diff --git a/docs/installation.md b/docs/installation.md index da556dd..39ec530 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -181,6 +181,63 @@ The minimal installation includes the UI, headless interfaces, OpenClaw, and Ope ## Usage +### Direct IP access with MetalLB + +On clusters with MetalLB installed, add this to your installation values: + +```yaml +mosaicUi: + service: + type: LoadBalancer + annotations: + metallb.io/address-pool: afoa +``` + +Replace `afoa` with an existing MetalLB pool permitted to allocate an address to +this namespace and Service. The cluster administrator reserves reachable IPs +outside DHCP and configures the pool and its L2 or BGP advertisement. The agent +chart does not install MetalLB or allocate network ranges. + +After installing, find the assigned address: + +```bash +kubectl -n mosaic get service mosaic-ui +``` + +Open `http://:3000` and use the browser credentials described below. +No DNS name, certificate, Gateway or port forwarding is required. Login remains +required, but HTTP does not encrypt credentials or traffic. For encrypted access, +use the HTTPS endpoint below. The default Service remains `ClusterIP`. + +### HTTPS access + +For a shared HTTPS endpoint, attach the UI to a site-managed Gateway: + +```yaml +mosaicUi: + httpRoute: + enabled: true + hostname: afoa.example.com + parentRefs: + - name: afoa-gateway + namespace: afoa + sectionName: https +``` + +The chart creates an `HTTPRoute` to the internal UI Service and preserves the +existing login. The site manages Kgateway, the MetalLB address, DNS and a trusted +certificate covering the hostname. The Gateway must be in the same cluster and +allow routes from the agent namespace. For BCM-managed NMC installations, use a +dedicated HTTPS Gateway to follow the Run:ai endpoint pattern. Keep development +and shared-user releases in separate namespaces to isolate upgrades. + +Check that the Gateway reports `Accepted=True` and `Programmed=True`, and the +route reports `Accepted=True` and `ResolvedRefs=True`. Verify certificate trust, +login and a streamed response from the users' network before changing existing +access paths. Keep all site infrastructure configuration in tracked source. + +Port forwarding remains available: + After installing with either inference path, open the UI by running: ```bash diff --git a/docs/nmc_installation.md b/docs/nmc_installation.md index 5199f2e..b6e5883 100644 --- a/docs/nmc_installation.md +++ b/docs/nmc_installation.md @@ -82,6 +82,31 @@ The command is a complete declaration and can be rerun unchanged. The SSH privat Before following the shared upgrade workflow in a new shell, run `module load kubernetes/k8s-admin` to select the NMC admin cluster. +## UI endpoint + +For direct IP access, use the [MetalLB Service settings](installation.md#direct-ip-access-with-metallb). +Supply an existing pool with a reserved, reachable address available to the agent +namespace. This path requires no DNS, certificate or Gateway; it serves HTTP and +preserves login authentication. + +For hostname-based HTTPS, follow the Base Command Manager-managed Run:ai pattern: +a reserved MetalLB IP, DNS name and dedicated Kgateway HTTPS listener in front +of the application's internal Service. Configure the Gateway in `k8s-admin`, +with a certificate trusted by clients and covering the chosen AFOA hostname. +Use the [HTTPS settings](installation.md#usage) to attach the chart's HTTPRoute. +The Gateway must allow routes from the agent namespace. + +The site owns MetalLB, IP reservations, Gateway, DNS and certificates; the agent +chart owns its Service and HTTPRoute. See the [BCM Gateway API release notes](https://docs.nvidia.com/base-command-manager/bcm-11-release-notes/bcm11-33-0.html). +BCM also supports [custom DNS zones](https://docs.nvidia.com/base-command-manager/product-manuals/admin/configuring) +through its preserved DNS include file. Clients must use a resolver that knows +the site's AFOA record and trust its certificate issuer. An IP assignment alone +does not configure DNS or client trust. + +Keep the shared-user release in a separate namespace from development, with its +own pods and storage. Preserve existing endpoints during validation. A Gateway +in `k8s-user` cannot directly reference the admin-cluster UI Service. + ## Self-hosted inference Deploy inference in `k8s-user` and connect the agent in `k8s-admin` to its HTTPS endpoint. If you already have an OpenAI-compatible endpoint, skip to [Install the agent](#install-the-agent). @@ -261,3 +286,19 @@ JS Use your configured served model name in the completion check if you changed the default. HTTP 200 for both requests establishes DNS, routing, TLS trust, authentication, and basic inference from the admin cluster. Then ask a short question through the agent UI to verify the full agent path. TLS errors require correcting certificate trust, timeouts require checking routing or server readiness, and 401 responses indicate mismatched keys. + + +### Reasoning-capable inference + +For a Nemotron endpoint that separates reasoning from final content, configure +vLLM with the model's supported reasoning parser (for this profile, +`--reasoning-parser=nemotron_v3`). Set `chatTemplateKwargs.enable_thinking: true` +in inference values and `llm.reasoning: true` in agent values. The agent then +requests low-effort reasoning and the compatibility proxy preserves that request. +The endpoint must return reasoning in its structured reasoning field; do not +strip planning text from final answers using phrase matching. + +These settings default to disabled to preserve existing endpoints. Coordinate +an inference rollout with every deployment using that endpoint; it reloads the +model. Verify multi-turn conversations, tool calls and final answers after the +rollout, including existing histories that contain a failed response. diff --git a/helm/afoa-inference/templates/vllm.yaml b/helm/afoa-inference/templates/vllm.yaml index 3ea56d7..7326730 100644 --- a/helm/afoa-inference/templates/vllm.yaml +++ b/helm/afoa-inference/templates/vllm.yaml @@ -43,7 +43,7 @@ spec: - --port=8000 - --tensor-parallel-size={{ .Values.tensorParallelSize }} - --served-model-name={{ .Values.servedModelName }} - - '--default-chat-template-kwargs={"enable_thinking":false}' + - {{ printf "--default-chat-template-kwargs=%s" (.Values.chatTemplateKwargs | toJson) | quote }} {{- range .Values.extraArgs }} - {{ . | quote }} {{- end }} diff --git a/helm/afoa-inference/values.yaml b/helm/afoa-inference/values.yaml index 133e1cc..8114f6f 100644 --- a/helm/afoa-inference/values.yaml +++ b/helm/afoa-inference/values.yaml @@ -7,6 +7,8 @@ model: nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 servedModelName: nemotron-3.5-super-vl gpuCount: 1 tensorParallelSize: 1 +chatTemplateKwargs: + enable_thinking: false runtimeClassName: nvidia nodeSelector: {} tolerations: [] diff --git a/helm/mosaic-stack/files/openclaw-seed/nemoclaw.json b/helm/mosaic-stack/files/openclaw-seed/nemoclaw.json index 0e94b50..5d8caf8 100644 --- a/helm/mosaic-stack/files/openclaw-seed/nemoclaw.json +++ b/helm/mosaic-stack/files/openclaw-seed/nemoclaw.json @@ -72,7 +72,7 @@ "observability", "slurm" ], - "thinkingDefault": "off", + "thinkingDefault": {{ ternary "low" "off" .Values.llm.reasoning | quote }}, "reasoningDefault": "off", {{- if .Values.openclaw.slack.enabled }} "maxConcurrent": {{ .Values.openclaw.slack.maxConcurrent }}, @@ -89,7 +89,7 @@ "id": "default", "name": "OpenClaw Assistant", "workspace": "~/.openclaw/workspace", - "thinkingDefault": "off", + "thinkingDefault": {{ ternary "low" "off" .Values.llm.reasoning | quote }}, "reasoningDefault": "off" } ] @@ -179,7 +179,7 @@ { "id": {{ include "mosaic-stack.llmModel" . | quote }}, "name": {{ include "mosaic-stack.llmModel" . | quote }}, - "reasoning": false, + "reasoning": {{ .Values.llm.reasoning | toJson }}, "input": [ "text" ], diff --git a/helm/mosaic-stack/templates/NOTES.txt b/helm/mosaic-stack/templates/NOTES.txt new file mode 100644 index 0000000..7107e10 --- /dev/null +++ b/helm/mosaic-stack/templates/NOTES.txt @@ -0,0 +1,11 @@ +{{/* SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 */}} +{{- if and .Values.observability.grafanaUrl (not .Values.observability.grafanaAuth.existingSecret) }} +Warning: Grafana is configured without authentication credentials. +If Grafana requires authentication, dashboard discovery, creation and display +will fail with HTTP 401. Set observability.grafanaAuth.existingSecret to a Secret +in this release's namespace containing the configured usernameKey and passwordKey +(defaults: username and password). +Credentials may be omitted for Grafana installations that allow anonymous access +with the permissions needed by these operations. Helm does not verify Grafana connectivity. +{{- end }} diff --git a/helm/mosaic-stack/templates/llm-compat.yaml b/helm/mosaic-stack/templates/llm-compat.yaml index e763a0d..fa780cc 100644 --- a/helm/mosaic-stack/templates/llm-compat.yaml +++ b/helm/mosaic-stack/templates/llm-compat.yaml @@ -30,7 +30,7 @@ data: if (req.method !== 'POST' || !req.url.endsWith('/chat/completions')) return body; try { const json = JSON.parse(body.toString('utf8')); - json.reasoning_effort = 'none'; + if (!{{ .Values.llm.reasoning | toJson }}) json.reasoning_effort = 'none'; return Buffer.from(JSON.stringify(json)); } catch { return body; @@ -71,7 +71,7 @@ spec: labels: app: {{ $c.name }} annotations: - checksum/config: {{ printf "%s-%s-%s" .Chart.Version $c.name (include "mosaic-stack.llmUpstreamBaseUrl" .) | sha256sum }} + checksum/config: {{ printf "%s-%s-%s-%v" .Chart.Version $c.name (include "mosaic-stack.llmUpstreamBaseUrl" .) .Values.llm.reasoning | sha256sum }} spec: {{- include "mosaic-stack.imagePullSecrets" . | nindent 6 }} {{- include "mosaic-stack.nodeSelector" . | nindent 6 }} diff --git a/helm/mosaic-stack/templates/mosaic-ui-httproute.yaml b/helm/mosaic-stack/templates/mosaic-ui-httproute.yaml new file mode 100644 index 0000000..15ae800 --- /dev/null +++ b/helm/mosaic-stack/templates/mosaic-ui-httproute.yaml @@ -0,0 +1,28 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- $route := .Values.mosaicUi.httpRoute | default dict }} +{{- if and .Values.modules.ui.enabled .Values.mosaicUi.enabled $route.enabled }} +{{- if not $route.parentRefs }} +{{- fail "mosaicUi.httpRoute.parentRefs is required when HTTPRoute is enabled" }} +{{- end }} +apiVersion: gateway.networking.k8s.io/v1 +kind: HTTPRoute +metadata: + name: mosaic-ui + labels: + {{- include "mosaic-stack.labels" . | nindent 4 }} +spec: + parentRefs: + {{- toYaml $route.parentRefs | nindent 4 }} + hostnames: + - {{ required "mosaicUi.httpRoute.hostname is required when HTTPRoute is enabled" $route.hostname | quote }} + rules: + - matches: + - path: + type: PathPrefix + value: / + backendRefs: + - name: mosaic-ui + port: {{ .Values.mosaicUi.service.port }} +{{- end }} diff --git a/helm/mosaic-stack/templates/mosaic-ui.yaml b/helm/mosaic-stack/templates/mosaic-ui.yaml index c7eeca8..fc28307 100644 --- a/helm/mosaic-stack/templates/mosaic-ui.yaml +++ b/helm/mosaic-stack/templates/mosaic-ui.yaml @@ -190,6 +190,10 @@ apiVersion: v1 kind: Service metadata: name: mosaic-ui + {{- with .Values.mosaicUi.service.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} labels: app: mosaic-ui {{- include "mosaic-stack.labels" . | nindent 4 }} @@ -200,5 +204,5 @@ spec: - name: http port: {{ .Values.mosaicUi.service.port }} targetPort: http - type: ClusterIP + type: {{ .Values.mosaicUi.service.type | default "ClusterIP" }} {{- end }} diff --git a/helm/mosaic-stack/tests/chart-render.test.mjs b/helm/mosaic-stack/tests/chart-render.test.mjs index 999031f..89aa5e4 100644 --- a/helm/mosaic-stack/tests/chart-render.test.mjs +++ b/helm/mosaic-stack/tests/chart-render.test.mjs @@ -8,6 +8,7 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { fileURLToPath } from "node:url"; import test from "node:test"; +import { runInNewContext } from "node:vm"; const chart = fileURLToPath(new URL("..", import.meta.url)); const slackManifest = fileURLToPath(new URL("../profiles/slack-app-manifest.json", import.meta.url)); @@ -1186,3 +1187,24 @@ test("configures a tenant-scoped VictoriaMetrics backend and native Grafana data assert.ok(render(...args, "--show-only", template).includes(endpoint)); } }); + + +test("reasoning endpoints preserve reasoning requests while legacy defaults remain compatible", () => { + for (const enabled of [false, true]) { + const args = ["--set", `llm.reasoning=${enabled}`]; + const seed = render(...args, "--show-only", "templates/openclaw-seed-configmap.yaml"); + const config = JSON.parse(configMapBlock(seed, "openclaw.json")); + assert.equal(config.models.providers.llm.models[0].reasoning, enabled); + assert.equal(config.agents.defaults.thinkingDefault, enabled ? "low" : "off"); + const proxy = render(...args, "--show-only", "templates/llm-compat.yaml"); + const code = configMapBlock(proxy, "server.js"); + const patch = runInNewContext(`${code}\npatchBody`, { + URL, Buffer, + process: { env: { UPSTREAM_BASE_URL: "https://inference.example.com/v1" } }, + require: () => ({ createServer: () => ({ listen() {} }) }), + }); + const body = { model: "example-model", reasoning_effort: "low", messages: [{ role: "user", content: "Inspect the available inventory." }] }; + const result = JSON.parse(patch({ method: "POST", url: "/v1/chat/completions" }, Buffer.from(JSON.stringify(body))).toString()); + assert.deepEqual(result, { ...body, reasoning_effort: enabled ? "low" : "none" }); + } +}); diff --git a/helm/mosaic-stack/values.yaml b/helm/mosaic-stack/values.yaml index cbce2c6..942de0d 100644 --- a/helm/mosaic-stack/values.yaml +++ b/helm/mosaic-stack/values.yaml @@ -101,6 +101,8 @@ kubernetes: llm: mode: vllm + # Enable only when the inference endpoint supports structured reasoning output. + reasoning: false requestCompatibility: enabled: true name: llm-openai-compat @@ -233,7 +235,13 @@ mosaicUi: size: 1Gi storageClassName: "" service: + type: ClusterIP port: 3000 + annotations: {} + httpRoute: + enabled: false + hostname: "" + parentRefs: [] auth: create: true name: mosaic-ui-auth