docs: Add AURA security and data-handling guide; correct OTEL content-recording docs - #95
promptless[bot] wants to merge 4 commits into
Conversation
…t-recording Document AURA's operator security and data-handling model (mezmo/aura PR #524): new aura/security.mdx covering the safety boundary, telemetry and data egress, credentials, data retention, RBAC/MCP permission boundaries, approval and fail-closed behavior, prompt injection, vulnerability reporting, and release integrity. Correct aura/cli-reference.mdx and aura/tracing-spans.mdx: OTEL_RECORD_CONTENT gates content only on AURA-owned spans, so false does not fully suppress export; Rig-owned spans record content regardless, truncated by OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT (65536 bytes) not OTEL_CONTENT_MAX_LENGTH. Add nav entry and Vale vocabulary.
Fold mezmo/aura PR #543 into the OTEL_RECORD_CONTENT documentation: when enabled, AURA-owned spans now also record the assembled system prompt (the preamble, including orchestration instructions and the skills catalog), truncated by OTEL_CONTENT_MAX_LENGTH and exported as leading system entries in llm.input_messages. Correct the Rig-owned span list in the cli-reference OTEL table.
![promptless[bot]](https://avatars.githubusercontent.com/in/979718?s=60&v=4){kind=small}
|
|
||
| Treat data returned by logs, runbooks, vector stores, webpages, and MCP servers as untrusted model input. | ||
|
|
||
| [aura-web-server](/aura/web-server-reference) has no end-user authentication and no identity-aware role-based access control (RBAC) for its OpenAI-compatible API. Bind it to a private interface, or place it behind an authenticated Transport Layer Security (TLS) gateway or service mesh. |
There was a problem hiding this comment.
aura-web-server has no end-user auth or identity-aware RBAC on its OpenAI-compatible API; bind to a private interface or place behind an authenticated TLS gateway/service mesh — verified verbatim against SECURITY.md "Safety Boundary" section.
Source: https://github.com/mezmo/aura/blob/6c4a5ba66d899a831ff2c589fafe029168af39da/SECURITY.md#L20-L22
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
|
||
| [aura-web-server](/aura/web-server-reference) has no end-user authentication and no identity-aware role-based access control (RBAC) for its OpenAI-compatible API. Bind it to a private interface, or place it behind an authenticated Transport Layer Security (TLS) gateway or service mesh. | ||
|
|
||
| An air-gapped deployment requires every enabled network dependency to be local or disabled. AURA does not implement a general-purpose egress firewall, so you enforce that boundary with your own network policy. See Data Retention and Large Language Model Egress below for the destinations to check against this rule. |
There was a problem hiding this comment.
An air-gapped deployment requires every enabled network dependency to be local or disabled; AURA implements no general-purpose egress firewall, so the operator enforces that boundary with network policy (specific destinations enumerated in the Data Retention and LLM Egress section below) -- verified against SECURITY.md's "Safety Boundary and Operator Responsibilities" air-gapped-deployment paragraph.
Source: https://github.com/mezmo/aura/blob/6c4a5ba66d899a831ff2c589fafe029168af39da/SECURITY.md#L24-L31
| Once OpenTelemetry Protocol (OTLP) export is on, treat the collector as a recipient of prompt, response, and tool content. `OTEL_RECORD_CONTENT=false` is **not** a complete content-suppression control. Do not set `OTEL_EXPORTER_OTLP_ENDPOINT` unless the collector and its downstream systems are approved to receive model and tool content. | ||
| </Warning> | ||
|
|
||
| AURA emits two kinds of spans. AURA-owned spans honor `OTEL_RECORD_CONTENT`; Rig-owned spans (created by the underlying Rig library) do not. For AURA-owned spans, `OTEL_RECORD_CONTENT=false` (the default) omits prompt and completion text, and tool arguments and results. `OTEL_RECORD_CONTENT=true` includes them, subject to `OTEL_CONTENT_MAX_LENGTH` (default 1000 bytes). Rig-owned spans (`chat`, `chat_streaming`, `agent.turn`, and `execute_tool`) record content independently of that flag. AURA translates their prompt, response, message, tool-argument, and tool-result attributes for export, subject to `OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT` (65,536 bytes by default), not `OTEL_CONTENT_MAX_LENGTH`. So setting `OTEL_RECORD_CONTENT=false` still exports content from Rig-owned spans. |
There was a problem hiding this comment.
OTEL_RECORD_CONTENT=false (default) omits AURA-owned span content; =true includes it bounded by OTEL_CONTENT_MAX_LENGTH (default 1000 bytes, per logging.rs); Rig-owned spans (chat, chat_streaming, agent.turn, execute_tool -- exact names per openinference_exporter.rs infer_span_kind) record content independently of that flag, bounded by OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT (65,536 bytes default, DEFAULT_ATTRIBUTE_VALUE_LENGTH_LIMIT const), not OTEL_CONTENT_MAX_LENGTH -- verified against SECURITY.md's "Operator OpenTelemetry Export" section plus source defaults/span names in logging.rs and openinference_exporter.rs.
Source: https://github.com/mezmo/aura/blob/6c4a5ba66d899a831ff2c589fafe029168af39da/SECURITY.md#L85-L98
|
|
||
| ## Credentials | ||
|
|
||
| Reference credentials with environment-variable placeholders such as `{{ env.OPENAI_API_KEY }}` instead of writing literal secrets into your TOML, and populate those environment variables from your secret manager. Initialization keeps the placeholder in `config.toml`, but if you enter a new key, the CLI can write it to a local `.env` file using normal filesystem permissions rather than a restrictive mode. Restrict that file and exclude it from version control. |
There was a problem hiding this comment.
If a user enters a new key during init, the CLI may write it to a local .env using the process's normal filesystem permissions rather than a dedicated restrictive mode; operator must restrict the file and exclude it from version control — verified against SECURITY.md "Credentials" section.
Source: https://github.com/mezmo/aura/blob/6c4a5ba66d899a831ff2c589fafe029168af39da/SECURITY.md#L105-L108
|
|
||
| ## RBAC and MCP Permission Boundaries | ||
|
|
||
| `mcp_filter` and `client_tool_filter` control which named tools an agent or worker sees. An explicit empty `mcp_filter` grants no MCP tools, while a missing filter can expose every discovered tool. Use explicit allow-lists for every agent and worker. |
There was a problem hiding this comment.
mcp_filter/client_tool_filter are capability-reduction controls, not user-aware RBAC; empty mcp_filter grants no MCP tools, missing filter can expose all discovered tools; enforce RBAC at ingress/MCP/IAM/k8s/network layers — verified against SECURITY.md "RBAC and MCP Permission Boundaries."
Source: https://github.com/mezmo/aura/blob/6c4a5ba66d899a831ff2c589fafe029168af39da/SECURITY.md#L166-L175
|
|
||
| ## Approval and Fail-Closed Behavior | ||
|
|
||
| The `[hitl]` policy gates only tool names that match its `require_approval` glob patterns. A matching call reaches its tool only after the configured route returns an `Approved` decision. The following outcomes all fail closed for that call: |
There was a problem hiding this comment.
[hitl] gates only tool names matching require_approval glob patterns; a matching call proceeds only after an Approved decision; denial/timeout/disconnect/shutdown/malformed-rejected/transport failures fail closed for that call but do not block the model from proposing another — verified against SECURITY.md "Approval and Fail-Closed Behavior" (note: source treats "Denial short-circuits that call" as distinct from the enumerated error-return outcomes; see corrections).
Source: https://github.com/mezmo/aura/blob/6c4a5ba66d899a831ff2c589fafe029168af39da/SECURITY.md#L193-L199
|
|
||
| ## Vulnerability Reporting | ||
|
|
||
| Report suspected vulnerabilities privately to [security@mezmo.com](mailto:security@mezmo.com). GitHub private vulnerability reporting is not enabled for the repository, so do not rely on its private-advisory URL. Include the affected version, deployment mode, reproduction steps, impact, and any proposed mitigation. |
There was a problem hiding this comment.
Report vulnerabilities privately to security@mezmo.com; GitHub private vulnerability reporting is not currently enabled for the repository, so do not rely on the repo's private-advisory submission URL — verified verbatim against SECURITY.md "Vulnerability Reporting."
Source: https://github.com/mezmo/aura/blob/6c4a5ba66d899a831ff2c589fafe029168af39da/SECURITY.md#L237-L240
|
|
||
| ## Release Integrity and Supply Chain | ||
|
|
||
| Release binaries and `.deb` and `.rpm` packages ship with SHA-256 checksums. The install script requires a matching checksum by default, and a mismatch is always fatal. Keep `AURA_REQUIRE_CHECKSUM=1`, or supply a separately obtained trusted checksum with `AURA_CHECKSUMS`. |
There was a problem hiding this comment.
Release binaries and .deb/.rpm packages ship with SHA-256 checksums; install script requires a matching checksum by default and a mismatch is always fatal; keep AURA_REQUIRE_CHECKSUM=1 or supply a trusted checksum via AURA_CHECKSUMS — verified against SECURITY.md and cross-checked against docs_repos/mezmo__documentation/aura/installation.mdx (AURA_REQUIRE_CHECKSUM default 1, mismatch always fatal, AURA_CHECKSUMS is a local checksums.txt path).
Source: https://github.com/mezmo/aura/blob/6c4a5ba66d899a831ff2c589fafe029168af39da/SECURITY.md#L253-L256
| When `OTEL_RECORD_CONTENT=true`, prompt/completion text and tool | ||
| arguments/results are recorded as span attributes, truncated to | ||
| `OTEL_CONTENT_MAX_LENGTH` (default 1000 bytes, rounded to a UTF-8 boundary). | ||
| `OTEL_RECORD_CONTENT` governs only AURA-owned spans. When `false` (the default), |
There was a problem hiding this comment.
OTEL_RECORD_CONTENT=false does not fully suppress content; Rig-owned chat/agent-turn/tool spans record prompt/response/structured-message/tool-argument/tool-result content independently of the flag, bounded by OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT (65,536 bytes default), not OTEL_CONTENT_MAX_LENGTH — verified against SECURITY.md.
Source: https://github.com/mezmo/aura/blob/6c4a5ba66d899a831ff2c589fafe029168af39da/SECURITY.md#L89-L93
| | `OTEL_LOG_LEVEL` | Override the OTel layer's filter. Default captures `aura=trace`, `aura_cli=info`, and rig spans. | | ||
| | `OTEL_RECORD_CONTENT` | When `true`, prompt/completion/tool args/results are recorded as span attributes. | | ||
| | `OTEL_CONTENT_MAX_LENGTH` | Max bytes for content attributes (default 1000, rounded down to UTF-8 boundary). | | ||
| | `OTEL_RECORD_CONTENT` | Gates content, including the assembled system prompt, on AURA-owned spans (`false` default omits it). Rig-owned `chat`, `chat_streaming`, `agent.turn`, and `execute_tool` spans record content regardless, so `false` does not fully suppress exported content. See [Security and Data Handling](/aura/security). | |
There was a problem hiding this comment.
OTEL_RECORD_CONTENT does not fully suppress content when false — Rig-owned spans record content independently, bounded by OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT — verified against SECURITY.md's Rig-owned-span content-export limitation.
Source: https://github.com/mezmo/aura/blob/6c4a5ba66d899a831ff2c589fafe029168af39da/SECURITY.md#L89-L93
|
|
||
| - **Model provider:** the system prompt, user input, relevant conversation history, and tool results placed into model context. | ||
| - **MCP server:** the selected tool name and its arguments, which can contain user input or model-derived data. | ||
| - **Optional destinations:** retrieval-augmented generation (RAG) services, approval webhooks, Redis-compatible session stores, and OTLP collectors, only when you configure them. An approval webhook receives the tool name and arguments under review, and by default that request is unsigned and can use plaintext `http://` unless you configure a signing secret (see Approval and Fail-Closed Behavior below). A Redis-compatible session store can hold operational data such as parked approval payloads and task state, so treat it as a destination with comparable sensitivity. |
There was a problem hiding this comment.
Approval webhook request body includes tool_name and arguments (see ApprovalRequestWire/ApprovalItem in protocol.rs and the documented webhook JSON payload in hitl.mdx); by default (no HMAC secret) the request is unsigned and http:// is permitted -- verified against route.rs EgressSigning::Disabled default and WebhookHmac::load_from_env (signing.rs L309-357, "unsigned and unverified" when unset).
|
|
||
| - **Model provider:** the system prompt, user input, relevant conversation history, and tool results placed into model context. | ||
| - **MCP server:** the selected tool name and its arguments, which can contain user input or model-derived data. | ||
| - **Optional destinations:** retrieval-augmented generation (RAG) services, approval webhooks, Redis-compatible session stores, and OTLP collectors, only when you configure them. An approval webhook receives the tool name and arguments under review, and by default that request is unsigned and can use plaintext `http://` unless you configure a signing secret (see Approval and Fail-Closed Behavior below). A Redis-compatible session store can hold operational data such as parked approval payloads and task state, so treat it as a destination with comparable sensitivity. |
There was a problem hiding this comment.
A Redis-compatible session store can hold operational data such as parked approval payloads and task state -- verified against configuration-reference.mdx's Session Store section: "cross-request session state (A2A tasks, parked HITL approvals) lives in process memory... configure a shared Redis/Valkey backend" for multi-pod deployments.
Source:
|
|
||
| AURA does not establish that an approver is human. A conversational decision comes from the connected client, and a webhook decision can come from a person or an automated service. Your deployment must authenticate users and enforce approver identity and authorization outside AURA. | ||
|
|
||
| Webhook hash-based message authentication code (HMAC) signing and verification are optional. Without a signing secret (the `AURA_HITL_WEBHOOK_SECRET` environment variable), AURA sends unsigned requests, accepts unsigned responses, and permits a plaintext `http://` URL. When you set that secret, AURA rejects unsigned or invalid responses and rejects plaintext URLs. Production should use TLS and HMAC. |
There was a problem hiding this comment.
The webhook HMAC signing secret is the environment variable AURA_HITL_WEBHOOK_SECRET (PRIMARY_SECRET_VAR const), read only via WebhookHmac::load_from_env (signing.rs L309-357) -- not a TOML field. Without it, load_from_env returns Ok(None) and logs "webhook traffic is unsigned and unverified" (L317-322); route.rs defaults EgressSigning::Disabled (permits plaintext http://) and rejects/misconfigures a plaintext http:// URL once a secret is configured (route.rs L488-498, boot guard validate_webhook_signing_config L674-687).
| Once OpenTelemetry Protocol (OTLP) export is on, treat the collector as a recipient of prompt, response, and tool content. `OTEL_RECORD_CONTENT=false` is **not** a complete content-suppression control. Do not set `OTEL_EXPORTER_OTLP_ENDPOINT` unless the collector and its downstream systems are approved to receive model and tool content. | ||
| </Warning> | ||
|
|
||
| AURA emits two kinds of spans. AURA-owned spans honor `OTEL_RECORD_CONTENT`; Rig-owned spans (created by the underlying Rig library) do not. For AURA-owned spans, `OTEL_RECORD_CONTENT=false` (the default) omits prompt and completion text, and tool arguments and results. `OTEL_RECORD_CONTENT=true` includes them, subject to `OTEL_CONTENT_MAX_LENGTH` (default 1000 bytes). Rig-owned spans (`chat`, `chat_streaming`, `agent.turn`, and `execute_tool`) record content independently of that flag. AURA translates their prompt, response, message, tool-argument, and tool-result attributes for export, subject to `OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT` (65,536 bytes by default), not `OTEL_CONTENT_MAX_LENGTH`. So setting `OTEL_RECORD_CONTENT=false` still exports content from Rig-owned spans. |
There was a problem hiding this comment.
OTEL_CONTENT_MAX_LENGTH default is 1000 bytes -- verified against logging.rs doc comment "Controlled by OTEL_CONTENT_MAX_LENGTH env var (default 1000)" (also stated identically in cli-reference.mdx and tracing-spans.mdx, both previously verified).
| Once OpenTelemetry Protocol (OTLP) export is on, treat the collector as a recipient of prompt, response, and tool content. `OTEL_RECORD_CONTENT=false` is **not** a complete content-suppression control. Do not set `OTEL_EXPORTER_OTLP_ENDPOINT` unless the collector and its downstream systems are approved to receive model and tool content. | ||
| </Warning> | ||
|
|
||
| AURA emits two kinds of spans. AURA-owned spans honor `OTEL_RECORD_CONTENT`; Rig-owned spans (created by the underlying Rig library) do not. For AURA-owned spans, `OTEL_RECORD_CONTENT=false` (the default) omits prompt and completion text, and tool arguments and results. `OTEL_RECORD_CONTENT=true` includes them, subject to `OTEL_CONTENT_MAX_LENGTH` (default 1000 bytes). Rig-owned spans (`chat`, `chat_streaming`, `agent.turn`, and `execute_tool`) record content independently of that flag. AURA translates their prompt, response, message, tool-argument, and tool-result attributes for export, subject to `OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT` (65,536 bytes by default), not `OTEL_CONTENT_MAX_LENGTH`. So setting `OTEL_RECORD_CONTENT=false` still exports content from Rig-owned spans. |
There was a problem hiding this comment.
Exact Rig-owned span names are chat, chat_streaming, agent.turn, and execute_tool -- verified against openinference_exporter.rs infer_span_kind match arms ("chat" | "chat_streaming" | "agent.turn" => "LLM"; "execute_tool" | "mcp.tool_call" => "TOOL") and corresponding unit tests (L492-499).
|
|
||
| ## Span attributes | ||
|
|
||
| *When `OTEL_RECORD_CONTENT=true`, AURA also records the assembled system prompt on the agent entry-point spans (`agent.stream`, `agent.prompt`, `agent.chat`) and on the `orchestration.planning` and `orchestration.worker` spans. See [Content recording](#content-recording).* |
There was a problem hiding this comment.
set_system_prompt_attribute (gated on should_record_content) records gen_ai.system_instructions; called on all five spans named in the doc sentence: agent.prompt and agent.chat via record_input_attributes (builder.rs L1140/L1165 calling L1568), orchestration.planning (orchestrator.rs L1468-1471), orchestration.worker (orchestrator.rs L3105-3109), and the agent.stream span via StreamOtelContext::record_input (aura-web-server/src/streaming/otel.rs L51-53), which is invoked for both the streaming and non-streaming HTTP completion paths (handlers.rs L546-559) -- verified against merged PR mezmo/aura#543 (commit 474556e).
| it omits prompt and completion text, and tool arguments and results. When `true`, | ||
| it includes them, along with the assembled system prompt (the preamble, which | ||
| includes the orchestration instructions and the skills catalog). AURA truncates | ||
| all of this content to `OTEL_CONTENT_MAX_LENGTH` (default 1000 bytes, rounded down |
There was a problem hiding this comment.
system_instructions_json truncates the prompt text via truncate_for_otel (OTEL_CONTENT_MAX_LENGTH) before JSON-serializing it as the gen_ai.system_instructions parts array, confirming the system prompt is truncated pre-serialization rather than post-serialization -- verified against merged PR mezmo/aura#543 (commit 474556e). Assembled preamble = orchestration instructions + skills catalog per builder.rs L436-438 (config.effective_preamble() + render_skill_catalog) and the PR's own commit message.
| to the nearest UTF-8 boundary). At export, the recorded system prompt appears as | ||
| leading `system` entries in the input messages (`llm.input_messages`) in Phoenix | ||
| and other OpenInference viewers. AURA translates the raw `gen_ai.system_instructions` | ||
| key into those leading `system` entries rather than exporting it directly. |
There was a problem hiding this comment.
At export, gen_ai.system_instructions is parsed (parse_system_instructions, L451) into leading role:"system" entries prepended to llm.input_messages (L233-252); span.attributes.retain(...) at L290-291 strips every gen_ai.* key, including the raw gen_ai.system_instructions, so it is translated and not exported verbatim -- verified against merged PR mezmo/aura#543 (commit 474556e).
| | `OTEL_LOG_LEVEL` | Override the OTel layer's filter. Default captures `aura=trace`, `aura_cli=info`, and rig spans. | | ||
| | `OTEL_RECORD_CONTENT` | When `true`, prompt/completion/tool args/results are recorded as span attributes. | | ||
| | `OTEL_CONTENT_MAX_LENGTH` | Max bytes for content attributes (default 1000, rounded down to UTF-8 boundary). | | ||
| | `OTEL_RECORD_CONTENT` | Gates content, including the assembled system prompt, on AURA-owned spans (`false` default omits it). Rig-owned `chat`, `chat_streaming`, `agent.turn`, and `execute_tool` spans record content regardless, so `false` does not fully suppress exported content. See [Security and Data Handling](/aura/security). | |
There was a problem hiding this comment.
OTEL_RECORD_CONTENT (via should_record_content) gates set_system_prompt_attribute, which records the assembled system prompt as gen_ai.system_instructions on AURA-owned spans -- verified against merged PR mezmo/aura#543 (commit 474556e). Default false confirmed at logging.rs L145 / L182-185 (bool_env("OTEL_RECORD_CONTENT", false)).
- Add PHOENIX_PROJECT_NAME (openinference.project.name routing) to tracing-spans and the CLI OTel env-var table, with the multi-replica same-value rule. - Reclassify agent-level spans to LLM and vector.search to RETRIEVER in the span-kinds table and trace diagrams; document retrieval.documents.*. - Record the token-count breaking change: llm.token_count.total and llm.token_count.tool_completion removed from AURA spans; per-turn prompt/completion counts now on agent.turn, with a migration Warning. - Document new span attributes (llm.provider, agent.name, llm.invocation_parameters, llm.tools.*, llm.prompt_template.*, aura.version, aura.mode, service.version) and the exception span event. - Document the OpenAI-compatible user and metadata request fields on the web server reference. - Extend the security page's OTLP content scope to the new content-bearing attribute families.
| [Phoenix](https://github.com/Arize-ai/phoenix) and other OpenInference-aware | ||
| observability tools. | ||
|
|
||
| Set `PHOENIX_PROJECT_NAME` to choose which Phoenix project receives your |
There was a problem hiding this comment.
PHOENIX_PROJECT_NAME sets the openinference.project.name OTLP resource attribute; when unset it falls back to service_name (OTEL_SERVICE_NAME, then "aura"); service.version resource attr is also set from CARGO_PKG_VERSION here -- verified against mezmo/aura PR #566 (merged commit c3b8b996), init_otel_provider in logging.rs.
| | `OTEL_RECORD_CONTENT` | Gates content, including the assembled system prompt, on AURA-owned spans (`false` default omits it). Rig-owned `chat`, `chat_streaming`, `agent.turn`, and `execute_tool` spans record content regardless, so `false` does not fully suppress exported content. See [Security and Data Handling](/aura/security). | | ||
| | `OTEL_CONTENT_MAX_LENGTH` | Max bytes for content attributes on AURA-owned spans (default 1000, rounded down to the nearest UTF-8 boundary). | | ||
| | `OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT` | Truncation limit for translated Rig-owned content (default 65,536 bytes, rounded down to the nearest UTF-8 boundary). | | ||
| | `PHOENIX_PROJECT_NAME` | Sets the `openinference.project.name` resource attribute. Phoenix routes traces to a project by that attribute alone and ignores `service.name` and `OTEL_SERVICE_NAME` for routing. When unset, AURA sets the attribute to the service name (`OTEL_SERVICE_NAME`, then `aura`) instead, so traces are grouped under a project of that name. | |
There was a problem hiding this comment.
PHOENIX_PROJECT_NAME table row: sets openinference.project.name resource attr; Phoenix routing ignores service.name/OTEL_SERVICE_NAME per source comment; unset falls back to service_name -- verified against mezmo/aura PR #566 (merged commit c3b8b996), init_otel_provider in logging.rs.
There was a problem hiding this comment.
(Line 65)
infer_span_kind match arms confirm exact reclassification: agent.stream/agent.prompt/agent.chat/orchestration.planning/orchestration.worker -> LLM; vector.search -> RETRIEVER; chat_completions/streaming_completion/orchestration/orchestration.iteration -> CHAIN (orchestration.planning removed from CHAIN); execute_tool/mcp.tool_call -> TOOL -- verified against mezmo/aura PR #566 (merged commit c3b8b996). Confirms diagrams and the OpenInference span kinds table.
|
|
||
| RAG-enabled agents emit an additional `vector.search (RETRIEVER)` span under | ||
| `execute_tool`, alongside `mcp.tool_call`, wherever the agent calls a | ||
| vector-search tool. |
There was a problem hiding this comment.
StreamOtelContext::record_input stamps ATTR_AURA_VERSION (aura.version, from CARGO_PKG_VERSION) and ATTR_AURA_MODE (aura.mode = "orchestration"/"single-agent") on the agent.stream root span; service.version is set separately on the tracer resource (logging.rs init_otel_provider) -- verified against mezmo/aura PR #566 (merged commit c3b8b996).
| `llm.token_count.prompt`, `llm.token_count.completion`, `llm.token_count.total` | ||
| `aura.version`, `aura.mode`. | ||
|
|
||
| `aura.version` and `aura.mode` are stamped on the streaming root so you can |
There was a problem hiding this comment.
set_llm_identifiers now sets both ATTR_LLM_SYSTEM (llm.system) and the new ATTR_LLM_PROVIDER (llm.provider) to the same provider string -- verified against mezmo/aura PR #566 (merged commit c3b8b996).
| filter traces by release and by serving path. `service.version` is set on the | ||
| tracer resource. | ||
|
|
||
| ### Agent and LLM spans |
There was a problem hiding this comment.
llm_invocation_parameters() builds the llm.invocation_parameters JSON (temperature, max_tokens, additional_params) from the effective LlmConfig; set_llm_invocation_parameters writes it as the ATTR_LLM_INVOCATION_PARAMETERS attribute -- verified against mezmo/aura PR #566 (merged commit c3b8b996).
| tracer resource. | ||
|
|
||
| ### Agent and LLM spans | ||
|
|
There was a problem hiding this comment.
McpManager::tool_schemas_json(filter) serializes MCP tool definitions as OpenAI-style function schemas for llm.tools.{i}.tool.json_schema; filter narrows by worker mcp_filter glob patterns, None includes every tool -- verified against mezmo/aura PR #566 (merged commit c3b8b996). Called with a per-worker filter in orchestrator.rs execute_task (L3118-3123) and unfiltered for single-agent spans in builder.rs otel_llm_tools().
| - `llm.tools.{i}.tool.json_schema` holds the MCP tool schemas each agent | ||
| advertises, filtered per worker. | ||
| - `user.id` comes from the request `user` field, and `metadata` comes from the | ||
| request metadata map. See the [web server reference](/aura/web-server-reference) |
There was a problem hiding this comment.
execute_task (the orchestration.worker span) calls set_llm_prompt_template with the WORKER_TASK_PROMPT_TEMPLATE and CONTEXT/YOUR_TASK variables, adding llm.prompt_template.template and llm.prompt_template.variables to that span -- verified against mezmo/aura PR #566 (merged commit c3b8b996).
| - `user.id` comes from the request `user` field, and `metadata` comes from the | ||
| request metadata map. See the [web server reference](/aura/web-server-reference) | ||
| for those request fields. | ||
|
|
There was a problem hiding this comment.
Module doc: token counts now live only on agent.turn spans (Rig fork records per-call model identifiers and usage there); Phoenix prices those LLM-kind spans and rolls turns up per worker/phase/trace; no Aura-owned span records token counts -- verified against mezmo/aura PR #566 (merged commit c3b8b996). Backs the Warning box (lines 135-141) and the removal of llm.token_count.total/tool_completion.
| @@ -102,13 +175,35 @@ spans (planning, worker). | |||
|
|
|||
| ## Content recording | |||
There was a problem hiding this comment.
set_llm_prompt_template always records ATTR_LLM_PROMPT_TEMPLATE (the static template); ATTR_LLM_PROMPT_TEMPLATE_VARIABLES is only recorded when should_record_content() is true, and each variable value is passed through truncate_for_otel -- verified against mezmo/aura PR #566 (merged commit c3b8b996).
| translates and exports these attributes regardless of `OTEL_RECORD_CONTENT`. | ||
| Content is truncated to `OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT` (default 65,536 | ||
| bytes, rounded down to the nearest UTF-8 boundary), not `OTEL_CONTENT_MAX_LENGTH`. Because of this, | ||
| `OTEL_RECORD_CONTENT=false` is not a complete content-suppression control. See [Security and Data Handling](/aura/security) |
There was a problem hiding this comment.
set_span_error now fires tracing::error!(error = msg) before span.set_status; the tracing-opentelemetry layer converts that error-field event into an OTel exception span event carrying exception.message -- verified against mezmo/aura PR #566 (merged commit c3b8b996). Anchor updated: this claim now lives in the dedicated "## Error spans" section (moved from within Content recording).
| "metadata": {"team": "sre"} | ||
| } | ||
| ``` | ||
|
|
There was a problem hiding this comment.
ChatCompletionRequest gained a new optional user: Option<String> field (OpenAI-compatible end-user identifier); pre-existing metadata: Option<HashMap<String,String>> is now consumed -- verified against mezmo/aura PR #566 (merged commit c3b8b996).
|
|
||
| Two optional OpenAI-compatible fields carry request context into traces: | ||
|
|
||
| - `user`: optional string. An end-user identifier that flows into the `user.id` span attribute. |
There was a problem hiding this comment.
StreamOtelContext::record_input sets ATTR_USER_ID (user.id) from the request's user field and ATTR_METADATA (metadata) from the request's metadata map (serialized to a JSON string in handlers.rs prepare_request) -- verified against mezmo/aura PR #566 (merged commit c3b8b996).
| filter traces by release and by serving path. `service.version` is set on the | ||
| tracer resource. | ||
|
|
||
| ### Agent and LLM spans |
There was a problem hiding this comment.
The exporter's gen_ai.* translation pass strips "gen_ai.agent.name" and re-emits it as "agent.name" (ungated by span kind, so it applies to whichever span carries the field). Rig's AgentBuilder.name(...) is what stamps gen_ai.agent.name on the agent.turn span: single-agent builds call .name(&config.agent.name) (builder.rs L479), the coordinator calls .name("coordinator") (orchestrator.rs L2125), and each worker calls .name(&worker_config.agent.name) (orchestrator.rs L2582/2629/2672/2707/2736/2770). Unit tests confirm the attribute lands on the agent.turn span specifically (test_translates_agent_metadata, L911-932, using the literal value "Unnamed Agent" as the pre-fix placeholder; test_pipeline_agent_turn_span, L1620-1676, asserting find_attr(turn, "agent.name")) -- verified against mezmo/aura PR #566 (merged commit c3b8b996). The doc no longer claims name sits directly on agent.stream or orchestration.worker.
|
|
||
| - Phoenix rolls the turn counts up as the single cost anchor, so every aggregate | ||
| (per worker, per phase, per trace) is computed from the turns. | ||
| - AURA's agent-level spans carry no token counts, so they price at zero and |
There was a problem hiding this comment.
set_retrieval_documents writes retrieval.documents.{i}.document.score unconditionally and retrieval.documents.{i}.document.content / .document.metadata only when should_record_content() (OTEL_RECORD_CONTENT) is true, per its own doc comment ("Scores are always recorded; content and metadata only when content recording is enabled"). infer_span_kind classifies the "vector.search" span name as RETRIEVER (openinference_exporter.rs L126). Wired from both static RAG tools (rag_tools.rs L71-99, create_vector_search_tool! macro: info_span!("vector.search", ...) then set_retrieval_documents(&span, &documents)) and the dynamic vector-store tool (vector_dynamic.rs L116-164) -- verified against mezmo/aura PR #566 (merged commit c3b8b996). The doc's claim that only "content" (not the score) follows the OTEL_RECORD_CONTENT gate is accurate and not an over-claim.
| content. | ||
|
|
||
| ## OpenInference span kinds | ||
|
|
There was a problem hiding this comment.
infer_span_kind(name: &str) has always matched purely on the span's name (never on attributes) -- this PR only changed which kind names map to (agent.stream/agent.prompt/agent.chat/orchestration.planning/orchestration.worker moved from AGENT/CHAIN to LLM; vector.search added as RETRIEVER), not the mechanism. The reworded doc sentence ("Each span's kind follows from its span name. The agent-level spans now map to LLM...") correctly describes a mapping change, not a switch from attribute-based to name-based inference -- verified against mezmo/aura PR #566 (merged commit c3b8b996).
…ity-data-handling # Conflicts: # aura/tracing-spans.mdx
Open in Promptless
Adds a new operator-facing security and data-handling reference for AURA at
/aura/security, derived from the newSECURITY.mdadded in mezmo/aura PR #524. The page covers AURA's safety boundary and operator responsibilities, the two independent telemetry mechanisms (Mezmo CLI product telemetry and operator OpenTelemetry export), credentials handling, data retention and LLM egress, RBAC and MCP permission boundaries, read-only versus mutating tools, approval and fail-closed behavior, prompt injection from logs and runbooks, vulnerability reporting, and release integrity.It also corrects and expands the OTEL content-recording documentation on two existing pages.
OTEL_RECORD_CONTENTgates content only on AURA-owned spans, soOTEL_RECORD_CONTENT=falsedoes not fully suppress exported content: Rig-owned chat, agent-turn, and tool spans record prompt, response, and tool content regardless of the flag, truncated byOTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT(65,536 bytes) rather thanOTEL_CONTENT_MAX_LENGTH(1,000 bytes).Folds in mezmo/aura PR #543: when
OTEL_RECORD_CONTENT=true, AURA-owned spans now also record the assembled system prompt (the preamble, which includes the orchestration instructions and the skills catalog), truncated byOTEL_CONTENT_MAX_LENGTH. At export it appears as leadingsystementries inllm.input_messagesin Phoenix and other OpenInference viewers. Theaura/tracing-spans.mdxcontent-recording and span-attributes sections and theaura/cli-reference.mdxOTEL environment-variable table are updated to reflect this and to cross-link the new security page.Folds in mezmo/aura PR #566 (fix empty assistant content 400, enrich Phoenix span). This change enriches the exported OpenTelemetry spans and moves where token counts live, so the tracing and reference pages are updated to match shipped behavior:
PHOENIX_PROJECT_NAMEenvironment variable — sets theopeninference.project.nameresource attribute, the only attribute Phoenix uses to route a trace to a project. Documented inaura/tracing-spans.mdxand theaura/cli-reference.mdxOTEL environment-variable table, with the multi-replica rule to set the same value on every pod so their traces group into one project.llm.token_count.totalandllm.token_count.tool_completionare no longer emitted on AURA-owned spans. Per-turnllm.token_count.promptandllm.token_count.completionon eachagent.turnspan replace them, and Phoenix aggregates cost from those turns. A migration warning tells operators to repoint dashboards and alerts.agent.stream,agent.prompt,agent.chat,orchestration.planning,orchestration.worker) now export as LLM, andvector.searchexports as RETRIEVER. The span-kinds table and trace diagrams are updated, and a new Retriever spans section documentsretrieval.documents.{i}.*.llm.provider,agent.name,llm.invocation_parameters,llm.tools.{i}.tool.json_schema,llm.prompt_template.template/.variables,aura.version,aura.mode,service.version, and anexceptionspan event carryingexception.message.userandmetadatafields on/v1/chat/completionsare documented inaura/web-server-reference.mdx, including that they surface only as span attributes on exported traces.aura/security.mdxOTLP export section now also lists the retrieval-document and prompt-template-variable content families as gated byOTEL_RECORD_CONTENT.Files: new
aura/security.mdx; edits toaura/cli-reference.mdx,aura/tracing-spans.mdx,aura/web-server-reference.mdx,docs.json(navigation), and the Vale vocabulary.Trigger Events
Tip: Filter the Dashboard by labels or assignees to focus on what matters to you 🔎