diff --git a/devops/developer-api/openapi.yaml b/devops/developer-api/openapi.yaml index b86b8de91..b1b03cdaf 100644 --- a/devops/developer-api/openapi.yaml +++ b/devops/developer-api/openapi.yaml @@ -513,6 +513,15 @@ paths: required: true schema: type: string + - name: period + in: query + description: >- + Period the time series cover: 1h, 3h, 12h, 1d, 3d, 7d or 30d. Defaults to 1h. 30d is + only available for databases on a Pro plan or with the Production Pack. + required: false + schema: + type: string + default: 1h responses: "200": description: Database stats retrieved successfully @@ -522,6 +531,169 @@ paths: $ref: "#/components/schemas/DatabaseStats" security: - basicAuth: [] + /redis/traffic/{id}: + get: + tags: + - redis + summary: Get Database Traffic + description: >- + This endpoint reports where a database's traffic comes from over a period. Available for + databases on a Pro plan or with the Production Pack. + x-mint: + href: /devops/developer-api/redis/get_database_traffic + operationId: getDatabaseTraffic + parameters: + - name: id + in: path + description: The ID of the database + required: true + schema: + type: string + - name: period + in: query + description: "The period every figure covers: 1h, 3h, 12h, 1d, 3d, 7d or 30d. Defaults to 1d." + required: false + schema: + type: string + default: 1d + responses: + "200": + description: Database traffic retrieved successfully + content: + application/json: + schema: + $ref: "#/components/schemas/DatabaseTraffic" + security: + - basicAuth: [] + /redis/insights/{id}: + get: + tags: + - redis + summary: Get Database Insights + description: >- + This endpoint returns the Insights series of a database: service latency percentiles, + read, write, engine, replication, lock-wait and Lua latency, expired entries, replication + lag, throughput, bandwidth and disk size, for the whole database, one replica or every + region, with each series' latest window compared to the one before it, and how much the + limit, security and Lua counters grew over the period. Latencies are in milliseconds. Available for databases on a Pro plan or + with the Production Pack. + x-mint: + href: /devops/developer-api/redis/get_database_insights + operationId: getDatabaseInsights + parameters: + - name: id + in: path + description: The ID of the database + required: true + schema: + type: string + - name: period + in: query + description: "The period of the series: 1h, 3h, 12h, 1d, 3d, 7d or 30d. Defaults to 3h." + required: false + schema: + type: string + default: 3h + - name: replica + in: query + description: >- + Restrict the series to one replica, by the key of a group whose kind is replica. + Other keys are rejected with 400. + required: false + schema: + type: string + - name: by + in: query + description: Set to region to receive one series per region next to the total. + required: false + schema: + type: string + enum: + - region + responses: + "200": + description: Database insights retrieved successfully + content: + application/json: + schema: + $ref: "#/components/schemas/DatabaseInsights" + security: + - basicAuth: [] + /redis/backup-analysis/{id}/{backupid}: + get: + tags: + - redis + summary: Get Backup Analysis + description: >- + This endpoint returns the state of a backup's analysis and, once completed, the report. + Daily backups of databases on a Pro plan or with the Production Pack are analyzed + automatically when they complete, except on regional and HIPAA databases and for backups + over 200 GiB. + x-mint: + href: /devops/developer-api/redis/backup/get_backup_analysis + operationId: getBackupAnalysis + parameters: + - name: id + in: path + description: The ID of the database + required: true + schema: + type: string + - name: backupid + in: path + description: The ID of the backup + required: true + schema: + type: string + responses: + "200": + description: Analysis state retrieved successfully + content: + application/json: + schema: + $ref: "#/components/schemas/BackupAnalysis" + security: + - basicAuth: [] + post: + tags: + - redis + summary: Analyze Backup + description: >- + This endpoint starts the offline analysis of a completed backup. The analysis scans the + backup without touching the running database and takes minutes for large backups; poll + the GET endpoint until the state is completed or failed. Available for databases on a Pro + plan or with the Production Pack; other databases get 404. Not available for RDB exports, + backups over 200 GiB, HIPAA databases or regional databases, nor for a backup whose + analysis is already pending or completed. + x-mint: + href: /devops/developer-api/redis/backup/analyze_backup + operationId: analyzeBackup + parameters: + - name: id + in: path + description: The ID of the database + required: true + schema: + type: string + - name: backupid + in: path + description: The ID of the backup + required: true + schema: + type: string + responses: + "200": + description: Analysis started + content: + application/json: + schema: + type: object + properties: + analysis_state: + type: string + example: pending + security: + - basicAuth: [] /redis/list-backup/{id}: get: tags: @@ -2138,11 +2310,17 @@ components: $ref: "#/components/schemas/TimeSeriesData" description: Disk usage over time example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + latency_p50: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: Median (p50) latency over time + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] latencymean: type: array items: $ref: "#/components/schemas/TimeSeriesData" - description: Average latency over time + description: Deprecated alias of latency_p50, kept for older clients. Carries the median, not the mean. example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] latency_99: type: array @@ -2150,11 +2328,17 @@ components: $ref: "#/components/schemas/TimeSeriesData" description: 99th percentile latency over time example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + read_latency_p50: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: Median (p50) read latency over time + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] read_latency_mean: type: array items: $ref: "#/components/schemas/TimeSeriesData" - description: Average read latency over time + description: Deprecated alias of read_latency_p50, kept for older clients. Carries the median, not the mean. example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] read_latency_99: type: array @@ -2162,11 +2346,17 @@ components: $ref: "#/components/schemas/TimeSeriesData" description: 99th percentile read latency over time example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + write_latency_p50: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: Median (p50) write latency over time + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] write_latency_mean: type: array items: $ref: "#/components/schemas/TimeSeriesData" - description: Average write latency over time + description: Deprecated alias of write_latency_p50, kept for older clients. Carries the median, not the mean. example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] write_latency_99: type: array @@ -2256,7 +2446,8 @@ components: example: 0 total_monthly_script_requests: type: integer - description: Total script requests in current month + description: Deprecated, always 0. Use daily_script_commands from the Prometheus integration + deprecated: true example: 0 queue_optimized: type: boolean @@ -2275,6 +2466,83 @@ components: format: float description: Total cost in current month example: 222.33902763855485 + rest_conn_count: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: Open REST connections over time + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + latency_p90: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: Overall 90th percentile latency over time, Pro and Production Pack only + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + command_latency_p50: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: Median time the engine spent executing commands, Pro and Production Pack only + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + command_latency_99: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: 99th percentile time the engine spent executing commands, Pro and Production Pack only + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + wait_latency_99: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: 99th percentile time commands waited for the execution lock, Pro and Production Pack only + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + disk_load_latency_99: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: 99th percentile time to load a key from disk, Pro and Production Pack only + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + disk_loads: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: Keys loaded from disk per ten-minute window, Pro and Production Pack only + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + lua_latency_p50: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: Median Lua script latency over time, Pro and Production Pack only + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + lua_latency_99: + type: array + items: + $ref: "#/components/schemas/TimeSeriesData" + description: 99th percentile Lua script latency over time, Pro and Production Pack only + example: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }] + lua_scripts: + type: array + description: Most executed Lua scripts since midnight UTC, Pro and Production Pack only + items: + $ref: "#/components/schemas/LuaScriptStat" + lua_script_count: + type: number + description: Number of distinct Lua scripts run since midnight UTC, Pro and Production Pack only + slow_commands: + type: array + description: Commands that exceeded the slow threshold today, Pro and Production Pack only + items: + $ref: "#/components/schemas/SlowCommandStat" + acl_user_logins: + type: array + description: Logins per ACL user today, Pro and Production Pack only + items: + type: object + properties: + username: + type: string + logins: + type: number command_counts: type: array description: Command-specific counts over time @@ -2288,6 +2556,335 @@ components: [{ "x": "2025-08-31 15:12:52.799480932 +0000 UTC", "y": 0 }], }, ] + LuaScriptStat: + type: object + properties: + sha: + type: string + script_name: + type: string + count: + type: number + description: Executions today + failures: + type: number + timeouts: + type: number + memory_limits: + type: number + command_count: + type: number + description: Redis commands the script ran + avg_duration_ns: + type: number + max_duration_ns: + type: number + avg_lua_duration_ns: + type: number + avg_command_duration_ns: + type: number + avg_allocated_bytes: + type: number + max_allocated_bytes: + type: number + SlowCommandStat: + type: object + properties: + command: + type: string + example: "EVALSHA" + count: + type: number + description: Executions today over the slow threshold + max_ms: + type: number + description: Slowest execution today in milliseconds + ClientNetwork: + type: object + properties: + prefix: + type: string + description: Client network masked to /24 for IPv4 or /48 for IPv6 + example: "203.0.113.0/24" + entry_region: + type: string + description: >- + Region of the Upstash proxies the network's requests came in through. A network + entering through several regions has one row per region. + example: "us-east-1" + provider: + type: string + description: aws, gcp, cloudflare, or internet when the address is not in a published range + region: + type: string + description: Provider region, empty for internet + count: + type: number + description: >- + Sampled observations. The proxies keep an hour of samples; for longer periods the + hourly figures are added up over time, so compare rows rather than read it as a + request count + weight: + type: number + description: Estimated observations, an upper bound, added up over time like count + share: + type: number + description: Share of all sampled observations, 0 to 1 + idle_seconds: + type: number + description: Seconds since the network was last seen; for periods over 1h, the lowest value over the period + TelemetryShare: + type: object + properties: + name: + type: string + example: "@upstash/redis@1.34.0" + count: + type: number + share: + type: number + InsightGroup: + type: object + properties: + key: + type: string + description: Key of the group in metrics and comparison for the all and region groups; pass a replica group's key as the replica parameter + label: + type: string + kind: + type: string + enum: + - all + - region + - replica + DatabaseInsights: + type: object + properties: + groups: + type: array + description: Every group the database has, regardless of the scope requested + items: + $ref: "#/components/schemas/InsightGroup" + metrics: + type: object + description: >- + Metric name to group key to data points. The all group is always present; region + groups are added with by=region. Metrics: latency_p50, latency_p90, latency_p99, + read_latency_99, write_latency_99, command_latency_99, replication_latency_999, + wait_latency_99, lua_latency_p50, lua_latency_99, expired_entries (per minute), + replication_lag (entries), throughput, bandwidth (bytes per second) and diskusage. + additionalProperties: + type: object + additionalProperties: + type: array + items: + $ref: "#/components/schemas/InsightPoint" + comparison_window: + type: string + description: >- + Length of the latest window the comparison averages: the whole period up to three + hours, the last three hours for longer periods. + example: "3h" + comparison: + type: object + description: >- + Metric name to group key to the mean over the latest window and over the equal window + right before it. previous is absent when that earlier window has no data. + additionalProperties: + type: object + additionalProperties: + type: object + properties: + current: + type: number + previous: + type: number + counters: + type: object + description: >- + Growth of each counter over the period, for the whole database: over_size_requests + (TCP and REST), over_size_record_accesses, throttled_commands (writes delayed by + persistence backpressure), undeliverable (requests and connections the proxies could + not hand to any replica), auth_failures (TCP and REST, including commands sent before + authenticating and REST requests over the rate limit), keyspace_notifications, connections_opened, bandwidth_ingress, + bandwidth_egress (bytes), rest_commands, lua_executions, lua_commands (commands run + inside scripts), lua_failures, lua_timeouts and lua_memory_limits. + additionalProperties: + type: object + properties: + total: + type: number + last_at: + type: string + format: date-time + description: End of the last bucket the counter grew in, absent when it did not grow + series: + type: array + description: Growth inside each bucket of the period + items: + $ref: "#/components/schemas/InsightPoint" + anomalies: + type: object + description: >- + Metric name to up to three anomalies of its all series, largest first: stretches where + the value stayed above both the period's median plus three standard deviations, + estimated as 1.4826 times the median absolute deviation, and 30% over the median. A + series with fewer than 8 points after its first positive value reports none. The median is raised to a small per-metric floor, so a + nearly idle series does not report huge changes. An empty stretch at the start of the + series is ignored, and a stretch longer than a third of the analyzed points is a level + change, not an anomaly. Every metric is present; diskusage is never checked and always + has an empty list. + additionalProperties: + type: array + items: + $ref: "#/components/schemas/InsightAnomaly" + InsightAnomaly: + type: object + properties: + start: + type: string + description: x of the first point above the threshold + end: + type: string + description: x of the last point above the threshold + peak: + type: string + description: x of the highest point + peak_value: + type: number + baseline: + type: number + description: Typical level of the period, the median raised to the metric's floor + change: + type: number + description: peak_value / baseline - 1, where 1 means +100% + InsightPoint: + type: object + properties: + x: + type: string + description: Timestamp of the point + y: + type: number + TrafficRoute: + type: object + properties: + proxy_region: + type: string + description: Region the requests entered in + member_region: + type: string + description: Region of the replica that served them + http_requests: + type: number + tcp_connections_opened: + type: number + tcp_connections: + type: number + description: Connections open now; every other figure covers the requested period + ingress_bytes: + type: number + description: Bytes clients sent + egress_bytes: + type: number + description: Bytes clients received + errors: + type: number + cross_region: + type: boolean + DatabaseTraffic: + type: object + properties: + period: + type: string + description: The period every figure covers + example: "1d" + database_region: + type: string + description: Primary region of the database, to compare client regions against + client_network_window: + type: string + description: Same as period + example: "1d" + client_networks: + type: array + items: + $ref: "#/components/schemas/ClientNetwork" + client_network_evictions: + type: number + description: >- + Networks the proxies pushed out of their sampling tables. A value comparable to the + observations means the list is a sample of the traffic rather than a ranking. + telemetry_window: + type: string + description: Same as period + example: "1d" + telemetry: + type: object + properties: + sdks: + type: array + items: + $ref: "#/components/schemas/TelemetryShare" + platforms: + type: array + items: + $ref: "#/components/schemas/TelemetryShare" + runtimes: + type: array + items: + $ref: "#/components/schemas/TelemetryShare" + retries: + type: object + properties: + requests: + type: number + retried: + type: number + rate: + type: number + routes_window: + type: string + description: Same as period + example: "1d" + routes: + type: array + items: + $ref: "#/components/schemas/TrafficRoute" + entry_regions: + type: array + items: + type: object + properties: + region: + type: string + share: + type: number + cross_region_share: + type: number + description: Share of bytes served by a replica in another region than the entry region + commands: + type: object + description: Command counts over the period, absent when the database saw none + properties: + rest_commands: + type: number + description: Commands received over REST + script_commands: + type: number + description: Commands that Lua scripts ran internally + BackupAnalysis: + type: object + properties: + analysis_state: + type: string + description: Empty when never analyzed, otherwise pending, completed or failed + example: completed + report: + type: object + description: >- + The analysis report, present when completed. Contains totals, per-type size and + element-count distributions, the biggest keys overall and per type, and a TTL profile. CommandCount: type: object properties: @@ -2351,6 +2948,10 @@ components: description: Size of the backup format: int64 example: 0 + analysis_state: + type: string + description: State of the backup's analysis when one was requested, pending, completed or failed + example: completed daily_backup: type: string description: Daily backup status diff --git a/docs.json b/docs.json index 062515736..0db890107 100644 --- a/docs.json +++ b/docs.json @@ -2347,6 +2347,8 @@ "GET /redis/databases", "GET /redis/database/{id}", "GET /redis/stats/{id}", + "GET /redis/traffic/{id}", + "GET /redis/insights/{id}", "POST /redis/database", "POST /redis/rename/{id}", "POST /redis/reset-password/{id}", @@ -2364,6 +2366,8 @@ "group": "Backup", "pages": [ "GET /redis/list-backup/{id}", + "GET /redis/backup-analysis/{id}/{backupid}", + "POST /redis/backup-analysis/{id}/{backupid}", "POST /redis/create-backup/{id}", "POST /redis/restore-backup/{id}", "PATCH /redis/enable-dailybackup/{id}", diff --git a/llms-full.txt b/llms-full.txt index 4f4817a58..026f8b351 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -14459,6 +14459,12 @@ not being used anymore; you should delete it. You can delete the API keys in the **Stats:** We will provide reports based on usage of your API keys. +# Analyze Backup +Source: https://upstash.com/docs/devops/developer-api/redis/backup/analyze_backup + +/devops/developer-api/openapi.yaml post /redis/backup-analysis/{id}/{backupid} +This endpoint starts the offline analysis of a completed backup. The analysis scans the backup without touching the running database and takes minutes for large backups; poll the GET endpoint until the state is completed or failed. Available for databases on a Pro plan or with the Production Pack; other databases get 404. Not available for RDB exports, backups over 200 GiB, HIPAA databases or regional databases, nor for a backup whose analysis is already pending or completed. + # Create Backup Source: https://upstash.com/docs/devops/developer-api/redis/backup/create_backup @@ -14483,6 +14489,12 @@ Source: https://upstash.com/docs/devops/developer-api/redis/backup/enable_dailyb /devops/developer-api/openapi.yaml patch /redis/enable-dailybackup/{id} This endpoint enables daily backup for a Redis database. +# Get Backup Analysis +Source: https://upstash.com/docs/devops/developer-api/redis/backup/get_backup_analysis + +/devops/developer-api/openapi.yaml get /redis/backup-analysis/{id}/{backupid} +This endpoint returns the state of a backup's analysis and, once completed, the report. Daily backups of databases on a Pro plan or with the Production Pack are analyzed automatically when they complete, except on regional and HIPAA databases and for backups over 200 GiB. + # List Backup Source: https://upstash.com/docs/devops/developer-api/redis/backup/list_backup @@ -14553,12 +14565,24 @@ Source: https://upstash.com/docs/devops/developer-api/redis/get_database /devops/developer-api/openapi.yaml get /redis/database/{id} This endpoint gets details of a database. +# Get Database Insights +Source: https://upstash.com/docs/devops/developer-api/redis/get_database_insights + +/devops/developer-api/openapi.yaml get /redis/insights/{id} +This endpoint returns the Insights series of a database: service latency percentiles, read, write, engine, replication, lock-wait and Lua latency, expired entries, replication lag, throughput, bandwidth and disk size, for the whole database, one replica or every region, with each series' latest window compared to the one before it, and how much the limit, security and Lua counters grew over the period. Latencies are in milliseconds. Available for databases on a Pro plan or with the Production Pack. + # Get Database Stats Source: https://upstash.com/docs/devops/developer-api/redis/get_database_stats /devops/developer-api/openapi.yaml get /redis/stats/{id} This endpoint gets detailed stats of a database. +# Get Database Traffic +Source: https://upstash.com/docs/devops/developer-api/redis/get_database_traffic + +/devops/developer-api/openapi.yaml get /redis/traffic/{id} +This endpoint reports where a database's traffic comes from over a period. Available for databases on a Pro plan or with the Production Pack. + # List Databases Source: https://upstash.com/docs/devops/developer-api/redis/list_databases @@ -82995,7 +83019,7 @@ This guide will walk you through the steps to seamlessly connect your Datadog ac **Integration Scope** -Upstash Datadog Integration only covers Pro databases or those included in the Enterprise Plan. +Upstash Datadog Integration only covers Pro databases, databases with the Production Pack, and those included in the Enterprise Plan. @@ -83056,6 +83080,38 @@ Upstash will suspend all metric publishing process after the you remove Datadog After removing the integration on the Upstash side, it's crucial to go to your Datadog account and remove any related API keys or configurations associated with the integration. +## Metrics Published + +Metrics arrive in Datadog under `upstash.db.` with `team` and `database` tags, refreshed every few minutes. Latencies are in milliseconds over the last ten minutes; a latency with no samples in that window is not sent. + +| Metric | What it measures | +|---|---| +| `dailyprocessedcommands`, `dailyreadcommands`, `dailywritecommands` | Commands today, and the read and write share | +| `monthlycommands` | Commands this month | +| `readpersecond`, `writepersecond`, `throughput` | Commands per second by type, and in total | +| `hitrate`, `missrate` | Key hits and misses per second | +| `connections`, `restconnections` | Open TCP and REST connections | +| `totaldatasize`, `keyspace` | Data size in bytes, and number of keys | +| `dailybandwidth`, `dailybandwidthin`, `dailybandwidthout` | Bandwidth today, total and by direction, bytes | +| `monthlycost` | Cost this month | +| `overalllatencyp50`, `overalllatencyp99` | Simple service latency: server time from reading a request to writing its response, for requests where every command is a single-key read or write; multi-key commands, scripts and cross-region requests are excluded | +| `readlatencyp50`, `readlatencyp99`, `writelatencyp50`, `writelatencyp99` | Read and write request latency | +| `overalllatencyp90` | Simple service latency, p90 | +| `enginelatencyp99` | Time the engine spent executing commands, excluding network and queueing | +| `lockwaitlatencyp99` | Time commands waited for the execution lock | +| `replicationlatencyp999` | Time writes were held because of replication, p99.9 | +| `lualatencyp50`, `lualatencyp99` | Lua script execution latency | +| `throttledcommands` | Writes delayed today because persistence was falling behind | +| `authfailures` | Refused authentications and commands sent before authenticating, per second, over TCP and REST. REST requests over the rate limit are counted here as well | +| `restcommands` | Commands received over REST today | +| `expiredentriespermin` | Keys removed per minute because their TTL passed | +| `replicationlag` | Updates written on the primary that have not yet been sent to the replica | +| `oversizerequests`, `oversizerecordaccesses` | Requests over the request size limit, and accesses to keys over the record size limit, today | +| `keyspacenotifications`, `connectionsopened` | Keyspace notifications sent and TCP connections opened, per second | +| `luaexecutions`, `luacommands` | Script executions today, and the commands they ran | +| `luafailures`, `luatimeouts`, `luamemorylimits` | Script runs today that failed, timed out or hit the memory limit | +| `undeliverable` | Requests and connections the Upstash proxies could not hand to any replica today | + ## Pricing If you choose to integrate Datadog via Upstash, there will be an additional cost of $5 per month. @@ -83271,6 +83327,8 @@ Requests enter Upstash at the region closest to the client, whether or not your Cross-region requests are expected. Writes always go to the primary region, and reads go to the nearest replica. If many reads cross regions, adding a [read region](/docs/redis/features/globaldatabase) close to those clients reduces their latency. +The routes, client networks and client types are also available from the [Get Database Traffic](/docs/devops/developer-api/redis/get_database_traffic) endpoint. + ### Where clients connect from The IP ranges your requests came from, busiest first, with the cloud each range belongs to, the entry region its requests used, and its share of the sampled requests. Hover a source to see its cloud region, when the range has one. Ranges are matched against the IP ranges that AWS, Google Cloud and Cloudflare publish. Serverless platforms show up as the cloud they run on, for example Vercel functions as AWS. Other addresses show up as Internet. @@ -83306,6 +83364,8 @@ Charts of throughput, bandwidth, disk size, latency, expired entries and replica | Expired entries | Keys removed per minute because their TTL passed. | | Replication lag | Updates written on the primary that have not been sent yet to the replica in each region. | +The same series, with each chart's comparison and anomalies, are available from the [Get Database Insights](/docs/devops/developer-api/redis/get_database_insights) endpoint. + ## Limits How close the database is to the limits of its plan, and the requests a limit refused or delayed. @@ -83338,7 +83398,7 @@ An analysis of what your keys hold, made from an offline scan of the latest anal * **Biggest keys**: the ten largest keys, with their type, size, element count and remaining TTL. Pick a type to see the largest keys of that type. * **Expiration profile**: keys grouped by remaining TTL range, with the number of keys, their share of the keys that have a TTL, and their size. The last row is the total of every key with a TTL. -The keyspace is analyzed after every daily backup, so [daily backups](/docs/redis/features/backup) must be turned on. The analysis is not available for HIPAA-compliant databases. +The keyspace is analyzed after every daily backup, so [daily backups](/docs/redis/features/backup) must be turned on. The analysis is not available for HIPAA-compliant or regional databases. To analyze another backup, use the [Analyze Backup](/docs/devops/developer-api/redis/backup/analyze_backup) endpoint. # Use IP Allowlist Source: https://upstash.com/docs/redis/howto/ipallowlist @@ -83534,54 +83594,47 @@ Total number of requests per day, over the last 5 days. ## Database Usage -If you click on the "Usage" tab, you can see more detailed charts about the usage of your database. +The "Usage" tab of a database shows detailed charts. Pick the period at the top of the tab: the past hour, 3 hours, 12 hours, day, 3 days or week. Databases on a Pro plan or with the Production Pack can also select the past month. Charts refresh every minute. + +### Top Commands + +The ten most executed commands over the period, one line each. ### Throughput Redis throughput chart -Throughput chart shows throughput values for reads, writes and commands (all -commands including reads and writes) per second. The chart covers the last 1 -hour and it is updated every 10 seconds. +Reads, writes and all commands per second. ### Service Time Latency Redis latency chart -This chart shows the processing time of the request between it is received by -the server and the response is sent to the caller. It shows the times in max, -mean, min, 99.9 percentile and 99.99 percentile. The chart covers the last 1 -hour and it is updated every 10 seconds. +The time between a request reaching the server and its response leaving it, for reads and writes separately, as the median (p50) and the 99th percentile (p99). Every latency chart describes a sliding ten-minute window, so it reacts within minutes; an idle database shows no data rather than zero. ### Data Size Redis data size chart -This chart shows the data size of your database. The chart covers the last 24 -hours and it is updated every 10 seconds. +The data size of your database on disk. ### Connections Redis connection count chart -This chart shows the number of active client connections. It shows the number of -open connections plus the number of short-lived connections that started and -terminated in 10 seconds period. The chart covers the last 1 hour and it is -updated every 10 seconds. +Open TCP connections and open REST connections. ### Key Space Redis keyspace chart -This chart shows the number of keys. The chart covers the last 24 hours and it -is updated every 10 seconds. +The number of keys in the database. ### Hits / Misses Redis cache hits and misses chart -This chart shows the number of hits per second and misses per second. The chart -covers the last 1 hour and it is updated every 10 seconds. +Key hits and misses per second. # Migrate Regional to Global Database Source: https://upstash.com/docs/redis/howto/migratefromregionaltoglobal @@ -84509,7 +84562,7 @@ To monitor your Upstash database in Prometheus and visualize metrics in Grafana, **Integration Scope** -Upstash Prometheus Integration only covers Pro databases or those included in the Enterprise Plan. +Upstash Prometheus Integration covers Pro databases, databases included in the Enterprise Plan, and databases with the Production Pack. @@ -84601,6 +84654,48 @@ Please check this address to access Upstash Grafana Dashboard ` | Executions of each command today, for example `command_GET` | +| `daily_used_bandwidth`, `daily_bandwidth_ingress`, `daily_bandwidth_egress` | Bandwidth today, total and by direction, bytes | +| `monthly_used_bandwidth` | Bandwidth this month, bytes | +| `current_client_count`, `total_client_count` | Open TCP connections, and connections opened since start | +| `rest_conn_count` | Open REST connections | +| `total_hits`, `total_misses` | Key hits and misses since start | +| `simple_service_latency_p50`, `_p90`, `_p99` | Simple service latency: server time from reading a request to writing its response, for requests where every command is a single-key read or write; multi-key commands, scripts and cross-region requests are excluded | +| `read_service_latency_p50`, `_p99`, `_p99_9` | Read request latency | +| `write_service_latency_p50`, `_p99`, `_p99_9` | Write request latency | +| `command_latency_p50`, `_p99` | Time the engine spent executing commands, excluding network and queueing | +| `wait_latency_p99` | Time commands waited for the execution lock | +| `replication_latency_p99_9` | Time writes were held because of replication: waiting for the backup to acknowledge, or backpressure while a replica catches up | +| `replication_backup_lag` | Updates written on the primary that have not yet been sent to the replica | +| `lua_latency_p50`, `_p99`, `_p99_9`, `_p99_99` | Lua script execution latency | +| `daily_script_commands`, `daily_commands_in_scripts` | Script executions today, and the commands they ran | +| `lua_script_count`, `lua_script_failures`, `lua_script_timeouts`, `lua_script_memory_limits` | Scripts tracked today, and how many runs failed, timed out or hit the memory limit | +| `throttled_commands` | Writes delayed today because persistence was falling behind | +| `auth_failures`, `rest_auth_failures` | Refused authentications over TCP and REST, and commands sent before authenticating, since the database started. REST requests over the rate limit are counted in `rest_auth_failures` as well | +| `tcp_over_size_requests`, `rest_over_size_requests` | Requests rejected for exceeding the request size limit today | +| `over_size_record_accesses` | Accesses to keys above the record size limit today | +| `monitor_count`, `monitored_commands` | Clients running MONITOR, and commands they observed | +| `keyspace_notifications` | Keyspace notifications sent since start | +| `inmemory_memory_expired_entry_count`, `persistence_expired_entry_count` | Keys removed because their TTL passed since start, in memory and on disk | +| `slow_command_`, `slow_command_count_` | Commands that took 500 milliseconds or longer today: the slowest run in nanoseconds and how often it happened, for example `slow_command_count_KEYS` | +| `proxy_undeliverable` | Requests and connections the proxies of a region could not hand to any replica today | + ## **Conclusion** You've now integrated your database with Upstash Prometheus, providing access to improved monitoring and analytics. @@ -85446,7 +85541,7 @@ For more details, see the [Redis pricing page](https://upstash.com/pricing/redis ### More Backup Capability -Backups up to 3 days are possible with the Prod Pack add-on. +Backups up to 3 days are possible with the Prod Pack add-on. Enabling Prod Pack turns on daily backups, and every daily backup of a global, non-HIPAA database is analyzed for the keyspace view of the Insights tab. The latest daily backup is included in Prod Pack at no extra charge; keeping the last 3 days bills the two older backups as usual. The first one is taken as soon as Prod Pack is enabled. They stay on while Prod Pack is enabled. Disabling Prod Pack turns off the daily backups it turned on, unless you chose to keep the last 3 days or the database is HIPAA-compliant; daily backups you had enabled before Prod Pack, and their retention, stay as they were. No backup is deleted: existing daily backups stay, and are billed as usual, until you delete them. ### Encryption at Rest diff --git a/llms.txt b/llms.txt index 4e5fde094..e9ae80877 100644 --- a/llms.txt +++ b/llms.txt @@ -139,10 +139,12 @@ Search these docs: GET https://upstash.com/docs/search?q= returns JSON { - [Authentication](https://upstash.com/docs/devops/developer-api/authentication.md): Authentication for the Upstash Developer API - [HTTP Status Codes](https://upstash.com/docs/devops/developer-api/http_status_codes.md): The Upstash API uses the following HTTP Status codes: - [Getting Started](https://upstash.com/docs/devops/developer-api/introduction.md) +- [Analyze Backup](https://upstash.com/docs/devops/developer-api/redis/backup/analyze_backup.md): This endpoint starts the offline analysis of a completed backup. The analysis scans the backup without touching the running database and takes minutes for large backups; poll the GET endpoint until the state is completed or failed. Available for databases on a Pro plan or with the Production Pack; other databases get 404. Not available for RDB exports, backups over 200 GiB, HIPAA databases or regional databases, nor for a backup whose analysis is already pending or completed. - [Create Backup](https://upstash.com/docs/devops/developer-api/redis/backup/create_backup.md): This endpoint creates a backup for a Redis database. - [Delete Backup](https://upstash.com/docs/devops/developer-api/redis/backup/delete_backup.md): This endpoint deletes a backup of a Redis database. - [Disable Daily Backup](https://upstash.com/docs/devops/developer-api/redis/backup/disable_dailybackup.md): This endpoint disables daily backup for a Redis database. - [Enable Daily Backup](https://upstash.com/docs/devops/developer-api/redis/backup/enable_dailybackup.md): This endpoint enables daily backup for a Redis database. +- [Get Backup Analysis](https://upstash.com/docs/devops/developer-api/redis/backup/get_backup_analysis.md): This endpoint returns the state of a backup's analysis and, once completed, the report. Daily backups of databases on a Pro plan or with the Production Pack are analyzed automatically when they complete, except on regional and HIPAA databases and for backups over 200 GiB. - [List Backup](https://upstash.com/docs/devops/developer-api/redis/backup/list_backup.md): This endpoint lists all backups for a Redis database. - [Restore Backup](https://upstash.com/docs/devops/developer-api/redis/backup/restore_backup.md): This endpoint restores data from an existing backup. - [Change Database Plan](https://upstash.com/docs/devops/developer-api/redis/change_plan.md): This endpoint changes the plan of a Redis database. @@ -154,7 +156,9 @@ Search these docs: GET https://upstash.com/docs/search?q= returns JSON { - [Enable Eviction](https://upstash.com/docs/devops/developer-api/redis/enable_eviction.md): This endpoint enables eviction for given database. - [Enable TLS](https://upstash.com/docs/devops/developer-api/redis/enable_tls.md): This endpoint enables tls on a database. - [Get Database](https://upstash.com/docs/devops/developer-api/redis/get_database.md): This endpoint gets details of a database. +- [Get Database Insights](https://upstash.com/docs/devops/developer-api/redis/get_database_insights.md): This endpoint returns the Insights series of a database: service latency percentiles, read, write, engine, replication, lock-wait and Lua latency, expired entries, replication lag, throughput, bandwidth and disk size, for the whole database, one replica or every region, with each series' latest window compared to the one before it, and how much the limit, security and Lua counters grew over the period. Latencies are in milliseconds. Available for databases on a Pro plan or with the Production Pack. - [Get Database Stats](https://upstash.com/docs/devops/developer-api/redis/get_database_stats.md): This endpoint gets detailed stats of a database. +- [Get Database Traffic](https://upstash.com/docs/devops/developer-api/redis/get_database_traffic.md): This endpoint reports where a database's traffic comes from over a period. Available for databases on a Pro plan or with the Production Pack. - [List Databases](https://upstash.com/docs/devops/developer-api/redis/list_databases.md): This endpoint list all databases of user. - [Move To Team](https://upstash.com/docs/devops/developer-api/redis/moveto_team.md): This endpoint moves database under a target team - [Rename Database](https://upstash.com/docs/devops/developer-api/redis/rename_database.md): This endpoint renames a database. diff --git a/redis/howto/datadog.mdx b/redis/howto/datadog.mdx index a8bbb276d..d9c76e968 100755 --- a/redis/howto/datadog.mdx +++ b/redis/howto/datadog.mdx @@ -8,7 +8,7 @@ This guide will walk you through the steps to seamlessly connect your Datadog ac **Integration Scope** -Upstash Datadog Integration only covers Pro databases or those included in the Enterprise Plan. +Upstash Datadog Integration only covers Pro databases, databases with the Production Pack, and those included in the Enterprise Plan. @@ -69,6 +69,38 @@ Upstash will suspend all metric publishing process after the you remove Datadog After removing the integration on the Upstash side, it's crucial to go to your Datadog account and remove any related API keys or configurations associated with the integration. +## Metrics Published + +Metrics arrive in Datadog under `upstash.db.` with `team` and `database` tags, refreshed every few minutes. Latencies are in milliseconds over the last ten minutes; a latency with no samples in that window is not sent. + +| Metric | What it measures | +|---|---| +| `dailyprocessedcommands`, `dailyreadcommands`, `dailywritecommands` | Commands today, and the read and write share | +| `monthlycommands` | Commands this month | +| `readpersecond`, `writepersecond`, `throughput` | Commands per second by type, and in total | +| `hitrate`, `missrate` | Key hits and misses per second | +| `connections`, `restconnections` | Open TCP and REST connections | +| `totaldatasize`, `keyspace` | Data size in bytes, and number of keys | +| `dailybandwidth`, `dailybandwidthin`, `dailybandwidthout` | Bandwidth today, total and by direction, bytes | +| `monthlycost` | Cost this month | +| `overalllatencyp50`, `overalllatencyp99` | Simple service latency: server time from reading a request to writing its response, for requests where every command is a single-key read or write; multi-key commands, scripts and cross-region requests are excluded | +| `readlatencyp50`, `readlatencyp99`, `writelatencyp50`, `writelatencyp99` | Read and write request latency | +| `overalllatencyp90` | Simple service latency, p90 | +| `enginelatencyp99` | Time the engine spent executing commands, excluding network and queueing | +| `lockwaitlatencyp99` | Time commands waited for the execution lock | +| `replicationlatencyp999` | Time writes were held because of replication, p99.9 | +| `lualatencyp50`, `lualatencyp99` | Lua script execution latency | +| `throttledcommands` | Writes delayed today because persistence was falling behind | +| `authfailures` | Refused authentications and commands sent before authenticating, per second, over TCP and REST. REST requests over the rate limit are counted here as well | +| `restcommands` | Commands received over REST today | +| `expiredentriespermin` | Keys removed per minute because their TTL passed | +| `replicationlag` | Updates written on the primary that have not yet been sent to the replica | +| `oversizerequests`, `oversizerecordaccesses` | Requests over the request size limit, and accesses to keys over the record size limit, today | +| `keyspacenotifications`, `connectionsopened` | Keyspace notifications sent and TCP connections opened, per second | +| `luaexecutions`, `luacommands` | Script executions today, and the commands they ran | +| `luafailures`, `luatimeouts`, `luamemorylimits` | Script runs today that failed, timed out or hit the memory limit | +| `undeliverable` | Requests and connections the Upstash proxies could not hand to any replica today | + ## Pricing If you choose to integrate Datadog via Upstash, there will be an additional cost of $5 per month. diff --git a/redis/howto/insights.mdx b/redis/howto/insights.mdx index de75b5707..896c13fa1 100644 --- a/redis/howto/insights.mdx +++ b/redis/howto/insights.mdx @@ -27,6 +27,8 @@ Requests enter Upstash at the region closest to the client, whether or not your Cross-region requests are expected. Writes always go to the primary region, and reads go to the nearest replica. If many reads cross regions, adding a [read region](/redis/features/globaldatabase) close to those clients reduces their latency. +The routes, client networks and client types are also available from the [Get Database Traffic](/devops/developer-api/redis/get_database_traffic) endpoint. + ### Where clients connect from The IP ranges your requests came from, busiest first, with the cloud each range belongs to, the entry region its requests used, and its share of the sampled requests. Hover a source to see its cloud region, when the range has one. Ranges are matched against the IP ranges that AWS, Google Cloud and Cloudflare publish. Serverless platforms show up as the cloud they run on, for example Vercel functions as AWS. Other addresses show up as Internet. @@ -68,6 +70,8 @@ Charts of throughput, bandwidth, disk size, latency, expired entries and replica | Expired entries | Keys removed per minute because their TTL passed. | | Replication lag | Updates written on the primary that have not been sent yet to the replica in each region. | +The same series, with each chart's comparison and anomalies, are available from the [Get Database Insights](/devops/developer-api/redis/get_database_insights) endpoint. + ## Limits How close the database is to the limits of its plan, and the requests a limit refused or delayed. @@ -106,4 +110,4 @@ An analysis of what your keys hold, made from an offline scan of the latest anal - **Biggest keys**: the ten largest keys, with their type, size, element count and remaining TTL. Pick a type to see the largest keys of that type. - **Expiration profile**: keys grouped by remaining TTL range, with the number of keys, their share of the keys that have a TTL, and their size. The last row is the total of every key with a TTL. -The keyspace is analyzed after every daily backup, so [daily backups](/redis/features/backup) must be turned on. The analysis is not available for HIPAA-compliant databases. +The keyspace is analyzed after every daily backup, so [daily backups](/redis/features/backup) must be turned on. The analysis is not available for HIPAA-compliant or regional databases. To analyze another backup, use the [Analyze Backup](/devops/developer-api/redis/backup/analyze_backup) endpoint. diff --git a/redis/howto/metrics-and-charts.mdx b/redis/howto/metrics-and-charts.mdx index 7c268b466..05674614b 100755 --- a/redis/howto/metrics-and-charts.mdx +++ b/redis/howto/metrics-and-charts.mdx @@ -43,7 +43,11 @@ Total number of requests per day, over the last 5 days. ## Database Usage -If you click on the "Usage" tab, you can see more detailed charts about the usage of your database. +The "Usage" tab of a database shows detailed charts. Pick the period at the top of the tab: the past hour, 3 hours, 12 hours, day, 3 days or week. Databases on a Pro plan or with the Production Pack can also select the past month. Charts refresh every minute. + +### Top Commands + +The ten most executed commands over the period, one line each. ### Throughput @@ -51,9 +55,7 @@ If you click on the "Usage" tab, you can see more detailed charts about the usag Redis throughput chart -Throughput chart shows throughput values for reads, writes and commands (all -commands including reads and writes) per second. The chart covers the last 1 -hour and it is updated every 10 seconds. +Reads, writes and all commands per second. ### Service Time Latency @@ -61,10 +63,7 @@ hour and it is updated every 10 seconds. Redis latency chart -This chart shows the processing time of the request between it is received by -the server and the response is sent to the caller. It shows the times in max, -mean, min, 99.9 percentile and 99.99 percentile. The chart covers the last 1 -hour and it is updated every 10 seconds. +The time between a request reaching the server and its response leaving it, for reads and writes separately, as the median (p50) and the 99th percentile (p99). Every latency chart describes a sliding ten-minute window, so it reacts within minutes; an idle database shows no data rather than zero. ### Data Size @@ -72,8 +71,7 @@ hour and it is updated every 10 seconds. Redis data size chart -This chart shows the data size of your database. The chart covers the last 24 -hours and it is updated every 10 seconds. +The data size of your database on disk. ### Connections @@ -81,10 +79,7 @@ hours and it is updated every 10 seconds. Redis connection count chart -This chart shows the number of active client connections. It shows the number of -open connections plus the number of short-lived connections that started and -terminated in 10 seconds period. The chart covers the last 1 hour and it is -updated every 10 seconds. +Open TCP connections and open REST connections. ### Key Space @@ -92,8 +87,7 @@ updated every 10 seconds. Redis keyspace chart -This chart shows the number of keys. The chart covers the last 24 hours and it -is updated every 10 seconds. +The number of keys in the database. ### Hits / Misses @@ -101,5 +95,4 @@ is updated every 10 seconds. Redis cache hits and misses chart -This chart shows the number of hits per second and misses per second. The chart -covers the last 1 hour and it is updated every 10 seconds. +Key hits and misses per second. diff --git a/redis/integrations/prometheus.mdx b/redis/integrations/prometheus.mdx index 8be559987..e93283beb 100755 --- a/redis/integrations/prometheus.mdx +++ b/redis/integrations/prometheus.mdx @@ -8,7 +8,7 @@ To monitor your Upstash database in Prometheus and visualize metrics in Grafana, **Integration Scope** -Upstash Prometheus Integration only covers Pro databases or those included in the Enterprise Plan. +Upstash Prometheus Integration covers Pro databases, databases included in the Enterprise Plan, and databases with the Production Pack. @@ -100,6 +100,48 @@ Please check this address to access Upstash Grafana Dashboard ` | Executions of each command today, for example `command_GET` | +| `daily_used_bandwidth`, `daily_bandwidth_ingress`, `daily_bandwidth_egress` | Bandwidth today, total and by direction, bytes | +| `monthly_used_bandwidth` | Bandwidth this month, bytes | +| `current_client_count`, `total_client_count` | Open TCP connections, and connections opened since start | +| `rest_conn_count` | Open REST connections | +| `total_hits`, `total_misses` | Key hits and misses since start | +| `simple_service_latency_p50`, `_p90`, `_p99` | Simple service latency: server time from reading a request to writing its response, for requests where every command is a single-key read or write; multi-key commands, scripts and cross-region requests are excluded | +| `read_service_latency_p50`, `_p99`, `_p99_9` | Read request latency | +| `write_service_latency_p50`, `_p99`, `_p99_9` | Write request latency | +| `command_latency_p50`, `_p99` | Time the engine spent executing commands, excluding network and queueing | +| `wait_latency_p99` | Time commands waited for the execution lock | +| `replication_latency_p99_9` | Time writes were held because of replication: waiting for the backup to acknowledge, or backpressure while a replica catches up | +| `replication_backup_lag` | Updates written on the primary that have not yet been sent to the replica | +| `lua_latency_p50`, `_p99`, `_p99_9`, `_p99_99` | Lua script execution latency | +| `daily_script_commands`, `daily_commands_in_scripts` | Script executions today, and the commands they ran | +| `lua_script_count`, `lua_script_failures`, `lua_script_timeouts`, `lua_script_memory_limits` | Scripts tracked today, and how many runs failed, timed out or hit the memory limit | +| `throttled_commands` | Writes delayed today because persistence was falling behind | +| `auth_failures`, `rest_auth_failures` | Refused authentications over TCP and REST, and commands sent before authenticating, since the database started. REST requests over the rate limit are counted in `rest_auth_failures` as well | +| `tcp_over_size_requests`, `rest_over_size_requests` | Requests rejected for exceeding the request size limit today | +| `over_size_record_accesses` | Accesses to keys above the record size limit today | +| `monitor_count`, `monitored_commands` | Clients running MONITOR, and commands they observed | +| `keyspace_notifications` | Keyspace notifications sent since start | +| `inmemory_memory_expired_entry_count`, `persistence_expired_entry_count` | Keys removed because their TTL passed since start, in memory and on disk | +| `slow_command_`, `slow_command_count_` | Commands that took 500 milliseconds or longer today: the slowest run in nanoseconds and how often it happened, for example `slow_command_count_KEYS` | +| `proxy_undeliverable` | Requests and connections the proxies of a region could not hand to any replica today | + ## **Conclusion** You've now integrated your database with Upstash Prometheus, providing access to improved monitoring and analytics. diff --git a/redis/overall/enterprise.mdx b/redis/overall/enterprise.mdx index 552c9a5c7..fbabc9ac1 100755 --- a/redis/overall/enterprise.mdx +++ b/redis/overall/enterprise.mdx @@ -48,7 +48,7 @@ For more details, see the [Redis pricing page](https://upstash.com/pricing/redis ### More Backup Capability -Backups up to 3 days are possible with the Prod Pack add-on. +Backups up to 3 days are possible with the Prod Pack add-on. Enabling Prod Pack turns on daily backups, and every daily backup of a global, non-HIPAA database is analyzed for the keyspace view of the Insights tab. The latest daily backup is included in Prod Pack at no extra charge; keeping the last 3 days bills the two older backups as usual. The first one is taken as soon as Prod Pack is enabled. They stay on while Prod Pack is enabled. Disabling Prod Pack turns off the daily backups it turned on, unless you chose to keep the last 3 days or the database is HIPAA-compliant; daily backups you had enabled before Prod Pack, and their retention, stay as they were. No backup is deleted: existing daily backups stay, and are billed as usual, until you delete them. ### Encryption at Rest