From b31be9e33c4aec0da9aa001adecad82111c82b44 Mon Sep 17 00:00:00 2001 From: Daniel Visca Date: Wed, 9 Sep 2026 07:20:55 -0700 Subject: [PATCH 1/3] Docs: Google Cloud Monitoring metrics installation page New installation page for pulling Google Cloud Monitoring metrics with the PostHog metrics agent (PostHog/posthog#97528): prerequisites and IAM, metric types vs descriptor filters, Docker and GKE Workload Identity setups, verification, config reference, and notes on sharding/quota/freshness/alpha. Also adds the page to the metrics install nav and the installation overview table. Generated-By: PostHog Desktop Task-Id: 025ef99f-893f-4131-b6f3-56c0c30a9c05 --- .../metrics/installation/google-cloud.mdx | 152 ++++++++++++++++++ contents/docs/metrics/installation/index.mdx | 3 +- src/navs/index.js | 1 + 3 files changed, 155 insertions(+), 1 deletion(-) create mode 100644 contents/docs/metrics/installation/google-cloud.mdx diff --git a/contents/docs/metrics/installation/google-cloud.mdx b/contents/docs/metrics/installation/google-cloud.mdx new file mode 100644 index 000000000000..95a5fc87ca76 --- /dev/null +++ b/contents/docs/metrics/installation/google-cloud.mdx @@ -0,0 +1,152 @@ +--- +title: Google Cloud Monitoring metrics installation +platformLogo: googleCloud +showStepsToc: true +--- + +import { Steps, Step } from "components/Docs/Steps"; +import MetricsNextSteps from "./_snippets/metrics-next-steps.mdx"; + +> **Note:** Metrics is in private alpha and the viewer is only turned on for selected teams, so you may not be able to view metrics you send yet. Setup details, including the ingestion endpoint, may change before general availability. + +Google Cloud Monitoring can't push metrics anywhere by itself, so the PostHog metrics agent pulls from it: set a GCP project and a list of metric types, and the agent polls the Cloud Monitoring API and forwards everything to PostHog. One `docker run` or one `helm install`, no application changes. + +This is the path for metrics only Google has – Cloud SQL, GKE, Cloud Run, load balancers, Pub/Sub, and the other managed services. For anything that exposes a `/metrics` endpoint, use [Docker](/docs/metrics/installation/docker) or [Kubernetes](/docs/metrics/installation/kubernetes) scraping instead; for code you control, the [SDK or OTLP paths](/docs/metrics/installation) need no agent at all. The same agent can scrape and pull at once. + + + + + +You need: + +- A GCP project with metrics in Cloud Monitoring +- A GCP service account with the **Monitoring Viewer** role (`roles/monitoring.viewer`, which allows `monitoring.timeSeries.list`) in that project +- Docker, or a GKE cluster with `helm` v3 +- Your PostHog project token + + + + + +You'll need your PostHog project token to authenticate metrics requests. This is the same token you use for capturing events with the PostHog SDK. + +> **Important:** Use your **project token**, which starts with `phc_`. Do **not** use a personal API key (which starts with `phx_`). + +You can find your project token in [Project Settings](https://app.posthog.com/settings). + + + + + +Tell the agent which metric types to pull. An explicit list: + +``` +compute.googleapis.com/instance/cpu/utilization +cloudsql.googleapis.com/database/cpu/utilization +``` + +Or pull a whole family with a [metric descriptor filter](https://cloud.google.com/monitoring/api/v3/filters): + +``` +metric.type = starts_with("compute.googleapis.com/instance/cpu/") +``` + +Each entry costs one Cloud Monitoring API call per pull, so a filter that covers a prefix is cheaper than a long list of names. Browse available types in [Google's metrics list](https://cloud.google.com/monitoring/api/metrics). + + + + + +### Docker + +Create a JSON key for the service account, make it world-readable inside the container (the agent runs as a non-root user, so `chmod 644 sa.json`), then: + +```bash +docker run -d --name posthog-metrics-agent \ + -e POSTHOG_API_KEY= \ + -e GCP_PROJECT_ID= \ + -e GCP_METRICS=compute.googleapis.com/instance/cpu/utilization,cloudsql.googleapis.com/database/cpu/utilization \ + -v "$PWD/sa.json:/etc/gcp/sa.json:ro" \ + -e GOOGLE_APPLICATION_CREDENTIALS=/etc/gcp/sa.json \ + posthog/metrics-agent:latest +``` + +For EU Cloud, add `-e POSTHOG_HOST=https://eu.i.posthog.com`. + +To use filters instead of a name list, swap in `GCP_METRIC_FILTERS` – filters are **semicolon**-separated, not comma-separated, because filter expressions can contain commas: + +```bash +-e GCP_METRIC_FILTERS='metric.type = starts_with("compute.googleapis.com/")' +``` + +### Kubernetes (Helm) + +On GKE, use [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) instead of a key file. Create the GCP service account, allow the agent's Kubernetes service account to impersonate it: + +```bash +gcloud iam service-accounts add-iam-policy-binding agent@.iam.gserviceaccount.com \ + --role roles/iam.workloadIdentityUser \ + --member "serviceAccount:.svc.id.goog[/posthog-metrics-agent]" +``` + +then install with the annotation: + +```yaml +# values.yaml +scrape: + annotationDiscovery: false +gcp: + projectId: + metrics: + - compute.googleapis.com/instance/cpu/utilization +serviceAccount: + annotations: + iam.gke.io/gcp-service-account: agent@.iam.gserviceaccount.com +``` + +```bash +helm install posthog-metrics-agent oci://ghcr.io/posthog/charts/posthog-metrics-agent \ + --set posthog.apiKey= \ + -f values.yaml +``` + +If you can't use Workload Identity, put the JSON key in a Kubernetes Secret and set `gcp.credentialsSecret.name` to its name; the chart mounts it for you. Scraping and pulling work together too: leave `scrape.annotationDiscovery` on and the agent does both. + + + + + +1. Check the agent started cleanly – `docker logs posthog-metrics-agent` or `kubectl logs -l app.kubernetes.io/name=posthog-metrics-agent` should show `Everything is ready. Begin running and processing data.` Credential and quota errors from Google appear in the same log. +2. Open [**Metrics**](https://app.posthog.com/metrics) in PostHog and filter to `service_name = google-cloud-monitoring`. Points appear within one pull interval (default 60 seconds). + +Set `GCP_SERVICE_NAME` (or `gcp.serviceName` in Helm) to change the `service_name` your pulled metrics arrive under – for example `gcp-production`. + + + View your metrics in PostHog + + + + + + + + +## Configuration reference + +| Variable | Default | Description | +| -------------------------------- | -------------------------- | ------------------------------------------------------------------------ | +| `GCP_PROJECT_ID` | – | GCP project to pull from. Setting it enables the source | +| `GCP_METRICS` | – | Comma-separated metric types to pull (this or filters, at least one) | +| `GCP_METRIC_FILTERS` | – | **Semicolon**-separated metric descriptor filters | +| `GCP_COLLECTION_INTERVAL` | `60s` | How often to poll. Values under `60s` are rejected by the collector | +| `GCP_SERVICE_NAME` | `google-cloud-monitoring` | `service_name` on every pulled metric | +| `GOOGLE_APPLICATION_CREDENTIALS` | – | Path to a service account JSON key. Omit when using Workload Identity | + +The Helm values mirror these under `gcp.`: `projectId`, `metrics`, `metricFilters`, `collectionInterval`, `serviceName`, `credentialsSecret.name`, `credentialsSecret.key`. For full control over the pull, mount a raw `metrics_list` at `/etc/posthog/gcp_metrics_list.yaml` (see the [agent README](https://github.com/PostHog/posthog/tree/master/products/metrics/agent)). + +## Notes and limits + +- **One puller only.** Cloud Monitoring can't be sharded: every agent instance would pull the same series and double-count. Don't combine with `SHARD_COUNT` / Helm `shards` – the agent refuses to start. Run a separate single instance for Cloud Monitoring alongside a sharded scrape fleet. +- **Quota and cost.** Each metric type or filter costs one `timeSeries.list` API call per interval. At the default 60 seconds that's 1,440 calls per entry per day, within the free Cloud Monitoring API allocation for typical lists – raise `GCP_COLLECTION_INTERVAL` for large lists. +- **Freshness.** Cloud Monitoring itself adds latency before a point is queryable (often 1–3 minutes for some services), so the newest data in PostHog lags real time by that plus one pull interval. +- **Alpha upstream.** The agent uses the OpenTelemetry Collector's `googlecloudmonitoring` receiver, which is alpha: metric naming and attributes can change when the collector version is bumped, and a restart can leave a gap of up to one pull interval. diff --git a/contents/docs/metrics/installation/index.mdx b/contents/docs/metrics/installation/index.mdx index 7e051d1033f3..64fc635da45b 100644 --- a/contents/docs/metrics/installation/index.mdx +++ b/contents/docs/metrics/installation/index.mdx @@ -8,7 +8,7 @@ There are three ways to get metrics into PostHog: - **PostHog SDKs**: if a PostHog SDK is already in your app, record metrics with the `posthog.metrics` API. No new packages, no extra authentication. - **OpenTelemetry (OTLP)**: if you use OpenTelemetry anywhere else, point your OTLP metrics exporter at PostHog. No PostHog packages required. -- **Metrics agent**: if your services already expose Prometheus `/metrics` endpoints, run the PostHog metrics agent with [Docker](/docs/metrics/installation/docker) or the [Kubernetes Helm chart](/docs/metrics/installation/kubernetes) to scrape and forward them, with no code changes. +- **Metrics agent**: if your services already expose Prometheus `/metrics` endpoints, run the PostHog metrics agent with [Docker](/docs/metrics/installation/docker) or the [Kubernetes Helm chart](/docs/metrics/installation/kubernetes) to scrape and forward them, with no code changes. The same agent can also [pull from Google Cloud Monitoring](/docs/metrics/installation/google-cloud) for managed-service metrics like Cloud SQL and load balancers. Already capturing metrics with Prometheus, StatsD, or Datadog? You don't need to replace anything: add the PostHog call next to your existing instrumentation, using the same metric name and attributes, and migrate at your own pace. @@ -22,5 +22,6 @@ Already capturing metrics with Prometheus, StatsD, or Datadog? You don't need to | [Other languages](/docs/metrics/installation/other) | Any OpenTelemetry-compatible OTLP metrics exporter | | [Docker](/docs/metrics/installation/docker) | Metrics agent container that scrapes Prometheus `/metrics` targets | | [Kubernetes](/docs/metrics/installation/kubernetes) | Helm chart that scrapes `prometheus.io/scrape` annotated pods | +| [Google Cloud](/docs/metrics/installation/google-cloud) | Metrics agent that pulls Google Cloud Monitoring metrics | > **Note:** Metrics uses the OpenTelemetry Protocol (OTLP) standard. If your language isn't listed, check the [OpenTelemetry documentation](https://opentelemetry.io/docs/) for compatible libraries and see [other languages](/docs/metrics/installation/other). diff --git a/src/navs/index.js b/src/navs/index.js index ecc64e7d6341..076f3b04cede 100644 --- a/src/navs/index.js +++ b/src/navs/index.js @@ -8316,6 +8316,7 @@ export const docsMenu = { { name: 'Other languages', url: '/docs/metrics/installation/other' }, { name: 'Docker', url: '/docs/metrics/installation/docker' }, { name: 'Kubernetes', url: '/docs/metrics/installation/kubernetes' }, + { name: 'Google Cloud', url: '/docs/metrics/installation/google-cloud' }, ], }, { From b31ee6507f29864816fd2114500f9e0d82ac1e31 Mon Sep 17 00:00:00 2001 From: Daniel Visca Date: Wed, 9 Sep 2026 07:23:26 -0700 Subject: [PATCH 2/3] Fix Vale EnDash error in google-cloud metrics docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces the bare en dash in a numeric range (1–3 minutes) with words per the PostHog docs style: en dashes take spaces. Generated-By: PostHog Desktop Task-Id: 025ef99f-893f-4131-b6f3-56c0c30a9c05 --- contents/docs/metrics/installation/google-cloud.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contents/docs/metrics/installation/google-cloud.mdx b/contents/docs/metrics/installation/google-cloud.mdx index 95a5fc87ca76..c610add90bcd 100644 --- a/contents/docs/metrics/installation/google-cloud.mdx +++ b/contents/docs/metrics/installation/google-cloud.mdx @@ -148,5 +148,5 @@ The Helm values mirror these under `gcp.`: `projectId`, `metrics`, `metricFilter - **One puller only.** Cloud Monitoring can't be sharded: every agent instance would pull the same series and double-count. Don't combine with `SHARD_COUNT` / Helm `shards` – the agent refuses to start. Run a separate single instance for Cloud Monitoring alongside a sharded scrape fleet. - **Quota and cost.** Each metric type or filter costs one `timeSeries.list` API call per interval. At the default 60 seconds that's 1,440 calls per entry per day, within the free Cloud Monitoring API allocation for typical lists – raise `GCP_COLLECTION_INTERVAL` for large lists. -- **Freshness.** Cloud Monitoring itself adds latency before a point is queryable (often 1–3 minutes for some services), so the newest data in PostHog lags real time by that plus one pull interval. +- **Freshness.** Cloud Monitoring itself adds latency before a point is queryable (often one to three minutes for some services), so the newest data in PostHog lags real time by that plus one pull interval. - **Alpha upstream.** The agent uses the OpenTelemetry Collector's `googlecloudmonitoring` receiver, which is alpha: metric naming and attributes can change when the collector version is bumped, and a restart can leave a gap of up to one pull interval. From c1d2b16dde57d48153cfeacf2be9a128e1ff2a0a Mon Sep 17 00:00:00 2001 From: Daniel Visca Date: Thu, 17 Sep 2026 10:26:39 -0700 Subject: [PATCH 3/3] metrics gcp docs: recommend uid-owned 0600 key copy, not chmod 644 The agent runs as uid 10001, so the safest key handoff is a dedicated copy owned by that uid with mode 0600 (the approach the agent README recommends), mounted into the container. `chmod 644` on the original service-account key would make the private key readable by every local user. Validated end-to-end against the real agent image: 0600 uid-owned and 644 root-owned both start and deliver metrics; 0600 root-owned is rejected by the agent's own entrypoint check. Generated-By: PostHog Desktop Task-Id: 6fb9271a-f921-4eac-a852-21d8691518c4 --- contents/docs/metrics/installation/google-cloud.mdx | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/contents/docs/metrics/installation/google-cloud.mdx b/contents/docs/metrics/installation/google-cloud.mdx index c610add90bcd..518bf8a8f678 100644 --- a/contents/docs/metrics/installation/google-cloud.mdx +++ b/contents/docs/metrics/installation/google-cloud.mdx @@ -59,14 +59,15 @@ Each entry costs one Cloud Monitoring API call per pull, so a filter that covers ### Docker -Create a JSON key for the service account, make it world-readable inside the container (the agent runs as a non-root user, so `chmod 644 sa.json`), then: +Create a JSON key for the service account. The agent runs as a non-root user (uid 10001), so keep a dedicated copy of the key readable only by that uid – do **not** `chmod 644` the original key, which would make the private key readable by every local user: ```bash +install -m 0600 -o 10001 sa.json /run/posthog-gcp/sa.json docker run -d --name posthog-metrics-agent \ -e POSTHOG_API_KEY= \ -e GCP_PROJECT_ID= \ -e GCP_METRICS=compute.googleapis.com/instance/cpu/utilization,cloudsql.googleapis.com/database/cpu/utilization \ - -v "$PWD/sa.json:/etc/gcp/sa.json:ro" \ + -v /run/posthog-gcp/sa.json:/etc/gcp/sa.json:ro \ -e GOOGLE_APPLICATION_CREDENTIALS=/etc/gcp/sa.json \ posthog/metrics-agent:latest ```