Skip to content

Added a generic HTTP logsink - #1063

Merged
javuto merged 1 commit into
developfrom
generic-http-logsink
Sep 17, 2026
Merged

javuto merged 1 commit into
developfrom
generic-http-logsink

Conversation

@javuto

@javuto javuto commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

Overview

Adds a new generic HTTP log sink (http) so osquery logs can be forwarded to any HTTP/HTTPS endpoint with configurable method, headers, format, and metadata wrapping. This is the twelfth sink type, joining Splunk, Graylog, Kafka, Kinesis, S3, Elastic, Logstash, DB, file, stdout, and none.

Configuration

The sink exposes seven fields through the existing dynamic form schema (no new frontend component needed):

Field Type Default Description
url string (required) — HTTP/HTTPS endpoint
method select POST HTTP method (POST or PUT)
format select json json (forward as-is), ndjson (one object per line), raw (bytes verbatim)
contentType string derived from format Overrides Content-Type header
headers string (JSON object) — Custom HTTP headers, e.g. {"Authorization":"Bearer xyz"}
includeMetadata boolean false Wraps each event with environment, uuid, log_type, timestamp
timeoutSeconds integer 30 Per-request timeout

Backend changes

Config (pkg/config)

  • loggers.go — new HTTPLogger struct with URL, method, headers (map[string]string), format, content type, timeout, and metadata toggle.
  • types.go — LoggingHTTP = "http" constant; HTTP *HTTPLogger field added to YAMLConfigurationLogger.
  • validation.go — LoggingHTTP registered as a valid logging type.

Exporter (pkg/logging)

  • http.go (new) — LoggerHTTP with a pooled http.Client, configurable timeout, and three encoding modes:
    • JSON — forwards the raw payload with application/json content type.
    • NDJSON — splits array payloads into newline-delimited JSON objects.
    • Raw — sends bytes verbatim with the configured content type.
    • When includeMetadata is true, each event is wrapped in an httpEnvelope adding environment, UUID, log type, and Unix timestamp.
    • Non-2xx responses are logged at warn level; errors do not stall the ingestion path.
  • exporter_adapters.go — Name/IsEnabled/Export methods so LoggerHTTP satisfies the DataExporter interface.
  • exporter_factory.go — config.LoggingHTTP case in CreateExporter.

Registry (pkg/logsinks)

  • logsinks.go — new config.LoggingHTTP entry in Registry with field specs for the dynamic form, a Build func, and a custom decodeHTTPConfig decoder that handles headers arriving as either a JSON object (API/YAML) or a JSON string (frontend single-line input). Seed path wired in configRowForType.
  • logsinks_test.go — TestRegistryCoversAllYAMLTypes updated to include http; added tests for registry presence, header decoding (object + string shapes), empty/null config, exporter build, and seed round-trip.

Frontend changes

  • LogSinksPage.tsx — added an icon (arrow-up/upload pictogram) for the http sink type in SINK_TYPE_ICONS.

Tests

  • pkg/logging/http_test.go (new) — 12 tests: defaults, nil config, JSON/NDJSON/raw formats, metadata envelopes (JSON + NDJSON), custom headers, PUT method, non-2xx response handling, Export adapter, Close idempotency.
  • pkg/logsinks/logsinks_test.go — 6 new tests: registry entry, header decoding variants, empty config, exporter build, seed round-trip.
  • pkg/config/validation_test.go — TestValidateTLSConfigValuesAcceptsHTTP.

Validation

  • go build ./... — pass
  • go test ./pkg/logging/... ./pkg/logsinks/... ./pkg/config/... — pass
  • golangci-lint run — 0 issues
  • npm run check (TypeScript) — pass
  • npm test (415 frontend tests) — pass
  • make openapi-check — no changes needed (sink types are served dynamically from the registry)

@javuto javuto added osctrl-tls osctrl-tls related changes 🗄️ logging Logging related issues ⭐️ frontend Frontend related issues labels Sep 17, 2026
@javuto
javuto merged commit 4201612 into develop Sep 17, 2026
8 checks passed
@javuto
javuto deleted the generic-http-logsink branch September 17, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🗄️ logging Logging related issues ⭐️ frontend Frontend related issues osctrl-tls osctrl-tls related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant