Logstreem
Vector

Vector

An HTTP sink config, plus the transform that makes the output query well.

Vector talks to the HTTP ingest endpoint. The sink below batches, compresses and retries with sensible defaults.

vector.yaml
sources:
  app_logs:
    type: file
    include: ["/var/log/app/*.log"]

transforms:
  shape:
    type: remap
    inputs: [app_logs]
    source: |
      . = parse_json!(.message) ?? { "message": .message }
      ._time = to_unix_timestamp(now(), unit: "milliseconds")
      .host = get_hostname!()

sinks:
  logstreem:
    type: http
    inputs: [shape]
    uri: "https://api.logstreem.com/v1/ingest/api-gateway-prod"
    method: post
    encoding:
      codec: json
    compression: gzip
    framing:
      method: newline_delimited
    request:
      headers:
        Authorization: "Bearer ${LOGSTREEM_TOKEN}"
        Content-Type: "application/x-ndjson"
    batch:
      max_events: 1000
      timeout_secs: 2
    buffer:
      type: disk
      max_size: 268435488

Use a disk buffer

The buffer block above survives an ingest outage and a Vector restart without losing events, and without growing memory. It is the single most valuable line in this config.

Verify

Shell
vector validate vector.yaml
vector top   # watch the sink's events-out counter
PreviousOpenTelemetryNextFluent Bit