Docs / Ingest protocol

Ingest protocol

How an agent delivers results to a regional endpoint such as network.pingform.org. Two transports, one semantic model.

Transport selection

The agent prefers gRPC. It falls back to WebSocket when ALPN negotiation yields http/1.1, when an intermediary rejects long-lived h2 streams, or when transport: websocket is set explicitly. Both carry the same payloads and both are authenticated with the client certificate issued at enrolment.

	gRPC	WebSocket
ALPN	h2	http/1.1
Framing	length-prefixed protobuf	binary frames, same protobuf
Streams	multiplexed on one connection	one connection per stream
Compression	gzip	permessage-deflate
Keepalive	h2 PING, 45s	ping frame, 30s
Throughput ceiling	higher	lower, more resilient to middleboxes

gRPC

Results are OpenTelemetry metrics, so the ingest surface is the standard OTLP collector service — no PingForm-specific proto to generate.

service:   opentelemetry.proto.collector.metrics.v1.MetricsService
method:    /Export
endpoint:  network.pingform.org:443
alpn:      h2
auth:      mTLS (client certificate from enrolment)

Each ExportMetricsServiceRequest carries one resource scope per vantage point. Check identity travels in resource attributes:

resource:
  pingform.workspace: wk_8f3c91ba
  pingform.agent:     agt_3f91c2
  pingform.vantage:   eu-central
metrics:
  - name: pingform.check.duration
    unit: ms
    histogram: { ... }
    attributes: { check.id: api-edge, check.type: http, result: ok }
  - name: pingform.check.result
    unit: "1"
    sum: { ... }

The server acknowledges per batch. On RESOURCE_EXHAUSTED the agent applies the returned retry-after and keeps spooling; nothing is dropped until the local buffer is full.

WebSocket

The live stream is an upgrade on the ingest host. The last path segment is your workspace ID, so each workspace has its own URL:

GET wss://network.pingform.org/_agent/v2/stream/wk_8f3c91ba
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Version: 13
Sec-WebSocket-Protocol: pingform.v2
Sec-WebSocket-Extensions: permessage-deflate

Requests without an Upgrade header receive 404; the path is not a browsable resource. After the handshake, frames are the same protobuf messages as the gRPC path, binary, one message per frame. The server sends a ping every 30 seconds and closes with 1011 if two consecutive pongs are missed.

Backpressure and ordering

Samples are idempotent: (check.id, vantage, window_start) is the dedup key, so a replayed batch after reconnect is harmless.
Ordering is not guaranteed across vantage points and is not required — every sample is self-timestamped.
The agent spools to disk when the connection is down and replays oldest-first on reconnect, rate-limited to 25% of the negotiated ceiling so live data is never starved.

Self-test payload

Every ingest host serves a fixed 2 MiB incompressible payload used by pingform-agent selftest:

GET https://network.pingform.org/_agent/v2/selftest/pf7k2m.bin
→ 200, content-length: 2097152, cache-control: no-store

It exists to separate throughput problems from connectivity problems. It is not part of the data plane and carries no workspace data.