Alerts, as a signed webhook

Three events, one body

A subscriber routes on event and can ignore the other two — but the shape below is the same object every channel renders from, so an email and this request are two views of one fact rather than two descriptions that agree today.

incident.opened

A component this account watches has gone degraded or outage

Sent once, when the incident engine opens an incident on a component of a venue on the account’s watchlist, at or above the channel’s severity floor. The engine’s unit is a component (rest:ticker, ws:book) rather than a venue, so a venue whose REST is fine and whose order book feed has stalled produces one of these and not two.

incident.escalated

An open incident got worse — degraded became outage

The same incident, not a second one: the engine escalates in place, so incidentId is unchanged and severity is now outage. An account told about the degradation has to be told it got worse, which is why this is its own event rather than a repeat of incident.opened. A subscriber that groups by incidentId gets the update it wants for free.

incident.closed

The component recovered

state is operational and endedAt is set — and endedAt is when recovery began, not when the engine’s two-minute confirmation hold expired. Recovery is deliberately slower to publish than failure. A close for an opening that was never delivered — below the floor, or inside a cooldown — is skipped with a reason rather than sent, so this event never arrives on its own.

How it arrives

POST as application/json to the URL the channel was created with, HTTPS only. Redirects are not followed: a subscriber answering 301 would send the payload, and the signature, to a host the account never authorised.

A webhook channel chooses one of three formats when it is created, and this document describes the default. up24 is the signed payload below. slack is an incoming-webhook body and pagerduty is an Events v2 call to PagerDuty's endpoint for the service region — neither reads a signature and neither is sent one, because for both of them the URL or the routing key is itself the credential. Everything below is true of the up24 format; the retry policy, the delivery log and the severity floor are true of all three.

An up24 request carries x-up24-signature: t=<unix seconds>,v1=<hex hmac>, an HMAC-SHA256 over <t>.<raw body> with the secret shown once when the channel was created. Verify it against the raw bytes, before parsing: re-serialising the JSON changes them. The timestamp is inside the signed string, so a captured body cannot be replayed later under its original signature — reject anything more than 300s out of date.

Answer 2xx within 10 seconds. Anything else is a failure and is retried with exponential backoff up to 5 attempts; 10 consecutive failures switch the channel off and say so on the dashboard. Delivery is at-least-once, so de-duplicate on eventId — and a test alert carries eventId: 0 and test: true, which means a subscriber keying off that id ignores tests without a special case.

How fast it arrives

Measured, not estimated: over the last 30 days, half of the alerts up24 delivered reached their channel within 8s of the incident starting, and 95% within 11s — across 4 delivered alerts, detection time included.

Measured end to end, from the timestamp up24 puts on the fault to the moment the request left for your channel — so the incident engine's own detection time is inside it. That is the honest boundary: the clock a desk cares about starts when the venue breaks, not when a monitoring service finishes making up its mind. Openings only, because on a recovery the same subtraction would measure how long the outage lasted rather than how fast anyone was told; deliveries that actually left, because a skip is a row in your log with a reason but it is not a notification; and nothing that sat over an hour in the outbox before it was sent, because an event delivered long after it happened is history arriving at once rather than up24 paging anybody. It is read live from /v1/notify, which is the delivery log with two percentiles taken over it and nothing else.

The body

{
  "event": "incident.opened",
  "kind": "opened",
  "eventId": 48213,
  "incidentId": 1907,
  "venue": "binance",
  "venueName": "Binance",
  "component": "ws:book",
  "severity": "outage",
  "state": "outage",
  "summary": "Order book stream silent past its stall threshold in 2 of 3 regions",
  "errorClass": "stalled",
  "startedAt": "2026-08-30T14:12:05.000Z",
  "endedAt": null,
  "occurredAt": "2026-08-30T14:12:15.000Z",
  "url": "https://up24.app/exchange/binance"
}

Every field, from the same definition the dispatcher builds the body with — so a field named here is a field that arrives. Adding one is safe and will happen; renaming one is not, and would be a dated entry in the changelog.

eventstring

incident.opened, incident.escalated or incident.closed — the conventional shape for a webhook, and the field to route on. kind is the same value without the prefix.

kindstring

Which of the three this is. escalated is separate from opened because a degradation becoming an outage is one incident throughout — the engine escalates in place rather than opening a second.

"opened""escalated""closed"

eventIdinteger

The event’s own id. Delivery is at-least-once by design, so de-duplicate on this. Zero on a test alert, which corresponds to no event and no incident — a subscriber that keys off these ids therefore ignores tests for free instead of storing one under a fabricated real id.

incidentIdinteger

The incident all three events of one fault share, and therefore the right thing to group by — or to use as a pager’s deduplication key, so an escalation updates the page it opened and a close resolves it.

venuestring

The venue, as its id on this site.

"binance""bybit""coinbase""kraken""okx""bitfinex""deribit""bitstamp""gemini""cryptocom""hyperliquid""upbit""binance-futures""bybit-futures""okx-futures"

venueNamestring

The venue’s display name, as the board writes it.

componentstring

One REST endpoint or one WebSocket stream of one venue — rest:ticker, ws:book. ws:conn is the shared socket itself, which is where a connection-scoped fault is recorded.

matches ^(rest|ws):[a-z]+$ · e.g. "rest:ticker"

severitystring

What the engine graded the incident. The two it can open at; a channel’s severity floor decides which of them reaches you.

"degraded""outage"

statestring

What the component is right now — operational on a close, which is the one event where this and severity deliberately disagree.

"operational""degraded""outage""unknown"

summarystring

One sentence, produced by a switch over states with the measured numbers substituted in. Never model-written, which is why it is safe to forward verbatim.

errorClassstring | null

The dominant failure behind the incident — one of timeout, dns, tls, conn, http_429, http_403, http_451, http_4xx, http_5xx, bad_body, stalled, rejected, refused, unknown, lag — or null where there was no single one. stalled and rejected come only from stream monitoring: a stalled feed is not a timeout, the socket is open and the venue has simply stopped saying anything.

startedAtstring (date-time)

When the incident began, which is not when this event was sent.

endedAtstring (date-time) | null

Null while the incident is open. On a close it is when recovery began, not when the engine’s two-minute confirmation hold expired.

occurredAtstring (date-time)

When this event was written, within one engine tick of the change it reports. The delivery timestamp is on the dashboard’s log, not in the body.

urlstring

Where a human goes to see it: the venue’s page on up24, with the window, the denominator and the vantage points the verdict was reached from.

testbooleanoptional

Present and true only for the test button on the dashboard. Absent on every real alert.

Verifying it

This is the part worth reading twice. A desk that pulls its quotes when up24 says a venue is down has handed an endpoint on the public internet the ability to take it out of a market — so the question is not whether the request parses, it is whether it came from us. Both snippets below are run against a real signed alert in up24's own test suite on every deploy, which is the only reason they are worth pasting.

Two things a framework will do to you if you let it. Verify the raw bytes: the signature covers what was sent, and a JSON round trip is not guaranteed to give them back — in Express that means express.raw({ type: 'application/json' }) on this route, and in most Python frameworks it means the request body property rather than the parsed one. And compare in constant time: timingSafeEqual, hmac.compare_digest, not ===. pause and resume are yours.

import { createHash, createHmac, timingSafeEqual } from 'node:crypto';

const SECRET = process.env.UP24_WEBHOOK_SECRET;

// Verify the raw bytes, before anything parses them. JSON.parse followed by
// JSON.stringify is not guaranteed to give back what was signed.
export function verifyUp24(secret, header, rawBody, nowSeconds = Math.floor(Date.now() / 1000)) {
  const parts = new Map(
    String(header ?? '')
      .split(',')
      .map((piece) => piece.split('='))
      .map(([key, value]) => [key?.trim() ?? '', value?.trim() ?? '']),
  );

  const t = Number(parts.get('t'));
  const provided = parts.get('v1') ?? '';
  if (!Number.isFinite(t) || provided === '') return false;

  // The timestamp is inside the signed string, so it cannot be moved forward
  // to make a captured body look fresh.
  if (Math.abs(nowSeconds - t) > 300) return false;

  const expected = createHmac('sha256', secret).update(`${t}.${rawBody}`).digest('hex');
  const a = createHash('sha256').update(expected).digest();
  const b = createHash('sha256').update(provided).digest();

  // Constant time. Hashing both sides to fixed 32-byte digests avoids RangeError on length mismatch.
  return timingSafeEqual(a, b);
}

// Deliveries are at-least-once, so the same eventId can arrive twice.
const seen = new Set();

export function onUp24Webhook(rawBody, signature) {
  if (!verifyUp24(SECRET, signature, rawBody)) return { status: 400 };

  const alert = JSON.parse(rawBody);

  // The dashboard's test button sends eventId 0 and test: true, so a
  // subscriber that keys off the id ignores tests without a special case.
  if (seen.has(alert.eventId)) return { status: 204 };
  seen.add(alert.eventId);

  if (alert.kind === 'closed') resume(alert.venue, alert.component);
  else pause(alert.venue, alert.component, alert.severity);

  return { status: 204 };
}

import hashlib
import hmac
import json
import time

# Verify the raw bytes, before anything parses them: a JSON round trip is not
# guaranteed to give back what was signed.
def verify_up24(secret: str, header: str, raw_body: bytes, now: int | None = None) -> bool:
    parts = dict(
        piece.split("=", 1) for piece in (header or "").split(",") if "=" in piece
    )
    try:
        t = int(parts["t"].strip())
        provided = parts["v1"].strip()
    except (KeyError, ValueError):
        return False

    # The timestamp is inside the signed string, so it cannot be moved forward
    # to make a captured body look fresh.
    if abs((int(time.time()) if now is None else now) - t) > 300:
        return False

    expected = hmac.new(
        secret.encode(), f"{t}.".encode() + raw_body, hashlib.sha256
    ).hexdigest()
    # Constant time: a byte-by-byte comparison of an HMAC leaks how much of a
    # guess was right.
    return hmac.compare_digest(expected, provided)


# Deliveries are at-least-once, so the same eventId can arrive twice.
seen: set[int] = set()


def on_up24_webhook(secret: str, headers, raw_body: bytes) -> int:
    if not verify_up24(secret, headers.get("x-up24-signature", ""), raw_body):
        return 400

    alert = json.loads(raw_body)

    # The dashboard's test button sends eventId 0 and test: true, so keying
    # off the id ignores tests without a special case.
    if alert["eventId"] in seen:
        return 204
    seen.add(alert["eventId"])

    if alert["kind"] == "closed":
        resume(alert["venue"], alert["component"])
    else:
        pause(alert["venue"], alert["component"], alert["severity"])

    return 204

Three formats, chosen per channel

The body above is one of three. A webhook channel picks its format when it is created and the choice is a property of the channel, not of the account — a desk can page on PagerDuty, post to a Slack channel and drive its own automation off the signed payload at the same time, from one watchlist. Each body below is rendered by the function the dispatcher builds it with, applied to the example payload above, so what is printed here is what arrives.

up24

The signed payload, and the only format that carries x-up24-signature. HTTPS only, and a redirect is never followed — answering 301 to somewhere else would send the body, and the signature, to a host the account never authorised.

slack

An incoming-webhook body. Unsigned by design: Slack reads no signature and offers nothing to sign with, so the URL is the credential. text travels beside blocks because a blocks-only body arrives as a blank push notification on a phone, which is the device an out-of-hours alert is read on.

pagerduty

Events v2. The routing key is a service's integration key, entered on the dashboard and stored the way the signing secret is. All three events of one incident share a dedup_key, so the escalation updates the page it opened and the recovery resolves it — outage maps to critical and degraded to warning, and a recovery for an opening that was never delivered is skipped with a reason rather than sent as a resolve for an incident that does not exist.

{
  "text": ":red_circle: Binance · ws:book · outage — Order book stream silent past its stall threshold in 2 of 3 regions",
  "blocks": [
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "*:red_circle: Binance · ws:book · outage*\nOrder book stream silent past its stall threshold in 2 of 3 regions"
      }
    },
    {
      "type": "context",
      "elements": [
        {
          "type": "mrkdwn",
          "text": "<https://up24.app/exchange/binance|What up24 measured> · detected 2026-08-30T14:12:15.000Z"
        }
      ]
    }
  ]
}

{
  "routing_key": "R0EXAMPLEROUTINGKEY0000000000000",
  "event_action": "trigger",
  "dedup_key": "up24-1907",
  "payload": {
    "summary": "Binance · ws:book · outage — Order book stream silent past its stall threshold in 2 of 3 regions",
    "source": "binance",
    "component": "ws:book",
    "severity": "critical",
    "timestamp": "2026-08-30T14:12:15.000Z",
    "custom_details": {
      "event": "incident.opened",
      "kind": "opened",
      "eventId": 48213,
      "incidentId": 1907,
      "venue": "binance",
      "venueName": "Binance",
      "component": "ws:book",
      "severity": "outage",
      "state": "outage",
      "summary": "Order book stream silent past its stall threshold in 2 of 3 regions",
      "errorClass": "stalled",
      "startedAt": "2026-08-30T14:12:05.000Z",
      "endedAt": null,
      "occurredAt": "2026-08-30T14:12:15.000Z",
      "url": "https://up24.app/exchange/binance"
    }
  },
  "links": [
    {
      "href": "https://up24.app/exchange/binance",
      "text": "Binance on up24"
    }
  ]
}

A webhook channel is created on the dashboard, which shows the signing secret once and then never again, and logs every attempt including the ones deliberately not made.

Or your own shape

up24 shipping the two above does not help a program that wants a third. Both transforms below are the ones up24 runs, written out as the pure functions they are; the test suite asserts that each produces exactly the body printed above, so a paste from here behaves the way the native format does. Start from up24, verify the signature first, and transform after.

// Events v2. The routing key is the integration key of a PagerDuty service,
// and it is the only credential involved — PagerDuty reads no signature, which
// is why up24 signs its own format and not this one.
const ENDPOINT = 'https://events.pagerduty.com/v2/enqueue';
const ACTION = { opened: 'trigger', escalated: 'trigger', closed: 'resolve' };
const SEVERITY = { outage: 'critical', degraded: 'warning' };

export function toPagerDuty(alert, routingKey) {
  // One PagerDuty incident per up24 incident. All three events carry the same
  // incidentId, so the escalation updates the trigger in place and the close
  // resolves it rather than opening a third page at three in the morning.
  const base = {
    routing_key: routingKey,
    event_action: ACTION[alert.kind],
    dedup_key: `up24-${alert.incidentId}`,
  };

  // A resolve carries the key and nothing else: there is no payload to grade
  // once the thing is over.
  if (base.event_action === 'resolve') return base;

  return {
    ...base,
    payload: {
      summary: `${alert.venueName} · ${alert.component} · ${alert.severity} — ${alert.summary}`,
      source: alert.venue,
      component: alert.component,
      severity: SEVERITY[alert.severity],
      timestamp: alert.occurredAt,
      // The whole up24 payload, so the responder has the denominator and the
      // vantage points without leaving PagerDuty.
      custom_details: alert,
    },
    links: [{ href: alert.url, text: `${alert.venueName} on up24` }],
  };
}

// An incoming webhook URL is itself the secret, and Slack reads no
// signature — so this body is unsigned by design. Post it as-is.
const DOT = { outage: ':red_circle:', degraded: ':large_orange_circle:' };

export function toSlack(alert) {
  const head =
    alert.kind === 'closed'
      ? `:large_green_circle: ${alert.venueName} · ${alert.component} recovered`
      : `${DOT[alert.severity]} ${alert.venueName} · ${alert.component} · ${alert.severity}`;

  // text is what a notification and a screen reader get; blocks are what the
  // channel shows. Sending only blocks is how a Slack alert arrives blank on a
  // phone.
  return {
    text: `${head} — ${alert.summary}`,
    blocks: [
      { type: 'section', text: { type: 'mrkdwn', text: `*${head}*\n${alert.summary}` } },
      {
        type: 'context',
        elements: [
          {
            type: 'mrkdwn',
            text: `<${alert.url}|What up24 measured> · detected ${alert.occurredAt}`,
          },
        ],
      },
    ],
  };
}