DEVBOOK

Webhooks

8 min read

Edit this pageReport / suggest

A webhook is an HTTP callback: when an event occurs in a producer system, it sends an HTTP POST with event data to a URL the consumer registered in advance. This inverts the communication direction compared to polling — instead of the consumer repeatedly asking “anything new?”, the producer pushes notifications in near real-time. You reach for webhooks when you need low-latency, push-based integration between systems that communicate over HTTP, especially across organizational boundaries where shared message brokers are impractical (payment providers, source control platforms, SaaS integrations).

A common webhook contract works like this: the consumer registers a callback URL and the producer sends an HTTP request when an event occurs. Authentication, signature algorithm, accepted acknowledgement codes, timeout, and retry schedule are provider-defined. The concrete receiver below models a provider that signs the raw JSON body with HMAC-SHA256, accepts a successful 2xx response, and retries transient delivery failures with exponential backoff.

sequenceDiagram
    participant Consumer as Consumer Service
    participant Producer as Producer Service

    Consumer->>Producer: Register callback URL + provider credentials
    Note over Producer: Event occurs internally
    Producer->>Consumer: POST /webhook with JSON payload + provider signature
    Consumer->>Consumer: Verify provider-defined signature
    Consumer->>Consumer: Check idempotency key and process event
    Consumer-->>Producer: Accepted 2xx response
    Note over Producer: Apply the documented timeout and retry policy

Webhooks complement Event-Driven Architecture — they are the HTTP-native way to deliver events between systems that do not share a message broker. For internal service-to-service communication within the same platform, Message Queues are usually a better fit because they provide built-in durability, fan-out, and back-pressure.

ASP.NET Core receiver

Authenticate the provider’s exact raw request bytes before parsing JSON. Reserializing a payload can change whitespace or property ordering and invalidate the signature.

using System.Security.Cryptography;
using System.Text;
 
app.MapPost("/webhooks/provider", async (
    HttpRequest request,
    IWebhookInbox inbox,
    IConfiguration configuration,
    CancellationToken ct) =>
{
    await using var buffer = new MemoryStream();
    await request.Body.CopyToAsync(buffer, ct);
    var body = buffer.ToArray();
 
    if (!request.Headers.TryGetValue("X-Signature", out var supplied) ||
        !Convert.TryFromHexString(
            supplied.ToString(),
            new byte[32],
            out var written) ||
        written != 32)
    {
        return Results.Unauthorized();
    }
 
    var secret = Encoding.UTF8.GetBytes(
        configuration["Webhooks:Secret"]!);
    var expected = HMACSHA256.HashData(secret, body);
    var actual = Convert.FromHexString(supplied.ToString());
 
    if (!CryptographicOperations.FixedTimeEquals(expected, actual))
    {
        return Results.Unauthorized();
    }
 
    var accepted = await inbox.StoreIfNewAsync(
        body,
        request.Headers,
        ct);
 
    return accepted ? Results.Accepted() : Results.Ok();
});

Adapt the signed payload to the provider contract: many providers sign timestamp + "." + rawBody, not the body alone. Validate timestamp skew, store the provider event ID under a unique constraint, and commit the inbox record or durable queue message before returning success. Background processing can happen afterward.

Polling interval versus durable webhook delivery contract

The visual shows the direction difference, not a reliability guarantee. Polling can be cheap when the interval is long or conditional requests return no body; webhooks can be expensive when retries storm or endpoints are slow.

QuestionPollingWebhook
FreshnessBounded by the interval plus API latencyUsually near real-time, bounded by delivery and retry delay
Missed changesFetch from a durable source cursor or updated_since windowProducer must retain deliveries or expose replay/reconciliation
Duplicate workOverlapping windows repeat recordsAt-least-once retries repeat events
Load ownerConsumer chooses interval and batch sizeProducer controls fan-out and retry schedule
Failure recoveryResume from cursor after outageReplay by event ID, dead-letter inspection, or reconciliation poll

A durable webhook contract needs more than POST:

  1. Sign the exact body plus a timestamp and identify the signing key version.
  2. Assign a stable event or delivery ID and document retry and ordering scope.
  3. Retry transient failures with bounded exponential backoff and jitter; do not retry permanent 4xx failures blindly.
  4. Require the receiver to persist the event before returning success and process it idempotently.
  5. Expose delivery logs, manual replay, retention, timeout, and terminal failure behavior.
  6. Provide a cursor-based list API so consumers can reconcile gaps after outages.

For an hourly billing export, polling may be simpler and more controllable. For PaymentCaptured, use a signed webhook for low latency, then poll GET /events?after=<cursor> periodically as a correctness safety net. Push handles the common path; pull repairs the gaps.

Tradeoffs: Webhooks vs Polling vs SSE vs WebSockets

ApproachDirectionLatencyComplexityConnectionBest fit
WebhooksPush (server to server)Near real-timeModerate (retry, signatures, idempotency)Per-event HTTP requestCross-org integrations, SaaS event delivery
PollingPull (consumer to producer)Interval-bound delayLowStateless per-requestSimple integrations, systems without webhook support
SSEPush (server to browser)Real-timeLowLong-lived HTTP streamOne-directional browser notifications, dashboards
WebSocketsBidirectionalReal-timeHigh (connection management, scaling)Persistent TCPChat, collaborative editing, real-time bidirectional flows

Decision heuristic:

  • Pick webhooks for server-to-server push across trust boundaries (Stripe payment events, GitHub repository events, CRM notifications).
  • Pick polling when the producer has no webhook support, or when near real-time latency is not required and simplicity matters most.
  • Pick SSE when you need server-to-browser push without the complexity of WebSockets.
  • Pick WebSockets when you need bidirectional real-time communication (client sends and receives).

Pitfalls

1) At-Least-Once Delivery Without Idempotency

  • What goes wrong: the producer retries after a timeout and the consumer processes the same event twice — double-charging a payment, sending duplicate notifications, or creating duplicate records.
  • Why it happens: network timeouts, producer retries, and load balancer replays mean the same webhook may arrive more than once. Exactly-once delivery over HTTP is not achievable.
  • Mitigation: use the delivery ID (e.g., X-GitHub-Delivery, Stripe event.id) as an idempotency key. Store processed IDs in durable storage and check before processing. Make state transitions conditional (UPDATE ... WHERE status = 'pending').

2) Slow Processing Causes Timeout and Retry Storm

  • What goes wrong: the consumer does heavy work synchronously in the webhook handler, exceeds the producer’s timeout (typically 5-30 seconds), and triggers retries that compound the load.
  • Why it happens: inline database writes, external API calls, or computation in the request path.
  • Mitigation: authenticate and durably store the inbox/queue record, then return 2xx without running downstream business work inline. A background worker performs the business logic.

3) Missing or Weak Signature Verification

  • What goes wrong: an attacker sends forged webhook payloads to the endpoint, triggering unauthorized actions (fraudulent refunds, fake deployment triggers, data manipulation).
  • Why it happens: the endpoint accepts any POST without verifying the provider’s signature, or an HMAC-based integration uses a non-constant-time comparison vulnerable to timing attacks.
  • Mitigation: verify the provider-defined signature over the exact documented input. For HMAC, use a constant-time comparison. Reject missing or invalid signatures and enforce the provider’s signed-timestamp tolerance to limit replay.

4) Endpoint Availability and Missed Events

  • What goes wrong: if the consumer is down during delivery and the producer exhausts its retry budget, events are permanently lost.
  • Why it happens: webhook producers typically retry for a limited window (hours to days) and then give up.
  • Mitigation: implement a reconciliation mechanism — periodically poll the producer’s event API to detect gaps. Monitor webhook delivery metrics. Some producers offer event replay or dead-letter inspection (use them).

Questions

References

Graph View

Backlinks