What is a webhook?

A webhook is a mechanism that allows one system to notify another system when an event occurs by sending an HTTP request to a specified URL. Instead of repeatedly asking "is it done yet?", you give the other system a URL and say "call me when it is done." In AI applications, webhooks are commonly used to receive notifications when asynchronous operations like batch jobs, model training runs, or agent tasks complete.

The webhook pattern is straightforward:

[1. Registration]: You provide a URL (your webhook endpoint) to the service you want to receive notifications from. This is typically done through an API call, a dashboard configuration, or as part of a request.

[2. Event occurs]: Something happens on the service side, such as a batch processing job completing, an async generation finishing, or a status change.

[3. Notification]: The service sends an HTTP POST request to your registered URL with a payload containing details about the event.

[4. Processing]: Your server receives the request, processes the payload, and returns an HTTP response (usually 200 OK) to acknowledge receipt.

This is the opposite of the typical client-server pattern. Instead of your code calling the API, the API calls your code. This inversion is why webhooks are sometimes called [reverse APIs] or [HTTP callbacks].

There are two fundamental approaches to learning about the status of asynchronous operations:

[Polling] means your code repeatedly sends requests to check the status of an operation. You might call a status endpoint every 10 seconds to see if a batch job has finished. This is simple to implement but wasteful: most of those requests return "still processing" and accomplish nothing. Frequent polling wastes API calls and bandwidth. Infrequent polling means you learn about completion with a delay.

[Webhooks] (push) mean the service notifies you proactively when something changes. You find out about events immediately, without wasting requests checking. The downside is that you need to operate a publicly accessible server that can receive incoming HTTP requests.

For most production applications, webhooks are preferred because they are more efficient, provide faster notification, and reduce unnecessary API traffic. Polling is simpler to implement for scripts or development environments where running a web server is inconvenient.

Webhooks appear throughout AI systems wherever asynchronous processing is involved:

[Batch job completion]: When you submit a batch of thousands of requests to an AI provider, the processing may take hours. A webhook notifies your system the moment results are ready, so you can immediately begin processing them.

[Agent status updates]: AI agents that perform multi-step tasks, such as researching a topic, writing a report, or executing a workflow, may take minutes or longer to complete. Webhooks can notify you of intermediate progress and final completion.

[Model training and fine-tuning]: Fine-tuning a model can take hours. A webhook tells your system when training is complete so you can automatically deploy the new model or run evaluation.

[Async content generation]: Some AI services process image generation, video generation, or audio generation asynchronously. Webhooks notify you when the generated content is ready for download.

[Moderation and review]: Content submitted for AI moderation or human-in-the-loop review can trigger a webhook when the review is complete, allowing your application to proceed with the approved or flagged content.

[Third-party integrations]: Services like Zapier, Make, and other automation platforms use webhooks extensively to connect AI services to other business tools.

To receive webhooks, you need an HTTP endpoint that can accept POST requests. Here is what a basic implementation involves:

[Create an endpoint]: Set up a route in your web application that accepts POST requests at a specific path like /webhooks/batch-complete.

[Parse the payload]: Read and parse the JSON body of the incoming request. The payload typically includes an event type, a resource identifier, relevant data, and a timestamp.

[Respond quickly]: Return a 200 OK response as fast as possible, ideally within a few seconds. Most webhook senders have timeout limits, and if your endpoint takes too long to respond, they may consider the delivery failed and retry.

[Process asynchronously]: If handling the webhook requires significant work (like downloading batch results and processing them), acknowledge the webhook immediately and queue the processing work for a background job.

Webhook endpoints are public URLs that accept incoming requests, which makes security critical:

[Signature verification]: Reputable services sign their webhook payloads using a shared secret. They include a signature in a request header (often X-Signature or similar), computed as an HMAC of the payload using your webhook secret. Your endpoint should recompute the signature and verify it matches before trusting the payload. This ensures the request genuinely came from the expected service and was not tampered with.

[HTTPS only]: Always use HTTPS for webhook endpoints. This encrypts the payload in transit and prevents eavesdropping or tampering.

[IP allowlisting]: Some providers publish the IP ranges their webhook deliveries originate from. Adding these to an allowlist provides an additional layer of verification.

[Payload validation]: Validate the structure and content of incoming payloads. Do not assume the payload is well-formed or trustworthy just because the signature is valid. Validate field types, required fields, and value ranges.

Webhooks involve network communication between two systems, which means things can go wrong. Reliable webhook implementations account for failures:

[Retries]: Most webhook senders retry failed deliveries. If your endpoint returns a 5xx error or times out, the sender will typically retry several times with increasing delays (exponential backoff). Your system should be designed to handle receiving the same webhook multiple times.

[Idempotency]: Because retries can deliver the same event more than once, your webhook handler should be [idempotent]: processing the same event twice should have the same effect as processing it once. Use the event's unique identifier to detect duplicates and skip reprocessing.

[Ordering]: Webhooks may arrive out of order, especially during retries. Do not assume that events arrive in the same order they occurred. Use timestamps or sequence numbers to handle ordering if it matters for your logic.

[Dead letter queues]: For critical webhooks, log failed processing attempts and maintain a queue of events that could not be processed. This allows you to investigate failures and replay events after fixing issues.

[Health monitoring]: Monitor your webhook endpoint's availability, response time, and error rate. If your endpoint goes down, you will miss webhook deliveries until it recovers, and retries may eventually exhaust.

Several patterns recur when using webhooks with AI systems:

[Status pipeline]: Register a webhook when starting an async job. When the webhook fires, check the final status, download results if successful, handle errors if not, and trigger the next step in your pipeline.

[Event-driven architecture]: Use webhooks as the glue between AI services and your application logic. A batch completion webhook might trigger result processing, which triggers a notification to users, which triggers analytics logging. Each step reacts to events rather than polling for changes.

[Fan-out]: A single webhook event might need to trigger multiple downstream actions. Use a message queue or event bus internally to distribute the event to multiple handlers.

[Webhook-to-queue bridge]: In high-volume systems, webhook receivers write events to a message queue (like Redis, RabbitMQ, or SQS) immediately and return 200. Background workers consume from the queue and handle the actual processing. This decouples receiving from processing and provides buffering.

Webhooks are a foundational building block for any AI application that involves asynchronous work. Understanding how to implement them securely and reliably will serve you well across many different projects and integrations.