Troubleshoot a dead webhook delivery

Correct the receiver; version 1.0.0 has no delivery replay UI, so use a controlled integration recovery plan. The guide then takes you through the correct route, the checks to complete before making changes, the workflow in order, and the evidence to review afterwards.

What you’ll accomplish

Correct the receiver; version 1.0.0 has no delivery replay UI, so use a controlled integration recovery plan. This section documents endpoint safety checks, event selection, HMAC verification, delivery retries and dead-letter behaviour.

Follow these steps

1. Go to Enterprise POS → Settings → Signed webhooks and open the workflow that matches “Troubleshoot a dead webhook delivery”. Confirm the page or record context before making a change.
2. Create a public HTTPS endpoint that can receive POST JSON.
3. Create a webhook with minimum event selection.
4. Copy the full secret once.
5. Check timestamp and signature before parsing/processing the event.
6. Return a successful 2xx response promptly.
7. Check downstream logs; the module retries transient failures through cron.
8. Disable the webhook when it should stop receiving events.

Fields and options to review

• Guide focus: Correct the receiver; version 1.0.0 has no delivery replay UI, so use a controlled integration recovery plan.
• Webhook name and HTTPS endpoint
• Selected event names
• One-time secret encrypted at rest
• Event ID, timestamp, signature and JSON payload
• Delivery status, attempt, next-attempt time, response code and excerpt

Rules the system enforces

• Endpoints must use HTTPS and must not resolve to localhost, .local, private, reserved or non-public addresses.
• All DNS results are validated and the first public IP is pinned for delivery.
• Redirects are disabled.
• Connect timeout is 5 seconds; total timeout is 10 seconds.
• Signature format is sha256=HMAC_SHA256(timestamp.payload, secret).
• Maximum attempts are 8; delay is 2^attempt × 30 seconds capped at 6 hours.
• After final failure the delivery becomes dead.

How to confirm it worked

• The specific outcome described by this guide is visible and remains tenant-scoped.
• A successful endpoint receives signed tenant event data.
• Transient failures are retried without duplicate queue creation for the same webhook/event.
• Disabled webhooks stop receiving new queued events.

Safety checks and troubleshooting

• Verify signatures using the raw request body.
• Reject stale/replayed timestamps under the receiver’s policy.
• Make receiver processing idempotent by event ID.
• Do not log secrets or full sensitive payloads.

Continue with related guidance