A client that sends a request and receives no response cannot know whether the request was processed. The connection may have failed before the request arrived, or after the API had processed it and before the response was delivered. If the request creates a payment or an order, retrying it may create a second one, and not retrying it may lose the first.
Payment providers such as Stripe, Adyen and PayPal resolve this with an idempotency key. The client sends a value of its choosing, such as a UUID, in a header with the request, and sends the same value again with every retry. The API guarantees that the operation runs once for that value. The IETF is standardising this as the Idempotency-Key header.
The Idempotency middleware provides that guarantee for Kipchak APIs. It is part of Kipchak Enterprise, and was listed on the public roadmap.
How it works
When a request arrives with an Idempotency-Key header, the middleware takes the key atomically in Valkey or Memcached. The route runs, and its response is stored against the key. Every later request with the same key receives that stored response, with its status, headers and body, and the header Idempotent-Replayed: true. The route does not run again.
The middleware also refuses requests that would make the guarantee meaningless:
| Situation | Response |
|---|---|
| The same key with a different request | 422, because the key has already been used for something else |
| The same key while the first request is still running | 409, with Retry-After: 1 |
| No key, where keys are required | 400 |
Two requests are the same when their method, path, query parameters, content type and body are the same.
A response is stored whatever its status. A declined payment or a validation error is the answer to that request, and a retry receives the same answer. Server errors and exceptions are the exception: they free the key, so that a retry runs the request again, because the operation may not have completed.
Concurrent retries
The difficult case is a retry that arrives while the first request is still running, on a different worker or a different server. Taking the key is a single atomic operation in the store, so only one of the requests can take it, and the other receives a 409.
With Valkey, a response is stored, or a key released, by a script that first checks that the request still holds the key. If a request runs for longer than the configured lock period and a retry takes the key over, the first request cannot then overwrite the retry’s result. It logs a warning instead.
In worker-mode testing on the Enterprise runtime image, 40 simultaneous copies of a request with the same key ran the route once. In a second test, 1,200 requests with retries spread across 300 keys ran the route exactly 300 times.
Keys belong to a consumer
Keys are scoped to the consumer who sent them, as defined by Kipchak Identity, so one client’s key can never match another client’s request. The middleware is added before the authentication middlewares, which means it runs after them: unauthenticated requests never take a key. Keys are stored as a hash of the consumer and the key, so they do not appear in the store in readable form.
When the store is unavailable
If Valkey or Memcached cannot be reached, the middleware cannot guarantee that a request runs once. By default it therefore refuses protected requests with 503 rather than run them without protection, and a client that retries later receives the correct result. An API for which availability matters more can be configured to run requests unprotected while the store is unavailable.
In testing, a Valkey outage produced 503 responses, and the API recovered when Valkey returned with no request run twice.
Installation
The Idempotency middleware is part of Kipchak Enterprise and is installed from the Enterprise Composer repository, with the driver for the store:
composer require kipchak/middleware-idempotency
composer require kipchak/driver-valkey # or kipchak/driver-memcached
It can protect every POST and PATCH request, or be attached to individual routes such as payment creation. The configuration reference is in the Idempotency middleware documentation.