Fieldnotes / engineering

Idempotency

Explanation

One intended operation

A caller asks for one release ticket. The intended effect is one saved ticket, not merely a successful network response.

Separate the business effect from the response that reports it.

  1. 0s The caller wants one release ticket. The service must save that effect.
  2. 7s The create request reaches the service. The service begins the operation.
  3. 14s The service saves ticket 47. The database records that effect.
  4. 21s A response reports ticket 47. This normal attempt succeeds.

A missing reply

The service saves a ticket but the reply is lost. The caller times out without knowing whether the operation happened.

A timeout does not prove that nothing was created. Retrying an unprotected create can duplicate the effect.

  1. 0s The caller sends a create request. The service receives it.
  2. 7s The service saves ticket 47. Its response will not reach the caller.
  3. 14s The caller stops waiting. Its outcome is unknown.
  4. 21s Another create could add a duplicate. The caller needs a stable operation ID.

Name the operation

The caller saves key k7 with its release request before sending. The service commits the ticket and key record in one database transaction.

The key identifies an intended operation. Atomic storage and uniqueness are needed to prevent concurrent duplicate effects in this design.

  1. 0s New attempt: save key k7 before sending. This time, protect the operation upfront.
  2. 7s Send the key with the request. The service checks its stored records.
  3. 14s The first attempt creates ticket 47. Store the ticket and key atomically.
  4. 21s Suppose this protected reply is lost too. Key k7 still identifies the saved ticket.

Retry the same key

After losing the response, the caller resends k7 with the same release payload. The service finds the previous result.

The retry reuses the recorded effect instead of creating a second ticket. This depends on the service contract and key retention.

  1. 0s The caller still needs its result. It reuses the saved key k7.
  2. 7s The service looks up k7. Its record points to ticket 47.
  3. 14s The existing result is returned. No second ticket is created.
  4. 21s The caller receives ticket 47. The intended effect remains one ticket.

Changed intent, new key

The caller now sends a different release payload using the old key. This service compares the stored payload before deciding whether replay is valid.

A key is not a general-purpose deduplication label. Reusing it for changed intent must follow the service contract; here the request is rejected.

  1. 0s Key k7 belongs to release v1. The new payload asks for release v2.
  2. 7s The service compares the payloads. The same key now has different intent.
  3. 14s The service rejects the mismatch. It does not overwrite the old operation.
  4. 21s A new operation needs a new key. A retry keeps its original identity.
Read the guide

Helpful first: Tool design, Timeouts and retries.

The idea

Idempotency means repeated identical requests have the same intended effect as one request. It does not promise identical responses, eventual success, or that all incidental effects, such as logging, happen once. HTTP defines PUT and DELETE as idempotent; a POST tool needs an explicit service contract before automatic retries are safe. See RFC 9110, section 9.2.2.

Where it helps

An agent runner can lose a tool response after the service commits. Treating that timeout as proof of failure can create another ticket on retry. Delegation makes this recovery decision part of your software: establish what the tool guarantees before enabling unattended writes.

Teach this before agents gain permission to create shared resources, because uncertain outcomes require a deliberate retry policy. Even teams consuming hosted tools need to ask for that contract and route unresolved outcomes to a person.

Imagine an in-house release-ticket tool backed by one PostgreSQL database. A runner saves operation key k7 with release v1 before calling it. The service creates ticket #47 and records the key together. Its reply disappears. Repeating k7 with v1 retrieves #47; reusing k7 with v2 is rejected. This is a proposed contract, not a claim about a commercial issue tracker.

What you will be able to check

  • Distinguish a retry from a newly intended action and preserve its operation key.
  • Write a tool contract covering changed parameters, concurrent attempts, and expiry.
  • Fault-test a committed write with a dropped response and inspect the resulting resource count.

What changes in the team

Today
An engineer checks the ticket list after a failed submission and decides whether to submit again.
With an agent
A durable runner retains the operation key and arguments across retries and restarts, then follows the tool’s documented recovery policy.
Still your responsibility
Engineers define intent, approve write permissions, and investigate outcomes the service cannot reconcile.

Work through an example

  1. Identify one intended release ticket. Generate its key once in the runner and persist it with the arguments. A later, genuinely separate ticket needs a new key even when its contents match; equal text alone does not establish identity.
  2. For this local design, require non-null caller, operation, and key fields under a composite, non-deferrable unique constraint. Within one transaction, claim that identity, insert the ticket, and save its identifier and parameters. PostgreSQL documents uniqueness and all-or-nothing transaction visibility; the application still needs a duplicate-handling branch.
  3. PostgreSQL’s unique-index check waits for an uncommitted competing insertion. If the competing transaction commits, roll back the losing transaction and read the saved record in a fresh one. Return its ticket identifier for matching arguments; reject a mismatch. Bound the wait and report uncertainty instead of issuing an uncoordinated write.
  4. Drop the first reply after commit and restart the runner. Its retry should take the lookup path. This single-database design does not make an external ticket API transactional with your local records: use that provider’s documented deduplication or reconciliation mechanism and test its failure window.

The engineering decision

Specify caller and operation scope, parameter comparison, retention, and concurrent-request behavior. Stripe illustrates why contracts matter: its API v1 can replay saved failures and permits pruning keys after 24 hours; v2 has different retry semantics. Never infer either policy from the word idempotent. A key is not authorization; keep access checks in the tool.

  • Generating a new key for every attempt defeats duplicate detection.
  • Checking for a key and then writing without database-enforced coordination lets concurrent callers both proceed.
  • Pruning records before delayed retries arrive can turn an old intent into a new action.
  • Putting “do not duplicate” in a prompt supplies neither durable coordination nor enforced permissions.

Try it with your team

Lose the reply, keep one ticket

  1. Build a local test endpoint with ticket and operation tables; document the duplicate-response contract.
  2. Inject a disconnect after commit, then retry with the same key and payload.
  3. Submit two matching calls concurrently, then retry that key with changed release data.
  4. Inspect stored rows and results; document how a retry after retention expiry is handled.

You have an answer when…

  • Lost replies and concurrent matching calls leave exactly one ticket and converge on its identifier.
  • Changed parameters produce a mismatch without altering the original ticket.
  • The team can identify the recovery path for expired keys and external side effects.

Read further

At the service boundary, can your team demonstrate one ticket after a lost reply and two concurrent attempts, then explain what happens after the key expires?

Original video comparisons