Idempotency and Retries for LLM Workflows

Read-only vs mutating steps

Idempotency key pattern

def charge_refund(order_id: str, idempotency_key: str, amount: cents):
    existing = db.get_idempotent_result(idempotency_key)
    if existing:
        return existing
    result = payment_api.refund(order_id, amount, idempotency_key)
    db.store_idempotent_result(idempotency_key, result)
    return result

Pass the same key from the agent tool call. Store in your DB, not just the payment provider.

LLM timeout ambiguity

When an HTTP call times out, the model may still have completed. Mitigations:

1. Request ID — pass provider request ID; query status if supported
2. Dedupe table — hash (user_id, task_id, step) to stored output
3. Checkpoint before expensive calls — LangGraph or custom state

Retry backoff

attempt 1: immediate
attempt 2: 2s + jitter
attempt 3: 4s + jitter
attempt 4: 8s + jitter
cap at 60s, max 5 attempts on 429

Route to fallback model after 2 failures on primary.

Agent checkpointing

LangGraph persists state at node boundaries. On failure, resume from last checkpoint instead of re-running retrieval and prior tool calls.

For n8n/Zapier: use execution IDs and step-level error workflows; avoid blind full-rerun on partial success.

Observability

Log in Langfuse:

• retry_count
• failure_reason (timeout, 429, validation, tool error)
• idempotency_key on mutating tools
• checkpoint_id on agent resume

Retry storms often mean rate limits too tight or prompts too large — fix root cause, not just retry count.