SCUA

How-to

Write handlers that survive snapshots

One of SCUA's promises is that a partition can be snapshotted — saved, moved to another machine, resumed later — without any custom serialization. That promise has a sharp edge the moment a partition does I/O: what happens if it's snapshotted while a request is in flight? A handler is halfway through an http.get, or a fs.read, and right then the partition is saved and later resumed somewhere else. The outbound request is gone with the old machine. What does the paused call return?

This page is about that one case, and the small habit that makes your handlers safe against it.

#The reload rule: Error("interrupted")

When a partition is loaded from a snapshot while a call was parked mid-I/O, that call resumes as Error("interrupted"). The request that was in flight does not come back — and, crucially, the runtime never re-issues it.

That "never re-issues" is the whole safety property. A blind retry of an in-flight request would mean a POST could fire twice — an order placed twice, a payment charged twice — because from the outside you can't tell whether the original request reached the server before the snapshot. So SCUA doesn't guess. It hands the interruption back to your code as an ordinary Error, and lets you decide what actually happened.

Every parking call — http.get, http.post, fs.read, and the rest — can therefore return Error("interrupted") after a reload. A robust handler accounts for it.

There's one sibling of the same rule. If the parked call had a deadline that already passed by the time the snapshot resumes, it comes back as Error("timeout") rather than Error("interrupted") — the wait outlived its budget instead of being cut short mid-flight. Both are ordinary Error values, and both mean the same thing for safety: the request was not re-issued, so before you assume anything about a side effect, reconcile against the durable side (below). A catch-all Error arm handles both; match "interrupted" and "timeout" separately only when you want to react differently.

#Don't retry blindly — reconcile

The wrong reflex is "an error, so try again". After an interruption you don't know whether the side effect already happened. The right move is to ask the durable side what's true, and act on the answer. This is reconcile on resume.

Say a handler places an order:

fn place_order(order)
  match http.post("https://api.shop/orders", encode(order))
    Ok(resp) -> record_placed(order.id, resp)
    Error(e) when e == "interrupted" -> reconcile(order)   -- snapshot mid-flight: find out what really happened
    Error(e) -> retry_later(order, e)                      -- an ordinary failure is safe to retry
  end
end

Notice the two Error arms do different things. An ordinary failure (connection refused, a timeout) never sent a completed request, so retrying is fine. An interruption is the one case where you must not assume — so it goes to reconcile, which asks the durable side:

fn reconcile(order)
  -- Ask whether the order already landed before the snapshot, instead of re-sending it.
  match http.get(`https://api.shop/orders?client_id={order.id}`)
    Ok(resp) when order_exists(resp) -> record_placed(order.id, resp)  -- it did land: adopt it
    Ok(_)                            -> place_order(order)             -- it didn't: now it's safe to send
    Error(_)                         -> retry_later(order, "reconcile failed")
  end
end

The pattern generalizes: after Error("interrupted"), query the durable system of record ("is order 42 already placed?", "does file out.tmp exist?") and reconcile against the answer, rather than re-firing the operation. A read like the GET above is safe to repeat; the non-idempotent POST is not, which is exactly why you check first.

#A catch-all is fine too

If a handler only does idempotent work — reads, gets, anything safe to repeat — you don't need a special interruption arm. Since Error("interrupted") is just an Error value, a catch-all Error arm handles it like any other failure:

match http.get(url)
  Ok(resp) -> use(resp)
  Error(e) -> print(`could not fetch (maybe interrupted): {e}`)   -- fine when a retry is harmless
end

The distinction only matters when a repeat would be unsafe. Reach for the explicit Error("interrupted") arm precisely where double-sending would hurt — the writes, the posts, the charges — and reconcile there.

#Why this is worth the small effort

The payoff is that a SCUA partition can be hibernated, migrated, or restored at any moment — including mid-request — and your handler stays correct without you tracking in-flight state by hand. The runtime guarantees at-most-once: it will never silently replay a request across a snapshot. Your one job is to handle the Error("interrupted") you get back, and for anything with a side effect, ask the durable side what happened rather than guessing.