Evolve your saved data safely

Game state outlives the code that wrote it. A player saves today, you ship a new build next week with a renamed field, and that old save still has to load. This page is about the part of that you control as a script author: designing your records and writing migrate hooks so old data turns into current data without breaking.

First, the boundary. SCUA organizes state into partitions, and the host engine that embeds SCUA is what actually writes a whole partition to disk and reads it back — relocating live state with no work from you. For that path your script doesn't call "save" or "load" itself (for writing your own save files from script, see the durable module below). What your script decides is the shape of the data and the rule for upgrading older shapes. When the host loads an old partition, that data flows into your current records, and the upgrade you describe here is what runs. The split is clean. The host moves bytes; your records and hooks decide what those bytes are allowed to become.

We'll simulate a load with an any-typed value standing in for "data the host just read," turned into a record with parse(value, Record). That conversion is where a load fills defaults, runs a migrate hook, and checks the result. A typed let on its own only checks a value, one level deep; it doesn't convert it, so old data needs parse.

#Additive changes are automatic

If the only thing you did between versions is add a field, and that field has a default, you don't have to do anything. Old data that lacks the field gets the default when parse converts it.

record Settings {
  volume: int,
  muted:  bool = false,   -- added in a later version
}

let old_save: any = { volume = 7 }    -- written before `muted` existed
let s = parse(old_save, Settings)
print(`volume {s.volume}, muted {s.muted}`)

Run it:

$ scua settings.scua
volume 7, muted false

The old save has no muted. It loaded anyway, with muted filled in from the default. That's the whole pattern for additive evolution: new field, give it a default, done. (See Records and gradual types for how defaults work.)

#Non-additive changes use a migrate hook

Renames, splits, and merges can't be inferred from a default, because the data is in a different shape, not just missing a piece. For those you write a migrate hook. It takes the old-shaped data and returns the current record.

record Hero {
  handle: string,
  coins:  int,
  lives:  int,
}
migrate Hero(old)
  return {
    handle = old.name  ?? old.handle,                -- renamed: name -> handle
    coins  = old.coins ?? (old.gold + old.silver),   -- merged: gold + silver -> coins
    lives  = old.lives ?? 3,                          -- new field: 3 for old heroes
  }
end

let v1: any = { name = "ed", gold = 500, silver = 20 }
let hero = parse(v1, Hero)
print(`{hero.handle}: {hero.coins} coins, {hero.lives} lives`)

Run it:

$ scua hero.scua
ed: 520 coins, 3 lives

Each field of the return handles one kind of change. The ?? operator is doing the work: it takes the left side unless it's missing, then falls back to the right. So old.name ?? old.handle reads name from a v1 save or handle from one that's already current. old.gold + old.silver collapses two old fields into one. old.lives ?? 3 gives heroes saved before lives existed a starting value.

There's a sharp edge here worth stating plainly: a migrate hook must return the complete current shape. Defaults are not applied to a migrate's return. That's why lives is set explicitly even though you might expect a default to cover it. The hook's job is to produce a finished record, so name every field.

parse only runs the hook when the data isn't already in the current shape. Data that's already complete is copied into the record as it is:

record Hero {
  handle: string,
  coins:  int,
  lives:  int,
}
migrate Hero(old)
  return {
    handle = old.name  ?? old.handle,
    coins  = old.coins ?? (old.gold + old.silver),
    lives  = old.lives ?? 3,
  }
end

let current: any = { handle = "ana", coins = 999, lives = 1 }
let hero = parse(current, Hero)
print(`{hero.handle}: {hero.coins} coins, {hero.lives} lives`)

Run it:

$ scua current.scua
ana: 999 coins, 1 lives

The hook didn't run and the values came through untouched, so a save written by the current version needs no upgrade code.

#The load is still a boundary

The migrate hook reshapes data, but it doesn't get to wave it through. Whatever comes out is still checked against the record before it becomes a trusted value. If you put a refinement on a field, a corrupt old save that migrates to a bad value is caught right at the load.

contract NonNeg(n) n >= 0 end
record Hero {
  handle: string,
  coins:  int where NonNeg,
  lives:  int,
}
migrate Hero(old)
  return {
    handle = old.name  ?? old.handle,
    coins  = old.coins ?? (old.gold + old.silver),
    lives  = old.lives ?? 3,
  }
end

let corrupt: any = { name = "glitch", gold = -80, silver = 0 }
try
  let h = parse(corrupt, Hero)
rescue err
  print(`load rejected: {err.record}.{err.field} = {err.value}`)
end

Run it:

$ scua guarded.scua
load rejected: Hero.coins = -80

The migrate ran, produced coins = -80, and the NonNeg refinement rejected it at the crossing. You catch that as a fault with the record, field, and value attached, so a bad load is a thing you can log and handle rather than a silent corruption. Contracts has more on refinements, and Errors and faults on catching faults.

#Saving from a script: the `durable` module

The host-writes-the-bytes model above is SCUA's relocation path — moving a whole live partition, byte-for-byte, with the same code on both ends. For the other case — writing a save file your script controls, that a future build with a changed schema must still load — there is the durable module. It turns a value into a typed, versioned binary blob and back, entirely from script:

import durable

record Player { handle: string, wins: int where wins >= 0 }
record Save   { season: int, players: { Player } }

let state: Save = { season = 3, players = [ { handle = "ed", wins = 12 } ] }

let blob = durable.encode(state, 2)   -- a `bytes` blob, stamped with schema version 2

durable.encode(value, schema_version?) preserves SCUA's value kinds — int vs float vs decimal vs big, plus string, bytes, and key — which JSON can't. Its on-disk layout is decoupled from memory, so you can reshape your records and old blobs still load. durable.decode(blob, Save) rebuilds the values and converts them as parse(v, Save) would — migrating and deeply validating every nested record and { Record } element. A corrupt blob and a save that doesn't fit are both an Error, with a message that says which and where:

match durable.decode(blob, Save)
  Ok(loaded) -> print(`season {loaded.season}, {len(loaded.players)} players`)
  Error(e) -> print(`not a valid save: {e.message}`)   -- bad bytes, or a corrupt nested Player
end

Saving can check too: durable.encode(state, 2, Save) checks the whole value against Save first and gives Ok(bytes), or an Error naming the bad value and no bytes, so a bug never writes a save that won't load. Without the type, durable.decode(blob) gives plain data and durable.encode(state, 2) writes what it is given.

A blob carries the version you stamped, readable without decoding the payload — handy for deciding how to upgrade:

match durable.version(blob)
  Ok(v) -> print(`this save is schema v{v}`)
  Error(e) -> print(e)
end

#Keeping saves across SCUA releases

The version you stamp is your data's schema. The blob also carries the format SCUA wrote it in, which changes only when SCUA's own encoding does. The promise for that format:

A blob written by SCUA 0.26 or later (format 2) loads in every later release. If the format ever changes, later releases keep reading format 2, so a save you keep for years still loads. Writing the newer format is then a choice you make, so an older build that must still read your files can.
A blob from a newer release is refused by name, not misread: durable.decode gives Error(durable blob is format 3, written by a newer SCUA; this release reads format 2).
Blobs from before 0.26 (format 1) no longer load.

durable.format(blob) reads the format without decoding anything, and durable.current_format() says what this release writes, so a program can turn away a newer file before it tries to load it:

let blob = durable.encode({ notes = [] }, 1)
match durable.format(blob)
  Ok(f) -> if f > durable.current_format() then print("written by a newer release") else print(`format {f}: loads here`) end
  Error(e) -> print(`not a save: {e}`)
end

An empty file and a foreign one are told apart too: durable.decode says not a durable blob (empty) or not a durable blob (bad magic).

durable.encode/decode are pure — they only turn a value into bytes and back, no capability needed. To put a save on disk, write those bytes with the fs capability: fs.write("user:saves/slot1.scd", durable.encode(state, 2)), and read them back with fs.read("user:saves/slot1.scd"). A game's saves belong in its per-user folder; Save the player's progress shows a save that survives a crash, a damaged file and an older build.

What can be saved. Every data value: int, float, decimal, big, money, datetime, string, bytes, key, arrays, tables and records, frame data tables, vectors/matrices/quaternions, colours, the geometry types, localized strings, and compiled regexes (which save as their pattern text). What cannot: things that are a reference to a running program or resource rather than data — a function or coroutine, a partition reference, a capability or host handle, a path, a text builder, and a tagged Ok/Error. Encoding one is a fault on encode, not a silent omission — lossy saves are how data quietly corrupts. NaN is also refused: it is never equal to itself, so a save containing one could not be compared or verified.

Reordering an enum or flags declaration is a schema change. A variant without a payload is stored as its position in the declaration, so a save records 1, not Green. Move a variant and every existing save changes meaning — silently, because the blob still decodes, the CRC still verifies, and durable.version still reports whatever number you stamped:

enum Colour { Red, Green, Blue }         -- v1 of your game
let blob = durable.encode({ favourite = Colour.Green }, 1)

enum Colour { Green, Red, Blue }         -- v2: the only change is the order
-- that same blob now loads with `favourite == Colour.Red`

Two habits make this a non-issue. Add new variants at the end rather than in alphabetical or logical position — appending never changes an existing value. When you do need to reorder, or when you are not sure, give the variants explicit values (enum Colour { Red = 1, Green = 2, Blue = 3 }), which pins each name to a number that no longer depends on where it sits. And when a stored variant's meaning changes for any reason, bump the schema version you pass to durable.encode and handle the old one on load, exactly as you would for a renamed field.

The format is compact — a typed, versioned save of a five-field record costs about 28 bytes, less than the same record as JSON — but it is not compressed. For large saves, compress the bytes before writing them (gzip/zstd in your host); repeated record shapes are highly redundant and compress very well.

#Verifying a save: integrity and tamper-evidence

Two different worries, two tools — both built on the durable format and SCUA's byte-determinism.

Did the save→load cycle keep everything? durable.verify_roundtrip(value) encodes, decodes, and re-encodes, and checks the bytes match exactly. A match on its result tells you whether anything was lost — a precise QA/CI guard that most languages can't offer (their re-serialization isn't byte-stable):

match durable.verify_roundtrip(state)
  Ok(_) -> ...          -- nothing lost
  Error(e) -> ...       -- a kind the format can't preserve, or a cycle
end

Did someone tamper with the save? durable.digest(value) is a SHA-256 over the value's durable encoding (and hash.sha256(bytes) hashes any bytes). It doesn't depend on how you built the table — the order you assigned fields, or keys you added and later removed — so the same data always digests the same way. It is sensitive to how a number is written: 1 and 1.0 are equal but digest differently, as do 10d and 10, or 1.0d and 1.00d. Pick one representation for anything you intend to sign, and stick to it. On its own a digest only catches accidents — a cheater can edit the save and recompute the hash. For real tamper-evidence, sign the digest with a key the player can't get. The crypto module does Ed25519:

import crypto
import hash
import durable

let keys = crypto.keypair(server_seed)?             -- server-side; seed is unpredictable, host-supplied
let sig  = crypto.sign(durable.digest(state), keys.secret)?  -- the server signs the save's digest
-- ... the client stores `sig` next to the save; on load it re-derives the digest and checks ...
let ok = crypto.verify(durable.digest(loaded), sig, keys.public)   -- public key only: verify, can't forge

This is the thin-backend pattern: the client keeps the save on its disk; the server holds only the secret key (and signs each save). The client can verify but not forge, so a small team gets cheat-resistant saves without standing up real backend infrastructure. Three cautions the cryptography demands:

The secret key must never reach the client — sign on the server.
A bare signature doesn't stop a player restoring an older, still-valid save. To prevent that, include a server-tracked sequence number in what you sign and check it on load.
durable.digest(value) covers the content, not the version stamp the blob carries. If you need to authenticate the exact bytes on disk (including the schema version), sign hash.sha256(blob) of the encoded blob instead of durable.digest(state), and verify against the bytes you read back.

SCUA gives you the primitives; your host owns the key and the server.

#Actor snapshots and your records

An actor snapshot (actors.snapshot) is a different kind of save: it holds live records, not data you convert on the way back in, so a revive checks each record type the actor holds by name, against the program doing the reviving. A migrate hook does not run there. What that means for a change:

Records the actor does not hold can change freely, and so can the order of declarations.
A held record can gain a field with a constant default, or an optional one: the revive fills it in every saved copy. Its fields can also change order.
A held record that lost a field, retyped one, or gained one with no default (or a computed one) is refused with identity_refused, and the message names the record and the field and shows both field lists. A record with where rules keeps its fields, and its rules, across a snapshot. Renaming a record, or moving it to another module, is renaming its type, so that is refused too.
A partition's own state works the same way: a new field with a constant initial value, [] or {} included, is filled in on revive, and reordering is fine. Removing a field, or adding one with a computed initial value, is refused, naming the partition.

When a change must break a save, move the actor across as data: revive it under the program that wrote it, ask it for its state, save that with durable.encode, and build the new shape from it with parse on the other side. Partitions and the actor model has the full list.

#Putting it together

Here is one program covering all four ideas: an additive default, a rename-and-merge migrate, a current-shape value skipping the hook, and a boundary rejection.

-- 1. Additive change: a new field with a default. Older saves missing it load fine.
record Settings {
  volume: int,
  muted:  bool = false,   -- added in a later version
}

let old_settings: any = { volume = 7 }     -- saved before `muted` existed
let s = parse(old_settings, Settings)
print(`volume {s.volume}, muted {s.muted}`)

-- 2. Non-additive change: a rename and a merge. A migrate hook reshapes old data.
--    The hook returns the COMPLETE current shape.
contract NonNeg(n) n >= 0 end
record Hero {
  handle: string,
  coins:  int where NonNeg,
  lives:  int,
}
migrate Hero(old)
  return {
    handle = old.name  ?? old.handle,                -- v1 stored `name`; renamed to `handle`
    coins  = old.coins ?? (old.gold + old.silver),   -- v1 had `gold` + `silver`; merged to `coins`
    lives  = old.lives ?? 3,                         -- brand-new field; give old heroes 3
  }
end

let v1: any = { name = "ed", gold = 500, silver = 20 }   -- an old save
let hero = parse(v1, Hero)
print(`{hero.handle}: {hero.coins} coins, {hero.lives} lives`)

-- 3. Already-current data skips the hook (it's already complete).
let v2: any = { handle = "ana", coins = 999, lives = 1 }
let hero2 = parse(v2, Hero)
print(`{hero2.handle}: {hero2.coins} coins, {hero2.lives} lives`)

-- 4. The load is a boundary: the result is still checked. Corrupt old data is caught here.
let corrupt: any = { name = "glitch", gold = -80, silver = 0 }
try
  let h = parse(corrupt, Hero)
rescue err
  print(`load rejected: {err.record}.{err.field} = {err.value}`)
end

Run it:

$ scua persist.scua
volume 7, muted false
ed: 520 coins, 3 lives
ana: 999 coins, 1 lives
load rejected: Hero.coins = -80

The takeaway for designing data that lasts: prefer additive changes with defaults, because they need no code. When you do have to reshape, write a migrate hook that reads the old fields with ?? and returns every field of the current record. Put refinements on the fields that have invariants, so a bad load fails loudly at the boundary instead of leaking a broken value into the rest of your game.

Write handlers that survive snapshots — the companion case: a partition snapshotted while a request is in flight. A parked I/O call resumes as an Error and is never re-issued, so you reconcile against the durable side rather than blind-retry.

#Additive changes are automatic

#Non-additive changes use a migrate hook

#The load is still a boundary

#Saving from a script: the durable module

#Keeping saves across SCUA releases

#Verifying a save: integrity and tamper-evidence

#Actor snapshots and your records

#Putting it together

#Related

#Saving from a script: the `durable` module