SCUA 0.36.0: ship your data with your program

In SCUA 0.36.0 a program can read the data it ships with, such as levels, templates, sprites or pages, from paths that start with res:. Reading and writing ordinary files works as before; res: is a second place, kept apart from the user's files. While you develop, the data sits in a res/ folder beside the program. When you ship, scua res build turns that folder into a pack, a zip archive that is checked before anything is read from it. Packs stack: a patch or a mod is one more pack on top, and a save records which packs it ran with. This part of the release is new, and we have marked it for evaluation, so tell us what you would change. 0.36.0 also compiles more of the code people write inside loops (a bit writer like a JPEG encoder's runs seven times faster than in 0.35.0), and it adds fs.seek, files opened for update, and DEFLATE compression.

#Using packed data

The files you already read and write work as before: fs.read("notes.txt") still reads a host file under --allow-fs. res: is for the data your program brings with it. Put that data in a folder named res next to the program, or next to scua.toml if the project has one:

keep/
  scua.toml
  game.scua
  res/maps/keep.json
  res/maps/mill.json
  res/text/greeting.txt

The ordinary fs verbs read it, with res: in front of the path:

import fs
import json

print(fs.read_text("res:text/greeting.txt")?)
for name in fs.list("res:maps")? do
  let map = json.decode(fs.read_text(`res:maps/{name}`)?)?
  print(`{name}: {map.name}, {map.enemies} enemies`)
end
fs.write("slot1.json", json.encode({ level = "keep" }))?

$ scua --allow-fs=save game.scua
Welcome back, traveller.
keep.json: The Keep, 4 enemies
mill.json: The Mill, 2 enemies

The save goes to save/slot1.json, a host file, under the --allow-fs grant as before. res: needs no grant. A program that has resources and no --allow-fs can still import fs and read res:, but not the user's files. Resources are read-only: writing to a res: path returns an Error that tells you to write to a host path instead. A maps/keep.json in the user's own folder is a different file from res:maps/keep.json, so a user's folder never replaces what the program ships.

Reading res/maps/keep.json as a host path works while you develop, because the folder is there, and then breaks once you ship. So scua warns about it:

$ scua --allow-fs=. oops.scua
scua: oops.scua:2: warning: "res/maps/keep.json" is the project's resource folder read as a host file; it is not there once shipped — write "res:maps/keep.json"
{ "name": "The Keep", "enemies": 4, "music": "drums.ogg" }

res: names match case exactly on every system, so a path that works on your Mac also works once the data is packed. scua-version = "0.36" in scua.toml makes an older scua ask for an upgrade instead of reading res:x as an ordinary file name.

#Building packed data

scua res build writes a pack. It writes the same bytes for the same input every time, and it refuses names that would not unpack on every system, such as two names that differ only in case:

$ scua res build res/ -o base.pak
wrote base.pak: 3 entries (0 deflated), 482 bytes; skipped 0
  content digest 2d90c10af9adf20ff591fb02d6749bbc0737f73084d9ec474e8c11d630141358
  sha256         2fe92e244c85c495bae128ed0654ffbafabcc823dcddc20e7bef08ced2140083

$ scua res list base.pak
base.pak: 3 entries, 482 bytes
  stored          59         59  aa1e71e4  maps/keep.json
  stored          59         59  b3811d94  maps/mill.json
  stored          24         24  4507e129  text/greeting.txt
content digest 2d90c10af9adf20ff591fb02d6749bbc0737f73084d9ec474e8c11d630141358

Files are stored uncompressed unless you ask for compression, because that is fastest to read. --deflate="*.json,*.txt" compresses the files you name. scua res check runs every check on every entry. A pack made with zip works too, when you make it from inside the folder.

The shipped program has no res/ folder. It mounts the pack instead, and reads it exactly as before:

$ scua --mount=base.pak --allow-fs=save game.scua
Welcome back, traveller.
keep.json: The Keep, 4 enemies
mill.json: The Mill, 2 enemies

Every pack is checked before a line of your program runs. One that fails stops the run with exit status 2 and a code that names the check, such as E-PACK-CORRUPT or E-PACK-DIGEST for a pack pinned with --mount=base.pak#sha256=… that doesn't match. A file damaged inside a pack is caught by its CRC when it is read.

#Patches and mods are packs on top

Each --mount goes above the ones before it, and the top layer wins. A patch is a pack that holds only the files it changes:

$ scua --mount=base.pak --mount=patch1.pak --allow-fs=save game.scua
Welcome back, traveller.
keep.json: The Keep, 6 enemies
mill.json: The Mill, 2 enemies

--mount="patches/*.pak" mounts every pack in a folder, patch2 before patch10, and a policy file can list the mounts in place of the flags. fs.where says which layer a file comes from:

import fs

let w = fs.where("res:maps/keep.json")?
print(`res:maps/keep.json comes from {w.layer.name}, which hides {len(w.shadows)} other copy`)
for layer in fs.mounts().layers do print(`  {layer.name}, files: {layer.entries}`) end

$ scua --mount=base.pak --mount=patch1.pak where.scua
res:maps/keep.json comes from patch1.pak, which hides 1 other copy
  patch1.pak, files: 1
  base.pak, files: 3

A program can also mount packs itself before it starts its first actor, which is all a mod loader needs. This one mounts every zip in the player's mods folder. { root = "*" } reads a zip whose files all sit inside one folder, which is what Finder's Compress makes:

import fs
import json

match fs.find("mods", { filter = ["*.zip", "*.pak"], kind = "file" })
  Ok(found) -> for m in found do
    match fs.mount(m.path, { root = "*" })
      Ok(layer) -> print(`mod {m.name}, files: {layer.entries}, overrides: {layer.overrides}`)
      Error(why) -> print(`skipped {m.name}: {why}`)
    end
  end
  Error(_) -> nil
end

print(fs.read_text("res:text/greeting.txt")?)
for name in fs.list("res:maps")? do
  let map = json.decode(fs.read_text(`res:maps/{name}`)?)?
  print(`{name}: {map.name}, {map.enemies} enemies`)
end

$ scua --mount=base.pak --mount=patch1.pak --allow-fs=user game.scua
skipped broken.zip: E-PACK-FORMAT: not a zip archive (the file is 9 bytes, smaller than an end record)
mod castle.zip, files: 1, overrides: 0
Welcome back, traveller.
castle.json: The Castle, 9 enemies
keep.json: The Keep, 6 enemies
mill.json: The Mill, 2 enemies

A broken mod comes back as an Error the program can skip. The game grants only the player's user folder, which holds mods and the saves. A pack you ship is refused if it sits inside a folder the program can write (E-PACK-WRITABLE), because the program could rewrite it for its next run.

#A save knows which packs it ran with

actors.snapshot records the res: layers the run had, and actors.revive compares them with the layers it finds. A save made with a mod that the player has since removed is refused, and the error names the missing mod, so a game can tell the player what to put back:

import actors
import fs
import sys

partition Player
  state level = "keep"
  on Go(to)
    level = to
  end
  on Show()
    print(`at {level}`)
  end
end

for m in fs.find("mods", { filter = ["*.zip"], kind = "file" })? do
  fs.mount(m.path, { root = "*" })?
end

if sys.args()[0] == "save" then
  let p = Player()
  tell p.Go("castle")
  wait(0ms)
  fs.write("slot1.save", actors.snapshot(p)?)?
  print("saved at castle")
else
  match actors.revive(fs.read("slot1.save")?)
    Ok(p) -> tell p.Show()
    Error(e) -> print(`can't load: {e.kind}, missing {e.missing}`)
  end
end

$ scua --mount=base.pak --allow-fs=user save.scua save
saved at castle

$ rm user/mods/castle.zip
$ scua --mount=base.pak --allow-fs=user save.scua load
can't load: res_missing, missing [mods/castle.zip]

$ scua --mount=base.pak --allow-fs=user --res-missing=allow save.scua load
at castle

{ res_missing = "allow" } in the program, or --res-missing=allow, loads the save anyway. A layer that changed since the save, such as a newer patch, loads and is noted in the log, and --strict-res refuses that too.

#Serve a site from a pack

scua serve serves a pack the way it serves a folder:

$ scua res build site/ -o site.pak --deflate="*.html,*.css"
wrote site.pak: 2 entries (1 deflated), 277 bytes; skipped 0
$ scua serve site.pak --port 8737
scua: serving site.pak on http://127.0.0.1:8737/  (Ctrl-C to stop)

$ curl -i http://127.0.0.1:8737/
HTTP/1.1 200 OK
content-length: 52
connection: keep-alive
content-type: text/html; charset=utf-8
cache-control: no-store
date: Fri, 16 Oct 2026 07:18:54 GMT

<!doctype html><title>Keep</title><h1>The Keep</h1>

#One file to ship

An application bundle, a .scuapp, can now carry the app's res: files inside it. They become the bottom layer, so --mount patches and mods go above them as they do above a folder. Each file is checked against the SHA-256 the bundle records for it, and none is read into memory at launch, so large assets cost nothing until the program reads them. scua res list and scua res check take a .scuapp too. scua-pkg's next release writes these bundles with bundle --app.

#Compiled loops do more

The JIT now compiles a small function into the loop that calls it, even when the function has a short while loop of its own, sets fields on a record passed to it, or appends to an array. A short while written in the loop itself compiles with the loop, and a loop that counts a number's bits runs as one step. Here is a bit writer of the kind a JPEG or PNG encoder uses:

fn put_bits(w, code, size)
  w.acc = (w.acc << size) | (code & ((1 << size) - 1))
  w.n = w.n + size
  while w.n >= 8 do
    w.out.push((w.acc >> (w.n - 8)) & 255)
    w.n = w.n - 8
  end
  w.acc = w.acc & ((1 << w.n) - 1)
end

fn bit_length(v)
  let a = v
  if a < 0 then a = -a end
  let n = 0
  while a > 0 do
    a = a >> 1
    n = n + 1
  end
  return n
end

fn encode(values)
  let w = { acc = 0, n = 0, out = [] }
  for v in values do
    let size = bit_length(v)
    put_bits(w, size, 4)
    put_bits(w, v, size)
  end
  return w.out
end

let values = []
let s = 7
for i in 0:10_000_000 do
  s = (s * 1103515245 + 12345) % 2147483648
  values.push((s >> 16) % 2001 - 1000)
end
let out = encode(values)
print(`{len(values)} values, {len(out)} bytes`)

$ scua bits.scua
10000000 values, 16247109 bytes

On an Apple M4 Max, the whole run takes 1.29 s on SCUA 0.35.0 and 0.18 s on 0.36.0, the best of three runs each. The output is the same.

The same work shows on a real program. A JPEG encoder written in SCUA, encoding a 12-megapixel image, took these times (encoding only, on an Apple Silicon Mac, the median of three runs):

	SCUA 0.35.0	SCUA 0.36.0
quality 85	3.81 s	0.47 s
quality 100, 4:4:4	9.18 s	1.31 s

Float loops over several arrays also compile to shorter code on x86. On Linux on an Intel i7-10700, the physics benchmark, which updates four float arrays element by element, went from 24.6 ms to 17.8 ms, where LuaJIT 2.1 takes 14.4 ms.

#Files and compression

fs.open(path, { mode = "update" }) writes into a file that is already there, keeping what it holds, and fs.seek moves to any position. Together they change bytes in place, such as a count in a file's header:

import fs

fs.write("scores.txt", "entries=0002\nada 31\ngrace 27\n")?

let f = fs.open("scores.txt", { mode = "update" })?
fs.seek(f, 0, { from = "end" })?
fs.write_chunk(f, "edsger 44\n")?
fs.seek(f, 8)?                  -- the count in the header
fs.write_chunk(f, "0003")?
fs.close(f)

print(fs.read_text("scores.txt")?)

$ scua --allow-fs=. scores.scua
entries=0003
ada 31
grace 27
edsger 44

The rest of the file and compression additions:

fs.write_chunk(f, data, { from, to }) writes a range of a value straight from it, so a large buffer goes out in pieces without copies.
fs.rename(from, to, { replace = false }) never replaces an existing file. The check and the move are one step, so when two writers race to save under the same name, exactly one wins.
bytes.deflate compresses DEFLATE data, as zlib (what PNG uses), gzip or raw, and bytes.deflater compresses data that arrives in pieces.
bytes.inflate can decode a stream split across several parts of its input (a PNG's image chunks), into a { u8 } array, and in place when you know the size.

#Upgrading to 0.36

A few changes can need edits.

A misspelt option is refused. An option name a function doesn't take used to be ignored. It now stops the program before anything is written:

$ scua --allow-fs=. opt.scua
scua: opt.scua:2: fs.write: unknown option 'exclusiv' — did you mean 'exclusive'?

An optional record is checked. A value passed as Prefs? now gets the same check as Prefs whenever it isn't nil, and a record nested in another is checked at every depth. Convert loose data, such as decoded JSON, with as:

import json

record Prefs { volume: int, muted: bool = false }

fn show(p: Prefs?)
  if p == nil then return "no prefs" end
  return `volume {p.volume}, muted {p.muted}`
end

let loaded = json.decode("{\"volume\": 8}")?
print(show(as(loaded, Prefs)))
print(show(loaded))

$ scua rec.scua
volume 8, muted false
scua: rec.scua:5: field 'muted' of Prefs is missing; use `as(value, Prefs)` to fill its default
  at show (rec.scua:5)
  at main (rec.scua:12)

A host path that starts with name: is reserved, since res: is the first such prefix:

$ scua --allow-fs=. colon.scua
scua: colon.scua:2: type error: `fs.read_text`: "user:save.json" is never a valid path — a path beginning `name:` is reserved for namespaces — only res: is defined

Listings come back in natural order. fs.list and fs.find on host paths sort as res: listings do, with runs of digits compared as numbers, so e1m2 comes before e1m10. The order used to be whatever the operating system gave, which differed between systems.

The full changelog has the rest, including wording changes to some messages.

#What's next

res: and packs are marked for evaluation in this release, so their details may still change: try them on a real project and tell us what gets in the way. scua-pkg's next release writes .scuapp bundles that carry their data, so a program and everything it reads can ship as one file.