Records and gradual types

SCUA is dynamic by default. You can write a whole program with no type annotations and it runs like any scripting language. Annotations are something you add where a compile-time check pays off, usually around the data that matters: saved state, messages between systems, the shape of an entity.

The rule that makes this work: annotations are erased. They tell the compiler what to check, and then they're gone. A correct program runs exactly the same with or without them. Adding a type never changes behaviour; it only adds a check that catches mistakes earlier.

#A record

A record declares a named type with a fixed set of fields:

record Player {
  name:  string,
  hp:    number,
  level: number = 1,
}

let hero: Player = { name = "Ed", hp = 100 }
print(`{hero.name}: {hero.hp} hp, level {hero.level}`)

Run it:

$ scua hero.scua
Ed: 100 hp, level 1

The literal is checked field by field against Player: every required field present, no unknown keys, each value the right type. Records are structural, so any table with the right fields fits the type; you don't construct them with a special syntax. (If you reach for Player{ … } out of habit from another language, the compiler reminds you: a record type isn't a value, so build one by assigning a table literal to a typed binding as above.)

#Field defaults and optional fields

A field with a default (level: number = 1) can be left out at construction. It's auto-filled, which is also how older saved data that predates the field gets a sensible value when loaded.

A field typed with a trailing ? is optional. It may be omitted, and reads back as nil:

record Hero {
  name:  string,
  title: string?,   -- optional
  level: number = 1 -- default
}
let a: Hero = { name = "Ed" }
let b: Hero = { name = "Mara", title = "the Bold", level = 9 }
print(`{a.name}, level {a.level}, title {a.title}`)
print(`{b.name}, level {b.level}, title {b.title}`)

$ scua optional.scua
Ed, level 1, title nil
Mara, level 9, title the Bold

A default gives an omitted field a value. An optional field is allowed to be absent. They're different tools: reach for a default when there's a sensible value, an optional when absence is meaningful.

A field default fills a field that is missing. Writing nil explicitly keeps the nil:

record Cfg { host: any = "localhost" }
let a: Cfg = { }              -- host is "localhost"
let b: Cfg = { host = nil }   -- host is nil

That differs from a parameter default, which fills a missing or nil argument. The reason is that a record is something you can ask about — a field can be there or not, and both can mean something — while a parameter always holds a value once the call starts.

#Type aliases

type names a structural shape without declaring a sealed record. Use it for shapes you want to refer to in several places:

type Profile = { name: string, title: string? }

let p: Profile = { name = "Ana" }   -- title omitted
print(p.name)

$ scua profile.scua
Ana

#What gets checked

Typed bindings, parameters, and return types are all checked at compile time. Field access flows the field's type, so a chain of accesses stays checked. Introduce a mistake and the compiler locates it before the program runs:

record Player { name: string, hp: number }
let hero: Player = { name = "Ed", hp = "lots" }
print(hero.name)

$ scua mistake.scua
scua: mistake.scua:2: type error: field 'hp' is string, expected number

The same goes for arguments at a call site:

fn restore(p: number, amount: number) -> number
  return p + amount
end
print(restore("hello", 5))

$ scua arg.scua
scua: arg.scua:4: type error: argument 1 to 'restore' is string, expected number

A record literal written where the record type is declared builds a real record: a typed let (let p: Player = { … }), a return from a function declared -> Player, a field of that type, an element pushed into or stored in a { Player }, or an argument to a Player parameter of a function in the same file. Fields with a default, or marked optional, may be left out; the runtime fills them in (see Literals become records):

record Player { name: string, hp: number, mana: number = 100 }
fn heal(p: Player) -> number
  return p.hp + p.mana
end
print(heal({ name = "Ed", hp = 10 }))

$ scua literal.scua
110

When the literal is wrong, the error names the field, with the same hint a typed let gives. A misspelt field is also a missing one, so both are reported:

record Player { name: string, hp: number, mana: number = 100 }
fn heal(p: Player) -> number
  return p.hp + p.mana
end
print(heal({ naem = "Ed", hp = 10 }))

$ scua typo.scua
scua: typo.scua:5: type error: unknown field 'naem' on Player (did you mean 'name'?)
scua: typo.scua:5: type error: missing field 'name' of type string

Untyped code stays fully dynamic and pays nothing. A table with no annotation is never shape-checked; you can grow it freely.

#Every problem at once, and warnings

The checker reports everything it finds in one go, so you can fix a file in one pass: every error in file and line order, then the warnings. Any error stops the program before it starts.

record Player { name: string, hp: number }
let hero: Player = { name = "Ed", hp = "lots" }
let hp: int = hero.name
let n = tonumber([hero.hp])
print(hero.nam)

$ scua mistakes.scua
scua: mistakes.scua:2: type error: field 'hp' is string, expected number
scua: mistakes.scua:3: type error: value is string, but the binding is declared int
scua: mistakes.scua:5: type error: unknown field 'nam' on Player (did you mean 'name'?)
scua: mistakes.scua:4: warning: `tonumber` is given an array ({ number }), which is never a number, so this always gives nil — pass it a string or a number

A warning is code that runs but almost certainly doesn't do what it says. On its own it doesn't stop anything. To make it stop the run, start the file with --!strict: that file's warnings then fail the run as errors do, and nothing runs.

--!strict
let ports = [8080, 8081]
let port = tonumber(ports)
print("listening on", port)

$ scua strict.scua
scua: strict.scua:3: warning: `tonumber` is given an array ({ int }), which is never a number, so this always gives nil — pass it a string or a number
scua: nothing ran: 1 warning is in a --!strict file, where a warning fails the run

For a whole run, CI and agent harnesses pass --deny-warnings on the command line instead. Warnings in the packages you depend on are quiet unless you ask for them with --dep-warnings, and they never stop your run on their own.

When a log should hold only what stops the run, such as the output a tool or an agent reads line by line, pass --no-warnings. Errors still print and still stop the run; the warnings don't print:

let ports = [8080, 8081]
let port = tonumber(ports)
print("listening on", port)

$ scua --no-warnings quiet.scua
listening on nil

A warning in a --!strict file still prints, since that file made it an error and the run stops on it. Your editor keeps showing every warning.

To make that the default for everyone who runs a project, put it in the project's scua.toml:

[run]
warnings = false

The command line still wins: scua --warnings shows them, and --deny-warnings counts them as usual.

#The boundary rule: parse, don't validate

A type annotation on a parameter, a return or a let checks one level as the value crosses: its kind and, for a record, the kind of each of its own fields and its own where rule. parse(value, T), a decoder given a type and an encoder given a type check all the way down, running where rules at every level, and parse and the decoders also convert, filling defaults and running migrate hooks, without changing the value they are given.

#An annotation checks one level

For a record, one level means:

• a scalar field (int, float, number, string, bool, decimal, money, big, bytes, key) holds a value of that kind;
• an enum or flags field holds a variant of that declaration;
• a field typed as a record holds a table, and a collection field ({ string }, { Item }) holds an array. What is inside them isn't looked at;
• a { f32 }-style numeric buffer holds values of its element type. One that already has that type is kept; any other array whose values fit is copied into a new buffer, which the field then holds.

An annotation never converts: no default is filled and no migrate runs, so a field the value lacks is refused even when it has a default. A record literal written where the type is declared is the exception, because it is built there (see Literals become records).

A shape type, written as a table ({ id: string, n: int }) or named with type R = { id: string }, is checked the same way: the value must be a table holding each declared field at its kind. Extra fields are allowed, and a literal of a shape type stays a plain table rather than becoming a record. A map type such as { [string]: int } names no fields, so only its kind is checked.

A record literal is checked too when a field's value is only known at run time. -> Note returning { id = id } with an untyped id that holds 1 is refused, naming the field, rather than building a Note whose string field holds an int.

record Entry { id: string, amount: int = 0 }
record State { gold: int, ledger: { Entry } = [] }
fn sys(s: State) -> int return s.gold + 1 end

let bad: any = { gold = "lots", ledger = [] }
try print(sys(bad)) rescue e print(e.message) end
let short: any = { gold = 1 }
try print(sys(short)) rescue e print(e.message) end
let s: State = { gold = 1 }        -- a literal is built: `ledger` gets its default
print(sys(s), len(s.ledger))

$ scua crossing.scua
parameter 's' of 'sys': s.gold is string, expected int
parameter 's' of 'sys': s.ledger is missing; use `parse(value, State)` to fill its default
2 0

A bare collection is checked the same way: fn sum_amounts(xs: { Entry }) refuses a string, and doesn't walk the entries. P? and { T }? get the same check whenever the value isn't nil.

record Entry { id: string, amount: int = 0 }
fn sum_amounts(xs: { Entry }) -> int
  let n = 0
  for e in xs do n = n + e.amount end
  return n
end
let v: any = "hello"
try print(sum_amounts(v)) rescue e print(e.message) end
print(sum_amounts([]))

$ scua bare.scua
parameter 'xs' of 'sum_amounts' is string, expected { Entry }
0

Because an annotation looks one level deep, passing a record costs the same whatever it holds. A game state with a 50,000-entry ledger costs as much to hand to a typed function as one with an empty ledger.

A scalar annotation on a binding is checked too: let n: int = x faults if x holds anything but an int. The test is left out only when the value is already known to be an int, such as a literal or arithmetic on checked ints. A value read out of a record's field is always tested, since the record was checked one level:

let s: any = "seven"
try
  let n: int = s
  print(n)
rescue e
  print(e.message)
end

$ scua scalar_let.scua
binding 'n' is string, expected int

A parameter with a scalar type is checked whenever the program calls the function, and a declared return type every time the function returns, for each value a -> (int, string) function returns. The rules are the checker's: an int satisfies float and number, int? also accepts nil, and a default fills a missing argument before the check. NaN is a float, so a float parameter accepts it. Falling off the end of a function with a declared return type returns nil and is checked like any other return, so -> int needs a return on every path, where -> int? does not.

fn half(x: int) -> int return x // 2 end
let v: any = 2.5
try print(half(v)) rescue e print(e.message) end
fn pair(a: any) -> (int, string) return 1, a end
try print(pair(2)) rescue e print(e.message) end

$ scua scalars.scua
parameter 'x' of 'half' is float, expected int
return value 2 of 'pair' is int, expected string

Caught with try/rescue or pcall, a rejected crossing is a record, like a where refinement's: e.message is the text above, e.path the path to the value that didn't fit (s.gold), e.at the same path from inside the value (gold), and e.value what arrived.

#parse checks and converts all the way down

parse(value, T) builds a real T from loose data: a new record for every record in it, defaults filled, any migrate hook run, every nested record and every element of a typed collection or map checked with its where rules, all the way down, including self-recursive types like a linked list or a tree. T can be any type you could write in an annotation: parse(v, Player), parse(v, { Item }), parse(v, shapes.Pt). sys.parse_args(Args) does the same with the command line. Data from outside your program, such as a save, a JSON body or host input, goes through parse once, where it arrives, and typed code trusts it from then on:

record Item   { id: string }
record Player { name: string, hp: number, inventory: { Item } = [] }

let incoming: any = { name = "Ana", hp = 80, inventory = [{ id = "bow" }] }
let ana = parse(incoming, Player)   -- checked all the way down here, trusted afterward
print(`recruited {ana.name} with {ana.hp} hp and {len(ana.inventory)} item`)

let bad: any = { name = "Bo", hp = 70, inventory = [{ id = "axe" }, { id = 5 }] }
try parse(bad, Player) rescue e print(e.message) end

$ scua boundary.scua
recruited Ana with 80 hp and 1 item
parse(bad, Player): bad.inventory[1].id is int, expected string

parse never changes the value you give it. Its result is a new value, so a change you make to the result does not reach the input: convert once where the data arrives, and keep working on the result. A const (or freezed) value that needs nothing converted comes back as the same value, at no cost.

A field typed decimal or money also takes its exact text, "1.5" or "19.99 USD", which is how json.encode writes them, so a value saved as JSON reads back as it was.

Keys the record doesn't declare are left behind by parse, on purpose: data written by a newer version of your program, or by another service, often carries fields this one doesn't know yet. For data that has to be exact, such as a hand-written config where a misspelt key would quietly use the default, pass { unknown = "refuse" }. It refuses a key no field names, at every level, and names the field it was probably meant to be:

record Pity   { hard: int = 90, soft: int = 70 }
record Banner { name: string, pity: Pity = {} }

let raw: any = { name = "Legends", pity = { hrad = 80 } }
print(parse(raw, Banner).pity.hard)                    -- `hrad` is left behind; `hard` keeps 90
try
  parse(raw, Banner, { unknown = "refuse" })
rescue e
  print(e.message)
  print(e.at)
end

$ scua strict.scua
90
parse(raw, Banner): raw.pity.hrad is not a field of Pity; did you mean hard?
pity.hrad

To use the same options in many places, name them once with a top-level const: const STRICT = { unknown = "refuse" }, then parse(raw, Banner, STRICT). The compiler reads the options, so they must be a table literal or a top-level const holding one.

A record with a migrate hook is the exception: its old data carries fields the hook renames, so it isn't checked for unknown keys.

A rejected parse is a fault, not an Error value, so it needs try … rescue (or pcall); the ? operator only unwraps Ok/Error.

#Decoding into a type

json.decode, durable.decode, toml.parse and ini.parse take the type as an optional second argument. With it, the result is Ok(T), checked and converted as parse would, and anything that doesn't fit is an Error holding the fault record, bad bytes included, so ? handles a bad save the way it handles a corrupt one and one Error(e) arm reads e.message for both. A decoder's own complaint about the bytes keeps its line and column where it gives them:

import json

record Line  { item: string, qty: int }
record Order { id: string, lines: { Line } }

fn load(text: string) -> Result
  let order = json.decode(text, Order)?
  return Ok(`order {order.id}: {len(order.lines)} lines`)
end

print(load("{\"id\": \"A1\", \"lines\": [{\"item\": \"tea\", \"qty\": 2}]}"))
match load("{\"id\": \"A2\", \"lines\": [{\"item\": \"tea\", \"qty\": true}]}")
  Ok(v) -> print(v)
  Error(e) -> print(e.message)
end
match load("{\"id\": ")
  Ok(v) -> print(v)
  Error(e) -> print(e.message, e.line, e.column)
end

$ scua orders.scua
Ok(order A1: 1 lines)
json.decode(text, Order): value.lines[0].qty is bool, expected int
json.decode(text, Order): invalid JSON at line 1, column 8 1 8

The options parse takes go after the type: json.decode(text, Config, { unknown = "refuse" }). The type is read where the call is written, so a decoder held in a variable (let d = json.decode) takes none. Without the type, a decoder gives plain data, which typed code checks only one level.

#What typed code trusts, and what it doesn't

Untyped code can still change a record after it was checked: a function with an untyped parameter can write x.items[0].id = 5. The next annotation doesn't look inside items, so the bad value is found where something reads it. A declared return type, a typed let, a typed parameter of the field's own type, or arithmetic catches it. Something that accepts any value, such as a template, .., print or ==, does not:

record Item { id: string }
record Bag  { items: { Item } = [], owner: string }
fn first_id(b: Bag) -> string return b.items[0].id end
fn first_label(b: Bag) return `item {b.items[0].id}` end
fn scribble(x) x.items[0].id = 5 end   -- untyped code writes the wrong kind

let bag = parse({ owner = "Ana", items = [{ id = "bow" }] }, Bag)
print(first_id(bag))
scribble(bag)
print(first_label(bag))
print(first_id(bag))

$ scua bag.scua
bow
item 5
scua: bag.scua:3: return value of 'first_id' is int, expected string
  at first_id (bag.scua:3)
  at main (bag.scua:11)

A record's own where rule runs at every annotation, over nested data the annotation didn't check, so write a where that reads only its own record's fields.

#Saving a checked value

To check a whole value all the way down before you save it, give durable.encode or json.encode the type: durable.encode(value, version, T), json.encode(value, T). Nothing is copied. If anything in the value doesn't fit, or has no saved form (a function left in a field typed any, say), you get an Error naming the path, and nothing is encoded:

import durable

record Player { name: string, rating: int where rating >= 0 }
record Save   { season: int, players: { Player } = [] }

fn save_bytes(s: Save) -> Result
  let blob = durable.encode(s, 2, Save)?
  return Ok(len(blob) > 0)
end
fn loosen(x) x.players[0].rating = -5 end

let s = parse({ season = 3, players = [{ name = "Ana", rating = 10 }] }, Save)
print(save_bytes(s))
loosen(s)
match save_bytes(s)
  Ok(v) -> print(v)
  Error(e) -> print(e.message)
end

$ scua save.scua
Ok(true)
durable.encode(s, 2, Save): s.players[0].rating violates Player's where rule (got -5)

Without the type, an encoder writes what it is given. Where a typed encode is written, the checker warns at each field of the type that could hold a value with no saved form (any, a collection of any, a function type). Save the player's progress shows a save that keeps the last good one when this happens.

#Literals become records

A record literal written where the record type is known is built as that record, with its defaults filled: a typed let, a return from a function declared -> Player (or -> { Player }), a field of that type, a record field's default, an element pushed into, inserted into or stored in a typed collection, and an argument to a typed parameter of a function in the same file. Untyped code can't add a key to it afterwards:

record Entry { id: string, amount: int = 1 }
record State { ledger: { Entry } = [] }
fn add_entry(s: State, id: string) push(s.ledger, { id = id }) end
fn tag(x) x.note = "loose" end

let s: State = {}
add_entry(s, "a")
print(s.ledger[0].amount)
let e: any = s.ledger[0]
try tag(e) rescue err print(err) end

$ scua ledger.scua
1
cannot add field `note` to Entry: a record has only the fields its declaration lists

The checker checks such a literal as the record it becomes: a required field left out, or a field the record doesn't have, is a compile error where the literal is written. A literal handed to a function through a variable, or to a method, is checked as it is and not built, so leaving out a field with a default is refused there; write parse({ players = 8 }, Settings) at the call.

#Checking everything while you test

Run with the deep mode to check every annotation all the way down, as parse does, without converting anything. It only adds faults, so a program that passes with it passes without it:

$ scua --check=deep bag.scua
bow
scua: bag.scua:4: parameter 'b' of 'first_label': b.items[0].id is int, expected string
  at first_label (bag.scua:4)
  at main (bag.scua:10)

#When the checker warns

A typed let or a return of a value typed any into a record that holds other records or collections checks only the outer record, which is rarely what loading code means. The checker says so:

import durable

record Player { name: string }
record Save   { season: int, players: { Player } = [] }
let v: any = durable.decode(durable.encode({ season = 1, players = [] }, 2))?
let s: Save = v

$ scua load.scua
scua: load.scua:6: warning: `v` is `any`, so `let s: Save = v` checks only Save's own fields; nested `players` is not checked. Use parse(v, Save) to check it all the way down.

Write durable.decode(blob, Save)? or parse(v, Save) instead, and the warning goes. A parameter never warns, since code that calls through function values passes any on purpose.

#What each operation promises

Every promise is about the moment the check ran. Removing an annotation never adds a fault, and nor does turning the deep mode off.

#Records are sealed

At runtime a record is a fixed-layout instance. Updating a declared field is fine. Adding a field that wasn't declared is not.

If the compiler can see the type, it catches the bad write directly:

record Player { name: string, hp: number }
let hero: Player = { name = "Ed", hp = 100 }
hero.hp = 90          -- ok: declared field
hero.trophies = 5     -- undeclared

$ scua sealed.scua
scua: sealed.scua:4: type error: unknown field 'trophies' on Player

When a record reaches code that has lost sight of its type, for instance an untyped function parameter, the seal still holds at runtime. Adding an undeclared field raises a fault you can catch:

record Player { name: string, hp: number }
let hero: Player = { name = "Ed", hp = 100 }

fn tamper(x)            -- x is untyped, so the checker can't see the shape
  x.trophies = 5
end

try
  tamper(hero)
rescue e
  print(`caught: {e}`)
end

$ scua sealed-runtime.scua
caught: cannot add field `trophies` to Player: a record has only the fields its declaration lists

A plain untyped table, by contrast, grows as much as you like. The seal is a property of records, not of every table. A record keeps its seal when it is sent to an actor, too: it arrives as the same record type, so a handler that wants extra fields copies what it needs into a table of its own.

There is no fixed limit on how many fields a record can declare. A binding surface with one field per bound function is a record like any other, however wide it is. (The technical ceiling is 65535, the same as Java's, and the compiler tells you the count if you ever reach it.) A plain table with more than sixteen keys keeps working too. It switches to hash-map storage, which is why its iteration order stops being insertion order at that point.

#Function types

A parameter, field, or binding can be typed with a function signature fn(T) -> R. The argument you pass must match the signature, and a call through a function value checks its arguments and yields the declared return type:

fn apply(f: fn(int) -> int, x: int) -> int
  return f(x)
end
fn double(n: int) -> int return n * 2 end
print(apply(double, 21))

let combine: fn(int, int) -> int = fn(a: int, b: int) -> int return a + b end
print(combine(40, 2))

$ scua fntype.scua
42
42

Still gradual: an unannotated function (its parameters are any) satisfies any function type, and fully untyped higher-order code is never checked.

#Evolving a schema

Saved data outlives the code that wrote it, so records have to handle older shapes.

Loading old data into a record is a conversion, and the conversion is parse(value, Record). A typed let on its own only checks a value; parse builds the record, and that is the step that upgrades old shapes.

Additive changes are automatic. Add a field with a default and old saves that lack it get the default when parse converts them. No extra code.

Non-additive changes, like renaming or splitting a field, need a migrate hook. It takes the old data and returns the current shape. parse runs it only when the data isn't already complete, so current data skips it:

record Account { handle: string, gold: number }
migrate Account(old)
  return { handle = old.name ?? old.handle, gold = old.gold }
end

let v1save: any = { name = "ed", gold = 500 }   -- old save, pre-rename
let acct = parse(v1save, Account)                   -- the hook upgrades it
print(`account {acct.handle} has {acct.gold} gold`)

$ scua migrate.scua
account ed has 500 gold

Here v1 stored name and v2 renamed it to handle. The ?? operator picks the first non-nil value, so the hook reads old.name from old saves and old.handle from anything already current.

#See also

• Pattern matching — destructuring records and tables.
• Enums — closed sets of named variants.
• Audience gates — gated fields and project, for sending an audience-safe view.
• Contracts — where refinements that constrain field values at the boundary.
• Errors and faults — try/rescue and the sealed-record fault.