SCUA 0.20.0 is out

SCUA can now exec. exec starts git, a build, a test suite, or whatever an agent just decided to run, streams the output as it arrives, and takes the whole process tree with it when you cancel. Alongside it: a list of commands you can declare up front so a program may run only those, month and weekday names in thirteen languages, frames that write CSV, and an exit status for each way a run can end.

#SCUA can now exec

import exec

match exec.run_any(["zig", "build", "test"])
  Ok(r)    -> print(`the build exited {r.status}`)
  Error(e) -> print(`it never started: {e.kind}`)
end

Run it with --allow-exec-any. Ok means the command ran and status is what it said, so a test suite failing is an Ok, because it worked. Error means it never started or could not finish, and kind is one of not_allowed, bad_argument, spawn_failed, timeout or unsupported, so a retry policy can tell "try again" from "never going to work" without parsing an error message.

There is no shell. argv is an array that goes to the operating system unparsed, so a branch name containing a semicolon stays a branch name and shell injection is not reachable. When you want pipes and globs, spawn ["sh", "-c", …] yourself.

A live command can be read as it produces output, polled, waited on and cancelled. exec.kill signals the whole process group, so a command that started its own children takes them with it. Every call carries a deadline that cannot be switched off, because a command burns wall clock instead of interpreter steps and --max-ops cannot see it. Captured output is capped, and when the cap fires you keep both ends of it: a build that fails after a megabyte of warnings puts the reason on the last line.

exec works on Windows with the same surface and the same guarantees. Cancellation there goes through a job object a child cannot escape, and which the OS tears down even if SCUA dies first. Under --io=async a command yields the way http and fs already do, so four actors each running a one-second command take about a second between them.

There is one thing to know before you switch it on. --allow-all does not grant exec, and it is the only capability it skips. Every other grant narrows what a script can reach. A command you run inherits your operating-system user instead, so a script holding exec and no fs can still run cat. Granting it hands over your account, which should be a deliberate choice.

#You can now declare which commands a program may run

When you already know what a program needs to run, say so, and then it can run only that. The declarations live in the capability file --allow-file reads:

exec = [
  ["zig", "build", "test"],
  ["git", "log", "--oneline", "-n", "{int}"],
  ["grep", "-c", "{string}", "--", "{string}"],
]

and the call drops the _any:

exec.run(["zig", "build", "test"])

The call site is the same line either way, so hardening a working program means pasting the argvs you already wrote into a file. If tightening security meant rewriting every call site, nobody would bother.

A declaration pins the program and its flags. Holes are for the values you pass, and your argv must match a declaration exactly: same length, literals identical. A hole fills exactly one argument and is never re-parsed, so a b; echo c is a single argument containing a space and a semicolon. A hole before a literal -- refuses anything starting with a dash, so a search term cannot turn into a flag and change what the command does. Anything unmatched comes back as Error({ kind = "not_declared" }) naming the closest declaration, which is usually one wrong flag away.

Mistakes in the file are refused at startup: a hole where the program name should be, an invented hole like {count}, an empty template. So is exec = true, because there is no "all commands" here. That is exec_any, and you have to name it.

#Logs can now go to standard error

When a caller parses your tool's stdout as JSON, the log lines need to go somewhere else. --log-stderr sends them to standard error:

$ scua --log-stderr tool.scua
{"matches":2,"file":"notes.txt"}

The log lines are still there, on the other channel, where the caller reading the answer will not trip over them:

[info] scanning 412 files
[warn] skipped 3 unreadable paths

It is a flag on the run rather than something the script chooses, because the program reading stdout is the one that needs it clean. You write logging the same way either way, and the flag combines with --log and --log-time.

#Improved exit codes from the scua command

When you run a script, the exit code scua returns tells you what kind of thing happened. A syntax error, a missing permission, a crash and a clean run that simply found no match each have their own status, so a wrapper can tell them apart without reading the output.

0, 1 and 2 are still yours to use however you like. scua reserves 3 to 6 for its own outcomes.

4 is the useful one if you run SCUA scripts from another program. It means the script itself is fine and the command line was wrong: it asked for a capability you did not grant. A wrapper that sees 4 can add the grant and run it again. Before, it saw 1 and had no way to tell that apart from a crash.

#Records now check the types you declared

A record field declared { string } or as an enum is now checked when the value arrives from outside, and the error says what arrived:

field 'files' of Args is string, expected { string }

This applies wherever outside data becomes a record: sys.parse_args, a typed let, a typed function parameter, a message arriving at a partition handler, and a loaded save. A list of scalars checks each element and names the index of the first wrong one. A flags field must hold a set of that flags type. A { f32 }-style numeric buffer field is converted to its declared element kind, so data arriving from JSON or a save file gets the packed storage the annotation promised.

Worth knowing when you write a tool: a command line carries text, so a { string } argument can only come from a JSON body on stdin, and an enum argument can't come from either route. Declare a string and convert it in your own code.

#fs now supports renaming files

import fs

fn replace(path, contents)
  fs.write(`{path}.new`, contents)?
  return fs.rename(`{path}.new`, path)
end

fs.rename replaces one name with another in a single step, so either the old file is there or the new one is. That is how you replace a file safely: write the new copy under a temp name, then rename it over the old one.

fs can also manage directories now: fs.mkdir creates one including missing parents, fs.remove takes a file or an empty directory, and fs.remove_all takes a tree. The granted root bounds all three, the way it bounds every other fs path, and the root itself is refused however you spell it.

fs.read and fs.read_text take { max_bytes = N }, in either direction, so a script reading a large export can raise the limit and one expecting a small config can lower it.

#Tasks now take turns

spawn do
  let i = 0
  while i < 3000000 do i = i + 1 end
  print("worker finished")
end
spawn do print("still here") end

The second task runs while the first is still counting. A long loop hands control back every few thousand steps and resumes exactly where it stopped, so a background job shares the program with everything else. There is nothing to switch on, and neither task is written differently.

Two caveats for now: this covers interpreted code, so a loop hot enough to be compiled still runs to completion, and it does not yet reach inside a partition handler. Budgets are untouched, so --max-ops=N still means this turn may run at most N steps however many times it paused. There is also --max-ops=none, for when you want no CPU limit at all.

#Dates and money in thirteen languages

time.format gained month and weekday names, so t.format("D MMM YYYY") gives 3 Jun 2024, and the AM/PM marker landed with them so h and hh work. The render layer gained matching components: @{d:month.short}, @{d:weekday}, @{d:day}, @{d:year}. One rule names all of them: a formatter names a field, and .short is the only modifier.

Both come off one table, so t.format("MMMM") and @{d:month} cannot disagree about what June is called. Thirteen languages ship, sys.locales() lists them, and anything else falls back to English.

Date order now follows the region rather than the language, so en_GB, en_AU, en_NZ, en_IE, en_IN and en_ZA are day-first. money.format follows the run locale too: a Euro amount reads 1.234.567,89 € under de_DE and 1 234 567,89 € under fr_FR. Under the neutral default and en_US the output is byte-identical to 0.19, so if you have never set a locale nothing moved.

Money values can also be taken apart now. money.amount(m) and money.code(m) read the two halves back out, and money.of(money.amount(m), money.code(m)) gives back exactly what you started with, at the original scale, including for zero-decimal currencies like JPY and three-decimal ones like KWD. Use those when you need the parts. Parsing the formatted string tells you little, since $ covers USD, CAD and AUD alike.

#Frames can now write CSV

f.to_csv()

That gives you the frame as RFC 4180 CSV, with the quoting handled: a field containing a comma, a quote, CR or LF is quoted, an inner quote is doubled, a nil cell is an empty field. csv(f.to_csv()) gives back an equal frame.

It belongs in the language because joining with commas works right up until a field contains one, and then writes a file that is wrong by one column from that row on.

Reading got stricter too. A cell is typed only when the typed value renders back to exactly the text that was there, so 42 is still an int and 1.50 is still an exact decimal, while 007, +7, -0 and 1_000 stay strings. Zip codes, product codes and zero-padded order ids survive the trip. Quoting is authoritative as well: a newline inside a quoted field stays inside it, and " padded " keeps its spaces.

#Also in 0.20.0

math.erf, math.erfc and math.lgamma arrived, the special functions statistics needs. Use math.erfc(x) rather than 1 - math.erf(x) for a tail, since erf(8) rounds to exactly 1.0 and the subtraction gives zero where the answer is about 1.1e-29.

hash.sha384 and hash.sha512 sit beside hash.sha256, take text or raw bytes, and run at native speed. They are the sizes specs ask for by name, HS384 and HS512 among them.

scua test runs every file you name, in any mix of files and directories, repeated as often as you like.

An HTTP server can log: a print inside an http.serve handler reaches the terminal now. A spawned task releases its memory when it finishes, which matters most to the servers that spawn one per request.

Error messages name what to do about them. A CPU budget that runs out gives the limit, the value and the flag that raises it, and points at the line inside the loop. An fs path refusal names which rule the path broke, rather than listing every rule and leaving you to pick. A missing import is reported as a missing import, at the import.

#What changes when you upgrade

If a script or CI step matches on exit code 1 to mean "the program is broken", it needs updating, because that case is now 2 to 6.

--jit-stats no longer switches the JIT back on. If you have ever measured with --jit=off --jit-stats and been surprised by the number, that was why, and it depended on the order you typed the flags in.

time.format("h:mm") now refuses. It used to render 02:05 and 14:05 identically. Use HH, or add the A marker that now exists.

csv() keeps leading zeros, so 007 stays 007, and it keeps whitespace inside quotes. It also refuses a duplicate header, and join refuses a suffix that collides with an existing column, because either one produced a frame that read back two different ways.

{ max = 0 } and { max_per_file = 0 } in fs.grep mean zero results, where they used to mean "no cap given".

#scua-pkg supports 0.20

scua-pkg 0.3.7 supports this release, so the package commands work against it. Its window is now 0.18 to 0.20, and 0.17 has dropped out of it.

#Next

The public registry is the launch step for scua-pkg, and it is the piece being built now: a static, CDN-served index and a signed transparency log, live from the first public publish, because first-seen history cannot be retrofitted afterwards.

Two questions from the team building on exec this month need a design decision rather than a patch, so both get written up first: whether log levels should go to standard error by default, and whether fs.grep should learn to match across lines.

The changelog has the full accounting.