SCUA 0.21.0 is out

SCUA can now read and write a file of any size. fs.open gives you a handle you pull a file through a line or a block at a time, so the memory you need is the size of one piece instead of the size of the file. Alongside it: a way to tell a symlink from a file before you replace it, a deadline on served requests, long jobs that run to completion, and faster string searching.

#fs can now read and write files of any size

import fs

match fs.open("access.log")
  Ok(f) -> do
    let n = 0
    let more = true
    while more do
      match fs.read_line(f)
        Ok(line) -> if line == nil then more = false else n = n + 1 end
        Error(why) -> do
          print(`read failed: {why}`)
          more = false
        end
      end
    end
    print(`{n} lines`)
    fs.close(f)
  end
  Error(why) -> print(`could not open: {why}`)
end

fs.read_line gives you text a line at a time. fs.read_chunk(f, max) gives you raw bytes a block at a time, for binary files or when lines are not the unit you want. fs.close releases the handle.

The two readers signal the end of the file differently, and it is the one thing to remember here. fs.read_chunk is finished when it hands back an empty block. fs.read_line is finished when it hands back nil, because an empty line is a real line: if "" meant finished, a reader would stop at the first blank line in your file and not tell you.

Writing works the same way. fs.open(path, { mode = "write" }) starts a new file and { mode = "append" } continues an existing one, or starts it if it is not there yet. fs.write_chunk(f, data) adds a piece and returns how many bytes it wrote. fs.write remains the one-shot verb for content you already hold; fs.write_chunk is for output you produce as you go.

Writes go straight to disk rather than sitting in a buffer, so a failure is reported by the call that caused it rather than turning up later when you close the file. If one fails partway, the error says how many bytes got through, and the file is exactly its previous length plus that many. You know what state it is in instead of having to assume the worst.

Close what you open. Nothing enforces it during a run, and a loop that leaks handles will exhaust the operating system's file descriptors, with the failure landing somewhere that looks unrelated to the file you forgot. A program holding an open file also cannot be snapshotted, so close it before a handler that needs to survive a snapshot ends.

There is more on this in reading a file too big to load, and a runnable example in examples/streaming_files.scua.

#fs can now tell a symlink from a file

fs.lstat(path) returns what fs.stat returns plus is_symlink, and it does not follow the link.

This matters when you replace a file. The crash-safe way to rewrite one is to write a temporary file and rename it over the original, which is a single step that either happened or did not. If the original is a symlink, that rename replaces the link and leaves the file it pointed at untouched. The write reports success and the file you meant to change is unchanged. fs.stat cannot see the difference because it follows the link. fs.lstat can, so you can check before you commit to the swap.

#Served requests now have a deadline

Granting --allow-serve arms a 30 second limit on each request, adjustable with --serve-timeout. A handler that runs past it gets a located error naming the limit, rather than holding the connection open while the caller waits.

Handlers can take as long as the deadline allows, so a request that parses a payload, queries something and renders a response is ordinary work.

#Long jobs run to completion

A program that simply has a lot to do runs to the end. Work is bounded by the budget you set with --max-ops, which still stops a runaway loop, and by nothing else.

That is what makes streaming a large file practical: reading a million lines is a million ordinary calls, and a job that size should finish rather than being cut off for its size alone.

#Searching a string from a position is much faster

str.find(s, needle, from) resolves the offset in constant time on plain ASCII, and returns its answer the same way. Searching from an offset is cheaper than scanning the whole string, which is what you would expect it to be. str.rfind, str.slice, str.index, str.code, str.chars and the padding functions all take the same path.

Text with accents, CJK or emoji returns exactly what it always has. This is a cheaper route to the same answer, not a change to any answer.

A scanner that walks an offset through a large document is the shape this helps most, and it is a common one: tokenizers, log parsers, anything that advances through text a token at a time.

#Also in this release

Arrays hold twice as many elements. Building a value past the per-value ceiling is an ordinary error you can catch. A refused snapshot names the handle blocking it, so an embedder can see that the answer is to close a stream first. The --jit-stats and --mem-stats flags print however the program ends, including through sys.emit and sys.fail, which is the shape most agent tools use. And a match arm with a stray second statement tells you to wrap the body in a do … end block.

The changelog has the complete list.