SCUA scripts can now call native libraries directly

A SCUA script can open a compiled library on the machine, describe one of its functions, and call it. Nobody writes a wrapper per function, in C or in anything else. The library is the same .dylib or .so a C program would link against, which is what native means here, and the mechanism is the one other languages call a foreign function interface, or FFI.

Until now a job that needed SQLite, or any other library that lives in C, was a reason to write the script in Python and use its ctypes. In a source build of 0.25.0 that reason is gone.

#SQLite from a script

This is the flavour of it. Each SQLite function is named, given its result and argument types from sqlite3.h, and called.

let lib = ffi.open("sqlite")
let Stmt = ffi.opaque("sqlite3_stmt")
let step = ffi.bind(lib, "sqlite3_step", "i32", [Stmt])
let column_text = ffi.bind(lib, "sqlite3_column_text", "cstr", [Stmt, "i32"])

while ffi.call(step, stmt) == 100 do -- SQLITE_ROW
    print(ffi.call(column_text, stmt, 0))
end

The rest of a working program is the same shape: opening the database through an out-parameter, preparing the statement, binding a value, closing things in the right order. It runs, and it reads as plainly as it looks. FFI interfaces are like that in most languages, and most scripts should never see one. The plan is a SQLite package that wraps these calls in a handful of functions, and the FFI is what that package is built from. What the raw layer gives the package author is worth having: ffi.opaque keeps a statement from being passed where a database is expected, an out-parameter slot names its destructor as it is declared, and a destructor that returns an int, as sqlite3_close_v2 does, has that result checked. If SQLite refuses to close because a statement is still live, the close faults with the symbol and the code instead of dropping the handle and leaking the connection.

#When you get a call wrong

A native library cannot check what you hand it, so the binding does. An argument of the wrong kind faults with its position, what the binding declared and what arrived: argument 2: expected i64, got string. A string that SQLite would keep after the call returns, sqlite3_bind_text with the static destructor, is refused at the call with the fix named: declare the argument {type = "cstr", retained = true} and pass ffi.native_string(lib, s), a copy that lives as long as its handle. Before that refusal existed, the same call segfaulted at the next step. A fault raised inside a callback of your own reaches you with its own message.

#What it reaches

Anything with a C interface has this shape. That includes a good part of the local AI tooling: llama.cpp, whisper.cpp and ONNX Runtime each ship a C API of the same kind as SQLite's. We have driven the FFI against SQLite. Those have the same kind of surface, so the door is open.

#What it needs today

The FFI is opt-in. It is in a source build made with zig build -Dffi=true, on macOS and Linux; Windows x86-64 is a preview tested under Wine. The downloaded release binaries do not include it.

A library is granted by the program that embeds SCUA, and there is no command-line flag for it, on purpose. A native library runs with the process's authority, so the decision to let a script reach one belongs to whoever runs the process. The example host in the repository does it in a few lines of C, and a script then opens the library by the name the host gave it. A Clang-based importer can generate the declarations from a header when there are many.

Native libraries and FFI has the build steps, the example host, and the limits.