A frame can now answer the questions you ask a
spreadsheet. describe() summarises every numeric column,
value_counts() counts a text one, and
group_by() computes as many aggregates as you name in a
single pass. A dozen new reducers came with them, and the exact ones
stay exact, so the median of a column of prices is a price and not a
float that nearly is. The maths module gained a base argument for
log, along with exp2 and clamp01,
and fmod now gives the answer C and Python give. Two
changes are breaking, so read the upgrade section before you
install.
#describe, value_counts and group_by
orders.describe() gives a frame back: one row per
numeric column, with the count, mean, standard deviation, minimum,
quartiles, median and maximum. Text columns are left out rather than
filled with blanks, since a column of region names has no mean.
orders.value_counts("region") is the question those columns
do answer, and hands back each distinct value with how often it occurs,
most frequent first.
group_by takes the key column and a table of the
aggregates you want:
orders.group_by("region", {
revenue = "sum:amount",
orders = "count",
biggest = "max:amount",
typical = "median:amount",
})That reads one pass over the rows and produces one column per entry.
Each value names a reducer and a column, or a bare "count"
for the number of rows in the group. The reducers are sum,
mean, median, min,
max, std, variance,
count, count_distinct, first,
last, product and mode, and every
one of them also works on a whole column:
orders.median("amount"),
orders.count_distinct("rep"). They are the same functions
in both places, so a grouped median and a plain one agree by
construction.
Money keeps its exactness through all of it. A decimal
column is exact by default, with nothing to add and no numeric type to
pick, and its total is a decimal. Its median is the exact midpoint of
the two middle values when there is an even number of rows, at one more
decimal place only if the midpoint needs one, so the midpoint of 20.00
and 35.00 is 27.50. The reducers that hand back a cell keep the column's
type too: the maximum of a money column is money, and the mode of a text
column is text. Standard deviation and variance are floats, because a
deviation of money is not a money amount.
Four reshaping verbs came with them. distinct() drops
duplicate rows and keeps the first of each in its original position,
drop(names) is the complement of pick,
rename leaves column order alone, and tail(n)
is the other end of a sort from head. sort_by
now takes several columns and a direction for each, so "by region, then
by amount descending" is one call.
All of this runs with no import, no install step, no virtual environment and no capability grant. In Python the same job usually starts by setting up a virtual environment and installing a package. Here a script that reads a CSV and answers a question about it is one file you can send someone.
#Log with a base, and a corrected fmod
math.log(x, base) takes an optional second argument, so
math.log(1024, 2) is 10. Bases 2 and 10 go through the
hardware's own implementations, which means
math.log(1000, 10) is 3 rather than 2.9999999999999996.
Computing it by hand as log(x)/log(b) gives the longer
answer, and so does Python.
math.exp2 is log2's inverse, which was
missing while log2 was there. math.clamp01(x)
limits a value to 0 through 1, the clamp that interpolation code reaches
for. And list.min, list.max and
list.mean now sit beside list.sum, which was
the only one of the four that existed. An empty list gives
nil from all three, because an empty list has no smallest
element and saying zero would be a claim about data you do not have.
math.fmod changed its answer. It was a floored
remainder, so math.fmod(-7, 3) gave 2, where C, Python and
JavaScript all give -1. It now matches them. The floored answer is what
the % operator gives and always did, so both conventions
are available where before there was only one under a name promising the
other.
#nan follows the same rule on every tier
nan is the float you get from a calculation with no
answer, and by the IEEE rule it is not equal to itself. That is now true
wherever your code runs. Before, the interpreter said a nan
equalled itself and the compiler disagreed, so the answer depended on
whether a function had been compiled yet, and scua test
reported the interpreter's answer. Ordering comparisons were already
correct and are unchanged. is(n, n) is still true, since
is asks whether two things are the same value and one value
is.
Arrays follow their elements now as well. [nan] == [nan]
is false and [0.0] == [-0.0] is true, both matching what
== says about the values inside.
#Enum payloads are checked
A record carried by an enum variant now gets the checks it gets
anywhere else. Build
Item.Text({ at = [72, 700], text = "hi" }) where
TextItem has defaults and where refinements,
and the defaults fill and the refinements run, which they did not
before. A literal with a field the record does not declare is a compile
error, the same one you get from let t: TextItem = { … }.
Values arriving from JSON are checked the same way, so
{"kind":"Text","value":[1,2]} is refused where it crosses
instead of failing later somewhere else.
A parameter or let typed with an enum is checked too.
fn f(p: Direction) used to accept any value at all.
#Upgrading
Two things to check before you install.
If you call math.fmod with operands of opposite signs,
its answer has changed. Use the % operator for the old
behaviour, which is unchanged.
If a program builds an enum variant from a value that does not satisfy the variant's declared payload type, or passes a non-variant where an enum type is declared, it now stops at that point with a located error rather than carrying the wrong value forward. The error names the record or enum it expected. Nothing that was already correct needs changing.
The full changelog has the rest, including a clearer message for a stray character and a build that tells you whether it was compiled for speed.