jf Documentation

Here are several examples of Jeff expressions and their results, along with some complete scripts and commands you can run on ANSI-compatible unix-like terminals.

Jeff/jf (Json Formulas) is just a command-line app which lets you view, transform, and convert data from/to common data formats, like JSON, JSONL, CSV, TSV, SSV, HTML, plain text, and many others. Input and output are JSON by default, unless overridden by named file/URI inputs, or via options -from and -to. Jeff even has special line-by-line input modes both for plain-text (called each-line) and for JSONL data (called each-value), enabling stream-processing.

Jeff also lets you run a command to transform data, run using a surprisingly capable scripting engine. If the command is a dot, it means keep input data as they are. Scripts give you all sorts of conveniences you may have come to expect from other popular tools, such as negative indexing, but that's barely scratching the surface: Jeff gives you many other handy features, such as automatic depth-first recursion for most operators and functions.

Scripts can use 200+ predefined funcs, not counting aliases; plenty of global variables with all sorts of data are also available. Some of these predefined functions/values are documented on this help page. Many names/aliases should be familiar if you know JavaScript, Go, or Python: just avoid package names, avoid underscores, keep all letters lowercase, and chances are you'll get the right Jeff names.

As frequently shown here, the most convenient way to call 1-input funcs on non-objects is to use them as fake-properties of their input. Since objects have real properties, you have to use parentheses to call funcs explicitly on them, as many examples show.

Jeff can even output documents via special comments in HTML and SVG files, when they're used via the template option, while still being able to (optionally) load inputs files. The results of all template expressions are always automatically escaped. You still want to ensure any URIs you emit are valid, as their escaping is safe but useless as (non-working) links.

Types

Here are all types functions support: values of types bigint, fraction, and skip aren't compatible with JSON.

Input/Output Formats

Json Convert (jc) is a tool which can convert several formats into JSON, especially those Jeff doesn't know how to load, such as YAML. Just cmd-line pipe jc in front of jf, and you're good to go.

Here are values options -from and -to accept: the default input type is `json`, unless filenames or webserver responses suggest otherwise; the default output is also `json`.

All IO types, except `envars`, and `live`, have a counterpart ending in `.gz` or `.gzip` (i.e. jsonl.gz), indicating it's GZIP encodied. All input types also have BZIPped counterparts, their names ending in `.bz`, `.bzip`, or `.bzip2`. Even IO aliases have such counterparts.

Examples of Various Syntax

Examples of Unary Operators

Examples of Binary Operators

Examples of Other Operators

Examples of Normal Functions

Examples of Evaluator Functions

def({
    f: (x, y) => `${x} + ${y} = ${x+y}`,
    g: str => str.trimmed(),
    h: (x, y, z) => x*y + z,
})

Incomplete Examples

These examples are missing concrete input: use them in a pipe, or add a filename/URI with relevant data right after, and they'll do work.

Encode generic data into base64.

jf -from=bytes -to=text in.base64

Encode generic data into a data URI.

jf -from=bytes -to=text in.datauri

Decode base64 or data-URI text into raw bytes.

jf -from=text -to=bytes in.debase64

Compress generic data as gzip.

jf -from=bytes -to=bytes.gz

Decompress generic gzip data.

jf -from=bytes.gz -to=bytes

Decompress generic bzip2 data.

jf -from=bytes.bz -to=bytes

Restrict line-width of plain-text paragraphs up to 75 symbols per line.

jf -from=blocks -to=text '[squeeze(v^space).folds(75): in] ^ 2*lf'

Mimic the `strings` cmd-line app available on linux/mac.

jf -from=bytes -to=lines in.strings

Complete Examples

The following aliases can spare you a lot of typing when using Jeff directly:

• alias jfa='jf -to=ansi'

• alias jfl='jf -to=lines'

• alias jfn='jf -from=null'

• alias jfna='jf -from=null -to=ansi'

• alias jfnl='jf -from=null -to=lines'

Wondering how long float64 mantissas will work as unix-style timestamps... are those in millionths or billionths of a second?

jf -from=n '2**{52,53,64}/{1e6,1e9}/day/365.25+1970'

A partial ASCII table.

jf -from=n -to=ansi '{v: v.ord: space+visascii}'

Calculate how long do 4GB of PCM-audio (.wav files) last, under various settings.

jf -from=n -to=a '
    // msg(`How long do 4 GBs last as PCM sound (.wav) data?`) |
    {44_100, 48_000, 96_000} * {mono: 2, stereo: 4} | _ // bytes per second
    . revalue(data_rate => (4*gb/data_rate).s2hms) // convert into time durations
    . rename(k + ` hz`)'

Another way to do it.

jf -from=n -to=a '
    // msg(`How long do 4 GBs last as PCM sound (.wav) data?`);
    {44_100, 48_000, 96_000} * {mono: 2, stereo: 4}; // bytes per second
    4*gb/_; _@s2hms // convert into time durations
    . rename(k + ` hz`)'

Find how many KBs the commonest MP3/AAC fixed data-rates take per minute.

jf -from=n '{`128k`: 16, `192k`: 24} * {v.fix1: v: .5*20.iota} * 60'

Same as above, but using a more general formula.

jf -from=n '{`${64*v}k`: 8*v: 2..4} * {v.fix1: v: .5*20.iota} * 60'

You say ... about 1500 each day, what range is that a year?

jf -from=n 1500.fuzz*[365,366].kv

Now you tell me it's about 1400-1500 a day, so I guess you meant to say between 1400 and 1500 a day, which makes the yearly ranges...

jf -from=n [1400,1500].kv*[365,366].kv

Get me a nested/recursive list of filenames and their sizes starting from the current folder.

jf -to=a -from=n dot.fs

Now get me the flattened version of the same thing.

jf -to=a -from=n dot.fs^slash

Approximate e using its truncated-series expansions.

jf -from=n '{
    e,
    approx: a = 0 | (0..20).map(v, a = a + v.factorial.inv | a)
}'

Same, but use fractions for better accuracy, since using float64s exceeds the actual e fairly quickly. This example also shows how to do it using the special dictionary-builder syntax.

jf -from=n '{
    e,
    approx: a = 0n; {v: a = a + v.bigint.factorial.inv | a.n: 0..20}
}'

Emit ANSI-colored tiles from exponentially-distributed random values: the colorscale used conveys the relative intensity of each value to the highest value in each line/array.

jf -from=null -to=ansi "(`tput lines`-3) . map((`tput cols`-6) . map(rexp().tile) ^ '')"

Get date/time-related info for your timezone.

jf . https://worldtimeapi.org/api/ip

Get date/time-related info for your timezone, then ignore info about your own IP address, so it doesn't show in the result.

jf 'in . del(`client_ip`)' https://worldtimeapi.org/api/ip

Start JSON webserver on http://localhost:1234 to get the current date/time.

jf -from=null 'servejson(1234, now)'

Start webserver on http://localhost:1234 to get the current date/time.

jf -from=null 'servetext(1234, now.weekday[:3] + ` ` + now.string)'

Start JSON webserver on http://localhost:1234 with various live services.

jf -from=null 'servejson(1234, {
    `/`: `scripts are available at now/, pow2/, base64/, and debase64/`,
    now: now,           // the current date/time
    pow2: q.x.n.pow2,   // 2**x, given query parameter x
    base64: q.x.parse.base64, // base64-encode parameter x, a numeric JSON array
    debase64: q.x.debase64,   // base64-decode query parameter x
})'

Show colored tiles of increasing value.

jf -from=n -to=a '(0..50) . map((v/50).tile) . join . quote(nan.tile)'

A quick way to view a few base-10 logarithms, using ANSI styles/graphics.

jf -from=n -to=a '{
    log10: 10.map(v, v.log10.fix2 + space + v.log10.tile),
    trend: 10.map(v.log10).spark,
}'

A complete shell function to Group Digits by adding commas to make numbers more readable.

gd() { jf -from=el "in . fields . map(v.tryn.nice(${1:--1})) ^ ' '" "${@:2}"; }

A complete shell function to REPeat the text given.

rep() { jf -from=null "repeat('${1:--}', ${2:-80}, '${3:-}')"; }

Show 1-level-deep entries of the folder given, sorting entries by size, largest to smallest. All entries with NaN sizes are subfolders.

echo 'data/folder' | jf -from=text -to=ansi 'in.ls.sort(-v)'

Show file-tree from the folder given, sorting file entries inside the top-level subfolders by file size, largest to smallest.

echo 'data/folder' | jf -from=text -to=ansi 'in.fs  .  map(v.unless(v.sort(-v)))'

Reshape headlines/data from live reuters feed.

jf -to=ansi 'in.rss.channel[0].item .
    map({title: v.title, link: v.link, when: v.pubDate })' \
"https://news.google.com/rss/search?q=when:12h+allinurl:reuters.com&ceid=US:en&hl=en-US&gl=US"

Reshape headlines/data from live reuters feed, and show it on a new browser tab using my other apps `vida` (VIew DAta) and `si` (Show It).

jf 'in.rss.channel[0].item .
    map({title: v.title, link: v.link, when: v.pubDate })' \
"https://news.google.com/rss/search?q=when:12h+allinurl:reuters.com&ceid=US:en&hl=en-US&gl=US" |
    vida -from=json -to=html | si

Get URI for JSON exchange rates from the Bank of Canada, since the start of the month: using shell substitution effectively prevents using the string-interpolation syntax, which would interfere with the shell's own substitution syntax.

echo https://www.bankofcanada.ca/valet/observations/group/FX_RATES_DAILY/json?\
start_date=`jf -from=null 'now.year + "-" + now.month.pad02 + "-01"'`

Get recent `live` exchange rates from the Bank of Canada, and transform values from the latest record. The shell-injected use of the current year is to get a URI for data since the beginning of the current year: this avoids downloading data for several years, thus speeding up things.

jf -to=ansi 'latest = in.observations.last;
    rates = {k-`FX`-`CAD`: +v.v: latest.drop(`d`)};
    { date: latest.d, rates.sort(k)... }' \
    https://www.bankofcanada.ca/valet/observations/group/FX_RATES_DAILY/json?start_date=`date +%Y`-01-01

Same as before, but show only 4 decimals, and make values (now strings) look green as they did before.

jf -to=ansi 'latest = in.observations.last;
    rates = {k-`FX`-`CAD`: +v.v: latest.drop(`d`)};
    { date: latest.d, rates.sort(k)@fix4@green... }' \
    https://www.bankofcanada.ca/valet/observations/group/FX_RATES_DAILY/json?start_date=`date +%Y`-01-01

Calculate the first 100 fibonacci numbers: using bigint values allows using different limits without worries, since float64 starts giving wrong results at around 80 items. If float64s were used instead, this algorithm would be constant-space and linear-time.

echo 100 | jf 'a = 1n; b = 0n; in.map(v, b = a+b; a = b; b)'

Calculate prime numbers using the algorithm with O(n**1.5) time-complexity.

echo 100 | jf 'in
.filter(n => n != 1 && n.sqrt.all(div => div == 1 || n % div != 0))
.map((k+1).nth, v)'

Calculate prime numbers in streaming mode.

seq 100 | jf -from=ev '
    (in != 1 && n.sqrt.all(div => div == 1 || n % div != 0)) ? in : skip
' | nl

A prime-numbers calculator as a complete self-running shell script.

#!/usr/bin/env -S jf -f
// find all prime numbers up to the input number given

if(type(in) == `number`, in, error(`input must be a number`)) .
filter(n => n != 1 && n.sqrt . every(div => n % div != 0 || div == 1))

Turn lines from paragraph blocks into objects, using NaN values as filler values, instead of the default null values.

seq 17 | awk 'NR%5 == 1 { print "" }; 1' | jf -from=blocks '
    keys = `one,two,three,four,five` / `,`;
    // turn each string-array into an object
    [keys ^ v@n: in]
    // use nan as the default missing value, replacing null
    .revalue(v ?? nan)'

Rename all files in folders called `a`, `b`, and `c`, so they all lose their original filename extensions.

jf -from=n -to=l '
    [`mv ${v.shquo} ${v.trimsuffix(v.ext).shquo}`: argv.files]' a/ b/ c/ | sh

Show all linux/mac search-path folders one per line.

echo $PATH | jf -from=t -to=l in/colon

Show all windows search-path folders one per line.

echo %PATH% | jf -from=t -to=l in/semi

Template Examples

These examples involve using option `-t` with the files whose contents are shown below, with no command, as all commands are in the template given, and with optional input files/URIs as usual.

Templates from HTML/SVG files automatically HTML/SVG escape jeff results as they're inserted into the output, avoiding injection tricks from potentially hostile data sources you may load.

Emit info based on the current time/date: this template is run with no input files/URIs.

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Simple HTML Template (<!--jf now.string -->)</title>
</head>

<body>
    <p>
        Right now it's
        <ul>
            <li><!--jf now[`hour`..`second`]@str@pad02.values() .join(`:`) --></li>
            <li><!--jf `${now.weekday[:3]}, ${now.day} ${now.monthname} ${now.year}` --></li>
        </ul>
    </p>
</body>

</html>

This one shows how to use a template loop, and is run with no input files/URIs.

<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Looped HTML Template (<!--jf now.string -->)</title>
</head>

<body>
    <ul>
    <!-- Normal HTML/SVG comments like these will end up in the result. -->
    <!--
        When template jeff expressions result in a skip value, nothing from
        the line they're on will be part of the output, which is handy when
        you need to assign values but not emit an empty line because of it.
    -->
    <!--jf max = 12; skip -->
    <!--for key, val := 1..max -->
        <!--for name := `abc` -->
        <li><!--jf name -->: <!--jf key -->: <!--jf val --></li>
        <!--end -->
    <!--end -->
    </ul>
</body>

</html>

This example emits a partial ASCII table, and is run with no input files/URIs.

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>ASCII Table</title>
    <style>
        body {
            font-family: monospace;
        }

        .c {
            color: darkmagenta;
        }

        .n {
            color: dodgerblue;
            margin-right: 2rem;
        }

    </style>
</head>

<body>
    <!--jf rows = (space+visascii).runes.chunk(12).tr(); skip -->
    <table>
        <tbody>
            <!--for r := rows -->
            <tr>
                <!--for v := r -->
                <td><span class="c"><!--jf v --></span>: <span class="n"><!--jf v == nil ? skip : v.ord --></span></td>
                <!--end -->
            </tr>
            <!--end -->
        </tbody>
    </table>
</body>

</html>

Coming from Excel

Coming from SQL

Coming from JavaScript

Coming from Python

Function Names

A recent run of the following command results in the table below.

jf -from=null -to=html 'funcs().toentries().map({name: v[0], type: v[1]})'