How sqx works

sqx rests on three ideas. Once you know them, every command and flag makes sense.

1. Your data becomes tables

sqx starts an in-memory SQLite database, loads your input into it, runs your SQL and prints the result.

  • Data piped into sqx becomes a table called stdin. Rename it with -t.
  • Each -f name=path becomes a table called name. With just -f path, the table is named after the file.
  • The format is detected automatically: JSON array, JSON Lines, TSV, a plain list of URLs or CSV.
  • Column names are cleaned up and column types are inferred, so WHERE latency_ms > 300 compares numbers.

Because it is SQLite, you get the full SQL language: joins, GROUP BY, window functions, CTEs, json_extract, json_each and more. sqx adds its own functions on top.

Read more in Load CSV, JSON and text.

2. Fetch once, query as often as you like

Every web function (http_*, html_*, dns_*) shares a cache on disk. However many functions you call on one URL, it is downloaded once, and later runs read it from the cache without touching the network.

That means you can refine a query twenty times while exploring a website, and only the first run waits for the network.

You want to…Use
Work without a network-offline
Refresh everything-force
Treat entries older than an hour as stale-cache-ttl 1h
Keep a separate cache per project-cache-db ./project-cache.db
Warm the cache for many URLs quicklysqx prefetch -c 25

Read more in The web cache.

3. Temporary by default, saved when you ask

  • A plain query runs in memory, and its tables disappear when the command ends.
  • sqx import -db project.db … saves files, stdin or URLs into a SQLite database file.
  • -db project.db -save <table> "<SQL>" saves a query result. Running it again replaces the table.
  • Tables from stdin and -f are never written into -db, so repeating a command never duplicates rows.

Read more in Save to a SQLite database.

Unix-friendly by design

sqx reads stdin and writes stdout, so it fits into pipelines. Output is a readable table in a terminal and CSV when piped, which means you can chain sqx into itself, into grep, xargs or any other tool. Errors go to stderr and the exit code is non-zero on failure.