SQL for Unix streams and the web.

cat users.csv | sqx "
  SELECT country, COUNT(*) AS customers
  FROM stdin GROUP BY country"

+---------+-----------+
| country | customers |
+---------+-----------+
| CN      | 1         |
| IN      | 2         |
| MX      | 1         |
| TR      | 1         |
| US      | 1         |
+---------+-----------+

Data in, SQL in the middle, answers out

Pipe in whatever you have. sqx loads it into an in-memory SQLite database, runs your query, and prints the result in the format you need.

You pipe in

• CSV
• TSV
• JSON
• JSON Lines
• logs
• URLs
• HTML

sqx SELECT … FROM stdin

You get back

• table
• CSV
• JSON
• Markdown
• raw values
• SQLite file

Why sqx?

Joining two CSV files, checking which of 500 URLs are broken, or finding an API key leaked in a page should not need a database server, credentials or a throwaway Python script. With sqx, you pipe data in and write the SQL you already know.

Three things you can do in one line

Join a CSV with a JSON Lines file

sqx -f users=users.csv -f events=events.jsonl "
  SELECT u.name, COUNT(e.id) AS events, ROUND(SUM(e.amount), 2) AS revenue
  FROM users u LEFT JOIN events e ON e.user_id = u.id
  GROUP BY u.id ORDER BY revenue DESC"

Audit a list of URLs

cat urls.txt | sqx -i lines "
  SELECT url, http_status(url) AS status, http_final_url(url) AS final_url, html_title(url) AS title
  FROM stdin"

Check SPF and DMARC for a list of domains

cat companies.csv | sqx "
  SELECT domain,
         CASE WHEN dns_txt_json(domain) LIKE '%v=spf1%' THEN 'yes' ELSE 'MISSING' END AS spf,
         CASE WHEN dns_txt_json('_dmarc.' || domain) LIKE '%v=DMARC1%' THEN 'yes' ELSE 'MISSING' END AS dmarc
  FROM stdin"

More in the cookbook.

Works with the tools you already know

sqx is one more command in your pipeline. Our short guides cover the classic Unix tools it pairs with, with everyday options, examples and links to the full manuals.

Browse all Unix tool guides →

Documentation

Frequently asked questions