awk process columns of text
awk splits each line into fields ($1, $2, …) and runs a short program on it. It is ideal for picking columns, filtering on a field value, and quick sums, especially on whitespace-separated output such as logs.
Availability Preinstalled on macOS, Linux and BSD (GNU awk is called gawk)
Syntax
awk [-F separator] 'pattern { action }' [file ...]Common options
| Option | What it does |
|---|---|
-F sep | Field separator, for example -F, for commas or -F'\t' for tabs |
-v name=value | Pass a variable into the program |
$1, $2 … $NF | Fields of the current line. $0 is the whole line, $NF the last field |
NR | Current line number |
BEGIN { } / END { } | Run code before the first line or after the last |
Examples
Print the second column
awk '{print $2}' file.txtLines where field 3 is over 100
awk '$3 > 100' metrics.txtSum a CSV column, skipping the header
awk -F, 'NR > 1 {total += $3} END {print total}' sales.csvUse awk with sqx
Turn a web server log into a table
In the common log format, $1 is the client IP and $9 the status code. awk pulls them out; sqx groups and sorts.
awk '{print $1","$9}' access.log | sqx -no-header "SELECT c1 AS ip, COUNT(*) AS errors FROM stdin WHERE c2 >= 400 GROUP BY c1 ORDER BY errors DESC LIMIT 10"awk -F, does not understand quoted CSV fields such as "Rao, Asha". For real CSV files, load them with sqx, which parses quotes correctly.
Full documentation
This page covers the essentials. For every option, see the official documentation:
Or read the manual in your terminal: man awk