While reading Masterminds of Programming I came across the interview with Alfred Aho and his thoughts about the design of the AWK programming language. I’ve always known about the existence of awk, but have never really felt the urge to dive in. Then I came across this sentence:
If I had to choose a word to describe our centering forces in language design, I’d say Kernighan emphasized ease of learning; Weiberger, soundness of implementation; and I, utility. I think AWK has all three of these properties.
This got me curious enough to put down the book and start looking through a few AWK tutorials. After mucking around with it for a day or two, I am really glad that I did so. AWK is a simple yet powerful language to do some quick data processing for which languages like Python or Ruby may be an overkill.
While learning, I’ve collected a few one-liners together, as a cheatsheat for myself.
Remove duplicate lines from a file
awk '!a[$0]++' data.txt
Print account name and role from /etc/passwd
awk -F : '/^[^(#|_)]/ {print $1, "=>", $5}' /etc/passwd
Prefixing line with increasing number
awk '{print FNR ". " $0}' data1.txt
Find sum of columns for each row
awk '{s=0; for (i=1; i<=NF; i++) s=s+$i; print s}' data1.txt
Find the line containing largest value in the first column
awk '$1 > max {max=$1; maxline=$0}; END{ print maxline}' data1.txt
Find and replace in file
# Replace only first instance in line
awk '{sub(/HTML/, "html")}; 1' data1.txt > data2.txt
# Replace all instances in line
awk '{gsub(/HTML/, "html")}; 1' data1.txt > data2.txt
Swap first two columns of every line
awk '{temp=$1; $1=$2; $2=temp} 1' data2.txt
Delete second column from every line
awk '{ $2 = ""; print }' data2.txt
Perform search similar to SQL ‘where’
awk '$2 == "hello"' data2.txt