Linux Text Processing

cat, head, cut, sort, uniq, and awk for daily data shaping.

  • awk
  • cut sort uniq
  • head tail
  • text processing
  • csv cli

Linux Text Processing

cat, head, cut, sort, uniq, and awk for daily data shaping.

Commands

CommandWhat it does
cat -n fileNumbered full output
head -n 20 and tail -n 20First and last lines
cut -d: -f1 /etc/passwdFirst colon field
`sort fileuniq -csort -nr`Frequency table
awk -F, print column 2Second CSV column

Example

sort access.log | uniq -c | sort -nr | head

FAQ

Huge file - should I cat it?

No: page with less, sample with head, tail, or shuf, or stream through awk without loading all.

CSV with quoted commas?

cut and awk break - use csvkit or Miller for real CSV parsing.

常見問題

Huge file - should I cat it?
No: page with less, sample with head, tail, or shuf, or stream through awk without loading all.
CSV with quoted commas?
cut and awk break - use csvkit or Miller for real CSV parsing.

Linux