Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save tanandy/2190af8446a093d4e91ccf6580316e7b to your computer and use it in GitHub Desktop.
Save tanandy/2190af8446a093d4e91ccf6580316e7b to your computer and use it in GitHub Desktop.
Nginx Log Analizer - Colorizer - Grep, Awk, Cut, Sort, Uniq, Ccze

To be honest, I use grep, awk, cut, sort, uniq and 'ccze -a' to colorize output from the commandline way more than anything else. It's worth learning them and key options - it will pay off many orders of magnitude.

A few quick commandline examples:

case insensitive match 'error' in nginx access.log and colorize output

grep -i error /var/log/nginx/access.log |ccze -A

how many 404 errors today?

grep "" 404 " /var/log/nginx/access.log |grep "18/Jul/2020" |wc -l

what caused 404 errors, how many times did each one happen, and sort based on # times, and colorize

grep "" 404 " /var/log/nginx/access.log |grep "18/Jul/2020" |cut -d " -f 2 |sort |uniq -c |sort -rh |ccze -A

and to answer your examples:

Total # of http requests

grep "18/Jul/2020" /var/log/nginx/access.log |wc -l

Total # of http requests that generated a 200 response code:

grep "18/Jul/2020" /var/log/nginx/access.log |grep "" 200 " |wc -l

total number of unique IPs for today's requests (if you're behind a firewall or vpc, your webserver needs to be configured properly to forward customer IPs, and customers could be behind firewalls too, so... don't put too much weight on this stat

grep "18/Jul/2020" /var/log/nginx/access.log |awk '{print $1}' |sort -u |wc -l

unique IPs today sorted by # of requests

grep "18/Jul/2020" /var/log/nginx/access.log |awk '{print $1}' |sort |uniq -c |sort -rh

urls of today's requests - do you mean the referrer url or the request?

top 20 referrer urls for today:

grep "18/Jul/2020" /var/log/nginx/access.log |cut -d " -f 4 |sort |uniq -c |sort -rh |head -20

top 20 webserver requests for today

grep "18/Jul/2020" /var/log/nginx/access.log |cut -d " -f 2 |sort |uniq -c |sort -rh |head -20


Ref (https://www.reddit.com/r/nginx/comments/htbhg2/comment/fyhzi5q/?utm_source=share&utm_medium=web2x&context=3)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment