Skip to content

Instantly share code, notes, and snippets.

@vicenterusso
Created October 27, 2022 14:03
Show Gist options
  • Save vicenterusso/6b119d99203acd79b1de4fa2ec05d2f5 to your computer and use it in GitHub Desktop.
Save vicenterusso/6b119d99203acd79b1de4fa2ec05d2f5 to your computer and use it in GitHub Desktop.
Nginx Log Analizer - Colorizer - Grep, Awk, Cut, Sort, Uniq, Ccze

To be honest, I use grep, awk, cut, sort, uniq and 'ccze -a' to colorize output from the commandline way more than anything else. It's worth learning them and key options - it will pay off many orders of magnitude.

A few quick commandline examples:

case insensitive match 'error' in nginx access.log and colorize output

grep -i error /var/log/nginx/access.log |ccze -A

how many 404 errors today?

grep "" 404 " /var/log/nginx/access.log |grep "18/Jul/2020" |wc -l

what caused 404 errors, how many times did each one happen, and sort based on # times, and colorize

grep "" 404 " /var/log/nginx/access.log |grep "18/Jul/2020" |cut -d " -f 2 |sort |uniq -c |sort -rh |ccze -A

and to answer your examples:

Total # of http requests

grep "18/Jul/2020" /var/log/nginx/access.log |wc -l

Total # of http requests that generated a 200 response code:

grep "18/Jul/2020" /var/log/nginx/access.log |grep "" 200 " |wc -l

total number of unique IPs for today's requests (if you're behind a firewall or vpc, your webserver needs to be configured properly to forward customer IPs, and customers could be behind firewalls too, so... don't put too much weight on this stat

grep "18/Jul/2020" /var/log/nginx/access.log |awk '{print $1}' |sort -u |wc -l

unique IPs today sorted by # of requests

grep "18/Jul/2020" /var/log/nginx/access.log |awk '{print $1}' |sort |uniq -c |sort -rh

urls of today's requests - do you mean the referrer url or the request?

top 20 referrer urls for today:

grep "18/Jul/2020" /var/log/nginx/access.log |cut -d " -f 4 |sort |uniq -c |sort -rh |head -20

top 20 webserver requests for today

grep "18/Jul/2020" /var/log/nginx/access.log |cut -d " -f 2 |sort |uniq -c |sort -rh |head -20


Ref (https://www.reddit.com/r/nginx/comments/htbhg2/comment/fyhzi5q/?utm_source=share&utm_medium=web2x&context=3)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment