Skip to content

Instantly share code, notes, and snippets.

@jsyi
Last active August 6, 2024 08:47
Show Gist options
  • Save jsyi/9d2d191c82ce0c1e32052476d90cc047 to your computer and use it in GitHub Desktop.
Save jsyi/9d2d191c82ce0c1e32052476d90cc047 to your computer and use it in GitHub Desktop.
Use AWS CLI to query AWS CloudWatch Logs and determine the most recent entry into a Log Group
aws logs describe-log-groups | jq ".logGroups[].logGroupName" | grep -E "homeplus|mcdonalds" | xargs -n 1 -t aws logs describe-log-streams --query "logStreams[*].lastEventTimestamp" --log-group | jq "max/1000|floor" | xargs -t -n 1 date -r
# aws logs describe-log-groups | jq ".logGroups[].logGroupName"
# Get LogGroup names
# grep -E "homeplus|mcdonalds"
# (Optional) Filter LogGroup names
# xargs -n 1 -t aws logs describe-log-streams --query "logStreams[*].lastEventTimestamp" --log-group | jq "max/1000|floor"
# Get last event timestamp for each LogGroup.
# (Optional) Convert milliseconds timestamp to seconds timestamp.
# xargs -t -n 1 date -r
# (MacOS X) Format seconds timestamp into human-readable format.
# (Other) Change `date -r` to `date -d` or other option appropriate to the installation of `date` included with your OS
# BONUS: After verifying LogGroups as old/unused delete them
# aws logs describe-log-groups | jq ".logGroups[].logGroupName" | grep -E "homeplus|mcdonalds" | grep -v "mcdonalds-v1-production" | xargs -n 1 -p aws logs delete-log-group --log-group-name
# grep -v "mcdonalds-v1-production"
# (Optional) Filter rule to excluded any LogGroups that should be kept.
# xargs -n 1 -p aws logs delete-log-group --log-group-name
# The -p option specifies that xargs should prompt for user confirmation before each command execution.
@richcarrot
Copy link

Thank you for making gist this public. I found your jq query very useful.

Here's the query I ended up with. It generates a CSV that has the name of each log group and the date of its latest entry. Just providing it in case you or someone else may find it useful.

aws logs describe-log-groups
| jq ".logGroups[].logGroupName" 
| xargs -n 1 -t aws logs describe-log-streams --query "logStreams[*].[arn,lastEventTimestamp]" --descending --max-items 1  --log-group-name 
| jq '
    .[]
    | {
         arn: (.[0] | capture("log-group:(?<loggroup>.*):log-stream").loggroup), 
         timestamp: (.[1] | . /1000 | strftime("%Y-%m-%d"))
    }
    | [.arn, .timestamp] 
    | @csv'
> log-groups-most-recent-event.csv     

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment