Skip to content

Instantly share code, notes, and snippets.

View matwerber1's full-sized avatar

Mathew Werber matwerber1

  • Seattle
View GitHub Profile
@matwerber1
matwerber1 / readme.md
Created December 13, 2023 08:01
Querying AWS Cost and Usage V2 (CURv2) with Amazon Athena - DDL and Queries

DDL for Athena Table

This could be improved, but does the trick for now. This is for a report configured as "daily" with resource IDs:

CREATE EXTERNAL TABLE IF NOT EXISTS `org_data`.`curv2` (
  `discount_bundled_discount` string,
  `discount_total_discount` string,
  `identity_line_item_id` string,
  `identity_time_interval` string,
@matwerber1
matwerber1 / aws-profile
Created November 27, 2023 05:46
Super Duper AWS CLI Profile Switching Script
#! /bin/bash -i
set -e
# Constants
PROFILE_NICKNAME_FILE=".awsprofile.env"
AWS_CONFIG_FILE="$AWS_CONFIG_FILE"
# Variables
original_request="$1"
requested_profile=""
@matwerber1
matwerber1 / compose.yml
Last active November 9, 2023 18:16
AWS - Containers - Compose - mount ~/.aws/cred
version: "3.8"
services:
AwsCLI:
image: public.ecr.aws/aws-cli/aws-cli
container_name: aws-cli
platform: linux/x86_64
volumes:
- ~/.aws/:~/.aws/ # Mount local credentials into container to use AWS CLI/SDK
environment:

CloudTrail Lake Snippets

CloudTrail Lake supports Presto SQL. This doc focuses less on SQL itself and more on snippets specific to CloudTrail Lake schema.

Example Query

This query ties together key concepts and, in one shape or another, is often the starting point for my analyses of events:

SELECT
@matwerber1
matwerber1 / aws-cli-eksctl-create-iamidentitymapping
Last active October 13, 2023 04:06
Snippet of Amazon EKS aws-auth configmap that grants cluster admin permissions to a specific AWS SSO role/permission set
# Example command
eksctl create iamidentitymapping \
--cluster YOUR_CLUSTER_NAME \
--arn arn:aws:iam::999999999999:role/AWSReservedSSO_YOUR-ROLE_NAME_xxxxxxxxxxxc \
--username cluster-admin \
--group system:masters
@matwerber1
matwerber1 / gist:48ecc538ce03ce3d1708121bd34eba80
Last active October 17, 2023 20:32
Cleaning up my gists, found this. Not sure if it works, but making it public in case it helps.
/* eslint-disable no-prototype-builtins */
var express = require('express');
var ip = require("ip");
var fetchTimeout = require('fetch-timeout');
/*
This app creates a simple HTTP listener and responds to GET / requests by
displaying basic info about the container (e.g. IP address) and, optionally,
ECS metadata if the container is running on EC2 or Fargate via ECS.
@matwerber1
matwerber1 / requirements.txt
Created September 22, 2023 01:01
cloudmapper-python-requirements-for-m1-arm
astroid==2.15.6
autoflake==2.2.1
autopep8==2.0.4
boto3==1.28.52
botocore==1.31.52
certifi==2023.7.22
chardet==5.2.0
charset-normalizer==3.2.0
contourpy==1.1.1
coverage==7.3.1
@matwerber1
matwerber1 / javascript-regex-for-aws.md
Last active October 13, 2023 04:16
Javascript regex patterns for AWS

AWS Resource Regex Patterns

AWS Backup

These were extracted from the open source AWS CLI code on GitHub for AWS Backup:

ruleName: /^[a-zA-Z0-9\-_\.]{1,50}$/,
@matwerber1
matwerber1 / bigquery-sql-for-gmail-logs.sql
Last active September 9, 2024 16:06
Example SQL for Gmail Logs data written to BigQuery daily_ table
-- I was helping a company using Google Workspace for email, and they were having challenges
-- with email deliverability due to domain reputation. While it's true that gmail provides
-- provides detailed logs, I found their docs to be only marginaly helpful and lacking
-- more robust, real-world ways to interpret the data. I also struggled to find quality examples
-- when searching unofficial sources (blogs, etc.).
--
-- The query below is my best effort attempt to translate gmail logs into something half-way usable.
-- I'd be shocked if there weren't mistakes or areas for improvement, but either way, sharing in
-- the hope that it at least helps give a jumping-off point for anyone else that finds themselves
-- saying "OK, I enabled Gmail Logs for BigQuery.... now what?".
@matwerber1
matwerber1 / aws-cli-ecs-exec-with-fargate.sh
Last active October 13, 2023 04:18
Example to start an ECS Fargate task an existing cluster, task definition, and VPC, and then use ECS Exec to open an interactive shell with a containr in the task
CLUSTER="your_cluster_name"
REGION="cluster_region"
PRIVATE_SUBNETS="subnet-someSubnet123,subnet-someOtherSubnet456"
TASK_SECURITY_GROUP="security-group-for-ECS-task-ID"
PUBLIC_IP_SETTING="DISABLED"
TASK_CONTAINER_NAME_FOR_ECS_EXEC="container-name-for-ecs-exec"
# Run a Fargate task with ECS Exec enabled:
RUN_TASK_RESULT=$(
aws ecs run-task \