Skip to content

Instantly share code, notes, and snippets.

View ottokruse's full-sized avatar

Otto Kruse ottokruse

  • AWS
View GitHub Profile
@ottokruse
ottokruse / download_log_stream.sh
Created March 27, 2024 12:29
Download full AWS CloudWatch log stream
#!/bin/bash -e
log_group_name="/aws/sagemaker/TrainingJobs"
log_stream_name=${1}
output_file=$(echo "${1}.txt" | sed 's/\//_/g')
next_token=""
echo "Storing full logs to: $output_file"
# Clear output file
@ottokruse
ottokruse / cdk-snippet.ts
Created May 25, 2022 07:55
Create an AWS Lambda Layer that includes pandas and pyarrow
// Creating an AWS Lambda Layer with pandas and pyarrow is harder than it might seem,
// as simply `pip install pandas pyarrow` will lead to a deployment package that is > 250 MB
// which is not allowed by AWS Lambda.
// In this snippet, that deployment package is trimmed down, to make it fit (and still work)
import * as lambda from "aws-cdk-lib/aws-lambda";
const layerInstallCommand = [
"bash",
"-c",
@ottokruse
ottokruse / main.py
Created March 25, 2022 16:17
Trade a Cognito User Pool JWT for AWS credentials with a Cognito Identity Pool
import boto3
session = boto3.Session()
cognito_client = session.client("cognito-identity")
id_token = "<jwt>"
identity_response = cognito_client.get_id(
IdentityPoolId="<identity pool id>",
Logins={"cognito-idp.<region>.amazonaws.com/<user pool id>": id_token}, # Only need to provide this here as well, if the Identity Pool doesn't allow unauthenticated identities
@ottokruse
ottokruse / index.ts
Last active February 10, 2022 12:19
TypeScript: create new type that overrides types of some fields of base type
/**
* Let's say our base type is `Person` and we want to create a type like it,
* where some fields are encrypted into a Buffer instead
*/
type Encrypted<Base, EncryptedFields extends keyof Base> = {
[field in keyof Base]: field extends EncryptedFields ? Buffer : Base[field];
};
interface Person {
@ottokruse
ottokruse / buildspec.yaml
Created January 11, 2022 13:34
AWS CLI assume role in AWS CodeBuild through ECS credentials
version: '0.2'
phases:
build:
commands:
- |
cat << EOF > aws_config
[profile test]
role_arn = arn:aws:iam::123456789012:role/my-role-to-assume
role_session_name = IntegrationTest
credential_source = EcsContainer
@ottokruse
ottokruse / debounce.ts
Last active April 21, 2022 13:48
TypeScript debounce that keeps argument types of debounced function
export function debounce<T extends (...args: any[]) => void>(
func: T,
timeout = 300
) {
let timer: ReturnType<typeof setTimeout>;
return (...args: Parameters<T>) => {
if (timer) clearTimeout(timer);
timer = setTimeout(() => func(...args), timeout);
};
}
@ottokruse
ottokruse / cw-metric.ts
Last active March 3, 2021 08:37
Simple AWS Lambda logger that uses CloudWatch Embedded Metric Format to create custom CloudWatch metrics (TypeScript)
// https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html
export function logCustomMetric(metric: {
dimensions: {
[key: string]: string;
};
namespace: string;
metrics: {
unit: Unit;
name: string;
@ottokruse
ottokruse / package_lambda.py
Created October 12, 2020 10:44
Package Lambda functions and layers with CDK bundling, use cache to prevent unnecessary docker runs
from typing import List
import os
import shutil
import hashlib
import tempfile
from aws_cdk import core, aws_lambda
import jsii
from pathlib import Path
@ottokruse
ottokruse / verify-jwt-rs256.ts
Last active July 29, 2024 01:29
Verify JWT signature, for RS256 in NodeJS (code in TypeScript). Only needs public key modulus and exponent (i.e. not as PEM/DER)
/*
This gist's core function is verifyJwt, whose purpose is to verify JWT's signed
using RS256
The public key needs to be provided as n (modulus) and e (exponent).
JWT algorithm RS256 in fact means RSASSA-PKCS1-v1_5 using SHA-256:
https://tools.ietf.org/html/rfc7518#section-3.1
The specification of RSASSA-PKCS1-v1_5 specifies the steps to verify signatures:
@ottokruse
ottokruse / invoke-aws-api-gateway-with-cognito-client-credentials.py
Last active August 11, 2020 11:22
Invoke an AWS API Gateway Endpoint using Cognito Client Credentials
"""
This example does a HTTP POST, change it to whatever you want.
The code here assumes that:
- Your API Gateway has been setup with a Cognito User Pool authorizer
- That User Pool authorizer is set up with http header "authorization" as Token Source
- The API Gateway Method you are invoking is set up to require at least one OAuth scope
"""
from urllib.request import Request, urlopen