Eric E. Intelrunner

Architecting my way out of cloud disaster. I'm a neophyte dev, lifelong data junkie, and analyst by trade. Also - I have ducks, and they are just odd.

3 followers · 7 following

DoiT International
OH
www.linkedin.com/in/ericethridge

View GitHub Profile

Recently created

Least recently created

Recently updated

Least recently updated

Intelrunner / supported_services.json

Last active August 21, 2024 13:41

For Google Cloud: A JSON list of all supported services that can be used in "Restricting resource usage" in Org Policies

	[
	"accessapproval.googleapis.com",
	"aiplatform.googleapis.com",
	"alloydb.googleapis.com",
	"analyticshub.googleapis.com",
	"anthosedge.googleapis.com",
	"apigateway.googleapis.com",
	"apigee.googleapis.com",
	"apigeeconnect.googleapis.com",
	"apigeeregistry.googleapis.com",

Intelrunner / bq_recommender.sh

Created August 19, 2024 16:30

To provide $USER view access to your BQ Admin Project ($PROJECT_ID) for reviewing Recommendation Engine reccs. View Only.

	# Ensure these environment variables are set before running the script:
	# export PROJECT_ID=<your-project-id>
	# export USER_EMAIL=<user-email>
	# export EXPIRATION_DATE=<expiration-date> # Format: YYYY-MM-DD

	# Creates the custom role for BigQuery Reservations Viewer
	gcloud iam roles create bigqueryReservationsViewer \
	--project="$PROJECT_ID" \
	--title="BigQuery Reservations Viewer" \
	--permissions="bigquery.reservations.list,bigquery.reservationAssignments.list,bigquery.capacityCommitments.list" \

Intelrunner / big-ole-file.py

Last active August 6, 2024 20:36

Builds a 1.3GB file full of random values. Used for testing transfer speeds. Update the # of rows to make a even chonkier file.

	# This is not an original work, but crafted based on: https://gist.github.com/momota/ba302f0f0720ff5b2445fb81820c5b82
	# I updated it to make a file closer to the size I needed consistantly. All praise goes to: @momota and @andrewFarley for
	# The original gist.

	import csv
	import random

	# 1000000 and 62 == roughly 1.3GB (will take a bit of time, go get a coffee)
	rows = 1200000
	columns = 62

Intelrunner / set_crontab.sh

Last active July 29, 2024 17:14

Set a crontab to run a script every 15 min

	#!/bin/bash

	# Create the crontab entry
	(crontab -l 2>/dev/null; echo "/5 * * * source /var/user-scripts/backup_script.sh") \| crontab -

Intelrunner / mongodb_backup.sh

Last active July 29, 2024 04:53

mongodb_backup_setup

	#!/bin/bash

	# Check to see if the backup directory exists
	# If not, create it
	if [ ! -d "/var/mongobackup" ]; then
	sudo mkdir /var/mongobackup
	fi
	sudo chmod 777 /var/mongobackup

	mongodb_pwd=$(curl "http://metadata.google.internal/computeMetadata/v1/instance/attributes/MONGODB_PWD" -H "Metadata-Flavor: Google")

Intelrunner / mongod.conf

Last active July 28, 2024 19:35

mongod.conf - base from 6.0 w/auth enabled for pull on startup

	# Where and how to store data.
	storage:
	dbPath: /var/lib/mongodb
	# engine:
	# wiredTiger:

	# where to write logging data.
	systemLog:
	destination: file
	logAppend: true

Intelrunner / google_cloud_storage_soft_delete_set.py

Created June 4, 2024 13:55

Update all buckets in project to X soft_delete_duration (Google Cloud)

	from google.cloud import storage
	import os
	def update_soft_delete():
	"""Lists all buckets in a project."""

	storage_client = storage.Client()
	buckets = storage_client.list_buckets()

	for bucket in buckets:
	print ("****************")

Intelrunner / convert_pdf.py

Last active May 8, 2024 13:28

This will do a quick rip and save of all tables inside of a PDF. It is not pretty but it does work. Ignore the errors about jpye.

	"""
	Instructions
	- Download to folder
	- > Run "pip3 freeze > requirements.txt"
	- > Run pip3 "install -r requirements.txt"
	- > run python3 -m main.py
	- Enjoy
	"""

	from tabula import convert_into, read_pdf

Intelrunner / soft_delete_clear.py

Last active April 24, 2024 16:45

Loops through all projects a user has available in gcloud and updates all of their buckets to clear the soft delete policy

	import subprocess

	# Function to run shell commands
	def run_command(command):
	try:
	output = subprocess.check_output(command, shell=True, text=True)
	return output.strip() # Return output with whitespace stripped
	except subprocess.CalledProcessError as e:
	print(f"Command failed with error: {e}")
	return None

Intelrunner / bq_check.sh

Created March 5, 2024 14:22

Check all projects in a GCP org to see if an API is enabled.

	#!/bin/bash

	# The Organization ID (Replace with your organization's ID)
	ORG_ID="your-organization-id"

	# Gather all projects under the specified organization
	PROJECT_IDS=$(gcloud projects list --organization=${ORG_ID} --format="value(projectId)")

	# Loop through each project
	for PROJECT_ID in ${PROJECT_IDS}; do

NewerOlder