Skip to content

Instantly share code, notes, and snippets.

View zwolf's full-sized avatar

Zach Wolfenbarger zwolf

View GitHub Profile
@camallen
camallen / unretire_workflow_subjects.rb
Created September 25, 2020 13:07
Unretire subects for a workflow
workflow_ids = []
subject_set_ids = []
workflow_ids.map do |wf_id|
subject_set_ids.map do |set_id|
set_subject_ids = SetMemberSubject.where(subject_set_id: set_id).pluck(:subject_id)
# UNCOMMMENT ONE OF THESE, I DO A SANITY CHECK COUNT FIRST, THEN UNRETIRE
# run a count for sanity check
# SubjectWorkflowStatus.where.not(retired_at: nil).where(workflow_id: wf_id, subject_id: set_subject_ids).count
# RUN THE UNRETIRE
SubjectWorkflowStatus.where.not(retired_at: nil).where(workflow_id: wf_id, subject_id: set_subject_ids).update_all(retired_at: nil, retirement_reason: nil)
@amy-langley
amy-langley / move_small.rb
Created August 14, 2019 16:38
like move_internal.rb, but does not loop, just finds the first few and quits immediately. it needs to be fast so that it moves things before reduce workers can pick them up.
# this assumes the rails console has preloaded the resources; accordingly,
# you should invoke this script as
#
# rails c < move_small.rb
begin
args_list = []
jobs_list = []
puts 'finding jobs'
Sidekiq::Queue.new('internal').each do |job|
@amy-langley
amy-langley / move_internal.rb
Created August 14, 2019 16:37
A script to move all TESS reduce worker tasks from the `internal` queue to the `holding` queue
# this assumes the rails console has preloaded the resources; accordingly,
# you should invoke this script as
#
# rails c < move_internal.rb
begin
args_list = []
jobs_list = []
puts 'finding jobs'
Sidekiq::Queue.new('internal').each do |job|
@adammcmaster
adammcmaster / org_workflow_stats.py
Created May 3, 2019 11:36
Print workflow stats for an organisation
"""
Requires panoptes-client and (on Python 2) futures.
pip install panoptes-client futures
"""
from panoptes_client import Panoptes, Organization
ORG_ID = 16
Panoptes.connect(
username='',
@camallen
camallen / clear_sidekiq_locks_redis_cli.sh
Created November 2, 2018 13:37
Clear out sidekiq unique & congestion locks from redis
# ssh onto the redis node in question and run these cmds
# congestion:* keys are for https://github.com/parrish/Sidekiq-Congestion
# uniquejobs:* https://github.com/mhenrixon/sidekiq-unique-jobs
# get a count of the locks
redis-cli -n 0 --scan --pattern 'congestion:*' | wc -l
redis-cli -n 0 --scan --pattern 'uniquejobs:*' | wc -l
# clear them for both key namespaces
@camallen
camallen / test_file_mime_type.py
Created September 21, 2018 08:17
Python magic lib test file mime types
# https://github.com/ahupp/python-magic#usage
import magic, csv
file_paths = (
'480_CornellFeeders_20171024_0921_000.mp4',
'480_CornellFeeders_20171024_0921_000.mp4',
'480_CornellFeeders_20171024_0921_001.mp4',
'480_CornellFeeders_20171024_0921_002.mp4'
)
@camallen
camallen / project_classifications_csv_dump_export.rb
Last active June 18, 2019 10:25
Manual classification csv exports for a panoptes project
# Manual csv classifications dump
# ensure the config/database.yml is configured to use the read replica database and not the production db.
#
# run via rails runner from the panoptes cmd line via
# rails r project_classifications_csv_dump_export.rb
require 'csv'
PROJECT_ID = 1
@dleske
dleske / k8s-update-secret.md
Last active January 29, 2024 17:12
k8s: Updating a Secret

Hopefully helped another k8s newbie with the following. The question was, how do you update a single key in a secret in k8s? I don't know anything about secrets but I will probably want to know this in the future, so here we go.

First, to create a dummy secret:

apiVersion: v1
kind: Secret
metadata:
  name: test-secret
data:
@camallen
camallen / find_database_relation_sized.sql
Created November 21, 2017 13:27
List top 10 table sizes and report the index usage
select relation, pg_size_pretty(total_size), pg_size_pretty(size), pg_size_pretty(total_size - size) as index_size from
(SELECT relname AS "relation", pg_total_relation_size(C.oid) AS "total_size", pg_relation_size(C.oid) AS "size"
FROM pg_class C LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
WHERE nspname NOT IN ('pg_catalog', 'information_schema')
ORDER BY pg_relation_size(C.oid) DESC
) as derived
LIMIT 10;
@troyharvey
troyharvey / deployment.yml
Last active September 13, 2024 19:00
Using Kubernetes envFrom for environment variables
# Use envFrom to load Secrets and ConfigMaps into environment variables
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: mans-not-hot
labels:
app: mans-not-hot
spec:
replicas: 1