Skip to content

Instantly share code, notes, and snippets.

@fgregg
fgregg / randomize_reviewers.py
Last active July 24, 2024 23:37
Randomize Reviewers
import csv
import itertools
import math
import random
import string
import sys
import click
from pulp import PULP_CBC_CMD, LpBinary, LpMaximize, LpProblem, LpVariable, lpSum, value
@fgregg
fgregg / census-api-proxy.js
Created April 1, 2022 00:58
Code for Cloudflare Worker to be Census API Proxy
addEventListener("fetch", (event) => {
event.respondWith(
handleRequest(event).catch(
(err) => new Response(err.stack, { status: 500 })
)
);
});
@fgregg
fgregg / illinois_library_service_areas.geojson
Created January 16, 2022 02:23
Geojson for Illinois Library Service Areas
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@fgregg
fgregg / cyborg_scrape.py
Last active January 26, 2021 14:18
cyborg pattern
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
with webdriver.Chrome() as driver:
# load the page
driver.get("https://dapps.36thdistrictcourt.org/ROAWEBINQ/Default.aspx")
@fgregg
fgregg / locator.md
Last active April 16, 2020 21:16
notes on inmate locator

should be able to scrape changing bookingNumber in the data payload.

For Justin Walker:

curl 'https://inmatelocator.ccsheriff.org/InmateLocator/Details'
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:75.0) Gecko/20100101 Firefox/75.0'
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'
-H 'Accept-Language: en-US,en;q=0.5'
--compressed
@fgregg
fgregg / copy_from_generator.py
Last active March 4, 2020 21:47
Copy from generator
import psycopg2
import io
import csv
import itertools
class Readable(object):
def __init__(self, iterator):
self.output = io.StringIO()
@fgregg
fgregg / batch.py
Created February 25, 2020 22:48
offset batch
def batched_result_generator(cur):
offset = 0
batchsize = 100
exhausted = False
while not exhausted:
results = cur.execute('select ... from offset = ? limit = ?', (offset, batchsize)).fetchall()
if len(results) < batchsize:
exhausted = True
for row in results:
yield row
@fgregg
fgregg / app_ideas.md
Created February 20, 2020 19:00
app ideas
  • councilmatic for ann arbor
  • twitter bot for the wolverine
  • clearstreets for ann arbor
  • twitter bot for new restaurants in washtenaw county
  • plotting bad sidewalks in AA
  • pollution in michigan water
@fgregg
fgregg / ward_community_area_overlap.csv
Created December 14, 2018 19:46
Ward Community Area Overlap
ward community proportion_of_ward_in_community_area
1 WEST TOWN 0.583178464274206
1 LOGAN SQUARE 0.392380054759725
1 HUMBOLDT PARK 5.05596447686876e-05
1 NORTH CENTER 0.00942203809405585
1 LINCOLN PARK 0.0149688832272447
10 HEGEWISCH 0.265895481584491
10 SOUTH CHICAGO 0.0768757437832051
10 RIVERDALE 3.43949272079429e-09
10 EAST SIDE 0.151634348793321
@fgregg
fgregg / plots.R
Created September 22, 2018 20:36
Declines in Felony Cases
library(tidyverse)
library(lubridate) # we'll use this for some date processing
# Read in the data
intake <- read.csv("Intake.csv")
initiation <- read.csv("Initiation.csv")
# Turn RECEIVED_DATE from strings to date objects
intake <- intake %>%
mutate(RECEIVED_DATE = as.Date(RECEIVED_DATE, '%m/%d/%Y'))