Skip to content

Instantly share code, notes, and snippets.

View philshem's full-sized avatar
💭
🐙

@philshem philshem

💭
🐙
View GitHub Profile
#!/usr/bin/python3
musical_notes = list('CDEFGAB')
#musical_notes = list('CDEFGAH')
fp = 'yob2020.txt'
with open(fp, 'r') as f:
lines = f.read().split('\n')
names = [x.split(',')[0].upper() for x in lines if len(x) > 0]
import requests
df = pd.read_html(requests.get('https://www.slickcharts.com/sp500',headers={'User-agent': 'Mozilla/5.0'}).text)[0]
print(df)
df.to_csv('sp.csv',index=False)
@philshem
philshem / publibike_zuerich_elevation.csv
Created February 19, 2021 19:19
Elevation of Publibike stations around Zürich
station_id latitude longitude elevation elevation_above_lake
pb_110 47.396401 8.4860017 399.0173645 -6.982635498
pb_149 47.392336 8.491061 399.4473877 -6.552612305
pb_286 47.3985553 8.493822 401.0709534 -4.929046631
pb_90 47.3917964 8.5056978 402.3087158 -3.69128418
pb_395 47.3669233 8.5420304 402.3295898 -3.670410156
pb_152 47.389823 8.49163 402.6643982 -3.335601807
pb_155 47.374711 8.543042 402.9417419 -3.058258057
pb_450 47.350578 8.560587 403.0110779 -2.988922119
pb_130 47.391366 8.488108 403.0958862 -2.90411377
@philshem
philshem / Collect Parler Metadata.ipynb
Created January 12, 2021 07:25 — forked from kylemcdonald/Collect Parler Metadata.ipynb
Collect video URLs and GPS data for Parler videos.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@philshem
philshem / forbes400.csv
Last active December 7, 2020 10:11
scraper: Forbes' richest 400, 30 years
We can't make this file beautiful and searchable because it's too large.
name,lastName,uri,imageUri,worthChange,age,source,industry,gender,timestamp,headquarters,state,realTimeWorth,date,realTimeRank,realTimePosition,squareImage,title,category,government,position,rank,worth,year,source_url,salary,managementAssets,pay
Abigail Johnson,Johnson,abigail-johnson,abigail-johnson,0.0,58.0,money management,Finance & Investments,F,2020-12-07 08:50:01.435,"Milton, MA",Massachusetts,14974.7,1993-01-01 00:00:00,39.0,39.0,//specials-images.forbesimg.com/imageserve/5f15ddd274642d00070464eb/416x416.jpg?background=000000&cropX1=62&cropX2=1355&cropY1=144&cropY2=1437,,,,,,,1990,http://www.forbes.com/ajax/list/data?year=1990&uri=forbes-400&type=person,,,
Aerin Lauder,Lauder,aerin-lauder,aerin-lauder,0.0,50.0,cosmetics,Fashion & Retail,F,2020-12-07 08:50:01.435,"New York, NY",New York,3215.528,1993-01-01 00:00:00,285.0,285.0,https://specials-images.forbesimg.com/imageserve/5f46b424808a289780083f05/416x416.jpg?background=000000&cropX1=895&cropX2=2304&cropY1=287&cropY2=1696,,,,,,,1990,http://www.forbes.
x y y_ideal
10 10.9277728455939 10
10.25 10.5933490961181 10.25
10.5 14.0852031368451 10.5
10.75 12.4232922691896 10.75
11 14.7683460256395 11
11.25 9.93532391180117 11.25
11.5 12.4777201750146 11.5
11.75 11.3343548513887 11.75
12 10.2801940515687 12
@philshem
philshem / at.20200927.csv
Last active September 27, 2020 18:44
scrape individual state data from https://coronadata.at used for this answer https://opendata.stackexchange.com/a/18094/1511
We can make this file beautiful and searchable if this error is corrected: It looks like row 7 should actually have 11 columns, instead of 10 in line 6.
Datum,Tests,Fälle (Gesamt),Fälle (Aktiv),/ 100.000 Einwohner,Genesene,Verstorbene,Hospitalisiert,Intensivstation,state,url
27.09.2020 20:00,46.351 +476 (1%),758 +9 (1%),141 0 (0%),48,606 +9 (2%),11 +0 (0%),8 +1 (14%),1 0 (0%),burgenland,https://www.coronatracker.at/burgenland
26.09.2020 20:00,45.875 +592 (1%),749 +20 (3%),141 +6 (4%),48,597 +14 (2%),11 +0 (0%),7 +4 (133%),1 +1 (100%),burgenland,https://www.coronatracker.at/burgenland
25.09.2020 20:00,45.283 +908 (2%),729 +8 (1%),135 -5 (-4%),46,583 +13 (2%),11 +0 (0%),3 -4 (-57%),0 0 (0%),burgenland,https://www.coronatracker.at/burgenland
24.09.2020 20:00,44.375 +528 (1%),721 +4 (1%),140 -1 (-1%),48,570 +5 (1%),11 +0 (0%),7 +3 (75%),0 0 (0%),burgenland,https://www.coronatracker.at/burgenland
23.09.2020 20:00,43.847 +999 (2%),717 +33 (5%),141 +23 (19%),48,565 +10 (2%),11 +0 (0%),4 -3 (-43%),0 -1 (-100%),burgenland,https://www.coronatracker.at/burgenland
22.09.2020 20:00,42.848 +303 (1%),684 +14 (2%),118 +2 (2%),40,555 +12 (2%),11 +0 (0%),7 -2 (-22%),1 0 (0%),b
@philshem
philshem / 53827775.py
Created September 1, 2020 07:41
refactor stackoverflow answer to read unread emails and also fix for loop https://stackoverflow.com/a/53827775/2327328
import imaplib
import email
# adapted from: https://stackoverflow.com/a/53827775/2327328
def read_email_from_gmail():
mail = imaplib.IMAP4_SSL('imap.gmail.com')
mail.login('MYEMAIL','MYSECRET')
mail.select('inbox')
import pandas as pd
# https://burntsushi.net/stuff/worldcitiespop.csv
df = pd.read_csv('worldcitiespop.csv',low_memory=False)
df = df.query('Country == "us"')
print(len(df))
# prints 141989
@philshem
philshem / data.json
Last active August 24, 2020 03:26
sample data
{
"1960":316.91,
"1961":317.64,
"1962":318.45,
"1963":318.99,
"1964":319.62,
"1965":320.04,
"1966":321.38,
"1967":322.16,
"1968":323.04,