Skip to content

Instantly share code, notes, and snippets.

@sflender

sflender/soup_seafearers_scrape.py

Created August 13, 2020 05:30
Show Gist options
  • Save sflender/b5334701f1d6f5c7b2954bbf7a2b6078 to your computer and use it in GitHub Desktop.
Save sflender/b5334701f1d6f5c7b2954bbf7a2b6078 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
soup_seafearers_scrape.py
import pandas as pd
import numpy as np
def get_values_for_key(key, entries):
return [(str(e).split(key)[1].split(';')[0].split('>')[1].split('<')[0] if key in str(e) else 'missing') for e in entries ]
df = pd.DataFrame()
df['id'] = np.arange(0,len(entries),1)
for key in ['Flag','Abandoned','Notified','Port of abandonment']:
df[key] = get_values_for_key(key, entries)
df['abandoned_year'] = [x[-4:] for x in df.Abandoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment