Skip to content

Instantly share code, notes, and snippets.

@jQwotos
Created October 17, 2017 13:39
Show Gist options
  • Save jQwotos/37c54992881824d7084680ee91c9633c to your computer and use it in GitHub Desktop.
Save jQwotos/37c54992881824d7084680ee91c9633c to your computer and use it in GitHub Desktop.
A basic scraper that scrapes search results from flipp.com
import requests
BASE_URL = 'https://flipp.com'
BACKEND_URL = 'https://backflipp.wishabi.com/flipp'
SEARCH_URL = '%s/items/search' % BACKEND_URL
ITEM_URL = '%s/items/' % BACKEND_URL
def scrape_item(item_id):
return requests.get(
"%s/%s" % (ITEM_URL, item_id,)
).json()
def search(query, postal_code):
data = requests.get(
SEARCH_URL,
params = {
'q': query,
'postal_code': postal_code,
}
).json()
return [
scrape_item(x.get('flyer_item_id'))
for x in data.get('items')
]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment