Last active
August 14, 2018 11:28
-
-
Save skpyns/d3193514f74ef3c589dc7984b06ed4bd to your computer and use it in GitHub Desktop.
Fetching a number of ads of used bicycles
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests, bs4 | |
# Making a request | |
r = requests.get('https://www.2bike.rs/cikloberza/mali-oglasi/delovi-20/ramovi-i-delovi-za-ram-181/ramovi-21?fl20[]=138&fl12[]=72') | |
if r.status_code != 200: | |
raise Exception("Error!") | |
# Making a soup and locating an object to be retrieved | |
soup = bs4.BeautifulSoup(r.text, 'lxml') | |
ads = soup.find('div', {'class': 'total'}) | |
print('Number of ads: ' + ads.text) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
As you requested feed back on Reddit here is some.
You can be more specific in the error message you print like:
raise Exception(f'Error! The status code {r.status_code} was returned from the server')
Also when web scrapping any site you need to remember that stuff can change and any point and won't be consistent. So with that in mind I would change this line
ads = soup.find('div', {'class': 'total'})
toads = soup.find_all('div', {'class': 'total'})
as there are two of them on the page. And you never know they might be different values in each. You can then decide what you want to do with the "ads" list. You can print all of them, merge them, just print the frist one, etc.By the work Good Work!