SEcond last line of the code is generating an error.
SyntaxError: 'yield' outside function
What is the role of this line at the last of your code?
yield None
Try removing it
The yield is a part of this if block:
if (len(image_url_list)>3):
d = {
'Item Title' : item_title,
'Description' : item_description,
'Item Price' : item_price,
'Savings' : item_savings,
'Details' : idetail
}
CATEGORY_NAME = response.meta['dir_name']
ITEM_DIR_URL = os.path.join(self.BASE_DIR,os.path.join(CATEGORY_NAME,item_title))
if not os.path.exists(ITEM_DIR_URL):
os.makedirs(ITEM_DIR_URL)
with open(os.path.join(ITEM_DIR_URL,'metadata.txt'),'w') as f:
json.dump(d,f)
for i,img_url in enumerate(image_url_list):
r = requests.get(img_url)
with open(os.path.join(ITEM_DIR_URL,"image_{}.jpg".format(i)),'wb') as f:
f.wite(r.content)
print("--> Successfully Saved \""+item_title+"\" data at:"+ITEM_DIR_URL)
yield d
but the problem is that this if block isn’t a part of any function. A ‘yield’ can only exist inside a function. Maybe the problem originated because of some mixed up indentation.
Hope this helps!
Thank you for your help. But now the problem is that Pepperfry Data folder is been created with its sub-folders but every sub-folders are empty, no data is being extracted! HELP
Try this one thing. Replace the contents of the file settings.py with this:
I have changed the User Agent and set Obey Robots.txt to False.
Let me know if this helps.
No man, it’s still not working. The result is same as it was earlier.
I think the problem is not with the code, rather, some changes in the pepperfry website. Let me look into it. Till then, you can move on to the other sections of the course.
Hey man,
Here is the working code: https://ide.codingblocks.com/s/260867
Some xpaths were wrong earlier because of changes in the website.
Also, ensure these lines in the settings.py:
USER_AGENT = 'Mozilla/5.0'
ROBOTSTXT_OBEY = False
Hope this helps! You can mark the doubt as resolved now in the My Doubts section.
I hope I’ve cleared your doubt. I ask you to please rate your experience here
Your feedback is very important. It helps us improve our platform and hence provide you
the learning experience you deserve.
On the off chance, you still have some questions or not find the answers satisfactory, you may reopen
the doubt.
No bro, everything is same as it was.
Pepper Fry is a real live website. You have to understand that real websites do not remain the same for such long periods of time. I posted that code 18 days ago and it was working fine then. Now, between this time, Pepper Fry could have made some changes to their website and subsequently, the doc structure as well as the xpaths also change. You have to keep updating them to keep your code working.
Hope you understand!