I am facing key not found error.
I have already set the name property but still it’s giving errors.
Key error while running in cmd
The exact error is : Spider not found: quotes_spider
- ! scrapy startproject myproject
A new folder of myproject was created
Then Inside the spiders folder I created a .py file whose code is as follows:
import scrapy
class QuotesSpider(scrapy.Spider):
name = “quotes_spider”
def start_requests(self):
# making a list of urls
urls = [
"http://quotes.toscrape.com/page/1/",
"http://quotes.toscrape.com/page/2/",
"http://quotes.toscrape.com/page/3/"
]
# making requests
# Generator function
for url in urls:
yield scrapy.Request(url = url, callback = self.parse)
# creating a parsing method
def parse(self,response):
# retrieving page ids for filenaming purpose
page_id = response.url.split('/')[-2]
filename = "quotes-%s.html"%page_id
# creating the file
with open(filename , 'wb') as f:
f.write(response.body)
# saving it in the directory
self.log('Saved file - %s'%filename)
Then I run this on jupyter and saved it.
Then in command prompt, I wrote:
…DATA SCIENCE\DATA ACQUISITION\Scrapy Tutorial\myproject> scrapy crawl quotes_spider
Ok , This time I downloaded sublime text and stored the file with .py extension and now it works fine.