Not able to run the spider

#1

my project is in C:\Users\bhard\myproject\myproject\spiders
directory
and when I am running the command
C:\Users\bhard\myproject>scrapy crawl qoute_spider
in cmd then following error is occuring
NameError: name ‘null’ is not defined

#2

hey @bhardwajyash249 ,
can you check that in your code, there is usage or declaration of variable null.

I guess you are using it ,but had not defined it.

#3

I have found the mistake I was saving my file as qoutes_spider.py but when I saved is simply as qoutes_spider then the issue is resolved

#4

now another error is coming

#5

KeyError: ‘Spider not found: quotes_spider’

#6

actually the error is not resolved but actually got behind another error.
SO , just share me your code.

#7

I have made another project named tutorial and write this code in a file named qoutes_spider in spider fol:

#8

import scrapy class QuotesSpider(scrapy.Spider): name = “qoutes” def start_requests(self): urls = [ ‘http://quotes.toscrape.com/page/1/’, ‘http://quotes.toscrape.com/page/2/’, ] for url in urls: yield scrapy.Request(url=url, callback=self.parse) def parse(self, response): page = response.url.split("/")[-2] filename = f’quotes-{page}.html’ with open(filename, ‘wb’) as f: f.write(response.body) self.log(f’Saved file {filename}’)

#9

and following error is coming after I typed C:\Users\bhard\tutorial\tutorial\spiders>scrapy crawl qoutes_spiderKeyError: ‘Spider not found: qoutes_spider’

#10

also I want to say that file is saved with .ipynb and not .py extension

#11

I am running on jupyter and writing the last command on cmd i.e., scrapy crawl quotes_spider I am not using any software

#12

hey @bhardwajyash249 ,
it wont work on jupyter notebook
scrapy crawl uses some other files too , that are only present in project when we create using scrapy commands

#13

so where should I run it ?

#14

please can you suggest some software where to run it will it run on vscode?

#15

you dont need any software.
make a project using scrapy only and it will create all the required files

#16

I hope I’ve cleared your doubt. I ask you to please rate your experience here
Your feedback is very important. It helps us improve our platform and hence provide you
the learning experience you deserve.

On the off chance, you still have some questions or not find the answers satisfactory, you may reopen
the doubt.