I set USER_AGENT = ‘Chrome/92.0.4515.131’, in settings.py but now i am getting None as response, following error is displayed when trying to perform response.css :
‘NoneType’ object has no attribute ‘css’
Getting None response
hey @shubhrs25_01611cd3f0e6aa02,
one thing i guess user_agent need to be changed , second is share me your code.
code link: https://drive.google.com/file/d/1L5lVoRpr5SORiP7qQLm_aMK1kzAWCJLi/view?usp=sharing also i have changed my USER_AGENT to Chrome/92.0.4515.131 as I have mentioned above
hey @shubhrs25_01611cd3f0e6aa02 ,
sorry for the delay.
Just upload your full folder of scrapy project on github and share me the link…
hey @shubhrs25_01611cd3f0e6aa02,
can you please share me the output on the command prompt that you are getting , and command that you are using to to crawl on this scrapper.
desired commands and outputs: https://drive.google.com/drive/folders/1nIrPUwNYpFaoTA5EzykqQqDCYkoV-Bha?usp=sharing
hey @shubhrs25_01611cd3f0e6aa02,
first thing is don’t use shell command , go with crawl only for now.
and for crawl your code is working correct , you haven’t provided anything what to be on retrieving the page in your parse function so add your logic there and work ahead.
I hope I’ve cleared your doubt. I ask you to please rate your experience here
Your feedback is very important. It helps us improve our platform and hence provide you
the learning experience you deserve.
On the off chance, you still have some questions or not find the answers satisfactory, you may reopen
the doubt.
but sir without the shell how can actually find the components, and come up with a logic
what components are you referring to ?