Scrape a shopping website

#1

how to store the data in the form of table in csv file while using scrapy ? I mean in video bhaiya stored data of each quote in a separate dictionary not in one table

#2

hey @stutijain578 ,
you can use pandas for this functionality.
First you need to create an empty table with schema that you want to have .
and then you need to just append that dictionary to it , and at the end it will create a table the way you wanted to have.

I hopet his helps you.
Thank You. :slightly_smiling_face:.

#3

image
image2850×1494 372 KB

what to do next?

#4

def store(L):
_____df = pd.DataFrame(L,columns=['imageurl','title','price'])
_____df.to_csv("books1.csv")

in the end add a condition
if next_page is None:
_____store(L)

change your variable with L.

I hope this works.

#5

each time i run it, it says no active project

#6

you need to run from its root directory with command scrapy crawl …
are you doing the same thing, ?

#7

how can i get the path of the directory on mac?

#8

what command are you using to run this code.

#9

scrapy crawl books_spider -o all_books.csv -t csv

#10

can you please show me a screenshot of the command prompt while you are running this code.