GitHub - Hardysong/web-crawler-spider-: 基于Scrapy开发的网络爬虫，用于爬取大众点评，链家和搜房的数据

Hardysong / web-crawler-spider- Public

Notifications You must be signed in to change notification settings
Fork 8
Star 16

基于Scrapy开发的网络爬虫，用于爬取大众点评，链家和搜房的数据

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
spiders		spiders
README		README
__init__.py		__init__.py
__init__.pyc		__init__.pyc
items.py		items.py
items.pyc		items.pyc
pipelines.py		pipelines.py
settings.py		settings.py
settings.pyc		settings.pyc

Repository files navigation

Based on the Scrapy lib
get web data of dazongdianping（大众点评）,soufang（搜房） and lianjia（链家） data.
Other users can modify the content of python file to satisfy owner application

网络爬虫，用于爬取大众点评，搜房，链家上的数据

if users want export data as csv format , input : scrapy crawl amazon -t csv -o **.csv --loglevel=INFO

recommend is export the data into database (MySQL).details please reference web


Anyother questions pleasure connect the author: hardysong110@163.com