Skip to content

A multi_threading and a single_threading baidu picture crawler

License

Notifications You must be signed in to change notification settings

XueHansel/crawler_image

Repository files navigation

This is a repository for baidu image crawler

multi_threading crawler

It supports multi-threaded downloads, so download speed is very fast.

Put the labels that you want to download in name.txt

In name.txt file, Each line will include one label.

single_threading crawler

This single_threading crawler speed maybe slower than multi_threading.

Put the labels that you want to download in name.txt

In name.txt file, Each line will include one label.

data_del.py

If you have download many pictures, some pictues maybe be damaged. When you use this picture by PIL, may be you'll get EXIF warinig.

So you must delete this damaged image.

The chinese tutorial was posted in https://zhuanlan.zhihu.com/p/97060258

About

A multi_threading and a single_threading baidu picture crawler

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages