Skip to content

This project is created for the purpose of crawling professors' information, including name, contact, PhD degree, research interest, and personal statement.

Notifications You must be signed in to change notification settings

HAOzj/Crawler_of_professor

Repository files navigation

Project

This project is created to make an inventory of professors' profiles, including name, contact, PhD degree, and research interest.

files

file function
conf_loader.py save params for crawlers and elastic-search
bs.py parse html to get information of profs
__main__.py save profs' information to local elastic-search
diy_search.ipynb launch flexible search

prerequisites

elastic-search

download a proper version from official website and run a client

deployment

pip install -r requirements.txt  
python3 __main__.py

About

This project is created for the purpose of crawling professors' information, including name, contact, PhD degree, research interest, and personal statement.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages