Skip to content

a lightweight web crawler using Ghost Driver and PhantomJS

License

Notifications You must be signed in to change notification settings

hackerzpf/ghost-crawler

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GhostCrawler

A lightweight web crawler that uses the Ghost Driver implementation of the WebDriver Wire Protocol from the PhantomJS project.

Pre-requisites

Install PhantomJS using homebrew.

$ brew update && brew install phantomjs

Make sure PhantomJS is running on port 9134.

$ phantomjs --webdriver=9134
PhantomJS is launching GhostDriver...
[INFO  - 2014-10-30T20:58:59.516Z] GhostDriver - Main - running on port 9134

Usage

TODO: Write usage instructions here

Roadmap

  1. bin directory
  2. command line arguments
  3. website dependency graph
  4. screenshot image diffing

About

a lightweight web crawler using Ghost Driver and PhantomJS

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Ruby 93.0%
  • HTML 7.0%