Add option to respect or ignore nofollow directives #7
Open
Description
Offer an option to allow spidering links of URLs with a nofollow directive.
{{ message }}
No one assigned
No projects
None yet
No branches or pull requests
Offer an option to allow spidering links of URLs with a nofollow directive.