Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ensure extraHops also apply to maxDepth #694

Merged
merged 2 commits into from
Sep 30, 2024
Merged

ensure extraHops also apply to maxDepth #694

merged 2 commits into from
Sep 30, 2024

Conversation

ikreymer
Copy link
Member

@ikreymer
ensure extraHops also apply to maxDepth
9f04651 
- if extraHops is set, crawler should visit pages beyond maxDepth
- currently returning out of scope at depth limit even if extraHops is set
- adjust isInScope and isAtMaxDepth to account for extraHops
- fixes #693
@ikreymer ikreymer requested a review from tw4l September 30, 2024 17:32
tw4l
tw4l approved these changes Sep 30, 2024
View reviewed changes
Copy link
Member

@tw4l tw4l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes look good and tested against reproduction example in the issue

@ikreymer ikreymer merged commit 728f002 into main Sep 30, 2024
2 of 4 checks passed
@ikreymer ikreymer deleted the fix-extra-hops-and-max-depth branch September 30, 2024 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tw4l tw4l tw4l approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug]: Crawl Configuration Inconsistency: Max Depth and Include Any Linked Page
2 participants
@ikreymer @tw4l