LinkChecker should support not only specifying the maximum number of active threads overall, but more importantly the maximum number of concurrent requests for a single host. Otherwise you can overwhelm a single host with too many concurrent requests while requests for other hosts continue to work just fine.

(This is a real world problem, I just encountered it, not something theoretical.)