Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rand-user-agent生成不合理的UA导致爬虫被识别 #13353

Closed
1 task done
NightSpaceC opened this issue Sep 21, 2023 · 1 comment · Fixed by #13356
Closed
1 task done

rand-user-agent生成不合理的UA导致爬虫被识别 #13353

NightSpaceC opened this issue Sep 21, 2023 · 1 comment · Fixed by #13356
Labels
Bug Ping: Pinged Count: 1/1 Route: v2 v2 route related RSS bug Something isn't working

Comments

@NightSpaceC
Copy link
Contributor

Routes

/bilibili/user/video/:uid/:disableEmbed?

Full routes

/bilibili/user/video/1858861103

Related documentation

https://docs.rsshub.app/zh/routes/social-media#bilibili-up-zhu-tou-gao

What is expected?

正常访问API获取内容

What is actually happening?

在访问频率稍高时（约15/s）被网站屏蔽

Deployment information

Self-hosted

Deployment information (for self-hosted)

OS: Windows, Node: v18.17.0

Additional info

`rand-user-agent` 库生成了例如 `Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Code/1.82.2 Chrome/114.0.5735.289 Electron/25.8.1 Safari/537.36` 的不合理UA

手动设置真实的浏览器UA在同样数量的请求下没有出现这个问题

这应该是一个普遍的存在的问题

This is not a duplicated issue

  • I have searched existing issues to ensure this bug has not already been reported
@NightSpaceC NightSpaceC added the RSS bug Something isn't working label Sep 21, 2023
@github-actions
Copy link
Contributor

Searching for maintainers:
  • /bilibili/user/video/:uid/:disableEmbed?: @DIYgod

To maintainers: if you are not willing to be disturbed, list your username in scripts/workflow/test-issue/call-maintainer.js. In this way, your username will be wrapped in an inline code block when tagged so you will not be notified.

如果所有路由都无法匹配，issue 将会被自动关闭。如果 issue 和路由无关，请使用 NOROUTE 关键词，或者留下评论。我们会重新审核。
If all routes can not be found, the issue will be closed automatically. Please use NOROUTE for a route-irrelevant issue or leave a comment if it is a mistake.

@TonyRL TonyRL mentioned this issue Sep 21, 2023
Merged
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Ping: Pinged Count: 1/1 Route: v2 v2 route related RSS bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

chore: fix random ua generation TonyRL/RSSHub
1 participant
@NightSpaceC