Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Instagram: Can't download public Reel, "Unable to extract username" #5967

Open
10 tasks done
jazzido opened this issue Jan 5, 2023 · 8 comments
Open
10 tasks done

Instagram: Can't download public Reel, "Unable to extract username" #5967

jazzido opened this issue Jan 5, 2023 · 8 comments
Labels
site-bug Issue with a specific website

Comments

@jazzido
Copy link

jazzido commented Jan 5, 2023

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

Region

No response

Provide a description that is worded well enough to be understood

Reel can't be downloaded

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://www.instagram.com/reel/Cm1zP8OB9fm/']
[debug] User config: []
[debug] System config: []
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2023.01.02 [d83b0ad] (pip)
[debug] Python 3.11.1 (CPython x86_64 64bit) - macOS-13.0.1-x86_64-i386-64bit (OpenSSL 1.1.1s  1 Nov 2022)
[debug] exe versions: ffmpeg 5.1.2 (setts), ffprobe 5.1.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.16.0, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1754 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2023.01.02, Current version: 2023.01.02
yt-dlp is up to date (2023.01.02)
[Instagram] Extracting URL: https://www.instagram.com/reel/Cm1zP8OB9fm/
[Instagram] Cm1zP8OB9fm: Setting up session
[Instagram] Cm1zP8OB9fm: Downloading JSON metadata
WARNING: [Instagram] unable to extract username; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
ERROR: [Instagram] Cm1zP8OB9fm: Unable to extract video url; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/local/Cellar/yt-dlp/2023.1.2/libexec/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 680, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.1.2/libexec/lib/python3.11/site-packages/yt_dlp/extractor/instagram.py", line 486, in _real_extract
    video_url = self._og_search_video_url(webpage, secure=False)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.1.2/libexec/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 1371, in _og_search_video_url
    return self._html_search_regex(regexes, html, name, **kargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.1.2/libexec/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 1264, in _html_search_regex
    res = self._search_regex(pattern, string, name, default, fatal, flags, group)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.1.2/libexec/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 1228, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
@jazzido jazzido added site-bug Issue with a specific website triage Untriaged issue labels Jan 5, 2023
@bashonly
Copy link
Member

bashonly commented Jan 5, 2023 •
edited
Loading

Something has changed on instagram's end. This is not the usual rate-limit block/redirect, their graphql is sending an empty JSON response:

{
  "data": {
    "shortcode_media": null
  },
  "status": "ok"
}

@jazzido Are you able to open this URL in your browser without being logged in? (use private/incognito tab)

A possible workaround to download would be to pass --cookies-from-browser (when logged in with the browser), but it also looks like the authenticated API call details have changed slightly too, so use your IG account at your own risk

EDIT: It seems like only certain IG posts exhibit this behavior. I am still able to browse and download others without being logged in. The "shortcode_media": null JSON response should be handled by the extractor to give a more descriptive error message.

@bashonly bashonly removed the triage Untriaged issue label Jan 5, 2023
@bashonly bashonly moved this to instagram in Major sites Jan 5, 2023
@jazzido
Copy link
Author

jazzido commented Jan 5, 2023

Thanks for the quick reply, @bashonly

@jazzido Are you able to open this URL in your browser without being logged in? (use private/incognito tab)

It redirects me to the login page, so no.

A possible workaround to download would be to pass --cookies-from-browser (when logged in with the browser), but it also looks like the authenticated API call details have changed slightly too, so use your IG account at your own risk

That worked prefectly, thanks!

@firofame
Copy link

same issue

Example URL: https://www.instagram.com/reel/CnzB0EyPL_3/?igshid=Yzg5MTU1MDY=

yt-dlp -f "bestvideoext=mp4+bestaudioext=m4a/mp4" 'https://www.instagram.com/reel/CnzB0EyPL_3/?igshid=Yzg5MTU1MDY='
[Instagram] CnzB0EyPL_3: Downloading webpage
WARNING: [Instagram] unable to extract username; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
ERROR: [Instagram] CnzB0EyPL_3: Unable to extract video url; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

@firofame firofame mentioned this issue Jan 24, 2023
Closed
@egakupi
Copy link

egakupi commented Jun 30, 2023

There seems to be some restrictions because of location. I can download from the link from one country and can't from another

@bashonly bashonly mentioned this issue Sep 7, 2023
Closed
11 tasks
@bashonly bashonly changed the title Instagram: Can't download public Reel Instagram: Can't download public Reel, "Unable to extract username" Sep 7, 2023
@gamer191
Copy link
Collaborator

Example URL: https://www.instagram.com/reel/CnzB0EyPL_3/?igshid=Yzg5MTU1MDY=

I can't reproduce for this url, but I can reproduce for https://www.instagram.com/reel/Cw1yI2_SmQf/ (which was shared in #8048). Interestingly, that video is age-restricted, so it's likely just not possible to download that video without cookies

@xannythepleb
Copy link

Example URL: https://www.instagram.com/reel/CnzB0EyPL_3/?igshid=Yzg5MTU1MDY=

I can't reproduce for this url, but I can reproduce for https://www.instagram.com/reel/Cw1yI2_SmQf/ (which was shared in #8048). Interestingly, that video is age-restricted, so it's likely just not possible to download that video without cookies

I think age restriction is at least part of the issue. Because you must be signed in to view age restricted profiles, it needs a valid session. But it's partly geographic too.

From a UK or EU VPN server:

[Instagram] Extracting URL: https://www.instagram.com/reel/Cw75Ynlr40N/
[Instagram] Cw75Ynlr40N: Setting up session
WARNING: [Instagram] Cw75Ynlr40N: No csrf token set by Instagram API
[Instagram] Cw75Ynlr40N: Downloading JSON metadata
WARNING: [Instagram] Cw75Ynlr40N: General metadata extraction failed (some metadata might be missing).
[Instagram] Cw75Ynlr40N: Downloading webpage
WARNING: [Instagram] unable to extract shared data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
WARNING: [Instagram] Main webpage is locked behind the login page. Retrying with embed webpage (some metadata might be missing).
[Instagram] Cw75Ynlr40N: Downloading embed webpage

From a Swiss, US, Canadian VPN server:

[Instagram] Extracting URL: https://www.instagram.com/reel/Cw75Ynlr40N/
[Instagram] Cw75Ynlr40N: Setting up session
WARNING: [Instagram] Cw75Ynlr40N: No csrf token set by Instagram API
[Instagram] Cw75Ynlr40N: Downloading JSON metadata
WARNING: [Instagram] unable to extract username; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
ERROR: [Instagram] Cw75Ynlr40N: Unable to extract video url; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U

Judging by this it actually seems to get further in the first instance - it gets to the login page and prompts for credentials, whereas it can't seem to find the login page at all elsewhere.

This was referenced Feb 28, 2024
Closed
Closed
Open
@bashonly bashonly mentioned this issue Mar 6, 2024
Closed
10 tasks
@plasticalligator
Copy link

I just wanted to add the comment here that it is absolutely disgusting that these companies are implementing what surmounts to absolute dogshit tactics whose primary fallout is effectively suppressing independent journalism. On the day of Judgment the CEOs of these organizations ought to be quartered and hung in front of their respective office buildings on a livestream using their own platform. Fuck them so much. May every single bone in their bodies be broken and shoved up their assholes sideways.

@bashonly bashonly mentioned this issue Apr 27, 2024
Closed
@bashonly bashonly mentioned this issue Jun 1, 2024
Closed
11 tasks
@bashonly
Copy link
Member

bashonly commented Jun 1, 2024 •
edited
Loading

All of the videos that result in this error are not viewable in a browser without being logged in -- I get the same nearly-empty JSON response in my browser too.

Workaround: pass --cookies-from-browser or --cookies, but be aware that instagram does not hesitate to ban accounts when it detects them using 3rd party tools.

Proper solution: raise a better error when we get an API response like this

@bashonly bashonly mentioned this issue Sep 19, 2024
Closed
10 tasks
@bashonly bashonly mentioned this issue Sep 27, 2024
Closed
9 tasks
@bashonly bashonly mentioned this issue Dec 11, 2024
Closed
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
site-bug Issue with a specific website
Projects
Major sites
Status: instagram
Milestone
No milestone
Development

No branches or pull requests
7 participants
@jazzido @firofame @plasticalligator @egakupi @gamer191 @bashonly @xannythepleb