How to get remote webSocketDebuggerUrl (or connect without GUID) #940

davidwindell · 2017-10-02T21:17:25Z

We're using puppeteer.connect to connect to a remote Chrome instance:

puppeteer.connect(
  { "browserWSEndpoint": "ws://{remoteip}:9222/devtools/browser/fa60c034-422d-4f2c-bbeb-17a2cfd690f2"}
);

Is there any way we can avoid needing to know the GUID? Perhaps by a flag when starting the remote Chrome (it's on a secure internal network).

Alternatively, what would be the best way in node to look up the browserWSEndpoint from http://{remoteip}:9222/json/version?

Or are we approaching this the wrong way?

The text was updated successfully, but these errors were encountered:

kdekooter · 2017-10-04T07:23:38Z

As to your second question: http://{remoteip}:9222/json/version returns JSON. You could use the request library to call this URL and do a JSON.parse() on the response which gives you an object with the id as an attribute.

davidwindell · 2017-10-04T13:51:01Z

Thanks @kdekooter, I've put together the below script which is working (I'm new to node so feel free to comment on my appalling structure and handling of async):

generate.js

/*
 * A puppeteer script to generate a PDF from stdin using a remote chrome instance.
 * The generated PDF content is returned to stdout.
 *
 * Usage: node generate.js <host:port> <format>
 */
'use strict';

const puppeteer = require('puppeteer');
const http = require('http');

var html     = '';
var endpoint = process.argv[2];
var format   = process.argv[3] ? process.argv[3] : 'A4';

process.stdin.setEncoding('utf8');

process.stdin.on('readable', () => {
    const chunk = process.stdin.read();
    if (chunk !== null) {
        html += chunk;
    }
});

process.stdin.on('end', () => {
    http.get(`http://${process.argv[2]}/json/version`, (res) => {
        res.setEncoding('utf8');
        let rawData = '';
        res.on('data', (chunk) => { rawData += chunk; });
        res.on('end', () => {
            try {
                const parsedData = JSON.parse(rawData);
                endpoint = parsedData.webSocketDebuggerUrl;

                (async () => {
                    const browser = await puppeteer.connect({browserWSEndpoint: endpoint});
                    const page    = await browser.newPage();

                    await page.setJavaScriptEnabled(false);
                    await page.goto(`data:text/html,${html}`, {waitUntil: 'networkidle'});

                    const pdf = await page.pdf({format: format});

                    await process.stdout.write(pdf);

                    await browser.close();
                })();
            } catch (e) {
                console.error(`Unable to generate PDF: ${e.message}`);
            }
        });
    }).on('error', (e) => {
        console.error(`Unable to connect to Chrome: ${e.message}`);
    });
});

For example: echo "<html>foo</html>" | node generate.js 127.0.0.1:9222 > foo.pdf

If anyone has any suggestions to speed this up and/or avoid the browserWSEndpoint lookup I'd love to hear them 👍

aslushnikov · 2017-10-04T21:08:22Z

Is there any way we can avoid needing to know the GUID? Perhaps by a flag when starting the remote Chrome (it's on a secure internal network).

@davidwindell why do you want to avoid this? Since you're the one who starts chrome, you can get its GUID and share with all interested parties.

davidwindell · 2017-10-04T22:05:20Z

@aslushnikov we launch the chrome instance automatically in a disposable docker container and communicating/broadcasting the Guid just wouldn't work. A reboot of the host means all dependent services have to look it up.

aslushnikov · 2017-10-04T22:35:00Z

@davidwindell I see, thanks. This seems to be a general service discovery problem, but yes, you
can hack this in with the script you've suggested.

drorweissweiss · 2017-10-13T11:17:28Z

@davidwindell , @kdekooter
I'm also trying to discover the endpoint's browser id so I can use browserWSEndpoint, but I don't see any id field in the object returning from http://{remoteip}:9222/json/version:

{
"Browser": "HeadlessChrome/61.0.3163.91",
"Protocol-Version": "1.2",
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/61.0.3163.91 Safari/537.36",
"V8-Version": "6.1.534.37",
"WebKit-Version": "537.36 (@2f544949507dc5330b714cb017e3f584e791a1bf)"
}

I'm starting my headless-chrome instance using these flags:

--headless
--remote-debugging-address=0.0.0.0
--remote-debugging-port=9222
--no-sandbox
--disable-gpu
--disable-sync
--disable-translate
--disable-extensions
--disable-default-apps
--disable-background-networking
--safebrowsing-disable-auto-update
--mute-audio
--no-first-run
--hide-scrollbars
--metrics-recording-only
--ignore-certificate-errors

Am I missing any step to detect the id (perhaps did you guys started your headless-chrome instance with a special flag that allows this information broadcast) ?

davidwindell · 2017-10-13T13:13:11Z

@drorweissweiss this is only in the latest versions of Chrome (62+ I think, certainly 63). Caught me out at first!

seeekr · 2018-04-12T23:13:23Z

FYI: In Chrome 65+ (maybe earlier) there is a webSocketDebuggerUrl property instead of id in the /json/version endpoint's json response.

ja8zyjits · 2018-10-16T13:01:23Z

@aslushnikov for the new version chromium 69 is there a way to connect with remote headless chromium without knowing the GUID. I know there is a way through service discovery /json but since i would be having few chromium running behind my load balancer I would like the lb to redirect dynamically.

Possible Way

Is there a way to start chromium with a custom GUID so that my supervisord can autorestart and apply the same GUID which my puppeteer would be trying to access and it will also make the task of lb to redirect without worrying about GUID matching issues.

Thanks

ginuerzh · 2020-03-30T11:15:49Z

@aslushnikov for the new version chromium 69 is there a way to connect with remote headless chromium without knowing the GUID. I know there is a way through service discovery /json but since i would be having few chromium running behind my load balancer I would like the lb to redirect dynamically.

Possible Way

Is there a way to start chromium with a custom GUID so that my supervisord can autorestart and apply the same GUID which my puppeteer would be trying to access and it will also make the task of lb to redirect without worrying about GUID matching issues.

Thanks

@ja8zyjits did you find any solution to the dynamic guid when deploying multiple chrome instances?
I want to deploy the headless chrome on k8s as a deployment, but the inconsistent urls make it impossible to scale up.

puneetjindal · 2020-04-05T11:26:00Z

@ginuerzh i would also be interested to understand the same!

qiankunxienb · 2021-04-20T07:31:56Z

Does anyone have the solution of this question?

Jaqenhghar · 2022-05-11T07:26:56Z

@davidwindell , @kdekooter I'm also trying to discover the endpoint's browser id so I can use browserWSEndpoint, but I don't see any id field in the object returning from http://{remoteip}:9222/json/version:

{
"Browser": "HeadlessChrome/61.0.3163.91",
"Protocol-Version": "1.2",
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/61.0.3163.91 Safari/537.36",
"V8-Version": "6.1.534.37",
"WebKit-Version": "537.36 (@2f544949507dc5330b714cb017e3f584e791a1bf)"
}

I'm starting my headless-chrome instance using these flags:

--headless
--remote-debugging-address=0.0.0.0
--remote-debugging-port=9222
--no-sandbox
--disable-gpu
--disable-sync
--disable-translate
--disable-extensions
--disable-default-apps
--disable-background-networking
--safebrowsing-disable-auto-update
--mute-audio
--no-first-run
--hide-scrollbars
--metrics-recording-only
--ignore-certificate-errors

Am I missing any step to detect the id (perhaps did you guys started your headless-chrome instance with a special flag that allows this information broadcast) ?

It is in /json/list

aslushnikov closed this as completed Oct 4, 2017

ja8zyjits mentioned this issue Oct 17, 2018

Connect chrome without GUID #3420

Closed

lukaszfiszer mentioned this issue Nov 13, 2018

Feature proposal: Ability to pass browserUrl to puppeteer.connect #3537

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to get remote webSocketDebuggerUrl (or connect without GUID) #940

How to get remote webSocketDebuggerUrl (or connect without GUID) #940

davidwindell commented Oct 2, 2017 •

edited

Loading

kdekooter commented Oct 4, 2017 •

edited

Loading

davidwindell commented Oct 4, 2017

aslushnikov commented Oct 4, 2017

davidwindell commented Oct 4, 2017

aslushnikov commented Oct 4, 2017

drorweissweiss commented Oct 13, 2017

davidwindell commented Oct 13, 2017

seeekr commented Apr 12, 2018

ja8zyjits commented Oct 16, 2018

ginuerzh commented Mar 30, 2020

Possible Way

puneetjindal commented Apr 5, 2020

qiankunxienb commented Apr 20, 2021

Jaqenhghar commented May 11, 2022

How to get remote webSocketDebuggerUrl (or connect without GUID) #940

How to get remote webSocketDebuggerUrl (or connect without GUID) #940

Comments

davidwindell commented Oct 2, 2017 • edited Loading

kdekooter commented Oct 4, 2017 • edited Loading

davidwindell commented Oct 4, 2017

aslushnikov commented Oct 4, 2017

davidwindell commented Oct 4, 2017

aslushnikov commented Oct 4, 2017

drorweissweiss commented Oct 13, 2017

davidwindell commented Oct 13, 2017

seeekr commented Apr 12, 2018

ja8zyjits commented Oct 16, 2018

Possible Way

ginuerzh commented Mar 30, 2020

Possible Way

puneetjindal commented Apr 5, 2020

qiankunxienb commented Apr 20, 2021

Jaqenhghar commented May 11, 2022

davidwindell commented Oct 2, 2017 •

edited

Loading

kdekooter commented Oct 4, 2017 •

edited

Loading