Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mega update #18

Merged
merged 36 commits into from
Aug 30, 2018
Merged

Mega update #18

merged 36 commits into from
Aug 30, 2018

Conversation

ian-r-rose
Copy link
Collaborator

Mostly complete update of the dask JupyterLab extension. Depends on dask/distributed#2185.

  • Puts a dask dashboard launcher in the left panel, which can be used to launch different dashboard panels into the main area:
    image
  • Allows the user to set the URL of the dashboard bokeh server.
  • Remembers the current URL, so it is persistent across page refreshes.
  • Allows a default setting for the url via the JupyterLab settings system.

TODO: It would be nice to have a button to auto-detect the client dashboard url from the current notebook.

cc @mrocklin @canavandl

@blink1073
Copy link
Collaborator

Nice!

@mrocklin
Copy link
Member

What is the right way to build and install a development lab-extension like this?

The installation instructions we have in our README appear to be out-of-date.

After googling around I found https://jupyterlab.readthedocs.io/en/stable/developer/extension_dev.html#extension-authoring

But got this:

mrocklin@carbon:~/workspace/dask-labextension$ jupyter labextension install  # install the current directory as an extension
> /home/mrocklin/Software/node-v7.7.3-linux-x64/bin/npm pack /home/mrocklin/workspace/dask-labextension
jupyterlab-dask-0.1.0.tgz

Errored, use --debug for full output:
ValueError: "/home/mrocklin/workspace/dask-labextension" is not a valid extension:
schemaDir is empty: "schema"

@blink1073
Copy link
Collaborator

Ah, it looks like the schema part of the package.json metadata needs to be removed since there are currently no schema.

@mrocklin
Copy link
Member

Yup. That works. Thanks @blink1073 .

I updated our dev installation instructions in the README. If someone has a minute to take a look at that file that would be welcome. I'm not sure if the installation requirements remain correct. I also don't know what the minimum JLab version should be, although it looks like error reporting is good enough when installing the extension that we can possibly omit this from our instructions.

@blink1073
Copy link
Collaborator

I'd say remove the version specifier. You also don't need to enable an extension. Otherwise, LGTM.

@mrocklin
Copy link
Member

OK, some questions:

So if I start a scheduler, open up a pane pointing to a plot, then close that scheduler and open another on the same port the plot in JLab becomes non-responsive. This isn't too surprising, knowing when to refresh the iframe would be difficult here. Given this, what is the best user experience that we can easily provide here? My first attempt was to click on the plot button again, but that just highlighted the exisitng non-responsive plot again. I found that I had to close the pane, then open it back up again with the plot-button.

Entering localhost:8787 into the DASK DASHBOARD URL field presents a message about no dashboard being present there. The user needs to explicitly enter in the http://. Is this something that we can add in for them if they are lazy, or is it better to be explicit?

Seeing the Dask icon present in the JupyterLab left-bar makes me unreasonably happy. I'm inclined to color the buttons to a Dask orange as well. Is here the right place to accomplish that? (I'm still figuring out how to update things efficiently, and so haven't been able to try this out directly yet.)

diff --git a/style/index.css b/style/index.css
index 1d9cf90..a674d73 100644
--- a/style/index.css
+++ b/style/index.css
@@ -25,6 +25,8 @@
 .dask-DashboardListing-item button {
   flex: 1 1 auto;
   height: var(--dask-launch-button-height);
+  background-color: #ECB172;
+  border: 1px solid #ECB172;
 }
 
 .dask-DashboardListing-item button:disabled {

@ian-r-rose
Copy link
Collaborator Author

The schema directory was something I forgot to add to the repo, sorry. It should be fixed now.

@ian-r-rose
Copy link
Collaborator Author

I have added a search button that tries to query the currently active kernel for the dask dashboard URL. If none is found, it does nothing:
image
If we find it is unreliable, it's easy enough to remove.

@mrocklin
Copy link
Member

Functionally this works quite well for me. My coloring is a bit off though:

image

@ian-r-rose
Copy link
Collaborator Author

What is the problem with the coloring? I used the HTML codes you posted above, and black text for the wording.

@ian-r-rose
Copy link
Collaborator Author

Ah, I see: the search icon. Fix incoming.

@mrocklin
Copy link
Member

Right, you beat me to the response :)

@mrocklin
Copy link
Member

This works quite well for me. I can't think of anything I would want to change.

The check for the file dask_horizontal.svg will change in the next release. I'll push up a change here once dask/distributed#2223 is merged in.

@ian-r-rose
Copy link
Collaborator Author

Great. Eventually that check should probably be replaced with a server-side component to this extension to handle the proxy-ing. That would also help to cut down on mixed-content issues in HTTP/HTTPS settings. For now this seems to work okay.

src/widget.tsx Outdated
@@ -397,7 +397,8 @@ export namespace DaskDashboardLauncher {

export const DEFAULT_ITEMS = [
{ route: 'individual-graph', label: 'Graph' },
{ route: 'individual-load', label: 'Load' },
{ route: 'individual-nbytes', label: 'Memory Use' },
{ route: 'individual-nprocessing', label: 'Processing Tasks' },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ian-r-rose the "Load" plot was split into two upstream. I found that I needed to update these manually. I know that at one point we discussed getting this information from Dask itself. I'm inclined to leave this as future work, but wanted to point it out here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I had wanted to do that, but was stymied by the CORS issues. Once we move some logic into a server-side proxy we should be able to serve the individual routes dynamically, but for now I think this is the only way.

@mrocklin
Copy link
Member

From my perspective this is good. Upstream has stabilized. I'm inclined to merge this and release. Any objections or additional work that should be done before that happens?

@ian-r-rose
Copy link
Collaborator Author

No objections, I think we are good to go.

@mrocklin mrocklin merged commit 528d390 into dask:master Aug 30, 2018
@ian-r-rose
Copy link
Collaborator Author

Shall I publish to npm?

@mrocklin
Copy link
Member

OK then, this is in. Thank you @ian-r-rose for doing this work. You should have an invitation giving you commit rights to this repository.

Shall I publish to npm?

Yes, that would be most welcome. Presumably also to PyPI? I don't understand the release process of JupyterLab plugins well, so I'd be happy to defer to you on this.

I can work on some docs within the mainline Dask docs.

@ian-r-rose
Copy link
Collaborator Author

Since there is no python package, there is nothing to publish to PyPI. The user will install the extension using jupyter labextension install dask-labextension. Speaking of which, should we stick with that name?

@mrocklin
Copy link
Member

mrocklin commented Aug 30, 2018 via email

@ian-r-rose
Copy link
Collaborator Author

I would say that strong naming conventions have not been established. I have personally been naming things of the form jupyterlab-dask, but that is not universal. Ultimately not a huge deal either way.

@mrocklin
Copy link
Member

That would be fine with me. So too would dask-jupyterlab (dask-foo is the convention behind dask sub-projects). I'm inclined to leave this decision with you as primary author if you're up for it.

@ian-r-rose
Copy link
Collaborator Author

I kept it as-is. So the install would be jupyter labextension install dask-labextension. It is now live!

@mrocklin
Copy link
Member

dashboard-jupyterlab

Uploading an image here for posterity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants