Basic support for running in Jupyter and VS Code interactive console (h…

…oechenberger#52) * Basic support for running in Jupyter Fixes hoechenberger#51 This adds support for re-using an existing event loop, which allows us to run inside Jupyter. The support is currently very basic, i.e., each progress bar update prints a new line, and we don't display any nice emojis. But at least downloading works! * Fix emoji support! 😍 * Show widgets in VS Code * Add changelong and bump version * Update readme * Update linter settings
adam2392 · Oct 2, 2021 · 66a62fb · 66a62fb
1 parent d49bece
commit 66a62fb
Show file tree

Hide file tree

Showing 5 changed files with 58 additions and 12 deletions.
diff --git a/.github/workflows/run-linter.yml b/.github/workflows/run-linter.yml
@@ -38,7 +38,8 @@ jobs:
         python -m pip install flake8
     - name: Lint with flake8
       run: |
-        # stop the build if there are Python syntax errors or undefined names
-        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
+        # stop the build if there are Python syntax errors, undefined names,
+        # or trailing whitespaces
+        flake8 . --count --select=E9,F63,F7,F82,W291,W293 --show-source --statistics
         # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
         flake8 . --count --exit-zero --max-line-length=127 --statistics
diff --git a/CHANGES.md b/CHANGES.md
@@ -1,5 +1,10 @@
 # Changelog
 
+## 2021.9
+
+- Add basic support for running Jupyter Notebooks / Jupyter Lab, interactive
+  IPython sessions, and in the VS Code interactive Jupyter window.
+
 ## 2021.8
 
 - Retry downloads if a `ReadError` has occurred.

diff --git a/README.md b/README.md
@@ -19,6 +19,24 @@ conda install -c conda-forge openneuro-py
 pip install openneuro-py
 ```
 
+## Jupyter and IPython support
+
+To get basic support for Jupyter Lab, Jupyter Notebook, IPython interactive
+sessions, and VS Code's interactive Jupyter interface, you will also need to
+install `ipywidgets`:
+
+### via conda
+
+```shell
+conda install -c conda-forge ipywidgets
+```
+
+### via pip
+
+```shell
+pip install ipywidgets
+```
+
 ## Basic usage – Python interface
 
 ```python

diff --git a/openneuro/download.py b/openneuro/download.py
@@ -1,4 +1,5 @@
 import sys
+import os
 import fnmatch
 import hashlib
 import asyncio
@@ -13,7 +14,14 @@
 
 import requests
 import httpx
-from tqdm.asyncio import tqdm
+
+# Manually "enforce" notebook mode in VS Code to get progress bar widgets
+# Can be removed once https://github.com/tqdm/tqdm/issues/1213 has been merged
+if 'VSCODE_PID' in os.environ:
+    from tqdm.notebook import tqdm
+else:
+    from tqdm.auto import tqdm
+
 import click
 import aiofiles
 from sgqlc.endpoint.requests import RequestsEndpoint
@@ -22,11 +30,14 @@
 from .config import default_base_url
 
 
-try:
-    sys.stdout.reconfigure(encoding='utf-8')
+if sys.stdout.encoding.lower() == 'utf-8':
     stdout_unicode = True
-except AttributeError:
-    stdout_unicode = False
+else:
+    try:
+        sys.stdout.reconfigure(encoding='utf-8')
+        stdout_unicode = True
+    except AttributeError:
+        stdout_unicode = False
 
 
 # HTTP server responses that indicate hopefully intermittent errors that
@@ -42,9 +53,9 @@
     requests.exceptions.ConnectTimeout,
     requests.exceptions.ReadTimeout,
 
-    # "peer closed connection without sending complete message body 
+    # "peer closed connection without sending complete message body
     #  (incomplete chunked read)"
-    httpx.RemoteProtocolError  
+    httpx.RemoteProtocolError
 )
 
 # GraphQL endpoint and queries.
@@ -394,7 +405,7 @@ async def _retrieve_and_write_to_disk(
                   total=remote_file_size, unit='B',
                   unit_scale=True, unit_divisor=1024,
                   leave=False) as progress:
-            
+
             num_bytes_downloaded = response.num_bytes_downloaded
             # TODO Add timeout handling here, too.
             async for chunk in response.aiter_bytes():
@@ -634,7 +645,18 @@ def download(*,
                   max_retries=max_retries,
                   retry_backoff=retry_backoff,
                   max_concurrent_downloads=max_concurrent_downloads)
-    asyncio.run(_download_files(**kwargs))
+
+    # Try to re-use event loop if it already exists. This is required e.g.
+    # for use in Jupyter notebooks.
+    try:
+        loop = asyncio.get_running_loop()
+    except RuntimeError:
+        loop = None
+
+    if loop:
+        loop.create_task(_download_files(**kwargs))
+    else:
+        asyncio.run(_download_files(**kwargs))
 
     msg_finished = f'Finished downloading {dataset}.'
     if stdout_unicode:

diff --git a/setup.cfg b/setup.cfg
@@ -1,6 +1,6 @@
 [metadata]
 name = openneuro-py
-version = 2021.8
+version = 2021.9
 author = Richard Höchenberger <richard.hoechenberger@gmail.com>
 author_email = richard.hoechenberger@gmail.com
 url = https://github.com/hoechenberger/openneuro-py