Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to connect to my Hub at http://tf-hub-0:8081/hub/api (attempt 1/5). Is it running? #78

Closed
gangliao opened this issue Jan 3, 2018 · 7 comments
Closed

Failed to connect to my Hub at http://tf-hub-0:8081/hub/api (attempt 1/5). Is it running? #78

gangliao opened this issue Jan 3, 2018 · 7 comments

Comments

@gangliao
Copy link
Contributor

gangliao commented Jan 3, 2018 •
edited
Loading

image

usermod: no changes
Execute the command as jovyan
[W 2018-01-03 06:36:22.606 SingleUserNotebookApp configurable:168] Config option `open_browser` not recognized by `SingleUserNotebookApp`.  Did you mean `browser`?
/opt/conda/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
  return f(*args, **kwds)
[I 2018-01-03 06:36:24.227 SingleUserNotebookApp handlers:46] jupyter_tensorboard extension loaded.
[I 2018-01-03 06:36:24.257 SingleUserNotebookApp extension:38] JupyterLab alpha preview extension loaded from /opt/conda/lib/python3.6/site-packages/jupyterlab
[I 2018-01-03 06:36:24.258 SingleUserNotebookApp extension:88] Running the core application with no additional extensions or settings
[I 2018-01-03 06:36:24.262 SingleUserNotebookApp singleuser:365] Starting jupyterhub-singleuser server version 0.8.1
[I 2018-01-03 06:36:44.287 SingleUserNotebookApp log:122] 302 GET /user/admin/ → /user/admin/tree? (@172.17.0.6) 0.72ms
[I 2018-01-03 06:36:44.288 SingleUserNotebookApp log:122] 302 GET /user/admin/ → /user/admin/tree? (@172.17.0.6) 0.53ms
[E 2018-01-03 06:36:44.288 SingleUserNotebookApp singleuser:354] Failed to connect to my Hub at http://tf-hub-0:8081/hub/api (attempt 1/5). Is it running?
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.6/site-packages/jupyterhub/singleuser.py", line 351, in check_hub_version
@jlewi
Copy link
Contributor

jlewi commented Jan 3, 2018

What URL are you accessing Jupyter at?

How are you connecting to Jupyter on your cluster? Are you exposing Jupyter Hub on an external IP? Are you connecting using kubectl port-forward?

The url http://tf-hub-0:8081 looks suspect. tf-hub-0 is the name of the JupyterHub pod and service; if you're outside the cluster those won't map to valid hostnames. However, the fact that this is returning a 500 though and not a 400, I think indicates the name is being mapped to an IP.

Were you able to reach the Jupyter login page?

Did you try connecting via kubectl port-forward

kubectl port-forward tf-hub-0 8100:8000

And then accessing http://localhost:8100?

/cc @yuvipanda @foxish

@harryliu123
Copy link

i have same case
i thank question is not registered tf-hub-0 on kube-dns

root@XXXX:~# kubectl exec busybox -- nslookup tf-hub-0
Server: 10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

nslookup: can't resolve 'tf-hub-0'
command terminated with exit code 1

@foxish
Copy link
Contributor

foxish commented Jan 5, 2018

http://tf-hub-0:8081 should be an in-cluster resolvable DNS name that the jupyter notebooks pods use to connect back to the hub.
Can you verify that the headless service has been created, and that kube-dns is enabled?

@gangliao
Copy link
Contributor Author

gangliao commented Jan 6, 2018

Yes, the problem is from kube-dns, after it was reconfigured, kubeflow works well. Thank you! @foxish

@jlewi
Copy link
Contributor

jlewi commented Jan 7, 2018

@gangliao For posterity, what did you have to do to reconfigure kube-dns? Also how are you running Kubernetes?

@jlewi jlewi mentioned this issue Jan 8, 2018
Closed
@gangliao
Copy link
Contributor Author

gangliao commented Jan 9, 2018

@jlewi My case is quite special. I find Flannel is configured incorrectly, then I reconfigure it and also restart my kube-dns, finally http://tf-hub-0:8081 can be resolved.

@jlewi
Copy link
Contributor

jlewi commented Jan 9, 2018

Thanks.

@jlewi jlewi closed this as completed Jan 9, 2018
@xieydd xieydd mentioned this issue May 14, 2018
Closed
yanniszark pushed a commit to arrikto/kubeflow that referenced this issue Nov 1, 2019
@stpabhi @k8s-ci-robot
Notebook controller requires rbac to create istio virtualservices (ku…
8ddefcb 
…beflow#78)

Signed-off-by: Abhilash Pallerlamudi <stp.abhi@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jlewi @foxish @gangliao @harryliu123