Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

API server failed to start up #19601

Closed
tj13 opened this issue Jan 13, 2016 · 6 comments
Closed

API server failed to start up #19601

tj13 opened this issue Jan 13, 2016 · 6 comments

Comments

@tj13
Copy link

tj13 commented Jan 13, 2016

tried to config one master node following the guide (http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode.html#master-node)
script master.sh ran successfully, but the api server failed to boot up.

Software Version:

  • K8S_VERSION=1.1.3
  • ETCD_VERSION=2.2.1
  • FLANNEL_VERSION=0.5.5

OS Version:

VERSION="2015.09"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2015.09"
PRETTY_NAME="Amazon Linux AMI 2015.09"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2015.09:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"

Docker: 1.7.1

Kernel Version:
Linux ip-172-0-11-22 4.1.10-17.31.amzn1.x86_64 #1 SMP Sat Oct 24 01:31:37 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

log message of kubelet:

I0113 15:44:42.517777 7987 server.go:770] Started kubelet
E0113 15:44:42.517812 7987 kubelet.go:756] Image garbage collection failed: unable to find data for container /
E0113 15:44:42.518437 7987 event.go:197] Unable to write event: 'Post http://localhost:8080/api/v1/namespaces/default/events: dial tcp 127.0.0.1:8080: connection refused' (may retry after sleeping)
I0113 15:44:42.518460 7987 server.go:89] Starting to listen read-only on 0.0.0.0:10255
I0113 15:44:42.518885 7987 server.go:72] Starting to listen on 0.0.0.0:10250
I0113 15:44:42.524222 7987 kubelet.go:777] Running in container "/kubelet"
I0113 15:44:42.696510 7987 factory.go:239] Registering Docker factory
I0113 15:44:42.698516 7987 factory.go:93] Registering Raw factory
I0113 15:44:42.698837 7987 kubelet.go:2300] Recording NodeHasSufficientDisk event message for node localhost
I0113 15:44:42.698862 7987 kubelet.go:2300] Recording NodeReady event message for node localhost
I0113 15:44:42.698871 7987 kubelet.go:869] Attempting to register node localhost
I0113 15:44:42.699523 7987 kubelet.go:872] Unable to register localhost with the apiserver: Post http://localhost:8080/api/v1/nodes: dial tcp 127.0.0.1:8080: connection refused
I0113 15:44:42.829361 7987 manager.go:1006] Started watching for new ooms in manager
I0113 15:44:42.830001 7987 oomparser.go:183] oomparser using systemd
I0113 15:44:42.842667 7987 manager.go:250] Starting recovery of all containers
I0113 15:44:42.868829 7987 manager.go:255] Recovery completed
I0113 15:44:42.880876 7987 container_manager_linux.go:215] Configure resource-only container /docker-daemon with memory limit: 2903034265
I0113 15:44:42.880910 7987 manager.go:104] Starting to sync pod status with apiserver
I0113 15:44:42.880963 7987 kubelet.go:1960] Starting kubelet main sync loop.
I0113 15:44:42.881004 7987 kubelet.go:2012] SyncLoop (ADD): "k8s-master-localhost_default"
E0113 15:44:42.881457 7987 kubelet.go:1915] error getting node: node 'localhost' is not in cache
E0113 15:44:42.884752 7987 kubelet.go:1356] Failed creating a mirror pod "k8s-master-localhost_default": Post http://localhost:8080/api/v1/namespaces/default/pods: dial tcp 127.0.0.1:8080: connection refused
E0113 15:44:42.884780 7987 kubelet.go:1361] Mirror pod not available
I0113 15:44:42.884839 7987 manager.go:1707] Need to restart pod infra container for "k8s-master-localhost_default" because it is not found
W0113 15:44:42.885688 7987 manager.go:108] Failed to updated pod status: error updating status for pod "k8s-master-localhost_default": Get http://localhost:8080/api/v1/namespaces/default/pods/k8s-master-localhost: dial tcp 127.0.0.1:8080: connection refused
I0113 15:44:42.900665 7987 kubelet.go:2300] Recording NodeHasSufficientDisk event message for node localhost
I0113 15:44:42.900693 7987 kubelet.go:2300] Recording NodeReady event message for node localhost
I0113 15:44:42.900751 7987 kubelet.go:869] Attempting to register node localhost
I0113 15:44:42.901194 7987 kubelet.go:872] Unable to register localhost with the apiserver: Post http://localhost:8080/api/v1/nodes: dial tcp 127.0.0.1:8080: connection refused
I0113 15:44:42.977270 7987 provider.go:91] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
I0113 15:44:42.977458 7987 docker.go:159] Pulling image gcr.io/google_containers/pause:0.8.0 without credentials
I0113 15:44:43.302487 7987 kubelet.go:2300] Recording NodeHasSufficientDisk event message for node localhost
I0113 15:44:43.302552 7987 kubelet.go:2300] Recording NodeReady event message for node localhost
@feihujiang
Copy link
Contributor

Please re-post your question to stackoverflow.

We are trying to consolidate the channels to which questions for help/support are posted so that we can improve our efficiency in responding to your requests, and to make it easier for you to find answers to frequently asked questions and how to address common use cases.

We regularly see messages posted in multiple forums, with the full response thread only in one place or, worse, spread across multiple forums. Also, the large volume of support issues on github is making it difficult for us to use issues to identify real bugs.

The Kubernetes team scans stackoverflow on a regular basis, and will try to ensure your questions don't go unanswered.

Before posting a new question, please search stackoverflow for answers to similar questions, and also familiarize yourself with:

Again, thanks for using Kubernetes.

The Kubernetes Team

@feihujiang
Copy link
Contributor

You should also post log message of kube-apiserver.
And what's the output when you run commands kubectl version and kubectl get nodes?

@tj13
Copy link
Author

tj13 commented Jan 14, 2016

$ ./kubectl version
error: couldn't read version from server: Get http://localhost:8080/api: dial tcp 127.0.0.1:8080: connection refused

$ ./kubectl get nodes
error: couldn't read version from server: Get http://localhost:8080/api: dial tcp 127.0.0.1:8080: connection refused

@Pensu
Copy link
Contributor

Pensu commented Jan 14, 2016

Can you paste the log of kube-apiserver? It's hard to figure out what's wrong with apiserver with logs of kubelet!

@tj13
Copy link
Author

tj13 commented Jan 14, 2016

@Pensu @feihujiang
where can i find the kube-apiserver log?
below is the command line to run kubernete
docker run
--net=host
--pid=host
--privileged
--restart=always
-d
-v /sys:/sys:ro
-v /var/run:/var/run:rw
-v /:/rootfs:ro
-v /dev:/dev
-v /var/lib/docker/:/var/lib/docker:rw
-v /var/lib/kubelet/:/var/lib/kubelet:rw
index.alauda.cn/googlecontainer/hyperkube:v${K8S_VERSION}
/hyperkube kubelet
--address=0.0.0.0
--allow-privileged=true
--enable-server
--api-servers=http://localhost:8080
--config=/etc/kubernetes/manifests-multi
--cluster-dns=10.0.0.10
--cluster-domain=cluster.local
--containerized
--v=2

@tj13
Copy link
Author

tj13 commented Jan 15, 2016

issue got fixed by self. it's caused by the GFW which blocks internet access to gcr.io.

@tj13 tj13 closed this as completed Jan 15, 2016
fgrzadkowski added a commit to fgrzadkowski/kubernetes that referenced this issue Feb 25, 2016
@fgrzadkowski
Make kubernetes#19601 backward compatible.
6f6cb45 
Fixes kubernetes#20047
gmarek pushed a commit that referenced this issue Feb 25, 2016
Merge pull request #20710 from fgrzadkowski/backward_compatible_release
13d2364 
Make #19601 backward compatible.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@feihujiang @Pensu @tj13