Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release tarballs are too big #28435

Closed
3 of 7 tasks
jbeda opened this issue Jul 3, 2016 · 35 comments
Closed
3 of 7 tasks

Release tarballs are too big #28435

jbeda opened this issue Jul 3, 2016 · 35 comments
Assignees
Labels
area/build-release area/provider/gcp Issues or PRs related to gcp provider area/release-eng Issues or PRs related to the Release Engineering subproject kind/feature Categorizes issue or PR as related to a new feature. lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/release Categorizes an issue or PR as relevant to SIG Release.

Comments

@jbeda
Copy link
Contributor

jbeda commented Jul 3, 2016

The Kubernetes 1.3 release tarball is 1.4G. That is insane. While we know that much of this is due to the fact that we include all architectures, it make kubernetes feel heavier than it is.

Much of the space is the fact that 4 server platforms are now created and packaged with everything else. Each target is a ~325M tarball. These tarballs include both individual binaries along with Docker images that include those binaries. They include kubectl (which is available naked in the release).

We also have 9 copies of kubectl for various targets totaling 450MB.

Things we can do to slim this down:

Also related: #27444 (Builds are taking too long)

@spxtr
Copy link
Contributor

spxtr commented Jul 3, 2016

$ du -sh server/bin/*
99M     server/bin/federation-apiserver
4.0K    server/bin/federation-apiserver.docker_tag
100M    server/bin/federation-apiserver.tar
57M     server/bin/federation-controller-manager
4.0K    server/bin/federation-controller-manager.docker_tag
58M     server/bin/federation-controller-manager.tar
132M    server/bin/hyperkube
105M    server/bin/kube-apiserver
4.0K    server/bin/kube-apiserver.docker_tag
106M    server/bin/kube-apiserver.tar
95M     server/bin/kube-controller-manager
4.0K    server/bin/kube-controller-manager.docker_tag
97M     server/bin/kube-controller-manager.tar
54M     server/bin/kubectl
48M     server/bin/kube-dns
103M    server/bin/kubelet
101M    server/bin/kubemark
49M     server/bin/kube-proxy
4.0K    server/bin/kube-proxy.docker_tag
178M    server/bin/kube-proxy.tar
57M     server/bin/kube-scheduler
4.0K    server/bin/kube-scheduler.docker_tag
58M     server/bin/kube-scheduler.tar

If the point of the release is for end-users to turn up a kubernetes cluster, do they even need the binaries? Why not just give them the image?

I also think it makes sense to split the release up by server architectures.

@david-mcmahon

@justinsb
Copy link
Member

justinsb commented Jul 3, 2016

Some previous discussion here: #9253 Apparently some of the duplication is/was for CoreOS?

Although kube-up uses this tarball, there's no need to: the expanded files are also available for direct download (kubectl, kubelet), and gcr.io has all the images.

@luxas
Copy link
Member

luxas commented Jul 4, 2016

I'm very interested in this and have done some (small) improvements already.

hyperkube admin xxx

Do you have a proposal for which commands admin would have? I'm a huge +1 on a such tool.
That would remove some of the duplication from various deployments in favor for a canonical one, written in go.

Reevaluate all of our targets. Do we really need 32bit Windows?

I've been thinking about if we should deprecate 386 builds in v1.4 and remove them in v1.5.
If we're doing other dramatic changes (like the hyperkube transition), then we might remove them now already.

Why are the federation targets new binaries? They add 150M uncompressed for amd64. (I don't know enough to say here but could these be folded in to hyperkube? I'm guessing they could.)

The federation binaries are already inside hyperkube. The individual binaries was likely added in order to comply with current standards (separate binary + bundle in hyperkube)
cc @kubernetes/sig-cluster-federation

Don't ship binaries that we don't really need. We included kubemark here. No end users will actually use that. This is 100M uncompressed.

This has also been brought up before, I think. I'd prefer moving it to a test target.
cc @kubernetes/sig-testing

@kubernetes/sig-cluster-lifecycle folks might also be interested in this.

@david-mcmahon
Copy link
Contributor

david-mcmahon commented Jul 6, 2016

cc @roberthbailey

@roberthbailey
Copy link
Contributor

So I just came across this (while downloading the 1.3.0 binaries) and I agree that this is completely ridiculous. We need an owner for this issue for 1.4.

@thockin
Copy link
Member

thockin commented Jul 7, 2016

@david-mcmahon Can we stick you with this for 1.4?

@alex-mohr
Copy link
Contributor

I think this falls under @david-mcmahon and the release infra area, so proactively assigning -- David, let us know if you disagree.

I think this is also interesting to install, because downloading a 1.4GB tarball that extracts to 3+GB is a relatively sad install process.

There may be short-term expedient things we could do, e.g. building multiple tar balls with only linux x86-64 or srcs while still supporting the one-big-ball, in addition to hyperkube-style minified builds.

@mikedanese
Copy link
Member

We explode those tars in GCS. We should document how to pull specific pieces of the release.

https://gist.github.com/mikedanese/d3eed580c7f0a0e5c13534220f88eb5a

@david-mcmahon
Copy link
Contributor

david-mcmahon commented Jul 7, 2016

This looks like a huge meta-issue with several solutions affecting many areas of the product.

If we all agree on @jbeda 's suggested solutions, each should be broken out into their own bug (grouped where appropriate) and assigned.

Many of these solutions potentially have product/user/usability ramifications and need the attention of the area owner(s).

I'll make a first pass at the original list. Others feel free to add/update.

@david-mcmahon
Copy link
Contributor

I am unlikely to have the bandwidth to drive this further this quarter or before 1.4. Area leads should be assigning the subtasks @jbeda identified and I filed issues for to get this done before 1.4. @thockin @mikedanese @davidopp @luxas @quinton-hoole @alex-mohr. Please cc anyone I've missed.

@david-mcmahon david-mcmahon removed their assignment Jul 22, 2016
k8s-github-robot pushed a commit that referenced this issue Oct 27, 2016
Automatic merge from submit-queue

Centos: download client and server tarballs instead of mondo-tarball

Part of #28629 / #28435.

This should be functionally the same, except that you will download ~1/3 the bytes.
k8s-github-robot pushed a commit that referenced this issue Oct 28, 2016
…tes-full-tarball

Automatic merge from submit-queue

Stop including arch-specific binaries in kubernetes.tar.gz

**What this PR does / why we need it**:
This PR stops including the kubernetes client and server tarballs in `kubernetes.tar.gz`. These should instead be downloaded separately for only the required architectures.

The `cluster/get-kube.sh` (https://get.k8s.io) and `cluster/get-kube-binaries.sh` handle downloading and extracting the necessary binaries for you automatically. Jenkins also uses this workflow now.

Anyone still expecting binaries in kubernetes.tar.gz will be broken, but I've at least fixed all occurrences in-tree.

**Which issue this PR fixes**: fixes #28629, chips away at #28435

**Release note**:
```release-note
Arch-specific server and client binaries are no longer included in `kubernetes.tar.gz`. You must manually download the necessary architecture-specific tarballs or use `kubernetes/cluster/get-kube-binaries.sh` to download them for you.
```

cc @luxas @zmerlynn @justinsb @mikedanese @thockin @spxtr
@dims
Copy link
Member

dims commented Nov 16, 2016

@ixdy work on this will move to 1.6? (This needs to be triaged as a release-blocker or not for 1.5)

1 similar comment
@dims
Copy link
Member

dims commented Nov 16, 2016

@ixdy work on this will move to 1.6? (This needs to be triaged as a release-blocker or not for 1.5)

@luxas luxas modified the milestones: v1.6, v1.5 Nov 16, 2016
@luxas
Copy link
Member

luxas commented Nov 16, 2016

If I understand correctly, @ixdy has already made some improvements by splitting the monotarball out to one tarball per arch.

But I think there's work left to do, so moving to v1.6

@ethernetdan
Copy link
Contributor

@ixdy can this be closed?

@liggitt
Copy link
Member

liggitt commented Mar 14, 2017

pre-existing for 1.6, not a release blocker

@liggitt liggitt modified the milestones: v1.7, v1.6 Mar 14, 2017
@roberthbailey roberthbailey modified the milestones: v1.8, v1.7 May 30, 2017
@roberthbailey roberthbailey added sig/release Categorizes an issue or PR as relevant to SIG Release. and removed sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. labels Jun 13, 2017
@k8s-github-robot
Copy link

[MILESTONENOTIFIER] Milestone Labels Incomplete

@ixdy @jbeda

Action required: This issue requires label changes. If the required changes are not made within 6 days, the issue will be moved out of the v1.8 milestone.

kind: Must specify at most one of ['kind/bug', 'kind/feature', 'kind/cleanup'].

Additional instructions available here

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 7, 2018
@roberthbailey
Copy link
Contributor

Is anyone from @kubernetes/sig-release-feature-requests working on this feature request?

@k8s-ci-robot k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Jan 8, 2018
@enisoc
Copy link
Member

enisoc commented Jan 8, 2018

Since 1.6 or so, we've had separate per-arch tarballs, and the main one is only a few MB with a script to download only the arch you're running on. I haven't heard any complaints about tarball size since then.

@roberthbailey
Copy link
Contributor

Should we close this then?

@jbeda
Copy link
Contributor Author

jbeda commented Jan 8, 2018

Close enough. I'm closing it. Victory is ours!

@jbeda jbeda closed this as completed Jan 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build-release area/provider/gcp Issues or PRs related to gcp provider area/release-eng Issues or PRs related to the Release Engineering subproject kind/feature Categorizes issue or PR as related to a new feature. lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/release Categorizes an issue or PR as relevant to SIG Release.
Projects
None yet
Development

No branches or pull requests