Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

infra: automatically build and push docker image to storage #141

Open
imrehg opened this issue Jul 25, 2020 · 0 comments
Open

infra: automatically build and push docker image to storage #141

imrehg opened this issue Jul 25, 2020 · 0 comments
Assignees
Labels
enhancement New feature or request infrastructure Issues regarding the infrastructure setup

Comments

@imrehg
Copy link
Contributor

imrehg commented Jul 25, 2020

Describe the feature request

Currently Docker images for the task runner need to be manually built and pushed. Ideally it would be automatic (and across dev/staging/prod at the same time).

Affected product area (please put an X in all the brackets that apply)

[ ] Documentation
[ ] Data access
[x] Infrastructure
[ ] Loader pipeline
[ ] Other (please describe)

Additional context

This needs some figuring out, as the source of the images is 3 different docker repositories, and would be used differently. And it would be good to not rebuild but reuse images when possible, so we use the exact same version.

Eg.:

  • PR's build various images: sha + merge commit versions
  • Push to dev with those tags
  • Staging step tries to pull from the dev repository first, and if succeeds, retags and pushes to staging
  • After merge, the same thing happens (and retagging to prod) but across dev/staging/prod

Likely also needs some life-cycle rules to clean up old images and keep the space down.

This whole complexity could be avoided if using a single registry, but it would be either Github (which might be paid, and AWS would likely need a token to pull as images on Github don't seem to be publicly pullable), or Docker Hub (extra account), or something else (why else?).

In practice, that image doesn't change often (as is currently), so might not worth the added effort.

@imrehg imrehg added the enhancement New feature or request label Jul 25, 2020
@imrehg imrehg self-assigned this Jul 25, 2020
@imrehg imrehg added the infrastructure Issues regarding the infrastructure setup label Jul 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request infrastructure Issues regarding the infrastructure setup
Projects
None yet
Development

No branches or pull requests

1 participant