Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable use of private Docker images #1699

Closed
asthomasdk opened this issue Aug 11, 2020 · 14 comments · Fixed by #8550
Closed

Enable use of private Docker images #1699

asthomasdk opened this issue Aug 11, 2020 · 14 comments · Fixed by #8550
Labels
feature: workspace image build meta: never-stale This issue can never become stale team: workspace Issue belongs to the Workspace team type: feature request New feature or request

Comments

@asthomasdk
Copy link

Please add support for private docker registries in Gitpod - especially the self-hosted version.

We use a lot of private Docker images in our various stacks - also for development.

Not being able to set this up for Gitpod means that we are unable to configure Gitpod workspaces for anything but simple repo editing.

I would expect that for the self-hosted version, using secrets or docker credentials from kubernetes would be a good option to support this.

@svenefftinge
Copy link
Member

For self-hosted you can use your own docker registry: https://www.gitpod.io/docs/self-hosted/latest/install/docker-registry/

@asthomasdk
Copy link
Author

Can this be set up to use the internal docker registry for workspace image builds (default), but also have access to the external docker registries we have our images in (Docker Hub and Gitlab) ?

I would expect that if the docker credentials in the kubernetes secret are correctly created, it would contain logins to multiple registries and work as long as the correct secret is referenced.

@geropl
Copy link
Member

geropl commented Aug 11, 2020

Hi @asthomasdk ,

this is not well (yet) supported in the config surface, but the code should be capable of doing so. Let's see if we can make it work (I assume you have a working config derived from the Gitpod self-hosted repo)

  1. gitpod.components.server.defaultBaseImageRegistryWhitelist needs a list of registry domains you want (all) your users to have access to
  2. The file registry-auth.json in the secret image-builder-registry-secret (referenced in values/registry.yaml) has to contain the docker secrets for these registries. The file looks like this:
{
	"auths": {
		"gcr.io": {
			"auth": "`+auth+`"
		}
	}
}	

Just add entries for your domains. There are several possibillites for the auth part:

  • use a JSON key (for Googles gcr.io) like in the Go script for Google Cloud
  • use the ~/.docker/config.json after you did docker login (most likely tight to your personal account!)
    But this depends a bit on the registries you use.

This is not easy for sure or straigth forward and we're working on streamlining things. But it's the best with the state right now.

@stale
Copy link

stale bot commented Oct 4, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the meta: stale This issue/PR is stale and will be closed soon label Oct 4, 2020
@stale stale bot closed this as completed Mar 19, 2021
@akosyakov akosyakov added meta: never-stale This issue can never become stale and removed meta: stale This issue/PR is stale and will be closed soon labels Jun 27, 2021
@akosyakov
Copy link
Member

It was requested in the community again: https://community.gitpod.io/t/using-private-docker-images-in-dockerfile/4001/2

@akosyakov akosyakov reopened this Jun 27, 2021
@asthomasdk
Copy link
Author

Any news or development thoughts on this?

I have managed to work around this by adding envrionment variables through my profile in gitpod. This is hack though, as it is very unsecure.

It would make sense to have custom docker registries that can be setup and managed just like the integrations to external git repos. Having to manually create secrets in kubernetes is a little clumsy ;-)

@asthomasdk
Copy link
Author

Hi @asthomasdk ,

this is not well (yet) supported in the config surface, but the code should be capable of doing so. Let's see if we can make it work (I assume you have a working config derived from the Gitpod self-hosted repo)

  1. gitpod.components.server.defaultBaseImageRegistryWhitelist needs a list of registry domains you want (all) your users to have access to
  2. The file registry-auth.json in the secret image-builder-registry-secret (referenced in values/registry.yaml) has to contain the docker secrets for these registries. The file looks like this:
{
	"auths": {
		"gcr.io": {
			"auth": "`+auth+`"
		}
	}
}	

Just add entries for your domains. There are several possibillites for the auth part:

  • use a JSON key (for Googles gcr.io) like in the Go script for Google Cloud
  • use the ~/.docker/config.json after you did docker login (most likely tight to your personal account!)
    But this depends a bit on the registries you use.

This is not easy for sure or straigth forward and we're working on streamlining things. But it's the best with the state right now.

Reading this again - does that mean that if I have the image-builder-registry-secret correctkly created, that this will be used generally by gitpod to access external registries? If that is the case, then it could mean that I can get references in my .gitpod.yml to a private image to work. Right now this has to be a public image for this to work.

@Davidffry
Copy link

And not for the self-hosted environment ?

Thanks ;-)

@kolloch
Copy link

kolloch commented Jan 7, 2022

We would love to have support for this cloud version. I am not sure if I should add an extra issue for this since this was labeled with "self-hosted".

Optimally, the integration would work for the pre-builds. gitlab registry would work for us or ECR.

As a workaround, being able to inject private environment variables into the prebuilt might help -- but it is not ideal.

@jmls
Copy link

jmls commented Jan 21, 2022

we also need this, quite urgently fopr the cloud version.

We have to use some private libraries & images so at the moment the prebuild stage does the docker login, then pulls the images and copies the required folders from the image into the workspace.

This is extremely inefficient , as we have 2x the required disk space (the docker image and the folder) but also slows down the prebuild.

It also means that we can't take advantage of the docker layers for efficiency

this could be implemented either as

  1. "docker repositories" section in the gitpod file,
  2. "docker repositories" section in either the team or user dasbboard
  3. a username:password (using gitpod variables) in the image name
    image: user@password:gitpod/workspace-full

@svenefftinge svenefftinge added team: workspace Issue belongs to the Workspace team and removed self-hosted labels Jan 21, 2022
@yevon
Copy link

yevon commented Jan 26, 2022

Gitpod installer makes references to this as if that was already supported in the Container Registry section. I was trying to set up this againts a gitlab repository but it is not working.

@clouedoc
Copy link

I need to use a development Docker image derived from our production Docker image to stay as close as possible to the production environment.
Our production containers contain intellectual property, so we can't make them public.
The lack of this feature is currently blocking us from using Gitpod.

@csweichel
Copy link
Contributor

We could make image builds aware of private registries. For example, we could introduce a special environment variable (on user and project level), e.g. GITPOD_REGISTRY_AUTH which carries essentially a dockerconfig json. During an image build these credentials would be available, so that one could build from a Dockerfile much like:

FROM your-private-registry.com/super-secret:image

This would NOT enable a .gitpod.yml like:

image: your-private-registry.com/super-secret:image

It would however, enable the use of private images.

As an addendum, we could merge the credentials provided this way with the docker config in regular workspaces, so that Docker in regular workspaces (and prebuilds) would gain access to that private registry.

@jmls
Copy link

jmls commented Mar 2, 2022

that would work for us

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature: workspace image build meta: never-stale This issue can never become stale team: workspace Issue belongs to the Workspace team type: feature request New feature or request
Projects
No open projects
Archived in project
Development

Successfully merging a pull request may close this issue.