Description
This proposal is a combined from the previous discussion in #30110 #3056 #5369 #31067 (comment) #30449
, from @justincormack @duglin and many others.
I propose adding an extra flag --mount to the RUN command in Dockerfile. This would make sure that certain external resources are present while the build step runs. These mounts will not end up in the final image.
Build cache for every RUN would be calculated same way as multi-stage build COPY --from= is implemented. The builder would calculate a secure hash for the source directory and validate if it is the same as in the previous invocation. If something changes in the mounted directory the RUN command executes again.
Use cases:
Avoid copying over files that are just needed for a build command and don't need to be in the final image. Simple example is
COPY dir /
RUN go build -o d ./dir
In that case, the source does not need to be copied(and potentially deleted) into the image. Instead the process could use it directly: RUN --mount=src=/dst,destination=/dst,ro=true go build ....
Mounts can also appear from other build stages using the same logic as COPY --from.
FROM debian AS foo
...
FROM alpine
RUN --mount=src=/,target=foo,from=foo ..
This can also be used as a workaround for the COPY --chown issue as with mounting, no extra layer is created between copy and any other operation running in the container. chown isn't a primary use case for it though, and maybe should be supported separately as well.
Mounts from other images can only be read-only.
Optional additional use case: tmpfs
If a command needs a place to store temporary files it would be good if it could just set some directory as tmpfs instead of trying to clean up after itself and doing unnecessary disk accesses.
RUN --mount=target=foo,type=tmpfs.
In the future, we could allow more types. It is necessary though that we don't allow anything that isn't reproducible or uses some external resource.
Optional additional use case: shared build cache
There is a widely requested feature for providing a way to have shared cache directories between builder invocations. Usually, people would want to work around it by introducing -v to docker build but that has many downsides #31499 (comment)
A solution for that could be to allow exposing a special kind of mount that is mutable and can be automatically cleared by the builder.
FROM scratch AS cache-for-go
FROM golang
RUN --mount=target=/go/pkg,from=cache-for-go,type=cache go build
Now everything written to /go/pkg by go would be already there next time builder runs(unless user runs docker system prune). The cache ID could be calculated after the program has executed, or maybe, in this case, it would be users responsibility to make sure that all data in this directory is optional and cache would only be invalidated if commands in cache-for–go definition change. Finding previous cache directory would be based on session ID from client like in #31829
Activity