The builder currently uses an API that was designed more around the existing functions in daemon package.

The main problems with this interface are:

• It is bloated, contains 17 functions that are very specific to docker.
• It is not isolated. The builder uses image and containers by IDs directly it means that it conflicts with other API users. For example deleting an image or container while the builder is running would cause an undefined behavior. Deleting a container while the copy operation is running even causes EBUSY and leaks data.
• It is inefficient, creating containers and mounting filesystems for every operation, while many commands like ENV just change a single value in a configuration struct.
• It can't be used for zero-copy commit because docker commit requires duplicating data.

The proposed new interface would be:

// Backend abstracts a dependency interface for the builder.
type Backend interface {
    // GetImage returns an image config and rootFS. Used by `FROM` and `--from`
    GetImage(ctx context.Context, refOrID string, opt ImageGetOpt) (config []byte, l releaseableLayer, error)

    // Execute a process in a sandbox environment. Used by `RUN`
    Exec(ctx context.Context, config types.RunConfig, layer releaseableLayer, opt_stdio stdio) (releaseableLayer, error)

    // Register image in imagestore
    Export(ctx context.Context, config []byte, legacyParentID string) (id string, err error)

    // Create a debug container. This is called when the build fails instead of releasing the layer.
    Debug(ctx context.Context, config types.RunConfig, layer releaseableLayer) (string, error)
}

type releaseableLayer interface {
  Release() error
  Mount() (string, error)
  Unmount() (string, error)
}

type ImageGetOpt struct {
  ForcePull bool
  ConfigOnly bool
  Auth map[]
}

releaseableLayer maps directly to the layer.Layer. Builder holds a reference to a layer and snapshot of image config until it runs. That makes sure that if images are deleted it doesn't affect the builder and everything would be cleaned up after the builder has finished.

Export is separated from commit so no hacks are needed to make sure that config of a container is in a correct state for committing. COPY/ADD would be implemented directly with layer access.

This is a big change and would need to be split up into multiple PRs. I propose following intermediate steps(open for discussion of course):

• General refactoring in builder to split different features and make further work easier(@dnephin is working on this)
• Implement GetImage and replace current pull methods (PR open [Builder] Expose GetImage interface for builder #33054)
• Combine Create/Attach/Start/Wait into same function, don't change Commit logic yet. (PR open [Builder] Refactor builder probe cache and container backend #33061)
• Implement Export, don’t try to integrate yet (@dnephin [Builder] Move file coping from the daemon to the builder #33454)
• Temporarily use GetImage to access image config committed with Commit so that Export and Commit can be used in parallel (@dnephin [Builder] Move file coping from the daemon to the builder #33454)
• Expose releasableLayer into builder. Remove CopyOnBuild, commit with Export (@dnephin [Builder] Move file coping from the daemon to the builder #33454)
• Change commands that update metadata to use Export
• Implement Exec/Debug and remove current Commit functions

@dnephin @vdemeester @duglin @AkihiroSuda