api/services/task: add RuntimeInfo() #8509

AkihiroSuda · 2023-05-11T14:27:16Z

The RuntimeInfo() method returns the OCI Runtime features that are implemented by the runtime.

Expected to be used for:

KEP-3857: Recursive Read-only (RRO) mounts kubernetes/enhancements#3858

RPC

Request:

message RuntimeInfoRequest {
        string runtime_path = 1;
        // Options correspond to CreateTaskRequest.options
        google.protobuf.Any options = 2;
}

Response:

message RuntimeVersion {
        string version = 1;
        string revision = 2;
}

message RuntimeInfo {
        string name = 1;
        RuntimeVersion version = 2;
        // Options correspond to RuntimeInfoRequest.Options
        google.protobuf.Any options = 3;
        // OCI-compatible runtimes should use https://github.com/opencontainers/runtime-spec/blob/main/features.md
        google.protobuf.Any features = 4;
        // Annotations of the shim. Irrelevant to features.Annotations.
        map<string, string> annotations = 5;
}

Examples

Use ctr task runtime-info to test this.

Default (runc)

$ sudo ctr task runtime-info                                                                                                                                                                                                                                       
{
    "Name": "io.containerd.runc.v2",
    "Version": {
        "Version": "v1.7.0-231-g36b30b3bb",
        "Revision": "v1.7.0-231-g36b30b3bb"
    },
    "Options": {},
    "Features": {
        "ociVersionMin": "1.0.0",
        "ociVersionMax": "1.1.0-rc.2",
        "hooks": [
            "prestart",
            "createRuntime",
            "createContainer",
            "startContainer",
            "poststart",
            "poststop"
        ],
        "mountOptions": [
            "async",
            "atime",
            "bind",
            "defaults",
            "dev",
            "diratime",
            "dirsync",
            "exec",
            "iversion",
            "lazytime",
            "loud",
            "mand",
            "noatime",
            "nodev",
            "nodiratime",
            "noexec",
            "noiversion",
            "nolazytime",
            "nomand",
            "norelatime",
            "nostrictatime",
            "nosuid",
            "nosymfollow",
            "private",
            "ratime",
            "rbind",
            "rdev",
            "rdiratime",
            "relatime",
            "remount",
            "rexec",
            "rnoatime",
            "rnodev",
            "rnodiratime",
            "rnoexec",
            "rnorelatime",
            "rnostrictatime",
            "rnosuid",
            "rnosymfollow",
            "ro",
            "rprivate",
            "rrelatime",
            "rro",
            "rrw",
            "rshared",
            "rslave",
            "rstrictatime",
            "rsuid",
            "rsymfollow",
            "runbindable",
            "rw",
            "shared",
            "silent",
            "slave",
            "strictatime",
            "suid",
            "symfollow",
            "sync",
            "tmpcopyup",
            "unbindable"
        ],
        "linux": {
            "namespaces": [
                "cgroup",
                "ipc",
                "mount",
                "network",
                "pid",
                "user",
                "uts"
            ],
            "capabilities": [
                "CAP_CHOWN",
                "CAP_DAC_OVERRIDE",
                "CAP_DAC_READ_SEARCH",
                "CAP_FOWNER",
                "CAP_FSETID",
                "CAP_KILL",
                "CAP_SETGID",
                "CAP_SETUID",
                "CAP_SETPCAP",
                "CAP_LINUX_IMMUTABLE",
                "CAP_NET_BIND_SERVICE",
                "CAP_NET_BROADCAST",
                "CAP_NET_ADMIN",
                "CAP_NET_RAW",
                "CAP_IPC_LOCK",
                "CAP_IPC_OWNER",
                "CAP_SYS_MODULE",
                "CAP_SYS_RAWIO",
                "CAP_SYS_CHROOT",
                "CAP_SYS_PTRACE",
                "CAP_SYS_PACCT",
                "CAP_SYS_ADMIN",
                "CAP_SYS_BOOT",
                "CAP_SYS_NICE",
                "CAP_SYS_RESOURCE",
                "CAP_SYS_TIME",
                "CAP_SYS_TTY_CONFIG",
                "CAP_MKNOD",
                "CAP_LEASE",
                "CAP_AUDIT_WRITE",
                "CAP_AUDIT_CONTROL",
                "CAP_SETFCAP",
                "CAP_MAC_OVERRIDE",
                "CAP_MAC_ADMIN",
                "CAP_SYSLOG",
                "CAP_WAKE_ALARM",
                "CAP_BLOCK_SUSPEND",
                "CAP_AUDIT_READ",
                "CAP_PERFMON",
                "CAP_BPF",
                "CAP_CHECKPOINT_RESTORE"
            ],
            "cgroup": {
                "v1": true,
                "v2": true,
                "systemd": true,
                "systemdUser": true,
                "rdma": true
            },
            "seccomp": {
                "enabled": true,
                "actions": [
                    "SCMP_ACT_ALLOW",
                    "SCMP_ACT_ERRNO",
                    "SCMP_ACT_KILL",
                    "SCMP_ACT_KILL_PROCESS",
                    "SCMP_ACT_KILL_THREAD",
                    "SCMP_ACT_LOG",
                    "SCMP_ACT_NOTIFY",
                    "SCMP_ACT_TRACE",
                    "SCMP_ACT_TRAP"
                ],
                "operators": [
                    "SCMP_CMP_EQ",
                    "SCMP_CMP_GE",
                    "SCMP_CMP_GT",
                    "SCMP_CMP_LE",
                    "SCMP_CMP_LT",
                    "SCMP_CMP_MASKED_EQ",
                    "SCMP_CMP_NE"
                ],
                "archs": [
                    "SCMP_ARCH_AARCH64",
                    "SCMP_ARCH_ARM",
                    "SCMP_ARCH_MIPS",
                    "SCMP_ARCH_MIPS64",
                    "SCMP_ARCH_MIPS64N32",
                    "SCMP_ARCH_MIPSEL",
                    "SCMP_ARCH_MIPSEL64",
                    "SCMP_ARCH_MIPSEL64N32",
                    "SCMP_ARCH_PPC",
                    "SCMP_ARCH_PPC64",
                    "SCMP_ARCH_PPC64LE",
                    "SCMP_ARCH_RISCV64",
                    "SCMP_ARCH_S390",
                    "SCMP_ARCH_S390X",
                    "SCMP_ARCH_X32",
                    "SCMP_ARCH_X86",
                    "SCMP_ARCH_X86_64"
                ],
                "knownFlags": [
                    "SECCOMP_FILTER_FLAG_TSYNC",
                    "SECCOMP_FILTER_FLAG_SPEC_ALLOW",
                    "SECCOMP_FILTER_FLAG_LOG"
                ],
                "supportedFlags": [
                    "SECCOMP_FILTER_FLAG_TSYNC",
                    "SECCOMP_FILTER_FLAG_SPEC_ALLOW",
                    "SECCOMP_FILTER_FLAG_LOG"
                ]
            },
            "apparmor": {
                "enabled": true
            },
            "selinux": {
                "enabled": true
            },
            "intelRdt": {
                "enabled": true
            }
        },
        "annotations": {
            "io.github.seccomp.libseccomp.version": "2.5.4",
            "org.opencontainers.runc.checkpoint.enabled": "true",
            "org.opencontainers.runc.commit": "v1.1.0-534-g26851168",
            "org.opencontainers.runc.version": "1.1.0+dev"
        }
    },
    "Annotations": null
}

crun (prior to v1.8.6)

The Features property is null because crun features was not implemented until crun v1.8.6

$ sudo ctr task runtime-info --runc-binary=crun
{
    "Name": "io.containerd.runc.v2",
    "Version": {
        "Version": "v1.7.0-231-g36b30b3bb",
        "Revision": "v1.7.0-231-g36b30b3bb"
    },
    "Options": {
        "binary_name": "crun"
    },
    "Features": null,
    "Annotations": null
}

`io.containerd.runsc.v1`

Fails because containerd-shim-runsc-v1 -info is not implemented yet

$ sudo ctr task runtime-info --runtime=io.containerd.runsc.v1
ctr: rpc error: code = Unknown desc = failed to get runtime info for "io.containerd.runsc.v1": failed to run [/usr/local/bin/containerd-shim-runsc-v1 -info]: exit status 2 (stderr: "flag provided but not defined: -info\nUsage of /usr/local/bin/containerd-shim-runsc-v1:\n  -address string\n    \tgrpc address back to main containerd\n  -bundle string\n    \tpath to the bundle if not workdir\n  -debug\n    \tenable debug output in logs\n  -id string\n    \tid of the task\n  -namespace string\n    \tnamespace that owns the shim\n  -publish-binary string\n    \tpath to publish binary (used for publishing events) (default \"containerd\")\n  -socket string\n    \tsocket path to serve\n  -v\tshow the shim version and exit\n")

runtime/v2/runc/manager/manager_linux.go

runtime/v2/shim/shim.go

mikebrow

LGTM
noting that for k8s they would likely need to use the CRI api ContainerStatus() .. see crictl info for current example of verbose output. Both apis should return the same information.

AkihiroSuda · 2023-05-29T16:45:25Z

LGTM noting that for k8s they would likely need to use the CRI api ContainerStatus() .. see crictl info for current example of verbose output. Both apis should return the same information.

In the CRI API, the runtime features are intended to be added to the StatusResponse message like this:
https://github.com/kubernetes/enhancements/pull/3858/files#diff-a5e3a174567e889120ab32af5f159c2bcddd459cc36abdf56f3eceb5ad86d6c5R446

Iceber · 2023-05-30T07:06:57Z

As a feature that runtime will call, the documentation can be updated to suggest that shim developers implement this flag
https://github.com/containerd/containerd/tree/main/runtime/v2#shim-authoring

AkihiroSuda · 2023-05-30T09:54:10Z

documentation

Added

runtime/v2/runc/manager/manager_linux.go

runtime/v2/manager.go

mikebrow · 2023-06-12T20:41:46Z

LGTM noting that for k8s they would likely need to use the CRI api .edited typo. Status() .. see crictl info for current example of verbose output. Both apis should return the same information.

In the CRI API, the runtime features are intended to be added to the StatusResponse message like this: https://github.com/kubernetes/enhancements/pull/3858/files#diff-a5e3a174567e889120ab32af5f159c2bcddd459cc36abdf56f3eceb5ad86d6c5R446

nod though it's called info command in crictl the api is cri Status() with StatusResponse message

AkihiroSuda · 2023-06-15T17:46:41Z

@dmcgowan PTAL

AkihiroSuda · 2023-09-05T20:12:52Z

Rebased

AkihiroSuda · 2023-09-05T21:29:53Z

/retest

runtime/v2/runc/manager/manager_linux.go

integration/client/client_test.go

dmcgowan · 2023-09-06T21:04:05Z

Could we consider adding this function to a new service definition rather than to task v1. We have rejected changes like this in the past because v1 task definition is stable and no plans for a task v2. I think we already need something like a runtime introspection API.

AkihiroSuda · 2023-09-07T00:51:33Z

Could we consider adding this function to a new service definition rather than to task v1. We have rejected changes like this in the past because v1 task definition is stable and no plans for a task v2. I think we already need something like a runtime introspection API.

I’m not sure. I don’t feel the runtime info is decouplable from the task service.

rata · 2023-09-08T14:42:54Z

client.go

+	Annotations map[string]string
+}
+
+func (c *Client) RuntimeInfo(ctx context.Context, runtimePath string, runtimeOptions interface{}) (*RuntimeInfo, error) {


I'm not sure at which layer, but it would be nice to cache this info instead of running it every time.

If we require a containerd restart when you upgrade runc, it seems like a reasonable trade off IMHO.

What do you think?

I'd say caching is the caller's responsibility

Sure, not here in the client. But I mean, probably the runc manager can cache it, right? Or at some other layer we are modifying here?

Runc binary can be updated without restarting the daemon, so I don’t think the daemon should cache the info.

AkihiroSuda · 2023-09-13T01:19:33Z

@dmcgowan What should we do with this?
Not sure how the runtime info can be decoupled from the task service.

Use `ctr task runtime-info` to test this. Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

AkihiroSuda · 2023-10-05T17:04:04Z

Rebased

AkihiroSuda · 2023-10-05T17:49:08Z

TODO: consider moving this to https://github.com/containerd/containerd/blob/main/api/services/introspection/v1/introspection.proto

k8s-ci-robot · 2023-11-29T05:40:03Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

AkihiroSuda · 2023-11-30T04:32:12Z

New PR:

Add PluginInfo to introspection API #9442

Moved the method from the task service to the introspection service.

AkihiroSuda added the impact/changelog label May 11, 2023

AkihiroSuda added this to the 2.0 milestone May 11, 2023

AkihiroSuda force-pushed the runtime-info branch 3 times, most recently from 9fcd53f to c9a386d Compare May 11, 2023 15:53

AkihiroSuda mentioned this pull request May 11, 2023

Support recursively read-only (RRO) mounts moby/moby#45278

Merged

AkihiroSuda requested a review from dmcgowan May 16, 2023 03:02

Iceber reviewed May 17, 2023

View reviewed changes

runtime/v2/runc/manager/manager_linux.go Show resolved Hide resolved

Iceber reviewed May 17, 2023

View reviewed changes

runtime/v2/shim/shim.go Show resolved Hide resolved

dmcgowan added the status/needs-discussion Needs discussion and decision from maintainers label May 17, 2023

mikebrow approved these changes May 26, 2023

View reviewed changes

AkihiroSuda mentioned this pull request May 30, 2023

KEP-3857: Recursive Read-only (RRO) mounts kubernetes/enhancements#3858

Merged

AkihiroSuda force-pushed the runtime-info branch from c9a386d to baf18a8 Compare May 30, 2023 09:54

Iceber reviewed May 30, 2023

View reviewed changes

runtime/v2/runc/manager/manager_linux.go Show resolved Hide resolved

AkihiroSuda force-pushed the runtime-info branch from baf18a8 to 8d6a5c4 Compare May 30, 2023 10:07

Iceber reviewed May 30, 2023

View reviewed changes

runtime/v2/manager.go Show resolved Hide resolved

AkihiroSuda force-pushed the runtime-info branch 2 times, most recently from 4eba2c4 to 8aa955f Compare September 5, 2023 20:11

AkihiroSuda mentioned this pull request Sep 5, 2023

Add support for userns in stateless and stateful pods with idmap mounts (KEP-127, k8s >= 1.27) #8287

Merged

3 tasks

rata reviewed Sep 6, 2023

View reviewed changes

runtime/v2/runc/manager/manager_linux.go Outdated Show resolved Hide resolved

rata reviewed Sep 6, 2023

View reviewed changes

integration/client/client_test.go Outdated Show resolved Hide resolved

AkihiroSuda force-pushed the runtime-info branch from 8aa955f to cf291ec Compare September 7, 2023 04:07

rata reviewed Sep 8, 2023

View reviewed changes

api/services/task: add RuntimeInfo()

08f8bcb

Use `ctr task runtime-info` to test this. Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

AkihiroSuda force-pushed the runtime-info branch from cf291ec to 08f8bcb Compare October 5, 2023 17:03

AkihiroSuda marked this pull request as draft October 5, 2023 17:49

k8s-ci-robot added the needs-rebase label Nov 29, 2023

AkihiroSuda mentioned this pull request Nov 30, 2023

Add PluginInfo to introspection API #9442

Merged

AkihiroSuda closed this Nov 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

api/services/task: add RuntimeInfo() #8509

api/services/task: add RuntimeInfo() #8509

AkihiroSuda commented May 11, 2023 •

edited

Loading

mikebrow left a comment

AkihiroSuda commented May 29, 2023

Iceber commented May 30, 2023 •

edited

Loading

AkihiroSuda commented May 30, 2023

mikebrow commented Jun 12, 2023

AkihiroSuda commented Jun 15, 2023

AkihiroSuda commented Sep 5, 2023

AkihiroSuda commented Sep 5, 2023

dmcgowan commented Sep 6, 2023

AkihiroSuda commented Sep 7, 2023

rata Sep 8, 2023

AkihiroSuda Sep 9, 2023

rata Sep 11, 2023

AkihiroSuda Sep 12, 2023

AkihiroSuda commented Sep 13, 2023

AkihiroSuda commented Oct 5, 2023

AkihiroSuda commented Oct 5, 2023

k8s-ci-robot commented Nov 29, 2023

AkihiroSuda commented Nov 30, 2023

api/services/task: add RuntimeInfo() #8509

api/services/task: add RuntimeInfo() #8509

Conversation

AkihiroSuda commented May 11, 2023 • edited Loading

RPC

Examples

Default (runc)

crun (prior to v1.8.6)

io.containerd.runsc.v1

mikebrow left a comment

Choose a reason for hiding this comment

AkihiroSuda commented May 29, 2023

Iceber commented May 30, 2023 • edited Loading

AkihiroSuda commented May 30, 2023

mikebrow commented Jun 12, 2023

AkihiroSuda commented Jun 15, 2023

AkihiroSuda commented Sep 5, 2023

AkihiroSuda commented Sep 5, 2023

dmcgowan commented Sep 6, 2023

AkihiroSuda commented Sep 7, 2023

rata Sep 8, 2023

Choose a reason for hiding this comment

AkihiroSuda Sep 9, 2023

Choose a reason for hiding this comment

rata Sep 11, 2023

Choose a reason for hiding this comment

AkihiroSuda Sep 12, 2023

Choose a reason for hiding this comment

AkihiroSuda commented Sep 13, 2023

AkihiroSuda commented Oct 5, 2023

AkihiroSuda commented Oct 5, 2023

k8s-ci-robot commented Nov 29, 2023

AkihiroSuda commented Nov 30, 2023

AkihiroSuda commented May 11, 2023 •

edited

Loading

`io.containerd.runsc.v1`

Iceber commented May 30, 2023 •

edited

Loading