Release Summary

The quark container v0.2.0 release includes adding AMD64 support to quark container, supporting UDP over RDMA, adding Egress and Ingress over Kubernetes over RDMA.

Key Features and Improvements

1. AMD64 support

• Add AMD64 support for Quark in addition to X86-64.
• Test environment: AMD Ryzen 9 6900HX

2. UDP over RDMA

• k8s can run pods using Quark runtime RDMA network
• Quark in k8s supports both TCP and UDP

3. Egress and Ingress networking in k8s RDMA network

• Add Egress gateway for container. Containers can visit external IPs.
• Inside k8s cluster, the traffic is RDMA. Will be changed to TCP in Egress gateway to visit external.
• Add Ingress gateway for container. User can define ingress rule for mapping between host port to containers.
• Ingress traffic visits host ip/port, and Ingress gateway changes to traffic to RDMA and redirects to corresponding pods.
• Add support for traffic from host machine to container network. The transition happens in Ingress gateway.

4. Reliability/Performance bug fixes

• Fix issue that quark container cannot be started randomly.
• Optimize hibernate feature.
• Add an inner loop inside event processing to improve perf.

Release Summary

The quark container v0.1.0 release includes supporting Container Runtime Interface (CRI), initial version of TCP Socket over RDMA (TSoR) control plane and data plane and fix multiple reliability/Performance bug fixes.

*Note: To set up k8s using quark container, it depends on quarkcni, please refer to k8s setup to configure.

Key Features and Improvements

1. CRI support

• Develop Quark contained Shim based on Containerd Shim V2.
• Support Sandbox/Subcontainer in Quark runtime
• Kubernetes integration test with MiniKube and K8S cluster.

2. TCP over RDMA support control plane

• k8s can run pods using Quark runtime
• Quark in k8s supports both RDMA CNI or other CNIs such as flannel
• Added Quark Control Manager (quarkcm) and Quark RDMA CNI (quarkcni)
• Quark Control Manager watches k8s cluster's node and pod and send changed data to RDMA service
• RDMA service setups connections between nodes automatically

3. TCP over RDMA support data plane

• RDMA support for following socket APIs:

  • bind
  • listen
  • connect
  • read* (a group of functions related to read: read, recvfrom, receive)
  • write (a group of functions related to write: write, sendto, send)
  • getsockname
  • getpeername

• RDMA service which acts as a standalone process to routing network traffic between local pods and remote nodes

  • Use share memory and lockless queue between RDMA service and quark container to gain best performance
  • Share RDMA Queue Pair between nodes for multiple quark containers to make RDMA NIC more scalable
  • Create RDMA channel state machine to simulate TCP state diagram

4. Reliability/Performance bug fixes

• Fix TLB shoot down handling bug
• Increase memory page management parallelism
• Change process memory space lock from mutex to upgradable RW lock