CRI API v1alpha2

Description

CRI API is the container runtime APIs provided by Kubernetes. CRI defines service interfaces for containers and images. iSulad uses CRI API to interconnect with Kubernetes.

The lifecycle of a container is isolated from that of an image. Therefore, two services are required. CRI API is defined using Protocol Buffers and is based on gRPC.

Currently, the default CRI API version used by iSulad is v1alpha2. The official API description file is as follows:

https://github.com/kubernetes/kubernetes/blob/release-1.14/pkg/kubelet/apis/cri/runtime/v1alpha2/api.proto,

iSulad uses the API description file of version 1.14 used by Pass, which is slightly different from the official API. The interfaces in this document prevail.

NOTE: For the WebSocket streaming service of CRI API, the listening address of the server is 127.0.0.1, and the port number is 10350. The port number can be configured through the --websocket-server-listening-port command option or in the daemon.json configuration file.

Interfaces

The following tables list the parameters that may be used by the interfaces. Some parameters cannot be configured.

Interface Parameters

• DNSConfig

  Specifies the DNS servers and search domains of a sandbox.

• Protocol

  Enum values of the protocols.

• PortMapping

  Specifies the port mapping configurations of a sandbox.

• MountPropagation

  Enum values for mount propagation.

• Mount

  Specifies a host volume to mount into a container. (Only files and folders are supported.)

• NamespaceOption

• Capability

  Contains information about the capabilities to add or drop.

• Int64Value

  Wrapper of the int64 type.

• UInt64Value

  Wrapper of the uint64 type.

• LinuxSandboxSecurityContext

  Specifies Linux security options for a sandbox.

  Note that these security options are not applied to containers in the sandbox and may not be applicable to a sandbox without any running process.

• LinuxPodSandboxConfig

  Sets configurations related to Linux hosts and containers.

• PodSandboxMetadata

  Stores all necessary information for building the sandbox name. The container runtime is encouraged to expose the metadata in its user interface for better user experience. For example, the runtime can construct a unique sandbox name based on the metadata.

• PodSandboxConfig

  Contains all the required and optional fields for creating a sandbox.

• PodSandboxNetworkStatus

  Describes the network status of the sandbox.

• Namespace

  Stores namespace options.

• LinuxPodSandboxStatus

  Describes the status of the Linux sandbox.

• PodSandboxState

  Enum values for sandbox states.

• PodSandboxStatus

  Describes the podsandbox status.

• PodSandboxStateValue

  Wrapper of PodSandboxState.

• PodSandboxFilter

  Filtering conditions when listing sandboxes. The intersection of multiple conditions is displayed.

• PodSandbox

  Minimal data that describes a sandbox.

• KeyValue

  Wrapper of a key-value pair.

• SELinuxOption

  SELinux labels to be applied to the container.

• ContainerMetadata

  ContainerMetadata contains all necessary information for building the container name. The container runtime is encouraged to expose the metadata in its user interface for better user experience. For example, the runtime can construct a unique container name based on the metadata.

• ContainerState

  Enum values for container states.

• ContainerStateValue

  Wrapper of ContainerState.

• ContainerFilter

  Filtering conditions when listing containers. The intersection of multiple conditions is displayed.

• LinuxContainerSecurityContext

  Security configuration that will be applied to a container.

• LinuxContainerResources

  Resource specification for the Linux container.

• Image

  Basic information about a container image.

• ImageSpec

  Internal data structure that represents an image. Currently, ImageSpec wraps only the container image name.

• StorageIdentifier

  Unique identifier of a storage device.

• FilesystemUsage

• AuthConfig

• Container

  Container description information, such as the ID and state.

• ContainerStatus

  Container status information.

• ContainerStatsFilter

  Filtering conditions when listing container states. The intersection of multiple conditions is displayed.

• ContainerStats

  Filtering conditions when listing container states. The intersection of multiple conditions is displayed.

• ContainerAttributes

  Basic information about the container.

• CpuUsage

  Container CPU usage.

• MemoryUsage

  Container memory usage.

• FilesystemUsage

  Usage of the writable layer of the container.

• Device

  Host volume to mount into a container.

• LinuxContainerConfig

  Configuration specific to Linux containers.

• ContainerConfig

  Required and optional fields for creating a container.

• NetworkConfig

  Runtime network configuration.

• RuntimeConfig

  Runtime network configuration.

• RuntimeCondition

  Runtime condition information.

• RuntimeStatus

  Runtime status.

Runtime Service

The runtime service contains interfaces for operating pods and containers, and interfaces for querying the configuration and status of the runtime service.

RunPodSandbox

Interface Prototype

rpc RunPodSandbox(RunPodSandboxRequest) returns (RunPodSandboxResponse) {}

Interface Description

Creates and starts a pod sandbox. The sandbox is in the ready state on success.

Precautions

1. The default image for starting the sandbox is rnd-dockerhub.huawei.com/library/pause-${machine}:3.0, where ${machine} indicates the architecture. On x86_64, the value of machine is amd64, on ARM64, the value of machine is aarch64. Currently, only the amd64 and aarch64 images can be downloaded from the rnd-dockerhub repository. If the images do not exist on the host, ensure that the host can download them from the rnd-dockerhub repository.
2. The container names use the field in PodSandboxMetadata and are separated by underscores (_). Therefore, the data in metadata cannot contain underscores. Otherwise, the sandbox runs successfully, but the ListPodSandbox interface cannot query the sandbox.

Parameter

Returns

StopPodSandbox

Interface Prototype

rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse) {}

Interface Description

Stops the pod sandbox, stops the sandbox container, and reclaims the network resources (such as IP addresses) allocated to the sandbox. If any running container belongs to the sandbox, the container must be forcibly terminated.

Parameter

Returns

RemovePodSandbox

Interface Prototype

rpc RemovePodSandbox(RemovePodSandboxRequest) returns (RemovePodSandboxResponse) {}

Interface Description

Removes a sandbox. If there are any running containers in the sandbox, they must be forcibly terminated and removed. This interface must not return an error if the sandbox has already been removed.

Precautions

1. When a sandbox is deleted, the network resources of the sandbox are not deleted. Before deleting the pod, you must call StopPodSandbox to remove the network resources. Ensure that StopPodSandbox is called at least once before deleting the sandbox.
2. If the container in a sandbox fails to be deleted when the sandbox is deleted, the sandbox is deleted but the container remains. In this case, you need to manually delete the residual container.

Parameter

Returns

PodSandboxStatus

Interface Prototype

rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse) {}

Interface Description

Queries the status of the sandbox. If the sandbox does not exist, this interface returns an error.

Parameter

Returns

ListPodSandbox

Interface Prototype

rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse) {}

Interface Description

Returns sandbox information. Conditional filtering is supported.

Parameter

Returns

CreateContainer

Interface Prototype

rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse) {}

Interface Description

Creates a container in a PodSandbox.

Precautions

• sandbox_config in CreateContainerRequest is the same as the configuration passed to RunPodSandboxRequest to create the PodSandbox. It is passed again for reference. PodSandboxConfig is immutable and remains unchanged throughout the lifecycle of a pod.
• The container names use the field in ContainerMetadata and are separated by underscores (_). Therefore, the data in metadata cannot contain underscores. Otherwise, the container runs successfully, but the ListContainers interface cannot query the container.
• CreateContainerRequest does not contain the runtime_handler field. The runtime type of the created container is the same as that of the corresponding sandbox.

Parameter

Supplementary Information

Unstructured key-value map that may be used to store and retrieve arbitrary metadata. Some fields can be transferred through this field because CRI does not provide specific parameters.

• Customization

Returns

StartContainer

Interface Prototype

rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {}

Interface Description

Starts a container.

Parameter

Returns

StopContainer

Interface Prototype

rpc StopContainer(StopContainerRequest) returns (StopContainerResponse) {}

Interface Description

Stops a running container. The graceful stop timeout can be configured. If the container has been stopped, no error can be returned.

Parameter

Returns

None

RemoveContainer

Interface Prototype

rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse) {}

Interface Description

Deletes a container. If the container is running, it must be forcibly stopped. If the container has been deleted, no error can be returned.

Parameter

Returns

None

ListContainers

Interface Prototype

rpc ListContainers(ListContainersRequest) returns (ListContainersResponse) {}

Interface Description

Returns container information. Conditional filtering is supported.

Parameter

Returns

ContainerStatus

Interface Prototype

rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse) {}

Interface Description

Returns container status information. If the container does not exist, an error is returned.

Parameter

Returns

UpdateContainerResources

Interface Prototype

rpc UpdateContainerResources(UpdateContainerResourcesRequest) returns (UpdateContainerResourcesResponse) {}

Interface Description

Updates container resource configurations.

Precautions

• This interface is used exclusively to update the resource configuration of a container, not a pod.
• Currently, the oom_score_adj configuration of containers cannot be updated.

Parameter

Returns

None

ExecSync

Interface Prototype

rpc ExecSync(ExecSyncRequest) returns (ExecSyncResponse) {}

Interface Description

Runs a command synchronously in a container and communicates using gRPC.

Precautions

This interface runs a single command and cannot open a terminal to interact with the container.

Parameter

Returns

Exec

Interface Prototype

rpc Exec(ExecRequest) returns (ExecResponse) {}

Interface Description

Runs a command in the container, obtains the URL from the CRI server using gRPC, and establishes a persistent connection with the WebSocket server based on the obtained URL to interact with the container.

Precautions

This interface runs a single command and can open a terminal to interact with the container. One of stdin, stdout, or stderr must be true. If tty is true, stderr must be false. Multiplexing is not supported. In that case, the outputs of stdout and stderr are combined into a single stream.

Parameter

Returns

Attach

Interface Prototype

rpc Attach(AttachRequest) returns (AttachResponse) {}

Interface Description

Takes over process 1 of the container, obtains the URL from the CRI server using gRPC, and establishes a persistent connection with the WebSocket server based on the obtained URL to interact with the container.

Parameter

Returns

ContainerStats

Interface Prototype

rpc ContainerStats(ContainerStatsRequest) returns (ContainerStatsResponse) {}

Interface Description

Returns information about the resources occupied by a single container. Only containers whose runtime type is lcr are supported.

Parameter

Returns

ListContainerStats

Interface Prototype

rpc ListContainerStats(ListContainerStatsRequest) returns (ListContainerStatsResponse) {}

Interface Description

Returns information about resources occupied by multiple containers. Conditional filtering is supported.

Parameter

Returns

UpdateRuntimeConfig

Interface Prototype

rpc UpdateRuntimeConfig(UpdateRuntimeConfigRequest) returns (UpdateRuntimeConfigResponse);

Interface Description

Provides standard CRI for updating pod CIDR of the network plugin. Currently, the CNI network plugins do not need to update the pod CIDR. Therefore, this interface only records access logs.

Precautions

This interface does not modify the system management information, but only records logs.

Parameter

Returns

None

Status

Interface Prototype

rpc Status(StatusRequest) returns (StatusResponse) {};

Interface Description

Obtains the network status of the runtime and pod. When the network status is obtained, the network configuration is updated.

Precautions

If the network configuration fails to be updated, the original configuration is not affected. The original configuration is overwritten only when the network configuration is updated successfully.

Parameter

Returns

Image Service

Provides gRPC APIs for pulling, viewing, and removing images from the image repository.

ListImages

Interface Prototype

rpc ListImages(ListImagesRequest) returns (ListImagesResponse) {}

Interface Description

Lists information about existing images.

Precautions

This interface is a unified interface. Images of embedded format can be queried using cri images. However, because embedded images are not in OCI standard, the query result has the following restrictions:

• The displayed image ID is digest of config of the image because embedded images do not have image IDs.
• digest cannot be displayed because embedded images have only digest of config, not digest of themselves, and digest does not comply with OCI specifications.

Parameter

Returns

ImageStatus

Interface Prototype

rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse) {}

Interface Description

Queries the details about a specified image.

Precautions

1. This interface is used to query information about a specified image. If the image does not exist, ImageStatusResponse is returned, in which Image is nil.
2. This interface is a unified interface. Images of embedded format cannot be queried because they do not comply with the OCI specification and lack some fields.

Parameter

Returns

PullImage

Interface Prototype

 rpc PullImage(PullImageRequest) returns (PullImageResponse) {}

Interface Description

Downloads an image.

Precautions

You can download public images or private images using the username, password, and authentication information. The server_address, identity_token, and registry_token fields in AuthConfig are not supported.

Parameter

Returns

RemoveImage

Interface Prototype

rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse) {}

Interface Description

Deletes a specified image.

Precautions

This interface is a unified interface. Images of embedded format cannot be deleted based on the image ID because they do not comply with the OCI specification and lack some fields.

Parameter

Returns

None

ImageFsInfo

Interface Prototype

rpc ImageFsInfo(ImageFsInfoRequest) returns (ImageFsInfoResponse) {}

Interface Description

Queries information about the file systems of an image.

Precautions

The queried information is the file system information in the image metadata.

Parameter

None

Returns

Constraints

1. If log_directory is configured in PodSandboxConfig when a sandbox is created, log_path must be specified in ContainerConfig when a container of the sandbox is created. Otherwise, the container may fail to be started or even deleted using CRI API.

   The actual LOGPATH of the container is log_directory/log_path. If log_path is not configured, the final LOGPATH changes to log_directory.

   • If the path does not exist, iSulad creates a soft link pointing to the final path of container logs when starting the container, and log_directory becomes a soft link. In this case, there are two situations:

     1. If log_path is not configured for other containers in the sandbox, when other containers are started, log_directory is deleted and points to log_path of the newly started container. As a result, the logs of the previously started container point to the logs of the container started later.
     2. If log_path is configured for other containers in the sandbox, LOGPATH of the container is log_directory/log_path. Because log_directory is a soft link, if log_directory/log_path is used as the soft link target to point to the actual log path of the container, the container creation fails.

   • If the path exists, iSulad attempts to delete the path (non-recursively) when starting the container. If the path is a folder that contains content, the deletion fails. As a result, the soft link fails to be created and the container fails to be started. When the container is deleted, the same symptom occurs. As a result, the container deletion fails.

2. If log_directory is configured in PodSandboxConfig when a sandbox is created and log_path is configured in ContainerConfig when a container is created, the final LOGPATH is log_directory/log_path. iSulad does not create LOGPATH recursively. Therefore, you must ensure that dirname(LOGPATH), that is, the parent directory of the final log directory, exists.

3. If log_directory is configured in PodSandboxConfig when a sandbox is created, and the same log_path is specified in ContainerConfig when two or more containers are created or containers in different sandboxes point to the same LOGPATH, when the containers are started successfully, the log path of the container that is started later overwrites that of the container that is started earlier.

4. If the image content in the remote image repository changes and the CRI image pulling interface is used to download the image again, the image name and tag of the local original image (if it exists) change to "none."

   Example:

   Local image:

   IMAGE                                        TAG                 IMAGE ID            SIZE
   rnd-dockerhub.huawei.com/pproxyisulad/test   latest              99e59f495ffaa       753kB
   

   After the rnd-dockerhub.huawei.com/pproxyisulad/test:latest image in the remote repository is updated and downloaded again:

   IMAGE                                        TAG                 IMAGE ID            SIZE
   <none>                                       <none>              99e59f495ffaa       753kB
   rnd-dockerhub.huawei.com/pproxyisulad/test   latest              d8233ab899d41       1.42MB
   

   Run the isula images command. REF is displayed as -.

   REF                                               IMAGE ID               CREATED              SIZE   
   rnd-dockerhub.huawei.com/pproxyisulad/test:latest d8233ab899d41          2019-02-14 19:19:37  1.42MB   
   -                                                 99e59f495ffaa          2016-05-04 02:26:41  753kB
   

5. The exec and attach interfaces of iSulad CRI API are implemented using WebSocket. Clients interact with iSulad using the same protocol. When using the exec or attach interface, do not transfer a large amount of data or files over the serial port. The exec or attach interface is used only for basic command interaction. If the user side does not process the data or files in a timely manner, data may be lost. In addition, do not use the exec or attach interface to transfer binary data or files.

6. The iSulad CRI API exec/attach depends on libwebsockets (LWS). It is recommended that the streaming API be used only for persistent connection interaction but not in high-concurrency scenarios, because the connection may fail due to insufficient host resources. It is recommended that the number of concurrent connections be less than or equal to 100.