Container Resource Management
- Container Resource Management
- Description
- Sharing Resources
- Restricting CPU Resources of a Running Container
- Restricting the Memory Usage of a Running Container
- Restricting I/O Resources of a Running Container
- Restricting the Rootfs Storage Space of a Container
- Restricting the Number of File Handles in a Container
- Restricting the Number of Processes or Threads that Can Be Created in a Container
- Configuring the ulimit Value in a Container
Description
You can use namespaces and cgroups to manage container resources. iSula can use cgroup v1 and cgroup v2 to restrict resources. cgroup v2 is an experimental feature and cannot be put into commercial use. When the system is configured to support only cgroup v2 and cgroup v2 is mounted to the /sys/fs/cgroup directory, iSula uses cgroup v2 for resource management. Whether cgroup v1 or cgroup v2 is used to manage container resources, iSula provides the same interface for users to implement resource restriction.
Sharing Resources
Description
Containers or containers and hosts can share namespace information mutually, including PID, network, IPC, and UTS information.
NOTE: When namespace information is shared with a host, the namespace isolation mechanism is unavailable. As a result, information on the host can be queried and operated in a container, causing security risks. For example, if --pid=host is used to share the PID namespace of a host, information about other processes on the host can be viewed, causing information leakage or even killing the host process. Exercise caution when using the shared host namespace function to ensure security.
Usage
When running the isula create/run command, you can set the namespace parameters to share resources. For details, see the following parameter description table.
Parameters
You can specify the following parameters when running the isula create/run command:
Example
If two containers need to share the same PID namespace, add --pid container:<containerID> when running the container. For example:
isula run -tid --name test_pid busybox sh
isula run -tid --name test --pid container:test_pid busybox sh
Restricting CPU Resources of a Running Container
Description
You can set parameters to restrict the CPU resources of a container.
Usage
When running the isula create/run command, you can set CPU-related parameters to limit the CPU resources of a container. For details about the parameters and values, see the following table.
Parameters
You can specify the following parameters when running the isula create/run command:
Example
To restrict a container to use a specific CPU, add --cpuset-cpus number when running the container. For example:
isula run -tid --cpuset-cpus 0,2-3 busybox sh
NOTE: You can check whether the configuration is successful. For details, see "Querying Information About a Single Container."
Restricting the Memory Usage of a Running Container
Description
You can set parameters to restrict the memory usage of a container.
Usage
When running the isula create/run command, you can set memory-related parameters to restrict memory usage of containers. For details about the parameters and values, see the following table.
Parameters
You can specify the following parameters when running the isula create/run command:
Example
To set the upper limit of the memory of a container, add --memory <number>[<unit>] when running the container. For example:
isula run -tid --memory 1G busybox sh
Restricting I/O Resources of a Running Container
Description
You can set parameters to limit the read/write speed of devices in the container.
Usage
When running the isula create/run command, you can set --device-read-bps/--device-write-bps <device-path>:<number>[<unit>] to limit the read/write speed of devices in the container.
Parameters
When running the isula create/run command, set --device-read/write-bps.
64-bit integer The value is a positive integer. The value can be 0, indicating that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB. |
Example
To limit the read/write speed of devices in the container, add --device-write-bps/--device-read-bps <device-path>:<number>[<unit>] when running the container. For example, to limit the read speed of the device /dev/sda in the container busybox to 1 Mbit/s, run the following command:
isula run -tid --device-read-bps /dev/sda:1mb busybox sh
To limit the write speed, run the following command:
isula run -tid --device-write-bps /dev/sda:1mb busybox sh
Restricting the Rootfs Storage Space of a Container
Description
When the overlay2 storage driver is used on the EXT4 file system, the file system quota of a single container can be set. For example, the quota of container A is set to 5 GB, and the quota of container B is set to 10 GB.
This feature is implemented by the project quota function of the EXT4 file system. If the kernel supports this function, use the syscall SYS_IOCTL to set the project ID of a directory, and then use the syscall SYS_QUOTACTL to set the hard limit and soft limit of the corresponding project ID.
Usage
Prepare the environment.
Ensure that the file system supports the Project ID and Project Quota attributes, the kernel version is 4.19 or later, and the version of the peripheral package e2fsprogs is 1.43.4-2 or later.
Before mounting overlayfs to a container, set different project IDs for the upper and work directories of different containers and set inheritance options. After overlayfs is mounted to a container, the project IDs and inherited attributes cannot be modified.
Set the quota as a privileged user outside the container.
Add the following configuration to daemon:
-s overlay2 --storage-opt overlay2.override_kernel_check=true
Daemon supports the following options for setting default restrictions for containers:
--storage-opt overlay2.basesize=128M specifies the default limit. If --storage-opt size is also specified when you run the isula run command, the value of this parameter takes effect. If no size is specified during the daemon process or when you run the isula run command, the size is not limited.
Enable the Project ID and Project Quota attributes of the file system.
Format and mount the file system.
mkfs.ext4 -O quota,project /dev/sdb mount -o prjquota /dev/sdb /var/lib/isulad
Parameters
When running the create/run command, set --storage-opt.
Example
In the isula run/create command, use the existing parameter --storage-opt size=value to set the quota. The value is a positive number in the unit of [kKmMgGtTpP]?[iI]?[bB]?. If the value does not contain a unit, the default unit is byte.
$ [root@localhost ~]# isula run -ti --storage-opt size=10M busybox
/ # df -h
Filesystem Size Used Available Use% Mounted on
overlay 10.0M 48.0K 10.0M 0% /
none 64.0M 0 64.0M 0% /dev
none 10.0M 0 10.0M 0% /sys/fs/cgroup
tmpfs 64.0M 0 64.0M 0% /dev
shm 64.0M 0 64.0M 0% /dev/shm
/dev/mapper/vg--data-ext41
9.8G 51.5M 9.2G 1% /etc/hostname
/dev/mapper/vg--data-ext41
9.8G 51.5M 9.2G 1% /etc/resolv.conf
/dev/mapper/vg--data-ext41
9.8G 51.5M 9.2G 1% /etc/hosts
tmpfs 3.9G 0 3.9G 0% /proc/acpi
tmpfs 64.0M 0 64.0M 0% /proc/kcore
tmpfs 64.0M 0 64.0M 0% /proc/keys
tmpfs 64.0M 0 64.0M 0% /proc/timer_list
tmpfs 64.0M 0 64.0M 0% /proc/sched_debug
tmpfs 3.9G 0 3.9G 0% /proc/scsi
tmpfs 64.0M 0 64.0M 0% /proc/fdthreshold
tmpfs 64.0M 0 64.0M 0% /proc/fdenable
tmpfs 3.9G 0 3.9G 0% /sys/firmware
/ #
/ # dd if=/dev/zero of=/home/img bs=1M count=12 && sync
dm-4: write failed, project block limit reached.
10+0 records in
9+0 records out
10432512 bytes (9.9MB) copied, 0.011782 seconds, 844.4MB/s
/ # df -h | grep overlay
overlay 10.0M 10.0M 0 100% /
/ #
Constraints
The quota applies only to the rw layer.
The quota of overlay2 is for the rw layer of the container. The image size is not included.
The kernel supports and enables this function.
The kernel must support the EXT4 project quota function. When running mkfs, add -O quota,project. When mounting the file system, add -o prjquota. If any of the preceding conditions is not met, an error is reported when --storage-opt size=value is used.
$ [root@localhost ~]# isula run -it --storage-opt size=10Mb busybox df -h Error response from daemon: Failed to prepare rootfs with error: time="2019-04-09T05:13:52-04:00" level=fatal msg="error creating read- write layer with ID "a4c0e55e82c55e4ee4b0f4ee07f80cc2261cf31b2c2dfd628fa1fb00db97270f": --storage-opt is supported only for overlay over xfs or ext4 with 'pquota' mount option"
Description of the limit of quota:
- If the quota is greater than the size of the partition where user root of iSulad is located, the file system quota displayed by running the df command in the container is the size of the partition where user root of iSulad is located, not the specified quota.
- --storage-opt size=0 indicates that the size is not limited and the value cannot be less than 4096. The precision of size is one byte. If the specified precision contains decimal bytes, the decimal part is ignored. For example, if size is set to 0.1, the size is not limited. (The value is restricted by the precision of the floating point number stored on the computer. That is, 0.999999999999999999999999999 is equal to 1. The number of digits 9 may vary according to computers. Therefore, 4095.999999999999999999999999999 is equal to 4096.) Note that running isula inspect displays the original command line specified format. If the value contains decimal bytes, you need to ignore the decimal part.
- If the quota is too small, for example,--storage-opt size=4k, the container may fail to be started because some files need to be created for starting the container.
- The -o prjquota option is added to the root partition of iSulad when iSulad is started last time. If this option is not added during this startup, the setting of the container with quota created during the last startup does not take effect.
- The value range of the daemon quota --storage-opt overlay2.basesize is the same as that of --storage-opt size.
When storage-opt is set to 4 KB, the lightweight container startup is different from that of Docker.
Use the storage-opt size=4k and image rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest to run the container.
Docker fails to be started.
[root@localhost ~]# docker run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest docker: Error response from daemon: symlink /proc/mounts /var/lib/docker/overlay2/e6e12701db1a488636c881b44109a807e187b8db51a50015db34a131294fcf70-init/merged/etc/mtab: disk quota exceeded. See 'docker run --help'.
The lightweight container is started properly and no error is reported.
[root@localhost ~]# isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest 636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728 [root@localhost ~]# isula ps STATUS PID IMAGE COMMAND EXIT_CODE RESTART_COUNT STARTAT FINISHAT RUNTIME ID NAMES running 17609 rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest /bin/bash 0 0 2 seconds ago - runc 636480b1fc2c 636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728
During container startup, if you need to create a file in the rootfs directory of the container, the image size exceeds 4 KB, and the quota is set to 4 KB, the file creation will fail.
When Docker starts the container, it creates more mount points than iSulad to mount some directories on the host to the container, such as /proc/mounts and /dev/shm. If these files do not exist in the image, the creation will fail, therefore, the container fails to be started.
When a lightweight container uses the default configuration during container startup, there are few mount points. The lightweight container is created only when the directory like /proc or /sys does not exist. The image rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest in the test case contains /proc and /sys. Therefore, no new file or directory is generated during the container startup. As a result, no error is reported during the lightweight container startup. To verify this process, when the image is replaced with rnd-dockerhub.huawei.com/official/busybox-aarch64:latest, an error is reported when the lightweight container is started because /proc does not exist in the image.
[root@localhost ~]# isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/busybox-aarch64:latest 8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4 Error response from daemon: Start container error: runtime error: 8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4:tools/lxc_start.c:main:404 starting container process caused "Failed to setup lxc, please check the config file."
Other description:
When using iSulad with the quota function to switch data disks, ensure that the data disks to be switched are mounted using the prjquota option and the mounting mode of the /var/lib/isulad/storage/overlay2 directory is the same as that of the /var/lib/isulad directory.
NOTE: Before switching the data disk, ensure that the mount point of /var/lib/isulad/storage/overlay2 is unmounted.
Restricting the Number of File Handles in a Container
Description
You can set parameters to limit the number of file handles that can be opened in a container.
Usage
When running the isula create/run command, set the --files-limit parameter to limit the number of file handles that can be opened in a container.
Parameters
Set the --files-limit parameter when running the isula create/run command.
Example
When running the container, add --files-limit n. For example:
isula run -ti --files-limit 1024 busybox bash
Constraints
If the --files-limit parameter is set to a small value, for example, 1, the container may fail to be started.
[root@localhost ~]# isula run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64 004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4 Error response from daemon: Start container error: Container is not running:004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4
Docker will be started successfully, and the value of files.limit cgroup is max.
[root@localhost ~]# docker run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64 ef9694bf4d8e803a1c7de5c17f5d829db409e41a530a245edc2e5367708dbbab [root@localhost ~]# docker exec -it ef96 cat /sys/fs/cgroup/files/files.limit max
The root cause is that the startup principles of the lxc and runc processes are different. After the lxc process creates the cgroup, the files.limit value is set, and then the PID of the container process is written into the cgroup.procs file of the cgroup. At this time, the process has opened more than one handle. As a result, an error is reported, and the startup fails. After you create a cgroup by running the runc command, the PID of the container process is written to the cgroup.procs file of the cgroup, and then the files.limit value is set. Because more than one handle is opened by the process in the cgroup, the file.limit value does not take effect, the kernel does not report any error, and the container is started successfully.
Restricting the Number of Processes or Threads that Can Be Created in a Container
Description
You can set parameters to limit the number of processes or threads that can be created in a container.
Usage
When creating or running a container, use the --pids-limit parameter to limit the number of processes or threads that can be created in the container.
Parameters
When running the create/run command, set the --pids-limit parameter.
Example
When running the container, add --pids-limit n. For example:
isula run -ti --pids-limit 1024 busybox bash
Constraints
During container creation, some processes are created temporarily. Therefore, the value cannot be too small. Otherwise, the container may fail to be started. It is recommended that the value be greater than 10.
Configuring the ulimit Value in a Container
Description
You can use parameters to control the resources for executed programs.
Usage
Set the --ulimit parameter when creating or running a container, or configure the parameter on the daemon to control the resources for executed programs in the container.
Parameters
Use either of the following methods to configure ulimit:
When running the isula create/run command, use --ulimit <type>=<soft>[:<hard>] to control the resources of the executed shell program.
Use daemon parameters or configuration files.
For details, see --default-ulimits in Configuration Mode.
--ulimit can limit the following types of resources:
Example
When creating or running a container, add --ulimit <type>=<soft>[:<hard>]. For example:
isula create/run -tid --ulimit nofile=1024:2048 busybox sh
Constraints
The ulimit cannot be configured in the daemon.json and /etc/sysconfig/iSulad files (or the iSulad command line). Otherwise, an error is reported when iSulad is started.