Container Resource Management

Sharing Resources

Description

Containers or containers and hosts can share namespace information mutually, including PID, network, IPC, and UTS information.

Usage

When running the isula create/run command, you can set the namespace parameters to share resources. For details, see the following parameter description table.

Parameters

You can specify the following parameters when running the lcrc create/run command:

Parameter

Description

Value Range

Mandatory or Not

--pid

Specifies the PID namespace to be shared.

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

No

--net

Specifies the network namespace to be shared.

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

No

--ipc

Specifies the IPC namespace to be shared.

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

No

--uts

Specifies the UTS namespace to be shared.

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

No

Example

If two containers need to share the same PID namespace, add –pid container:<containerID> when running the container. For example:

isula run -tid --name test_pid busybox sh
isula run -tid --name test --pid container:test_pid busybox sh

Restricting CPU Resources of a Running Container

Description

You can set parameters to restrict the CPU resources of a container.

Usage

When running the isula create/run command, you can set CPU-related parameters to limit the CPU resources of a container. For details about the parameters and values, see the following table.

Parameters

You can specify the following parameters when running the isula create/run command:

Parameter

Description

Value Range

Mandatory or Not

--cpu-period

Limits the CPU CFS period in a container.

64-bit integer

No

--cpu-quota

Limits the CPU CFS quota.

64-bit integer

No

--cpu-shares

Limits the CPU share (relative weight).

64-bit integer

No

--cpuset-cpus

Limits the CPU nodes.

A character string. The value is the number of CPUs to be configured. The value ranges from 0 to 3, or 0 and 1.

No

--cpuset-mems

Limits the memory nodes used by cpuset in the container.

A character string. The value is the number of CPUs to be configured. The value ranges from 0 to 3, or 0 and 1.

No

Example

To restrict a container to use a specific CPU, add –cpuset-cpus number when running the container. For example:

isula run -tid --cpuset-cpus 0,2-3 busybox sh

NOTE:
You can check whether the configuration is successful. For details, see “Querying Information About a Single Container.”

Restricting the Memory Usage of a Running Container

Description

You can set parameters to restrict the memory usage of a container.

Usage

When running the isula create/run command, you can set memory-related parameters to restrict memory usage of containers. For details about the parameters and values, see the following table.

Parameters

You can specify the following parameters when running the isula create/run command:

Parameter

Description

Value Range

Mandatory or Not

--memory

Specifies the upper limit of the memory usage of a container.

64-bit integer The value is a non-negative number. The value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

No

--memory-reservation

Specifies the soft upper limit of the memory of a container.

64-bit integer The value is a non-negative number. The value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

No

--memory-swap

Specifies the upper limit of the swap memory of the container.

64-bit integer The value can be -1 or a non-negative number. The value -1 indicates no limit, and the value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

No

--kernel-memory

Specifies the upper limit of the kernel memory of the container.

64-bit integer The value is a non-negative number. The value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

No

Example

To set the upper limit of the memory of a container, add –memory <number>[<unit>] when running the container. For example:

isula run -tid --memory 1G busybox sh

Restricting I/O Resources of a Running Container

Description

You can set parameters to limit the read/write speed of devices in the container.

Usage

When running the isula create/run command, you can set –device-read-bps/–device-write-bps <device-path>:<number>[<unit>] to limit the read/write speed of devices in the container.

Parameters

When running the isula create/run command, set –device-read/write-bps.

Parameter

Description

Value Range

Mandatory or Not

--device-read-bps/--device-write-bps

Limits the read/write speed of devices in the container.

64-bit integer The value is a positive integer. The value can be 0, indicating that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

No

Example

To limit the read/write speed of devices in the container, add –device-write-bps/–device-read-bps <device-path>:<number>[<unit>] when running the container. For example, to limit the read speed of the device /dev/sda in the container busybox to 1 Mbit/s, run the following command:

isula run -tid --device-write /dev/sda:1mb busybox sh

To limit the write speed, run the following command:

isula run -tid read-bps /dev/sda:1mb busybox sh

Restricting the Rootfs Storage Space of a Container

Description

When the overlay2 storage driver is used on the EXT4 file system, the file system quota of a single container can be set. For example, the quota of container A is set to 5 GB, and the quota of container B is set to 10 GB.

This feature is implemented by the project quota function of the EXT4 file system. If the kernel supports this function, use the syscall SYS_IOCTL to set the project ID of a directory, and then use the syscall SYS_QUOTACTL to set the hard limit and soft limit of the corresponding project ID.

Usage

  1. Prepare the environment.

    Ensure that the file system supports the Project ID and Project Quota attributes, the kernel version is 4.19 or later, and the version of the peripheral package e2fsprogs is 1.43.4-2 or later.

  2. Before mounting overlayfs to a container, set different project IDs for the upper and work directories of different containers and set inheritance options. After overlayfs is mounted to a container, the project IDs and inherited attributes cannot be modified.

  3. Set the quota as a privileged user outside the container.

  4. Add the following configuration to daemon:

    -s overlay2 --storage-opt overlay2.override_kernel_check=true
    
  5. Daemon supports the following options for setting default restrictions for containers:

    –storage-opt overlay2.basesize=128M specifies the default limit. If –storeage-opt size is also specified when you run the isula run command, the value of this parameter takes effect. If no size is specified during the daemon process or when you run the isula run command, the size is not limited.

  6. Enable the Project ID and Project Quota attributes of the file system.

    • Format and mount the file system.

      # mkfs.ext4 -O quota,project /dev/sdb
      # mount -o prjquota /dev/sdb /var/lib/isulad
      

Parameters

When running the create/run command, set –storage-opt.

Parameter

Description

Value Range

Mandatory or Not

--storage-opt size=${rootfsSize}

Restricts the root file system (rootfs) storage space of the container.

The size parsed by rootfsSize is a positive 64-bit integer expressed in bytes. You can also set it to ([kKmMgGtTpP])?[iI]?[bB]?$.

No

Example

In the isula run/create command, use the existing parameter –storage-opt size=value to set the quota. The value is a positive number in the unit of [kKmMgGtTpP]?[iI]?[bB]?. If the value does not contain a unit, the default unit is byte.

$ [root@localhost ~]# isula run -ti --storage-opt size=10M busybox
/ # df -h
Filesystem                Size      Used Available Use% Mounted on
overlay                  10.0M     48.0K     10.0M   0% /
none                     64.0M         0     64.0M   0% /dev
none                     10.0M         0     10.0M   0% /sys/fs/cgroup
tmpfs                    64.0M         0     64.0M   0% /dev
shm                      64.0M         0     64.0M   0% /dev/shm
/dev/mapper/vg--data-ext41
                          9.8G     51.5M      9.2G   1% /etc/hostname
/dev/mapper/vg--data-ext41
                          9.8G     51.5M      9.2G   1% /etc/resolv.conf
/dev/mapper/vg--data-ext41
                          9.8G     51.5M      9.2G   1% /etc/hosts
tmpfs                     3.9G         0      3.9G   0% /proc/acpi
tmpfs                    64.0M         0     64.0M   0% /proc/kcore
tmpfs                    64.0M         0     64.0M   0% /proc/keys
tmpfs                    64.0M         0     64.0M   0% /proc/timer_list
tmpfs                    64.0M         0     64.0M   0% /proc/sched_debug
tmpfs                     3.9G         0      3.9G   0% /proc/scsi
tmpfs                    64.0M         0     64.0M   0% /proc/fdthreshold
tmpfs                    64.0M         0     64.0M   0% /proc/fdenable
tmpfs                     3.9G         0      3.9G   0% /sys/firmware
/ # 
/ # dd if=/dev/zero of=/home/img bs=1M count=12 && sync
dm-4: write failed, project block limit reached.
10+0 records in
9+0 records out
10432512 bytes (9.9MB) copied, 0.011782 seconds, 844.4MB/s
/ # df -h | grep overlay
overlay                  10.0M     10.0M         0 100% /
/ # 

Constraints

  1. The quota applies only to the rw layer.

    The quota of overlay2 is for the rw layer of the container. The image size is not included.

  2. The kernel supports and enables this function.

    The kernel must support the EXT4 project quota function. When running mkfs, add -O quota,project. When mounting the file system, add -o prjquota. If any of the preceding conditions is not met, an error is reported when –storage-opt size=value is used.

    $ [root@localhost ~]# isula run -it --storage-opt size=10Mb busybox df -h
    Error response from daemon: Failed to prepare rootfs with error: time="2019-04-09T05:13:52-04:00" level=fatal msg="error creating read-
    write layer with ID "a4c0e55e82c55e4ee4b0f4ee07f80cc2261cf31b2c2dfd628fa1fb00db97270f": --storage-opt is supported only for overlay over
    xfs or ext4 with 'pquota' mount option"
    
  3. Description of the limit of quota:

    1. If the quota is greater than the size of the partition where user root of iSulad is located, the file system quota displayed by running the df command in the container is the size of the partition where user root of iSulad is located, not the specified quota.
    2. –storage-opt size=0 indicates that the size is not limited and the value cannot be less than 4096. The precision of size is one byte. If the specified precision contains decimal bytes, the decimal part is ignored. For example, if size is set to 0.1, the size is not limited. (The value is restricted by the precision of the floating point number stored on the computer. That is, 0.999999999999999999999999999 is equal to 1. The number of digits 9 may vary according to computers. Therefore, 4095.999999999999999999999999999 is equal to 4096.) Note that running isula inspect displays the original command line specified format. If the value contains decimal bytes, you need to ignore the decimal part.
    3. If the quota is too small, for example,** –storage-opt size=4k**, the container may fail to be started because some files need to be created for starting the container.
    4. The -o prjquota option is added to the root partition of iSulad when iSulad is started last time. If this option is not added during this startup, the setting of the container with quota created during the last startup does not take effect.
    5. The value range of the daemon quota –storage-opt overlay2.basesize is the same as that of –storage-opt size.
  4. When storage-opt is set to 4 KB, the lightweight container startup is different from that of Docker.

    Use the storage-opt size=4k and image rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest to run the container.

    Docker fails to be started.

    [root@localhost ~]# docker run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest
    docker: Error response from daemon: symlink /proc/mounts /var/lib/docker/overlay2/e6e12701db1a488636c881b44109a807e187b8db51a50015db34a131294fcf70-init/merged/etc/mtab: disk quota exceeded.
    See 'docker run --help'.
    

    The lightweight container is started properly and no error is reported.

    [root@localhost ~]# isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest
    636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728
    [root@localhost ~]# isula ps
    STATUS  PID   IMAGE                                                 COMMAND   EXIT_CODE RESTART_COUNT STARTAT       FINISHAT RUNTIME ID           NAMES                                                            
    running 17609 rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest /bin/bash 0         0             2 seconds ago -        lcr     636480b1fc2c 636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728 
    

    During container startup, if you need to create a file in the rootfs directory of the container, the image size exceeds 4 KB, and the quota is set to 4 KB, the file creation will fail.

    When Docker starts the container, it creates more mount points than iSulad to mount some directories on the host to the container, such as /proc/mounts and /dev/shm. If these files do not exist in the image, the creation will fail, therefore, the container fails to be started.

    When a lightweight container uses the default configuration during container startup, there are few mount points. The lightweight container is created only when the directory like /proc or /sys does not exist. The image rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest in the test case contains /proc and /sys. Therefore, no new file or directory is generated during the container startup. As a result, no error is reported during the lightweight container startup. To verify this process, when the image is replaced with rnd-dockerhub.huawei.com/official/busybox-aarch64:latest, an error is reported when the lightweight container is started because /proc does not exist in the image.

    [root@localhost ~]# isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/busybox-aarch64:latest
    8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4
    Error response from daemon: Start container error: runtime error: 8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4:tools/lxc_start.c:main:404 starting container process caused "Failed to setup lxc,
    please check the config file."
    
  5. Other description:

    When using iSulad with the quota function to switch data disks, ensure that the data disks to be switched are mounted using the prjquota option and the mounting mode of the /var/lib/isulad/storage/overlay2 directory is the same as that of the /var/lib/isulad directory.

    NOTE:
    Before switching the data disk, ensure that the mount point of /var/lib/isulad/storage/overlay2 is unmounted.

Restricting the Number of File Handles in a Container

Description

You can set parameters to limit the number of file handles that can be opened in a container.

Usage

When running the isula create/run command, set the –files-limit parameter to limit the number of file handles that can be opened in a container.

Parameters

Set the –files-limit parameter when running the isula create/run command.

Parameter

Description

Value Range

Mandatory or Not

--files-limit

Limits the number of file handles that can be opened in a container.

64-bit integer The value can be 0 or a negative number, but cannot be greater than 2 to the power of 63 minus 1. The value 0 or a negative number indicates no limit.

During container creation, some handles are opened temporarily. Therefore, the value cannot be too small. Otherwise, the container may not be restricted by the file limit. If the value is less than the number of opened handles, the cgroup file cannot be written. It is recommended that the value be greater than 30.

No

Example

When running the container, add –files-limit n. For example:

isula run -ti --files-limit 1024 busybox bash

Constraints

  1. If the –files-limit parameter is set to a small value, for example, 1, the container may fail to be started.

    [root@localhost ~]# isula run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64
    004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4
    Error response from daemon: Start container error: Container is not running:004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4
    

    Docker will be started successfully, and the value of files.limit cgroup is max.

    [root@localhost ~]# docker run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64
    ef9694bf4d8e803a1c7de5c17f5d829db409e41a530a245edc2e5367708dbbab
    [root@localhost ~]# docker exec -it ef96 cat /sys/fs/cgroup/files/files.limit
    max
    

    The root cause is that the startup principles of the lxc and runc processes are different. After the lxc process creates the cgroup, the files.limit value is set, and then the PID of the container process is written into the cgroup.procs file of the cgroup. At this time, the process has opened more than one handle. As a result, an error is reported, and the startup fails. After you create a cgroup by running the runc command, the PID of the container process is written to the cgroup.procs file of the cgroup, and then the files.limit value is set. Because more than one handle is opened by the process in the cgroup, the file.limit value does not take effect, the kernel does not report any error, and the container is started successfully.

Restricting the Number of Processes or Threads that Can Be Created in a Container

Description

You can set parameters to limit the number of processes or threads that can be created in a container.

Usage

When creating or running a container, use the –pids-limit parameter to limit the number of processes or threads that can be created in the container.

Parameters

When running the create/run command, set the –pids-limit parameter.

Parameter

Description

Value Range

Mandatory or Not

--pids-limit

Limits the number of file handles that can be opened in a container.

64-bit integer The value can be 0 or a negative number, but cannot be greater than 2 to the power of 63 minus 1. The value 0 or a negative number indicates no limit.

No

Example

When running the container, add –pids-limit n. For example:

isula run -ti --pids-limit 1024 busybox bash

Constraints

During container creation, some processes are created temporarily. Therefore, the value cannot be too small. Otherwise, the container may fail to be started. It is recommended that the value be greater than 10.

Configuring the ulimit Value in a Container

Description

You can use parameters to control the resources for executed programs.

Usage

Set the –ulimit parameter when creating or running a container, or configure the parameter on the daemon to control the resources for executed programs in the container.

Parameters

Use either of the following methods to configure ulimit:

  1. When running the isula create/run command, use –ulimit <type>=<soft>[:<hard>] to control the resources of the executed shell program.

    Parameter

    Description

    Value Range

    Mandatory or Not

    --ulimit

    Limits the resources of the executed shell program.

    64-bit integer The value of the soft limit must be less than or equal to that of the hard limit. If only the soft limit is specified, the value of the hard limit is equal to that of the soft limit. Some types of resources do not support negative numbers. For details, see the following table.

    No

  2. Use daemon parameters or configuration files.

    For details, see –default-ulimits in Deployment Mode.

    –ulimit can limit the following types of resources:

    Type

    Description

    Value Range

    core

    limits the core file size (KB)

    64-bit integer, without unit. The value can be 0 or a negative number. The value -1 indicates no limit. Other negative numbers are forcibly converted into a large positive integer.

    cpu

    max CPU time (MIN)

    data

    max data size (KB)

    fsize

    maximum filesize (KB)

    locks

    max number of file locks the user can hold

    memlock

    max locked-in-memory address space (KB)

    msgqueue

    max memory used by POSIX message queues (bytes)

    nice

    nice priority

    nproc

    max number of processes

    rss

    max resident set size (KB)

    rtprio

    max realtime priority

    rttime

    realtime timeout

    sigpending

    max number of pending signals

    stack

    max stack size (KB)

    nofile

    max number of open file descriptors

    64-bit integer, without unit. The value cannot be negative. A negative number is forcibly converted to a large positive number. In addition, "Operation not permitted" is displayed during the setting.

Example

When creating or running a container, add –ulimit <type>=<soft>[:<hard>]. For example:

isula create/run -tid --ulimit nofile=1024:2048 busybox sh

Constraints

The ulimit cannot be configured in the daemon.json and /etc/sysconfig/iSulad files (or the iSulad command line). Otherwise, an error is reported when iSulad is started.

Rating

search results about are found