Installation and Configuration

Installation Methods

iSulad can be installed by running the yum or rpm command. The yum command is recommended because dependencies can be installed automatically.

This section describes two installation methods.

  • (Recommended) Run the following command to install iSulad:

    $ sudo yum install -y iSulad
    
  • If the rpm command is used to install iSulad, you need to download and manually install the RMP packages of iSulad and all its dependencies. To install the RPM package of a single iSulad (the same for installing dependency packages), run the following command:

    $ sudo rpm -ihv iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm
    

Deployment Configuration

Configuration Mode

The iSulad server daemon isulad can be configured with a configuration file or by running the isulad –xxx command. The priority in descending order is as follows: CLI > configuration file > default configuration in code.

NOTE:
If systemd is used to manage the iSulad process, modify the OPTIONS field in the /etc/sysconfig/iSulad file, which functions the same as using the CLI.

  • CLI

    During service startup, configure iSulad using the CLI. To view the configuration options, run the following command:

    $ isulad --help
    lightweight container runtime daemon
    
    Usage:  isulad [global options]
    
    GLOBAL OPTIONS:
    
          --authorization-plugin            Use authorization plugin
          --cgroup-parent                   Set parent cgroup for all containers
          --cni-bin-dir                     The full path of the directory in which to search for CNI plugin binaries. Default: /opt/cni/bin
          --cni-conf-dir                    The full path of the directory in which to search for CNI config files. Default: /etc/cni/net.d
          --default-ulimit                  Default ulimits for containers (default [])
      -e, --engine                          Select backend engine
      -g, --graph                           Root directory of the iSulad runtime
      -G, --group                           Group for the unix socket(default is isulad)
          --help                            Show help
          --hook-spec                       Default hook spec file applied to all containers
      -H, --host                            The socket name used to create gRPC server
          --image-layer-check               Check layer intergrity when needed
          --image-opt-timeout               Max timeout(default 5m) for image operation
          --insecure-registry               Disable TLS verification for the given registry
          --insecure-skip-verify-enforce    Force to skip the insecure verify(default false)
          --log-driver                      Set daemon log driver, such as: file
      -l, --log-level                       Set log level, the levels can be: FATAL ALERT CRIT ERROR WARN NOTICE INFO DEBUG TRACE
          --log-opt                         Set daemon log driver options, such as: log-path=/tmp/logs/ to set directory where to store daemon logs
          --native.umask                    Default file mode creation mask (umask) for containers
          --network-plugin                  Set network plugin, default is null, suppport null and cni
      -p, --pidfile                         Save pid into this file
          --pod-sandbox-image               The image whose network/ipc namespaces containers in each pod will use. (default "rnd-dockerhub.huawei.com/library/pause-${machine}:3.0")
          --registry-mirrors                Registry to be prepended when pulling unqualified images, can be specified multiple times
          --start-timeout                   timeout duration for waiting on a container to start before it is killed
      -S, --state                           Root directory for execution state files
          --storage-driver                  Storage driver to use(default overlay2)
      -s, --storage-opt                     Storage driver options
          --tls                             Use TLS; implied by --tlsverify
          --tlscacert                       Trust certs signed only by this CA (default "/root/.iSulad/ca.pem")
          --tlscert                         Path to TLS certificate file (default "/root/.iSulad/cert.pem")
          --tlskey                          Path to TLS key file (default "/root/.iSulad/key.pem")
          --tlsverify                       Use TLS and verify the remote
          --use-decrypted-key               Use decrypted private key by default(default true)
      -V, --version                         Print the version
       --websocket-server-listening-port    CRI websocket streaming service listening port (default 10350)
    

    Example: Start iSulad and change the log level to DEBUG.

    $ isulad -l DEBUG
    
  • Configuration file

    The iSulad configuration file is /etc/isulad/daemon.json. The parameters in the file are described as follows:

    Parameter

    Example

    Description

    Remarks

    -e, --engine

    "engine": "lcr"

    iSulad runtime, which is Icr by default.

    None

    -G, --group

    "group": "isulad"

    Socket group.

    None

    --hook-spec

    "hook-spec": "/etc/default/isulad/hooks/default.json"

    Default hook configuration file for all containers.

    None

    -H, --host

    "hosts": "unix:///var/run/isulad.sock"

    Communication mode.

    In addition to the local socket, the tcp://ip:port mode is supported. The port number ranges from 0 to 65535, excluding occupied ports.

    --log-driver

    "log-driver": "file"

    Log driver configuration.

    None

    -l, --log-level

    "log-level": "ERROR"

    Log output level.

    None

    --log-opt

    "log-opts": {

    "log-file-mode": "0600",

    "log-path": "/var/lib/isulad",

    "max-file": "1",

    "max-size": "30KB"

    }

    Log-related configuration.

    You can specify max-file, max-size, and log-path. max-file indicates the number of log files. max-size indicates the threshold for triggering log anti-explosion. If max-file is 1, max-size is invalid. log-path specifies the path for storing log files. The log-file-mode command is used to set the permissions to read and write log files. The value must be in octal format, for example, 0666.

    --start-timeout

    "start-timeout": "2m"

    Time required for starting a container.

    None

    --runtime

    "default-runtime": "lcr"

    Container runtime, which is lcr by default.

    If neither the CLI nor the configuration file specifies the runtime, lcr is used by default. The priorities of the three specifying methods are as follows: CLI > configuration file > default value lcr. Currently, lcr and kata-runtime are supported.

    None

    "runtimes":  {
        "kata-runtime": {
          "path": "/usr/bin/kata-runtime",
          "runtime-args": [
            "--kata-config",
            "/usr/share/defaults/kata-containers/configuration.toml"
          ]
        }
    }

    When starting a container, set this parameter to specify multiple runtimes. Runtimes in this set are valid for container startup.

    Runtime whitelist of a container. The customized runtimes in this set are valid. kata-runtime is used as the example.

    -p, --pidfile

    "pidfile": "/var/run/isulad.pid"

    File for storing PIDs.

    This parameter is required only when more than two container engines need to be started.

    -g, --graph

    "graph": "/var/lib/isulad"

    Root directory for iSulad runtimes.

    -S, --state

    "state": "/var/run/isulad"

    Root directory of the execution file.

    --storage-driver

    "storage-driver": "overlay2"

    Image storage driver, which is overlay2 by default.

    Only overlay2 is supported.

    -s, --storage-opt

    "storage-opts": [ "overlay2.override_kernel_check=true" ]

    Image storage driver configuration options.

    The options are as follows:

    overlay2.override_kernel_check=true #Ignore the kernel version check.
    overlay2.size=${size} #Set the rootfs quota to ${size}.
    overlay2.basesize=${size} #It is equivalent to overlay2.size.

    --image-opt-timeout

    "image-opt-timeout": "5m"

    Image operation timeout interval, which is 5m by default.

    The value -1 indicates that the timeout interval is not limited.

    --registry-mirrors

    "registry-mirrors": [ "docker.io" ]

    Registry address.

    None

    --insecure-registry

    "insecure-registries": [ ]

    Registry without TLS verification.

    None

    --native.umask

    "native.umask": "secure"

    Container umask policy. The default value is secure. The value normal indicates insecure configuration.

    Set the container umask value.

    The value can be null (0027 by default), normal, or secure.

    normal #The umask value of the started container is 0022.
    secure #The umask value of the started container is 0027 (default value).

    --pod-sandbox-image

    "pod-sandbox-image": "rnd-dockerhub.huawei.com/library/pause-aarch64:3.0"

    By default, the pod uses the image. The default value is rnd-dockerhub.huawei.com/library/pause-${machine}:3.0.

    None

    --network-plugin

    "network-plugin": ""

    Specifies a network plug-in. The value is a null character by default, indicating that no network configuration is available and the created sandbox has only the loop NIC.

    The CNI and null characters are supported. Other invalid values will cause iSulad startup failure.

    --cni-bin-dir

    "cni-bin-dir": ""

    Specifies the storage location of the binary file on which the CNI plug-in depends.

    The default value is /opt/cni/bin.

    --cni-conf-dir

    "cni-conf-dir": ""

    Specifies the storage location of the CNI network configuration file.

    The default value is /etc/cni/net.d.

    --image-layer-check=false

    "image-layer-check": false

    Image layer integrity check. To enable the function, set it to true; otherwise, set it to false. It is disabled by default.

    When iSulad is started, the image layer integrity is checked. If the image layer is damaged, the related images are unavailable. iSulad cannot verify empty files, directories, and link files. Therefore, if the preceding files are lost due to a power failure, the integrity check of iSulad image data may fail to be identified. When the iSulad version changes, check whether the parameter is supported. If not, delete it from the configuration file.

    --insecure-skip-verify-enforce=false

    "insecure-skip-verify-enforce": false

    Indicates whether to forcibly skip the verification of the certificate host name/domain name. The value is of the Boolean type, and the default value is false. If this parameter is set to true, the verification of the certificate host name/domain name is skipped.

    The default value is false (not skipped). Note: Restricted by the YAJL JSON parsing library, if a non-Boolean value that meets the JSON format requirements is configured in the /etc/isulad/daemon.json configuration file, the default value used by iSulad is false.

    --use-decrypted-key=true

    "use-decrypted-key": true

    Specifies whether to use an unencrypted private key. The value is of the Boolean type. If this parameter is set to true, an unencrypted private key is used. If this parameter is set to false, the encrypted private key is used, that is, two-way authentication is required.

    The default value is true, indicating that an unencrypted private key is used. Note: Restricted by the YAJL JSON parsing library, if a non-Boolean value that meets the JSON format requirements is configured in the /etc/isulad/daemon.json configuration file, the default value used by iSulad is true.

    --tls

    "tls":false

    Specifies whether to use TLS. The value is of the Boolean type.

    This parameter is used only in -H tcp://IP:PORT mode. The default value is false.

    --tlsverify

    "tlsverify":false

    Specifies whether to use TLS and verify remote access. The value is of the Boolean type.

    This parameter is used only in -H tcp://IP:PORT mode.

    --tlscacert

    --tlscert

    --tlskey

    "tls-config": {

    "CAFile": "/root/.iSulad/ca.pem",

    "CertFile": "/root/.iSulad/server-cert.pem",

    "KeyFile":"/root/.iSulad/server-key.pem"

    }

    TLS certificate-related configuration.

    This parameter is used only in -H tcp://IP:PORT mode.

    --authorization-plugin

    "authorization-plugin": "authz-broker"

    User permission authentication plugin.

    Only authz-broker is supported.

    --cgroup-parent

    "cgroup-parent": "lxc/mycgroup"

    Default cgroup parent path of a container, which is of the string type.

    Specifies the cgroup parent path of a container. If --cgroup-parent is specified on the client, the client parameter prevails.

    Note: If container A is started before container B, the cgroup parent path of container B is specified as the cgroup path of container A. When deleting a container, you need to delete container B and then container A in sequence. Otherwise, residual cgroup resources exist.

    --default-ulimits

    "default-ulimits": {

    "nofile": {

    "Name": "nofile",

    "Hard": 6400,

    "Soft": 3200

    }

    }

    Specifies the ulimit restriction type, soft value, and hard value.

    Specifies the restricted resource type, for example, nofile. The two field names must be the same, that is, nofile. Otherwise, an error is reported. The value of Hard must be greater than or equal to that of Soft. If the Hard or Soft field is not set, the default value 0 is used.

    --websocket-server-listening-port

    "websocket-server-listening-port": 10350

    Specifies the listening port of the CRI WebSocket streaming service. The default port number is 10350.

    Specifies the listening port of the CRI websocket streaming service.

    If the client specifies --websocket-server-listening-port, the specified value is used. The port number ranges from 1024 to 49151.

    Example:

    $ cat /etc/isulad/daemon.json
    {
        "group": "isulad",
        "default-runtime": "lcr",
        "graph": "/var/lib/isulad",
        "state": "/var/run/isulad",
        "engine": "lcr",
        "log-level": "ERROR",
        "pidfile": "/var/run/isulad.pid",
        "log-opts": {
            "log-file-mode": "0600",
            "log-path": "/var/lib/isulad",
            "max-file": "1",
            "max-size": "30KB"
        },
        "log-driver": "stdout",
        "hook-spec": "/etc/default/isulad/hooks/default.json",
        "start-timeout": "2m",
        "storage-driver": "overlay2",
        "storage-opts": [
            "overlay2.override_kernel_check=true"
        ],
        "registry-mirrors": [
            "docker.io"
        ],
        "insecure-registries": [
            "rnd-dockerhub.huawei.com"
        ],
        "pod-sandbox-image": "",
        "image-opt-timeout": "5m",
        "native.umask": "secure",
        "network-plugin": "",
        "cni-bin-dir": "",
        "cni-conf-dir": "",
        "image-layer-check": false,
        "use-decrypted-key": true,
        "insecure-skip-verify-enforce": false
    }
    

    NOTICE:
    The default configuration file /etc/isulad/daemon.json is for reference only. Configure it based on site requirements.

Storage Description

File

Directory

Description

\*

/etc/default/isulad/

Stores the OCI configuration file and hook template file of iSulad. The file configuration permission is set to 0640, and the sysmonitor check permission is set to 0550.

\*

/etc/isulad/

Default configuration files of iSulad and seccomp.

isulad.sock

/var/run/

Pipe communication file, which is used for the communication between the client and iSulad.

isulad.pid

/var/run/

File for storing the iSulad PIDs. It is also a file lock to prevent multiple iSulad instances from being started.

\*

/run/lxc/

Lock file, which is created during iSulad running.

\*

/var/run/isulad/

Real-time communication cache file, which is created during iSulad running.

\*

/var/run/isula/

Real-time communication cache file, which is created during iSulad running.

\*

/var/lib/lcr/

Temporary directory of the LCR component.

\*

/var/lib/isulad/

Root directory where iSulad runs, which stores the created container configuration, default log path, database file, and mount point.

/var/lib/isulad/mnt/: mount point of the container rootfs.

/var/lib/isulad/engines/lcr/: directory for storing LCR container configurations. Each container has a directory named after the container.

Constraints

  • In high concurrency scenarios (200 containers are concurrently started), the memory management mechanism of Glibc may cause memory holes and large virtual memory (for example, 10 GB). This problem is caused by the restriction of the Glibc memory management mechanism in the high concurrency scenario, but not by memory leakage. Therefore, the memory consumption does not increase infinitely. You can set MALLOC_ARENA_MAX to reducevirtual memory error and increase the rate of reducing physical memory. However, this environment variable will cause the iSulad concurrency performance to deteriorate. Set this environment variable based on the site requirements.

    To balance performance and memory usage, set MALLOC_ARENA_MAX to 4. (The iSulad performance on the ARM64 server is affected by less than 10%.)
    
    Configuration method:
    1. To manually start iSulad, run the export MALLOC_ARENA_MAX=4 command and then start iSulad.
    2. If systemd manages iSulad, you can modify the /etc/sysconfig/iSulad file by adding MALLOC_ARENA_MAX=4.
    
  • Precautions for specifying the daemon running directories

    Take –root as an example. When /new/path/ is used as the daemon new root directory, if a file exists in /new/path/ and the directory or file name conflicts with that required by iSulad (for example, engines and mnt), iSulad may update the original directory or file attributes including the owner and permission.

    Therefore, please note the impact of re-specifying various running directories and files on their attributes. You are advised to specify a new directory or file for iSulad to avoid file attribute changes and security issues caused by conflicts.

  • Log file management:

    NOTICE:
    Log function interconnection: logs are managed by systemd as iSulad is and then transmitted to rsyslogd. By default, rsyslog restricts the log writing speed. You can add the configuration item $imjournalRatelimitInterval 0 to the /etc/rsyslog.conf file and restart the rsyslogd service.

  • Restrictions on command line parameter parsing

    When the iSulad command line interface is used, the parameter parsing mode is slightly different from that of Docker. For flags with parameters in the command line, regardless of whether a long or short flag is used, only the first space after the flag or the character string after the equal sign (=) directly connected to the flag is used as the flag parameter. The details are as follows:

    1. When a short flag is used, each character in the character string connected to the hyphen (-) is considered as a short flag. If there is an equal sign (=), the character string following the equal sign (=) is considered as the parameter of the short flag before the equal sign (=).

      isula run -du=root busybox is equivalent to isula run -du root busybox, isula run -d -u=root busybox, or isula run -d -u root busybox. When isula run -du:root is used, as -: is not a valid short flag, an error is reported. The preceding command is equivalent to isula run -ud root busybox. However, this method is not recommended because it may cause semantic problems.

    2. When a long flag is used, the character string connected to is regarded as a long flag. If the character string contains an equal sign (=), the character string before the equal sign (=) is a long flag, and the character string after the equal sign (=) is a parameter.

      isula run --user=root busybox
      

      or

      isula run --user root busybox
      
  • After an iSulad container is started, you cannot run the isula run -i/-t/-ti and isula attach/exec commands as a non-root user.

  • When iSulad connects to an OCI container, only kata-runtime can be used to start the OCI container.

Daemon Multi-Port Binding

Description

The daemon can bind multiple UNIX sockets or TCP ports and listen on these ports. The client can interact with the daemon through these ports.

Port

Users can configure one or more ports in the hosts field in the /etc/isulad/daemon.json file, or choose not to specify hosts.

{
    "hosts": [
        "unix:///var/run/isulad.sock",
        "tcp://localhost:5678",
        "tcp://127.0.0.1:6789"
    ]
}

Users can also run the -H or –host command in the /etc/sysconfig/iSulad file to configure a port, or choose not to specify hosts.

OPTIONS='-H unix:///var/run/isulad.sock --host tcp://127.0.0.1:6789'

If hosts are not specified in the daemon.json file and iSulad, the daemon listens on unix:///var/run/isulad.sock by default after startup.

Restrictions

  • Users cannot specify hosts in the /etc/isulad/daemon.json and /etc/sysconfig/iSuald files at the same time. Otherwise, an error will occur and iSulad cannot be started.

    unable to configure the isulad with file /etc/isulad/daemon.json: the following directives are specified both as a flag and in the configuration file: hosts: (from flag: [unix:///var/run/isulad.sock tcp://127.0.0.1:6789], from file: [unix:///var/run/isulad.sock tcp://localhost:5678 tcp://127.0.0.1:6789])
    
  • If the specified host is a UNIX socket, the socket must start with unix:// followed by a valid absolute path.

  • If the specified host is a TCP port, the TCP port number must start with tcp:// followed by a valid IP address and port number. The IP address can be that of the local host.

  • A maximum of 10 valid ports can be specified. If more than 10 ports are specified, an error will occur and iSulad cannot be started.

Configuring TLS Authentication and Enabling Remote Access

Description

iSulad is designed in C/S mode. By default, the iSulad daemon process listens only on the local/var/run/isulad.sock. Therefore, you can run commands to operate containers only on the local client iSula. To enable iSula's remote access to the container, the iSulad daemon process needs to listen on the remote access port using TCP/IP. However, listening is performed only by simply configuring tcp ip:port. In this case, all IP addresses can communicate with iSulad by calling isula -H tcp://remote server IP address:port, which may cause security problems. Therefore, it is recommended that a more secure version, namely Transport Layer Security (TLS), be used for remote access.

Generating TLS Certificate

  • Example of generating a plaintext private key and certificate

    #!/bin/bash
    set -e
    echo -n "Enter pass phrase:"
    read password
    echo -n "Enter public network ip:"
    read publicip
    echo -n "Enter host:"
    read HOST
    
    echo " => Using hostname: $publicip, You MUST connect to iSulad using this host!"
    
    mkdir -p $HOME/.iSulad
    cd $HOME/.iSulad
    rm -rf $HOME/.iSulad/*
    
    echo " => Generating CA key"
    openssl genrsa -passout pass:$password -aes256 -out ca-key.pem 4096
    echo " => Generating CA certificate"
    openssl req -passin pass:$password -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem -subj "/C=CN/ST=zhejiang/L=hangzhou/O=Huawei/OU=iSulad/CN=iSulad@huawei.com"
    echo " => Generating server key"
    openssl genrsa -passout pass:$password -out server-key.pem 4096
    echo " => Generating server CSR"
    openssl req -passin pass:$password -subj /CN=$HOST -sha256 -new -key server-key.pem -out server.csr
    echo subjectAltName = DNS:$HOST,IP:$publicip,IP:127.0.0.1 >> extfile.cnf
    echo extendedKeyUsage = serverAuth >> extfile.cnf
    echo " => Signing server CSR with CA"
    openssl x509 -req -passin pass:$password -days 365 -sha256 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile extfile.cnf
    echo " => Generating client key"
    openssl genrsa -passout pass:$password -out key.pem 4096
    echo " => Generating client CSR"
    openssl req -passin pass:$password -subj '/CN=client' -new -key key.pem -out client.csr
    echo " => Creating extended key usage"
    echo extendedKeyUsage = clientAuth > extfile-client.cnf
    echo " => Signing client CSR with CA"
    openssl x509 -req -passin pass:$password -days 365 -sha256 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cert.pem -extfile extfile-client.cnf
    rm -v client.csr server.csr extfile.cnf extfile-client.cnf
    chmod -v 0400 ca-key.pem key.pem server-key.pem
    chmod -v 0444 ca.pem server-cert.pem cert.pem
    
  • Example of generating an encrypted private key and certificate request file

    #!/bin/bash
    
    echo -n "Enter public network ip:"
    read publicip
    echo -n "Enter pass phrase:"
    read password
    
    # remove certificates from previous execution.
    rm -f *.pem *.srl *.csr *.cnf
    
    
    # generate CA private and public keys
    echo 01 > ca.srl
    openssl genrsa -aes256 -out ca-key.pem -passout pass:$password 2048
    openssl req -subj '/C=CN/ST=zhejiang/L=hangzhou/O=Huawei/OU=iSulad/CN=iSulad@huawei.com' -new -x509 -days $DAYS -passin pass:$password -key ca-key.pem -out ca.pem
    
    # create a server key and certificate signing request (CSR)
    openssl genrsa -aes256 -out server-key.pem -passout pass:$PASS 2048
    openssl req -new -key server-key.pem -out server.csr -passin pass:$password -subj '/CN=iSulad'
    
    echo subjectAltName = DNS:iSulad,IP:${publicip},IP:127.0.0.1 > extfile.cnf
    echo extendedKeyUsage = serverAuth >> extfile.cnf
    # sign the server key with our CA
    openssl x509 -req -days $DAYS -passin pass:$password -in server.csr -CA ca.pem -CAkey ca-key.pem -out server-cert.pem -extfile extfile.cnf
    
    # create a client key and certificate signing request (CSR)
    openssl genrsa -aes256 -out key.pem -passout pass:$password 2048
    openssl req -subj '/CN=client' -new -key key.pem -out client.csr -passin pass:$password
    
    # create an extensions config file and sign
    echo extendedKeyUsage = clientAuth > extfile.cnf
    openssl x509 -req -days 365 -passin pass:$password -in client.csr -CA ca.pem -CAkey ca-key.pem -out cert.pem -extfile extfile.cnf
    
    # remove the passphrase from the client and server key
    openssl rsa -in server-key.pem -out server-key.pem -passin pass:$password
    openssl rsa -in key.pem -out key.pem -passin pass:$password
    
    # remove generated files that are no longer required
    rm -f ca-key.pem ca.srl client.csr extfile.cnf server.csr
    

APIs

{
    "tls": true,
    "tls-verify": true,
    "tls-config": {
		"CAFile": "/root/.iSulad/ca.pem",
		"CertFile": "/root/.iSulad/server-cert.pem",
		"KeyFile":"/root/.iSulad/server-key.pem"
    }
}

Restrictions

The server supports the following modes:

  • Mode 1 (client verified): tlsverify, tlscacert, tlscert, tlskey
  • Mode 2 (client not verified): tls, tlscert, tlskey

The client supports the following modes:

  • Mode 1 (verify the identity based on the client certificate, and verify the server based on the specified CA): tlsverify, tlscacert, tlscert, tlskey
  • Mode 2 (server verified): tlsverify, tlscacert

Mode 1 is used for the server, and mode 2 for the client if the two-way authentication mode is used for communication.

Mode 2 is used for the server and the client if the unidirectional authentication mode is used for communication.

NOTICE:

  • If RPM is used for installation, the server configuration can be modified in the /etc/isulad/daemon.json and /etc/sysconfig/iSulad files.
  • Two-way authentification is recommended as it is more secure than non-authentication or unidirectional authentication.
  • GRPC open-source component logs are not taken over by iSulad. To view gRPC logs, set the environment variables gRPC_VERBOSITY and gRPC_TRACE as required.
      

Example

On the server:

 isulad -H=tcp://0.0.0.0:2376 --tlsverify --tlscacert ~/.iSulad/ca.pem --tlscert ~/.iSulad/server-cert.pem --tlskey ~/.iSulad/server-key.pem

On the client:

 isula version -H=tcp://$HOSTIP:2376 --tlsverify --tlscacert ~/.iSulad/ca.pem --tlscert ~/.iSulad/cert.pem --tlskey ~/.iSulad/key.pem

devicemapper Storage Driver Configuration

To use the devicemapper storage driver, you need to configure a thinpool device which requires an independent block device with sufficient free space. Take the independent block device /dev/xvdf as an example. The configuration method is as follows:

  1. Configuring a thinpool

  2. Stop the iSulad service.

    # systemctl stop isulad
    
  3. Create a logical volume manager (LVM) volume based on the block device.

    # pvcreate /dev/xvdf
    
  4. Create a volume group based on the created physical volume.

    # vgcreate isula /dev/xvdf
    Volume group "isula" successfully created:
    
  5. Create two logical volumes named thinpool and thinpoolmeta.

    # lvcreate --wipesignatures y -n thinpool isula -l 95%VG
    Logical volume "thinpool" created.
    
    # lvcreate --wipesignatures y -n thinpoolmeta isula -l 1%VG
    Logical volume "thinpoolmeta" created.
    
  6. Convert the two logical volumes into a thinpool and the metadata used by the thinpool.

    # lvconvert -y --zero n -c 512K --thinpool isula/thinpool --poolmetadata isula/thinpoolmeta
    
    WARNING: Converting logical volume isula/thinpool and isula/thinpoolmeta to
    thin pool's data and metadata volumes with metadata wiping.
    THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
    Converted isula/thinpool to thin pool.
    

  

  1. Modifying the iSulad configuration files

  2. If iSulad has been used in the environment, back up the running data first.

    # mkdir /var/lib/isulad.bk
    # mv /var/lib/isulad/* /var/lib/isulad.bk
    
  3. Modify configuration files.

    Two configuration methods are provided. Select one based on site requirements.

    • Edit the /etc/isulad/daemon.json file, set storage-driver to devicemapper, and set parameters related to the storage-opts field. For details about related parameters, see Parameter Description. The following lists the configuration reference:

      {
          "storage-driver": "devicemapper"
          "storage-opts": [
              "dm.thinpooldev=/dev/mapper/isula-thinpool",
              "dm.fs=ext4",
                  "dm.min_free_space=10%"
          ]
      }
      
    • You can also edit /etc/sysconfig/iSulad to explicitly specify related iSulad startup parameters. For details about related parameters, see Parameter Description. The following lists the configuration reference:

      OPTIONS="--storage-driver=devicemapper --storage-opt dm.thinpooldev=/dev/mapper/isula-thinpool --storage-opt dm.fs=ext4 --storage-opt dm.min_free_space=10%"
      
  4. Start iSulad for the settings to take effect.

    # systemctl start isulad
    

Parameter Description

For details about parameters supported by storage-opts, see Table 1.

Table 1 Parameter description

Parameter

Mandatory or Not

Description

dm.fs

Yes

Specifies the type of the file system used by a container. This parameter must be set to ext4, that is, dm.fs=ext4.

dm.basesize

No

Specifies the maximum storage space of a single container. The unit can be k, m, g, t, or p. An uppercase letter can also be used, for example, dm.basesize=50G. This parameter is valid only during the first initialization.

dm.mkfsarg

No

Specifies the additional mkfs parameters when a basic device is created. For example: dm.mkfsarg=-O ^has_journal

dm.mountopt

No

Specifies additional mount parameters when a container is mounted. For example: dm.mountopt=nodiscard

dm.thinpooldev

No

Specifies the thinpool device used for container or image storage.

dm.min_free_space

No

Specifies minimum percentage of reserved space. For example, dm.min_free_space=10% indicates that storage-related operations such as container creation will fail when the remaining storage space falls below 10%.

Precautions

  • When configuring devicemapper, if the system does not have sufficient space for automatic capacity expansion of thinpool, disable the automatic capacity expansion function.

    To disable automatic capacity expansion, set both thin_pool_autoextend_threshold and thin_pool_autoextend_percent in the /etc/lvm/profile/isula-thinpool.profile file to 100.

    activation {   
      thin_pool_autoextend_threshold=100   
      thin_pool_autoextend_percent=100 
    }
    
  • When devicemapper is used, use Ext4 as the container file system. You need to add –storage-opt dm.fs=ext4 to the iSulad configuration parameters.

  • If graphdriver is devicemapper and the metadata files are damaged and cannot be restored, you need to manually restore the metadata files. Do not directly operate or tamper with metadata of the devicemapper storage driver in Docker daemon.

  • When the devicemapper LVM is used, if the devicemapper thinpool is damaged due to abnormal power-off, you cannot ensure the data integrity or whether the damaged thinpool can be restored. Therefore, you need to rebuild the thinpool.

Precautions for Switching the devicemapper Storage Pool When the User Namespace Feature Is Enabled on iSula

  • Generally, the path of the deviceset-metadata file is /var/lib/isulad/devicemapper/metadata/deviceset-metadata during container startup.
  • If user namespaces are used, the path of the deviceset-metadata file is /var/lib/isulad/userNSUID.GID/devicemapper/metadata/deviceset-metadata.
  • When you use the devicemapper storage driver and the container is switched between the user namespace scenario and common scenario, the BaseDeviceUUID content in the corresponding deviceset-metadata file needs to be cleared. In the thinpool capacity expansion or rebuild scenario, you also need to clear the BaseDeviceUUID content in the deviceset-metadata file. Otherwise, the iSulad service fails to be restarted.
Rating

search results about are found