LTS

    Innovation Version

      Configuring Networking for a Secure Container

      TAP-based Network Support

      The secure container technology is implemented based on QEMU VMs. For a physical machine system, a secure container is equivalent to a VM. Therefore, the secure container may connect the VM to an external network in the Neutron network by using the test access point (TAP) technology. You do not need to pay attention to TAP device creation and bridging. You only need to hot add the specified TAP device (with an existing host) to the VM in the pause container and update the NIC information.

      Related commands are as follows:

      1. Run the following command to add a TAP NIC for a started container:

        $ cat ./test-iface.json | kata-runtime kata-network add-iface 6ec7a98 -
        

        In the preceding command, 6ec7a98 is the truncated container ID, and test-iface.json is the file that describes the NIC information. The following is an example:

        {
            "device": "tap-test", 
            "name": "eth-test", 
            "IPAddresses": [
                {
                    "address": "172.16.0.3", 
                    "mask": "16"
                }
            ], 
            "hwAddr":"02:42:20:6f:a3:69",
            "mtu": 1500,
            "vhostUserSocket":"/usr/local/var/run/openvswitch/vhost-user1",
            "queues":5
        }
        

        The fields in the JSON file are described as follows:

        Field

        Mandatory/Optional

        Description

        device

        Mandatory

        Name of the NIC on a host. The value can contain a maximum of 15 characters, including letters, digits, underscores (_), hyphens (-), and periods (.). It must start with a letter. The device name must be unique on the same host.

        name

        Mandatory

        Name of the NIC in the container. The value can contain a maximum of 15 characters, including letters, digits, underscores (_), hyphens (-), and periods (.). It must start with a letter. The name must be unique in the same sandbox.

        IPAddresses

        Optional

        IP address of the NIC. Currently, one IP address can be configured for each NIC. If no IP address is configured for the NIC, no IP address will be configured in the container, either.

        hwAddr

        Mandatory

        MAC address of the NIC.

        mtu

        Mandatory

        MTU of the NIC. The value ranges from 46 to 9600.

        vhostUserSocket

        Optional

        Socket path for DPDK polling. The path contains a maximum of 128 bytes. The naming rule can contain digits, letters, and hyphens (-). The path name must start with a letter.

        queues

        Optional

        Number of NIC queues. If this parameter is not set, the default value 0 is used.

        The following describes the output of the kata-runtime kata-network add-iface command for adding NICs:

        • If the command is successfully executed, the NIC information in JSON format is returned from standard output (stdout). The content in JSON format is the same as the input NIC information.

          Example:

          $ kata-runtime kata-network add-iface <container-id> net.json 
          {"device":"tap_test","name":"eth-test","IPAddresses":[{"Family":2,"Address":"173.85.100.1","Mask":"24"}],"mtu":1500,"hwAddr":"02:42:20:6e:03:01","pciAddr":"01.0/00"}
          
        • If the command fails to be executed, null is returned from stdout.

          Example:

          $ kata-runtime kata-network add-iface <container-id> netbad.json 2>/dev/null
          null
          

        NOTE:
        If an IP address is specified for an NIC that is successfully added, Kata adds a default route whose destination is in the same network segment as the IP address of the NIC. In the preceding example, after the NIC is added, the following route is added to the container:

        [root@6ec7a98 /]# ip route  
        172.16.0.0/16 dev eth-test proto kernel scope link src 172.16.0.3  
        
      2. Run the following command to view the added NICs:

        $ kata-runtime kata-network list-ifaces 6ec7a98
        [{"name":"eth-test","mac":"02:42:20:6f:a3:69","ip":["172.16.0.3/16"],"mtu":1500}]
        

        The information about the added NICs is displayed.

        The following describes the output of the kata-runtime kata-network list-ifacescommand for listing added NICs:

        • If the command is executed successfully, information about all NICs inserted into the pod in JSON format is returned from stdout.

          If multiple NICs are inserted into the pod, the NIC information in JSON array format is returned.

          $ kata-runtime kata-network list-ifaces <container-id>
          [{"name":"container_eth","mac":"02:42:20:6e:a2:59","ip":["172.17.25.23/8"],"mtu":1500},{"name":"container_eth_2","mac":"02:90:50:6b:a2:29","ip":["192.168.0.34/24"],"mtu":1500}]
          

          If no NIC is inserted into the pod, null is returned from stdout.

          $ kata-runtime kata-network list-ifaces <container-id>
          null
          
        • If the command fails to be executed, null is returned from stdout, and error description is returned from standard error (stderr).

          Example:

          $ kata-runtime kata-network list-ifaces <container-id>
          null
          
      3. Add a route for a specified NIC.

        $ cat ./test-route.json | kata-runtime kata-network add-route 6ec7a98 -
        [{"dest":"default","gateway":"172.16.0.1","device":"eth-test"}]
        

        The following describes the output of the kata-runtime kata-network add-route command for adding a route to a specified NIC:

        • If the command is executed successfully, the added route information in JSON format is returned from stdout.

          Example:

          $ kata-runtime kata-network add-route <container-id> route.json 
          [{"dest":"177.17.0.0/24","gateway":"177.17.25.1","device":"netport_test_1"}]
          
        • If the command fails to be executed, null is returned from stdout, and error description is returned from standard error (stderr).

          Example:

          $ kata-runtime kata-network add-route <container-id> routebad.json 2>/dev/null
          null
          

        Key fields are described as follows:

        • dest: Network segment corresponding to the route. The value is in the format of <ip>/<mask>. <ip> is mandatory. There are three cases:

          1. Both IP address and mask are configured.
          2. If only an IP address is configured, the default mask is 32.
          3. If "dest":"default" is configured, there is no destination by default. In this case, the gateway needs to be configured.
        • gateway: Next-hop gateway of the route. When "dest":"default" is configured, the gateway is mandatory. In other cases, this parameter is optional.

        • device: Name of the NIC corresponding to the route, which is mandatory. The value contains a maximum of 15 characters.

        NOTE:
        If a route is added for the loopback device lo in the container, the device name corresponding to the device field in the route configuration file is lo.

      4. Run the following command to delete a specified route:

        $ cat ./test-route.json | kata-runtime kata-network del-route 6ec7a98 -
        

        The fields in the test-route.json file are the same as those in the JSON file for adding a route.

        The following describes the output of thekata-runtime kata-network del-route command for deleting a specified route:

        • If the command is executed successfully, the added route information in JSON format is returned from stdout.

          Example:

          $ kata-runtime kata-network del-route <container-id> route.json 
          [{"dest":"177.17.0.0/24","gateway":"177.17.25.1","device":"netport_test_1"}]
          
        • If the command fails to be executed, null is returned from stdout, and error description is returned from standard error (stderr).

          Example:

          $ kata-runtime kata-network del-route <container-id> routebad.json 2>/dev/null
          null
          

        NOTE:

        • In the input fields, dest is mandatory, and both device and gateway are optional. Kata performs fuzzy match based on different fields and deletes the corresponding routing rules. For example, if dest is set to an IP address, all rules of this IP address will be deleted.
        • If the route of the loopback device lo in the container is deleted, the device name corresponding to the device field in the route configuration file is lo.
      5. Run the following command to delete an NIC:

        $ cat ./test-iface.json | kata-runtime kata-network del-iface 6ec7a98 -
        

        NOTE:
        When deleting an NIC, you can only delete it based on the name field in the NIC container. Kata does not identify other fields.

        The following describes the output of the kata-runtime kata-network del-ifacecommand for deleting NICs:

        • If the command is executed successfully, null is returned from stdout.

          Example:

          $ kata-runtime kata-network del-iface <container-id> net.json
          null
          
        • If the command fails to be executed, the information about NICs that fail to be deleted in JSON format is returned from stdout, and error description is returned from stderr.

          Example:

          $ kata-runtime kata-network del-iface <container-id> net.json
          {"device":"tapname_fun_012","name":"netport_test_1","IPAddresses":[{"Family":0,"Address":"177.17.0.1","Mask":"8"}],"mtu":1500,"hwAddr":"02:42:20:6e:a2:59","linkType":"tap"}
          

      The preceding are common commands. For details about the command line interfaces, see APIs.

      Kata IPVS Subsystem

      The secure container provides an API for adding the ipvs command and setting the IPVS rule for the container. The functions include adding, editing, and deleting virtual services, adding, editing, and deleting real servers, querying IPVS service information, setting connection timeout, clearing the system connection cache, and importing rules in batches.

      1. Add a virtual service address for the container.

        kata-runtime kata-ipvs ipvsadm --parameters "--add-service --tcp-service 172.17.0.7:80 --scheduler rr --persistent 3000" <container-id>
        
      2. Modify virtual service parameters of a container.

        kata-runtime kata-ipvs ipvsadm --parameters "--edit-service --tcp-service 172.17.0.7:80 --scheduler rr --persistent 5000" <container-id>
        
      3. Delete the virtual service address of a container.

        kata-runtime kata-ipvs ipvsadm --parameters "--delete-service --tcp-service 172.17.0.7:80" <container-id>
        
      4. Add a real server for the virtual service address.

        kata-runtime kata-ipvs ipvsadm --parameters "--add-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80 --weight 100" <container-id>
        
      5. Modify real server parameters of a container.

        kata-runtime kata-ipvs ipvsadm --parameters "--edit-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80 --weight 200" <container-id>
        
      6. Delete a real server from a container.

        kata-runtime kata-ipvs ipvsadm --parameters "--delete-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80" <container-id>
        
      7. Query service information.

        kata-runtime kata-ipvs ipvsadm --parameters "--list" <container-id>
        
      8. It takes a long time to import rules one by one. You can write rules into a file and import them in batches.

        kata-runtime kata-ipvs ipvsadm --restore - < <rule file path> <container-id>
        

        NOTE:
        By default, the NAT mode is used for adding a single real server. To add real servers in batches, you need to manually add the -m option to use the NAT mode.
        The following is an example of the rule file content:
        -A -t 10.10.11.12:100 -s rr -p 3000
        -a -t 10.10.11.12:100 -r 172.16.0.1:80 -m
        -a -t 10.10.11.12:100 -r 172.16.0.1:81 -m
        -a -t 10.10.11.12:100 -r 172.16.0.1:82 -m

      9. Clear the system connection cache.

        kata-runtime kata-ipvs cleanup --parameters "--orig-dst 172.17.0.4 --protonum tcp" <container-id>
        
      10. Set timeout interval for TCP, TCP FIN, or UDP connections.

        kata-runtime kata-ipvs ipvsadm --parameters "--set 100 100 200" <container-id>
        

        NOTE:

        1. Each container supports a maximum of 20000 iptables rules (5000 services and three servers/services). Both add-service and add-server are rules.
        2. Before importing rules in batches, you need to clear existing rules.
        3. No concurrent test scenario exists.
        4. The preceding are common commands. For details about the command line interfaces, see APIs.

      Bug Catching

      Buggy Content

      Bug Description

      Submit As Issue

      It's a little complicated....

      I'd like to ask someone.

      PR

      Just a small problem.

      I can fix it online!

      Bug Type
      Specifications and Common Mistakes

      ● Misspellings or punctuation mistakes;

      ● Incorrect links, empty cells, or wrong formats;

      ● Chinese characters in English context;

      ● Minor inconsistencies between the UI and descriptions;

      ● Low writing fluency that does not affect understanding;

      ● Incorrect version numbers, including software package names and version numbers on the UI.

      Usability

      ● Incorrect or missing key steps;

      ● Missing prerequisites or precautions;

      ● Ambiguous figures, tables, or texts;

      ● Unclear logic, such as missing classifications, items, and steps.

      Correctness

      ● Technical principles, function descriptions, or specifications inconsistent with those of the software;

      ● Incorrect schematic or architecture diagrams;

      ● Incorrect commands or command parameters;

      ● Incorrect code;

      ● Commands inconsistent with the functions;

      ● Wrong screenshots.

      Risk Warnings

      ● Lack of risk warnings for operations that may damage the system or important data.

      Content Compliance

      ● Contents that may violate applicable laws and regulations or geo-cultural context-sensitive words and expressions;

      ● Copyright infringement.

      How satisfied are you with this document

      Not satisfied at all
      Very satisfied
      Submit
      Click to create an issue. An issue template will be automatically generated based on your feedback.
      Bug Catching
      编组 3备份