Configuring Networking for a Secure Container
TAP-based Network Support
The secure container technology is implemented based on QEMU VMs. For a physical machine system, a secure container is equivalent to a VM. Therefore, the secure container may connect the VM to an external network in the Neutron network by using the test access point (TAP) technology. You do not need to pay attention to TAP device creation and bridging. You only need to hot add the specified TAP device (with an existing host) to the VM in the pause container and update the NIC information.
Related commands are as follows:
Run the following command to add a TAP NIC for a started container:
$ cat ./test-iface.json | kata-runtime kata-network add-iface 6ec7a98 -In the preceding command, 6ec7a98 is the truncated container ID, and test-infs.json is the file that describes the NIC information. The following is an example:
{ "device": "tap-test", "name": "eth-test", "IPAddresses": [ { "address": "172.16.0.3", "mask": "16" } ], "hwAddr":"02:42:20:6f:a3:69", "mtu": 1500, "vhostUserSocket":"/usr/local/var/run/openvswitch/vhost-user1", "queues":5 }The fields in the JSON file are described as follows:
The following describes the output of the kata-runtime kata-network add-iface command for adding NICs:
If the command is successfully executed, the NIC information in JSON format is returned from standard output (stdout). The content in JSON format is the same as the input NIC information.
Example:
$ kata-runtime kata-network add-iface <container-id> net.json {"device":"tap_test","name":"eth-test","IPAddresses":[{"Family":2,"Address":"173.85.100.1","Mask":"24"}],"mtu":1500,"hwAddr":"02:42:20:6e:03:01","pciAddr":"01.0/00"}If the command fails to be executed, null is returned from stdout.
Example:
$ kata-runtime kata-network add-iface <container-id> netbad.json 2>/dev/null null
NOTE:
If an IP address is specified for an NIC that is successfully added, Kata adds a default route whose destination is in the same network segment as the IP address of the NIC. In the preceding example, after the NIC is added, the following route is added to the container:[root@6ec7a98 /]# ip route 172.16.0.0/16 dev eth-test proto kernel scope link src 172.16.0.3Run the following command to view the added NICs:
$ kata-runtime kata-network list-ifaces 6ec7a98 [{"name":"eth-test","mac":"02:42:20:6f:a3:69","ip":["172.16.0.3/16"],"mtu":1500}]The information about the added NICs is displayed.
The following describes the output of the **kata-runtime kata-network list-ifaces **command for listing added NICs:
If the command is executed successfully, information about all NICs inserted into the pod in JSON format is returned from stdout.
If multiple NICs are inserted into the pod, the NIC information in JSON array format is returned.
$ kata-runtime kata-network list-ifaces <container-id> [{"name":"container_eth","mac":"02:42:20:6e:a2:59","ip":["172.17.25.23/8"],"mtu":1500},{"name":"container_eth_2","mac":"02:90:50:6b:a2:29","ip":["192.168.0.34/24"],"mtu":1500}]If no NIC is inserted into the pod, null is returned from stdout.
$ kata-runtime kata-network list-ifaces <container-id> nullIf the command fails to be executed, null is returned from stdout, and error description is returned from standard error (stderr).
Example:
$ kata-runtime kata-network list-ifaces <container-id> null
Add a route for a specified NIC.
$ cat ./test-route.json | kata-runtime kata-network add-route 6ec7a98 - [{"dest":"default","gateway":"172.16.0.1","device":"eth-test"}]The following describes the output of the kata-runtime kata-network add-route command for adding a route to a specified NIC:
If the command is executed successfully, the added route information in JSON format is returned from stdout.
Example:
$ kata-runtime kata-network add-route <container-id> route.json [{"dest":"177.17.0.0/24","gateway":"177.17.25.1","device":"netport_test_1"}]If the command fails to be executed, null is returned from stdout, and error description is returned from standard error (stderr).
Example:
$ kata-runtime kata-network add-route <container-id> routebad.json 2>/dev/null null
Key fields are described as follows:
dest: Network segment corresponding to the route. The value is in the format of <ip>/<mask>. <ip> is mandatory. There are three cases:
- Both IP address and mask are configured.
- If only an IP address is configured, the default mask is 32.
- If "dest":"default" is configured, there is no destination by default. In this case, the gateway needs to be configured.
gateway: Next-hop gateway of the route. When "dest":"default" is configured, the gateway is mandatory. In other cases, this parameter is optional.
device: Name of the NIC corresponding to the route, which is mandatory. The value contains a maximum of 15 characters.
NOTE:
If a route is added for the loopback device lo in the container, the device name corresponding to the device field in the route configuration file is lo.Run the following command to delete a specified route:
$ cat ./test-route.json | kata-runtime kata-network del-route 6ec7a98 -The fields in the test-route.json file are the same as those in the JSON file for adding a route.
The following describes the output of the** kata-runtime kata-network del-route** command for deleting a specified route:
If the command is executed successfully, the added route information in JSON format is returned from stdout.
Example:
$ kata-runtime kata-network del-route <container-id> route.json [{"dest":"177.17.0.0/24","gateway":"177.17.25.1","device":"netport_test_1"}]If the command fails to be executed, null is returned from stdout, and error description is returned from standard error (stderr).
Example:
$ kata-runtime kata-network del-route <container-id> routebad.json 2>/dev/null null
NOTE:- In the input fields, dest is mandatory, and both device and gateway are optional. Kata performs fuzzy match based on different fields and deletes the corresponding routing rules. For example, if dest is set to an IP address, all rules of this IP address will be deleted.
- If the route of the loopback device lo in the container is deleted, the device name corresponding to the device field in the route configuration file is lo.
Run the following command to delete an NIC:
$ cat ./test-iface.json | kata-runtime kata-network del-iface 6ec7a98 - NOTE:
When deleting an NIC, you can only delete it based on the name field in the NIC container. Kata does not identify other fields.The following describes the output of the **kata-runtime kata-network del-iface **command for deleting NICs:
If the command is executed successfully, null is returned from stdout.
Example:
$ kata-runtime kata-network del-iface <container-id> net.json nullIf the command fails to be executed, the information about NICs that fail to be deleted in JSON format is returned from stdout, and error description is returned from stderr.
Example:
$ kata-runtime kata-network del-iface <container-id> net.json {"device":"tapname_fun_012","name":"netport_test_1","IPAddresses":[{"Family":0,"Address":"177.17.0.1","Mask":"8"}],"mtu":1500,"hwAddr":"02:42:20:6e:a2:59","linkType":"tap"}
The preceding are common commands. For details about the command line interfaces, see APIs.
Kata IPVS Subsystem
The secure container provides an API for adding the ipvs command and setting the IPVS rule for the container. The functions include adding, editing, and deleting virtual services, adding, editing, and deleting real servers, querying IPVS service information, setting connection timeout, clearing the system connection cache, and importing rules in batches.
Add a virtual service address for the container.
kata-runtime kata-ipvs ipvsadm --parameters "--add-service --tcp-service 172.17.0.7:80 --scheduler rr --persistent 3000" <container-id>Modify virtual service parameters of a container.
kata-runtime kata-ipvs ipvsadm --parameters "--edit-service --tcp-service 172.17.0.7:80 --scheduler rr --persistent 5000" <container-id>Delete the virtual service address of a container.
kata-runtime kata-ipvs ipvsadm --parameters "--delete-service --tcp-service 172.17.0.7:80" <container-id>Add a real server for the virtual service address.
kata-runtime kata-ipvs ipvsadm --parameters "--add-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80 --weight 100" <container-id>Modify real server parameters of a container.
kata-runtime kata-ipvs ipvsadm --parameters "--edit-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80 --weight 200" <container-id>Delete a real server from a container.
kata-runtime kata-ipvs ipvsadm --parameters "--delete-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80" <container-id>Query service information.
kata-runtime kata-ipvs ipvsadm --parameters "--list" <container-id>It takes a long time to import rules one by one. You can write rules into a file and import them in batches.
kata-runtime kata-ipvs ipvsadm --restore - < <rule file path> <container-id> NOTE:
By default, the NAT mode is used for adding a single real server. To add real servers in batches, you need to manually add the -m option to use the NAT mode.
The following is an example of the rule file content:
-A -t 10.10.11.12:100 -s rr -p 3000
-a -t 10.10.11.12:100 -r 172.16.0.1:80 -m
-a -t 10.10.11.12:100 -r 172.16.0.1:81 -m
-a -t 10.10.11.12:100 -r 172.16.0.1:82 -mClear the system connection cache.
kata-runtime kata-ipvs cleanup --parameters "--orig-dst 172.17.0.4 --protonum tcp" <container-id>Set timeout interval for TCP, TCP FIN, or UDP connections.
kata-runtime kata-ipvs ipvsadm --parameters "--set 100 100 200" <container-id> NOTE:- Each container supports a maximum of 20000 iptables rules (5000 services and three servers/services). Both add-service and add-server are rules.
- Before importing rules in batches, you need to clear existing rules.
- No concurrent test scenario exists.
- The preceding are common commands. For details about the command line interfaces, see APIs.
