LTS

    Innovation Version

      StratoVirt VFIO Instructions

      Device Passthrough Management

      With device passthrough, a virtualization platform can enable VMs to directly use hardware devices, improving VM performance. This chapter describes the device passthrough feature supported by StratoVirt.

      Prerequisites

      To use device passthrough, a host must meet the following requirements:

      1. Enable the Input/Output Memory Management Unit (IOMMU).

        The IOMMU enables PCI/PCIe resources to be directly allocated to VMs.

        Run the following command on the host to check whether the IOMMU is enabled:

        dmesg | grep iommu
        

        If it is enabled, the following information is displayed:

        iommu: Default domain type: Translated
        hibmc-drm 0000:0a:00.0: Adding to iommu group 0
        ehci-pci 0000:7a:01.0: Adding to iommu group 1
        ehci-pci 0000:ba:01.0: Adding to iommu group 2
        ohci-pci 0000:7a:00.0: Adding to iommu group 3
        ohci-pci 0000:ba:00.0: Adding to iommu group 4
        xhci_hcd 0000:7a:02.0: Adding to iommu group 5
        ......
        

        If it is not enabled, no command output is displayed or only the following information is displayed:

        iommu: Default domain type: Translated
        

        Enable IOMMU:

        1.Add the following boot parameters for the kernel: intel_iommu=on and iommu=pt;

        $ vim /boot/grub2/grub.cfg
        linux /vmlinuz-5.15.0+ root=/dev/mapper/openeuler-root ro resume=/dev/mapper/openeuler-swap rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap crashkernel=512M intel_iommu=on iommu=pt
        

        2.Reboot the host OS.

      2. Load the vfio-pci kernel module.

        modprobe vfio-pci
        lsmod | grep vfio_pci
        

        If the vfio-pci module is successfully loaded, the following information is displayed:

        vfio_pci              327680  0
        vfio_virqfd           327680  1 vfio_pci
        vfio                  327680  2 vfio_iommu_type1,vfio_pci
        
      3. Unbind the PCI device from the host and bind it to the vfio-pci driver again.

        If Hi1822 NICs are directly connected through the Virtual Function I/O (VFIO), check the information about the PCI devices that correspond to the NICs first.

        $ lspci -v | grep "Eth" | grep 1822
        03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
        04:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
        05:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
        06:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
        

        Select the PCI device whose bus ID is 03, slot ID is 00, and function ID is 0, that is, 03:00.0. Unbind it from the host.

        echo 0000:03:00.0 > /sys/bus/pci/devices/0000:03:00.0/driver/unbind
        

        Finally bind the PCI device to the vfio-pci driver.

        lspci -ns 0000:03:00.0 |awk -F':| ' '{print 5" "6}' > /sys/bus/pci/drivers/vfio-pci/new_id
        

        After the NIC is bound to the vfio-pci driver, the NIC information cannot be queried on the host. Only the PCI device information can be queried.

      VFIO Device Passthrough

      Introduction

      The VFIO is a user-mode device driver solution provided by the kernel. The VFIO driver can securely present capabilities such as device I/O, interrupt, and DMA to user space. After StratoVirt uses the VFIO device passthrough solution, the I/O performance of VMs is greatly improved.

      Using VFIO Passthrough

      StratoVirt interconnects with libvirt to enable you to manage and configure VMs by modifying corresponding XML files. The following describes how to enable VFIO passthrough by modifying the XML file of a VM.

      Step 1 Modify the XML file.

      (1) Run the following command on the host to query the CPU architecture information:

      uname -m
      

      (2) For the AArch64 and x86_64 architectures, download the StratoVirt XML file stratovirt_aarch64.xml or stratovirtvirt_x86.xml and save it to any directory, for example, /home.

      cp stratovirt/docs/stratovirt_$arch.xml /home
      

      (3) Modify the VFIO configuration in the XML file based on the site requirements. bus, slot, and function specify the PCI device bound to the vfio-pci driver. The related configurations are as follows:

      <!-- vfio -->
      <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
          <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </source>
      </hostdev>
      

      In the preceding example, the device type is PCI, and managed='yes' indicates that libvirt unbinds the PCI device from the host and rebinds it to the vfio-pci driver. In thesource item, the domain, bus, slot, and function of the VFIO passthrough device are configured.

      Step 2 Create and log in to a VM using the libvirt command line.

      $ virsh create stratovirt_$arch.xml
      $ virsh list --all
      Id 	Name 		State
      --------------------
      1 	StratoVirt 	running
      # virsh console 1
      

      Step 3 View and use the VFIO passthrough NIC on the VM.

      (1) Check the NIC information before configuration.

      $ ip a
      1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
      	link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      	inet 127.0.0.1/8 scope host lo
          	valid_lft forever preferred_lft forever
      2: enp1s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
      	link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff
      

      (2) Dynamically configure the IP address of the NIC.

      dhclient
      

      (3) Check whether the IP address is configured successfully.

      $ ip a
      1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
      	link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      	inet 127.0.0.1/8 scope host lo
      		valid_lft forever preferred_lft forever
      2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
      	link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff
      	inet 192.168.1.3/16 brd 192.168.255.255 scope global dynamic enp1s0
      		valid_lft 86453sec preferred_lft 86453sec
      

      The preceding command output indicates that the IP address 192.168.1.3 is successfully assigned and the VM can directly use the configured NIC.

      Note: If the passthrough NIC is not connected to a physical network, network information cannot be obtained.

      Unbinding the VFIO Driver

      To unbind a passthrough NIC from a VM, log in to the host and run the following command to bind the NIC to the host again.hinic indicates the NIC driver type.

      echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/unbind
      echo 0000:03:00.0 > /sys/bus/pci/drivers/hinic/bind
      

      Note: Before binding a VFIO driver, you can run the ethtool -i enp0 command on the host to obtain the NIC driver type.enp0 indicates the name of the corresponding NIC.

      SR-IOV Passthrough

      Introduction

      When VFIO passthrough is enabled, VMs can directly access hardware, but each device can be exclusively used by only one VM. The SR-IOV passthrough technology can virtualize a physical function (PF) into multiple virtual functions (VFs) and directly pass the VFs to different VMs. This technology increases the number of available devices.

      Procedure

      Step 1 Create multiple VFs.

      The sriov_numvfs file is used to describe the count of VFs provided by SR-IOV and is stored in /sys/bus/pci/devices/domain:bus:slot.function/. For example, for the device whose bus ID is 03, slot ID is 00, and function ID is 0 in the preceding example, you can run the following command to create four VFs:

      echo 4 > /sys/bus/pci/devices/0000\:03\:00.0/sriov_numvfs
      

      Step 2 Verify that the VFs are successfully created.

      lspci -v | grep "Eth" | grep 1822
      

      If the following information is displayed, four VFs 03:00.1, 03:00.2, 03:00.3, and 03:00.4 are successfully created:

      03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
      03:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:00.4 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      

      Step 3 All the created VFs can be passed to VMs. The method for using an SR-IOV device is the same as that for using a common PCI device.

      Bug Catching

      Buggy Content

      Bug Description

      Submit As Issue

      It's a little complicated....

      I'd like to ask someone.

      PR

      Just a small problem.

      I can fix it online!

      Bug Type
      Specifications and Common Mistakes

      ● Misspellings or punctuation mistakes;

      ● Incorrect links, empty cells, or wrong formats;

      ● Chinese characters in English context;

      ● Minor inconsistencies between the UI and descriptions;

      ● Low writing fluency that does not affect understanding;

      ● Incorrect version numbers, including software package names and version numbers on the UI.

      Usability

      ● Incorrect or missing key steps;

      ● Missing prerequisites or precautions;

      ● Ambiguous figures, tables, or texts;

      ● Unclear logic, such as missing classifications, items, and steps.

      Correctness

      ● Technical principles, function descriptions, or specifications inconsistent with those of the software;

      ● Incorrect schematic or architecture diagrams;

      ● Incorrect commands or command parameters;

      ● Incorrect code;

      ● Commands inconsistent with the functions;

      ● Wrong screenshots.

      Risk Warnings

      ● Lack of risk warnings for operations that may damage the system or important data.

      Content Compliance

      ● Contents that may violate applicable laws and regulations or geo-cultural context-sensitive words and expressions;

      ● Copyright infringement.

      How satisfied are you with this document

      Not satisfied at all
      Very satisfied
      Submit
      Click to create an issue. An issue template will be automatically generated based on your feedback.
      Bug Catching
      编组 3备份