StratoVirt VFIO Instructions

Device Passthrough Management

With device passthrough, a virtualization platform can enable VMs to directly use hardware devices, improving VM performance. This chapter describes the device passthrough feature supported by StratoVirt.

Prerequisites

To use device passthrough, a host must meet the following requirements:

Enable the Input/Output Memory Management Unit (IOMMU).

The IOMMU enables PCI/PCIe resources to be directly allocated to VMs.

Run the following command on the host to check whether the IOMMU is enabled:

# dmesg | grep iommu

If it is enabled, the following information is displayed:

iommu: Default domain type: Translated
hibmc-drm 0000:0a:00.0: Adding to iommu group 0
ehci-pci 0000:7a:01.0: Adding to iommu group 1
ehci-pci 0000:ba:01.0: Adding to iommu group 2
ohci-pci 0000:7a:00.0: Adding to iommu group 3
ohci-pci 0000:ba:00.0: Adding to iommu group 4
xhci_hcd 0000:7a:02.0: Adding to iommu group 5
......

If it is not enabled, no command output is displayed or only the following information is displayed:

iommu: Default domain type: Translated

Enable IOMMU：

1.Add boot parameters for linux kernel: intel_iommu=on iommu=pt;

vim /boot/grub2/grub.cfg
linux /vmlinuz-5.15.0+ root=/dev/mapper/openeuler-root ro resume=/dev/mapper/openeuler-swap rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap crashkernel=512M intel_iommu=on iommu=pt

2.Reboot Host OS;

Load the vfio-pci kernel module.

# modprobe vfio-pci
# lsmod | grep vfio_pci

If the vfio-pci module is successfully loaded, the following information is displayed:

vfio_pci              327680  0
vfio_virqfd           327680  1 vfio_pci
vfio                  327680  2 vfio_iommu_type1,vfio_pci

Unbind the PCI device from the host and bind it to the vfio-pci driver again.

If Hi1822 NICs are directly connected through the Virtual Function I/O (VFIO), check the information about the PCI devices that correspond to the NICs first.

# lspci -v | grep "Eth" | grep 1822
03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
04:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
05:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
06:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)

Select the PCI device whose bus ID is 03, slot ID is 00, and function ID is 0, that is, 03:00.0. Unbind it from the host.

# echo 0000:03:00.0 > /sys/bus/pci/devices/0000:03:00.0/driver/unbind

Finally bind the PCI device to the vfio-pci driver.

lspci -ns 0000:03:00.0 |awk -F':| ' '{print 5" "6}' > /sys/bus/pci/drivers/vfio-pci/new_id

After the NIC is bound to the vfio-pci driver, the NIC information cannot be queried on the host. Only the PCI device information can be queried.

VFIO Device Passthrough

Introduction

The VFIO is a user-mode device driver solution provided by the kernel. The VFIO driver can securely present capabilities such as device I/O, interrupt, and DMA to user space. After StratoVirt uses the VFIO device passthrough solution, the I/O performance of VMs is greatly improved.

Using VFIO Passthrough

StratoVirt interconnects with libvirt to enable you to manage and configure VMs by modifying corresponding XML files. The following describes how to enable VFIO passthrough by modifying the XML file of a VM.

Step 1 Modify the XML file.

(1) Run the following command on the host to query the CPU architecture information:

# uname -m

(2) For the AArch64 and x86_64 architectures, download the StratoVirt XML file stratovirt_aarch64.xml or stratovirtvirt_x86.xml and save it to any directory, for example, /home.

# cp stratovirt/docs/stratovirt_$arch.xml /home

(3) Modify the VFIO configuration in the XML file based on the site requirements. bus, slot, and function specify the PCI device bound to the vfio-pci driver. The related configurations are as follows:

<!-- vfio -->
<hostdev mode='subsystem' type='pci' managed='yes'>
<driver name='vfio'/>
<source>
    <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</source>
</hostdev>

In the preceding example, the device type is PCI, and managed='yes' indicates that libvirt unbinds the PCI device from the host and rebinds it to the vfio-pci driver. In thesource item, the domain, bus, slot, and function of the VFIO passthrough device are configured.

Step 2 Create and log in to a VM using the libvirt command line.

# virsh create stratovirt_$arch.xml
# virsh list --all
Id 	Name 		State
--------------------
1 	StratoVirt 	running
# virsh console 1

Step 3 View and use the VFIO passthrough NIC on the VM.

(1) Check the NIC information before configuration.

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
	link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
	inet 127.0.0.1/8 scope host lo
    	valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
	link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff

(2) Dynamically configure the IP address of the NIC.

# dhclient

(3) Check whether the IP address is configured successfully.

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
	link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
	inet 127.0.0.1/8 scope host lo
		valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
	link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff
	inet 192.168.1.3/16 brd 192.168.255.255 scope global dynamic enp1s0
		valid_lft 86453sec preferred_lft 86453sec

The preceding command output indicates that the IP address 192.168.1.3 is successfully assigned and the VM can directly use the configured NIC.

Note: If the passthrough NIC is not connected to a physical network, network information cannot be obtained.

Unbinding the VFIO Driver

To unbind a passthrough NIC from a VM, log in to the host and run the following command to bind the NIC to the host again.hinic indicates the NIC driver type.

# echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/unbind
# echo 0000:03:00.0 > /sys/bus/pci/drivers/hinic/bind

Note: Before binding a VFIO driver, you can run the ethtool -i enp0 command on the host to obtain the NIC driver type.enp0 indicates the name of the corresponding NIC.

SR-IOV Passthrough

Introduction

When VFIO passthrough is enabled, VMs can directly access hardware, but each device can be exclusively used by only one VM. The SR-IOV passthrough technology can virtualize a physical function (PF) into multiple virtual functions (VFs) and directly pass the VFs to different VMs. This technology increases the number of available devices.

Procedure

Step 1 Create multiple VFs.

The sriov_numvfs file is used to describe the count of VFs provided by SR-IOV and is stored in /sys/bus/pci/devices/domain:bus:slot.function/. For example, for the device whose bus ID is 03, slot ID is 00, and function ID is 0 in the preceding example, you can run the following command to create four VFs:

# echo 4 > /sys/bus/pci/devices/0000\:03\:00.0/sriov_numvfs

Step 2 Verify that the VFs are successfully created.

# lspci -v | grep "Eth" | grep 1822

If the following information is displayed, four VFs 03:00.1, 03:00.2, 03:00.3, and 03:00.4 are successfully created:

03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
03:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
03:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
03:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
03:00.4 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)

Step 3 All the created VFs can be passed to VMs. The method for using an SR-IOV device is the same as that for using a common PCI device.

Bug Catching

Buggy Content

Bug Description

Submit As Issue

It's a little complicated....

I'd like to ask someone.

Just a small problem.

I can fix it online!

Bug Type

Specifications and Common Mistakes

● Misspellings or punctuation mistakes;

● Incorrect links, empty cells, or wrong formats;

● Chinese characters in English context;

● Minor inconsistencies between the UI and descriptions;

● Low writing fluency that does not affect understanding;

● Incorrect version numbers, including software package names and version numbers on the UI.

Usability

● Incorrect or missing key steps;

● Missing prerequisites or precautions;

● Ambiguous figures, tables, or texts;

● Unclear logic, such as missing classifications, items, and steps.

Correctness

● Technical principles, function descriptions, or specifications inconsistent with those of the software;

● Incorrect schematic or architecture diagrams;

● Incorrect commands or command parameters;

● Incorrect code;

● Commands inconsistent with the functions;

● Wrong screenshots.

Risk Warnings

● Lack of risk warnings for operations that may damage the system or important data.

Content Compliance

● Contents that may violate applicable laws and regulations or geo-cultural context-sensitive words and expressions;

● Copyright infringement.

We may contact you through email in regard to the bug fixing solution. You will also be notified through email if you win any prize during the activity.

How satisfied are you with this document

Not satisfied at all

Very satisfied

By submitting the contents, you fully understand and agree to the terms of the openEuler Privacy Statement.

Submit

Click to create an issue. An issue template will be automatically generated based on your feedback.