TLBi Optimization ​

1.1 Overview

1.2 Constraints

1.3 Usage

Overview ​

This feature disables the global broadcast of TLBi instructions on the host OS and limits the TLB flush scope in the guest OS to the CPUs that have executed the VM process, thereby narrowing the TLB broadcast scope. In many-core multi-VM scenarios, this reduces the TLB flush overhead and effectively improves the linearity of the unixbench score.

Constraints ​

1. This feature applies only to ARM64.
2. To enable this feature, add the following kernel compilation parameters: CONFIG_ARM64_TLBI_IPI and CONFIG_ARM64_KVM_HCR_NOFB.
3. To enable this feature, the boot parameter kvm-arm.hcr_nofb=1 is required.

Usage ​

Compiling and Replacing the Kernel ​

1. In the kernel root directory, run make openeuler_defconfig to generate the .config file.

2. Run make menuconfig to specify kernel compilation parameters. Enable CONFIG_ARM64_TLBI_IPI and CONFIG_ARM64_KVM_HCR_NOFB, and then check the corresponding configuration items in the .config file to ensure that the settings take effect.

3. Compile the RPM package: make binrpm-pkg -j. The generated RPM package is stored in the rpmbuild/RPMS/aarch64 directory.

4. In the directory where the RPM package is stored, run rpm -ivh --force kernel*.rpm to install the new kernel.

5. Run the grub2-set-default command to set the default boot kernel. The following uses the 24.03 LTS-SP1 version as an example:

   shell

   # Query the default boot kernel currently set.
   [root@localhost ~]# grub2-editenv list
   saved_entry=openEuler (6.12.0) 24.03 (LTS-SP1)
   
   # Query all kernel versions currently installed.
   [root@localhost ~]# grep "^menuentry" /etc/grub2-efi.cfg
   menuentry 'openEuler (6.12.0) 24.03 (LTS-SP1)' --class openeuler --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-6.12.0-advanced-887bd997-932e-4cd0-84e3-ec71abbe368b' {
   menuentry 'openEuler (6.6.0-77.0.0.81.oe2403sp1.aarch64) 24.03 (LTS-SP1)' --class openeuler --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-6.6.0-77.0.0.81.oe2403sp1.aarch64-advanced-887bd997-932e-4cd0-84e3-ec71abbe368b' {
   
   # Set the default boot kernel to 'openEuler (6.6.0-77.0.0.81.oe2403sp1.aarch64) 24.03 (LTS-SP1)'.
   [root@localhost ~]# grub2-set-default 'openEuler (6.6.0-77.0.0.81.oe2403sp1.aarch64) 24.03 (LTS-SP1)'
   
   # Query the default boot kernel currently set again.
   [root@localhost ~]# grub2-editenv list
   saved_entry=openEuler (6.6.0-77.0.0.81.oe2403sp1.aarch64) 24.03 (LTS-SP1)

Setting Kernel Boot Parameters ​

1. Edit the /etc/default/grub file and add kvm-arm.hcr_nofb=1 to GRUB_CMDLINE_LINUX.

2. View the modification result. An example is as follows:

   shell

   [root@localhost ~]# cat /etc/default/grub | grep GRUB_CMDLINE_LINUX
   GRUB_CMDLINE_LINUX="rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap video=VGA-1:640x480-32@60me cgroup_disable=files apparmor=0 crashkernel=1024M,high smmu.bypassdev=0x1000:0x17 smmu.bypassdev=0x1000:0x15 arm64.nopauth console=tty0 kvm-arm.hcr_nofb=1"

3. Run the grub2-mkconfig -o /boot/efi/EFI/openEuler/grub.cfg command to update the grub configuration file.

Restarting the Host and Checking the Parameter Settings ​

1. Restart the host.
2. Run the zcat /proc/config.gz | grep CONFIG_ARM64_TLBI_IPI command and confirm that the output is CONFIG_ARM64_TLBI_IPI=y.
3. Run the zcat /proc/config.gz | grep CONFIG_ARM64_KVM_HCR_NOFB command and confirm that the output is CONFIG_ARM64_KVM_HCR_NOFB=y.
4. Run the cat /proc/cmdline command and ensure that the output contains kvm-arm.hcr_nofb=1.