FUSE fastpath Feature Description and Usage Guide ​

Introduction ​

FUSE (Filesystem in Userspace) allows non-root users to create their own file systems in user space. Developers can quickly implement new and custom file systems without modifying kernel code or having kernel module development experience. FUSE has the advantages of simplifying file system development, extending file system capabilities, and improving security and stability. However, its use is limited because it has a great impact on I/O performance. To improve the performance of FUSE, openEuler provides the fastpath feature for FUSE. This feature accelerates FUSE in single-thread and multi-thread scenarios through pre-created thread core binding, shared memory, and fast process switching. Specifically, it includes the following three technologies:

1. When a user-space file system is mounted, a corresponding FUSE daemon is created on each CPU and bound to the corresponding CPU. After a user process delivers I/O, the FUSE daemon on the same CPU is directly woken up. This reduces the overhead of creating and destroying threads and makes full use of the CPU cache (the user process and the FUSE daemon usually access the same memory). After each thread is bound to a core, it is equivalent to having a percpu variable. User processes on different CPUs do not compete for locks after delivering I/O, improving parallelism.
2. A shared memory is created between the FUSE kernel module and the FUSE daemon, and the FUSE header information is directly placed in the shared memory, avoiding the overhead of addressing and copying part of the data.
3. The fast thread switching technology enables explicit switching of the thread running on a CPU in the kernel without the regular thread switching procedure, achieving higher CPU utilization.

Usage ​

Environment Requirements ​

• Hardware requirements: ARM64 architecture processor.
• Software requirements: Currently, the FUSE fastpath feature is provided only on openEuler 22.03 LTS SP4 and requires the cooperation of the kernel mode and user mode. To enable the fastpath feature, the minimum kernel version is 5.10.0-264.0.0. In user mode, only libfuse has been adapted, and the minimum version of the fuse3 software package is 3.10.5-11. Developers can obtain updated software packages from the openEuler update source.

Usage Method ​

Dynamic Library Linking ​

The user-mode part of the fastpath feature is provided as the libfuse dynamic library in the fuse3 software package. For a user-space file system that uses the libfuse dynamic library, you can link the library as follows:

gcc program.o -L/path/to/lib -lfuse -o program

For a user-space file system that has been compiled into a binary file, developers can run the following command to view the currently linked dynamic library:

ldd /path/to/binary

If the currently linked dynamic library is not the target version, use either of the following methods:

• Method 1: Update /etc/ld.so.conf.

  Open or create a configuration file, for example, /etc/d.so.conf.d/newlib.conf, and add the path of the library:

  bash

  /path/to/lib

  Run the following command to update the cache:

  bash

  sudo ldconfig

• Method 2: Set LD_LIBRARY_PATH.

  You can set the environment variable LD_LIBRARY_PATH to specify the path of the library. This method does not require administrator permissions, but it takes effect only for the current session.

  bash

  export LD_LIBRARY_PATH=/path/to/lib:$LD_LIBRARY_PATH

Adaptation and Enabling ​

After confirming that the correct version of the dynamic library is linked, developers can enable fastpath by adding parameters when creating a FUSE session. Specifically, the following three configurations are included:

• use_fastpath: Enables fastpath, that is, the preceding pre-created thread core binding, shared memory, and fast process switching functions.
• no_interrupt: Does not process interrupt requests.
• no_forget: Does not process forget requests.

no_forget and no_interrupt can improve performance but cause some functions to be unavailable. Therefore, enable them only after confirming that the user-space file system does not use these two types of requests. Otherwise, problems may occur.

Note that the added parameters are the parameters passed when the fuse_session_new function that creates a FUSE session in libfuse is used, rather than the parameters added when the binary file required for mounting the user-space file system is used. The specific adaptation method varies with user-space file systems. Take the demo (passthrough_hp) provided by libfuse as an example. Its code needs to be adapted as follows:

diff --git a/example/passthrough_hp.cc b/example/passthrough_hp.cc
index 872fc73..1f96820 100644
--- a/example/passthrough_hp.cc
+++ b/example/passthrough_hp.cc
@@ -1146,7 +1146,10 @@ static cxxopts::ParseResult parse_options(int argc, char **argv) {
         ("help", "Print help")
         ("nocache", "Disable all caching")
         ("nosplice", "Do not use splice(2) to transfer data")
-        ("single", "Run single-threaded");
+        ("single", "Run single-threaded")
+        ("nointerrupt", "Do not process interrupt request")
+        ("noforget", "Do not process forget request")
+        ("usefastpath", "use fastpath");
 
     // FIXME: Find a better way to limit the try clause to just
     // opt_parser.parse() (cf. https://github.com/jarro2783/cxxopts/issues/146)
@@ -1225,7 +1228,10 @@ int main(int argc, char *argv[]) {
     if (fuse_opt_add_arg(&args, argv[0]) ||
         fuse_opt_add_arg(&args, "-o") ||
         fuse_opt_add_arg(&args, "default_permissions,fsname=hpps") ||
-        (options.count("debug-fuse") && fuse_opt_add_arg(&args, "-odebug")))
+        (options.count("debug-fuse") && fuse_opt_add_arg(&args, "-odebug")) ||
+        (options.count("nointerrupt") && fuse_opt_add_arg(&args, "-ono_interrupt")) ||
+        (options.count("noforget") && fuse_opt_add_arg(&args, "-ono_forget")) ||
+        (options.count("usefastpath") && fuse_opt_add_arg(&args, "-ouse_fastpath")))
         errx(3, "ERROR: Out of memory");
 
     fuse_lowlevel_ops sfs_oper {};

Mounting method:

passthrough_hp --usefastpath /path/to/src /path/to/mnt

After entering passthrough_hp, the parameter --usefastpath is parsed as the -ouse_fastpath parameter, which is used to create a new FUSE session. For a session with this parameter, fastpath is enabled when the session is mounted and the FUSE daemon is created.