qtfs Shared File System Architecture and Usage

Introduction

qtfs is a shared file system project. It can be deployed on either a host-DPU hardware architecture or on two hosts. qtfs works in client-server mode, allowing the client to access specified file systems on the server in the same way that local files are accessed.

qtfs provides the following features:

• Mount point propagation

• Sharing of special file systems such as proc, sys, and cgroup

• Shared read and write of remote files

• Remote mounting of server file systems on the client

• Customized processing of special files

• Remote FIFO, Unix sockets, and epoll that allow the client and server to access the files as if they were like local

• Bottom-layer host-DPU communication over the PCIe protocol, outperforming the network

• Kernel module development, preventing intrusive modification to the kernel

Software Architecture

Installation

Perform operations in the following qtfs-related directories:

• qtfs: code of the client kernel module. Compile the client .ko file in this directory.

• qtfs_server: code of the server kernel module. Compile the server .ko files and related programs in this directory.

• qtinfo: diagnosis tool that is used to check the status of file systems and change the log level.

• demo, test, and doc: demo programs, test programs, and project documents.

• Root directory: code of common modules used by the client and server.

Configure the kernel compilation environment on two servers (or VMs).

1. The kernel version must be 5.10 or later.
2. Install the kernel development package by running yum install kernel-devel.
3. Assume that the host IP address is 192.168.10.10 and the DPU IP address is 192.168.10.11.

Install the qtfs server.

    1. cd qtfs_server
    2. make clean && make
    3. insmod qtfs_server.ko qtfs_server_ip=192.168.10.10 qtfs_server_port=12345 qtfs_log_level=WARN
    4. nohup ./engine 16 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 &

Install the qtfs client.

    1. cd qtfs
    2. make clean && make
    3. insmod qtfs.ko qtfs_server_ip=192.168.10.10 qtfs_server_port=12345 qtfs_log_level=WARN
    4. cd ../ipc/
    5. make clean && make && make install
    6. nohup udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 &

Usage

After the installation is complete, mount the server file system to the client. For example:

    mount -t qtfs / /root/mnt/

The file system is visible to the client. Access /root/mnt on the client to view and operate files on the server.