D-FOT User Guide ​

Overview ​

Dynamic Feedback-directed Optimization Tool (D-FOT) is a dynamic feedback-directed optimization framework. It aims to enable fully automatic and non-intrusive feedback-directed optimization for applications, comprehensively improving the usability and performance of this feature. The framework is planned to support two types of dynamic optimizations: startup time optimization and runtime optimization. Currently, startup time optimization has been implemented based on the oeAware online tuning framework. Startup time optimization: During application running, sampling and feedback-directed optimization are automatically performed. Once optimization is complete, the optimized version automatically takes over the next startup. You do not need to modify the startup procedure, and the optimized version is automatically started after the application is restarted. Runtime optimization: During application running, sampling and feedback-directed optimization are automatically performed. You do not need to restart the application, and no user intervention is required throughout the process. The optimized version can be enabled after only a few interruptions.

Software Architecture Description ​

This framework consists of the following modules:

• libkperf-based sampling data processing
• sysboost-based startup takeover and binary optimization feature enablement
• llvm-bolt-based binary feedback-directed optimization
• Implementation of the oeAware tuning plugin dfot_tuner_sysboost

Dependency Items ​

OS requirements: Currently, only openEuler 22.03 LTS SP4 is supported. The following dependent components can be installed using yum.

Procedure ​

Prerequisites for Applications to Be Optimized ​

1. The binary to be optimized must contain relocation information. For self-compiled software, the -Wl,-q linker option must be added during compilation. For example, for MySQL, add cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DBUILD_CONFIG=mysql_release -DWITH_BOOST=../boost -DCMAKE_C_LINK_FLAGS="-Wl,-q" -DCMAKE_CXX_LINK_FLAGS="-Wl,-q". For openEuler software packages, the relocation package of the corresponding application will be provided and can be directly installed.
2. Check whether the target binary contains relocation information: When the -Wl,-q linker option is effective, the binary will include RELA sections. You can check this using readelf -SW /path/to/bin. For example, in MySQL, before adding the option, only the .rela.dyn and .rela.plt sections exist. After option addition, more than 10 RELA sections appear, including .rela.text and .rela.eh_frame. If -Wl,-q is not effective, when manual perf sampling and perf2bolt are performed or llvm-bolt is used to perform optimization (with or without sysboost), the alarm BOLT-WARNING: non-relocation mode for AArch64 is not fully supported is reported.

D-FOT Preparations ​

Install D-FOT using yum or build it from the source code using the following commands. (If libkperf or oeAware-manager is also built from the source code, you need to specify the library and include path when running cmake.)

# Install dependency packages.
yum install boost-devel log4cplus-devel numactl-devel libkperf-devel oeAware-manager-devel
# Download the source code and compile it.
git clone https://atomgit.com/openeuler/D-FOT.git
cd D-FOT
mkdir build && cd build
cmake ../ -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_SKIP_RPATH=TRUE
make -j`nproc`
cd ..

cp build/libdfot.so /lib64/oeAware-plugin/
chmod 440 /lib64/oeAware-plugin/libdfot.so
mkdir -p /etc/dfot/
cp configs/dfot.ini /etc/dfot/

Configuration Modification ​

Modify /etc/dfot/dfot.ini. The configuration items can be set according to the following description. Public configuration: [general]

Plugin Usage ​

See https://atomgit.com/openeuler/oeAware-manager/tree/master/docs.

If D-FOT is installed using yum, you can directly run the following commands to enable plugin instances:

# Restart the oeAware service.
systemctl restart oeaware
# Enable the collection plugin.
oeawarectl -e pmu_sampling_collector
# Enable the feedback-directed optimization tuning plugin.
oeawarectl -e dfot_tuner_sysboost

# Disable the two plugins.
oeawarectl -d pmu_sampling_collector
oeawarectl -d dfot_tuner_sysboost

If D-FOT is built from the source code, copy libdfot.so to the default plugin directory /lib64/oeAware-plugin/ of oeAware, and then load and activate the plugin instance. Note that the permission of the plugin must be 440:

# Restart the oeAware service (the plugins in /lib64/oeAware-plugin/ are automatically loaded).
systemctl restart oeaware
# Alternatively, do not restart the service and manually run oeawarectl to load the plugin.
oeawarectl -l libdfot.so
# Then, perform the preceding enable operations.

For other operations, see the oeAware user manual.

Example ​

The following uses the TPCC test scenario of mysql-8.0.25 as an example:

# 1. Prepare the environment.
yum install libkperf oeaware-manager sysboost llvm-bolt D-FOT

# 2. Build MySQL.
# -Wl,-q is mandatory for optimization. -fPIE and -Wl,-zmax-page-size=0x200000 -Wl,-zcommon-page-size=0x200000 are optional. They are mandatory if sysboost huge page preloading optimization is required.
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DBUILD_CONFIG=mysql_release -DWITH_BOOST=../boost -DCMAKE_C_FLAGS="-fPIE" -DCMAKE_CXX_FLAGS="-fPIE" -DCMAKE_C_LINK_FLAGS="-Wl,-q -Wl,-zmax-page-size=0x200000 -Wl,-zcommon-page-size=0x200000 -pie" -DCMAKE_CXX_LINK_FLAGS="-Wl,-q -Wl,-zmax-page-size=0x200000 -Wl,-zcommon-page-size=0x200000 -pie"

# 3. Start oeAware and the optimization plugins.
# Fill in /etc/dfot/dfot.ini based on the actual application information, and then run the following commands.
systemctl start sysboost
systemctl start oeaware
oeawarectl -e pmu_sampling_collector
oeawarectl -e dfot_tuner_sysboost

# 4. Run the TPCC test.
...

# 5. Restart MySQL.
# You can check /etc/dfot/dfot.log in advance to determine whether optimization has been performed. After the application is restarted, the optimized version is used.

Constraints ​

1. Due to the limitations of the sysboost implementation, D-FOT cannot be used in virtualization scenarios (KVM virtual machines: supported but live migration is not supported; Docker containers: not supported).
2. The target application to be optimized must include relocation information.
3. When D-FOT is used as the oeAware tuning plugin, the fixed sampling frequency of the current oeAware collection plugin pmu_sampling_collector is 100, which may cause a long sampling time (for reference, the default sampling frequency of perf is 4000).

Future Plans ​

• [ ] Virtualization scenario support
• [ ] Runtime optimization support
• [ ] Enhanced binary optimization