astream
Introduction
astream is a tool for prolonging the service life of drives. It monitors directories based on the inotify mechanism of Linux and works with the stream allocation rules for application scenarios defined by users to set stream information for matched files when they are created. Then, the stream information is transparently transmitted to an NVMe SSD with the multi-stream feature enabled through the kernel. This allows the file storage to be better classified as identified by the stream information and reduces the workload of drive garbage collection, thereby lowering the write amplification factor (WAF) and prolonging the service life of the drive. astream focuses on database applications that have workloads with the same or similar lifecycles, such as MySQL.
Installation
After configuring the Yum source of openEuler 23.03, install astream using the yum command.
yum install astream
Usage
Before getting into the usage of astream, you need to understand the stream allocation rule file required for starting astream.
Stream Allocation Rule File
Introduction
The stream allocation rule file allows you to define stream information rules for workloads based on their data lifecycle.
Each line of the stream allocation rule file defines a rule, for example, ^/data/mysql/data/undo 4 means that any file with the undo prefix in /data/mysql/data is allocated with stream 4.
Example
A complete stream allocation rule file for MySQL is as follows:
^/data/mysql/data/ib_logfile 2
^/data/mysql/data/ibdata1$ 3
^/data/mysql/data/undo 4
^/data/mysql/data/mysql-bin 5
The file defines four rules for stream information:
- A file whose absolute path is prefixed with /data/mysql/data/ib_logfile is allocated with stream 2.
- A file whose absolute path is prefixed with /data/mysql/data/ibdata1 is allocated with stream 3.
- A file whose absolute path is prefixed with /data/mysql/data/undo is allocated with stream 4.
- A file whose absolute path is prefixed with /data/mysql/data/mysql-bin is allocated with stream 5.
Usage
Starting the astream Daemon
Assume that the rule files stream_rule1.txt and stream_rule2.txt are in the /home directory.
Monitoring a single directory:
astream -i /data/mysql/data -r /home/stream_rule1.txtMonitoring multiple directories:
astream can monitor multiple directories. Each directory requires a stream allocation rule file.
For example, to monitor two directories, run the following command:
astream -i /data/mysql-1/data /data/mysql-2/data -r /home/stream_rule1.txt /home/stream_rule2.txt
The preceding command is used to monitor the following directories:
- /data/mysql-1/data, whose stream allocation rule file is /home/stream_rule1.txt.
- /data/mysql-2/data, whose stream allocation rule file is /home/stream_rule2.txt.
Command Options
astream [options]
| Option | Description | Example |
|---|---|---|
| -h | Displays help information | astream -h |
| -l | Sets the log level for astream monitoring. Log levels include debug (1), info (2), warn (3), and error (4). | astream -i /data/mysql/data -r /home/rule.txt -l 2 |
| -i | Specifies the directories to be monitored. Separate multiple directories with spaces. | This option is used with -r. See the example below. |
| -r | Specifies the stream allocation rule files corresponding to the monitored directories. Each parameter of -r corresponds to a parameter of -i. | astream -i /data/mysql/data -r /home/rule.txt |
| stop | Stops the astream daemon gracefully. | astream stop |
Restrictions
Restrictions for using astream are as follows.
Function Restrictions
- Only NVMe SSDs with the multi-stream feature enabled are supported.
- A maximum of 5 streams can be allocated. The number of streams are limited by the BLK_MAX_WRITE_HINTS constant in the kernel and the maximum stream number supported by an NVMe SSD.
Operation Restrictions
- Root privileges are required for running the astream daemon during the test.
- The drive to be tested must have enough I/O stress and space usage. When the WAF is high, the multi-stream feature will benefit more from astream.
Precautions
- When the astream daemon is running, do not delete the monitored directory and then create it again. Otherwise, you need to restart the astream daemon to monitor the created directory.
- You can use regular expressions in rule files to match multiple files.
- The NVMe SSD used must support the multi-stream feature of NVMe 1.3 to run the test.
