astream
Introduction
astream is a tool for prolonging the service life of drives. It monitors directories based on the inotify mechanism of Linux and works with the stream allocation rules for application scenarios defined by the user to set stream information for matched files when they are created. Then, the stream information is transparently transmitted to the NVMe SSD with the multi-stream feature enabled through the kernel. Finally, file storage can be better classified and stored according to the identifier of the stream information, thereby reducing the workload of drive garbage collection and write amplification factor of the drive, and prolonging the service life of the drive. astream focuses on database applications that have workload characteristics with the same or similar lifecycle, such as MySQL.
Installation
After configuring the Yum source of openEuler 22.09, install astream using the yum command.
yum install astream
Usage
Before getting into the usage of astream, you need to understand the stream allocation rule file required for starting astream.
Stream Allocation Rule File Example
Introduction
The stream allocation rule file allows you to define stream information rules for workloads based on their data lifecycle.
Each line of the stream allocation rule file defines a rule, for example, ^/data/mysql/data/undo 4. It means that any file with the undo prefix in /data/mysql/data is allocated with stream 4.
Example
A complete stream allocation rule file for MySQL is as follows:
^/data/mysql/data/ib_logfile 2
^/data/mysql/data/ibdata1$ 3
^/data/mysql/data/undo 4
^/data/mysql/data/mysql-bin 5
The file defines four rules for stream information:
A file whose absolute path is prefixed with /data/mysql/data/ib_logfile is allocated with stream 2.
A file whose absolute path is prefixed with /data/mysql/data/ibdata1 is allocated with stream 3.
A file whose absolute path is prefixed with /data/mysql/data/undo is allocated with stream 4.
A file whose absolute path is prefixed with /data/mysql/data/mysql-bin is allocated with stream 5.
Usage
Starting the astream Daemon
Assume that the rule files stream_rule1.txt and stream_rule2.txt are in the /home directory.
Monitoring a single directory:
astream -i /data/mysql/data -r /home/stream_rule1.txtMonitoring multiple directories:
astream can monitor multiple directories. Each directory requires a stream allocation rule file.
For example, to monitor two directories:
astream -i /data/mysql-1/data /data/mysql-2/data -r /home/stream_rule1.txt /home/stream_rule2.txt
The preceding command is used to monitor the following directories:
- /data/mysql-1/data, whose stream allocation rule file is /home/stream_rule1.txt.
- /data/mysql-2/data, whose stream allocation rule file is /home/stream_rule2.txt.
Command Options
astream [options]
| Option | Description | Example |
|---|---|---|
| -h | Displays help information | astream -h |
| -l | Sets the log level for astream monitoring. Log levels include the debug(1), info(2), warn(3), and error(4) levels. | astream -i /data/mysql/data -r /home/rule.txt -l 2 |
| -i | Specifies the directories to be monitored. Separate multiple directories with spaces. | This option is used with -r. See the example below. |
| -r | Specifies the stream allocation rule files corresponding to the monitored directories. Each parameter of -r corresponds to a parameter of -i. | astream -i /data/mysql/data -r /home/rule.txt |
| stop | Stops the astream daemon gracefully. | astream stop |
Restrictions
Restrictions for using astream are as follows.
Function Restrictions
- Only the NVMe SSDs with the multi-stream feature enabled are supported.
- A maximum of 5 streams can be allocated. The number of streams are limited by the BLK_MAX_WRITE_HINTS constant in the kernel and the maximum stream number supported by the NVMe SSD.
Operation Restrictions
- Run the astream daemon with root privileges during the test.
- The drive to be tested must have enough I/O pressure and space usage. In that case, the write amplification factor is high, and the multi-stream feature benefits more from astream.
Precautions
- When the astream daemon is running, do not delete the monitored directory and create it again. The created directory is not monitored until the astream daemon is restarted.
- You can use regular expression to match multiple files in the rule file.
- The NVMe SSD used in the test implements the multi-stream feature of NVMe 1.3.
