Application Scenarios

You can use functions provided by A-Tune through the CLI client atune-adm. This chapter describes the functions and usage of the A-Tune client.

Overview

  • The root permission is required to use A-Tune.

  • You can run the atune-adm help/–help/-h command to query commands supported by atune-adm.

  • All example commands are used in single-node mode. For distributed mode, specify an IP address and port number. For example:

    #  atune-adm -a 192.168.3.196 -p 60001 list
    
  • The define, update, undefine, collection, train, and **upgrade **commands do not support remote execution.

  • In the command format, brackets ([]) indicate that the parameter is optional, and angle brackets (<>) indicate that the parameter is mandatory. The actual parameters prevail.

  • In the command format, meanings of each command are as follows:

    • WORKLOAD_TYPE: name of a user-defined workload type. For details about the supported workload types, see the query result of the list command.
    • PROFILE_NAME: user-defined profile name.
    • PROFILE_PATH: path of the user-defined profile.

Querying Workload Types

list

Function

Query the supported workload types, profiles, and the values of Active.

Format

atune-adm list

Example

# atune-adm list

Support WorkloadTypes:
+-----------------------------------+------------------------+-----------+
| WorkloadType                      | ProfileName            | Active    |
+===================================+========================+===========+
| default                           | default                | true      |
+-----------------------------------+------------------------+-----------+
| webserver                         | ssl_webserver          | false     |
+-----------------------------------+------------------------+-----------+
| big_database                      | database               | false     |
+-----------------------------------+------------------------+-----------+
| big_data                          | big_data               | false     |
+-----------------------------------+------------------------+-----------+
| in-memory_computing               | in-memory_computing    | false     |
+-----------------------------------+------------------------+-----------+
| in-memory_database                | in-memory_database     | false     |
+-----------------------------------+------------------------+-----------+
| single_computer_intensive_jobs    | compute-intensive      | false     |
+-----------------------------------+------------------------+-----------+
| communication                     | rpc_communication      | false     |
+-----------------------------------+------------------------+-----------+
| idle                              | default                | false     |
+-----------------------------------+------------------------+-----------+

NOTE:
If the value of Active is true, the profile is activated. In the example, the profile of the default type is activated.

Workload Type Analysis and Auto Optimization

analysis

Function

Collect real-time statistics from the system to identify and automatically optimize workload types.

Format

atune-adm analysis [OPTIONS]

Parameter Description

  • OPTIONS

    Parameter

    Description

    --model, -m

    Model generated by user-defined training

Example

  • Use the default model for classification and identification.

    # atune-adm analysis
    
  • Use the user-defined training model for recognition.

    # atune-adm analysis --model /usr/libexec/atuned/analysis/models/new-model.m
    

User-defined Model

A-Tune allows users to define and learn new models. To define a new model, perform the following steps:

  1. Run the define command to define workload_type and profile.
  2. Run the collection command to collect the profile data corresponding to workload_type.
  3. Run the train command to train the model.

define

Function

Add a user-defined workload type and the corresponding profile optimization item.

Format

atune-adm define <WORKLOAD_TYPE> <PROFILE_NAME> <PROFILE_PATH>

Example

Add a workload type. Set workload type to test_type, profile name to test_name, and configuration file of an optimization item to example.conf.

# atune-adm define test_type test_name ./example.conf

The example.conf file can be written as follows (the following optimization items are optional and are for reference only). You can also run the atune-adm info command to view how the existing profile is written.

[main]
# list its parent profile
[tip]
# the recommended optimization, which should be performed manunaly
[check]
# check the environment
[affinity.irq]
# to change the affinity of irqs
[affinity.task]
# to change the affinity of tasks
[bios]
# to change the bios config
[bootloader.grub2]
# to change the grub2 config
[kernel_config]
# to change the kernel config
[script]
# the script extention of cpi
[sysctl]
# to change the /proc/sys/* config
[sysfs]
# to change the /sys/* config
[systemctl]
# to change the system service config
[ulimit]
# to change the resources limit of user

collection

Function

Collect the global resource usage and OS status information during service running, and save the collected information to a CSV output file as the input dataset for model training.

NOTE:

  • This command depends on the sampling tools such as perf, mpstat, vmstat, iostat, and sar.
  • Currently, only the Kunpeng 920 CPU is supported. You can run the dmidecode -t processor command to check the CPU model.

Format

atune-adm collection <OPTIONS>

Parameter Description

  • OPTIONS

    Parameter

    Description

    --filename, -f

    Name of the generated CSV file used for training: name-timestamp.csv

    --output_path, -o

    Path for storing the generated CSV file. The absolute path is required.

    --disk, -b

    Disk used during service running, for example, /dev/sda.

    --network, -n

    Network port used during service running, for example, eth0.

    --workload_type, -t

    Workload type, which is used as a label for training.

    --duration, -d

    Data collection time during service running, in seconds. The default collection time is 1200 seconds.

    --interval, -i

    Interval for collecting data, in seconds. The default interval is 5 seconds.

Example

# atune-adm collection --filename name --interval 5 --duration 1200 --output_path /home/data --disk sda --network eth0 --workload_type test_type 

train

Function

Use the collected data to train the model. Collect data of at least two workload types during training. Otherwise, an error is reported.

Format

atune-adm train <OPTIONS>

Parameter Description

  • OPTIONS
    Parameter Description
    –data_path, -d Path for storing CSV files required for model training
    –output_file, -o Model generated through training

Example

Use the CSV file in the data directory as the training input. The generated model new-model.m is stored in the model directory.

# atune-adm train --data_path /home/data --output_file /usr/libexec/atuned/analysis/models/new-model.m 

undefine

Function

Delete a user-defined workload type.

Format

atune-adm undefine <WORKLOAD_TYPE>

Example

Delete the test_type workload type.

# atune-adm undefine test_type 

Querying Profiles

info

Function

View the profile content of a workload type.

Format

atune-adm info <WORKLOAD_TYPE_>_

Example

View the profile content of webserver.

# atune-adm info webserver

*** ssl_webserver:

#
# webserver tuned configuration
#
[main]
#TODO CONFIG

[kernel_config]
#TODO CONFIG

[bios]
#TODO CONFIG

[sysfs]
#TODO CONFIG

[sysctl]
fs.file-max=6553600
fs.suid_dumpable = 1
fs.aio-max-nr = 1048576
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_local_port_range = 1024     65500
net.ipv4.tcp_max_tw_buckets = 5000
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_mem =  362619      483495   725238
net.ipv4.tcp_rmem = 4096         87380   6291456
net.ipv4.tcp_wmem = 4096         16384   4194304
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

[systemctl]
sysmonitor=stop
irqbalance=stop

[bootloader.grub2]
selinux=0
iommu.passthrough=1

[tip]
bind your master process to the CPU near the network = affinity
bind your network interrupt to the CPU that has this network = affinity
relogin into the system to enable limits setting = OS

[script]
openssl_hpre = 0
prefetch = off

[ulimit]
{user}.hard.nofile = 102400
{user}.soft.nofile = 102400

[affinity.task]
#TODO CONFIG

[affinity.irq]
#TODO CONFIG

[check]
#TODO CONFIG

  

Updating a Profile

You can update the existing profile as required.

update

Function

Update an optimization item of a workload type to the content in the new.conf file.

Format

atune-adm update <WORKLOAD_TYPE> <PROFILE_NAME> <PROFILE_FILE>

Example

Update the workload type to test_type and the optimization item of test_name to new.conf.

# atune-adm update test_type test_name ./new.conf

Activating a Profile

profile

Function

Manually activate a profile of a workload type.

Format

**atune-adm profile **<WORKLOAD_TYPE>

Parameter Description

You can run the list command to query the supported workload types.

Example

Activate the profile configuration of webserver.

# atune-adm profile webserver

Rolling Back Profiles

  

rollback

Functions

Roll back the current configuration to the initial configuration of the system.

Format

atune-adm rollback

Example

# atune-adm rollback

Updating Database

  

upgrade

Function

Update the system database.

Format

atune-adm upgrade <DB_FILE>

Parameter Description

  • DB_FILE

    New database file path.

Example

The database is updated to new_sqlite.db.

# atune-adm upgrade ./new_sqlite.db

Querying System Information

  

check

Function

Check the CPU, BIOS, OS, and NIC information.

Format

atune-adm check

Example

# atune-adm check
 cpu information:
     cpu:0   version: Kunpeng 920-6426  speed: 2600000000 HZ   cores: 64
     cpu:1   version: Kunpeng 920-6426  speed: 2600000000 HZ   cores: 64
 system information:
     DMIBIOSVersion: 0.59
     OSRelease: 4.19.36-vhulk1906.3.0.h356.eulerosv2r8.aarch64
 network information:
     name: eth0              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth1              product: HNS GE/10GE/25GE Network Controller
     name: eth2              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth3              product: HNS GE/10GE/25GE Network Controller
     name: eth4              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth5              product: HNS GE/10GE/25GE Network Controller
     name: eth6              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth7              product: HNS GE/10GE/25GE Network Controller
     name: docker0           product:

Automatic Parameter Optimization

A-Tune provides the automatic search capability for optimal configurations, eliminating the need for repeated manual parameter adjustment and performance evaluation. This greatly improves the search efficiency of optimal configurations.

Tuning

Function

Use the specified project file to search the dynamic space for parameters and find the optimal solution under the current environment configuration.

Format

NOTE:
Before running the command, ensure that the following conditions are met:

  1. The YAML configuration file of the server has been edited and placed in the /etc/atuned/tuning/ directory on the server by the server administrator.
  2. The YAML configuration file of the client has been edited and placed in an arbitrary directory on the client.

atune-adm tuning [OPTIONS] <PROJECT_YAML>

Parameter Description

  • OPTIONS

    Parameter

    Description

    --restore, -r

    Restores the initial configuration before tuning.

    --project, -p

    Specifies the project name in the YAML file to be restored.

    NOTE:
    The preceding two parameters must be used at the same time, and the -p parameter must be followed by the specific project name.

  • PROJECT_YAML: YAML configuration file of the client.

Configuration Description

Table 1 YAML file on the server

Name

Description

Type

Value Range

project

Project name.

Character string

-

startworkload

Script for starting the service to be optimized.

Character string

-

stopworkload

Script for stopping the service to be optimized.

Character string

-

maxiterations

Maximum number of optimization iterations, which is used to limit the number of iterations on the client. Generally, the more optimization iterations, the better the optimization effect, but the longer the time required. Set this parameter based on the site requirements.

Integer

>10

object

Parameters to be optimized and related information.

For details about the object configuration items, see Table 2.

-

-

**Table 2** Description of object configuration items

Name

Description

Type

Value Range

name

Parameter to be optimized.

Character string

-

desc

Description of parameters to be optimized.

Character string

-

get

Script for querying parameter values.

-

-

set

Script for setting parameter values.

-

-

needrestart

Specifies whether to restart the service for the parameter to take effect.

Enumeration

true or false

type

Parameter type. Currently, the discrete and continuous types are supported.

Enumeration

discrete or continuous

dtype

This parameter is available only when type is set to discrete. Currently, only int and string are supported.

Enumeration

int, string

scope

Parameter setting range. This parameter is valid only when type is set to discrete and dtype is set to int, or type is set to continuous.

Integer

The value is user-defined and must be within the valid range of this parameter.

step

Parameter value step, which is used when dtype is set to int.

Integer

This value is user-defined.

items

Enumerated value of which the parameter value is not within the scope. This is used when dtype is set to int.

Integer

The value is user-defined and must be within the valid range of this parameter.

options

Enumerated value range of the parameter value, which is used when dtype is set to string.

Character string

The value is user-defined and must be within the valid range of this parameter.

ref

Recommended initial value of the parameter

Integer or character string

The value is user-defined and must be within the valid range of this parameter.

Table 3 Description of configuration items of a YAML file on the client

Name

Description

Type

Value Range

project

Project name, which must be the same as that in the configuration file on the server.

Character string

-

iterations

Number of optimization iterations.

Integer

≥ 10

benchmark

Performance test script.

-

-

evaluations

Performance test evaluation index.

For details about the evaluations configuration items, see Table 4.

-

-

**Table 4** Description of evaluations configuration item

Name

Description

Type

Value Range

name

Evaluation index name.

Character string

-

get

Script for obtaining performance evaluation results.

-

-

type

Specifies a positive or negative type of the evaluation result. The value positive indicates that the performance value is minimized, and the value negative indicates that the performance value is maximized.

Enumeration

positive or negative

weight

Weight of the index. The value ranges from 0 to 100.

Integer

0-100

threshold

Minimum performance requirement of the index.

Integer

User-defined

Example

The following is an example of the YAML file configuration on a server:

project: "example"
maxiterations: 10
startworkload: ""
stopworkload: ""
object :
  -
    name : "vm.swappiness"
    info :
        desc : "the vm.swappiness"
        get : "sysctl -a | grep vm.swappiness"
        set : "sysctl -w vm.swappiness=$value"
        needrestart: "false"
        type : "continuous"
        scope :
          - 0
          - 10
        ref : 1
  -
    name : "irqbalance"
    info :
        desc : "system irqbalance"
        get : "systemctl status irqbalance"
        set : "systemctl $value sysmonitor;systemctl $value irqbalance"
        needrestart: "false"
        type : "discrete"
        options:
          - "start"
          - "stop"
        dtype : "string"
        ref : "start"
  -
    name : "net.tcp_min_tso_segs"
    info :
        desc : "the minimum tso number"
        get : "cat /proc/sys/net/ipv4/tcp_min_tso_segs"
        set : "echo $value > /proc/sys/net/ipv4/tcp_min_tso_segs"
        needrestart: "false"
        type : "continuous"
        scope:
          - 1
          - 16
        ref : 2
  -
    name : "prefetcher"
    info :
        desc : ""
        get : "cat /sys/class/misc/prefetch/policy"
        set : "echo $value > /sys/class/misc/prefetch/policy"
        needrestart: "false"
        type : "discrete"
        options:
          - "0"
          - "15"
        dtype : "string"
        ref : "15"
  -
    name : "kernel.sched_min_granularity_ns"
    info :
        desc : "Minimal preemption granularity for CPU-bound tasks"
        get : "sysctl kernel.sched_min_granularity_ns"
        set : "sysctl -w kernel.sched_min_granularity_ns=$value"
        needrestart: "false"
        type : "continuous"
        scope:
          - 5000000
          - 50000000
        ref : 10000000
  -
    name : "kernel.sched_latency_ns"
    info :
        desc : ""
        get : "sysctl kernel.sched_latency_ns"
        set : "sysctl -w kernel.sched_latency_ns=$value"
        needrestart: "false"
        type : "continuous"
        scope:
          - 10000000
          - 100000000
        ref : 16000000

  

The following is an example of the YAML file configuration on a client:

project: "example"
iterations : 10
benchmark : "sh /home/Benchmarks/mysql/tunning_mysql.sh"
evaluations :
  -
    name: "tps"
    info:
        get: "echo -e '$out' |grep 'transactions:' |awk '{print $3}' | cut -c 2-"
        type: "negative"
        weight: 100
        threshold: 100

  

Example

  • Perform tuning.

    # atune-adm tuning example-client.yaml
    
  • Restore the initial configuration before tuning. The example value is the project name in the YAML file.

    # atune-adm tuning --restore --project example
    
Rating

search results about are found