---
url: /en/docs/common/faq/caselibrary/pvs_vgs.md
---
# "Unknown" Errors in pvs or vgs

## Symptom

When running `pvs` or `vgs` commands, physical volumes (PVs) or volume groups (VGs) may display "unknown" status.

![](./figures/pvs-1.png)

![](./figures/pvs-2.png)

## Possible Causes

This issue typically occurs due to corrupted metadata, preventing the system from reading complete information about the storage components. The affected PV or VG becomes unusable until repaired. For verification, you can examine raw disk data using the `hexdump` utility (see [LVM Label Corruption](./lvm.md) for details).

## Solution

### Fixing Unrecognized VGs

```bash
pvck --repair --file /etc/lvm/backup/vg /dev/sdb
```

VG backup files are stored in `/etc/lvm/backup`.

### Repairing Unrecognized PVs

```bash
pvcreate --uuid xxx --restorefile /etc/lvm/backup/vg /dev/sdc
vgcfgrestore --file /etc/lvm/backup/vg vg
vgchange -ay vg
```

The UUID can be found in the backup files located at `/etc/lvm/backup`.
After performing these repairs, verify the fix by running `pvs` or `vgs` again. Normal output indicates successful restoration.

---

---
url: /en/docs/common/faq/caselibrary/mountsysroot.md
---
# /sysroot Mount Failure

## Context

### System Specifications

Hardware platform: TaiShan200 (model 1280V2)
Kernel version: 5.10.0-153.1.0.81.oe2203sp2.aarch64

### Software Version

openEuler 22.03-LTS-SP2

## Symptom

During repeated power cycling tests, intermittent mount failures occur when the system fails to load ext4 or VFAT driver into the kernel during boot.

![](./figures/fd22e53b-5775-40ac-b194-6932ad81958e.png)

## Possible Causes

This low-probability boot-time issue requires meticulous investigation. The diagnostic process follows a progressive elimination approach.

### 1. Mount Operation Failure

![](./figures/8df63e36-e202-4d8b-b477-4bd0c1d9d826.png)

Debugging with `udev.log_priority=debug rd.debug=1` reveals:

![](./figures/cda96442-d6ba-4ab0-8d56-87b248b0ef41.png)

The system call fails during mount. Additional kernel logging shows:

![](./figures/98bb38d3-bbbf-475c-b78d-bf4b2bf08528.png)

Critical findings regarding VFAT mounting:

1. The kernel attempts to load the required module via `request_module1` when `get_fs_type1` detects its absence.
2. Though the execution path reaches user-space modprobe invocation (`call_modprobe->call_usermodehelper_exec`), it returns error code 256.

**Diagnosis**: User-space module loading fails with exit code 256, warranting investigation into the modprobe failure mechanism.

### 2. Module Load Failure

Because user-space logs are directed to the command line and unavailable in emergency mode, kernel-level debug prints were implemented:

![image](./figures/f1269fd0-7fb4-462c-a21c-043778edace0.png)

![image](./figures/3c883235-950a-45d1-b10d-3ee4cbda5cd2.png)

Log analysis confirms that the driver failed to load during kernel execution via `modprobe`.

**DIagnosis**: Either `module_sig_check` or `setup_load_info` returned error -129, causing the failure.

### 3. Signature Verification Failure

The verification process fails with error code -129, occurring through the following call chain:

```txt
load_module->module_sig_check->mod_verify_sig->verify_pkcs7_signature->verify_pkcs7_message_sig->pkcs7_validate_trust->pkcs7_validate_trust_one->verify_signature->public_key_verify_signature->crypto_akcipher_verify->pkcs1pad_verify->pkcs1pad_verify_complete
```

![image](./figures/7be4d825-680e-4dc0-989c-ae01843f90be.png)
![image](./figures/50c5f99c-f86d-4e23-bd4e-36b536837312.png)

Log analysis shows that while `out_buf` remains unchanged in both successful and failed verifications, the values of `req_ctx->out_buf + ctx->key_size` differ, indicating an anomaly in the data.
The problematic data is retrieved via `sg_pcopy_to_buffer` and corresponds to the digest portion of the signature. This digest is produced by hashing the original data and encrypting it with a private key. During verification, the public key decrypts the signature to extract the digest, while the system independently recalculates the digest from the original data. A mismatch between these values suggests either data corruption or tampering.

Additional logging confirmed that `pkcs7->data` remains consistent in both passing and failing cases, narrowing the issue to the digest comparison phase.
![image](./figures/64f55b4f-466f-4e56-86bc-c8714c3a1e22.png)

Further investigation revealed that all failures occurred when using the `sha256-ce` cryptographic driver, which employs ARMv8 CPU instructions for accelerated hashing. Since the issue could not be reproduced on other systems with the same kernel version, the problem appears to be hardware-specific.

![image](./figures/c2b268e3-78f1-4d7c-b459-50b90e73c5b2.png)

A test script repeatedly loading and unloading the kernel module successfully reproduced the error in the affected environment.

![image](./figures/ff7fd456-3f56-4e1c-9bd6-ace9565c271f.png)

**Diagnosis**: The evidence points to a CPU hardware issue.

### 4. CPU Hardware Defect

The failure does not occur in standard test environments.
Hardware diagnostics revealed:

1. The affected processor is from an early engineering sample batch.

2. Specific Arm instruction executions fail on:

   ```txt
   AdvsimdLoadStore
   LCRTSveVectorMove
   ```

3. Diagnostic tests confirm failures exclusively on CPU cores 130 and 131.

   When the kernel module stress test script is pinned to cores 130/131, the signature verification failure consistently occurs. The same test passes when restricted to core 129.

![image](./figures/22e3a767-9e64-4561-be56-76b92e3c17ad.png)

**Diagnosis**: Defective CPU silicon in cores 130/131 causes cryptographic instruction failures.

## Solution

**Hardware replacement**:

Physically replacing the faulty CPU with a production-qualified unit provides a permanent solution.

**Software mitigation**:

Implement CPU affinity controls to exclude the malfunctioning cores from critical operations as a temporary mitigation.

---

---
url: /zh/docs/common/faq/caselibrary/zabbix.md
---
# 22.03 LTS 安装zabbix 教程

## openEuler 最小安装

关闭防火墙

```text
systemctl stop firewalld
systemctl disable firewalld
```

## MYSQL服务安装配置

1. 安装mysql

   ```text
   dnf install mysql mysql-server mysql-common mysql-libs mysql-devel mysql-selinux --nogpgcheck
   ```

2. 启动mysql服务

   ```text
   systemctl enable mysqld
   systemctl start mysqld
   systemctl status mysqld
   ```

3. 配置密码

   ```mysql
   mysql -uroot -p
   > password 回车
   ALTER USER 'root'@'localhost' IDENTIFIED BY '密码';
   ```

## 安装ZABBIX服务

```text
dnf config-manager --add-repo https://repo.oepkgs.net/openeuler/rpm/openEuler-22.03-LTS/contrib/others/aarch64/
dnf clean all && dnf makecache
dnf install zabbix-server-mysql zabbix-web-mysql zabbix-nginx-conf zabbix-sql-scripts zabbix-agent --nogpgcheck
```

## 配置和启动zabbix进程

1. 配置zabbix

   ```mysql
   mysql -uroot -p
   > password
   create database zabbix character set utf8mb4 collate utf8mb4_bin;
   create user zabbix@localhost identified by '密码';
   grant all privileges on zabbix.* to zabbix@localhost;
   set global log_bin_trust_function_creators = 1;
   quit;
   ```

2. 初始化架构和数据

   ```text
   zcat /usr/share/doc/zabbix-sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
   ```

   Disable log\_bin\_trust\_function\_creators option after importing database schema.

   ```mysql
   mysql -uroot -p
   > password
   set global log_bin_trust_function_creators = 0;
   quit;
   ```

3. 修改zabbix server配置数据库和PHP

   ```text
   vi  /etc/zabbix/zabbix_server.conf 
   ---
   DBPassword=密码
   ---
   vi /etc/nginx/conf.d/zabbix.conf
   ---
   listen 8080;#取消注释
   server_name example.com;#取消注释
   ---
   ```

4. 升级net-snmp

   ```text
   dnf install net-snmp  net-snmp-devel net-snmp-utils --nogpgcheck
   ```

5. 启动zabbix进程

   ```text
   systemctl restart zabbix-server zabbix-agent nginx php-fpm
   systemctl enable zabbix-server zabbix-agent nginx php-fpm
   ```

6. 访问zabbix首页

   默认端口是8080

---

---
url: /en/docs/common/faq/server/atune_faqs.md
---
# A-Tune FAQ

## 1: An error occurs when the  **train** command is used to train a model, and the message "training data failed" is displayed

Cause: Only one type of data is collected by using the  **collection**command.

Solution: Collect data of at least two data types for training.

## 2: atune-adm cannot connect to the atuned service

Possible cause:

1. Check whether the atuned service is started and check the atuned listening address.

   ```shell
   systemctl status atuned
   netstat -nap | grep atuned
   ```

2. The firewall blocks the atuned listening port.

3. The HTTP proxy is configured in the system. As a result, the connection fails.

Solution:

1. If the atuned service is not started, run the following command to start the service:

   ```shell
   systemctl start atuned
   ```

2. Run the following command on the atuned and atune-adm servers to allow the listening port to receive network packets. In the command,  **60001**  is the listening port number of the atuned server.

   ```shell
   iptables -I INPUT -p tcp --dport 60001 -j ACCEPT
   iptables -I INPUT -p tcp --sport 60001 -j ACCEPT
   ```

3. Run the following command to delete the HTTP proxy or disable the HTTP proxy for the listening IP address without affecting services:

   ```shell
   no_proxy=$no_proxy, Listening_IP_address
   ```

## 3: The atuned service cannot be started, and the message "Job for atuned.service failed because a timeout was exceeded." is displayed

Cause: The hosts file does not contain the localhost information.

Solution: Add localhost to the line starting with  **127.0.0.1**  in the  **/etc/hosts**  file.

```text
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
```

---

---
url: /zh/docs/common/faq/server/atune_faqs.md
---
# A-Tune常见问题与解决方法

## **问题1：train命令训练模型出错，提示“training data failed”**

原因：collection命令只采集一种类型的数据。

解决方法：至少采集两种数据类型的数据进行训练。

## **问题2：atune-adm无法连接atuned服务**

可能原因：

1. 检查atuned服务是否启动，并检查atuned侦听地址。

   ```shell
   # systemctl status atuned
   # netstat -nap | grep atuned
   ```

2. 防火墙阻止了atuned的侦听端口。

3. 系统配置了http代理导致无法连接。

解决方法：

1. 如果atuned没有启动，启动该服务，参考命令如下：

   ```shell
   # systemctl start atuned
   ```

2. 分别在atuned和atune-adm的服务器上执行如下命令，允许侦听端口接收网络包，其中60001为atuned的侦听端口号。

   ```shell
   # iptables -I INPUT -p tcp --dport 60001 -j ACCEPT
   # iptables -I INPUT -p tcp --sport 60001 -j ACCEPT
   ```

3. 不影响业务的前提下删除http代理，或对侦听IP不进行http代理，命令如下：

   ```shell
   # no_proxy=$no_proxy,侦听地址
   ```

## **问题3：atuned服务无法启动，提示“Job for atuned.service failed because a timeout was exceeded.”**

原因：hosts文件中缺少localhost配置

解决方法：在/etc/hosts文件中127.0.0.1这一行添加上localhost

```ini
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
```

---

---
url: /en/docs/common/faq/caselibrary/isulad.md
---
# Accessing iSulad Services Through gRPC and REST Protocols

## gRPC Service Access

The default communication protocol between iSula and iSulad is gRPC, which is mandatory for CRI implementations in Kubernetes environments.

Developers integrating with iSulad via gRPC should consult the protocol buffer definitions at <https://atomgit.com/openeuler/iSulad/tree/master/src/api/services>, organized by functional modules.

For debugging gRPC interactions, install the community-developed grpcurl utility:

```sh
go get github.com/fullstorydev/grpcurl
go install github.com/fullstorydev/grpcurl/cmd/grpcurl
```

Sample debugging command:

```sh
grpcurl -plaintext -H 'tls_mode:0' -proto=api.proto -import-path=../ -unix=true /var/run/isulad.sock runtime.v1alpha2.ImageService.ListImages
```

## REST Service Access

To configure REST-based communication:

1. Clone the [iSulad repository](https://atomgit.com/openeuler/iSulad).
2. Follow the [build instructions](https://atomgit.com/openeuler/iSulad/blob/master/docs/build_docs/README.md).

Enable REST mode during compilation:

```sh
cmake -DENABLE_GRPC=OFF .../
```

Test REST endpoints using `curl`:

```sh
curl -d "{}" --unix-socket /var/run/isulad.sock http://localhost/ContainerService/Version
```

---

---
url: /zh/docs/common/contribute/ai_service_config.md
---
# AI 服务配置指导

## 操作指导

1. 登录`https://cloud.siliconflow.cn`，并为自己的账户充值。
2. 获取 API 密钥，如下图所示。

   ![get api key](figures/get_apikey.png)
3. 进入模型广场，选择合适的模型，点击进入详情页。

   ![models](figures/models.png)
4. 复制模型名字。

   ![copy model name](figures/copy_model_name.png)
5. 进入 DocMate 的 AI 服务配置页，填写以下配置项：

   * 基础URL：`https://api.siliconflow.cn/v1`。
   * API密钥：步骤2中获取的API密钥。
   * 模型名称：步骤3中获取的模型名称，为保障稳定运行，建议填写`Qwen/Qwen3-235B-A22B-Instruct-2507`。

   ![api service config](figures/api_service_config.png)

---

---
url: /zh/docs/common/faq/caselibrary/anaconda.md
---
# anaconda安装常见磁盘问题

## 场景1：安装系统找不到系统盘

### 问题背景

机器出现安装界面中找不到磁盘进行安装。
![image](./figures/磁盘消失.png)

### 现象描述

1.安装界面上无法查看到磁盘；
2.CTRL+ALT+F2切换后台查看使用lsblk可以看到磁盘，并且有相关的分区信息。
![image](./figures/磁盘消失_lsblk.png)

### 原因分析

CTRL+ALT+F2切换后台查看/tmp/storage.log日志，看到扫描磁盘时出现报错，可能是存在分区残留信息导致。

### 解决方案

使用fdisk命令删除分区删除，格式化磁盘。

## 场景2：安装界面卡在磁盘扫描

### 问题背景

在安装系统的时候发现系统一直卡在扫盘导致无法点击选择硬盘的按钮。

### 现象描述

安装界面卡在磁盘扫描，等待一直无法成功。
![image](./figures/920b卡在扫盘.png)

### 原因分析

1. CTRL+ALT+F2切换后台查看/tmp/storage.log日志，看到扫描磁盘一直卡住。
2. 在后台输出相同命令，也同样无法执行完成，磁盘上有脏数据导致扫盘的时候获取不到数据导致的，需要进行格式化硬盘。

### 解决方案

格式化磁盘后重新进行安装。

---

---
url: /en/docs/common/faq/server/applicationdev_faqs.md
---
# Application Development FAQ

## 1. Self-compilation of Some Applications Depending on the **java-devel** Package Fails

### Symptom

The self-compilation of some applications that depend on java-devel fails when the rpmbuild command is executed.

### Possible Causes

To provide OpenJDK features that are updated and compatible with Java applications, the openEuler provides OpenJDK of multiple versions, such as OpenJDK 1.8.0 and OpenJDK 11. The compilation of some applications depends on the **java-devel** package. When the **java-devel** package is installed, the system installs java-11-openjdk of a later version by default. As a result, the compilation of these applications fails.

### Solution

You need to run the following command to install java-1.8.0-openjdk and then run the **rpmbuild** command to perform self-compilation:

```shell
# yum install java-1.8.0-openjdk

```

---

---
url: /zh/docs/common/faq/caselibrary/audit.md
---
# audit写日志占满分区空间出现系统停机

## 问题背景

系统异常停机，需要排查停机原因。

## 现象描述

查看日志记录是由audit执行停机动作，按照预期已配置日志回滚，不应该停机。
查看audit日志目录，存在记录大量日志且已占满空间，如下：

```txt
-r--------. 1 root root 6291639  May 14 04:10 audit.log.968
-r--------. 1 root root 6291629  May 14 03:28 audit.log.969
-r--------. 1 root root 6291630  May 14 02:45 audit.log.970
-r--------. 1 root root 6291627  May 14 02:03 audit.log.971
-r--------. 1 root root 6291546  May 14 01:20 audit.log.972
-r--------. 1 root root 6291689  May 14 00:38 audit.log.973
-r--------. 1 root root 6291705  May 13 23:57 audit.log.974
-r--------. 1 root root 6291528  May 13 23:14 audit.log.975
...
```

## 原因分析

从现象上看，疑似audit日志回滚能力失效，检查auditd.conf配置：

```txt
...
max_log_file = 6 // 日志文件大小6MB
num_logs = 5     // 最大日志文件数5
...
admin_space_left = 50
admin_space_left_action = halt //空间小于admin_space_left执行停机
...
```

按照上述配置，最多出现5个日志文件，但实际超过设定值，不符合预期；系统停机是配置的默认动作，符合预期。

检查message日志，排查audit是否出现异常信息：

```txt
...
2024-06-09T04:59:46.424433+08:00 localhost auditd[21699]: Audit daemon rotating long files with keep option
...
```

从日志来看，触发了audit日志回滚，但是使用“keep option”选项，查看auditd.conf配置：

```txt
...
max_log_file_action = keep_logs //keep_logs 与rotate相似，但忽略num_logs选项
...
```

综上所述，此案例下audit写日志占满空间是由于配置“max\_log\_file\_action = keep\_logs”导致"num\_logs = 5"参数失效。

## 解决方案

将max\_log\_file\_action设置为rotate即可。

---

---
url: /en/docs/common/faq/caselibrary/anaconda.md
---
# Common Drive Issues in the Anaconda Installer

## 1. Missing System Drives During Installation

### Context

The installation interface fails to display available drives for OS installation.

![](./figures/drives_missing.png)

### Symptom

* The installation wizard shows no available drives.
* Drive partitions remain visible via `lsblk` in the terminal (**Ctrl+Alt+F2**)

![](./figures/drives_missing_lsblk.png)

### Possible Causes

System logs at **/tmp/storage.log** indicate drive scanning failures, typically caused by residual partition metadata.

### Solution

1. Access the terminal (**Ctrl+Alt+F2**).
2. Use `fdisk` to remove existing partitions.
3. Reformat the drive.

## 2. Installation Hangs on Drive Scan

### Context

Installation process freezes during drive scanning.

### Symptom

The system becomes unresponsive at drive scanning stage.

![](./figures/drive_scanning_stuck.png)

### Possible Causes

1. System logs at **/tmp/storage.log** show frozen drive scanning operation.
2. Manual drive scan commands fail to complete, indicating drive corruption or invalid sector data.

### Solution

1. Force reboot the system.
2. Reformat affected drive.
3. Restart the installation process.

---

---
url: /en/docs/common/faq/caselibrary/umask.md
---
# Configuring Container umask Values in Docker

## Context

Container umask configuration directly impacts file/directory permissions of newly created resources, affecting container security posture.

## Symptom

Standard openEuler containers initialize with default umask 0027. After modification, the **others** group loses access permissions to newly created files/directories.

![image](./figures/docker_umask_config_1.PNG)

## Possible Causes

The default umask was intentionally set to 0027 in runc implementation to enhance container security and mitigate potential attacks. This modification intentionally restricts **others** group access to new resources.

## Solution

**Option 1:**

Modify Docker service configuration file **/etc/sysconfig/docker**:

Add `--exec-opt native.umask=normal` parameter to OPTIONS line, then restart dockerd:

```bash
systemctl restart docker
```

![image](./figures/docker_umask_config_2.PNG)

**Option 2:**

Include `--annotation native.umask=normal` parameter when executing `docker run|exec` commands:

```bash
docker run --annotation native.umask=normal XXX
docker exec --annotation native.umask=normal XXX
```

![image](./figures/docker_umask_config_3.PNG)

---

---
url: /en/docs/common/faq/community_tools/dde_faqs.md
---
# DDE FAQ

## 1. After DDE Is Installed, Why Are the Computer and Recycle Bin Icons Not Displayed on the Desktop When I Log in as the **root** User

### Issue

After the DDE is installed, the computer and recycle bin icon is not displayed on the desktop when a user logs in as the **root** user.

![img](./figures/dde-1.png)

### Cause

The **root** user is created before the DDE is installed. During the installation, the DDE does not add desktop icons for existing users. This issue does not occur if the user is created after the DDE is installed.

### Solution

Right-click the icon in the launcher and choose **Send to Desktop**. The icon functions the same as the one added by DDE.

![img](./figures/dde-2.png)

---

---
url: /zh/docs/common/faq/community_tools/dde_faqs.md
---
# DDE常见问题与解决方法

## **问题1:安装DDE后，root帐户登录桌面无计算机和回收站图标**

### 问题描述

安装DDE后，root帐户登录桌面无计算机和回收站图标

![img](./figures/dde-1.png)

### 问题原因

由于root用户在安装DDE前已创建，而DDE在安装时不会对已经创建的用户进行新增桌面图标操作。DDE安装后新建用户无此问题。

### 解决方法

用户可通过启动器中右键对应图标发送到桌面即可，无任何功能差异。

![img](./figures/dde-2.png)

---

---
url: /zh/docs/common/contribute/doc_tools_introduction.md
---
# Doc Tools

本插件集成了markdownlint、链接失效等多种常见文档问题的自动化检测修复功能，同时提供文档预览等功能，旨在提升文档开发体验。

## 安装

在 Visual Studio Code 中搜索插件并安装：

![Doc Tools Installation](figures/install_doctools.png)

## 功能总览

### 静态检查

| 名称 | 功能 |
| -----| ----|
| [Markdown Lint](./doc_tools_static_check.md#markdown-lint) | Markdown 语法检查 |
| [Tag Closed Check](./doc_tools_static_check.md#tag-closed-check) | Html 标签闭合检查 |
| [Link Validity Check](./doc_tools_static_check.md#link-validity-check) | 链接可访问性检查 |
| [Resource Existence Check](./doc_tools_static_check.md#resource-existence-check) | 资源有效性检查 |
| [Toc Check](./doc_tools_static_check.md#toc-check) | 目录文件规范性检查 |
| [CodeSpell Check](./doc_tools_static_check.md#codespell-check) | 单词拼写检查  |
| [Filename Check](./doc_tools_static_check.md#filename-check) | Markdown 文件命名规范检查  |
| [Punctuation Check](./doc_tools_static_check.md#punctuation-check) | 标点符号检查  |

### 高级功能

| 名称 | 功能 |
| -----| ----|
| [文档预览](./doc_tools_functions.md#文档预览) | 预览 openEuler 文档风格的页面  |
| [目录生成](./doc_tools_functions.md#目录生成) | 自动生成 \_toc.yaml  |
| [批量检查链接可访问性](./doc_tools_functions.md#批量检查链接可访问性) | 批量检查选中目录下 Markdown 和 \_toc.yaml 所有链接的可访问性 |
| [批量检查文件命名规范](./doc_tools_functions.md#批量检查文件命名规范) | 批量检查选中目录下文件和子目录的命名规范性 |
| [批量检查中英文文档名称一致性](./doc_tools_functions.md#检查中英文文档名称一致性) | 批量检查选中目录下 Markdown 中英文文档名称的一致性 |
| [生成链接锚点并复制](./doc_tools_functions.md#生成链接锚点并复制) | 将选中标题生成链接锚点并复制 |

## 全局配置

插件支持以下配置项（可在 VSCode 设置中搜索 `docTools.scope` 或通过 `settings.json` 进行配置）：

* `docTools.scope`
  * 类型：`boolean`
  * 说明：是否检查范围仅限于 `docs/zh` 和 `docs/en` 目录
  * 默认：`false`

### 全局配置示例

```json
{
  "docTools.scope": false // 启用检查范围仅限于 docs/zh 和 docs/en 目录，默认检查项目全局文档
}
```

---

---
url: /en/docs/common/faq/caselibrary/docker.md
---
# Docker Container Terminates After a Certain Period

## Context

A Docker container launches successfully but unexpectedly stops after operating for a duration without other observable errors.

## Symptom

When a Docker container image is executed, the CLI remains inactive before automatically closing after prolonged idleness. This behavior persists even for containers launched in background via `docker run -d`.

![image](./figures/docker-container-auto-exit5.PNG)

## Root Cause

The container image includes a predefined 300-second timeout (TMOUT) setting in **/etc/profile**. When users access the container via SSH and maintain terminal inactivity beyond this threshold, the system forcibly terminates the session.

![image](./figures/docker-container-auto-exit1.PNG)

## Resolution

1. Deploy the container with existing TMOUT configuration.

   ```shell
   docker run -itd XXX bash
   ```

2. Adjust container settings through interactive execution.

   ```shell
   docker exec -it $container_id bash
   ```

   1. Disable the timeout by setting `TMOUT=0` in **/etc/profile**:

      ![image](./figures/docker-container-auto-exit2.PNG)

   2. Ensure profile loading by adding `source /etc/profile` to **/root/.bashrc**:

      ![image](./figures/docker-container-auto-exit3.PNG)

3. Reinitialize the container to apply permanent changes:

   ```shell
   docker restart $container_id
   ```

   ![image](./figures/docker-container-auto-exit4.PNG)

---

---
url: /en/docs/common/faq/cloud/docker_faqs.md
---
# Docker FAQ

## 1. Additional Mount Point in Docker v18.09.9 Compared to v19.03.0 and Later

In Docker version 18.09.9, containers have an extra mount point compared to those launched in Docker v19.03.0 and later. This is because the default `ipcmode` in v18.09 is set to `shareable`, which creates an additional `shmpath` mount point. To resolve this, either update the `ipcmode` option to `private` in the Docker configuration file or upgrade to a newer Docker version.

---

---
url: /zh/docs/common/faq/caselibrary/docker.md
---
# docker容器运行一定时间后退出

## 问题背景

docker正常启动容器，容器运行一段时间后，在无其他异常的情况下，容器会退出。

## 现象描述

运行docker容器镜像，字符界面长时间处于空闲状态，一段时间后字符界面会自动退出，包括docker run -d后台启动运行容器，也会有一段时间后自动退出现象。

![image](./figures/docker容器运行自动退出5.PNG)

## 原因分析

运行容器镜像，当开发者通过ssh登陆且字符界面长时间处于空闲状态，字符界面会自动退出。这是由于容器镜像在制作时在/etc/profile文件中设置了TMOUT字段，当前此值默认为300S。

![image](./figures/docker容器运行自动退出1.PNG)

## 解决方案

1. 首先运行一个设置了TMOUT环境变量的容器

   ```shell
   docker run -itd XXX bash
   ```

2. 执行exec命令修改容器内配置

   ```shell
   docker exec -it $container_id bash
   ```

   1. 先修改/etc/profile中的TMOUT环境变量值为0

      ![image](./figures/docker容器运行自动退出2.PNG)

   2. 再在/root/.bashrc中增加一行source /etc/profile

      ![image](./figures/docker容器运行自动退出3.PNG)

3. 最终重启容器，TMOUT已经被设置为0，此时容器就不会再因为超时退出

   ```shell
   docker restart $container_id
   ```

   ![image](./figures/docker容器运行自动退出4.PNG)

---

---
url: /zh/docs/common/faq/cloud/docker_faqs.md
---
# Docker常见问题与解决方法

## **问题1：docker v18.09.9拉起的容器挂载点相比docker v19.03.0及以后的版本多一个**

原因：18.09版本的docker，默认ipcmode为shareable，该配置会多挂载一个shmpath挂载点。
解决方法：结合实际情况修改docker配置文件中的ipcmode选项为private，或者使用新版本的docker。

---

---
url: /zh/docs/common/faq/caselibrary/docker_1.md
---
# docker服务启动失败常见问题汇总

## 场景1： 报错Non existing device xxx-thinpool和Unable to take ownership of thin-pool

### 问题背景

docker配置thinpool存储驱动启动服务。

### 现象描述

执行systemctl restart docker发现如下报错：

![thinpool](./figures/docker启动thinpool报错.png)

### 原因分析

这种通常是thinpool损坏导致的，需要进行thinpool的恢复和重建。

### 解决方案

可参考如下步骤对thinpool进行重建流程：

1. 对于防止systemctl重启导致的恢复异常重建之前应将其关闭。

   ```bash
   mv /usr/lib/systemd/system/docker.service /usr/lib/systemd/system/docker.service_bak
   ```

2. 删除/var/lib/docker目录下除hooks文件夹之外的所有文件。

   ```bash
   cd /var/lib/docker/
   rm -rf !(hooks)
   ```

3. 执行重建thinpool的操作。

   ```bash
   lvremove /dev/docker/thinpool
   lvcreate --wipesignatures y -n thinpool docker -L 19g（95%VG）
   lvcreate --wipesignatures y -n thinpoolmeta docker -L 0.2g（1%VG）
   lvconvert -y --zero n -c 512K --thinpool docker/thinpool --poolmetadata docker/thinpoolmeta
   lvchange --metadataprofile docker-thinpool docker/thinpool
   ```

4. 打开之前操作的docker重拉。

   ```bash
   mv /usr/lib/systemd/system/docker.service_bak /usr/lib/systemd/system/docker.service
   systemctl enable docker.service
   ```

5. 重启docker。

   ```bash
   systemctl start docker
   ```

systemctl status docker 结果中查看docker是否running。

## 场景2：systemd报错timeout

### 问题背景

正常启动docker.service服务。

### 现象描述

启动超时，导致dockerd服务启动失败：

![image](./figures/docker启动服务超时.png)

### 原因分析

通常是docker服务启动时间太长，导致systemd拉起服务超时90s。

### 解决方案

这种情况一般是docker服务启动的时候需要清理的内容太多，可能的原因有容器数量太多或者环境cpu压力较大，建议拉长超时时间重试。

1. 在/usr/lib/systemd/system/docker.service的\[Service]添加`TimeoutSec=0`。

2. 重启docker服务systemctl restart docker。

## 场景3：报错unable to configure the Docker daemon with file /etc/docker/daemon.json和unable to configure the Docker daemon with file /etc/sysconfig/docker

### 问题背景

修改docker daemon.json配置文件后重新启动docker服务。

### 现象描述

dockerd服务启动失败：
![image](./figures/docker启动配置文件错误.png)

### 原因分析及解决方案

一般是配置文件有问题，具体到报错中提示的文件中定位，找到错误的配置并修改即可。

## 场景4：systemctl status docker不打印日志，但是启动不了

### 问题背景

执行systemctl status docker命令查看docker服务日志。

### 现象描述

docker服务不打印日志，且启动失败：

![image](./figures/docker启动失败且不打印日志.png)

### 原因分析及解决方案

#### 原因1

一般出现于日志服务有问题的情况。

#### 解决方案

1. 检查rsyslog服务是否正常运行。

2. 如果没有运行的话可以先重启日志服务。

3. 根据docker服务启动的报错日志进行下一步定位。

#### 原因2

/run被占满，docker出现过日志打印占满/run目录的问题。

#### 解决方案

1. 进入`/var/run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/`目录，执行`du -sh *`找到空间占用较大的目录
2. 进入步骤1中找到的目录，执行`echo "" > log.json`清空该异常日志文件，**注意不能删除该文件，删除会导致容器运行异常。**

## 场景5：dockerd链接/var/run/containerd/containerd.sock失败

### 问题背景

启动docker.service服务。

### 现象描述

docker服务启动失败并报错:

![image](./figures/docker启动链接containerd.sock失败.png)

### 原因分析

该场景常见于在/var/run/containerd目录下存在非法的containerd.sock文件，比如该文件为一个失效的软链接的情况。

### 解决方案

删除错误文件并重启dockerd即可恢复。

## 场景6：创建/var/run/docker目录失败导致服务启动失败

### 问题背景

启动docker服务。

### 现象描述

服务启动失败并报错:

![image](./figures/docker启动创建docker目录失败.png)

### 原因分析

常见于/var/run目录有问题的情况，比如软链接失效。

### 解决方案

使用ln命令重建软链接，重启docker服务恢复。

## 场景7：docker服务初始化，报错no space left

### 问题背景

启动docker服务。

### 现象描述

docker服务启动失败并报错:

![image](./figures/docker启动报错no_spcae_left.png)

### 原因分析

常见于给docker预留的存储空间被占满的情况：

![image](./figures/docker启动报错no_space_left2.png)

### 解决方案

重新预分配，将给docker的存储空间增大。

## 场景8：containerd、containerd-shim、runc二进制权限不对导致docker服务无法启动

### 问题背景

启动docker服务。

### 现象描述

docker服务启动失败并报错:

![image](./figures/docker启动二进制权限不正确.png)

### 原因分析及解决方案

从报错来看是找不到runc二进制，实际上是可执行权限被去掉导致的，修改为原本的的权限之后恢复。

更多常见相关问题可在后续继续补充。

---

---
url: /zh/docs/common/faq/caselibrary/umask.md
---
# docker配置容器umask值

## 问题背景

容器的umask配置成不同的值会影响到容器中新创建的文件和目录的权限，影响容器使用的安全性。

## 现象描述

正常openeuler容器启动，容器默认的umask值为0027，修改后others群组将无法访问新建文件或目录。

![image](./figures/docker配置umask值1.PNG)

## 原因分析

为了容器使用安全性，避免容器受到攻击，修改runc的实现，将默认umask修改为0027，修改后others群组将无法访问新建文件或目录。

## 解决方案

方案一：
可以修改docker服务启动配置文件/etc/sysconfig/docker, 在OPTIONS行添加--exec-opt native.umask=normal参数，并重启dockerd服务。

```bash
systemctl restart docker
```

![image](./figures/docker配置umask值2.PNG)

方案二：

使用docker run/exec命令行时增加--annotation native.umask=normal参数。

```bash
docker run --annotation native.umask=normal XXX
docker exec --annotation native.umask=normal XXX
```

![image](./figures/docker配置umask值3.PNG)

---

---
url: /zh/docs/common/contribute/doc_mate.md
---
# DocMate 智能写作助手

## 功能介绍

DocMate 是专为开源社区文档打造的 VS Code 智能写作助手。它为文档创作提供全流程的 AI 支持，有效提升文档的规范性与写作效率，让每一位开发者都能轻松写出专业级技术文档。

* 文档检查
  * 中文错别字检查；
  * 标点符号规范；
  * 空格规范；
  * 格式规范；
  * 风格一致性；
  * 超链接检查；
  * 术语规范。
* 文本润色
  * 表达优化：提升文档的清晰度和专业性；
  * 结构调整：优化段落结构与语句逻辑；
  * 语言精炼：简化技术表达，使其更加简明易懂。
* 智能重写
  * 支持自定义重写指令，满足多样化的内容需求。

## 使用方法

* 安装插件
  1. 打开 VS Code，进入扩展商店（Ctrl+Shift+X）；

  2. 搜索“DocMate”，点击安装；

     ![docmate check](figures/docmate_install.png)

  3. 点击左侧活动栏的**DocMate**图标，打开专用侧边栏。
* 配置AI服务
  1. 点击侧边栏右上角的**设置**图标；
  2. 用户需自行获取模型服务，可参考[指导](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/contribute/ai_service_config.md)进行配置；
* 基本使用

  1. 打开文档：在 VS Code 中打开 Markdown 文件；
  2. 选择文本：选中需要处理的文本内容；
  3. 按需进行检查、润色或改写；
  4. 查看处理结果和建议，可一键应用改进建议。

  ![docmate use](figures/docmate_use.png)
* 检查项配置

  1. 点击侧边栏右上角的**检查规则管理**图标；
  2. 查看默认检查规则；
  3. 通过开关控制，灵活启用/禁用检查项；
  4. 点击右上角的**新建规则**按钮，自定义符合特定需求的检查规则。

  ![docmate config](figures/docmate_config.png)

## 使用示例

* 检查

  ![docmate check](public_sys_resources/docmate_check.gif)

* 润色

  ![docmate improve](public_sys_resources/docmate_improve.gif)

* 改写

  ![docmate rewrite](public_sys_resources/docmate_rewrite.gif)

---

---
url: /en/docs/common/contribute/documentation_writing_specifications.md
---
# Documentation Writing Specifications

This writing specification outlines the requirements for the structure, content elements, and language style of documents in the openEuler docs repository to ensure a consistent style across openEuler documentation.

Before starting to write openEuler documentation, familiarize yourself with this specification. **Improvement suggestions are welcome**.

## Document Structure Specifications

Feature manuals typically include an overview, background introduction, operational documentation (installation, deployment, and usage guide), frequently asked questions, and appendix. Developers may add or remove sections based on project needs.
For reference, see the [A-Tune example](https://docs.openeuler.org/en/docs/22.03_LTS_SP2/docs/A-Tune/A-Tune.html), which includes the content below.

### Overview

Provide a brief introduction to the feature definition and functionality, followed by a description of the target audience.

**Example**:

```markdown
This document describes how to install and use A-Tune, which is a performance self-optimization software for openEuler.

This document is intended for developers, open-source enthusiasts, and partners who use the openEuler system and want to know and use A-Tune. You need to have basic knowledge of the Linux OS.
```

### Background Introduction

Background documentation should cover the feature context, introduction, and architecture. Common titles include: `Understanding xxx`.

### Operational Documentation

* Requirements
  Specify the hardware and software environment, permissions, and other prerequisites needed to perform the operations.

  **Example**:

  ```markdown
  Hardware requirements: xxx processor.
  Software requirements: openEuler version xx, root privileges.
  ```

* Steps
  Operational steps include **installation and deployment** and **usage instructions**.

  Guidelines for writing steps:

  * Each step should describe a single action. Avoid combining multiple actions into one step.
  * Clearly indicate optional conditions if steps are optional.
  * For steps involving interface calls (such as tools or SQL statements), provide explanations for the interfaces used.

* Result Verification
  Describe how to verify the correctness of the operation results. If verification is closely tied to a step, include it within that step. For example, the return information from executing an SQL statement.

### Appendix

The appendix can provide definitions for terms and abbreviations.

## Content Element Specifications

### Naming

When creating new documents, add a MarkDown file (with the `.md` extension) to the appropriate directory.

* **Rule**: Ensure the document name is unique and does not conflict with existing files. Rename if necessary.
* **Rule**: Use **English** for all document names.
* **Rule**: Avoid parentheses in file names, as they can disrupt directory display. Replace them with underscores (`_`) or hyphens (`-`).

**Example**:

```text
installation_and_deployment.md  # Document for "Installation and Deployment"
```

### Headings

* **Rule**: Headings should clearly and concisely summarize the section content without omitting key details.
* **Rule**: For procedural documents, use verb-object structures (for example, "Requesting Permissions"). Ensure consistency in heading structures for the same level and type.
* **Rule**: Avoid ending headings with punctuation. Use parentheses for additional context and exclude special characters like `?`.
* **Rule**: Separate headings from body text with a blank line.
* **Rule**: Format headings with `#` followed by a space and the heading text. Increment heading levels one at a time, starting with the top-level heading.

**Example**:

```markdown
# Level 1 Heading
## Level 2 Heading
### Level 3 Heading
#### Level 4 Heading
##### Level 5 Heading
###### Level 6 Heading
```

### Body

**Formatting instructions**:

* *Italic*: Enclose text in single asterisks (`*`) for italic formatting.

  ```txt
  *italic text*
  ```

* **Bold**: Enclose text in double asterisks (`**`) for bold formatting.

  ```txt
  **bold text**
  ```

* ***Bold italic***: Enclose text in triple asterisks (`***`) for bold italic formatting.

  ```txt
  ***bold italic text***
  ```

* Escape: Use a backslash (`\`) to escape special characters.

  ```txt
  \<escaped markup symbol>
  ```

**Rule**: Always use the backslash (`\`) to escape characters as required.

**Rule**: Separate consecutive escape characters with a space, for example, `\{ \}`.

**Rule**: Maintain both Chinese and English versions of documentation. For translation support, contact <liujingrong3@huawei.com> or <wudonger@huawei.com>.

### Images

**Usage**:

```bash
![alt text](./path/to/image.png)

![alt text](./path/to/image.png "optional title")
```

**Rule**: Place all images in the **figures** subdirectory of the document folder. For example, the [A-Tune User Guide](https://docs.openeuler.org/en/docs/22.03_LTS_SP2/docs/A-Tune/A-Tune.html) stores its images in [this directory](https://atomgit.com/openeuler/docs-centralized/tree/stable2-22.03_LTS_SP2/docs/en/docs/A-Tune/figures). Always use **relative paths** for references.

**Rule**: Only use original or properly licensed images to avoid copyright issues.

**Rule**: Position images adjacent to their relevant text sections.

**Rule**: The preferred image format is PNG, with JPG as an alternative. Images must not exceed 640 pixels in height or 393 pixels in width, and should ideally be under 150 KB in file size.

**Rule**: For screenshots, crop to focus on essential content within these dimensions. Use red borders or text labels to emphasize important details in graphics.

**Example**:

```markdown
![](./figures/ci_check_result.jpg)
```

The `./` prefix in the path is mandatory for proper online display.

### Code Blocks

Code examples illustrate how to implement specific features, serving as references for developers during coding and debugging.

**Rule**: Ensure the code is logically and syntactically correct.
**Rule**: Clearly separate input and output sections where applicable.
**Rule**: Include comments to explain critical steps in the code.
**Rule**: Enclose inline code and commands in single backticks (for example, `code snippet`).
**Rule**: Format block code with either triple backquotes or four-space indentation (no TABs), preceded and followed by blank lines.

**Examples**

* Inline code

  ```markdown
  The `printf()` function
  ```

* Block code

  ```python
  #!/usr/bin/env python3
  print("Hello, World!")
  ```

  ```c
  #include <stdio.h>
  int main(void)
  {
      printf("Hello world\n");
  }
  ```

### Lists

* **Unordered lists**: Represented by asterisks (`*`), plus signs (`+`), or hyphens (`-`), each followed by a space. Maintain uniform markers within a list.

  ```markdown
  * First item
  * Second item
  * Third item

  + First item
  + Second item
  + Third item


  - First item
  - Second item
  - Third item
  ```

* **Ordered lists**: Numbered items with a trailing period (`.`).

  ```markdown
  1. First item
  2. Second item
  3. Third item
  ```

* **Nested lists**: Sub-items indented by four spaces (no TABs).

  ```markdown
  1. First item:
      - Sub-item A
      - Sub-item B
  2. Second item:
      - Sub-item A
      - Sub-item B
  ```

**Rule**:Use ordered lists when items follow a clear sequence or logical order.

**Rule**:Use unordered lists for parallel relationships or multiple-choice options.

**Rule**:Omit punctuation for terms or phrases in list items.

**Rule**:Include periods for complete sentences in list items.

**Rule**:If mixing phrases and sentences is unavoidable, apply periods to all items.

**Rule**:Alternatively, separate items with semicolons, ending the final item with a period.

### Annotation Symbols

The following annotation symbols may appear in documentation to indicate different scenarios and levels of importance. Select the appropriate symbol based on the significance of the information being highlighted.

| Symbol | Purpose/Meaning | Usage |
|--------|----------------|-------|
| **Warning** | Failure to follow this warning may cause task interruption or unexpected results, though recovery is possible. | `> [!WARNING]Warning`  `> Content` |
| **Note** | Provides helpful tips or useful reference information. | `> [!NOTE]Note`  `> Content` |

> \[!NOTE]Note
>
> * Choose the appropriate annotation symbol based on the documentation context and apply the correct styling.
> * Notes/Warnings can contain nested ordered/unordered lists, but avoid tables and code blocks.
> * To prevent style breaks, ensure `>` remains continuous.
> * Keep note/warning content concise. Consider placing lengthy explanations in the main text or splitting them into sections. Avoid excessive empty lines within styled blocks.

### Links

**Rule**: Verify link destinations exist to prevent navigation errors. Use standard Markdown syntax instead of HTML.

**Examples**:

```markdown
- Website link
This is the link to the [openEuler website](https://www.openeuler.org/en/).

- Relative path
[CI Pipeline Rules](./ci_rules.md)
```

### Tables

**Rule:** Use standard Markdown table syntax in documentation. Avoid HTML table formatting.

**Example:**

```markdown
| Header 1 | Header 2 |
| -------- | -------- |
| Cell 1   | Cell 2   |
| Cell 3   | Cell 4   |
```

**Alignment options:**

* `-:` Right-aligned content
* `:-` Left-aligned content
* `:-:` Centered content

**Rule**: Omit punctuation when all cells in a column contain terms/phrases.

**Rule**: Use periods when all cells contain complete sentences.

**Rule**: Apply periods uniformly if mixed content cannot be avoided.

### Punctuation

**Rule:** For numbered/bulleted lists, use periods consistently if items are complete sentences. Omit punctuation if all items are phrases. **Maintain uniformity: either apply punctuation throughout or omit it entirely.**

**Rule:** Always use half-width (ASCII) numerals.

**Rule:** Reserve exclamation marks exclusively for warnings about critical consequences involving equipment safety or personal harm. Avoid exclamation marks in all other contexts.

## Language Style Specifications

**Rule:** Submissions must exclusively pertain to openEuler features.

**Rule:** Content must not include sensitive information or material exhibiting strong racial/gender discrimination.

**Rule:** All submissions must be original work without intellectual property infringement.

**Rule:** Content must remain factual and objective. Avoid exaggerated promotional language.

**Unacceptable documentation practices**

Submitting excessive pull requests in a short timeframe via automated tools to address trivial issues (such as typos, grammar errors, date inaccuracies, and awkward phrasing) without substantive value.

---

---
url: /en/docs/common/faq/caselibrary/efivars.md
---
# EFI Variables Installation Errors

## 1. Failed to Create EFI Boot Entry

### Context

The installer displays a warning about being unable to create an EFI boot entry, though installation can continue after dismissing the alert.
![image](./figures/installation_unable_to_add_bootloader.png)

### Symptom

1. Access the terminal (**Ctrl+Alt+F2**) and examine **/tmp** logs.

2. **storage.log** reports "no space left" errors.

   ![image](./figures/installation_no_bootloader_space.png)

3. The **/sys/firmware/efi/efivars/** directory contains an excessive number of boot entries.
   ![image](./figures/installation_ls_bootloader.png)

### Possible Causes

The BIOS variable storage area has reached capacity, preventing new EFI boot entries.

### Resolution

Clear the BIOS variable cache.

## 2. Mount Error 32

### Context

The installer fails with "mount failed: error 32" during operation.

![image](./figures/installation_bootloader_error.png)

### Symptom

The error occurs when **/sys/firmware/efi/efivars/** is mounted.

1. Access the terminal (**Ctrl+Alt+F2**) and examine **/tmp** logs.
2. **storage.log** reports space allocation failures.

   ![image](./figures/installation_bootloader_mount32.png)

Error messages suggest potential file system corruption (bad superblock) in efivarfs, requiring BIOS-level investigation.

### Possible Causes

Hardware-related BIOS abnormalities or corrupted variable storage in flash memory cause UEFI variable service failures, leading to mount errors during OS installation.

### Resolution

Re-flash the BIOS firmware.

---

---
url: /en/docs/common/faq/general/general_faq.md
---
# General Community Questions

## What is openEuler

The OpenAtom openEuler project, short for openEuler, is an open-source OS project incubated and operated by the OpenAtom Foundation.
It started as a simple server OS but has now blossomed into a full-blown digital infrastructure OS, supporting server, cloud, edge, and embedded deployments. This Linux distribution is compatible with multiple instruction set architectures and ideal for a wide range of operational technology applications, enabling OT-ICT convergence.

## What is the openEuler community like

Established officially on December 31, 2019, the openEuler community operates as a global hub for developers worldwide, aiming to foster an open, diverse, and architecture-inclusive software ecosystem tailored for wide-ranging digital infrastructure needs. openEuler collaborates closely with both upstream and downstream communities to ensure continuous tech improvement and timely release of new versions.

## What instruction set architectures does openEuler support

With active collaboration from leading chip vendors like Intel and AMD, openEuler supports multiple processor architectures, including **x86**, **Arm**, **SW64**, **RISC-V**, and **LoongArch**, with plans to expand to PowerPC.

openEuler is optimized for a wide range of CPU chips, such as Loongson 3 series, Zhaoxin KaiXian and KaiSheng, Intel Sierra Forest and Granite Rapids, and AMD EPYC Milan and Genoa. openEuler's compatibility extends beyond the CPU, encompassing NIC, RAID, Fibre Channel, GPU & AI, DPU, SSD, and security cards.

By offering a unified OS that can run on various devices, openEuler facilitates streamlined application development, allowing developers to target a wide range of hardware without significant code modification.

## How often does openEuler release a new version

openEuler releases two types of community versions: long-term support (LTS) and innovation versions.

LTS versions, like openEuler 20.03 LTS and openEuler 22.03 LTS, are released every two years and provide community support for four years. This includes two years of maintenance support and two years of extended support.

Innovation versions are released every six months, with each receiving community support for six months.

Prior to the end of any version's lifecycle, users will receive notifications from our [mailing lists](https://www.openeuler.org/en/community/mailing-list/) three months in advance.

## What are openEuler's special interest groups all about and How can I join one

The openEuler community is home to 100+ SIGs, each dedicated to a specific project or topic. These groups drive innovation in areas like toolchains, architectures, desktop environments, universal middleware, cloud-native infrastructure, and more! Our SIGs are hot on the heels of trends like AI, embedded systems, RISC-V, security, and compliance. They manage repositories, contribute to code, and even help shape community governance & operations.

You can find the full list of openEuler SIGs and their descriptions [sig list](https://www.openeuler.org/en/sig/sig-list/).

* Interested in joining an existing SIG? Send an email to the group's email address or contact the maintainers directly.

* Can I start my own SIG? Absolutely! We have a simple and easy process for [setting up a new SIG](https://www.openeuler.org/en/sig/sig-guidance/).

## How can I contribute to openEuler

Whether you're a coding whiz or an enthusiastic non-coder, there's a place for you in our community. Here's how to get started:

1. Sign the [CLA](https://clasign.osinfra.cn/sign/gitee_openeuler-1611298811283968340) as an individual, employee or corporation.
2. Head over to our [SIG List](https://www.openeuler.org/en/sig/sig-list/) to see ongoing projects and discussions. Join an existing SIG or [start a new one](https://www.openeuler.org/en/sig/sig-guidance/).
3. Submit/address issues, contribute code/packages/ projects, and participate in non-code contributions.
   * Submit/Address issues on the [QuickIssue](https://quickissue.openeuler.org/en/issues/) page where you can sign in with your Gitee, GitHub, email, or other account.
   * Contribute code to our **source code repository** on [Gitee](https://gitee.com/openeuler) or our mirrored repository on [GitHub](https://github.com/openeuler-mirror). Rest assured, we review PRs regularly.
   * Contribute packages/projects to our **software package repository** on [Gitee](https://atomgit.com/src-openeuler) or visit our website's [Contribute Software Package](https://software-pkg.openeuler.org/en/package) page.
4. Join in our community activities. We host a wide range of activities, including meetings, summits, live streams, and meetups.

Every contribution, big or small, is valued! Check out our [contribution guide](https://www.openeuler.org/en/community/contribution/detail.html) to learn more.

## How can I stay informed about openEuler and chat with fellow users

Here's how to stay informed about our developments and chat with fellow users:

* Visit our official website for usage guides and white papers.
* Explore our [MOOCs](https://www.openeuler.org/en/learn/mooc/) for in-depth tutorials.
* Follow us on social media ([LinkedIn](https://www.linkedin.com/company/openeuler/posts/?feedView=all), [X](https://x.com/openEuler), and [YouTube](https://www.youtube.com/@openeuler)) for the latest news on open source & OS industry events, partnerships, and technical solutions.
* Subscribe to our [mailing lists](https://www.openeuler.org/en/community/mailing-list/) to receive updates on SIG news.
* Engage in discussions and ask questions on the [openEuler Forum](https://forum.openeuler.org/) or join the [r/openEuler](https://www.reddit.com/r/openEuler/) subreddit on Reddit for real-time communication. While the openEuler Forum's official English version is under construction, feel free to post in English on the existing forum and connect with other users!

## Hmm, openEuler... Who's using it

openEuler isn't just open-source — it's powering real innovation from semiconductors to a wide range of industries like operating systems, Internet, finance, carrier, electric power, manufacturing, energy, education, transportation, healthcare, and other fields. Companies tailor openEuler to their needs, creating commercial and enterprise distributions for both internal and external usage, and some of these companies have implemented large-scale deployments of these distributions.

We're all about making the future brighter and more open-source! Check out our [success stories](https://www.openeuler.org/en/showcase/).

## What does "noise" in the openEuler refer to

OS noise includes non-application computing tasks executed during service running, such as:

* System/User-mode daemon processes
* Interrupt processing
* Processes in user mode or kernel
* Memory management and scheduling overhead
* Non-computing tasks in service applications (e.g., monitoring logs and thread communication)
* Resource competition (e.g., cache misses and page faults)

## Where can I find common repositories for openEuler

You can find sorted and classified repositories for various openEuler versions on the [openEuler Forum](https://forum.openeuler.org/t/topic/768).

---

---
url: /en/docs/common/faq/general/project_intro_faq.md
---
# General Feature Questions

## What can I implement using openEuler WSL

You can implement the following using openEuler WSL:

* Deploy and use an openEuler LTS version on Windows.
* Create a smooth cross-platform development experience leveraging Visual Studio Code and openEuler WSL.
* Build a Kubernetes cluster in openEuler WSL.
* Use openEuler command-line programs or scripts to process files in Windows or WSL.

## What does the hmdfs of openEuler do

hmdfs stands for HarmonyOS Distributed File System. It is a soft bus-based distributed file system ported from the OpenHarmony community. hmdfs provides a globally consistent access view for each device dynamically connected to a network via the distributed soft bus (DSoftBus). It allows you to implement high-performance read and write operations on files using basic file system APIs, achieving low latency.

## What is the SysCare of openEuler

SysCare is a system-level hotfix software that provides security patches and hotfixes for OSs. It can fix system errors without requiring host restarts. SysCare combines kernel-mode and user-mode hot patching to manage system repairs, saving time for users to focus on other aspects of their business. In the future, live OS upgrades will be provided to enhance O\&M efficiency.

## What is A-Ops

A-Ops is an OS-oriented O\&M platform that provides intelligent O\&M solutions covering data collection, health check, fault diagnosis, and fault rectification. The A-Ops project includes the following sub-projects: Gala (fault detection), X-diagnosis (fault locating), and Apollo (vulnerability rectification).

## What capabilities does secGear provide

secGear provides:

* Architecture compatibility: It masks differences between various SDK APIs by sharing the same set of source code across multiple architectures.
* Easy development: The development tools and general-purpose security components allow users to focus on services, significantly improving development efficiency.
* High performance: The switchless feature improves interaction performance between the rich execution environment (REE) and trusted execution environment (TEE) by more than 10-fold in typical scenarios, such as frequent REE-TEE interactions and big data interaction.

## What security technologies are used in AI for OS

Vulnerability discovery: Automatic vulnerability discovery is crucial to OS security. It identifies defects using code analysis, fuzz testing, or both. Traditional fuzz testing tools are often both random and blind when it comes to generating and selecting seeds, mutations, testing, and feedback. In addition, code analysis relies on defect pattern libraries, which need to manually be built by experts. AI improves this by detecting patterns in defect code datasets to enhance the precision and efficiency of vulnerability identification.

Intrusion detection: Modern security threats, such as Advanced Persistent Threats (APT), are sophisticated and persistent. Traditional security defenses often fail against unknown threats. AI enhances security by deeply analyzing data, extracting key features from high-dimensional datasets, and identifying system abnormalities effectively. This improves the accuracy and timeliness of attack blocking methods, such as in abnormal traffic and side-channel attack detection.

## What are the advantages of the multi-level scheduling framework provided by openEuler

openEuler's multi-level scheduling framework allows you to choose the most suitable scheduling model for your needs and provides the following advantages:

* Higher flexibility and portability compared to traditional process/thread scheduling models.
* Faster model switching and scheduling thanks to the use of lightweight scheduling models.

## How does openEuler ensure security

openEuler ensures security by providing the following comprehensive security features:

* Authenticity protection
* Integrity protection
* Confidentiality protection

## What security isolation technologies does openEuler provide for the industrial sector

* Service isolation: Isolates potentially vulnerable services from known sources to minimize the impact of attacks on other system components.
* Code restriction: Limits code from untrusted sources to reduce potential harm to other system components.

---

---
url: /en/docs/common/faq/community_tools/isocut_faqs.md
---
# isocut FAQ

## 1. Default RPM Package List Causes System Installation Failure

### Context

When using isocut to trim ISO images, users specify required software packages via the configuration file **/etc/isocut/rpmlist**.

Since different OS versions may have reduced package sets, the default configuration only includes the kernel package to ensure successful ISO trimming. This guarantees the default configuration always produces a valid ISO image.

### Symptom

The trimmed ISO image created with default settings may be successfully generated but fail during system installation.

The installation reports missing packages as shown in the error screenshot:

![](./figures/lack_pack.png)

### Possible Causes

The default RPM package list lacks essential packages required for system installation. The specific missing packages vary across OS versions, as shown in the error message during installation.

### Solution

1. Add missing packages:

   1. Identify required RPM packages from the installation error message.
   2. Append these packages to **/etc/isocut/rpmlist**.
   3. Rebuild the ISO image.

   Example modified **rpmlist** configuration based on the reported error:

   ```shell
   $ cat /etc/isocut/rpmlist
   kernel.aarch64
   lvm2.aarch64
   chrony.aarch64
   authselect.aarch64
   shim.aarch64
   efibootmgr.aarch64
   grub2-efi-aa64.aarch64
   dosfstools.aarch64
   ```

---

---
url: /zh/docs/common/faq/community_tools/isocut_faqs.md
---
# isocut常见问题与解决方法

## **问题1:默认 rpm 包列表安装系统失败**

### 背景描述

用户使用 isocut 裁剪镜像时通过配置文件 /etc/isocut/rpmlist 指定需要安装的软件包。

由于不同版本会有软件包减少，可能导致裁剪镜像时出现缺包等问题。
因此 /etc/isocut/rpmlist 中默认只包含 kernel 软件包。
保证默认配置裁剪镜像必定成功。

### 问题描述

使用默认配置裁剪出来的 iso 镜像，能够裁剪成功，但是安装可能失败。

安装报错缺包，报错截图如下：

![](./figures/lack_pack.png)

### 原因分析

使用默认配置的 RPM 软件包列表，裁剪的 iso 镜像在安装时缺少必要的 RPM 包。
缺少的包如报错的图示，并且在不同版本中，缺少的 RPM 包也可能是不同的，以安装时实际报错为准。

### 解决方案

1. 增加缺少的包

   1. 根据报错的提示整理缺少的 RPM 包列表
   2. 将上述 RPM 包列表添加到配置文件 /etc/isocut/rpmlist 中。
   3. 再次裁剪安装 iso 镜像

   以问题描述中的缺包情况为例，修改 rpmlist 配置文件如下：

   ```shell
   $ cat /etc/isocut/rpmlist
   kernel.aarch64
   lvm2.aarch64
   chrony.aarch64
   authselect.aarch64
   shim.aarch64
   efibootmgr.aarch64
   grub2-efi-aa64.aarch64
   dosfstools.aarch64
   ```

---

---
url: /en/docs/common/faq/cloud/isula_build_faqs.md
---
# isual-build FAQ

## 1. isula-build Image Pull Error: Connection Refused

When pulling an image, isula-build encounters the error: `pinging container registry xx: get xx: dial tcp host:repo: connect: connection refused`.

This occurs because the image is sourced from an untrusted registry.

To resolve this, edit the isula-build registry configuration file located at **/etc/isula-build/registries.toml**. Add the untrusted registry to the `[registries.insecure]` section and restart isula-build.

---

---
url: /zh/docs/common/faq/cloud/isula_build_faqs.md
---
# isual-build常见问题与解决方法

## **问题1：isula-build拉取镜像报错：pinging container registry xx: get xx: dial tcp host:repo: connect: connection refused**

原因：拉取的镜像来源于非授信仓库。

解决方法：修改isula-build镜像仓库的配置文件/etc/isula-build/registries.toml，将该非授信仓库加入\[registries.insecure]，重启isula-build。

---

---
url: /en/docs/common/faq/cloud/isula_faqs.md
---
# iSula FAQ

## 1. Changing iSulad Default Runtime to `lxc` Causes Container Startup Error: Failed to Initialize Engine or Runtime

**Cause**: iSulad uses `runc` as its default runtime. Switching to `lxc` without the required dependencies causes this issue.

**Solution**: To set `lxc` as the default runtime, install the `lcr` and `lxc` packages. Then, either configure the `runtime` field in the iSulad configuration file to `lcr` or use the `--runtime lcr` flag when launching containers. Avoid uninstalling `lcr` or `lxc` after starting containers, as this may leave behind residual resources during container deletion.

## 2. Error When Using iSulad CRI V1 Interface: rpc error: code = Unimplemented desc =

**Cause**: iSulad supports both CRI V1alpha2 and CRI V1 interfaces, with CRI V1alpha2 enabled by default. Using CRI V1 requires explicit configuration.

**Solution**: Enable the CRI V1 interface by modifying the iSulad configuration file at **/etc/isulad/daemon.json**.

```json
{
    "enable-cri-v1": true,
}
```

When compiling iSulad from source, include the `cmake` option `-D ENABLE_CRI_API_V1=ON` to enable CRI V1 support.

---

---
url: /zh/docs/common/faq/caselibrary/isulad.md
---
# isulad使用grpc与rest直接调用服务方法

## isulad使用grpc直接调用服务方法

isula与isulad之间的通信默认使用grpc，且k8s场景CRI接口调用只能使用grpc。

若用户想在第三方组件中直接通过gRPC连接向iSulad请求服务，可参照<https://atomgit.com/openeuler/iSulad/tree/master/src/api/services>不同功能模块目录下的proto文件获取gRPC请求格式。

调试grpc调用可使用grpcurl工具，grpcurl 是 Go 语言开源社区开发的工具，需要手工安装：

```sh
$ go get github.com/fullstorydev/grpcurl
$ go install github.com/fullstorydev/grpcurl/cmd/grpcurl
```

调试grpc调用时可参照以下命令：

```sh
grpcurl -plaintext -H 'tls_mode:0' -proto=api.proto -import-path=../ -unix=true /var/run/isulad.sock runtime.v1alpha2.ImageService.ListImages
```

## isulad使用rest直接调用服务方法

若isula与isulad之间想要使用更轻量级的rest进行通信，需要在[iSulad](https://atomgit.com/openeuler/iSulad) 仓库获取iSulad源码进行源码编译，源码编译的教程可参照：<https://atomgit.com/openeuler/iSulad/blob/master/docs/build_docs/README_zh.md>

在源码编译时使用如下编译选项可更换isula与isulad的交互方式为rest：

```sh
cmake -DENABLE_GRPC=OFF …/
```

调试rest调用可直接使用curl工具，调试的具体方式可参照以下命令：

```sh
curl -d "{}" --unix-socket /var/run/isulad.sock http://localhost/ContainerService/Version
```

---

---
url: /zh/docs/common/faq/cloud/isula_faqs.md
---
# iSulad常见问题与解决方法

## **问题1：修改`iSulad`默认运行时为`lxc`，启动容器报错：Failed to initialize engine or runtime**

原因：`iSulad`默认运行时为`runc`，设置默认运行时为`lxc`时缺少依赖。

解决方法：若需修改`iSulad`默认运行时为`lxc`，需要安装`lcr`、`lxc`软件包依赖，且配置`iSulad`配置文件中`runtime`为`lcr`
或者启动容器时指定`--runtime lcr`。启动容器后不应该随意卸载`lcr`、`lxc`软件包，否则可能会导致删除容器时的资源残留。

## **问题2：使用`iSulad` `CRI V1`接口，报错：rpc error: code = Unimplemented desc =**

原因：`iSulad`同时支持`CRI V1alpha2`和`CRI V1`接口，默认使用`CRI V1alpha2`，若使用`CRI V1`，需要开启相应的配置。

解决方法：在`iSulad`配置文件`/etc/isulad/daemon.json`中开启`CRI V1`的配置。

```json
{
    "enable-cri-v1": true,
}
```

若使用源码编译`iSulad`，还需在编译时增加`cmake`编译选项`-D ENABLE_CRI_API_V1=ON`。

---

---
url: /en/docs/common/faq/caselibrary/sssnic.md
---
# Kernel Hot patch Creation Issue: dmesg Reporting Missing sssnic Module

## Context

During kernel hot patch creation for openEuler LTS SP3, an unexpected dependency on the unmodified sssnic driver module caused activation failures. System logs indicate the livepatch system cannot find the required sssdk module, though the patch never intentionally modified this network driver module.

### Version Information

* Kernel: 5.10.0-182.0.0.95.oe2203sp3.aarch64
* kpatch: 0.9.5-7.oe2203sp3.aarch64

### Symptom

The hot patch creation command `./make_hotpatch -d .new -i procversion` executed successfully but produced a non-functional patch. Diagnostic logs showed the system incorrectly marked the sssnic driver as modified, creating an unnecessary module dependency. Since this network driver is not loaded by default, the hot patch fails to activate with a missing module error.

```shell
[166439.721426] klp_procversion: tainting kernel with TAINT_LIVEPATCH
[166439.760137] livepatch: module 'sssdk' not loaded
```

## Possible Causes

1. The hot patch tool incorrectly identified changes in two driver components (`sss_tool_nic_func.c` and `sss_tool_sdk.c`) during ELF section comparison, despite no intentional modifications.

2. Investigation revealed the build system always recompiles the sssnic module during incremental builds. kpatch detects binary differences between these compiled objects, erroneously including them in the hot patch.

## Solution

**Option 1:**
When creating hot patches not related to the sssnic module, you can exclude this module during difference detection by modifying the **/usr/libexec/kpatch/kpatch-cc** file. Add the sssnic source code path to the ignore list, then rebuild the hot patch to eliminate sssnic module dependencies and ensure proper hot patch functionality.

```shell
diff --git a/kpatch-build/kpatch-cc b/kpatch-build/kpatch-cc
index 80d310c...688d92b 100755
--- a/kpatch-build/kpatch-cc
+++ b/kpatch-build/kpatch-cc
@@ -49,7 +49,8 @@ if [[ "$TOOLCHAINCMD" =~ ^(.*-)?gcc$ ||
                arch/powerpc/kernel/prom_init.o|\
                lib/*|\
                .*.o|\
-               */.lib_exports.o)
+               */.lib_exports.o|\
+               drivers/net/ethernet/3snic/sssnic/*)
                    break
                    ;;
                *.o)
```

**Option 2:**
When compiler optimizations cause unmodified functions to be mistakenly flagged as changed, use the `KPATCH_IGNORE_FUNCTION` macro to exclude these functions from hot patch generation.

The build log reveals two modified functions in the sssnic module:

```txt
Testing patch file(s)
Reading special section data
Building original source
Building patched source
Extracting new and modified ELF sections
sss_tool_nic_func.o: changed function: sss_tool_ioctl
sss_tool_sdk.o: changed function: sss_tool_get_hw_drv_version
version.o: changed function: version_proc_show
```

Add the following statements after the function declarations in their respective files (note: place `KPATCH_IGNORE_FUNCTION` after function declarations to avoid symbol lookup errors):

```c
#include "/usr/share/kpatch/patch/kpatch-macros.h"
KPATCH_IGNORE_FUNCTION(sss_tool_ioctl);
```

```c
#include "/usr/share/kpatch/patch/kpatch-macros.h"
KPATCH_IGNORE_FUNCTION(sss_tool_get_hw_drv_version);
```

Re-run the command `./make_hotpatch -d .new -i procversion` to rebuild the hot patch. This will resolve errors related to the unloaded sssnic module.

---

---
url: /en/docs/common/faq/cloud/kmesh_faqs.md
---
# Kmesh FAQ

## 1. Kmesh Service Exits with an Error When Started in Cluster Mode without Control Plane IP Address Configuration

![](./figures/not_set_cluster_ip.png)

Cause: When operating in cluster mode, Kmesh requires communication with the control plane to fetch configuration details. Without the correct control plane IP address, the service cannot proceed and exits with an error.

Solution: Follow the cluster mode setup instructions in the Kmesh installation and deployment guide to properly configure the control plane IP address.

## 2. Kmesh Service Displays "Get Kube Config Error!" during Startup

![](./figures/get_kubeconfig_error.png)

Cause: In cluster mode, Kmesh attempts to retrieve the control plane IP address from the k8s configuration. If the kubeconfig file path is not set in the environment, the service cannot access the kubeconfig and throws this error. (Note: This issue does not occur if the control plane IP address is manually specified in the Kmesh configuration file.)

Solution: Set up kubeconfig using the following commands:

```shell
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

---

---
url: /zh/docs/common/faq/cloud/kmesh_faqs.md
---
# Kmesh常见问题与解决方法

## **问题1：在使用集群启动模式时，若没有配置控制面程序ip信息，Kmesh服务启动后会报错退出**

![](./figures/not_set_cluster_ip.png)

原因：集群启动模式下，Kmesh服务需要跟控制面程序通信，然后从控制面获取配置信息，因此需要设置正确的控制面程序ip信息。

解决方法：参考Kmesh安装与部署章节中集群启动模式，设置正确的控制面程序ip信息。

## **问题2：Kmesh服务在启动时,提示"get kube config error!"**

![](./figures/get_kubeconfig_error.png)

原因：集群启动模式下，Kmesh服务会根据k8s的配置，自动获取控制面程序ip信息，若环境中没有配置k8s的kubeconfig路径，会导致获取kubeconfig失败，然后提示上述信息。（若已经手动修改Kmesh的配置文件，正确配置控制面程序ip信息，该问题可忽略）

解决方法：按如下方式配置kubeconfig：

```shell
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

---

---
url: /zh/docs/common/faq/cloud/kuasar_confidential_containers_question_answer.md
---
# Kuasar机密容器常见问题

## 调试方法

### 如何查看日志信息

```sh
$ journalctl -u isulad
```

### 如何加入调试工具到机密虚机

以cat命令为例：

```sh
$ls /var/lib/kuasar/cc-rootfs.img
/var/lib/kuasar/cc-rootfs.img
$mkdir cc-rootfs
$mount /var/lib/kuasar/cc-rootfs.img ./cc-rootfs
$cp /usr/bin/cat ./cc-rootfs/usr/bin/cat
$umount ./cc-rootfs
```

值得注意的是，cat的依赖比较简单，cc-rootfs.img中都有，但有的工具，比如curl、wget，依赖比较复杂，还需要拷贝依赖的so。
比如当宿主机环境的操作系统版本与kuasar版本一致时，可以用ldd /usr/bin/curl 查询依赖，并将所有so拷贝到cc-rootfs.img相应位置。

### 如何进入机密虚机

1.修改/var/lib/kuasar/cc-config.toml中的kernel\_params参数:  task.debug  task.debug\_shell=/xx/x
这里/xx/xx填rootfs镜像内sh命令所在地址，比如/bin/sh。

2.改好后，重启kuasar服务，拉新的sandbox，先不启动容器，执行

```sh
$socat - VSOCK-CONNECT:{vsock-id}:1025
```

此处vsock id为当前启动的沙箱命令（ps 查询qemu进程能看到）中，形如id=vsock-562081427,这个数字就是vsock id，每个沙箱都不一样。

## 常见问题

### 机密虚机内无法访问任何外部网络

确认方法：进入机密虚机，使用ping 或者 curl访问外部网络，发现无法ping通。

解决方案：需要排查json文件中"cri.sandbox.network.setup.v2"需要配置为"true"：

```json
cat pod.json
{
    "annotations": {
        "cri.sandbox.network.setup.v2": "true"
    },
}
```

### 机密虚机内拉加密镜像报错"failed to pull manifest error sending request for url ......"

确认方法：查看日志信息，报错"failed to pull manifest error sending request for url ......"，进入机密虚机，使用curl访问外部https网络地址，提示https不可信，缺少CA证书。但是使用curl -k命令可以访问成功。

解决方案：参考[《isulad+kuasar机密容器部署指南》](https://docs.openeuler.openatom.cn/zh/docs/24.03_LTS_SP2/cloud/container_runtime/kuasar/isulad_kuasar_confidential_containers_deployment_guide.html)配置机密容器参数章节，将镜像服务器的CA证书拷贝到cc-rootfs.img中。

### 机密虚机内拉加密镜像报错"failed to pull manifest unknown variant `NOT_FOUND`......"

确认方法：查看日志信息，报错"failed to pull manifest unknown variant `NOT_FOUND`......"。

解决方案：看container.json文件，确认镜像的地址确实和仓库中一致，尤其注意是不是镜像名称拼写错了，这个报错是因为镜像无法在镜像服务器中找到。

### 机密虚机内拉加密镜像报错"failed to get decrypt key"

确认方法：查看日志信息，报错"failed to get decrypt key"

解决方法：这个报错表明镜像已经可以成功拉取，但是在解密过程中出现错误，需要进一步排查。

#### task.aa\_ser\_url 配置问题

确认方法：检查/var/lib/kuasar/cc-config.toml中的kernel\_params参数中task.aa\_ser\_url，如果ip地址错误，就会导致获取密钥失败。

#### 其他

逐步确认打包加密镜像的过程，确认加密镜像打包正确，并且存放了正确的密钥在远程证明密钥托管服务器。

---

---
url: /en/docs/common/faq/cloud/kubernetes_faqs.md
---
# Kubernetes FAQ

## 1. Kubernetes + Docker Deployment Failure

Reason: Kubernetes dropped support for Kubernetes + Docker cluster deployments starting from version 1.21.

Solution: Use cri-dockerd + Docker for cluster deployment, or consider alternatives like containerd or iSulad.

## 2. Unable to Install Kubernetes RPM Packages via yum on openEuler

Reason: Installing Kubernetes-related RPM packages requires proper configuration of the EPOL repository in yum.

Solution: Follow the repository configuration guide provided in [this link](https://forum.openeuler.org/t/topic/768) to set up the EPOL repository in your environment.

---

---
url: /zh/docs/common/faq/cloud/kubernetes_faqs.md
---
# Kubernetes常见问题与解决方法

## **问题1：Kubernetes + docker为什么无法部署**

原因：Kubernetes自1.21版本开始不再支持Kubernetes + docker部署Kubernetes集群。

解决方法：改为使用cri-dockerd+docker部署集群，也可以使用containerd或者iSulad部署集群。

## **问题2：openEuler无法通过yum直接安装Kubernetes相关的rpm包**

原因：Kubernetes相关的rpm包需要配置yum的repo源有关EPOL的部分。

解决方法：[参考链接](https://forum.openeuler.org/t/topic/768)中repo源，重新配置环境中的EPOL源。

---

---
url: /en/docs/common/faq/caselibrary/crash.md
---
# kump FAQ

## 1. kdump Service Startup Failure

### Symptom

The `systemctl status kdump` command reports the service status as "failed."

### Possible Causes and Solutionss

1. The `crashkernel` parameter fails to reserve memory.

   The error log from `systemctl status kdump` includes:

   ```bash
   Aug 10 15:26:20 localhost.localdomain kdumpctl[772]: No memory reserved for crash kernel
   Aug 10 15:26:20 localhost.localdomain kdumpctl[772]: Starting kdump: [FAILED]
   ```

   The `crashkernel` parameter typically reserves memory in low memory (below 4 GB). Under heavy memory usage, this reservation may fail, preventing kdump from starting.

   **Solution**: Modify the boot parameter to `crashkernel=size,high`, enabling memory reservation from high memory.

### Possible Causes and Solutionss

1. The `crashkernel` parameter fails to reserve memory.

   The error log from `systemctl status kdump` includes:

   ```bash
   Aug 10 15:26:20 localhost.localdomain kdumpctl[772]: No memory reserved for crash kernel
   Aug 10 15:26:20 localhost.localdomain kdumpctl[772]: Starting kdump: [FAILED]
   ```

   The `crashkernel` parameter typically reserves memory in low memory (below 4 GB). Under heavy memory usage, this reservation may fail, preventing kdump from starting.

   **Solution**: Modify the boot parameter to `crashkernel=size,high`, enabling memory reservation from high memory.

2. Kernel configuration mismatch prevents `dracut` from creating `kdump.img`.

   `systemctl status kdump` shows errors similar to the follows:

   ```bash
   Aug 10 16:25:52 localhost.localdomain kdumpctl[3972]: dracut-install: ERROR: installing 'loop'
   Aug 10 16:25:52 localhost.localdomain kdumpctl[2225]: dracut: FAILED:  /usr/lib/dracut/dracut-install -D /var/tmp/dracut.a9swIC/initramfs -N mdio_gpi|usb_8d>
   Aug 10 16:25:52 localhost.localdomain dracut[2271]: FAILED:  /usr/lib/dracut/dracut-install -D /var/tmp/dracut.a9swIC/initramfs -N mdio_gpi|usb_8dev|et1011c>
   Aug 10 16:25:52 localhost.localdomain kdumpctl[2225]: dracut: installkernel failed in module squash
   Aug 10 16:25:52 localhost.localdomain dracut[2271]: installkernel failed in module squash
   Aug 10 16:25:53 localhost.localdomain kdumpctl[1541]: mkdumprd: failed to make kdump initrd
   Aug 10 16:25:53 localhost.localdomain kdumpctl[1541]: Starting kdump: [FAILED]
   ```

   This error occurs because `dracut` requires **squashfs.ko**, **loop.ko**, and **delay.ko**. If any of these modules are missing, `dracut` fails.

   This issue is unlikely in official openEuler LTS versions, as they include these .ko files. If you compiled the kernel, verify these configuration options:

   ```bash
   CONFIG_SQUASHFS=m

   CONFIG_BLK_DEV_LOOP=m

   CONFIG_OVERLAY_FS=m
   ```

   These options must be set to `m` to build the .ko files, not `y`.

3. KASLR is enabled and `/proc/sys/kernel/kptr_restrict`  is set to 2.

   `systemctl status kdump` returns these errors:

   ```bash
   Aug 10 14:55:04 localhost.localdomain kdumpctl[637422]: Can't find kernel text map area from kcore
   Aug 10 14:55:04 localhost.localdomain kdumpctl[637422]: Cannot load /boot/vmlinuz-4.18.0-147.5.2.1.h579.hugetlb.eulerosv2r10.x86_64+
   Aug 10 14:55:04 localhost.localdomain kdumpctl[637001]: kexec: failed to load kdump kernel
   Aug 10 14:55:04 localhost.localdomain kdumpctl[637001]: Starting kdump: [FAILED]
   ```

   This typically occurs on x86 systems, as Address Space Layout Randomization (KASLR) is not yet enabled on AArch64.

   With KASLR enabled, kdump cannot retrieve kernel layout information from **/proc/kcore**. Additionally, if **/proc/sys/kernel/kptr\_restrict** is set to 2, information in **/proc/kallsyms** is hidden. These combined factors prevent kdump from starting.

   **Solution**: Set **/proc/sys/kernel/kptr\_restrict** to 1, which allows only the root user to view **/proc/kallsyms**. Then, start kdump as **root**.

## 2. kdump Service Active, But vmcore Generation Fails

### Symptom

`systemctl status kdump` indicates the service is active, but no vmcore file is generated after a system crash and reboot.

### Possible Causes and Solutions

1. Insufficient memory is reserved for `crashkernel`, resulting in out-of-memory (OOM).

   The crash kernel requires sufficient memory to launch. An OOM error likely occurs because a kernel object consumes excessive memory. Official openEuler versions generally avoid this issue, but self-compiled kernels require careful attention. Check serial port logs to confirm if an OOM error occurred.

   **Solution**: Increase the value of the `crashkernel` boot parameter. If memory reservation fails after the increase, use `crashkernel=size,high` to reserve memory.

2. `SECTIONS_SIZE_BITS` is incompatible.

   The `makedumpfile` tool (invoked by the kdump service) completes the vmcore dump. The `SECTIONS_SIZE_BITS` definition within `makedumpfile` must match the kernel. `SECTIONS_SIZE_BITS` is defined in the kernel file **arch/arm64/include/asm/sparsemem.h**. Official openEuler AArch64 versions define it as 27, and the `SECTIONS_SIZE_BITS` in kdump is modified to 27 to match. However, the community source code sets `SECTIONS_SIZE_BITS` to 30, which is incompatible with kdump and causes `makedumpfile` to fail vmcore generation.

   **Solution**: Modify `SECTIONS_SIZE_BITS` in the kernel source code **arch/arm64/include/asm/sparsemem.h** to 27.

3. Out-of-band hardware watchdog resets, interrupting the vmcore dump.

   kdump vmcore dumps can be time-consuming, depending on system memory usage and drive write speeds. An out-of-band hardware watchdog might interrupt the vmcore dump process.

   **Solution**: Disable the out-of-band hardware watchdog or reset its timeout value in kdump.

4. Drive reporting is abnormal.

   Improper drive reporting can prevent vmcore from being saved correctly. Check serial port logs to confirm these issues.

## 3. crash Tool Fails to Parse the Generated vmcore

### Symptom

Parsing the generated vmcore with `crash vmcore vmlinux` results in an error, preventing normal parsing.

### Possible Causes and Solutions

1. vmcore and vmlinux versions do not match.

   `crash` requires a vmlinux file compiled from the kernel source code to parse a vmcore. The vmlinux version must match the system version that dumped the vmcore for successful parsing.

   **Solution**: Use a vmlinux file with the same version as the vmcore.

2. Missing `strings` Command

   `crash` relies on the `strings` command for vmcore parsing. Its absence causes parsing failures.

   **Solution**: The `binutils` package provides the `strings` command. Install binutils or manually copy the `strings` command and its dependencies.

3. vmcore is corrupted.

   Check **kdump\_status.log** to determine if the kdump vmcore dump process completed successfully.

   **Solution**: Trigger a system panic again to generate a new vmcore.

---

---
url: /zh/docs/common/faq/caselibrary/crash.md
---
# kump 常见问题

## 场景一：kdump服务启动失败

### 问题现象

systemctl status kdump查看状态为failed

### 原因分析及解决方案

1. crashkernel启动参数预留内存失败

   systemctl status kdump有以下报错：

   ```bash
   Aug 10 15:26:20 localhost.localdomain kdumpctl[772]: No memory reserved for crash kernel
   Aug 10 15:26:20 localhost.localdomain kdumpctl[772]: Starting kdump: [FAILED]
   ```

   crashkernel默认在低端内存（4G）预留，系统内存紧张时，可能会导致预留内存失败，进而导致kdump服务启动失败。

   **解决方案**：crashkernel=size,high，允许内核从高端内存预留。

2. 内核CONFIG不匹配导致dracut裁剪kdump.img失败

   systemctl status kdump有以下报错：

   ```bash
   Aug 10 16:25:52 localhost.localdomain kdumpctl[3972]: dracut-install: ERROR: installing 'loop'
   Aug 10 16:25:52 localhost.localdomain kdumpctl[2225]: dracut: FAILED:  /usr/lib/dracut/dracut-install -D /var/tmp/dracut.a9swIC/initramfs -N mdio_gpi|usb_8d>
   Aug 10 16:25:52 localhost.localdomain dracut[2271]: FAILED:  /usr/lib/dracut/dracut-install -D /var/tmp/dracut.a9swIC/initramfs -N mdio_gpi|usb_8dev|et1011c>
   Aug 10 16:25:52 localhost.localdomain kdumpctl[2225]: dracut: installkernel failed in module squash
   Aug 10 16:25:52 localhost.localdomain dracut[2271]: installkernel failed in module squash
   Aug 10 16:25:53 localhost.localdomain kdumpctl[1541]: mkdumprd: failed to make kdump initrd
   Aug 10 16:25:53 localhost.localdomain kdumpctl[1541]: Starting kdump: [FAILED]
   ```

   报错的原因是dracut会依赖squashfs.ko、loop.ko和delay.ko，如果缺失，就会导致dracut失败。

   这个问题一般不会在正式的openEuler LTS版本上出现，因为正式版本都包含这三个ko。如果是自行编译的内核，注意以下CONFIG的取值：

   ```bash
   CONFIG_SQUASHFS=m

   CONFIG_BLK_DEV_LOOP=m

   CONFIG_OVERLAY_FS=m
   ```

   另外，设置为y也是不行的，必须设置为m，编译出ko才可以。

3. KASLR生效且/proc/sys/kernel/kptr\_restrict设置为2

   systemctl status kdump有以下报错：

   ```bash
   Aug 10 14:55:04 localhost.localdomain kdumpctl[637422]: Can't find kernel text map area from kcore
   Aug 10 14:55:04 localhost.localdomain kdumpctl[637422]: Cannot load /boot/vmlinuz-4.18.0-147.5.2.1.h579.hugetlb.eulerosv2r10.x86_64+
   Aug 10 14:55:04 localhost.localdomain kdumpctl[637001]: kexec: failed to load kdump kernel
   Aug 10 14:55:04 localhost.localdomain kdumpctl[637001]: Starting kdump: [FAILED]
   ```

   一般出现在x86上，目前arm64的KASLR未生效。

   在KASLR生效时，kdump无法从/proc/kcore中获取内核的布局信息；如果同时设置/proc/sys/kernel/kptr\_restrict为2，/proc/kallsyms中的信息也会被隐藏。两者同时发生导致kdump启动失败。

   **解决方案**：设置/proc/sys/kernel/kptr\_restrict为1，仅允许root用户查看/proc/kallsyms，以root用户启动kdump可以成功。

## 场景二：kdump服务正常但无法生成vmcore

### 问题现象

systemctl status kdump查看状态为active，但是系统挂掉重启之后，没有生成vmcore。

### 原因分析及解决方案

1. crashkernel预留内存太小，导致crash内核oom

   crash内核启动需要足够的内存，oom大概率是由于ko耗内存较多，正式的openEuler版本一般没有类似问题，自行编译的内核需要特别注意这种可能。另外，可以查看串口日志确认是否是发生了oom的问题。

   **解决方案**：crashkernel启动参数调大，如果调大后预留内存失败，可以使用crashkernel=size,high的方式预留内存。

2. SECTIONS\_SIZE\_BITS未适配

   vmcore的转储由makedumpfile工具（由kdump服务调用）来完成，makedumpfile中的SECTIONS\_SIZE\_BITS定义需要和内核保持一致。SECTIONS\_SIZE\_BITS定义在内核文件arch/arm64/include/asm/sparsemem.h中，正式的openEuler arm64版本定义为27，kdump中的SECTIONS\_SIZE\_BITS也适配修改为27。但是在社区源码中SECTIONS\_SIZE\_BITS的值为30，和kdump不匹配，就会导致makedumpfile生成vmcore失败。

   **解决方案**：内核源码arch/arm64/include/asm/sparsemem.h中的SECTIONS\_SIZE\_BITS修改为27

3. 带外硬件狗复位，导致转储vmcore的过程中断

   受系统使用内存大小以及落盘速度的影响，kdump转储vmcore耗时可能较长，如果存在带外硬件狗时，有可能中断vmcore的转储流程。

   **解决方案**：关闭带外硬件狗或者在kdump中重置带外硬件狗超时时间。

4. 磁盘上报有问题

   vmcore没有正常保存可能是因为保存vmcore的磁盘没有正常上报，可以通过串口日志来确认此类问题。

## 场景三：生成的vmcore，crash工具无法正常解析

### 问题现象

在生成vmcore之后，使用crash vmcore vmlinux解析时出错，无法正常解析

### 原因分析及解决方案

1. vmcore和vmlinux版本不匹配

   crash在解析vmcore时需要有内核源码编译出的vmlinux，同时，vmlinux的版本需要和转储vmcore的系统版本一致，crash才能正常解析。

   **解决方案**：使用和vmcore版本一致的vmlinux。

2. 环境缺少strings命令

   crash解析vmcore时需要依赖strings命令，缺少该命令会导致crash解析失败。

   **解决方案**：strings命令由binutils包提供，可以安装binutils包或者手动拷贝strings命令以及依赖库。

3. 原因三：vmcore有损坏

   可以查看kdump\_status.log获取kdump转储vmcore时的流程，是否完整执行结束。

   **解决方案**：重新触发系统panic生成vmcore。

---

---
url: /en/docs/common/faq/caselibrary/lvm.md
---
# LVM Label Corruption

## Symptom

The `pvs` utility fails to show physical volume details, and `blkid` cannot recognize the `LVM2_member` label.
Expected output:

![](./figures/lvm-1.png)

Error state:

![](./figures/lvm-2.png)

## Possible Causes

Analysis of the metadata region with `hexdump` confirms label corruption.

![](./figures/lvm-3.png)

![](./figures/lvm-4.png)

## Solution

1. Check the backup files in **/etc/lvm/backup** to find the PV UUID, then restore using `pvcreate`:

   ![](./figures/lvm-5.png)

   Command:

   ```shell
   pvcreate --uuid D8v9Qw-1EJw-cmRc-nY2y-AZbC-8eLd-5Cpi2s --restorefile /etc/lvm/backup/vg /dev/sdb
   ```

2. Restore the volume group configuration:

   ```shell
   vgcfgrestore --file /etc/lvm/backup/vg vg
   ```

3. Complete the recovery by activating the volume group:

   ```shell
   vgchange -ay vg
   ```

---

---
url: /zh/docs/common/faq/caselibrary/lvm.md
---
# LVM标签损坏

## 现象描述

pvs查询不到pv，blkid查询不到LVM2\_member标签。
正常情况：

![](./figures/lvm-1.png)

异常情况：

![](./figures/lvm-2.png)

## 原因分析

hexdump查看元数据区域，发现标签损坏。

![](./figures/lvm-3.png)

![](./figures/lvm-4.png)

## 解决方法

1. 在/etc/lvm/backup目录查询卷组备份信息，获取pv的uuid，使用pvcreate命令恢复。

   ![](./figures/lvm-5.png)

   命令如下:

   ```
   pvcreate --uuid D8v9Qw-1EJw-cmRc-nY2y-AZbC-8eLd-5Cpi2s --restorefile /etc/lvm/backup/vg /dev/sdb
   ```

2. pv复之后，再恢复卷组。

   ```
   vgcfgrestore --file /etc/lvm/backup/vg vg
   ```

3. 最后激活卷组

   ```
   vgchange -ay vg
   ```

---

---
url: /zh/docs/common/contribute/markdownlint_rules.md
---
# markdownlint 检查规则

本文介绍了 markdownlint v0.38.0 版本规则以及 openEuler Docs 仓的规则设置，参照依据 <https://github.com/DavidAnson/markdownlint/blob/v0.38.0/doc/Rules.md> 。对 markdownlint 规则有任何疑问或交流，请联系 ECHO[@ECHO](https://gitee.com/echo10111111)。

## markdownlint 介绍

markdownlint 是一款检查 Markdown 文件格式的工具，可以根据设置的规则对 Markdown 文件进行全面的检查。文档写作时可以借助 VSCode 等工具的 markdownlint 插件修复格式问题。

## openEuler docs 仓规则设置

* openEuler 规则采用如下方案：

  * MD003 (标题样式) 规则将参数 `style`设置为`atx`。
  * MD029（有序列表的前缀序号）规则将参数`style`设置为`ordered`。
  * 屏蔽 MD004（无序列表）这条规则。
  * 屏蔽 MD007（无序列表缩进）这条规则。
  * 屏蔽 MD009（行尾空格）这条规则。
  * 屏蔽 MD013（行的长度）这条规则。
  * 屏蔽 MD014（命令前使用$而不显示输出）这条规则。
  * 屏蔽 MD020（closed atx样式的标题内没有空格）这条规则。
  * 屏蔽 MD021（closed atx样式的标题内有多个空格）这条规则。
  * 屏蔽 MD024（不能有重复内容的标题）这条规则。
  * 屏蔽 MD025 （文档中有多个顶级标题）这条规则。
  * 屏蔽 MD027（块引用符号后的多个空格）这条规则。
  * 屏蔽 MD033（内联HTML）这条规则。
  * 屏蔽 MD036（使用强调标记代替标题）这条规则。
  * 屏蔽 MD046（代码块样式）这条规则。

* ruby 文件的书写方式如下：

  ```bash
  all
  rule 'MD003', :style => :atx
  rule 'MD029', :style => :ordered
  exclude_rule 'MD004'
  exclude_rule 'MD007'
  exclude_rule 'MD009'
  exclude_rule 'MD013'
  exclude_rule 'MD014'
  exclude_rule 'MD020'
  exclude_rule 'MD021'
  exclude_rule 'MD024'
  exclude_rule 'MD025'
  exclude_rule 'MD027'
  exclude_rule 'MD033'
  exclude_rule 'MD036'
  exclude_rule 'MD046'
  ```

## 规则介绍

### MD001 - 标题级别一次只能增加一个级别

* **错误示例**

  ```text
  # Header1

  ### Header3
  ```

* **正确示例**

  ```text
  # Header1

  ## Header2

  ### Header3

  #### Header4
  ```

### MD003 - 标题样式

* **参数**

  * `style`：指定文档标题的样式，有 `consistent`、`atx`、`atx_closed`、`setext`、`setext_with_atx`、`setext_with_atx_closed`六种，默认为 `consistent`。

* **错误示例**

  ```text
  # ATX style H1

  ## Closed ATX style H2 ##

  Setext style H1
  ===============
  ```

* **正确示例**

  ```text
  # ATX style H1

  ## ATX style H2
  ```

`setext_with_atx` 和 `setext_with_atx_closed` 设置允许在使用了 Setext 样式标题（只支持 1 级和 2 级标题）的文档中使用 3 级及以上的 ATX 样式标题：

* **正确示例**

  ```text
  Setext style H1
  ===============

  Setext style H2
  ---------------

  ### ATX style H3
  ```

> \[!NOTE]说明
> 配置的标题样式可以是一个具体的样式（atx、atx\_closed、setext、setext\_with\_atx、setext\_with\_atx\_closed），也可以通过 consistent 要求所有标题样式与第一个标题样式匹配。
> 水平分隔线直接放在一行文本下方会将该行文本变成二级 Setext 样式标题，从而可能触发此规则。

### MD004 - 无序列表样式

**本仓已屏蔽这条规则。**

* **参数**

  * `style`：指定无序列表的样式，有 `consistent(定义时符号前后保持一致)`、`asterisk(用星号定义)`、`plus(用加号定义)`、`dash(用减号定义)`、`sublist(定义多重列表的时候用不同的符号定义)`五种，默认为 `consistent`。

* **错误示例**

  ```text
  * Item 1
  + Item 2
  ```

* **正确示例**

  ```text
  * Item 1
  * Item 2
  ```

### MD005 - 同一级别的列表项缩进不一致

* **错误示例**

  ```text
  * Item1
      * nested item 1
      * nested item 2
     * A misaligned item
  ```

* **正确示例**

  ```text
  * Item1
      * nested item 1
      * nested item 2
      * nested item 3
  ```

### MD007 - 无序列表缩进

**本仓已屏蔽这条规则。**

* **参数**

  * `ident`：指定无序列表嵌套时缩进的空格数，默认值是2。
  * `start_indent`：第一级缩进空格数，当 start\_indented 启用时，默认值是2。
  * `start_indented`：是否缩进列表的第一级，默认值是false。

* **错误示例**

  ```text
  * List item
    * Nested list item indented by 4 spaces
  ```

* **正确示例**

  ```text
  * List item
      * Nested list item indented by 2 spaces
  ```

### MD009 - 行尾空格

**本仓已屏蔽这条规则。**

* **参数**
  * `br_spaces`：指定在行尾可以添加的空格的数目，默认值为0，空格数目建议大于等于2，如果小于2，会默认为0。
  * `list_item_empty_lines`：允许列表项中的空行有空格，默认值为false。
  * `strict`：包含不必要的换行，默认值为false。

* **正确示例**

  ```text
  Text text text
  text[2 spaces]
  ```

  ```text
  - list item text
  [2 spaces]
  list item text  
  ```

### MD010 - 不能使用tab键缩进，要使用空格

* **参数**

  * `code_blocks`：指定本条规则在代码块里是否 (true or false) 生效,默认是 true。
  * `ignore_code_languages`：要忽略的围栏代码语言列表，默认是 \[]。
  * `spaces_per_tab`：每个硬制表符对应的空格数，默认是 1。

* **错误示例**

  ```text
  Some text

      * hard tab character used to indent the list item
  ```

* **正确示例**

  ```text
  Some text

      * Spaces used to indent the list item instead
  ```

### MD011 - 反向链接语法

* **错误示例**

  ```text
  (Incorrect link syntax)[http://www.example.com]
  ```

* **正确示例**

  ```text
  [Correct link syntax](http://www.example.com)
  ```

### MD012 - 多个连续的空行

* **参数**

  * `maximum`：指定文档中可以连续的最多的空行数，默认值是1。

* **错误示例**

  ```text
  Some text here


  Some more text here
  ```

* **正确示例**

  ```text
  Some text here

  Some more text here
  ```

> \[!NOTE]说明
> 如果代码块内有多个连续的空行，将不会触发此规则。
> `maximum`参数可用于配置允许的最大连续空行数。

### MD013 - 行的长度

**本仓已屏蔽这条规则。**

* **参数**

  * `line_length`：指定行的最大长度，默认是80。
  * `heading_line_length`：指定标题行的最大的长度，默认是80。
  * `code_block_line_length`：代码块的最大字符数，默认是80。
  * `code_blocks`：指定规则是否(true or false)对代码块生效，默认是true。
  * `tables`：指定规则是否(true or false)对表格生效，默认是true。
  * `headings`：指定规则是否(true or false)对标题生效，默认是true。
  * `stern`：严格长度检查，默认是false。
  * `strict`：严格长度检查，默认是false。

### MD014 - 命令前使用$而不显示输出

**本仓已屏蔽这条规则。**

* **错误示例**

  ```text
  ls
  cat foo
  less bar
  ```

* **正确示例**

  ```text
  ls
  cat foo
  less bar
  ```

  ```text
  $ ls
  foo bar
  $ cat foo
  Hello world
  $ cat bar
  baz
  ```

  在代码块中，终端命令前不需要要有$，但是如果代码中既有终端命令，也有命令的输出，则终端前可以有$。

### MD018 - atx样式的标题后没有空格

* **错误示例**

  ```text
  #Header1

  ##Header2
  ```

* **正确示例**

  ```text
  # Header1

  ## Header2
  ```

### MD019 - atx样式的标题后有多个空格

* **错误示例**

  ```text
  #  Header1

  ##  Header2
  ```

* **正确示例**

  ```text
  # Header1

  ## Header2
  ```

### MD020 - closed atx样式的标题内没有空格

**本仓已屏蔽这条规则。**

* **错误示例**

  ```text
  #Header1#

  ##Header2##
  ```

* **正确示例**

  ```text
  # Header1 #

  ## Header2 ##
  ```

### MD021 - closed atx样式的标题内有多个空格

**本仓已屏蔽这条规则。**

* **错误示例**

  ```text
  #  Header1  #

  ##  Header2  ##
  ```

* **正确示例**

  ```text
  # Header1 #

  ## Header2 ##
  ```

### MD022 - 标题行的上下行应该都是空行

* **参数**

  * `lines_above`：指定标题行上方的空行数，默认值是1。
  * `lines_below`：指定标题行下方的空行数，默认值是1。

* **错误示例**

  ```text
  # Header1
  Some text

  Some more text
  ## Header2
  ```

* **正确示例**

  ```text
  # Header1

  Some text

  Some more text

  ## Header2
  ```

### MD023 - 标题必须从行首开始

* **错误示例**

  ```text
  Some text
    ## Indented header
  ```

* **正确示例**

  ```text
  Some text

  ## Header
  ```

像块引用这样的场景会“缩进”行首，因此以下写法也是正确的：

* **正确示例**

  ```text
  > # Heading in Block Quote
  ```

### MD024 - 不能有重复内容的标题

**本仓已屏蔽这条规则。**

* **参数**

  * `siblings_only`：仅检查同级标题，默认是false。

* **错误示例**

  ```text
  # Some text

  ## Some text
  ```

* **正确示例**

  ```text
  # Some text

  ## Some more text
  ```

如果参数`siblings_only`设置为 true，则允许不同父级下的标题重复（这在变更日志中很常见）：

```text
# Change log

## 1.0.0

### Features

## 2.0.0

### Features
```

### MD025 - 文档中有多个顶级标题

**本仓已屏蔽这条规则。**

* **参数**

  * `level`：指定文档最高级的标题，默认值是1。
  * `front_matter_title`：用于匹配 front matter 中标题的正则表达式，默认 ^\s*title\s*\[:=]。

* **错误示例**

  ```text
  # Top level header

  # Another top level header
  ```

* **正确示例**

  ```text
  # Title

  ## Header

  ### Another header
  ```

### MD026 - 标题行尾的标点符号

* **参数**

  * `punctuation`：指定标题行尾不能有的标点符号，默认值是".,;:!?"。

* **错误示例**

  ```text
  # This is a header.
  ```

* **正确示例**

  ```text
  # This is a header
  ```

### MD027 - 块引用符号后的多个空格

**本仓已屏蔽这条规则。**

* **参数**

  * `list_items`：包含列表项，默认是true。

* **错误示例**

  ```text
  >  This is a block quote with bad indentation
  >  there should only be one
  ```

* **正确示例**

  ```text
  > This is a block quote with bad indentation
  > there should only be one
  ```

在块引用内推断预期的列表缩进可能比较困难；将`list_items`参数设置为 false 可对有序和无序列表项禁用此规则。

### MD028 - 块引用内的空行

* **错误示例**

  ```text
  > This is a blockquote
  > which is immediately followed by

  > this blockquote. Unfortunately
  > in some parsers, this are treated as the same blockquote.
  ```

* **正确示例**

  ```text
  > This is a blockquote.
  >
  > This is the same blockquote.
  ```

### MD029 - 有序列表的前缀序号

* **参数**

  * `style`：指定前缀序号的格式，有 `one`(只用1做前缀)，`ordered`(从1开始的加1递增数字做前缀)两种，默认值是 `one`。**本仓设置为`ordered`**。

* **错误示例**

  ```text
  1. Do this
  1. Do that
  1. Done
  ```

* **正确示例**

  ```text
  1. Do this
  2. Do that
  3. Done
  ```

### MD030 - 列表标记后的空格

* **参数**

  * `ul_single`：无序列表单个段落的前缀符号和文字之间的空格数，默认值是1。
  * `ol_single`：有序列表单个段落的前缀符号和文字之间的空格数，默认值是1。
  * `ul_multi`：无序列表多个段落的前缀符号和文字之间的空格数，默认值是1。
  * `ol_multi`：有序列表单个段落的前缀符号和文字之间的空格数，默认值是1。

* **错误示例**

  ```text
  *  Foo
  *  Bar
  *  Baz
  ```

* **正确示例**

  ```text
  * Foo
  * Bar
  * Baz
  ```

### MD031 - 单独的代码块前后需要用空格隔开（除非是在文档的开头或者结尾）

* **错误示例**

  ````text
  Some text
  ```
  Code block
  ```

  ```
  Another code block
  ```
  Some more text
  ````

* **正确示例**

  ````text
  Some text

  ```
  Code block
  ```

  ```
  Another code block
  ```

  Some more text
  ````

### MD032 - 列表前后需要用空格隔开（除非是在文档的开头或者结尾）

* **错误示例**

  ```text
  Some text
  * Some
  * List

  1. Some
  2. List
  Some text
  ```

* **正确示例**

  ```text
  Some text

  * Some
  * List

  1. Some
  2. List

  Some text
  ```

### MD033 - 内联HTML

**本仓已屏蔽这条规则。**

* **参数**

  * `allowed_elements`：允许的元素，默认值是 1。

* **错误示例**

  ```text
  <h1>Inline HTML header</h1>
  ```

* **正确示例**

  ```text
  # Markdown header
  ```

> \[!NOTE]说明
> 要允许特定的 HTML 元素，请使用`allowed_elements`参数。

### MD034 - 使用纯URL

* **错误示例**

  ```text
  For more information, see http://www.example.com/.
  ```

* **正确示例**

  ```text
  For more information, see <http://www.example.com/>.
  ```

### MD035 - 水平线样式

* **参数**

  * `style`：指定创建水平线的方式，有 `consistent`、`***`、`---`或其他指定水平线的字符串，默认值是 `consistent`。

* **错误示例**

  ```text
  ---

  - - -

  ***

  * * *

  ****
  ```

* **正确示例**

  ```text
  ---

  ---
  ```

### MD036 - 使用强调标记代替标题

**本仓已屏蔽这条规则。**

* **参数**

  * `punctuation`：指定用于结尾的标点符号，以此符号结尾的强调不会被视为以强调代替标题，默认值是".,;:!?"

* **错误示例**

  ```text
  **My document**

  Lorem ipsum dolor sit amet...

  _Another section_

  Consectetur adipiscing elit, sed to eiusmod
  ```

* **正确示例**

  ```text
  # My document

  Lorem ipsum dolor sit amet...

  ## Another section

  Consectetur adipiscing elit, sed to eiusmod
  ```

### MD037 - 强调标记内强调的符号和强调的文字之间不能有空格

* **错误示例**

  ```text
  Here is some ** bold ** text

  Here is some * italic * text

  Here is some more __ bold __ text

  Here is some more _ italic _ text
  ```

* **正确示例**

  ```text
  Here is some **bold** text

  Here is some *italic* text

  Here is some more __bold__ text

  Here is some more _italic_ text
  ```

### MD038 - 单反引号和之间的内容不能有空格

* **错误示例**

  ```text
  ` some text `
  `some text `
  ` some text`
  ```

* **正确示例**

  ```text
  `some text`
  ```

### MD039 - 链接文本和包围它的中括号之间内容不能有空格

* **错误示例**

  ```text
  [ a link ](http://www.example.com/)
  ```

* **正确示例**

  ```text
  [a link](http://www.example.com/)
  ```

### MD040 - 代码块应指定代码块的编程语言

* **错误示例**

  ````text
  ```
  #!/bin/bash
  echo Hello world
  ```
  ````

* **正确示例**

  ````text
  ```bash
  #!/bin/bash
  echo Hello world
  ```
  ````

* **常用的代码块编程语言**

  | 语言支持         | 关键字   |
  | ---------------- | -------- |
  | Python           | python   |
  | C                | cpp      |
  | Java             | java     |
  | Shell            | bash     |
  | Markdown         | markdown |
  | JavaScript       | js       |
  | CSS              | css      |
  | SQL              | sql      |
  | PHP              | php      |
  | Text             | text     |
  | XML              | html     |
  | Bat              | bat      |
  | Protocol Buffers | protobuf |

### MD041 - 文件中的第一行应该是顶级标题

* **参数**

  * `level`：指定文档最高级的标题，默认值是1。
  * `allow_preamble`：允许标题前有内容，默认值是false。
  * `front_matter_title`：用于匹配 front matter 中标题的正则表达式，默认值是^\s*title\s*\[:=]。

* **错误示例**

  ```text
  This is a file without a header
  ```

* **正确示例**

  ```text
  # File with header

  This is a file with a top level header
  ```

### MD042 - 无空链接

* **错误示例**

  ```text
  [an empty link]()
  ```

  ```text
  [an empty fragment](#)
  ```

* **正确示例**

  ```text
  [a valid link](https://example.com/)
  ```

  ```text
  [a valid fragment](#fragment)
  ```

### MD043 - 必需的标题结构

* **参数**

  * `headings`：标题列表，默认 \[]。
  * `match_case`：匹配标题的大小写，默认值是false。

* **正确示例**

  ```text
  # Heading
  ## Item
  ### Detail
  ```

将`headings`参数设置为：
\[
"# Heading",
"## Item",
"### Detail"
]

### MD044 - 专有名词应使用正确的大小写

* **参数**

  * `code_blocks`：包含代码块，默认值是true。
  * `html_elements`：包含 HTML 元素，默认值是true。
  * `names`：专有名词列表，默认值是\[]。

* **正确示例**
  语言“JavaScript”通常首字母 J 和 S 大写——尽管有时 s 或 j 以小写形式出现。要强制执行正确的大小写，请在 names 数组中指定所需的大小写。

  \[
  "JavaScript"
  ]

### MD045 - 图像应有替代文本（alt 文本）

* **正确示例**

  ```text
  ![Alternate text](image.jpg)
  ```

  ```text
  ![Alternate text][ref]

  ...

  [ref]: image.jpg "Optional title"
  ```

### MD046 - 代码块样式

**本仓已屏蔽这条规则。**

* **参数**

  * `style`：指定代码块定义格式，有 `fenced（使用三个反引号）`，`indented（使用缩进）`，`consistent（上下文一致）`三种，默认值是 `fenced`。

* **错误示例**

  ```text
  Some text.

      Code block

  Some more text.
  ```

* **正确示例**

  ````text
  Some text.

  ```ruby
  Code block
  ```

  Some more text.
  ````

### MD046 -  代码块风格

* **参数**

  * `style`：指定代码块定义格式，有 `consistent（一致）`，`fenced（围栏式）`，`indented（缩进式）`三种，默认值是 `consistent`。

* **错误示例**

  ````text
  Some text.

      # Indented code

  More text.

  ```ruby
  # Fenced code
  ```

  More text.
  ````

* **正确示例**

  ```text
  Some text.

      # Indented code

  More text.

      # Fenced code

  More text.
  ```

### MD047 - 文件应以单个换行符结尾

* **错误示例**

  ```text
  # Header

  This file ends without a newline.[EOF]
  ```

* **正确示例**

  ```text
  # Header

  This file ends with a newline.
  [EOF]
  ```

### MD048 - 代码围栏风格

* **参数**

  * `style`：指定代码块定义格式，有 `consistent（一致）`，`backtick（反引号）`，`tilde（波浪线）`三种，默认值是 `consistent`。

* **错误示例**

  ````text
  ```ruby
  # Fenced code
  ```

  ~~~ruby
  # Fenced code
  ~~~
  ````

* **正确示例**

  ````text
  ```ruby
  # Fenced code
  ```

  ```ruby
  # Fenced code
  ```
  ````

### MD049 - 强调样式

* **参数**

  * `style`：指定代码块定义格式，有 `consistent（一致）`，`asterisk（星号）`，`underscore（下划线）`三种，默认值是 `consistent`。

* **错误示例**

  ```text
  *Text*
  _Text_
  ```

* **正确示例**

  ```text
  *Text*
  *Text*
  ```

### MD050 - 加粗样式

* **参数**

  * `style`：指定代码块定义格式，有 `consistent（一致）`，`asterisk（星号）`，`underscore（下划线）`三种，默认值是 `consistent`。

* **错误示例**

  ```text
  **Text**
  __Text__
  ```

* **正确示例**

  ```text
  **Text**
  **Text**
  ```

### MD051 - 链接片段应有效

* **参数**

  * `ignore_case`：忽略片段大小写，默认值是false。
  * `ignored_pattern`：用于忽略额外片段的模式，默认为空字符串。

* **错误示例**

  ```text
  # Heading Name

  [Link](#fragment)
  ```

* **正确示例**

  ```text
  # Heading Name

  [Link](#heading-name)
  ```

### MD052 - 参考链接和图片应使用已定义的标签

* **参数**

  * `ignored_labels`：需忽略的链接标签，默认值：\["x"]。
  * `ignored_pattern`：是否包含快捷语法，默认值是false。

  参考链接和图片有以下三种形式：
  Full: \[text]\[label]
  Collapsed: \[label]\[]
  Shortcut: \[label]

  Full: !\[text]\[image]
  Collapsed: !\[image]\[]
  Shortcut: !\[image]

* **正确示例**

  ```text
  [label]: https://example.com/label
  [image]: https://example.com/image
  ```

### MD053 - 链接和图片参考定义应为必需

* **参数**

  * `ignored_definitions`：要忽略的定义，默认值：\["//"]。

* **正确示例**

  ```text
  [//]: # (This behaves like a comment)
  ```

### MD054 - 链接和图片样式

* **参数**

  * `autolink`：允许自动链接，默认值是true。
  * `collapsed`：允许折叠参考链接和图片，默认值是true。
  * `full`：允许完整参考链接和图片，默认值是true。
  * `inline`：允许内联链接和图片，默认值是true。
  * `shortcut`：允许快捷参考链接和图片，默认值是true。
  * `url_inline`：允许内联链接使用 URL 作为文本，默认值是true。

* **正确示例**

将 inline参数设置为 false将禁用内联链接和图片：

```text
<https://example.com>
```

将 inline参数设置为 false将禁用内联链接和图片：

```text
[link](https://example.com)

![image](https://example.com)
```

将 full参数设置为 false将禁用完整参考链接和图片：

```text
[link][url]

![image][url]

[url]: https://example.com
```

将 collapsed参数设置为 false将禁用折叠参考链接和图片：

```text
[url][]

![url][]

[url]: https://example.com
```

将 shortcut参数设置为 false将禁用快捷参考链接和图片：

```text
[url]

![url]

[url]: https://example.com
```

将 url\_inline参数设置为 false可防止使用具有相同绝对 URL 文本/目标且无标题的内联链接，因为此类链接可以转换为自动链接：

```text
[https://example.com](https://example.com)
```

要修复 url\_inline违规，可使用更简单的自动链接语法：

```text
<https://example.com>
```

### MD055 - 表格管道符样式

* **参数**

  * `style`：表格管道符样式，可选值：consistent/ leading\_and\_trailing/ leading\_only/ no\_leading\_or\_trailing/ trailing\_only，默认值是consistent。

* **错误示例**

  ```text
  | Header | Header |
  | ------ | ------
    Cell   | Cell   |
  ```

  ```text
  | Header | Header |
  | ------ | ------ |
  | Cell   | Cell   |
  This text is part of the table
  ```

* **正确示例**

  ```text
  | Header | Header |
  | ------ | ------ |
  | Cell   | Cell   |
  ```

  ```text
  | Header | Header |
  | ------ | ------ |
  | Cell   | Cell   |

  This text is part of the table
  ```

### MD056 - 表格列数一致性

* **错误示例**

  ```text
  | Header | Header |
  | ------ | ------ |
  | Cell   | Cell   |
  | Cell   |
  | Cell   | Cell   | Cell   |
  ```

* **正确示例**

  ```text
  | Header | Header |
  | ------ | ------ |
  | Cell   | Cell   |
  | Cell   | Cell   |
  | Cell   | Cell   |
  ```

### MD058 - 表格前后应包含空白行

* **错误示例**

  ```text
  Some text
  | Header | Header |
  | ------ | ------ |
  | Cell   | Cell   |
  > Blockquote
  ```

* **正确示例**

  ```text
  Some text

  | Header | Header |
  | ------ | ------ |
  | Cell   | Cell   |

  > Blockquote
  ```

### MD059 -  链接文本应具有描述性

* **参数**

  * `prohibited_texts`：禁止使用的链接文本，默认值：\["click here","here","link","more"]。

* **错误示例**

  ```text
  Some text
  [click here](...)
  [link](...)
  ```

* **正确示例**

  ```text
  Some text
  [Download the budget document](...)
  [CommonMark Specification](...)
  ```

## VSCode 中 Markdown 插件

markdownlint 扩展库包含 markdown 文件规则库，以保证 markdown 文件与其标准保持一致。添加配置后，markdownlint 可以自动检查文档错误。

### 安装

* 按下 `Ctrl_Shift+X`以打开扩展选项卡。
* 输入 `markdownlint` 以找到扩展。
* 点击 `Install` 按钮，然后再点击`Enable`按钮。

### 配置

注意：VSCode 中 markdownlint 参照的版本是 David Anson 拟定的，与 openEuler 仓使用的 markdownlint 官方 v0.38.0 版本有差异。为了与 openEuler 仓配置的规则保持一致，可参考下方配置项。

* 在命令面板（`Ctrl+Shift+P`）中输入`Open Settings (JSON)`命令。
* 在 Json 对象中添加如下配置：

  ```bash
  "markdownlint.config":{
     "default":true,
     "MD003":{"style":"atx"},
     "MD029":{"style":"ordered"},
     "MD004":false,
     "MD007":false,
     "MD009":false,
     "MD013":false,
     "MD014":false,
     "MD020":false,
     "MD021":false,
     "MD024":false,
     "MD025":false,
     "MD033":false,
     "MD036":false,
     "MD042":false,
     "MD043":false,
     "MD044":false,
     "MD045":false,
     "MD046":false,
     "MD048":false,
     "MD049":false,
     "MD050":false,
     "MD051":false,
     "MD052":false,
     "MD053":false,
     "MD055":false,
     "MD056":false,
     "MD057":false
  }
  ```

---

---
url: /zh/docs/common/contribute/markdownlint_tools.md
---
# markdownlint 错误修复工具

markdownlint-cli2 适用于批量修改 markdownlint 低错问题，如空行、缩进等，大大提高文档开发效率，但是复杂问题仍需要手动修复。

## 安装与配置

### 安装 Node.js + npm

进入 [Node.js](https://nodejs.org/zh-cn)官网下载 Node.js，并按照提示完成安装。

分别执行如下两条命令，如果显示版本号，则说明安装成功。

```shell
node -v
npm -v
```

### 安装 markdownlint-cli2

执行如下命令，安装 markdownlint-cli2。

```shell
npm install markdownlint-cli2 --global
```

如果遇到类似以下错误，可能权限问题导致的。

```txt
npm error code EACCES
npm error syscall mkdir
npm error path /usr/local/lib/node_modules/markdownlint-cli2
npm error errno -13
```

以管理员身份解决该问题：如果是 Mac 或 Linux 系统，可以在命令前加 `sudo`；如果是 `Windows` 系统，在命令提示符或者PowerShell中以管理员身份运行命令。

### 配置 markdownlint-cli2

markdownlint-cli2支持指定检查项，配置文件默认名为.markdownlint.json。将配置文件与待检查的 markdown 文件放在同一文件夹（如果要检查多个文件，则放在其共同的最上级文件夹），markdownlint-cli2就会自动读取并执行。

./markdownlint.json 文件示例如下：

```bash
{
  "MD003":{"style":"atx"},
  "MD029":{"style":"ordered"},
  "MD004":false,
  "MD007":false,
  "MD009":false,
  "MD013":false,
  "MD014":false,
  "MD020":false,
  "MD021":false,
  "MD024":false,
  "MD025":false,
  "MD027":false,
  "MD033":false,
  "MD036":false,
  "MD042":false,
  "MD043":false,
  "MD044":false,
  "MD045":false,
  "MD046":false,
  "MD048":false,
  "MD049":false,
  "MD050":false,
  "MD051":false,
  "MD052":false,
  "MD053":false,
  "MD055":false,
  "MD056":false,
  "MD057":false
}
```

## 检查与修复

### 检查

执行如下命令，检查指定的 markdown 文件或文件夹。

```bash
markdownlint-cli2 "**/*.md"
```

其中，文件路径可以是一个或多个文件名，也可以是通配符，或是文件夹。注意当被检测的文件夹中包含非.md格式的文件时，可能出现错误，导致检测失败。

### 修正

修正markdownlint的错误，可以使用`--fix`参数。将在源文件上直接修正错误语法，不创建备份。执行如下命令：

```bash
markdownlint-cli2 --fix "**/*.md"
```

---

---
url: /en/docs/common/faq/server/migration_faqs.md
---
# Migration FAQ

## What guidelines and resources can I refer to during migration

You can refer to the following resources to help you migrate to openEuler:

* [x2openEuler document](https://docs.openeuler.org/zh/docs/20.03_LTS_SP1/docs/x2openEuler/x2openEuler.html)
* [x2openEuler download](https://repo.oepkgs.net/openEuler/rpm/openEuler-22.03-LTS/contrib/x2openEuler/noarch/Packages)
* [openEuler repositories](https://forum.openeuler.org/t/topic/768)
* [x2openEuler compatibility database (CentOS 7/8 to openEuler 22.03 LTS)](https://repo.oepkgs.net/openEuler/rpm/openEuler-22.03-LTS/contrib/x2openEuler/noarch/Packages)

## Why does openEuler have less available memory than CentOS even when the amount of allocated physical memory is the same

The difference in available memory is due to the difference in the amount of memory allocated for the crashkernel (memory area used during kernel crashes). For instance, with 4 GB of physical memory:

* CentOS: 3.7 GB available (161 MB reserved for crashkernel)
* openEuler: 3.3 GB available (512 MB reserved for crashkernel)

To match CentOS's memory availability, you can reduce openEuler's crashkernel reservation to 256 MB in the GRUB configuration file (/boot/grub2/grub.cfg).

## What do I do if software packages cannot be parsed by macros during migration

This issue often occurs during the migration from CentOS or Fedora to other systems and is caused by different operating systems having different macro definitions.

There are two solutions to this issue:

1. Query the specific meaning of the macros and replace them in the SPEC file with their expanded values.
2. Introduce the software packages that provide the macro definitions into the corresponding repository and add them to **BuildRequires**. This allows the software packages that fail to run to be parsed by macros.

## How do I prepare the environment and check the prerequisites for VM live migration

Before performing a VM live migration, it is essential to prepare two PMs (source and destination) and check necessary conditions to ensure a smooth migration process. These checks include:

Permission check: ensures that the current user has permission to perform live migration.

Network check: verifies network connectivity between the source and destination PMs, ensuring they are in the same network segment.

Storage resource check: verifies that the source and destination PMs can access the same storage resources and ensures that the destination PM has sufficient CPU, memory, and storage resources.

VM state check: confirms that the VM to be migrated is running.

Additionally, you need to set live migration parameters such as maximum downtime and bandwidth during migration as required. You also need to identify whether shared or non-shared storage is being used. If the storage mode is non-shared, you may need to perform additional operations such as using NFS to achieve shared storage.

## What is VM live migration and how does it differ from cold migration

Virtual machine (VM) live migration is a technology that enables the seamless migration of a running VM, including its in-memory and on-drive data, to another physical server without shutting down the VM. This process is transparent to users, meaning that there is no perceived service interruption or performance degradation. Live migration is typically used for hardware maintenance, upgrades, load balancing, and ensuring high availability of critical services.

In contrast, VM cold migration (also known as static migration) requires VM shutdown before migration. This means services on the VM are unavailable during the migration process. Cold migration is suitable for scenarios where downtime is acceptable, such as batch processing jobs or migrating non-critical services.

## How to migrate SQL Server data from Windows to openEuler

To migrate SQL Server data from Windows to openEuler, follow these steps:

1. Back up the SQL Server database on Windows. You can use SQL Server Management Studio (SSMS) or SQL statements to perform the backup.
2. Once the backup is complete, transfer the backup file to openEuler using the SCP command or any other relevant method.
3. Create a new backup directory on openEuler and move the backup file into this directory.
4. Use the sqlcmd utility to execute SQL statements for restoring the database. If the database includes auxiliary files, make sure to add the **MOVE** option for these files in the **RESTORE DATABASE** statement.
5. Finally, verify the success of the data migration by listing all databases.

## Why is hardware compatibility testing necessary during the migration from CentOS to openEuler

Migrating from CentOS to openEuler involves not only changing the OS but also replacing, adapting, migrating, and re-building the application software and service systems running on the OS. Ensuring hardware compatibility is essential to guarantee system stability and service continuity during the migration and to prevent application failures or performance degradation afterward. Therefore, hardware compatibility testing is a key part of OS migration.

## What can x2openEuler migration tool do and how is it used for migration assessment

The x2openEuler migration tool provided by the openEuler community is primarily used for migration assessment and has the following functions:

* Software assessment: It assesses applications in various formats, including RPM, TAR, ZIP, GIP, and JAR packages, Python scripts, shell scripts, and binary files by scanning the dependent software package list, and generates assessment reports in HTML format.
* Configuration collection and assessment: It supports the collection of user environment data and generates JSON files. This includes hardware configurations, configuration interfaces, kernel option configurations, system configurations (**sysctl**, **proc**, and **sys**), environment variables, services, processes, ports, command interfaces, system call items, and device driver interfaces. Then, it completes configuration analysis and assessment.
* Hardware assessment: It evaluates the compatibility of the server and boards (such as RAID, NIC, FC, IB, GPU, SSD, TPM, etc.) with openEuler's compatibility list. These functions help you identify potential compatibility issues before migration to ensure a smooth migration process.

---

---
url: /en/docs/common/faq/server/kernel_faqs.md
---
# nvwa FAQ

## 1. After the `nvwa Update` Command Is Executed, the System Is Not Upgraded

Cause: An error occurs when the running information is retained or the kernel is replaced.

Solution: View logs to find the error cause.

## 2. After the Acceleration Feature Is Enabled, the `nvwa` Command Fails to Be Executed

Cause: NVWA provides many acceleration features, including quick kexec, pin memory, and cpu park. These features involve the cmdline configuration and memory allocation. When selecting the memory, run cat /proc/iomemory to ensure that the selected memory does not conflict with that of other programs. If necessary, run the dmesg command to check whether error logs exist after the feature is enabled.

## 3. After the Hot Upgrade, the Related Process Is Not Recovered

Cause: Check whether the nvwa service is running. If the nvwa service is running, the service or process may fail to be recovered.

Solution: Run the service `nvwa status` command to view the NVWA logs. If the service fails to be started, check whether the service is enabled, and then run the `systemd` command to view the logs of the corresponding service. Further logs are stored in the process or service folder named after the path specified by **criu\_dir**. The dump.log file stores the logs generated when the running information is retained, and the restore.log file restores the logs generated for process recovery.

## 4. The Recovery Fails, and the Log Displays "Can't fork for 948: File exists"

Cause: The kernel hot upgrade tool finds that the PID of the program is occupied during program recovery.

Solution: The current kernel does not provide a mechanism for retaining PIDs. Related policies are being developed. This restriction will be resolved in later kernel versions. Currently, you can only manually restart related processes.

## 5. When the `nvwa` Command Is Used to save and Recover a Simple Program (Hello World), the System Displays a Message Indicating That the Operation Fails or the Program Is Not Running

Cause: There are many restrictions on the use of CRIU.

Solution: View the NVWA logs. If the error is related to the CRIU, check the dump.log or restore.log file in the corresponding directory. For details about the usage restrictions related to the CRIU, see [CRIU WiKi](https://criu.org/What_cannot_be_checkpointed).

---

---
url: /en/docs/common/faq/caselibrary/caselibrary_menu.md
---
# openEuler Case Collection

| No. | Source             | Category        | Keywords                      | Title                                                                                                                                                    |
| --- | ------------------ | --------------- | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1   | Developer feedback | Core services   | systemd-logind, NIS           | [systemd-logind.service Failed to Retrieve NIS User Information](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/systemd-logind.md) |
| 2   | Developer feedback | Storage         | Mount, sysroot, cryptography  | [/sysroot Mount Failure](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/mountsysroot.md)                                           |
| 3   | Developer feedback | Core services   | Audit, Logs                   | [System Halt Caused by Audit Logs Consuming Drive Space](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/audit.md)                  |
| 4   | Developer feedback | Containers      | Docker, umask                 | [Configuring Container umask Values in Docker](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/umask.md)                            |
| 5   | Developer feedback | Containers      | Docker, container termination | [Docker Container Terminates After a Certain Period](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/docker.md)                     |
| 6   | Developer feedback | Core services   | kpatch, hot patch error       | [Kernel Hot patch Creation Issue: dmesg Reporting Missing sssnic Module](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/sssnic.md) |
| 7   | Developer feedback | Compute         | 22.03-LTS, Zabbix setup       | [Zabbix Installation Guide for openEuler 22.03 LTS](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/zabbix.md)                      |
| 8   | Developer feedback | Containers      | iSulad, gRPC, Rest            | [Accessing iSulad Services Through gRPC and REST Protocols](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/isulad.md)              |
| 9   | Developer feedback | Storage/Compute | LVM, storage                  | [LVM Label Corruption](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/lvm.md)                                                      |
| 10  | Developer feedback | Containers      | pvs, vgs                      | ["Unknown" Errors in pvs or vgs](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/pvs_vgs.md)                                        |
| 11  | Developer feedback | Storage/Compute | kdump, vmcore                 | [kump FAQ](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/crash.md)                                                                |
| 12  | Developer feedback | Installation    | efivars                       | [EFI Variables Installation Errors](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/efivars.md)                                     |
| 13  | Developer feedback | Installation    | Rebranding                    | [Rebranding FAQ](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/rebranding.md)                                                      |
| 14  | Developer feedback | Installation    | Anaconda                      | [Common Drive Issues in the Anaconda Installer](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/anaconda.md)                        |
| 15  | Developer feedback | Installation    | System files                  | [System File Recovery FAQ](https://atomgit.com/openeuler/docs/blob/stable-common/docs/en/faq/caselibrary/sysfile.md)                                              |

---

---
url: >-
  /en/docs/common/faq/community_tools/deployment-guide-for-network-environment-faqs.md
---
# openEuler Copilot System FAQ (Network Environment)

## 1. HuggingFace Connection Issue

```text
File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.eanconectionError: <urlib3.comnection.Hipscomnection object at exfblab6490>: Failed to establish a new conmection: [Errno 101] Network is unreachable
```

* Solution

```bash
pip3 install -U huggingface_hub
export HF_ENDPOINT=https://hf-mirror.com
```

## 2. Querying the RAG Service API

* Enter the pod for RAG.

```bash
curl  -k -X POST "http://localhost:8005/kb/get_answer"  -H "Content-Type: application/json"  -d '{ \
      "question": "", \
      "kb_sn": "default_test", \
      "fetch_source": true  }'
```

## 3. Helm Upgrade Cluster Errors

```text
Error: INSTALLATI0N FAILED: Kubernetes cluster unreachable: Get "http:/localhost:880/version": dial tcp [:1:8089: connect: connection refused
```

or

```text
Error: UPGRADE FAILED: Kubernetes cluster unreachable: the server could not find the requested resource
```

* Solution

```bash
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
```

## 4. Pod Log Access Issues

```text
[root@localhost euler-copilot]# kubectl logs rag-deployservice65c75c48d8-44vcp-n euler-copilotDefaulted container "rag" out of: rag.rag-copy secret (init)Error from server: Get "https://172.21.31.11:10250/containerlogs/euler copilot/rag deploy"service 65c75c48d8-44vcp/rag": Forbidden
```

* Solution

Ensure the local Kubernetes node IP address is excluded from proxy settings:

```bash
cat /etc/systemd/system/k3s.service.env
http_proxy="http://172.21.60.51:3128"
https_proxy="http://172.21.60.51:3128"
no_proxy=172.21.31.10 # Add the node IP address.
```

## 5. Streaming Response Issues in LLM API in the GPU environment

Curl requests fail when `"stream": true` but succeed with `"stream": false`.

```bash
curl http://localhost:30000/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer sk-123456"   -d '{
"model": "qwen1.5",
"messages": [
{
"role": "system",
"content": "You are an sentiment analysis expert..."
},
{
"role": "user",
"content": "Hello"
}
],
"stream": true,
"n": 1,
"max_tokens": 32768
}'
```

* Solution:

```bash
pip install Pydantic=1.10.13
```

## 6. SGLang Deployment

```bash
# 1. Activate the Conda environment (assuming the name is "myenv").
conda activate myenv

# 2. Install sglang[all] and flashinfer.
pip install sglang[all]==0.3.0
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/

# 3. Launch the server.
python -m sglang.launch_server --served-model-name Qwen2.5-32B --model-path Qwen2.5-32B-Instruct-AWQ --host 0.0.0.0 --port 8001 --api-key sk-12345 --mem-fraction-static 0.5 --tp 8
```

* Verification:

```bash
pip show sglang
pip show flashinfer
```

* Note:

1. API key: Ensure the `--api-key` value is valid.
2. Model path: Verify the `--model-path` points to an existing model directory.
3. CUDA version: `flashinfer` requires CUDA 12.1 and PyTorch 2.4.
4. Thread pool size: Adjust `--tp` (for example, `--tp 8` for 8 GPUs) based on available resources.

## 7. Embedding API Request

```bash
curl -k -X POST http://$IP:8001/embedding \
     -H "Content-Type: application/json" \
     -d '{"texts": ["sample text 1", "sample text 2"]}'
# Replace $IP with the internal network address of the vectorize embedding service.
```

## 8. Certificate Generation

```bash
# Download mkcert: https://github.com/FiloSottile/mkcert/releases
# x86_64
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64
# arm64
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-arm64
# 2. Generate keys.
mkcert -install
mkcert example.com # Supports domains or IP addresses.
# 3. Copy certificates and keys to /home/euler-copilot-framework_openeuler/euler-copilot-helm/chart_ssl/traefik-secret.yaml, then apply:
kubectl apply -f traefik-secret.yaml
```

---

---
url: >-
  /en/docs/common/faq/community_tools/deployment-guide-for-offline-environment-faqs.md
---
# openEuler Copilot System FAQ (Network Environment)

## 1. HuggingFace Connection Issue

```text
File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.eanconectionError: <urlib3.comnection.Hipscomnection object at exfblab6490>: Failed to establish a new conmection: [Errno 101] Network is unreachable
```

* Solution

```bash
pip3 install -U huggingface_hub
export HF_ENDPOINT=https://hf-mirror.com
```

## 2. Querying the RAG Service API

* Enter the pod for RAG.

```bash
curl  -k -X POST "http://localhost:8005/kb/get_answer"  -H "Content-Type: application/json"  -d '{ \
      "question": "", \
      "kb_sn": "default_test", \
      "fetch_source": true  }'
```

## 3. Helm Upgrade Cluster Errors

```text
Error: INSTALLATI0N FAILED: Kubernetes cluster unreachable: Get "http:/localhost:880/version": dial tcp [:1:8089: connect: connection refused
```

or

```text
Error: UPGRADE FAILED: Kubernetes cluster unreachable: the server could not find the requested resource
```

* Solution

```bash
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
```

## 4. Pod Log Access Issues

```text
[root@localhost euler-copilot]# kubectl logs rag-deployservice65c75c48d8-44vcp-n euler-copilotDefaulted container "rag" out of: rag.rag-copy secret (init)Error from server: Get "https://172.21.31.11:10250/containerlogs/euler copilot/rag deploy"service 65c75c48d8-44vcp/rag": Forbidden
```

* Solution

Ensure the local Kubernetes node IP address is excluded from proxy settings:

```bash
cat /etc/systemd/system/k3s.service.env
http_proxy="http://172.21.60.51:3128"
https_proxy="http://172.21.60.51:3128"
no_proxy=172.21.31.10 # Add the node IP address.
```

## 5. Streaming Response Issues in LLM API in the GPU environment

Curl requests fail when `"stream": true` but succeed with `"stream": false`.

```bash
curl http://localhost:30000/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer sk-123456"   -d '{
"model": "qwen1.5",
"messages": [
{
"role": "system",
"content": "You are an sentiment analysis expert..."
},
{
"role": "user",
"content": "Hello"
}
],
"stream": true,
"n": 1,
"max_tokens": 32768
}'
```

* Solution:

```bash
pip install Pydantic=1.10.13
```

## 6. SGLang Deployment

```bash
# 1. Activate the Conda environment (assuming the name is "myenv").
conda activate myenv

# 2. Install sglang[all] and flashinfer.
pip install sglang[all]==0.3.0
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/

# 3. Launch the server.
python -m sglang.launch_server --served-model-name Qwen2.5-32B --model-path Qwen2.5-32B-Instruct-AWQ --host 0.0.0.0 --port 8001 --api-key sk-12345 --mem-fraction-static 0.5 --tp 8
```

* Verification:

```bash
pip show sglang
pip show flashinfer
```

* Note:

1. API key: Ensure the `--api-key` value is valid.
2. Model path: Verify the `--model-path` points to an existing model directory.
3. CUDA version: `flashinfer` requires CUDA 12.1 and PyTorch 2.4.
4. Thread pool size: Adjust `--tp` (for example, `--tp 8` for 8 GPUs) based on available resources.

## 7. Embedding API Request

```bash
curl -k -X POST http://$IP:8001/embedding \
     -H "Content-Type: application/json" \
     -d '{"texts": ["sample text 1", "sample text 2"]}'
# Replace $IP with the internal network address of the vectorize embedding service.
```

## 8. Certificate Generation

```bash
# Download mkcert: https://github.com/FiloSottile/mkcert/releases
# x86_64
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64
# arm64
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-arm64
# 2. Generate keys.
mkcert -install
mkcert example.com # Supports domains or IP addresses.
# 3. Copy certificates and keys to /home/euler-copilot-framework_openeuler/euler-copilot-helm/chart_ssl/traefik-secret.yaml, then apply:
kubectl apply -f traefik-secret.yaml
```

---

---
url: /zh/docs/common/faq/community_tools/openeuler_intelligence_deployment_faqs.md
---
# openEuler Intelligence 常见问题与解决方法-网络环境下部署

## 1. 解决 Hugging Face 连接错误

```python
urllib3.exceptions.NewConnectionError: urllib3.connection.HTTPSConnection object, Failed to establish a new connection: [Errno 101] Network is unreachable
```

**解决方案**：

* 更新 `huggingface_hub` 包到最新版本。

  ```bash
  pip3 install -U huggingface_hub
  ```

* 如果网络问题依旧存在，可以尝试使用镜像站点作为端点。

  ```bash
  export HF_ENDPOINT=https://hf-mirror.com
  ```

## 2. 在 RAG 容器中调用问答接口

```bash
curl -k -X POST "http://localhost:9988/kb/get_answer" \
     -H "Content-Type: application/json" \
     -d '{"question": "您的问题", "kb_sn": "default_test", "fetch_source": true}'
```

## 3. 解决 helm upgrade 错误

```text
Error: UPGRADE FAILED: Kubernetes cluster unreachable
```

**解决方案**：

```bash
echo "export KUBECONFIG=/etc/rancher/k3s/k3s.yaml" >> /root/.bashrc
source /root/.bashrc
```

## 4. 查看 Pod 日志失败

**解决方案**：

```bash
cat /etc/systemd/system/k3s.service.env  # 确认 no_proxy 包含本机 IP
```

## 5. GPU 环境流式回复问题

**解决方案**：

```bash
pip install pydantic==1.10.13
# 请求中将 "stream": true 改为 false
```

## 6. sglang 模型部署

```bash
conda create --prefix=/root/py310 python==3.10.14
conda activate /root/py310
pip install "sglang[all]==0.3.0"
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/

python -m sglang.launch_server \
    --served-model-name Qwen2.5-32B \
    --model-path Qwen2.5-32B-Instruct-AWQ \
    --host 0.0.0.0 \
    --port 8001 \
    --api-key "sk-12345" \
    --mem-fraction-static 0.5 \
    --tp 8
```

## 7. 获取 Embedding

```bash
curl -k -X POST http://localhost:11434/v1/embeddings \
     -H "Content-Type: application/json" \
     -d '{"input": "The food was delicious...", "model": "bge-m3", "encoding_format": "float"}'
```

## 8. 生成证书

为了生成自签名证书，首先下载 [mkcert](https://github.com/FiloSottile/mkcert/releases)工具，然后运行以下命令：

```bash
mkcert -install
mkcert example.com
```

最后，将生成的证书和私钥拷贝到 values.yaml 中, 并应用至 Kubernetes Secret.

```bash
vim /home/euler-copilot-framework_openeuler/deploy/common/chart_ssl/traefik-secret.yaml
```

```bash
kubectl apply -f traefik-secret.yaml
```

## 9. 资源不足时，所有pod状态pending？

```bash
kubectl top nodes
df -h  # 确保磁盘空间 >30%
```

参考该链接挂载空间较大的磁盘[How to move k3s data to another location](https://mrkandreev.name/snippets/how_to_move_k3s_data_to_another_location/)

## 10. 无法插网线的离线环境安装k3s后启动失败？

k3s无法找到route和ipv6\_route，报错："no default routes found in '/proc/net/route' or '\proc/net\ipv6\_route'"，无法使用k3s，可以通过创建创建虚拟网络接口配置解决

**解决办法：**

```bash
# 注意：服务器器显示时间需要和当前网络时间保持一致
mkdir -p /etc/systemd/system/k3s.service.d/
cat <<EOF | sudo tee /etc/systemd/system/k3s.service.d/override.conf
[Service]
ExecStartPre=/sbin/ip link add dummy0 type dummy
ExecStartPre=/sbin/ip addr add 203.0.113.254/31 dev dummy0
ExecStartPre=/sbin/ip link set dummy0 up
ExecStartPre=/sbin/ip route add default via 203.0.113.254 dev dummy0
EOF
systemctl daemon-reload
systemctl restart k3s
```

## 11. x86架构且CPU型号E系列的服务器知识库文件解析至71%失败？

OpenGauss数据库的指令集加速特性默认开启，其对运行环境有特定要求：在x86架构下，Docker镜像中的OpenGauss服务需要CPU支持AVX512指令集；在ARM架构下则需要支持Neon指令集。由于x86架构且CPU型号E系列的服务器是低功耗x86计算，仅支持AVX2指令集，不支持AVX512，导致在执行数据插入操作时出现指令集不兼容错误。

**解决办法：**
由于不加指令集性能会受限，x86环境下如果环境不支持指令集加速，使用l2距离构建索引并查询，需要单独构建rag镜像，不能确保准确率是否会下降，建议更换支持AVX512的CPU型号进行部署。

```bash
# 查看是否支持avx512：
cat /proc/cpuinfo| grep avx512
lscpu | grep -i avx
```

## 12. 工作流理解上下文耗费时间很久？

大模型使用带reasoning的模型，例如deepseek-r1-distill-qwen32B

**解决办法：**

更换无思考过程的模型，不支持使用Deepseek R1系列和Qwen3系列

---

---
url: /zh/docs/common/contribute/doc_review_test_process.md
---
# openEuler 官方文档检视测试流程

## 概述

**背景**：openEuler 官方文档从24.03 LTS SP2版本起，文档已下沉至各sig组。\
下沉前：所有发布在官网的文档都在openeuler/docs仓存储，由doc-sig负责合入。\
下沉后：\
基础特性文档（发行说明、安装指南）还放在docs仓，由doc-sig负责合入。\
增量特性文档：oeaware用户指南、syscare用户指南，放在特性对应的代码仓，由sig maintainer负责合入。\
**问题**：文档下沉至sig组代码仓后，文档pr检视合入由sig maintainer决定，doc-sig检视/测试验收是非必须的，由sig maintainer决定并邀请，存在文档检视、测试不充分，导致文档质量无法得到保证。

## 检视测试流程

![image](figures/检视测试流程.png)

**版本发布之前**：

1. 版本关键角色，根据版本需求，确定文档清单和交付计划，文档工程师跟踪版本文档需求交付进展；
2. openeuler/docs 仓拉取新分支；
3. 代码仓拉取新分支；
4. 特性 owner 贡献文档，提 PR；
5. 特性 sig maintainer 检视文档；
6. 文档工程师检视文档，转测试；
7. 测试工程师检视文档；
8. 特性 owner 修改 pr 意见；
9. 特性 sig maintainer 合入文档。

**版本发布之后**：

1. 用户使用文档之后，发现文档问题；
2. 提交文档issue（基础特性文档提到 docs 仓，增量特性文档提到各个 sig 组代码仓；如果提到 docs 仓，则由 doc sig maintainer 转到对应代码仓）；
3. 特性 owner 处理 issue，提交文档 PR；
4. sig maintainer 检视文档；
5. 文档工程师检视文档，如仅低错修改，无需转测试；如涉及实际内容修改，则转测试（每周或双周固定时间转测试，与update版本issue转测节奏保持一致）；
6. 测试工程师检视文档；
7. 特性 owner 修改 pr 意见；
8. sig maintainer 合入文档。

---

---
url: /zh/docs/common/faq/caselibrary/caselibrary_menu.md
---
# openEuler 案例汇总

|  序号 |  问题来源 | 特性分类  | 关键字  | 标题  |
|---|---|---|---|---|
| 1 |  开发者反馈 | 基础服务  | systemd-logind,NIS  | [systemd-logind服务无法获取NIS服务器上的用户信息](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/systemd-logind.md)   |
| 2 | 开发者反馈 | storage  | mount,sysroot,crypto   |  [mount/sysroot失败的问题](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/mountsysroot.md)  |
| 3  |  开发者反馈 | 基础服务  | audit,log  | [audit写日志占满分区空间出现系统停机](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/audit.md)   |
| 4 | 开发者反馈 | 容器 | docker,umask | [docker配置容器umask值](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/umask.md) |
| 5 | 开发者反馈 | 容器 | docker,容器运行退出 | [docker容器运行一定时间后自动退出](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/docker.md) |
| 6 | 开发者反馈 | 基础服务 | kpatch，热补丁激活失败 | [制作内核热补丁，插入时dmesg提示缺少sssnic模块](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/sssnic.md) |
| 7 | 开发者反馈 | 计算 | 22.03-LTS,安装zabbix教程 | [openEuler-22.03-LTS安装zabbix教程](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/zabbix.md) |
| 8 | 开发者反馈 | 容器 | isulad,grpc,restful | [isulad使用grpc与rest直接调用服务方法](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/isulad.md) |
| 9 | 开发者反馈 | 存算 | lvm,存算 | [LVM标签损坏](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/lvm.md) |
| 10 | 开发者反馈 | 容器 | pvs、vgs | [pvs或vgs出现Unknown问题](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/pvs_vgs.md) |
| 11 | 开发者反馈 | 存算 | kdump、kexe-tools，vmcore | [kdump常见问题](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/crash.md) |
| 12 | 开发者反馈 | 安装 | efivars | [安装出现efivars报错](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/efivars.md) |
| 13 | 开发者反馈 | 安装 | 换标 | [换标常见问题](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/rebranding.md) |
| 14 | 开发者反馈 | 安装 | anaconda | [anaconda安装常见磁盘问题](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/anaconda.md) |
| 15 | 开发者反馈 | 安装 | 系统文件 | [系统文件恢复问题](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/sysfile.md) |
| 16 | 开发者反馈 | 容器 | docker服务启动 | [docker服务启动问题](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/faq/caselibrary/docker_1.md) |

---

---
url: /zh/docs/common/contribute/openeuler_contribution_guide.md
---
# openEuler开源社区贡献指南

# 前言

openEuler 是一个由全球贡献者共建的高效、稳定、安全的开源操作系统和社区，致力于打造持续创新的技术平台。本文以向 Gitee 上的 openEuler 社区贡献代码为例，系统指导贡献者完成从入门到实践的全流程，助力社区发展。

# 开源基础

## 初识开源社区

### 介绍

开源即开放源代码（Open Source Code），也称为源代码公开，指的是一种软件发布模式。开源社区又称开放源代码社区（Open Source Code Community or Open Source Community），一般由拥有共同兴趣爱好的人所组成，根据相应的开源软件许可证协议公布软件源代码的网络平台，同时也为网络成员提供一个自由学习交流的空间。从一定程度上讲，开源社区与开源产品的边界较为模糊，如Linux既指开源社区，由参与者共同管理，同时又指一套性能稳健的操作系统，其内核（Linux kernel）成为许多派生操作系统如Ubuntu的搭建基础。（百度百科）\
开源社区发展迅速，贡献者遍布全球且呈高速增长之势。据全球最大开源项目托管平台 GitHub 统计，2020年较上一年新增了1600万开发者用户，预计2025年开发者用户的数量将达到1亿。这其中，在 GitHub 的中国开发者数量及开源贡献度增长已成为全球最快。GitHub 预测到2030年中国开发者将成为全球最大的开源群体。（电子信息产业网）

### 主流开源社区

| 名称          | 简述                                                         | 网址                                                         |
| :------------ | :----------------------------------------------------------- | :----------------------------------------------------------- |
| Linux内核社区 | 由全球不同地方的开发人员组成，它以代码、风格、开发流程的高质量标准著称。 | <https://www.kernel.org/>           |
| Alpha         | Alpha处理器在Linux领域中很受欢迎，尤其用以处理高性能计算。   | <https://www.alpha.org/>             |
| PowerPC       | 提供对使用PowerPC微处理器的Mac计算机的支持，同时也会支持一些IBM的系统。 | <https://www.powerprogress.org/en/> |
| Apache        | 以讨论Linux/Unix类操作系统技术、软件开发技术、数据库技术和网络应用技术等为主的开源技术社区网站。其宗旨是给所有爱好Linux/Unix技术、开源技术的朋友提供一个自由、开放、免费的交流空间。 | <https://community.apache.org/> |
| SourceForge   | SourceForge.net (SF.net)是开源软件的开发者进行开发管理的集中式场所，也是全球最大开源软件开发平台和仓库，由VA Software提供主机，并运行SourceForge软件。 | <https://sourceforge.net/>         |
| Google Source | 谷歌的Android代码开源网站，包含了谷歌的各代Nexus的源码，这些源码都是跟随Android的版本演进。 | <https://android.googlesource.com>; <https://github.com/android/> |

### 开源社区的角色

> ![image](figures/contribution-guide-01.png)

### 贡献开源的意义

**巩固现有技能**：无论是撰写代码、设计用户界面、图形设计、撰写文档、亦或是组织活动，假如你有实践的愿望，你总能在开源项目中找到自己的位置。\
**建立声誉（职业口碑）**：由于开源下所有工作都是公开的，开源项目会是一个很好展示你实力的地方。\
**遇见知己**：开源项目一般都会有一个和谐、热心的社区。很多人便是通过技术研讨会亦或是聊天室的深入探讨建立起深厚友谊。\
**学习领导和管理的艺术**：开源为实践领导力和管理技能提供了很好的机会，比如解决冲突、组织团队、工作的优先级排列。\
**寻找导师/帮助他人**：和他人在一个共享的项目下工作，这意味着需要向他人解释清楚自己是如何做的，同理，也需要向他人求助，询问别人是如何做的。相互学习和彼此教学对于每位参与者都能满载而归。\
**鼓励做出改变**：在开源的世界里，贡献者不一定非得是拥有大量经验的人。在开源的项目中，没有那么多的顾忌，你只需要做就可以了，哪怕只是纠正网站一个小小的拼写错误。开源让人们在很舒服的状态做事，而这才是这个世界应有的体验。(开源社)

### 开源软件许可证协议

开源许可协议（License）是指开源社区为了维护作者和贡献者的合法权利，保证软件不被一些商业机构或个人窃取，影响软件的发展而开发的协议。License的条款由权利、义务、限制三种组成：\
● 权利：你可以使用该软件做什么事情\
● 义务：你使用这个软件时必须履行什么样的义务\
● 限制：你不能够做什么事情\
开源许可证是一类组合。对于大部分开源许可证，权利（红框）均授予，著作权人几乎不承担任何义务（蓝框），只是被授权人承担的义务又较多不同（绿框）。  (License遵从性指导书)
![image](figures/contribution-guide-02.png)

以下为openEuler docs SIG的许可证实例：
![image](figures/contribution-guide-03.png)

## 初识openEuler

### 介绍

openEuler 是一个开源、免费的 Linux 发行版平台，通过开放的社区形式与全球的开发者共同构建一个开放、多元和架构包容的软件生态体系。同时，openEuler也是一个创新平台，鼓励任何人在该平台上提出新想法、开拓新思路、实践新方案。\
openEuler的愿景是：通过社区合作，打造创新平台，构建支持多处理器架构、统一和开放的操作系统openEuler，推动软硬件生态繁荣发展。

### openEuler组织架构

![image](figures/contribution-guide-04.png)

### openEuler贡献流程

![image](figures/contribution-guide-05.png)

（openEuler官网）

# Gitee基础

Gitee 是开源中国社区2013年推出的基于 Git 的代码托管服务，目前已经成为国内知名的代码托管平台，致力于为国内开发者提供优质稳定的托管服务。Gitee 除了提供最基础的Git代码托管之外，还提供代码在线查看、历史版本查看、Fork、Pull Request（PR）、打包下载任意版本、Issue、Wiki 、保护分支、代码质量检测、PaaS项目演示等方便管理、开发、协作、共享的功能。本章介绍Gitee贡献相关概念和操作。 （Gitee网站）

## Issue提交指南

### Issue简介

**名词解释**：Issue是指一项待完成的工作，这个工作可以是问题、事务、需求和建议等。每一个Issue都包含该工作的所有信息和历史，便于后来的人了解该项工作的所有方面和过程。\
**来源和作用**：Issue的概念起源于客服部门，用户打电话反馈问题，客服就创建一个工单（ticket），后续每一个处理步骤、每一次和用户的交流都要更新到工单内，记录全部的过程信息，这就是Issue的前身。随着后来的不断扩展，逐步演变成制定和实施软件开发计划的全功能项目管理工具。\
openEuler社区直接使用Gitee提供的Issue跟踪和管理系统。

### Issue基本功能

Gitee的每一个仓库内都有一个Issue面板。

![image](figures/contribution-guide-06.png)

进入该面板，点击“+新建Issue”，就可以新建一个Issue，可以选择需求的类别。目前openEuler有任务、需求、缺陷、版本、翻译、CVE和安全问题等类别，每个类别对应一个提交模板，如下图所示。

![image](figures/contribution-guide-07.png)

### 需求类Issue提交和处理指导

不同类型的Issue都有各自完整的提交和处理流程，下面以需求类Issue为例。参考以下流程图。
![image](figures/contribution-guide-08.png)

**步骤 1** 新建需求类Issue（Issue状态：待办）
请进入需求对应的团队或项目的repository内，选择Issue面板，点击“新建Issue”。\
如果不确定该需求对应的团队或项目，请在community-issue中创建，会有社区的开发者帮助进行归属等信息的确认。\
在标题栏的单选下拉框将Issue类型设置成“需求”，系统会自动调出需求模板。\
在标题栏**简要描述需求的要点**。\
在详细说明框内说明需求的场景和价值。\
*请注意：清晰完整的描述有助于团队成员理解，并被更快的接受和排入开发计划。*

**步骤 2** 团队成员审核Issue（Issue状态——通过：进行中；拒绝：已拒绝 ）\
**2.1** 审核后接纳
团队成员（maintainer或者committer）审核后认为可以接纳该需求，则由审核人补充接纳需求的相关信息，并包含：\
● 检查并设置该需求所属的项目。\
● 设置该需求建议合入的里程碑信息（规划版本信息）。\
● 设置该需求的优先级标签，请在标签栏选择“feature:High”、“feature:Medium”或者"feature:Low"。\
完成以上的信息以后，请将该Issue的“当前状态”调整成“进行中”（**进入步骤3**）。\
**2.2** 描述不清晰挂起\
团队成员审核后认为描述的信息不清晰，可以在该Issue的评论区留言或在邮件列表中讨论，让提交人继续补充相关信息。\
● 如果在一个月内提交人未及时补充相关信息，则系统会自动关闭该问题（**跳到步骤4**）。\
● Issue提交人补充信息后，可以通过评论让团队成员审核（**跳到步骤2**）。\
**2.3** 审核后不接纳\
团队成员审核后，由于需求价值不高等原因认为暂不接纳，可以在评论区留言或在邮件列表中讨论说明原因。确认后将Issue的“当前状态”调整成“已拒绝”（**跳到步骤4**）。

**步骤 3** 认领或分派Issue（Issue状态：开启）\
已经进入开发阶段的需求，可以主动认领，也可以由团队成员分配。可以在评论框内输入/assign来把Issue分配给自己，或分配给其他人。

**步骤 4** **关闭Issue**，关闭Issue有三种情况：\
需求完成后关闭，可以由认领人手工修改状态，也可以通过关联PR后，由PR审核通过后系统自动关闭。\
需求被拒接关闭，由审核人手工修改状态。\
需求超期后关闭，由系统自动根据需求停滞的时间进行超期关闭的操作。\
（openEuler Issue提交指南）

## PR提交指南

Pull Request（PR）是贡献者修改源代码后，请求目标仓库采纳该修改时提交的一种请求。

### PR提交前验证

提交PR是对项目上的master主干的一次合入申请，为保证合入质量，这个动作是需要小心谨慎的。在提交PR之前，请先完成本地验证，以便在一定程度上保证在提交PR后的持续集成测试的通过。由于不同项目的本地验证方法不同，可以参考此章节内容以获取帮助。

### PR提交

**步骤 1** 提交PR请求。\
在Gitee上提交PR的具体操作，请参考下文Gitee工作流说明。为了更快得到响应，可以给PR添加标签，或提供更详细的信息，这里需要特别提示的是：\
● **关联Issue**：如果提交的PR是针对某个Issue的，请在提交的描述框内添加“#”字符，此时机器人会自动关联出当前存在的Issue，你可以通过此种方式快速链接到关联的Issue。\
● **标注优先级**：可以在创建PR的时候，选择PR的优先级。或者在评论区通过/priority high给PR添加高优先级标签。\
● **标注是解决bug的合入**：可以在描述框通过输入/kind bug来标注该PR是合入解决问题的代码，以便于大家更快的回应你的PR请求。\
● **标注所属SIG**：为了方便查找，也可以在描述框通过输入sig sig-name来标识该PR所属的SIG。

**步骤 2** 分配评审人。\
提交PR以后，社区机器人会自动分配评审人，你也可以指定评审人。指定评审人有两种方式，可以在创建PR的时候，在右侧的下拉框中选择评审人。也可以在评论框中输入/assign @reviewer把该PR分配给对应的人。如果想把PR提交给项目的核心成员评审，以便于更快的获得批准，可以有两种方式获取到核心成员的信息：\
**方式一**：该Repository的owners文件（该文件通常在该repository的根目录下）中查看，此文件保存的是所有该Repository的评审人列表。\
**方式二**：可以到该项目所属的SIG的首页内查看README.md文件，此文件会列出该SIG的负责人，所有项目以及项目的负责人。

**步骤 3** 自动化测试。\
如果您提交成功以后，看到PR上有openeuler-cla/no的标签，说明您还未和社区签署贡献者协议，请您先按照社区机器人的提示，完成贡献者协议的签署。具体操作详见后文。\
提交成功以后，社区机器人会启动自动化测试，为了避免浪费评审人时间，通常只有自动化测试通过的PR，评审人才会参与评审。你可以在PR的下方看到自动化测试的结果。\
如果自动化测试失败，您可以通过“Build Details”查看失败的原因。

> ![image](figures/contribution-guide-09.png)

点击“Build Details”，可以看到具体的log信息。然后可以在里面搜索“Error”，快速的定位到错误的信息。\
修改后，你可以在评论框输入/retest命令，让社区机器人重新发起一次自动化测试。

**步骤 4** PR审核。\
如果审核人通过你的PR，会在评论区添加/lgtm和/approve，以表示对本次PR提交的认同。\
审核人可以在评论区发表意见，也可以在审核文件的时候，在发现问题处添加审核意见。无论哪种方式，都会在评论区显示出来。区别是，后者的评论会显示出“代码评论”，你可以通过“详情”查看评论具体指向的出处。\
为了表示对评审人意见的尊重，如果对意见有异议，请回复该意见说明原因；如果接纳评审人意见，也请做出简单的回应，便于确认后续的提交是否已按照所有接纳意见完成修改。\
**请注意，在使用/approve前至少要有一个/lgtm。**

### 未完成PR标记

如果想在PR请求完成之前先征求大家的意见，有两种方法可以实现此目的：

1. 可以在评论区添加hold或hold-cancel标签
2. 可以在PR请求的标题中添加WIP或\[WIP]前缀

当存在这两个标签时，将不会考虑合并你的PR请求。

（openEuler PR提交指南）

## Gitee贡献工作流

> ![image](figures/contribution-guide-10.png)

**步骤 1** 开展工作流前的准备。\
安装Git：请先确保你的电脑上已经安装了Git软件。
在开展Gitee的工作流之前，需要先在openEuler的代码托管平台的上找到感兴趣的Repository。如果还未找到对应的Repository，请参考此章节的内容。

**步骤 2** 从云上fork代码分支。\
找到并打开对应的Repository的首页。\
点击右上角的 Fork 按钮，按照指引，建立一个属于个人的云上fork分支。

> ![image](figures/contribution-guide-11.png)\
> ![image](figures/contribution-guide-12.png)

**步骤 3** 把fork分支复制到本地。\
请按照以下的复制过程将Repository内的代码下载到你的在计算机上。

1. 创建本地工作目录：需要创建本地工作目录，以便于本地代码的查找和管理。

   `mkdir /YOUR_PATH/src/gitee.com/${your_working_dir}`

2. 完成git上用户名和邮箱的全局配置（如果之前已经完成过此项配置，请忽略）。\
   把git上的 user 设置成你Gitee的个人名称：

   `git config --global user.name "your Gitee Name"`

   配置你的git邮箱：

   `git config --global user.email "email@your_Gitee_email"`

3. 完成SSH公钥注册（如果没有完成此注册，每次都要重新输入账户和密码）。\
   ① 生成ssh公钥。

   `ssh-keygen -t rsa -C "email@your_Gitee_email"`

   `cat ~/.ssh/id_rsa.pub`

   ② 登录你个人的远程仓库网站Gitee账户并添加你的ssh公钥。\
   请在Gitee网页点击右上角的“个人头像”进入个人Gitee账户，并点击个人头像下的“个人设置”，进入个人设置页面。在“个人设置->安全设置”下，点击“SSH公钥”，在“添加公钥”内把cat命令获取到的ssh公钥添加进去。\
   ③ 在个人电脑上完成Gitee在SSH上的登记。

   `ssh -T git@gitee.com`

   > ![image](figures/contribution-guide-13.png)

   **步骤 4** 克隆远程仓库到本地。\
   ● 请注意openEuler有几个组织，请确认你所下载的远程仓库的组织名称\
   ● 可以在repository内复制远程仓库的拷贝地址，得到$remote\_link：

   > ![image](figures/contribution-guide-14.png)

   ● 在本地电脑执行拷贝命令：\
   把远程 fork 仓库克隆到本地

   `git clone https://gitee.com/$user_name/$repository_name`

   设置本地工作目录的 upstream 源（被 fork 的上游仓库）

   `git remote add upstream https://gitee.com/openeuler/$repository_name`

   设置同步方式

   `git remote set-url --push upstream no_push`

   **步骤 5** 拉分支（可选）。

   `git fetch upstream`

   `git checkout master`

   `git rebase upstream/master`

   从这里拉分支:

   `git checkout -b work`

   然后在 work 分支上编辑和修改代码。

   **步骤 6** 保持你的分支与master同步。

   `While on your work branch`

   `git fetch upstream`

   `git rebase upstream/master`

   执行merge的时候，请不要使用 git pull 替代上面的 fetch/rebase。因为这种方式会使提交历史变得混乱，并使代码难以理解。\
   **步骤 7** 在本地工作目录提交变更。

   提交你的变更

   `git add .`

   `git commit -m "提交原因"`

   **步骤 8** 在Gitee上创建一个 pull request。

   1. 访问你在 <https://gitee.com/$user/openEuler>的页面。
   2. 把你的分支选到提交使用的 work 分支上，点击+ Pull Request 。具体位置如下图所示：
   3. 在创建新PR界面，确认源分支和目标分支，选择创建。

   > ![image](figures/contribution-guide-15.png)

   **步骤 9** 查看和回应代码审查意见。\
   你提交PR申请后，PR被分配给一个或多个检视者。这些检视者将进行检视，以确保提交的正确性，不仅包括代码的正确，也包括注释和文档等。\
   (Gitee工作流说明)

## Git与 VSCode基础

### Git

Git是一个免费的、开源的分布式版本控制系统，可以有效、高速地处理项目版本管理。

#### 基本概念

**工作区**：就是你在电脑里能看到的目录。\
**暂存区**：英文叫 stage 或 index。一般存放在 .git 目录下的 index 文件（.git/index）中，所以我们把暂存区有时也叫作索引（index）。\
**版本库**：工作区有一个隐藏目录 .git，这个不算工作区，而是 Git 的版本库。

> ![image](figures/contribution-guide-16.png)

● 图中左侧为工作区，右侧为版本库。在版本库中标记为 "index" 的区域是暂存区（stage/index），标记为 "master" 的是 master 分支所代表的目录树。\
● 图中我们可以看出此时 "HEAD" 实际是指向 master 分支的一个“游标”。所以图示的命令中出现 HEAD 的地方可以用 master 来替换。\
● 当对工作区修改（或新增）的文件执行 git add 命令时，暂存区的目录树被更新，同时工作区修改（或新增）的文件内容被写入到对象库中的一个新的对象中，而该对象的ID被记录在暂存区的文件索引中。\
● 当执行提交操作（git commit）时，暂存区的目录树写到版本库（对象库）中，master 分支会做相应的更新。即 master 指向的目录树就是提交时暂存区的目录树。\
● 当执行 git reset HEAD 命令时，暂存区的目录树会被重写，被 master 分支指向的目录树所替换，但是工作区不受影响。\
● 当执行 git rm --cached \<file> 命令时，会直接从暂存区删除文件，工作区则不做出改变。

#### Git 创建仓库

● Git 使用 git init 命令来初始化一个 Git 仓库，Git 的很多命令都需要在 Git 的仓库中运行，所以 git init 是使用 Git 的第一个命令。在执行完成 git init 命令后，Git仓库会生成一个.git 目录，该目录包含了资源的所有元数据，其他的项目目录保持不变。\
如果当前目录下有几个文件想要纳入版本控制，需要先用 git add 命令告诉Git开始对这些文件进行跟踪，然后提交：

`$ git add *.c`

`$ git add README`

`$ git commit -m '初始化项目版本'`

以上命令将目录下以 .c 结尾及README文件提交到仓库中。

> 注： 在Linux系统中，commit 信息使用单引号 '，在Windows系统，commit信息使用双引号 "。

所以在 git bash 中 git commit -m '提交说明' 这样是可以的，在Windows命令行中就要使用双引号 git commit -m "提交说明"。\
● 我们使用 git clone 从现有 Git 仓库中拷贝项目。\
克隆仓库的命令格式为：

`git clone <repo名>`

如果我们需要克隆到指定的目录，可以使用以下命令格式：

`git clone <repo名> <directory目录>`

参数说明：\
repo: Git 仓库。\
directory: 本地目录。\
比如，要克隆 Ruby 语言的 Git 代码仓库 Grit，可以用下面的命令：

`$ git clone git://github.com/schacon/grit.git`

执行该命令后，会在当前目录下创建一个名为grit的目录，其中包含一个 .git 的目录，用于保存下载下来的所有版本记录。\
如果要自己定义要新建的项目目录名称，可以在上面的命令末尾指定新的名字：

`$ git clone git://github.com/schacon/grit.git mygrit`

● 我们使用如下命令设置提交代码时的用户信息。

`$ git config --global user.name xxx`

`$ git config --global user.email xxx@xxx.com`

如果去掉 --global 参数只对当前仓库有效。

#### Git 基本操作

Git常用的是以下6个命令：**git clone**、**git push**、**git add** 、**git commit**、**git checkout**、**git pull**

> ![image](figures/contribution-guide-17.png)

一个简单的操作步骤：

`$ git init`

`$ git add .`

`$ git commit`

● git init - 初始化仓库。\
● git add . - 添加文件到暂存区。\
● git commit - 将暂存区内容添加到仓库中。
(Git菜鸟教程)

#### Git常用命令

##### 创建仓库命令

| 命令      | 说明                                   |
| :-------- | :------------------------------------- |
| git init  | 初始化仓库                             |
| git clone | 拷贝一份远程仓库，也就是下载一个项目。 |

##### 提交与修改命令

| 命令       | 说明                                     |
| :--------- | :--------------------------------------- |
| git add    | 添加文件到暂存区。                       |
| git status | 查看仓库当前的状态，显示有变更的文件。   |
| git diff   | 比较文件的不同，即暂存区和工作区的差异。 |
| git commit | 提交暂存区到本地仓库。                   |
| git reset  | 回退版本。                               |
| git rm     | 将文件从暂存区和工作区中删除。           |
| git mv     | 移动或重命名工作区文件。                 |

##### 提交日志

| 命令             | 说明                                   |
| :--------------- | :------------------------------------- |
| git log          | 查看历史提交记录。                     |
| git blame \<file> | 以列表形式查看指定文件的历史修改记录。 |

##### 远程操作

| 命令       | 说明                 |
| :--------- | :------------------- |
| git remote | 远程仓库操作。       |
| git fetch  | 从远程获取代码库。   |
| git pull   | 下载远程代码并合并。 |
| git push   | 上传远程代码并合并。 |

#### Git 特殊操作

##### 处理冲突提交

如果发现提交的PR带有以下的标记，说明你提交的PR和本地存在冲突，需要处理冲突。

> ![image](figures/contribution-guide-18.png)

**步骤 1** 先将分支切换到master上，并完成master的rebase。

`git checkout master`

`git fetch upstream`

`git rebase upstream/master`

**步骤 2** 再将分支切换到您使用的分支上，并开始rebase。

`git checkout yourbranch`

`git rebase master`

**步骤 3** 此时你可以在git上看到冲突的提示，你可以通过vi等工具查看冲突。

**步骤 4** 解决冲突以后，再把修改提交上去。

`git add .`

`git rebase --continue`

`git push -f origin yourbranch`

##### 合并提交

如果你提交了一个PR以后，根据检视意见完成修改并再次提交了PR，不想让审阅者看到多次提交的PR，因为这不便于继续在检视中修改，那么可以合并提交的PR。合并提交的PR是通过压缩commit来实现的。\
**步骤 1** 现在本地分支上查看日志。

`git log`

**步骤 2** 然后把顶部的n个提交记录聚合到一起进入，注意n是一个数字。

`git rebase -i HEAD~n`

把需求压缩的日志前面的pick都改成s，s是squash的缩写。注意必须保留一个pick，如果将所有的pick都改成了s就没有合并的目标了，会发生错误。

**步骤 3** 修改完成以后，按ESC键，再输入:wq，会跳出一个界面，问你是否进入编辑提交备注的页面，输入e以后，进入合并提交备注的页面。请把需要合并的备注都删掉，只保留合并目标的备注，再按ESC键，输入:wq保存退出即可。

**步骤 4** 最后完成提交。

`git push -f origin yourbranch`

**步骤 5** 回到gitee上的PR提交页面查看，您就可以看到之前的提交已经合并了。\
详细请见：<https://www.runoob.com/git/git-workflow.html>。\
（Gitee工作流说明）

### VS Code

Visual Studio Code（简称 VS Code）是由微软开发的轻量级代码编辑器，支持包括 Markdown 在内的多种语言和格式，内置了命令行工具和 Git 版本控制系统。VSCode 中很多操作可以通过软件内命令行来使用，呼出软件内命令行的默认快捷键是 F1 或 Ctrl+Shift+P，请牢记。本文内有>前缀的命令是 VS Code 命令行的命令，否则是 Git Bash 命令。

> ![image](figures/contribution-guide-19.png)

#### 软件与扩展安装

1. 安装并配置好 Git。
2. 下载安装 VS Code。
3. 可安装中文语言包，语言包在扩展商店中以扩展的形式提供。要打开扩展商店：点击左侧边栏的扩展图标，或快捷键 Ctrl+Shift+X，或呼出命令行，输入>extensions: install extensions 以命令打开。\
   注：VSCode 软件内命令行一般不需要输入完整命令，可输入>extensions 或 >install extensions 之后用键盘上下键选择命令补全。可以输入中文命令描述。\
   点击\[安装]按钮即可安装，点击项目可进入商店页面查看详情。
4. 安装 GitLens 扩展，用于 Git 相关操作。

> ![image](figures/contribution-guide-20.png)

#### VSCode界面和Git

为方便解说UI内容，先将Gitee仓库用VS Code打开，这里假定你已经把 openEuler/docs仓fork到了自己的远程仓，并且 >git: clone 或用 git bash clone 到了本地计算机上：\
打开本地仓：在 VS Code 中用 >git: clone 克隆完成后会提示是否打开。如果是用 Git Bash 克隆的本地仓：在Windows资源管理器中打开本地仓所在文件夹，右键菜单通过Code打开，或在VSCode中点击文件>打开文件夹，或 >File: Open Folder。打开界面如下图。

> ![image](figures/contribution-guide-21.png)

图中已经点击预览 README.md。单击是预览文件，文件名在打开的编辑器中显示为斜体；双击是打开文件，文件名不显示斜体。预览时做任何改动也会打开文件。\
右下角显示 Markdown 处可以点击更改语言模式，如果编辑器没有自动检测到Markdown语法可以手动选择。\
查看（预览或打开）文件时，时间线会显示当前文件的提交历史，底栏会显示光标当前所在行的作者和提交时间。点击时间线中任意提交历史可以比较查看当前文件在本次提交中的改动。

> ![image](figures/contribution-guide-22.png)

上图在 README.md 中做了改动但没有保存，文件名前会加上圆点图标，并且有文字提示 1 个未保存。\
如要对比本地的两个文件，在文件树（即图中 DOCS）或打开的编辑器中右键点击一个文件选择以进行比较，再右键另一个文件与已选项目进行比较，或者按住 Ctrl 选择两个文件将已选项进行比较。改动后未保存的文件也可以与已保存的版本比较。\
文件树中可以进行新建文件/文件夹、复制粘贴、重命名等常规操作。在资源管理器页，按 Ctrl+E 可以在文件树中以文件名或路径进行搜索。\
在编辑器中更改并保存文件后，有改动的文件会出现在更改中。点击这里的文件可以显示对比，即本地最新的文件与最后一次拉取的文件对比。\
在更改文件树中的文件名右侧点击 + 号将文件放入暂存区。等效于对当前文件使用 >git: stage changes 命令。\
也可以 Ctrl+左键多选文件后点击 + 批量放入暂存区。命令 >git:stage all changes 可以把所有更改的文件放入暂存区。\
点击暂存区中文件名右侧的 - 可以移出暂存区。\
要提交暂存区中的文件，按源代码管理项下输入框中的提示，输入提交信息按 Ctrl+Enter 提交。等效于 >git: commit staged。\
未暂存的文件可以使用 >git: commit 或 >git: commit all 提交，确认后文件可跳过暂存直接提交。\
提交的文件存放于本地库中，还未发布到远端库。可以使用 >git: undo last commit 撤消最后一次提交。\
提交后，窗口左下角会显示提交的数量，可以点击此处发布到远端库（同时会拉取远端库的新提交）。如果要只发布而不拉取，可以使用 >git: push。如果显示远端库有新提交，先拉取至本地，以免提交后产生冲突。

> ![image](figures/contribution-guide-23.png)

**COMMITS** 项显示当前分支中的所有提交历史。点击提交项可以展开显示更改的文件，点击文件可以显示与上个版本比较的改动。

> ![image](figures/contribution-guide-24.png)

FILE HISTORY 即编辑器中当前打开文件的提交历史，类似于资源管理器页中的时间线。\
BRANCHES 显示本地分支中的提交。参见下文 REMOTES。\
REMOTES 显示远端库中所有分支的提交。右键选择 Switch to Branch... 切换到另一分支，并将这个分支保存到本地，显示在 BRANCHES 中。点击左下此处也可以切换分支。

> ![image](figures/contribution-guide-25.png)\
> STASHES 储藏区。修改后的文件如果不满意，不想放在本次提交里，或者有未提交的更改但想要切换到其他分支进行操作，可以 >git: stash 先放入储藏区。\
> TAGS 列出每个标签对应的提交。\
> SEARCH & COMPARE 提供搜索提交和比较分支的功能。Search Commits... 可以选择按消息内容、作者、文件等搜索。Compare References... 可以选中两个分支进行比较，提交比较和文件树比较。文件树比较会显示文件路径变化，以及内容有差异的同名文件。\
> ![image](figures/contribution-guide-26.png)

创建图中的比较时，先选择了 master 分支（显示在右侧），后选择了 stable2-21.09 分支（显示在左侧）。\
文件树中 - 号表示在右侧分支中缺少的文件，+ 号表示右侧分支中多出的文件，± 表示同名文件内容有差异，点击打开对比视图。

#### 保持本地提交记录和远端上游仓同步

为防止创建的Pull Request与上游仓内容冲突，强烈建议每次提交前将上游仓的 commit 拉取到当前分支。

**步骤 1** 在VS Code中添加上游仓。以主仓openEuler/docs为例，在网页端克隆/下载按钮复制SSH地址 git@gitee.com:xxxx.git。运行以下命令添加主仓，显示在源代码管理页的REMOTE下:

`git remote add upstream git@gitee.com:xxxx.git`

upstream 为自定义名称

**步骤 2** 抓取主仓的commits。

`git fetch upstream`\
`git fetch upstream`

**步骤 3** 将主仓的commits合并到本地的分支。以 master 分支为例：

`git checkout master`

↑切换到 master 分支，如已在 VS Code 中切换则省略

`git merge upstream/master`

**步骤 4** 解决源代码管理中提示的冲突（如有），然后提交自己的改动。

#### 合并commits

为保持提交记录简洁（以及满足部分仓库的要求），一个 PR 应当只包含一个 commit。如果多个 commits已经提交到远端库，靠回退版本 (git reset)来合并 commits。\
假设**源代码管理 > COMMITS** 当前的 commit 记录如下：

```text
   YOUR_COMMIT_3    
   YOUR_COMMIT_2 
   YOUR_COMMIT_1 
   OTHERS_COMMIT_2
   OTHERS_COMMIT_1 
```

要合并 YOUR\_COMMIT\_，（记录下 commit message）右键点击 OTHERS\_COMMIT\_2 选择 Reset Current Branch to Commit...，然后选择 Soft Reset，即可回退到 OTHERS\_COMMIT\_2 的版本，三个 YOUR\_COMMIT\_ 的改动会回到上方更改区域，重新填写commit message 强制提交即可。\
版本回退后，因为本地的版本早于远端库的版本，必须**使用 force push 覆盖远端库**：打开 VS Code 选项，搜索 Allow Force Push 项并勾选，然后按 F1 呼出命令行，输入 push force 等类似关键字，选择相关选项。

#### 推荐扩展

● Markdown Editor (zaaack.markdown-editor)，支持所见即所得模式和分屏预览模式的 Markdown 编辑器。\
● Markdown Preview Enhanced (shd101wyy.markdown-preview-enhanced)，增强 Markdown 预览功能，包含 TOC 自动生成功能。快捷方式同样为 Ctrl+Shift+V，覆盖 VSCode 自带的预览查看器。安装后如果需要使用自带的预览查看器，在编辑器窗口顶端的文件标签上右键 -> 打开预览。自带的查看器中，双击句子可以跳转到原文相应句子。\
自动生成 TOC：>markdown preview enhanced: create toc （需要保持预览窗口打开）。保存时自动更新 TOC。
● GitHub Markdown Preview (bierner.github-markdown-preview)，或仅安装其中的核心扩展 Markdown Preview GitHub Styling (bierner.markdown-preview-github-styles)，以 GitHub 格式显示 Markdown 预览，效果接近Gitee网页。\
● Code Spell Checker (streetsidesoftware.code-spell-checker)，自然语言拼写检查扩展，除英语外还有多种语言可选。\
● CJK Word Handler (sharzyl.cjk-word-handler)。VSCode 默认以空格、","、"." 等英文符号作为分隔符，导致整句中文被识别为一个整词，使用 Ctrl+←/→ 相关操作时非常不便。这个扩展可以让 VSCode 支持中文分词逻辑。
● Bookmarks (alefragnani.bookmarks)，以行为坐标添加书签。\
（VSCode for openEuler Docs Globalization）。

## 参考链接

<https://kaiyuanshe.cn/article/github/how-to-participate-in-open-source/>\
<https://docs.qq.com/doc/DSnJVSmhDcHhrV25L>\
<https://www.openeuler.org/zh/>\
<https://gitee.com/openeuler/community/blob/master/zh/contributors/issue-submit.md>\
<https://gitee.com/openeuler/community/blob/master/zh/contributors/pull-request.md>\
<https://gitee.com/openeuler/community/blob/master/zh/contributors/Gitee-workflow.md>\
<https://www.runoob.com/git/git-tutorial.html>\
<https://gitee.com/openeuler/G11N/blob/master/learning-materials/open-source-basics/vscode-for-g11n.md>\
<https://gitee.com/openeuler/globalization/blob/master/openeuler-g11n-contribution-guide.md>\
<https://gitee.com/openeuler/G11N/blob/master/learning-materials/open-source-basics/markdown-basics.md>

---

---
url: /en/docs/common/faq/server/installation_faq1.md
---
# OS Installation FAQ 1

## 1. openEuler Fails to Start After It Is Installed to the Second Drive

### Symptom

The OS is installed on the second drive  **sdb**  during the installation, causing startup failure.

### Possible Causes

When openEuler is installed to the second drive, MBR and GRUB are installed to the second drive  **sdb**  by default. The following two situations may occur:

1. openEuler installed on the first drive is loaded and started if it is complete.
2. openEuler installed on the first drive fails to be started from hard drives if it is incomplete.

The preceding two situations occur because the first drive  **sda**  is booted by default to start openEuler in the BIOS window. If openEuler is not installed on the  **sda**  drive, system restart fails.

### Solutions

This problem can be solved using either of the following two methods:

* During the openEuler installation, select the first drive or both drives, and install the boot loader on the first drive  **sda**.
* After installing openEuler, restart it by modifying the boot option in the BIOS window.

## 2. openEuler Enters Emergency Mode After It Is Started

### Symptom

openEuler enters emergency mode after it is powered on.

![fig](./figures/en-us_image_0229291264.jpg)

### Possible Causes

Damaged OS files result in drive mounting failure, or overpressured I/O results in drive mounting timeout (threshold: 90s).

An unexpected system power-off and low I/O performance of drives may also cause the problem.

### Solutions

1. Log in to openEuler as the  **root**  user.
2. Check and restore files by using the file system check (fsck) tool, and restart openEuler.

   > ![fig](./public_sys-resources/icon-note.gif) **NOTE:**
   > The fsck tool checks and maintains inconsistent file systems. If the system is powered off or a drive is faulty, run the  **fsck**  command to check file systems. Run the  **fsck.ext3 -h**  and  **fsck.ext4 -h**  commands to view the usage method of the fsck tool.

If you want to disable the timeout mechanism of drive mounting, add  **x-systemd.device-timeout=0**  to the  **etc/fstab**  file. For example:

```sh
#
# /etc/fstab
# Created by anaconda on Mon Sep 14 17:25:48 2015
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/openEuler-root / ext4 defaults,x-systemd.device-timeout=0 0 0
UUID=afcc811f-4b20-42fc-9d31-7307a8cfe0df /boot ext4 defaults,x-systemd.device-timeout=0 0 0
/dev/mapper/openEuler-home /home ext4 defaults 0 0
/dev/mapper/openEuler-swap swap swap defaults 0 0
```

## 3. openEuler Fails to Be Reinstalled When an Unactivated Logical Volume Group Exists

### Symptom

After a drive fails, openEuler fails to be reinstalled because a logical volume group that cannot be activated exists in openEuler.

### Possible Causes

During the installation of openEuler, a logical volume group cannot be activated.

### Solutions

Before reinstalling openEuler, restore the abnormal logical volume group to the normal status or clear it. For example:

* Restore the logical volume group.
  1. Run the following command to clear the active status of the abnormal logical volume group to ensure that the error message "Can't open /dev/sdc exclusively mounted filesystem" is not displayed:

     ```sh
      vgchange -a n testvg32947
     ```

  2. Run the following command to recreate a physical volume based on the backup file:

     ```sh
     pvcreate --uuid JT7zlL-K5G4-izjB-3i5L-e94f-7yuX-rhkLjL --restorefile /etc/lvm/backup/testvg32947 /dev/sdc
     ```

  3. Run the following command to restore the logical volume group information:

     ```sh
     vgcfgrestore testvg32947
     ```

  4. Run the following command to reactivate the logical volume group:

     ```sh
      vgchange -ay testvg32947
     ```

* Run the following commands to clear the logical volume group:

  ```sh
  vgchange -a n testvg32947
  vgremove -y testvg32947
  ```

## 4. An Exception Occurs During the Selection of the Installation Source

### Symptom

After the installation source is selected, the message "Error checking software selection" is displayed.

### Possible Causes

This is because the software package dependency in the installation source is abnormal.

### Solutions

Check whether the installation source is abnormal. Use the new installation source.

## 5. kdump Service Fails to Be Enabled

### Symptom

Run the  **systemctl status kdump**  command. The following information is displayed, indicating that no memory is reserved.

![fig](./figures/en-us_image_0229291280.png)

### Possible Causes

The kdump service requires the system to reserve memory for running the kdump kernel. However, the system does not reserve memory for the kdump service. As a result, the kdump service cannot be started.

### Solutions

For the scenario where the OS has been installed

1. Add  **crashkernel=1024M,high**  to  **/boot/efi/EFI/openEuler/grub.cfg**.
2. Restart the system for configuration to take effect.
3. Run the following command to check the kdump status:

   ```sh
   systemctl status kdump
   ```

   If the following information is displayed, the kdump status is  **active**, indicating that the kdump service is enabled. No further action is required.

   ![fig](./figures/en-us_image_0229291272.png)

### Parameter Description

The following table describes the parameters of the memory reserved for the kdump kernel.

**Table  1**  crashkernel parameters

| Kernel Boot Parameter | Description | Default Value | Remarks |
| --- | --- | --- | --- |
| crashkernel=X | Reserve X of the physical memory for kdump when the physical memory is less than 4 GB. | None. You can adjust the value as required. | This configuration method is used only when the memory is less than 4 GB. Ensure that the continuous available memory is sufficient. |
| crashkernel=X@Y | Reserve X of the memory at the start address Y for kdump. | None. You can adjust the value as required. | Ensure that the X of the memory at the start address Y is not reserved for other modules. |
| crashkernel=X,high | Reserve 256 MB of the physical memory for kdump when the physical memory is less than 4 GB, and X of the physical memory for kdump when the physical memory is greater than or equal to 4 GB. | None. You can adjust the value based as required. The recommended value is **1024M,high**. | Ensure that 256 MB of the memory is reserved for continuous use when the physical memory is less than 4 GB and X of the memory is reserved when the physical memory is greater than or equal to 4 GB. The actual reserved memory size equals 256 MB plus X. |
| crashkernel=X,lowcrashkernel=Y,high | Reserve X of the physical memory for kdump when the physical memory is less than 4 GB and Y of the physical memory for kdump when the physical memory is greater than or equal to 4 GB. | None. You can adjust the value as required. | Ensure that X of the memory is reserved for continuous use when the physical memory is less than 4 GB and Y of the memory is reserved when the physical memory is greater than or equal to 4 GB. The actual reserved memory size equals X plus Y. |

## 6. Fails to Select Only One Drive for Reinstallation When openEuler Is Installed on a Logical Volume Consisting of Multiple Drives

### Symptom

If openEuler is installed on a logical volume consisting of multiple drives, an error message will be displayed as shown in  [Figure 1](#fig115949762617)  when you attempt to select one of the drives for reinstallation.

**Figure  1**  Error message
![fig](./figures/Configuration_error_prompt.png)

### Possible Causes

The previous logical volume contains multiple drives. If you select one of the drives for reinstallation, the logical volume will be damaged.

### Solutions

The logical volume formed by multiple drives is equivalent to a volume group. Therefore, you only need to delete the corresponding volume group.

1. Press  **Ctrl**+**Alt**+**F2**  to switch to the CLI and run the following command to find the volume group:

   ```sh
   vgs
   ```

   ![fig](./figures/en-us_image_0231657950.png)

2. Run the following command to delete the volume group:

   ```sh
   vgremove euleros
   ```

3. Run the following command to restart the installation program for the modification to take effect:

   ```sh
   systemctl restart anaconda
   ```

   > ![fig](./public_sys-resources/icon-note.gif) **NOTE:**
   > You can also press  **Ctrl**+**Alt**+**F6**  to return to the GUI and click  **Refresh**  in the lower right corner to refresh the storage configuration.

## 7. openEuler Fails to Be Installed on an x86 PM in UEFI Mode due to Secure Boot Option Setting

### Symptom

During the installation of openEuler on an x86 PM in UEFI mode, the system stays at the "No bootable device" page and the installation cannot continue because  **secure boot**  is set to  **enabled**  (by default, it is set to  **disabled**), as shown in  [Figure 2](#fig115949762618).

**Figure  2**  Dialog box showing "No bootable device"&#x20;
![fig](./figures/No-bootable-device.png)

### Possible Causes

After  **Secure Boot**  is set to  **Enabled**, the mainboard verifies the boot program and OS. If the boot program and OS are not signed using the corresponding private key, they cannot pass the authentication of the built-in public key on the mainboard.

### Solutions

Access the BIOS, set  **Secure Boot**  to  **Disabled**, and reinstall the openEuler.

1. During the system startup, press  **F11**  and enter the password  **Admin@9000**  to access the BIOS.

   ![fig](./figures/BIOS.png)

2. Choose  **Administer Secure Boot**.

   ![fig](./figures/security.png)

3. Set  **Enforce Secure Boot**  to  **Disabled**.

   ![fig](./figures/select.png)

   > ![fig](./public_sys-resources/icon-note.gif) **NOTE:**
   > After  **Enforce Secure Boot**  is set to  **Disabled**, save the settings and exit. Then, reinstall the system.

## 8. pmie\_check Is Reported in the messages Log During openEuler Installation

### Symptom

During the OS installation, if you click  **Server > Performance tool**, PCP is installed. After the OS is installed and restarted, an error "pmie\_check failed in /usr/share/pcp/lib/pmie" is displayed in the  **/var/log/messages**  log.

### Possible Causes

anaconda does not support the installation of SELinux policy module in the chroot environment. During the pcp-selinux installation, the postin script fails to execute the PCP-related SELinux policy module. As a result, an error is reported after the OS is restarted.

### Solutions

After the OS is installed and restarted, perform either of the following two operations:

1. Install SElinux policy module pcpupstream.

   ````sh
   /usr/libexec/pcp/bin/selinux-setup /var/lib/pcp/selinux install "pcpupstream"

   ```sh

   ````
2. Reinstall pcp-selinux

   ````sh
   sudo dnf reinstall pcp-selinux

   ```sh
   ````

## 9. Installation Fails when a User Selects Two Drives with OS Installed and Customizes Partitioning

### Symptom

During the OS installation, the OS has been installed on two drives. In this case, if you select one drive for custom partitioning, and click **Cancel** to perform custom partitioning on the other drive, the installation fails.

![fig](./figures/cancle_drive.png)

![fig](./figures/custom_paratition.png)

### Possible Causes

A user selects a drive for partitioning twice. After the user clicks **Cancel** and then selects the other drive, the drive information is incorrect. As a result, the installation fails.

### Solutions

Select the target drive for custom partitioning. Do not frequently cancel the operation. If you have to cancel and select another drive, you are advised to reinstall the OS.

### Learn More About the Issue at

<https://gitee.com/src-openeuler/anaconda/issues/I29P84?from=project-issue>

## 10. vmcore Fails to Be Generated by Kdump on the PM with LSI MegaRAID Card Installed

### Symptom

After the Kdump service is deployed, kernel breaks down due to the manual execution of the  **echo c > /proc/sysrq-trigger** command or kernel fault. When Kdump enables second kernel, an error "BRCM Debug mfi stat 0x2d, data len requested/completed 0x200/0x0" is reported in the MegaRAID driver, as shown in the following figure. As a result, vmcore fails to be generated.

![Error information](./figures/Megaraid_IO_Request_uncompleted.png)

### Possible Causes

The  **reset\_devices** parameter is configured by default and is enabled during second kernel startup, making MegaRAID driver or drive faulty. An error is reported when the vmcore file is dumped ana accesses the MegaRAID card. As a result, vmcore fails to be generated.

### Solutions

Delete the **reset\_devices** parameter in the  **etc/sysconfig/kdump**  file on a PM, as shown in the following figure. Therefore, the I/O request will be responded when the MegaRAID driver resets the device during the second kernel startup, and vmcore will be successfully generated.

![Deleting reset\_devices](./figures/reset_devices.png)

---

---
url: /en/docs/common/faq/server/installation_faq2.md
---
# OS Installation FAQ 2

## 1. Failed to Start the Raspberry Pi

### Symptom

After the Raspberry Pi image released by the openEuler is written to the SD card, the Raspberry Pi fails to be started.

### Possible Causes

The possible causes are as follows:

1. The downloaded image file is incomplete. To avoid this problem, ensure that the image passes the integrity verification.
2. An error occurs when the image is written to the SD card. In most cases, the error occurs when the image is written to the SD card in the Windows environment using the application software.

### Solution

Re-write the complete image to the SD card.

## 2. Failed to Connect to Wi-Fi by Running the nmcli Command

### Symptom

Failed to connect to the Wi-Fi network by running the `nmcli dev wifi connect SSID password PWD` command. An error message, for example, `Error: Connection activation failed: (7) Secrets were required, but not provided.`, is displayed.

### Possible Causes

The command to be executed does not have a password. Note that if the password contains special characters, use single quotation marks to quote the password. If you fail to connect to the Wi-Fi network by running the `nmcli` command line, you are advised to use the nmtui utility for connection.

### Solution

Run the `nmtui` command to enter the nmtui utility. Perform the following steps to connect to the Wi-Fi network:

1. Select **Edit a connection** and press **Enter**. The window for editing network connections is displayed.

2. Press the right arrow key on the keyboard to select **Add**, and then press **Enter**. The window for creating a network connection is displayed.

3. Set the connection type to **Wi-Fi**, press the right arrow key on the keyboard to select **Create**, and press **Enter**. The page for editing Wi-Fi connection information is displayed.

4. On the Wi-Fi connection information page, edit the following information. Other information depends on the specific requirements. After the editing is complete, select **OK** and press **Enter** to return to the window for editing network connections.
   1. Enter the name of the Wi-Fi connection in the **Profile name** text box. You can use the default name, for example, **Wi-Fi connection 1**.
   2. Enter **wlan0** in the **Device** text box.
   3. Enter the SSID of the Wi-Fi network to be connected in the **SSID** text box.
   4. In the **Security** area, select the Wi-Fi password encryption mode as required, for example, **WPA & WPA2 Personal**.
   5. Enter the Wi-Fi password in the **Password** text box.

5. Select **Back** to return to the home screen of the nmtui utility.

6. Select **Activate a connection** and press **Enter**. The window for activating network connections is displayed.

7. Check whether the added Wi-Fi connection is activated. The name of an activated Wi-Fi connection is marked with an asterisk (\*). If the Wi-Fi connection is not activated, select the Wi-Fi connection, press the right arrow key on the keyboard to select **Activate**, and press **Enter** to activate the connection. After the activation is complete, select **Back** and press **Enter** to return to the home screen of the nmtui utility.

8. Select **Quit**, press the right arrow key on the keyboard to select **OK**, and press **Enter** to exit the nmtui utility.

## 3. Failed to Install the TensorFlow and Related Packages

### Symptom

Failed to install the TensorFlow and related packages using **yum**.

### Possible Causes

The dependencies of TensorFlow have not been upgraded to the version that adapts to TensorFlow 2.12.1. You need to manually install the dependencies using **pip**.

### Solution

1. Run `yumdownloader python3-tensorflow` to download the TensorFlow RPM package.
2. Run `rpm -ivh --nodeps python3-tensorflow` to install the package.
3. Install TensorFlow dependencies.
   1. Use **pip** to install dependencies: `pip3 install tensorflow-estimator==2.12.0 keras==2.12.0 protobuf==3.20.3`
   2. Use **yum** to install other dependencies: `yum install python3-termcolor python3-future python3-numpy python3-six python3-astunparse python3-google-pasta python3-opt-einsum python3-typing-extensions python3-wrapt python3-h5py python3-grpcio python3-absl-py python3-flatbuffers python3-gast`
4. Use **yum** to install related packages. For example, run `yum install python-keras-rl2` to install python-keras-rl2.

---

---
url: /en/docs/common/contribute/directory_structure_introductory.md
---
# Overview

## Introduction

This document describes the development and release workflow for openEuler documents, along with the structure of the documentation repository. It also specifies the exact locations of each manual within the repository.

![image](figures/architecture.png)

The diagram above illustrates the documentation development and release process for openEuler.

* The Document Center organizes community content into service scenarios and tools:
  * Service scenarios: server, virtualization, cloud, edge computing, and embedded.
  * Tools: community tools, DevOps, AI, graphical desktops, cloud-native tools, O\&M, and security.
* Release process:
  * Each scenario and tool has an associated directory structure file (**\_toc.yaml**). These files reside in the **openEuler/docs** repository and are managed centrally by the Doc SIG.
  * The SIG responsible for a document must link its directory structure file to the directory structure file of the relevant scenario or tool to ensure the document appears under the correct module.
* Documentation development:
  * Documents are created in the **openEuler/docs** repository and the **docs** repositories of individual SIGs.
  * Core feature documentation (such as release notes, quick start guides, installation, upgrades, administrator guides, configuration and logical volumes, network setup, and troubleshooting) is housed in the **openEuler/docs** repository and maintained by the Doc SIG.
  * Feature-specific documentation, like the A-Tune User Guide, x2openEuler USer Guide, and oeAware User Guide, falls under the purview of the respective SIGs and is stored in their **docs** repositories.
  * Each SIG maintains two types of files in the documentation repository: content files and directory structure files (**\_toc.yaml**). Content files store the actual documentation, while directory structure files define the chapter hierarchy.

The **docs** directory in the repository contains content published on the official website. It includes **en** and **zh** subdirectories for English and Chinese documentation, respectively, mirroring the website structure. The repository also features an **archive** directory for documents not yet ready for publication. Once finalized, these documents are moved to the **docs** directory for website display.

```text
├─docs     <!-- Published documentation -->
│  ├─en    <!-- English content -->
│  └─zh    <!-- Chinese content -->
├─archive  <!-- Unpublished documentation -->
```

## Document Repository Structure Overview

### Scenarios

The Document Center organizes content into five business scenarios: server, virtualization, cloud, edge computing, and embedded. Each scenario maps to a specific subdirectory under **docs/{zh|en}** in the repository: **server**, **virtualization**, **cloud**, **edge\_computing**, and **embedded**. The tools module is represented by the **tools** subdirectory.

Below is the scenario-related directory structure (the following directory structures use **zh** as an example):

```text
├─Archive
├─docs
│  ├─en
│  └─zh
│     ├─server          <!-- Server-->
│     ├─virtualization  <!-- Virtualization-->
│     ├─cloud           <!-- Cloud -->
│     ├─edge_computing  <!-- Edge computing -->
│     ├─embedded        <!-- Embedded -->
│     └─tools           <!-- Tools -->
```

The tools module is further divided into submodules: community tools, DevOps, AI, desktop, cloud-native tools, O\&M, and security. These are organized under the **tools** directory with corresponding subdirectories: **community\_tools**, **devops**, **ai**, **desktop**, **cloud**, **maintenance**, and **security**.

Below is the tool-related directory structure:

```text{9-16}
├─docs
│  ├─en
│  └─zh
│     ├─server              <!-- Server -->
│     ├─virtualization      <!-- Virtualization -->
│     ├─cloud               <!-- Cloud -->
│     ├─edge_computing      <!-- Edge computing -->
│     ├─embedded            <!-- Embedded -->
│     └─tools               <!-- Tools -->
│        ├─community_tools  <!-- Community tools -->
│        ├─devops           <!-- DevOps (community services) -->
│        ├─ai               <!-- AI -->
│        ├─desktop          <!-- Desktops -->
│        ├─cloud            <!-- Cloud-native tools -->
│        ├─maintenance      <!-- O&M -->
│        └─security         <!-- Security -->
```

### Directories

Each scenario is organized into specific directories. For instance, the server scenario includes first-level directories like release notes, quick start, installation and upgrade, system administration, O\&M, and security.

Directory structure for the server scenario:

```text{4-17}
├─docs
│  ├─en
│  └─zh
│      ├─server                     <!-- Scenario: server -->
│      │  ├─releasenotes            <!-- First-level directory: release notes -->
│      │  ├─quickstart              <!-- First-level directory: quick start -->
│      │  ├─installation_upgrade    <!-- First-level directory: installation and upgrade -->
│      │  ├─administration          <!-- First-level directory: system administration -->
│      │  ├─maintenance             <!-- First-level directory: O&M -->
|      |  ├─security                <!-- First-level directory: security -->
│      │  ├─memory_storage          <!-- First-level directory: memory and storage -->
│      │  ├─network                 <!-- First-level directory: networking -->
│      │  ├─performance             <!-- First-level directory: performance tuning -->
│      │  ├─development             <!-- First-level directory: application development -->
│      │  ├─high_availability       <!-- First-level directory: high availability -->
│      │  ├─diversified_computing   <!-- First-level directory: diversified computing -->
│      │  └─_toc.yaml
│      ├─virtualization             <!-- Scenario: virtualization -->
│      ├─cloud                      <!-- Scenario: cloud -->
│      ├─edgecomputing              <!-- Scenario: edge computing -->
│      ├─embedded                   <!-- Scenario: embedded -->
│      └─tools                      <!-- Scenario: tools -->
```

Some first-level directories are further divided into second-level directories. For example, the performance tuning directory under the server scenario includes subdirectories for overview, CPU tuning, system tuning, and tuning framework.

Example directory structure for performance tuning:

```text{13-22}
├─docs
│  ├─en
│  └─zh
│      ├─server                      <!-- Scenario: server -->
│      │  ├─releasenotes             <!-- First-level directory: release notes -->
│      │  ├─quickstart               <!-- First-level directory: quick start -->
│      │  ├─installation_upgrade     <!-- First-level directory: installation and upgrade -->
│      │  ├─administration           <!-- First-level directory: system administration -->
│      │  ├─maintenance              <!-- First-level directory: O&M -->
|      |  ├─security                 <!-- First-level directory: security -->
│      │  ├─memory_storage           <!-- First-level directory: memory and storage -->
│      │  ├─network                  <!-- First-level directory: networking -->
│      │  ├─performance              <!-- First-level directory: performance tuning -->
│      │  │  ├─overall               <!-- Second-level directory: overview -->
│      │  │  │  └─system_resource
│      │  │  ├─cpu_optimization      <!-- Second-level directory: CPU tuning -->
│      │  │  │  ├─kae
│      │  │  │  └─sysboost
│      │  │  ├─system_optimization   <!-- Second-level directory: system tuning -->
│      │  │  │  └─atune
│      │  │  └─tuning_framework      <!-- Second-level directory: tuning framework -->
│      │  │      └─oeaware
│      │  ├─development              <!-- First-level directory: application development -->
│      │  ├─high_availability        <!-- First-level directory: high availability -->
│      │  ├─diversified_computing    <!-- First-level directory: diversified computing -->
│      │  └─_toc.yaml
│      ├─virtualization              <!-- Scenario: virtualization -->
│      ├─cloud                       <!-- Scenario: cloud -->
│      ├─edge_computing              <!-- Scenario: edge computing -->
│      ├─embedded                    <!-- Scenario: embedded -->
│      └─tools                       <!-- Scenario: tools -->
```

### Manuals

The directories contains various manuals. Taking the O\&M directory under the server scenario as an example, it includes eight manuals, each corresponding to a directory in the documentation repository.

Here is the directory structure for O\&M under the server scenario:

```text{9-17}
├─docs
│   ├─en
│   └─zh
│      ├─server                     <!-- Scenario: server -->
│      │  ├─releasenotes            <!-- First-level directory: release notes -->
│      │  ├─quickstart              <!-- First-level directory: quick start -->
│      │  ├─installation_upgrade    <!-- First-level directory: installation and upgrade -->
│      │  ├─administration          <!-- First-level directory: system administration -->
│      │  ├─maintenance             <!-- First-level directory: O&M -->
│      │  │  ├─aops                 <!-- Manual: A-Ops User Gudie -->
│      │  │  ├─common_skills        <!-- Manual: Common Skills -->
│      │  │  ├─common_tools         <!-- Manual: Commonly Used Tools for Location and Demarcation -->
│      │  │  ├─gala                 <!-- Manual: gala User Guide -->
│      │  │  ├─kernel_live_upgrade  <!-- Manual: Kernel Live Upgrade Guide -->
│      │  │  ├─syscare              <!-- Manual: SysCare User Guide -->
│      │  │  ├─sysmonitor           <!-- Manual: sysmonitor User Guide -->
│      │  │  └─trouble_shooting     <!-- Manual: Troubleshooting -->
|      |  ├─security                <!-- First-level directory: security -->
│      │  ├─memory_storage          <!-- First-level directory: memory and storage -->
│      │  ├─network                 <!-- First-level directory: networking -->
│      │  ├─performance             <!-- First-level directory: performance tuning -->
│      │  ├─development             <!-- First-level directory: application development -->
│      │  ├─high_availability       <!-- First-level directory: high availability -->
│      │  ├─diversified_computing   <!-- First-level directory: diversified computing -->
│      │  └─_toc.yaml
│      ├─virtualization             <!-- Scenario: virtualization -->
│      ├─cloud                      <!-- Scenario: cloud -->
│      ├─edge_computing             <!-- Scenario: edge computing -->
│      ├─embedded                   <!-- Scenario: embedded -->
│      └─tools                      <!-- Scenario: tools -->
```

Each manual includes one or more content files (.md files) corresponding to one or more chapters, along with a directory structure file (**\_toc.yaml**). For example, the *Kernel Live Upgrade Guide* manual consists of three chapters: Installation and Deployment, Usage Guide, and Common Problems and Solutions.

```text{14-18}
├─docs
│  ├─en
│  └─zh
│      ├─server                                        <!-- Scenario: server -->
│      │  ├─quickstart                                 <!-- First-level directory: release notes -->
│      │  ├─releasenotes                               <!-- First-level directory: quick start -->
│      │  ├─installation_upgrade                       <!-- First-level directory: installation and upgrade -->
│      │  ├─administration                             <!-- First-level directory: system administration -->
│      │  ├─maintenance                                <!-- First-level directory: O&M -->
│      │  │  ├─aops                                    <!-- Manual: A-Ops User Gudie -->
│      │  │  ├─common_skills                           <!-- Manual: Common Skills -->
│      │  │  ├─common_tools                            <!-- Manual: Commonly Used Tools for Location and Demarcation -->
│      │  │  ├─gala                                    <!-- Manual: gala User Guide -->
│      │  │  ├─kernel_live_upgrade                     <!-- Manual: Kernel Live Upgrade Guide -->
│      |  │  │  ├─installation-and-deployment.md       <!-- Chapter: Installation and Deployment -->
│      |  │  │  ├─usage-guide.md                       <!-- Chapter: Usage Guide -->
│      |  │  │  ├─common-problems-and-solutions.md     <!-- Chapter: Common Problems and Solutions -->
│      |  │  │  └─_toc.yaml
|      |  ├─security                                   <!-- First-level directory: security -->
│      │  ├─memory_storage                             <!-- First-level directory: memory and storage -->
│      │  ├─network                                    <!-- First-level directory: networking -->
│      │  ├─performance                                <!-- First-level directory: performance tuning -->
│      │  ├─development                                <!-- First-level directory: application development -->
│      │  ├─high_availability                          <!-- First-level directory: high availability -->
│      │  ├─diversified_computing                      <!-- First-level directory: diversified computing -->
│      │  └─_toc.yaml│
│      ├─cloud                                         <!-- Scenario: virtualization -->
│      ├─edge_computing                                <!-- Scenario: cloud -->
│      ├─embedded                                      <!-- Scenario: edge computing -->
│      ├─tools                                         <!-- Scenario: embedded -->
│      └─virtualization                                <!-- Scenario: tools -->
```

## Directory Structure File Format

Every scenario and manual includes an **\_toc.yaml** file to organize the directory structure. The example below illustrates the placement of the **\_toc.yaml** file for the virtualization scenario, with other scenarios adhering to the same logic.

```text
├─docs
│  └─zh
│     ├─virtualization                                 <!-- Scenario: virtualization -->
│     │   ├─vitualization_platform                     <!-- First-level directory: virtualization platforms -->
│     │   |  ├─stratovirt                              <!-- Manual: StratoVirt User Guide-->
│     │   |  |   └─_toc.yaml // [!code highlight]
│     │   |  ├─virtualization                          <!-- Manual: Virtualization User Guide -->
│     │   |  |   └─_toc.yaml // [!code highlight]
│     │   └─_toc.yaml // [!code highlight]
```

### Manual Directory Structure File

Each manual must have an **\_toc.yaml** file to define the logical relationships among its chapters.

Here is the **\_toc.yaml** file for the *Kernel Live Upgrade Guide* manual:

```yaml
label: Kernel Live Upgrade Guide
isManual: true
description: User-space automation tool that facilitates rapid kernel restarts and program
  live migration, enabling kernel hot-swapping functionality
sections:
  - label: Installation and Deployment
    href: ./installation-and-deployment.md
  - label: Usage Guide
    href: ./usage-guide.md
  - label: Common Problems and Solutions
    href: ./common-problems-and-solutions.md
```

* **label**: The manual title.
* **isManual**: Flags this file as a manual directory structure file, differentiating it from scenario files.
* **description**: A concise overview of the manual.
* **sections**:
  * **label**: The chapter title.
  * **href**: Path to the document file (preferably relative).

### Scenario Directory Structure File

Every scenario includes a **\_toc.yaml** file that references the **\_toc.yaml** files of its associated manuals. For example, here is the structure for the server scenario:

```yaml
label: Server
sections:
  - label: Release Notes
    sections:
      - href: ./releasenotes/releasenotes/_toc.yaml
  - label: Quick Start
    sections:
      - href: ./quickstart/quickstart/_toc.yaml
  - label: Installation and Upgrade
    sections:
      - href: ./installation_upgrade/installation/_toc.yaml
      - href: ./installation_upgrade/upgrade/_toc.yaml
  - label: OS Administration
    sections:
      - href: ./administration/administrator/_toc.yaml
      - href: ./administration/sysmaster/_toc.yaml
      - href: ./administration/compa_command/_toc.yaml
  - label: O&M
    sections:
      - href: ./maintenance/aops/_toc.yaml
      - href: ./maintenance/gala/_toc.yaml
      - href: ./maintenance/sysmonitor/_toc.yaml
      - href: ./maintenance/kernel_live_upgrade/_toc.yaml
      - href: ./maintenance/syscare/_toc.yaml
      - href: ./maintenance/common_skills/_toc.yaml
      - href: ./maintenance/common_tools/_toc.yaml
      - href: ./maintenance/troubleshooting/_toc.yaml
  - label: Security
    sections:
      - href: ./security/secharden/_toc.yaml
      - href: ./security/trusted_computing/_toc.yaml
      - href: ./security/secgear/_toc.yaml
      - href: ./security/cve-ease/_toc.yaml
      - href: ./security/cert_signature/_toc.yaml
      - href: ./security/sbom/_toc.yaml
      - href: ./security/shangmi/_toc.yaml
  - label: Memory and Storage
    sections:
      - href: ./memory_storage/lvm/_toc.yaml
      - href: ./memory_storage/etmem/_toc.yaml
      - href: ./memory_storage/gmem/_toc.yaml
      - href: ./memory_storage/hsak/_toc.yaml
  - label: Network
    sections:
      - href: ./network/network_config/_toc.yaml
      - href: ./network/gazelle/_toc.yaml
  - label: Performance Optimization
    sections:
      - label: Overview
        sections:
          - href: ./system_resource/_toc.yaml
      - label: Tuning Framework
        sections:
          - href: ./oeaware/_toc.yaml
      - label: CPU Tuning
        sections:
          - href: ./sysboost/_toc.yaml
          - href: ./kae/_toc.yaml
      - label: System Tuning
        sections:
          - href: ./atune/_toc.yaml
  - label: Application Development
    sections:
      - href: ./development/application_dev/_toc.yaml
      - href: ./development/gcc/_toc.yaml
  - label: High Availability
    sections:
      - href: ./high_availability/ha/_toc.yaml
  - label: Diversified Computing
    sections:
      - href: ./diversified_computing/dpu_offload/_toc.yaml
      - href: ./diversified_computing/dpu_os/_toc.yaml
```

* **label**: The scenario title.
* **description**: A short description of the scenario.
* **sections**:
  * **label**: The name of the first-level directory.
  * **sections**:
    * **href**: A reference to the manual directory structure file.

## Document Storage Locations

openEuler documentation is hosted in the [openEuler/docs](https://atomgit.com/openeuler/docs) repository and the documentation repositories of each SIG. The tables below list the specific storage paths for each manual.

### Server

### Virtualization

### Cloud

### Edge Computing

### Embedded

### Tools

---

---
url: /en/docs/common/faq/community_tools/patch_tracking_faqs.md
---
# patch-tracking FAQ

## 1. Connection Refused When Accessing api.github.com

### Context

The following error may occur during patch-tracking execution:

```sh
Sep 21 22:00:10 localhost.localdomain patch-tracking[36358]: 2020-09-21 22:00:10,812 - patch_tracking.util.github_api - WARNING - HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /user (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0xfffe19d35820>: Failed to establish a new connection: [Errno 111] Connection refused'))
```

### Possible Causes

Unstable network connectivity between patch-tracking and GitHub API services

### Solution

Run patch-tracking in a network-stable environment that can reliably access GitHub API services, such as using [Huawei Cloud ECS](https://console.huaweicloud.com/).

---

---
url: /zh/docs/common/faq/community_tools/patch_tracking_faqs.md
---
# patch-tracking常见问题与解决方法

## **问题1:访问 api.github.com Connection refused 异常**

## 问题描述

patch-tracking 运行过程中，可能会出现如下报错：

```sh
 9月 21 22:00:10 localhost.localdomain patch-tracking[36358]: 2020-09-21 22:00:10,812 - patch_tracking.util.github_api - WARNING - HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /user (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0xfffe19d35820>: Failed to establish a new connection: [Errno 111] Connection refused'))   
```

## 原因分析

以上问题是 patch-tracking 与 GitHub API 服务之间网络访问不稳定导致。

## 解决方法

请确保在与 GitHub API 服务之间网络稳定的环境中（如使用[华为云ECS弹性云服务器](https://console.huaweicloud.com/)）运行 patch-tracking。

---

---
url: /zh/docs/common/faq/caselibrary/pvs_vgs.md
---
# pvs或vgs出现Unknown问题

## 现象描述

pvs或vgs的回显中，pv或vg报unknown错误。

![](./figures/pvs-1.png)

![](./figures/pvs-2.png)

## 原因分析

通常为元数据损坏，导致命令读取到不完整的信息，需要进行修复，否则无法正常使用pv或vg。可使用hexdump读取裸盘数据验证，见[LVM标签损坏](./lvm.md)。

## 解决方法

问题一：vg无法识别，报错unknown

```bash
pvck --repair --file /etc/lvm/backup/vg /dev/sdb
```

vg为卷组备份信息，保存在/etc/lvm/backup目录。

问题二：pv无法识别，报错unknown

```bash
pvcreate --uuid xxx --restorefile /etc/lvm/backup/vg /dev/sdc
vgcfgrestore --file /etc/lvm/backup/vg vg
vgchange -ay vg
```

uuid在备份文件/etc/lvm/backup中获取。

执行完修复步骤后，输入pvs或vgs命令，可看到正常回显则说明修复成功。

---

---
url: /en/docs/common/contribute/contribution_process.md
---
# Quick Start

## Overview

This guide provides  a structured approach to documentation development, covering three primary tasks: adding, modifying, and deleting documents.

**Required Skills:**

* Knowledge of the [openEuler documentation structure](./directory_structure_introductory.md)
* Proficiency in [Markdown writing specifications](./documentation_writing_specifications.md)

## Standard Workflow

Begin by determining the repository where the target document resides. Use the manual name to find the relevant [repository and directory path](./directory_structure_introductory.md#document-storage-locations). Below is a step-by-step workflow for document operations, illustrated using modifications to the *Installation Guide* for server environments.

1. Clone the repository.

   Fork the remote repository, clone it locally, and establish an upstream connection.

   ```bash
   git clone https://gitee.com/wu-donger/docs.git
   git remote add upstream https://atomgit.com/openeuler/docs.git
   ```

2. Switch to the target branch.

   Select the appropriate version branch (typically named `stable2-<version>`). For version 25.03:

   ```bash
   git fetch upstream
   git checkout stable2-25.03
   git rebase upstream/stable2-25.03
   git checkout -b work25.03
   ```

3. Commit changes.

   Apply changes following the dedicated sections for [adding](#adding-documents), [editing](#editing-documents), or [removing](#removing-documents) documents.

   ```bash
   git add .
   git commit -m "Brief description of changes"
   ```

4. Push changes to the remote repository and create a pull request (PR).

   Push changes and initiate a pull request:

   ```bash
   git push origin work25.03
   ```

## Adding Documents

This section guides developers through adding complete manuals. To add content to existing manuals (such as adding a "Common Tools" chapter to the *Upgrade Guide*), refer to [Editing Documents](#editing-documents).

The following example demonstrates adding an *Upgrade Guide* for server environments:

1. Create a storage directory.

   Identify the scenario for your *Upgrade Guide* and locate its [storage path](./directory_structure_introductory.md#document-storage-locations). Create a new folder to store all .md files for this guide:

   ```text{7-9}
   ├─docs
   |  ├─en
   |  └─zh
   |    └─Server                                                       # Scenario: Server
   |       └─InstallationUpgrade                                       # First-level directory: Installation and Upgrade
   |           ├─Installation                                          # Manual: Installation Guide
   |           └─Upgrade                                               # New folder: Upgrade Guide
   |               ├─openEuler_22.03_LTS_upgrade_and_downgrade.md      # Content file
   |               └─_toc.yaml                                         # Table of contents file
   ```

2. Edit **\_toc.yaml**.

   Create a **\_toc.yaml** file in your new directory to maintain chapter display logic:

   ```yaml
   label: Upgrade Guide                                                # Manual: Upgrade Guide // [!code ++]
   isManual: true                                                # Identifies this as a manual TOC file // [!code ++]
   description: Upgrade the openEuler OS.                           # Manual description // [!code ++]
   sections: // [!code ++]
     - label: Upgrade and Downgrade Guide                                          # Chapter: Upgrade and Downgrade Guide // [!code ++]
       href: ./openEuler_22.03_LTS_upgrade_and_downgrade.md      # Content file reference // [!code ++]
   ```

3. Link the manual to the scenario.

   Add a reference to your Upgrade Guide in the Server scenario's [\_toc.yaml](https://atomgit.com/openeuler/docs/blob/stable-25.03/docs/en/server/_toc.yaml):

   ```yaml
   label: Server                                                   # Scenario: Server
   sections:
     - label: Start Here                                           # First-level directory: Start Here
       sections:
         - href: ./releasenotes/releasenotes/_toc.yaml            # Manual: Release Notes
         - href: ./quickstart/quickstart/_toc.yaml                # Manual: Quick Start
     - label: Installation and Upgrade                                             # First-level directory: Installation and Upgrade
       sections:
         - href: ./installation_upgrade/installation/_toc.yaml    # Manual: Installation Guide
   ```

## Editing Documents

Document modifications fall into three categories:

* Content-only changes: Modify existing manual content without altering the TOC structure. For example, adding a new installation method to the "Installation Methods" chapter of the Installation Guide.

* Adding/removing chapters: Create or delete content files and update **\_toc.yaml** of the manual.

  Example: to add a "Common Upgrade Tools" chapter to the *Upgrade Guide*, first create **software.md** in the *Upgrade Guide* directory, then update [**\_toc.yaml**](https://atomgit.com/openeuler/docs/blob/stable-25.03/docs/en/server/installation_upgrade/upgrade/_toc.yaml) to reference **software.md**.

  ```yaml
  label: Upgrade Guide                                               # Manual: Upgrade Guide
  isManual: true                                               # Identifies this as a manual TOC file
  description: Upgrade the openEuler OS                          # Manual description
  sections:
    - label: Upgrade and Downgrade Guide                                         # Chapter: Upgrade and Downgrade Guide
      href: ./openEuler_22.03_LTS_upgrade_and_downgrade.md     # Content file path
    - label: Common Upgrade Tools                                       # New chapter: Common Upgrade Tools // [!code ++]
      href: ./software.md                                      # New content file path // [!code ++]
  ```

* Manual display position changes: Modify the scenario **\_toc.yaml** file to adjust manual ordering.

  Example: To move the *Upgrade Guide* above the *Installation Guide* in the server scenario, edit [**\_toc.yaml**](https://atomgit.com/openeuler/docs/blob/stable-25.03/docs/en/virtualization/_toc.yaml).

  ```yaml
  label: Server                                                      # Scenario: Server
  sections:
    - label: Start Here                                              # First-level directory: Start Here
      sections:
        - href: ./releasenotes/releasenotes/_toc.yaml               # Manual: Release Notes
        - href: ./quickstart/quickstart/_toc.yaml                   # Manual: Quick Start
    - label: 'Installation and Upgrade'                                              # First-level directory: Installation and Upgrade
      sections:
        - href: ./installation_upgrade/upgrade/_toc.yaml'           # Upgrade Guide now appears first // [!code ++]
        - href: ./installation_upgrade/installation/_toc.yaml'      # Manual: Installation Guide // [!code ++]
        - href: ./installation_upgrade/upgrade/_toc.yaml'          // [!code --]
    - label: 'System Administration'
      sections:
        - href: './administration/administrator/_toc.yaml'
        - href: './administration/sysmaster/_toc.yaml'
        - href: './administration/compa_command/_toc.yaml'
  ```

## Removing Documents

This section guides complete manual removal. To remove content from existing manuals (such as deleting the "Process Management" chapter from the *Administrator Guide*), refer to [Editing Documents](#editing-documents).

Example workflow for removing the *Virtualization User Guide* from the virtualization scenario:

1. Locate and delete the manual directory at the [storage location](./directory_structure_introductory.md#document-storage-locations).

2. Remove the manual reference from the virtualization scenario [**\_toc.yaml**](https://gitee.com/openeuler/docs/blob/25.03/docs/zh/virtualization/_toc.yaml) file.

   ```yaml
   label: Virtualization                                                          # Scenario: Virtualization
   sections:
     - label: Virtualization Platforms                                                  # First-level directory: Virtualization Platforms
       sections:
         - href: ./virtualization_platform/virtualization/_toc.yaml      # Removed reference to the Virtualization User Guide // [!code --]
         - href: ./virtualization_platform/stratovirt/_toc.yaml
   ```

## Temporary Documentation Storage

For documentation not yet ready for publication (under development or not mature enough for promotion), store files in the [**archive**](https://atomgit.com/openeuler/docs/tree/stable-25.03/archive) directory. These documents must still follow writing conventions and can be moved to appropriate directories when ready for publication.

> Note: Documents submitted to the **archive** directory are scheduled for future publication. Unmaintained documents will be purged.

---

---
url: /en/docs/common/faq/caselibrary/rebranding.md
---
# Rebranding FAQ

## 1. Installation Failure with Boot Loader Configuration Write Error

### Context

The installation fails when rebranding operations are performed using openEuler system packages.

### Symptom

Image installation fails with error "Unable to write boot loader configuration."

![image](./figures/rebranding_config.png)

### Possible Causes

Switching to the background terminal with **Ctrl+Alt+F2** reveals the following in **/tmp/anaconda.log**:

![image](./figures/rebranding_config_log.png)

The logs indicate the bootloader configuration file cannot be found.
This occurs because the installation process verifies the `ID` and `VARIANT_ID` values in the **/etc/os-release** file.

![image](./figures/rebranding_config_check.png)

### Solution

Modify the `os_id` and `variant_id` parameters under `Profile Detection` in the **openEuler.conf** file of Anaconda to match those in the **os-release** file of the rebranded system.

![image](./figures/rebranding_config_modification.png)

## 2. Installation Completes but System Fails to Boot

### Context

Following rebranding, the system image installs correctly but becomes unbootable post-installation.

### Symptom

Startup fails with an "EFI boot file not found" error.

![image](./figures/rebranding_incomplete.png)

### Possible Causes

Rebranding of software packages is incomplete, particularly affecting the GRUB bootloader configuration.

### Resolution

1. Verify boot entries for configuration errors:

   ![image](./figures/rebranding_incomplete_configuration.png)

2. See [System File Recovery FAQ](./sysfile.md) for adjusting boot configurations after entering the system.

3. Permanently fix it by updating **grub.cfg** in the build environment.

## 3. tk Installation Fails During Post-Rebranding Kernel Build

### Context

After rebranding, building the openEuler kernel via OBS triggers a tk installation failure.

![image](./figures/rebranding_kernel_build_error.png)

### Symptom

Installation verification of the tk package fails, and standalone installation also results in an error. The issue occurs during the `%post` phase of the component.

![image](./figures/rebranding_tk_install_failure.png)

### Possible Causes

Unrecognized `%ldconfig_post` or `%ldconfig_postun` macros in the spec file are being incorrectly processed as executable scripts.

![image](./figures/rebranding_tk_error.png)

### Resolution

Update macro definitions in the `rpm-config` configuration of the project.

## 4. POSTTRANS Scriptlet Execution Failure During Installation

### Context

`POSTTRANS` script errors occur when either rebranded software packages or locally built software images are installed.

![image](./figures/rebranding_script_error.png)

### Symptom

1. Switch to the background console using **Ctrl+Alt+F2** and change root into **/mnt/sysroot**.
2. Run `dnf history info 1` to identify the failing `POST` operation.
   ![image](./figures/rebranding_script_post.png)

### Possible Causes

The `%POSTTRANS` script fails during package installation due to missing .gz files.

### Solution

Analyze the `%post` script in the package `spec` file. After local verification of the modified script, retry the installation.

---

---
url: /en/docs/common/faq/server/syscare_faqs.md
---
# SysCare FAQ

## 1. "alloc upatch module memory failed"

Possible cause: The SELinux constraint is triggered.

Solution: Manually add policies according to the error. The policies to be added vary according to the actual situation. For details, see <https://gitee.com/openeuler/syscare/issues/I671H1>.

## 2. "patch file error 2"

Possible cause: The patch cannot be detected.

Solution: Use another patch.

## 3. "build project error 11"

Possible cause: The source package fails to be compiled.

Solution: Run `rpmbuild -ra *.src.rpm` to check if the source package can be compiled and the compilation dependencies are satisfied.

---

---
url: /zh/docs/common/faq/server/syscare_faqs.md
---
# SysCare常见问题与解决方法

## **问题1：报错：“alloc upatch module memory failed”**

原因：触发selinux约束。

解决方法：按照报错建议命令操作，手动添加策略，由于不同情况添加策略不同，无法穷尽枚举，参考该issue: <https://gitee.com/openeuler/syscare/issues/I671H1>。

## **问题2：报错: “patch file error 2”**

原因：补丁检测失败。

解决方法：补丁无法正常打入，更换补丁。

## **问题3：报错: “build project error 11”**

原因：源码包编译失败。

解决方法：尝试使用`rpmbuild -ra *.src.rpm`命令测试源码包是否可正常编译并满足其编译依赖。

---

---
url: /en/docs/common/faq/server/administration_faqs.md
---
# System ADministration FAQ

## 1. Why Is the Memory Usage of the libvirtd Service Queried by Running the systemctl and top Commands Different

### Symptom

The output of the  **systemctl**  and  **systemd-cgtop**  commands shows that the libvirtd service occupies more than 1.5 GB memory, but the output of the  **top**  command shows that the libvirtd service occupies about 70 MB memory.

### Possible Cause

The memory displayed in the services (including systemctl and systemd-cgtop) managed by systemd can be obtained from  **memory.usage\_in\_bytes**  in Cgroup. Running the  **top**  command is to query the memory information in the  **/proc**  directory. The query results are different because the statistical method varies.

Generally, the memory used by service processes has the following types:

* anon\_rss: anonymous pages in user mode address spaces, for example, memory allocated by calling the malloc function or the mmap function with configured  **MAP\_ANONYMOUS**. When the system memory is insufficient, this type of memory can be swapped by the kernel.
* file\_rss: mapped pages in user mode address spaces, including map file (such as mmap of a specified file) and map tmpfs (such as IPC shared memory). When the system memory is insufficient, the kernel can reclaim these pages. Data may need to be synchronized between the kernel and map file before reclamation.
* file\_cache: file cache (page in page cache of disk file), which is generated when a file is read or written. When the system memory is insufficient, the kernel can reclaim these pages. Data may need to be synchronized between the kernel and map file before reclamation.
* buffer pages: belongs to page cache, for example, cache generated when block device files are read.

anon\_rss and file\_rss belong to the resident set size (RSS) of processes, and file\_cache and buffer pages belong to page cache. In brief:

RSS in the output of the  **top**  command = anon\_rss + file\_rss; Shared memory (SHR) = file\_rss

**memory.usage\_in\_bytes**  in Cgroup = cache + RSS + swap

In conclusion, the definition of memory usage obtained by running the  **systemd**  command is different from that obtained by running the  **top**  command. Therefore, the query results are different.

## 2. An Error Occurs When stripsize Is Set to 4 During RAID 0 Volume Configuration

### Symptom

An error occurs when the  **stripsize**  parameter is set to  **4**  during RAID 0 volume configuration.

### Possible Cause

The 64 KB page table can be enabled only in the scenario where  **stripsize**  is set to  **64**.

### Solution

You do not need to modify the configuration file. When running the  **lvcreate** command on openEuler, set  **stripesize**  to  **64**  because the minimum supported stripe size is 64 KB.

## 3. Failed to Compile MariaDB Using rpmbuild

### Symptom

When you log in to the system as user  **root**  and run the  **rpmbuild**  command to compile the MariaDB source code, the compilation fails and the following information is displayed:

```text
+ echo 'mysql can'\''t run test as root'
mysql can't run test as root
+ exit 1
```

### Possible Cause

The MariaDB does not allow user  **root**  to execute test cases. However, test cases are automatically executed during compilation. As a result, the compilation process is blocked.

### Solution

Use a text editor, such as vi, to modify the value of the  **runtest**  variable in the  **mariadb.spec**  file.

Before the modification:

```text
%global runtest 1
```

After the modification:

```text
%global runtest 0
```

The modification disables the function of executing test cases during compilation, which does not affect the compilation and the RPM package content after compilation.

## 4. Failed to Start the SNTP Service Using the Default Configuration

### Symptom

The SNTP service fails to be started with the default configuration.

### Possible Cause

The domain name of the NTP server is not added to the default configuration.

### Solution

Modify the  **/etc/sysconfig/sntp**  file and add the domain name of the NTP server in China:  **0.generic.pool.ntp.org**.

## 5. Installation Failure Caused by Software Package Conflict, File Conflict, or Missing Software Package

### Symptom

Software package conflict, file conflict, or missing software packages may occur during software package installation. As a result, the upgrade is interrupted and the installation fails. The error information about software package conflict, file conflict, and missing software packages is as follows:

The following is an example of software package conflict error information (the conflict between  **libev-libevent-devel-4.24-11.oe1.aarch64**  and  **libevent-devel-2.1.11-2.oe1.aarch64**  is used as an example):

```text
package libev-libevent-devel-4.24-11.oe1.aarch64 conflicts with libevent-devel provided by libevent-devel-2.1.11-2.oe1.aarch64
 - cannot install the best candidate for the job
 - conflicting requests
```

The following is an example of file conflict error information (the  **/usr/bin/containerd**  file conflict is used as an example):

```text
Error: Transaction test error:
 file /usr/bin/containerd from install of containerd-1.2.0-101.oe1.aarch64 conflicts with file from package docker-engine-18.09.0-100.aarch64
 file /usr/bin/containerd-shim from install of containerd-1.2.0-101.oe1.aarch64 conflicts with file from package docker-engine-18.09.0-100.aarch64
```

The following is an example of the error message indicating that the  **blivet-data**  software package is missing:

```text
Error:
  Problem: cannot install both blivet-data-1:3.1.1-6.oe1.noarch and blivet-data-1:3.1.1-5.noarch
   - package python2-blivet-1:3.1.1-5.noarch requires blivet-data = 1:3.1.1-5, but none of the providers can be installed
   - cannot install the best update candidate for package blivet-data-1:3.1.1-5.noarch
   - problem with installed package python2-blivet-1:3.1.1-5.noarch(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
```

### Possible Cause

* In the software packages provided by openEuler, some software packages have different names but the same functions. As a result, the software packages cannot be installed at the same time.
* In the software packages provided by openEuler, some software packages have different names but the same functions. As a result, the files after installation are the same, causing file conflict.
* Some software packages are depended on by other software packages before the upgrade. After the software packages are upgraded, the software packages that depend on them may fail to be installed due to lack of software packages.

### Solution

If a software package conflict occurs, perform the following steps (the software package conflict in "Symptom" is used as an example):

1. According to the error message displayed during the installation, the software package that conflicts with the to-be-installed software package  **libev-libevent-devel-4.24-11.oe1.aarch64**  is  **libevent-devel-2.1.11-2.oe1.aarch64**.

2. Run the  **dnf remove**  command to uninstall the software package that conflicts with the software package to be installed.

   ```shell
   dnf remove libevent-devel-2.1.11-2.oe1.aarch64
   ```

3. Perform the installation again.

If a file conflict occurs, perform the following steps (the file conflict in "Symptom" is used as an example):

1. According to the error message displayed during the installation, the names of the software packages that cause the file conflict are  **containerd-1.2.0-101.oe1.aarch64**  and  **docker-engine-18.09.0-100.aarch64**.

2. Record the names of the software packages that do not need to be installed. The following uses  **docker-engine-18.09.0-100.aarch64**  as an example.

3. Run the  **dnf remove**  command to uninstall the software package that does not need to be installed.

   ```shell
   dnf remove docker-engine-18.09.0-100.aarch64
   ```

4. Perform the installation again.

If a software package is missing, perform the following steps (the missed software package in "Symptom" is used as an example):

1. Determine the name of the software package to be upgraded (**blivet-data-1:3.1.1-5.noarch**) and the name of the dependent software package (**python2-blivet-1:3.1.1-5.noarch**) based on the error information displayed during the upgrade.

2. Run the  **dnf remove**  command to uninstall the software package that depends on the upgrade package or add the  **--allowerasing**  parameter when upgrading the software package.
   * Run the  **dnf remove**  command to uninstall the software package that depends on the  **blivet-data-1:3.1.1-5.noarch**  software package.

     ```shell
     dnf remove python2-blivet-1:3.1.1-5.noarch
     ```

   * Add the  **--allowerasing**  parameter when upgrading the software package.

     ```shell
     yum update blivet-data-1:3.1.1-5.noarch -y --allowerasing
     ```

3. Perform the upgrade again.

### Installing Conflicting Instances

* A file conflict occurs.

  The **python3-edk2-devel.noarch** file conflicts with the **build.noarch** file due to duplicate file names.

  ```shell
  $ yum install python3-edk2-devel.noarch build.noarch
  ...
  Error: Transaction test error:
  file /usr/bin/build conflicts between attempted installs of python3-edk2-devel-202002-3.oe1.noarch and build-20191114-324.4.oe1.noarch
  ```

## 6. Failed to Downgrade libiscsi

### Symptom

libiscsi-1.19.0-4 or later fails to be downgraded to libiscsi-1.19.0-3 or earlier.

```text
Error:
Problem: problem with installed package libiscsi-utils-1.19.0-4.oe1.x86_64
- package libiscsi-utils-1.19.0-4.oe1.x86_64 requires libiscsi(x86-64) = 1.19.0-4.oe1, but none of the providers can be installed
- cannot install both libiscsi-1.19.0-3.oe1.x86_64 and libiscsi-1.19.0-4.oe1.x86_64
- cannot install both libiscsi-1.19.0-4.oe1.x86_64 and libiscsi-1.19.0-3.oe1.x86_64
- conflicting requests
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
```

### Possible Cause

In libiscsi-1.19.0-3 or earlier, binary files named **iscsi-xxx** are packed into the main package **libiscsi**. However, these binary files introduce improper dependency CUnit. To solve this problem, in libiscsi-1.19.0-4, these binary files are separated into the **libiscsi-utils** subpackage. The main package is weakly dependent on the subpackage. You can integrate or uninstall the subpackage during image building based on product requirements. If the subpackage is not integrated or is uninstalled, the functions of the **libiscsi** main package are not affected.
When libiscsi-1.19.0-4 or later is downgraded to libiscsi-1.19.0-3 or earlier and the **libiscsi-utils** subpackage is installed in the system, because libiscsi-1.19.0-3 or earlier does not contain **libiscsi-utils**, **libiscsi-utils** will fail to be downgraded. Due to the fact that **libiscsi-utils** depends on the **libiscsi** main package before the downgrade, a dependency problem occurs and the libiscsi downgrade fails.

### Solution

Run the following command to uninstall the **libiscsi-utils** subpackage and then perform the downgrade:

```shell
yum remove libiscsi-utils
```

## 7. Failed to Downgrade xfsprogs

### Symptom

xfsprogs-5.6.0-2 or later fails to be downgraded to xfsprogs-5.6.0-1 or earlier.

```text
Error:
Problem: problem with installed package xfsprogs-xfs_scrub-5.6.0-2.oe1.x86_64
- package xfsprogs-xfs_scrub-5.6.0-2.oe1.x86_64 requires xfsprogs = 5.6.0-2.oe1, but none of the providers can be installed
- cannot install both xfsprogs-5.6.0-1.oe1.x86_64 and xfsprogs-5.6.0-2.oe1.x86_64
- cannot install both xfsprogs-5.6.0-2.oe1.x86_64 and xfsprogs-5.6.0-1.oe1.x86_64
- conflicting requests
```

### Possible Cause

In xfsprogs-5.6.0-2, to reduce improper dependencies of the **xfsprogs** main package and separate experimental commands from the main package, the `xfs_scrub*` commands are separated into the **xfsprogs-xfs\_scrub** subpackage. The **xfsprogs** main package is weakly dependent on the **xfsprogs-xfs\_scrub** sub-package. You can integrate or uninstall the subpackage during image creation based on product requirements. If the subpackage is not integrated or is uninstalled, the functions of the **xfsprogs** main package are not affected.

When xfsprogs-5.6.0-2 or later is downgraded to xfsprogs-5.6.0-1 or earlier and the **xfsprogs-xfs\_scrub** subpackage is installed in the system, because xfsprogs-5.6.0-1 or earlier does not contain **xfsprogs-xfs\_scrub**, **xfsprogs-xfs\_scrub** will fail to be downgraded. Due to the fact that **xfsprogs-xfs\_scrub** depends on the **xfsprogs** main package before the downgrade, a dependency problem occurs and the xfsprogs downgrade fails.

### Solution

Run the following command to uninstall the **xfsprogs-xfs\_scrub** subpackage and then perform the downgrade:

```shell
yum remove xfsprogs-xfs_scrub
```

## 8. Failed to Downgrade elfutils

### Symptom

The dependency is missing. As a result, elfutils failed to be downgraded.

![](figures/1665628542704.png)

### Possible Cause

22.03-LTS, 22.03-LTS-Next: elfutils-0.185-12

master: elfutils-0.187-7

20.03-LTS-SP1: elfutils-0.180-9

In the preceding versions, the **eu-objdump**, **eu-readelf**, and **eu-nm** commands provided by the elfutils main package are split into the elfutils-extra subpackage. When elfutils-extra has been installed in the system and elfutils is downgraded, because an earlier version (such as the version of an earlier branch) cannot provide the corresponding elfutils-extra package, the elfutils-extra subpackage is not downgraded. However, the elfutils-extra subpackage depends on the elfutils package before the downgrade. As a result, the dependency problem cannot be resolved, and the elfutils subpackage fails to be downgraded.

### Solution

Run the following command to uninstall the elfutils-extra subpackage and then perform the downgrade:

```shell
yum remove -y elfutils-extra
```

## 9. CPython/Lib Detects CVE-2019-9674: Zip Bomb

### Symptom

**Lib/zipfile.py** in Python 3.7.2 or earlier allows remote attackers to create DoS requests using zip bombs, resulting in high resource consumption.

### Possible Cause

Remote attackers use zip bombs to cause denial of service, affecting target system services or even crashing the system. A zip bomb is a zip file with a high compression ratio. It may be several MB or dozens of MB in size. However, after decompression, a large amount of data is generated, consuming a large amount of resources.

### Solution

Add the alarm information to **zipfile** at <https://github.com/python/cpython/blob/3.7/Doc/library/zipfile.rst>.

## 1.: ReDoS Attack Occurs Due to Improper Use of glibc Regular Expressions

### Symptom

The regcomp/regexec interface of glibc is used for programming, or the glibc regular expressions, such as grep/sed, are used in shell commands. Improper regular expressions or inputs cause ReDoS attacks (CVE-2019-9192/CVE-2018-28796).
The typical regular expression pattern is the combination of the"reverse reference (\1)" with the "asterisk (\*)" (zero match or multiple matches), "plus sign (+)" (one match or multiple matches), or "{m,n}" (minimum match: m; maximum match: n); or the combination of ultra-long character strings with regular expressions. The following is an example:

```shell
$ echo D | grep -E "$(printf '(\0|)(\\1\\1)*')"Segmentation fault (core dumped)
$ grep -E "$(printf '(|)(\\1\\1)*')"
Segmentation fault (core dumped)
$ echo A | sed '/\(\)\(\1\1\)*/p'
Segmentation fault (core dumped)
$ time python -c 'print "a"*40000' | grep -E "a{1,32767}"
Segmentation fault (core dumped)
$ time python -c 'print "a"*40900' | grep -E "(a)\\1"
Segmentation fault (core dumped)
```

### Possible Cause

A core dump occurs on the process that uses the regular expression. The glibc regular expression is implemented using the NFA/DFA hybrid algorithm. The internal principle is to use a greedy algorithm for recursive query to match as many character strings as possible. The greedy algorithm causes the ReDoS attack when processing the recursive regular expression.

### Solution

1. Strict permission control is required to reduce the attack surface.

2. Ensure that the regular expression is correct. Do not enter an invalid regular expression or a combination of ultra-long character strings with regular expressions (references or asterisks) that may trigger infinite recursion.

   ```text
   # ()(\1\1)*
   # "a"*400000
   ```

3. After a user program detects a process exception, the user program can restart the process to restore services, improving program reliability.

## 1.: An Error Is Reported When gdbm-devel Is Installed or Uninstalled During the Installation and Uninstallation of httpd-devel and apr-util-devel

### Symptom

1. An error is reported when gdbm-devel-1.18.1-1 is installed or uninstalled.
2. After the error is rectified, gdbm and gdbm-devel are upgraded to the 1.18.1-2 version. However, the default version of gdbm-devel is still 1.18.1-1 when httpd-devel and apr-util-devel (dependent on gdbm-devel) are installed. As a result, the error persists.

### Possible Cause

1. The gdbm-devel-1.18.1-1 package does not contain the help package that provides `info`. As a result, the help package cannot be introduced when gdbm-devel is installed independently, and the following alarm information is displayed:

   ```text
   install-info: No such file or directory for /usr/share/info/gdbm.info.gz
   ```

2. By default, the gdbm-1.18.1-1 main package is installed in the system, but the gdbm-devel package is not installed. The software packages depending on gdbm-devel still match the version of the gdbm main package and install gdbm-devel-1.18.1-1. As a result, the error persists.

### Solution

1. Install gdbm-1.18.1-2 to upgrade gdbm. The error is rectified.
2. Upgrade gdbm, and then install gdbm-devel to make it depend on the gdbm of the later version. The error is rectified.

## 1.: An rpmdb Error Is Reported When Running the yum or dnf Command After the System Is Rebooted

### Symptom

1. After the system is rebooted, an error is reported when running an RPM-related command `yum` or `dnf` as follows:
   error: db5 error(-30973) from dbenv->open: BDB0087 DB\_RUNRECOVERY: Fatal error, run database recovery
   error: cannot open Packages index using db5 -  (-30973)
   error: cannot open Packages database in /var/lib/rpm
   Error: Error: rpmdb open failed

### Possible Cause

1. During an installation or upgrade, read and write operations are performed on the **/var/lib/rpm/\_\_db.00\*** file. If an unexpected interruption occurs, such as forced power-off, drive space full, or `kill -9`, the **\_db** file will be damaged. An error will be reported when a `dnf` or `yum` command is executed.

### Solution

Step 1 Run the `kill -9` command to terminate all running RPM-related commands.
Step 2 Run `rm -rf /var/lib/rpm/__db.00*` to delete all db.00 files.
Step 3 Run the `rpmdb --rebuilddb` command to rebuild the RPM database.

## 1.: Failed to Run `rpmrebuild -d /home/test filesystem` to Rebuild the filesystem Package

### Symptom

Failed to run the `rpmrebuild --comment-missing=y --keep-perm -b -d /home/test filesystem-3.16-3.oe1.aarch64` command to rebuild the **filesystem** package. The following information is displayed:

```text
/usr/lib/rpmrebuild/rpmrebuild.sh:Error:(RpmBuild) Package 'filesystem-3.16-3.oe1.aarch64' build failed.
/usr/lib/rpmrebuild/rpmrebuild.sh:Error: RpmBuild
```

### Possible Cause

The software package creates the directory in the **%pretrans -p** phase, and modify the directory in the **%ghost** phase. If you create a file or directory in the directory and use `rpmrebuild` to build the package, the created file or directory will be included in the package.

The root cause of the symptom is that **filesystem** creates the **/proc** directory in the **%pretrans** phase and modifies the directory in the **%ghost** phase, but some small processes are dynamically created during system running. As a result, `rpmrebuild` cannot include the processes in the package because they are not files or directories and fails to rebuild the package.

### Solution

Do not use `rpmrebuild` to rebuild the **filesystem** package.

## 1.: An Error Is Reported When modprobe or `insmod` Is Executed With the `-f` Option

### Symptom

An error is reported when `modprobe -f <mod>` or `insmod -f <mod>.ko.xz` is executed. For example, when `insmod -f xfs.ko.xz` is executed, error message **modprobe: ERROR: could not insert 'xfs': Key was rejected by service** is displayed.

Up till now (2022.09.20, kmod v30), this issue has not been fixed by the kmod community.Linux v5.17 [b1ae6dc](https://github.com/torvalds/linux/commit/b1ae6dc41eaaa98bb75671e0f3665bfda248c3e7) introduced support for compressed kernel modules, which is not supported by kmod.

### Possible Cause

`modprobe` and `insmod` use the `finit_module()` system call to load uncompressed ko files. For compressed ko files, kmod uses the `init_module()` system call to decompress them. `init_module()` does not take the ignore check flag. As a result, `mod_verify_sig()` is always executed by the kernel. the `-f` option of `modprobe` and `insmod` changes verification information about the ko file, resulting in verification failure of `mod_verify_sig()`.

### Solution

Do not use the `-f` option when running `insmod` or `modprobe` on compressed ko files.

---

---
url: /en/docs/common/faq/caselibrary/sysfile.md
---
# System File Recovery FAQ

## Context

During normal system usage, accidental deletion of system files or unintended modifications may prevent successful system boot. This requires file system recovery/repair or backup of critical data.

## Symptom

Missing or modified system files causing boot failure, requiring system file restoration.

## Solution

1. Mount the installation image and switch to the background console with **Ctrl+Alt+F2**.

   The drive will initially show as inactive and unavailable for direct operations.

   ![image](./figures/system_recovery_inactive.png)

2. Configure network and enable SSH service:

   ```txt
   ifconfig eth0 xx.xx.xx.xx netmask 255.255.255.0 up
   route add default gw xx.xx.xx.xx
   cp /etc/ssh/sshd_config.anaconda /etc/ssh/sshd_config
   systemctl restart sshd
   ```

3. Activate the system volume group:

   Use `vgchange -ay` to activate detected file systems.

   ![image](./figures/system_recovery_activate.png)

4. Mount partitions for system operations:
   1. Create a temporary directory `test` and mount the root partition **/dev/oprnruler/root** to it:

      ![image](./figures/system_recovery_mount_root.png)

   2. Mount the boot partition `/dev/sda2` to the temporary system's boot directory:

      ![image](./figures/system_recovery_mount_boot.png)

   3. The file system and data are now accessible for modifications.

---

---
url: /en/docs/common/faq/caselibrary/audit.md
---
# System Halt Caused by Audit Logs Consuming Drive Space

## Context

An unplanned system halt occurred, necessitating diagnosis.

## Symptom

Audit service logs confirmed it initiated the halt, contrary to configured log rotation settings.
The audit log directory contained excessive files occupying full drive capacity:

```txt
-r--------. 1 root root 6291639  May 14 04:10 audit.log.968
-r--------. 1 root root 6291629  May 14 03:28 audit.log.969
-r--------. 1 root root 6291630  May 14 02:45 audit.log.970
-r--------. 1 root root 6291627  May 14 02:03 audit.log.971
-r--------. 1 root root 6291546  May 14 01:20 audit.log.972
-r--------. 1 root root 6291689  May 14 00:38 audit.log.973
-r--------. 1 root root 6291705  May 13 23:57 audit.log.974
-r--------. 1 root root 6291528  May 13 23:14 audit.log.975
...
```

## Possible Causes

The issue appears to stem from failed audit log rotation. The **auditd.conf** settings is as follows:

```txt
...
max_log_file = 6 // 6 MB log file size limit
num_logs = 5     // Maximum of 5 log files
...
admin_space_left = 50
admin_space_left_action = halt // System halts if space drops below the threshold
...
```

The configuration allows only 5 log files, but the actual count exceeded this limit. The system halt was expected behavior.

Message logs revealed:

```txt
...
2024-06-09T04:59:46.424433+08:00 localhost auditd[21699]: Audit daemon rotating long files with keep option
...
```

The logs show rotation used the "keep option." The **auditd.conf** setting is as follows:

```txt
...
max_log_file_action = keep_logs // Similar to rotate but overrides num_logs setting.
...
```

The root cause is that `max_log_file_action = keep_logs` disabled the `num_logs = 5` limit, allowing logs to accumulate.

## Solution

Change `max_log_file_action` to `rotate`.

---

---
url: /en/docs/common/faq/server/system_management_faq.md
---
# System Performance FAQ

## When the NFS service is enabled on openEuler 22.03 SP1, why does the server response speed drop sharply from more than a gigabit to around 2 MB/s after less than a day of write operations

The performance drops sharply when the NFS server's cache uses more than 50% memory. This is mainly due to issues with memory allocation and reclamation mechanisms. The system reclaims memory in the background rather than during the memory allocation process. This is slower and will increase the waiting time (e.g., 500 ms latency) when the system cannot allocate enough memory in time. This issue is particularly pronounced under heavy loads since the NFS server requires substantial memory to process client requests. As the service runtime increases, the reduction in available memory leads to a dramatic performance drop, especially when there are intensive write operations.

## What do I do if a file fails to be created on openEuler due to an inode error of the Ext4 file system

Set the **rec\_len** field of the **dx\_node** block as follows:

Use `ext4_rec_len_from_disk()` to convert **rec\_len** to **65536** and then perform a comparison. This ensures that the **rec\_len** field is correctly set when a new **dx\_node** block is added.
Calculate and set a correct checksum for the node. This correction will prevent the inode error caused by incorrect checksums and thus allow the system to create and manage a large number of files correctly.

## How to reduce the excessive memory usage of service processes on openEuler due to the use of glibc tcache

Disable the tcache feature by setting the environment variables before starting the program. Specifically, add **GLIBC\_TUNABLES=glibc.malloc.tcache\_count=0** to **bash\_profile** to disable tcache. After the process starts, verify the process environment variables in **/proc/pid/environ** to ensure that the variable has been successfully added. Once tcache is disabled, the memory of this process will be managed in accordance with glibc 2.17, without any additional side effects. According to user feedback, this solution significantly reduces the memory usage of services on openEuler, resulting in lower memory usage than CentOS.

## When performing a fio multi-drive stress test, why is the performance of the Arm architecture only half that of the x86 architecture

The lower performance of the Arm architecture in the fio stress testing is due to differences in interrupt handling mechanisms. x86 achieves interrupt load balancing through APIC, while Arm's locality-specific peripheral interrupts (LPIs) are managed by Interrupt Translation Service (ITS), which by default assigns interrupts to the lowest-numbered core of each CPU. Therefore, performance bottlenecks occur on these cores during the multi-drive stress test.

## After the server is powered off and restarted, why does an I/O error occur when I run ls in the XFS file system

This error usually indicates that parts of the XFS file system failed to be loaded or read, possibly due to damaged file system metadata or incomplete write operations. If a power outage occurs during write operations, data may not be fully written to XFS, causing the XFS data to be inconsistent after the server reboots. Running a command such as ls at this time will result in an input/output error.

## What do I do if the new kernel is not used upon system boot

Replace the contents of the **/boot/grub2/grub.cfg** file with those of the **/boot/efi/EFI/xxxx/grub.cfg** file. This ensures that the system reads the correct configuration file containing the new kernel information upon system boot. In addition, check the system boot mode and confirm that UEFI is being used.

---

---
url: /en/docs/common/faq/caselibrary/systemd-logind.md
---
# systemd-logind.service Failed to Retrieve NIS User Information

## Context

When using NIS-synchronized users with the systemd-pam package installed, systemd fails to create corresponding UID files under **/run/systemd/users/** during user login, resulting in gnome-shell errors.

NIS provides centralized network resource management including users, passwords, home directories, and group information across multiple systems.

Related issue: <https://gitee.com/src-openeuler/systemd/issues/I9OXNC?from=project-issue>

### Version Information

systemd version: systemd-249-75.oe2203sp1.aarch64
systemd-pam version: systemd-pam-249-75.oe2203sp1.aarch64

## Symptom

A **ylp** NIS user (UID 1015) can be identified through the `id` command but lacks corresponding UID file creation in **/run/systemd/users/** after login:

```txt
[root@server1 ~]# id ylp
uid=1015(ylp) gid=1015(ylp) groups=1015(ylp)
[root@server1 ~]# ll /run/systemd/users/
total 16
-rw-r--r-- 1 root root 345 May 13 17:23 0
-rw-r--r-- 1 root root 243 May 10 17:40 1002
-rw-r--r-- 1 root root 252 May 13 17:20 1004
-rw-r--r-- 1 root root 274 May 10 18:12 971
```

## Possible Causes

1. Network connectivity issues preventing NIS server communication, as shown in error logs:

   ```txt
   systemd-logind[2989387]: yp_bind_client_create_v3: RPC: Remote system error - Address family not supported by protocol
   server1 sshd[2989498]: pam_systemd(sshd:session): Failed to create session: No such process
   server1 sshd[2989498]: pam_unix(sshd:session): session opened for user ylp(uid=1015) by (uid=0)
   server1 sshd[2989498]: pam_systemd(sshd:session): Failed to create session: Transport endpoint is not connected
   ```

2. Compatibility changes in systemd versions based on community discussions:

   * Version 235 added `IPAddressDeny=any` to services like systemd-logind.service, blocking external IP communication:\
     <https://github.com/systemd/systemd/pull/6921>

   * Version 239 further restricted network protocols (`AF_INET`/`AF_INET6`) for systemd-logind.service:\
     <https://github.com/systemd/systemd/issues/9072>

3. Systemd community's official stance on compatibility:

   * Compatibility notice added to NEWS documentation:\
     <https://github.com/systemd/systemd/pull/7343>

   * Maintainers recommend using NSCD/SSSD instead of direct NIS integration for security reasons:\
     <https://github.com/systemd/systemd/issues/7074#issuecomment-338157851>

## Solution

### Option 1

Add NSCD or SSSD services to access local cached data.

### Option 2

Manually enable network access for systemd-logind.service.

1. Check configuration files in **/usr/lib/systemd/system/systemd-logind.service.d/** on client machines for existing `IPAddressAllow` and `RestrictAddressFamilies` parameters. Use these commands to verify:

   ```bash
   grep -rn "IPAddressAllow"
   grep -rn "RestrictAddressFamilies"
   ```

   **Scenario 1**: If parameters do not exist, create a configuration file **systemd-logind-nis.conf** (with same permissions as other files in directory) containing:

   ```ini
   IPAddressAllow=NIS_server_IP_address_to_be_allowed
   RestrictAddressFamilies=AF_UNIX AF_NETLINK AF_INET AF_INET6
   ```

   **Scenario 2**: If parameters exist, append configurations to existing files:

   ```ini
   IPAddressAllow=Original_configuration Additional_NIS_server_IP
   RestrictAddressFamilies=Original_configuration AF_UNIX AF_NETLINK AF_INET AF_INET6
   ```

   **Scenario 3**: If a file contains only one of the parameters, append the configuration to the existing parameter in that file, and write the missing parameter to **/usr/lib/systemd/system/systemd-logind.service.d/systemd-logind-nis.conf** (refer to Scenarios 1 and 2).

2. Restart the service after configuration changes:

   ```bash
   systemctl daemon-reload
   systemctl restart systemd-logind.service
   ```

---

---
url: /zh/docs/common/faq/caselibrary/systemd-logind.md
---
# systemd-logind.service无法获取NIS服务器上的用户信息的解决方法

## 问题背景

在systemd-pam软件包已安装的前提下，使用NIS服务同步过来的用户登录时， /run/systemd/users/下面没有创建对应的uid文件，导致gnome-shell报错。

NIS（Network Information Service）是一种为网络中所有的机器提供网络信息的系统，包括用户名、密码、主目录、组信息等。NIS服务主要用于集中控制多个系统管理数据库的网络用品，其全称是Network Information Service。

相关issue：<https://gitee.com/src-openeuler/systemd/issues/I9OXNC?from=project-issue>

### 版本信息

systemd版本： systemd-249-75.oe2203sp1.aarch64

systemd-pam版本： systemd-pam-249-75.oe2203sp1.aarch64

## 现象描述

NIS服务器上创建了一个1015(ylp)用户，用户信息可以使用id命令查询到，但是登录1015用户后 /run/systemd/users/ 目录下没有uid对应的文件。

```txt
[root@server1 ~]# id ylp
uid=1015(ylp) gid=1015(ylp) groups=1015(ylp)
[root@server1 ~]# ll /run/systemd/users/
total 16
-rw-r--r-- 1 root root 345 May 13 17:23 0
-rw-r--r-- 1 root root 243 May 10 17:40 1002
-rw-r--r-- 1 root root 252 May 13 17:20 1004
-rw-r--r-- 1 root root 274 May 10 18:12 971
```

## 原因分析

1. 根据错误打印可以看出来是网络不通导致的连接不上NIS服务器。

   ```txt
   systemd-logind[2989387]: yp_bind_client_create_v3: RPC: Remote system error - Address family not supported by protocol
   server1 sshd[2989498]: pam_systemd(sshd:session): Failed to create session: No such process
   server1 sshd[2989498]: pam_unix(sshd:session): session opened for user ylp(uid=1015) by (uid=0)
   server1 sshd[2989498]: pam_systemd(sshd:session): Failed to create session: Transport endpoint is not connected
   ```

2. 查找systemd社区， 发现了类似的提交和讨论。以下两个提交都可能会导致systemd-logind无法与NIS服务器建立通信：

   * systemd在235版本对许多常驻的service（比如：systemd-logind.service）增加了`IPAddressDeny=any`，防止了与外部ip通信：<https://github.com/systemd/systemd/pull/6921>
   * systemd在239版本对systemd-logind.service的通信做了进一步的限制：禁止了`AF_INET` 和 `AF_INET6` 协议：<https://github.com/systemd/systemd/issues/9072>

3. 最终systemd社区对systemd-logind.service的兼容性变更做了一些解释以及文档承载：

   * NWES文档中增加了不兼容说明：<https://github.com/systemd/systemd/pull/7343>
   * systemd的开发者认为类似NIS的网络通信会增加遭受网络攻击的风险，最好使用 NSCD 或 SSSD （缓存远程用户信息的服务），systemd不会去改变这个默认的策略: <https://github.com/systemd/systemd/issues/7074#issuecomment-338157851>

## 解决方案

方案一：

添加NSCD或SSSD服务，以访问本地缓存数据。

方案二：

用户自行开启systemd-logind.service网络访问。

1. 首先在客户端的/usr/lib/systemd/system/systemd-logind.service.d/目录下查找 `IPAddressAllow`、`RestrictAddressFamilies`字段，保证这两个字段没有被其他服务重新设置。

   ```bash
   grep -rn "IPAddressAllow"
   grep -rn "RestrictAddressFamilies"
   ```

   **场景1**：如果有字段不存在，则在/usr/lib/systemd/system/systemd-logind.service.d/目录下新增配置文件systemd-logind-nis.conf（权限与其他文件保持一致），并在文件中新增`IPAddressAllow`和`RestrictAddressFamilies`中不存在的字段，IPAddressAllow字段增加`NIS服务器的IP地址`，RestrictAddressFamilies字段增加`AF_UNIX AF_NETLINK AF_INET AF_INET6`。如下所示:

   ```bash
   IPAddressAllow=需增加的NIS服务器的IP地址
   RestrictAddressFamilies=AF_UNIX AF_NETLINK AF_INET AF_INET6
   ```

   **场景2**：如果有文件中存在这两个字段，则在包含该字段的文件中追加配置。IPAddressAllow字段追加`NIS服务器的IP地址`，RestrictAddressFamilies字段追加`AF_UNIX AF_NETLINK AF_INET AF_INET6`。如下所示:

   ```bash
   IPAddressAllow=原配置 需增加的NIS服务器的IP地址
   RestrictAddressFamilies=原配置 AF_UNIX AF_NETLINK AF_INET AF_INET6
   ```

   **场景3**：如果有文件中只配置了其中一个字段，则在包含该字段的文件中追加配置，并将不存在的字段写入到/usr/lib/systemd/system/systemd-logind.service.d/systemd-logind-nis.conf中（参考场景1和场景2）。

2. 修改后需要在客户端重启systemd-logind.service服务。

   ```bash
   systemctl daemon-reload
   systemctl restart systemd-logind.service
   ```

---

---
url: /en/docs/common/faq/server/trusted_computing_faqs.md
---
# Trusted Computing FAQ

## 1. System Fails to Boot After IMA Appraisal Enforce Mode Is Enabled with the Default Policy

### Possible Causes

The default IMA policy may include checks for critical file access processes such as application execution and kernel module loading. If access to these critical files fails, the system may fail to boot. Common causes include:

1. The IMA verification certificate is not imported into the kernel, causing the digest list to fail verification.
2. The digest list file is not correctly signed, leading to verification failure.
3. The digest list file is not imported into the initrd, preventing the digest list from being loaded during the boot process.
4. The digest list file does not match the application, causing the application to fail matching the imported digest list.

### Solution

Enter the system in log mode to locate and fix the issue. Reboot the system, enter the GRUB menu, and modify the boot parameters to start in log mode:

```ini
ima_appraise=log
```

After the system boots, follow the steps below to troubleshoot.

**Step 1:** Check the IMA certificates in the key ring.

```shell
keyctl show %:.builtin_trusted_keys
```

For openEuler LTS versions, at least the following kernel certificates should exist (for other versions, reference based on their release dates):

If you have imported other kernel root certificates, use the `keyctl` command to confirm whether the certificates were successfully imported. By default, openEuler does not use the IMA key ring. If you are using it, check whether the user certificates exist in the IMA key ring with the following command:

```shell
keyctl show %:.ima
```

If the issue is that the certificate was not correctly imported, refer to *User Certificate Import* for troubleshooting.

**Step 2:** Check if the digest list contains signature information.

Query the digest list files in the current system with the following command:

```shell
ls /etc/ima/digest_lists | grep '_list-compact-'
```

For each digest list file, ensure that **one of the following three** signature conditions is met:

1. The digest list file has a corresponding **RPM digest list file**, and the `security.ima` extended attribute of the **RPM digest list file** contains a signature value. For example, for the bash package digest list, the digest list file path is:

   ```text
   /etc/ima/digest_lists/0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64
   ```

   The RPM digest list path is:

   ```text
   /etc/ima/digest_lists/0-metadata_list-rpm-bash-5.1.8-6.oe2203sp1.x86_64
   ```

   Check the RPM digest list signature by ensuring the `security.ima` extended attribute is not empty:

   ```shell
   getfattr -n security.ima /etc/ima/digest_lists/0-metadata_list-rpm-bash-5.1.8-6.oe2203sp1.x86_64
   ```

2. The `security.ima` extended attribute of the digest list file is not empty:

   ```shell
   getfattr -n security.ima /etc/ima/digest_lists/0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64
   ```

3. The digest list file contains signature information at the end. Verify if the file content ends with the `~Module signature appended~` magic string (supported in openEuler 24.03 LTS and later versions):

   ```shell
   tail -c 28 /etc/ima/digest_lists/0-metadata_list-compact-kernel-6.6.0-28.0.0.34.oe2403.x86_64
   ```

If the issue is that the digest list does not contain signature information, refer to *Digest List File Signing Methods* for troubleshooting.

**Step 3:** Verify the correctness of the digest list signature.

After ensuring that the digest list contains signature information, also ensure that the digest list is signed with the correct private key, meaning the signing private key matches the certificate in the kernel. In addition to manually checking the private key, users can check the dmesg logs or audit logs (default path: **/var/log/audit/audit.log**) for signature verification failures. A typical log output is as follows:

```ini
type=INTEGRITY_DATA msg=audit(1722578008.756:154): pid=3358 uid=0 auid=0 ses=1 subj=unconfined_u:unconfined_r:haikang_t:s0-s0:c0.c1023 op=appraise_data cause=invalid-signature comm="bash" name="/root/0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64" dev="dm-0" ino=785161 res=0 errno=0UID="root" AUID="root"
```

If the issue is incorrect signature information, refer to *Digest List File Signing Methods* for troubleshooting.

**Step 4:** Check if the digest list file is imported into the initrd.

Query whether the digest list file exists in the current initrd with the following command:

```shell
lsinitrd | grep 'etc/ima/digest_lists'
```

If no digest list file is found, users need to recreate the initrd and verify that the digest list is successfully imported:

```shell
dracut -f -e xattr
```

**Step 5:** Verify that the IMA digest list matches the application.

Refer to [Question 2](#2-file-execution-fails-after-ima-appraisal-enforce-mode-is-enabled).

## 2. File Execution Fails After IMA Appraisal Enforce Mode Is Enabled

### Possible Causes

After IMA appraisal enforce mode is enabled, if the content or extended attributes of a file configured with IMA policies are incorrect (for example, they do not match the imported digest list), file access may be denied. Common causes include:

1. The digest list was not successfully imported (refer to [Question 1](#1-system-fails-to-boot-after-ima-appraisal-enforce-mode-is-enabled-with-the-default-policy).
2. The file content or attributes have been tampered with.

### Solution

For scenarios where file execution fails, first ensure that the digest list file has been successfully imported into the kernel. Check the number of digest lists to determine the import status:

```shell
cat /sys/kernel/security/ima/digests_count
```

Next, use the audit logs (default path: **/var/log/audit/audit.log**) to identify which file failed verification and the reason. A typical log output is as follows:

```ini
type=INTEGRITY_DATA msg=audit(1722811960.997:2967): pid=7613 uid=0 auid=0 ses=1 subj=unconfined_u:unconfined_r:haikang_t:s0-s0:c0.c1023 op=appraise_data cause=IMA-signature-required comm="bash" name="/root/test" dev="dm-0" ino=814424 res=0 errno=0UID="root" AUID="root"
```

After identifying the file that failed verification, compare it with the TLV digest list to determine the cause of tampering. For scenarios where extended attribute verification is not enabled, only compare the SHA256 hash value of the file with the `IMA digest` entry in the TLV digest list. For scenarios where extended attribute verification is enabled, also compare the current file attributes with the extended attributes displayed in the TLV digest list.

Once the cause of the issue is determined, resolve it by restoring the file content and attributes or regenerating the digest list for the file, signing it, and importing it into the kernel.

## 3. Errors Occur During Packages Installation Across openEuler 22.03 LTS SP Versions After IMA Appraisal Mode Is Enabled

### Possible Causes

After IMA appraisal mode is enabled, installing packages from different SP versions of openEuler 22.03 LTS triggers the import of IMA digest lists. This process includes a signature verification step, where the kernel uses its certificates to verify the digest list signatures. Due to changes in signing certificates during the evolution of openEuler, backward compatibility issues may arise in certain cross-SP-version installation scenarios (there are no forward compatibility issues, meaning newer kernels can verify older IMA digest list files without problems).

### Solution

You are advised to ensure that the following signing certificates are present in the current kernel:

```shell
# keyctl show %:.builtin_trusted_keys
Keyring
 566488577 ---lswrv      0     0  keyring: .builtin_trusted_keys
 383580336 ---lswrv      0     0   \_ asymmetric: openeuler <openeuler@compass-ci.com> b675600b
 453794670 ---lswrv      0     0   \_ asymmetric: private OBS b25e7f66
 938520011 ---lswrv      0     0   \_ asymmetric: openeuler <openeuler@compass-ci.com> fb37bc6f
```

If any certificates are missing, you are advised to upgrade the kernel to the latest version:

```shell
yum update kernel
```

openEuler 24.03 LTS and later versions include dedicated IMA certificates and support certificate chain verification, ensuring the certificate lifecycle covers the entire LTS version.

## 4. IMA Digest List Import Fails Despite Correct Signatures After IMA Digest List Appraisal Mode Is Enabled

### Possible Causes

The IMA digest list import process includes a verification mechanism. If a digest list fails signature verification during import, the digest list import functionality is disabled, preventing even correctly signed digest lists from being imported afterward. Check the dmesg logs for the following message to confirm if this is the cause:

```shell
# dmesg
ima: 0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64 not appraised, disabling digest lists lookup for appraisal
```

If such a log is present, a digest list file with an incorrect signature was imported while IMA digest list appraisal mode was enabled, causing the functionality to be disabled.

### Solution

Reboot the system and fix the incorrect digest list signature information.

## 5. Importing User-Defined IMA Certificates Fails in openEuler 24.03 LTS and Later Versions

Linux kernel 6.6 introduced additional field validation restrictions for importing certificates. Certificates imported into the IMA key ring must meet the following constraints (following the X.509 standard format):

* It must be a digital signature certificate, meaning the `keyUsage=digitalSignature` field must be set.
* It must not be a CA certificate, meaning the `basicConstraints=CA:TRUE` field must not be set.
* It must not be an intermediate certificate, meaning the `keyUsage=keyCertSign` field must not be set.

## 6. kdump Service Fails to Start After IMA Appraisal Mode Is Enabled

After IMA appraisal enforce mode is enabled, if the IMA policy includes the following `KEXEC_KERNEL_CHECK` rule, the kdump service may fail to start:

```shell
appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig
```

The reason is that in this scenario, all files loaded via `kexec` must undergo integrity verification. As a result, the kernel restricts the loading of kernel image files by kdump to the `kexec_file_load` system call. This can be enabled by modifying the **/etc/sysconfig/kdump** configuration file:

```shell
KDUMP_FILE_LOAD="on"
```

Additionally, the `kexec_file_load` system call itself performs signature verification on the files. Therefore, the kernel image file being loaded must contain a valid secure boot signature, and the current kernel must include the corresponding verification certificate.

## 7. RAS Fails to Start After Installation

### Possible Causes

In the current RAS design logic, the program requires an `ecdsakey.pub` file in its working directory upon startup. This file serves as an authentication key for subsequent access. If the file is missing, RAS will fail to start.

### Solution

* Run `ras -T` to generate a test token—this will automatically create `ecdsakey.pub`.
* Alternatively, if deploying a custom OAuth2 service, save the corresponding JWT token verification public key as `ecdsakey.pub`.

## 8. RAS REST API Unreachable After Startup

By default, RAS starts in HTTPS mode, requiring a valid certificate for proper access. If it runs in HTTP mode, no certificate is needed.

---

---
url: /en/docs/common/faq/virtualization/virt_faq.md
---
# Virtualization FAQ

## 1. Why is the QEMU hot patch created with the libcareplus tool unable to be loaded

This issue occurs because the QEMU version is inconsistent with the hot patch version. You can download the source code of the corresponding QEMU version and use the buildID to ensure that the environment for creating a hot patch matches the environment for creating a QEMU package. If you do not have the environment for making the QEMU version, you can compile and install the QEMU version and use the buildID of **`/usr/libexec/qemu-kvm`** in the self-compiled package.

## 2. Why is the hot patch made using the libcareplus tool loaded but not working

Check whether the functions of the patch are infinite loop, non-exit, and recursive functions or initialization functions, inline functions, and short functions that are shorter than 5 bytes. These functions are within the constraints.

## 3. Why does the initial display of kvmtop show high variability from two samples taken 0.05 seconds apart

This occurs due to a known limitation in the open source top framework, with no current resolution available.

---

---
url: /en/docs/common/faq/community_tools/xfce_faq.md
---
# Xfce FAQ

## 1. Why Is the Background Color of the LightDM Login Page Black

The login page is black because `background` is not set in the default configuration file **/etc/lightdm/lightdm-gtk-greeter.conf** of lightdm-gtk.

Set `background=/usr/share/backgrounds/xfce/xfce-blue.jpg` in the `greeter` section at the end of the configuration file, and then run the `systemctl restart lightdm` command.

---

---
url: /zh/docs/common/faq/community_tools/xfce_faq.md
---
# xfce常见问题与解决方法

## **问题1:lightdm登录界面背景是黑色的**

原因： 登录界面是黑色的是因为lightdm-gtk默认配置文件/etc/lightdm/lightdm-gtk-greeter.conf中没有设置background。

解决方法：可以在该配置文件最后的\[greeter]段中设置 background=/usr/share/backgrounds/xfce/xfce-blue.jpg
然后使用“systemctl restart lightdm”命令就可以看到背景了。

---

---
url: /en/docs/common/faq/caselibrary/zabbix.md
---
# Zabbix Installation Guide for openEuler 22.03 LTS

## Minimal openEuler Setup

Disable the system firewall:

```shell
systemctl stop firewalld
systemctl disable firewalld
```

## MySQL Installation and Setup

1. Install MySQL components.

   ```shell
   dnf install mysql mysql-server mysql-common mysql-libs mysql-devel mysql-selinux --nogpgcheck
   ```

2. Configure MySQL service.

   ```shell
   systemctl enable mysqld
   systemctl start mysqld
   systemctl status mysqld
   ```

3. Set **root** password.

   ```mysql
   mysql -uroot -p
   > Press Enter when prompted
   ALTER USER 'root'@'localhost' IDENTIFIED BY 'secure_password';
   ```

## Zabbix Service Installation

```shell
dnf config-manager --add-repo https://repo.oepkgs.net/openeuler/rpm/openEuler-22.03-LTS/contrib/others/aarch64/
dnf clean all && dnf makecache
dnf install zabbix-server-mysql zabbix-web-mysql zabbix-nginx-conf zabbix-sql-scripts zabbix-agent --nogpgcheck
```

## Zabbix Environment Configuration

1. Prepare the database.

   ```mysql
   mysql -uroot -p
   > Enter your password
   create database zabbix character set utf8mb4 collate utf8mb4_bin;
   create user zabbix@localhost identified by 'zabbix_password';
   grant all privileges on zabbix.* to zabbix@localhost;
   set global log_bin_trust_function_creators = 1;
   quit;
   ```

2. Import initial data.

   ```shell
   zcat /usr/share/doc/zabbix-sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix
   ```

   Disable the temporary MySQL setting.

   ```mysql
   mysql -uroot -p
   > Enter your password
   set global log_bin_trust_function_creators = 0;
   quit;
   ```

3. Update configuration files.

   ```shell
   vi /etc/zabbix/zabbix_server.conf
   ---
   DBPassword=zabbix_password
   ---
   vi /etc/nginx/conf.d/zabbix.conf
   ---
   listen 8080; # Uncomment
   server_name example.com; # Uncomment
   ---
   ```

4. Update net-snmp.

   ```shell
   dnf install net-snmp net-snmp-devel net-snmp-utils --nogpgcheck
   ```

5. Launch zabbix services.

   ```shell
   systemctl restart zabbix-server zabbix-agent nginx php-fpm
   systemctl enable zabbix-server zabbix-agent nginx php-fpm
   ```

6. Access the web interface using port 8080.

---

---
url: /zh/docs/common/contribute/templates/feature_user_guide/description.md
---
# 介绍

## 简介

用户在初步接触特性时，需要对特性有一个整体、清晰的认知，以便于部署和应用特性。

## 架构介绍

特性的软件架构，组成，每个模块的含义，作用等。

## 规格

用户运营维护中需要了解并参考特性涉及的各种指标。

## 参考标准和协议

特性实现中遵循的协议和标准，同时提供协议和标准的版本信息。

## 可获得性

特性引入的最早软件版本信息及License支持情况。

## 约束与限制

用户需了解特性应用的限制和约束。

## 应用场景

描述何种情况下、如何应用该特性。提供具体清晰的任务场景，来源是用户实际使用情况。

## 原理描述

帮助用户了解特性的工作原理。包括基本技术原理和特性运作的业务流程。可提供流程图等。

---

---
url: /zh/docs/common/contribute/templates/feature_user_guide/usage.md
---
# 使用XXX

## 使用说明

请将每一步操作步骤都呈现出来，避免默认用户已知某些步骤而不提供的情况

## 验证说明

提供验证软件/特性生效的明确步骤。

---

---
url: /zh/docs/common/faq/server/kernel_faqs.md
---
# 内核热升级常见问题与解决方法

## **问题1:执行nvwa update后未升级**

原因：保留现场或者内核替换过程中出现错误。

解决方法：查看日志，找出错误原因。

## **问题2:开启加速特性后，nvwa执行命令失败**

原因：nvwa提供了诸多加速特性，包括quick kexec，pin memory，cpu park等等。这些特性都涉及到cmdline的配置和内存的分配，在选取内存时，通过cat /proc/iomemory确保选取的内存没有与其他程序冲突。

解决方法：必要时，通过dmesg查看使能特性后是否存在错误日志。

## **问题3:热升级后，相关现场未被恢复**

原因：首先检查nvwa服务是否运行，运行情况下，可能存在两种情况：一种是服务恢复失败，一种是进程恢复失败。

解决方法：通过service nvwa status查看nvwa的日志，如果是服务启动失败，首先确认是否使能了该服务，再通过systemd查看对应服务的日志。进一步的日志，去criu\_dir指定的路径对应命名的进程/服务文件夹中。其中dump.log为保存现场产生的日志，restore.log为恢复现场产生的。

## **问题4:恢复失败，日志显示Can't fork for 948: File exists**

原因：内核热升级工具在恢复程序过程中，发现程序的pid已经被占用。

解决方法：当前内核没有提供保留pid的机制，相关策略正在开发，预计会在将来的内核版本中解决这一限制，当前仅能手动重启相关进程。

## **问题5:使用nvwa去保存和恢复简单程序(hello world)，显示失败或者程序未在执行**

原因: criu使用存在诸多限制。

解决办法：查看nvwa的日志，如果显示是criu相关的错误，去相应的目录下检查dump.log或者restore.log，criu相关使用限制，可以参考[criu社区wiki](https://criu.org/What_cannot_be_checkpointed)。

---

---
url: /zh/docs/common/contribute/documentation_writing_specifications.md
---
# 写作规范

本写作规范针对openEuler docs 仓的文档结构、内容元素和语言风格提出规范要求，确保openEuler文档具备一致风格。

开发者开始openEuler文档写作前，建议先了解本规范内容，**欢迎提出改进意见**。

## 文档结构规范

特性手册内容一般包括概述、背景介绍、操作类文档（安装与部署、使用指南）、常见问题和附录，开发者可以根据项目实际情况增加或删减。
以[A-Tune 项目](https://docs.openeuler.org/zh/docs/22.03_LTS_SP2/docs/A-Tune/A-Tune.html)为例，可以参考如下内容：

### 概述

一句话介绍特性定义与功能，一句话说明读者对象。

【举例】

```markdown
本文档介绍openEuler系统性能自优化软件A-Tune的安装部署和使用方法，以指导开发者快速了解并使用A-Tune。
本文档适用于使用openEuler系统并希望了解和使用A-Tune的社区开发者、开源爱好者以及相关合作伙伴，使用人员需要具备基本的Linux操作系统知识。
```

### 背景介绍

背景介绍类文档写明特性背景与简介、架构说明等。常见文档标题：`认识xxx`。

### 操作类文档

* 环境要求

执行此操作需要准备的软硬件环境、权限以及其它约束条件。

【举例】

```markdown
硬件要求：xxx处理器。
软件要求：openEuler xx版本、root权限。
```

* 操作步骤

操作步骤文档包含**安装与部署**、**使用方法**等说明内容。

具体的操作步骤，需要注意如下事项：

```
- 建议一步一个操作步骤，不建议多个操作步骤合并在一个步骤中描写。
- 如果操作可选，要明确可选条件。
- 开发步骤中，涉及调用接口（例如使用了工具或者 SQL 语句），需要对使用的接口进行说明。
```

* 结果验证

说明如何验证操作结果正确。如果验证操作与步骤强相关，可以在步骤中描述。例如，执行 SQL 语句的返回信息。

### 附录

附录可包含术语与缩略语介绍。

## 内容元素规范

### 命名

对于新增文档，请在对应的文件目录下新增 MarkDown 文档（即以 .md 结尾的文件）。

【规则】zh/en目录下，新增文档名称不能与已有文档重名，如果有请重新命名。

【规则】文件名需要以**英文**小写命名。

【规则】若文件名有多个单词，请以下划线（\_）连接。

【规则】同一篇文档，中英文文档文件名保持一致。

【举例】

```text
installation_and_deployment.md  #新增‘安装与部署’文档
```

### 标题

【规则】标题尽量采用简洁的语句概况反映章节的中心内容，注意不要省略必要的信息。

【规则】操作类文档标题尽量用动宾结构（例如：申请权限）；相同级别，相同类型的标题结构保持一致。

【规则】标题不使用标点符号结尾，标题中尽量采用圆括号来表示补充说明，标题中不能出现特殊字符，如“？”。

【规则】标题与正文使用 1 整行换行隔开。

【规则】标题使用 “#” 空格连接标题名，标题级别一次只能增加一个级别且第一个标题应该是顶层标题。

【举例】

```markdown
# 一级标题
## 二级标题
### 三级标题
#### 四级标题
##### 五级标题
###### 六级标题
```

### 正文

【使用方法】

* 斜体：使用一个星号（\*）表示斜体。

  ```txt
  *斜体文本*
  ```

* 粗体：使用两个星号（\*\*）表示粗体。

  ```txt
  **粗体文本**
  ```

* 粗斜体：使用3个星号（\*\*\*）表示粗斜体。

  ```txt
  ***粗斜体文本***
  ```

* 转义：对特定内容使用转义符 \。

  ```txt
  \<转义的标记符号>
  ```

【规则】该转义的字符必须严格用转义符 \ 。

【规则】如果有连续两个转义符，转义符之间要有空格 { }。

【规则】国际化：需同时提供中英文文档，可联系[ECHO](https://gitee.com/echo10111111)和[wu-donger](https://gitee.com/wu-donger)协助翻译；如需参与英文文档贡献，可参考[英文版](../../en/contribute/documentation_writing_specifications.md)。

### 空格

【规则】编辑文档时中文和英文之间**建议**加空格，页面展示更美观，请保持全文一致。如果是产品名词如 “豆瓣FM”，请按照官方定义格式书写。

例如：openEuler 是一款开源操作系统。当前 openEuler 内核源于 Linux ，支持鲲鹏及其它多种处理器。

【规则】中文和数字之间加空格。

【规则】数字和单位之间不加空格。

【规则】全角标点与其他字符之间不加空格。

例如：刚刚成为了 openEuler 的 maintainer，好开心。

### 图片

【使用方法】

```bash
![alt 属性文本](图片地址)
 
![alt 属性文本](图片地址 "可选标题")
```

【规则】图片统一存放到文档同级目录下的 figures 文件夹中。例如，[《A-Tune用户指南》](https://docs.openeuler.org/zh/docs/22.03_LTS_SP2/docs/A-Tune/A-Tune.html)中的手册中使用的图片，统一存储在 [A-Tune](https://gitee.com/openeuler/docs-centralized/tree/stable2-22.03_LTS_SP2/docs/zh/docs/A-Tune/figures) 路径下。该文件夹下的文件引用图片时，使用相对引用。

【规则】请使用原创图片，避免存在知识产权侵权风险。

【规则】图文配合使用，切忌图文分离。

【规则】图片格式首选 png，此外也接受 jpg。图片的高不超过 640px，宽不超过 393px，图片大小建议不超过 150K。

【规则】中文用中文插图，英文用英文插图。

【规则】图片路径不能包含中文。

【规则】如果是截图，请在允许的范围内只保留有用的信息。图形中需要突出的关键信息，可增加红色框线或者文字备注说明。

【举例】
​图片以 `![](./figures/ci检查结果.jpg)` 格式书写，“./” 不可少，否则图片无法显示到现网。

### 代码块

代码示例说明了如何实现特定功能，开发人员使用代码示例来编写和调试代码。

【规则】代码的逻辑和语法正确。

【规则】代码的输入和输出尽可能的分开。

【规则】保证代码中关键步骤要有注释说明。

【规则】文中行内代码和命令行使用 1 对反引号，如： `代码块`。

【规则】块级代码使用 3 个反引号或 4 个空格（不能用 TAB 键）缩进，且上下均用整行隔开。

【举例】

* 行内代码

  ```txt
  `printf()` 函数
  ```

* 块级代码

  ```python
  #!/usr/bin/env python3
  print("Hello, World!");
  ```

  ```c
  #include  <stdio.h>
  int main(void)
  {
      printf("Hello world\n");
  }
  ```

### 列表

* 无序列表：无序列表使用星号（**\***）、加号（**+**）或是减号（**-**）作为列表标记，这些标记后面要添加一个空格，然后再填写内容。同一个无序列表，建议使用同一个符号。

  ```txt
  * 第一项
  * 第二项
  * 第三项
   
  + 第一项
  + 第二项
  + 第三项
   
   
  - 第一项
  - 第二项
  - 第三项
  ```

* 有序列表：有序列表使用数字并加上  **.**  号来表示。

  ```txt
  1. 第一项
  2. 第二项
  3. 第三项
  ```

* 嵌套列表：列表嵌套只需在子列表中的选项前面添加四个空格（注意不是Tab键）即可。

  ```txt
  1. 第一项：
      - 第一项嵌套的第一个元素
      - 第一项嵌套的第二个元素
  2. 第二项：
      - 第二项嵌套的第一个元素
      - 第二项嵌套的第二个元素
  ```

【规则】有明显先后逻辑顺序情况请使用有序列表，并列关系、多选一情况请使用无序列表。

【规则】当项目列表是术语、短语时，统一不加标点符号。

【规则】当项目列表是句子时，统一加句号。

【规则】特殊情况下如果不能避免出现短语和句子混合的情况，统一加句号。

【规则】项目列表前几项以分号结尾，最后一项以句号结尾的形式也可以接受。

### 注释符号

文档中会出现以下注释符号，代表不同的使用场景和提示程度。如果需要提示用户注意的信息，可以根据重要程度选择对应的注释符号。

| 注释符号 | 用途/含义                                                        | 使用方法                                                                    |
|--------|------------------------------------------------------------------|-----------------------------------------------------------------------------|
| 注意   | 如未按该注意事项操作，可能会导致任务中断或结果异常，但是可恢复。 | `> [!WARNING]注意`  `> 正文内容`   |
| 说明   | 提供帮助提示或有用的参考信息。                                     | `> [!NOTE]说明`  `> 正文内容`     |

> \[!NOTE]说明
>
> * 请根据文档具体场景选择对应的注释符号，并按照使用方法正确使用样式。方括号内是英文感叹号。
> * 说明/注意样式内可嵌套有序/无序列表，但不建议表格和代码块。
> * 为避免样式断开，需要保证 `>`连续。
> * 说明/注意内容避免过长，可考虑写在正文或者分段，请不要添加过多样式内空行。

### 链接

【规则】链接需要确保指向的目标文件存在，否则会造成链接跳转不正常，不建议使用 HTML 的链接样式。

【规则】引用如果是某篇文档，建议用书名号包裹。

【举例】

```markdown
- 网站链接
A-Tune 的安装步骤请参考[《安装与部署》](https://www.atune.com)。

- 相对路径
[文档开发流水线门禁](./ci_rules.md)
```

### 锚点

【规则】若要引用文档中的标题、图片或表格，可插入锚点，从而实现向文档特定位置的快速跳转。

【规则】锚点格式：将标题中大写字母转换为小写，空格替换为中划线`-`，并去除特殊符号。

【举例】

```markdown
# 安装前准备 A*

这是A-Tune的安装前准备。

...

参考[安装前准备](#安装前准备-a)章节。
```

```markdown
**图1** CI 检查结果
![CI 检查结果](./figures/ci检查结果.jpg)<a id="fig1"></a>

...

参考图1[CI 检查结果](#fig1)。
```

### 表格

【规则】markdown 文档中请使用以下方式创建表格，不建议使用 HTML 的表格样式。

【举例】

| 表头1         | 表头2      |
| ----------   | --------   |
| 单元格1       | 单元格2     |
| 单元格4       | 单元格4     |

【使用方法】
设置表格的对齐方式：

* -: 设置内容和标题栏居右对齐。
* :- 设置内容和标题栏居左对齐。
* :-: 设置内容和标题栏居中对齐。

【规则】当表格内一列全部是术语、短语时，统一不加标点符号。

【规则】当表格内一列全部是句子时，统一加句号。

【规则】特殊情况下如果不能避免出现短语和句子混合的情况，统一加句号。

### 标点符号

【规则】单位与数字之间不建议加空格，比如 50m，10kg，64Kbit/s。

【规则】对于有序/无序列表，如果是长句子，建议统一以句号结尾，如果是短语，结尾可不用标点。**重点是前后一致，要么都加，要么都不加**。

【规则】中文文档使用全角标点。

【规则】数字使用半角字符。

【规则】感叹号使用场景为可能引发严重后果的操作或设备安全、人身安全的警告。其他场景不允许使用感叹号。

【规则】文内引用其他文档时添加书名号，同时建议增加引用文档的跳转链接。例如：安装 openEuler 系统，安装方法参考《[openEuler 22.03 LTS SP2 安装指南](https://docs.openeuler.org/zh/docs/22.03_LTS_SP2/docs/Installation/installation.html)》。

### 操作步骤

【规则】当指引用户点击界面上的具体元素（如按钮、选项卡、菜单项）时，应将该元素的名称加粗。例如：登录到[openEuler官网](https://www.openeuler.openatom.cn/zh/)，点击顶部导航栏中的**文档**。

## 内容规范

【规则】内容中禁止包含“功能待完善”等字样。“待完善”是项目管理和开发跟踪的内部状态，不应直接传达给最终用户。

## 语言风格规范

【规则】提交内容必须是与 openEuler 特性相关内容。

【规则】内容不能包含敏感信息、有强烈的种族歧视或性别歧视的内容。

【规则】提交的内容必须是原创内容，不得侵犯他人知识产权。

【规则】提交的内容必须客观、真实，不允许使用夸大宣传等词汇。

**文档贡献中不受欢迎的行为**

短时间内通过自动化工具，提交大量的PR，提交大量的处理诸如拼写错误，语法错误，日期错误，语句不通顺等“无害的错误”的修正。
具体内容以及处理措施详见 [openEuler社区开发行为规范 V1.0](https://atomgit.com/openeuler/community/blob/master/zh/technical-committee/governance/openEuler%E7%A4%BE%E5%8C%BA%E5%BC%80%E5%8F%91%E8%A1%8C%E4%B8%BA%E8%A7%84%E8%8C%83.md)。

---

---
url: /zh/docs/common/faq/caselibrary/mountsysroot.md
---
# 出现mount/sysroot失败的问题

## 问题背景

### 硬软件信息

硬件环境：TaiShan200 (Model1280V2)

内核版本：5.10.0-153.1.0.81.oe2203sp2.aarch64

### 版本信息

openEuler 22.03-LTS-SP2

## 现象描述

重复多次上下电测试，mount时偶现`ext4/vfat`驱动没加载到内核，导致mount文件系统失败。

![image](./figures/fd22e53b-5775-40ac-b194-6932ad81958e.png)

## 原因分析

这个问题出现的概率低，现象表现又在启动过程，这个给定位分析带来了一些额外的工作量，分析的过程也是按抽丝剥茧的方式层层递进。

### 1. mount失败问题

![image](./figures/8df63e36-e202-4d8b-b477-4bd0c1d9d826.png)

Mount失败，打开调试后`udev.log_priority=debug rd.debug=1`，获取详细信息：

![image](./figures/cda96442-d6ba-4ab0-8d56-87b248b0ef41.png)

可以看到问题点是执行mount系统调用时失败，进一步增加内核打印。

![image](./figures/98bb38d3-bbbf-475c-b78d-bf4b2bf08528.png)

从上面截图里的信息可以看出，当前内核在挂载vfat文件系统时：

1. `get_fs_type1`，由于对应ko还未加载，所以调用`request_module1`去加载模块。
2. 根据图片中`request_module1`返回值 "大于" 0，可以看到，应该已经走到`call_modprobe->call_usermodehelper_exec`，用户态执行了请求，但是返回了256。

**分析结论**：用户态在执行modprobe时，返回错误码256，下面分析为什么会load module失败。

### 2. load module失败问题

用户态的日志由于输出到命令行，在紧急模式阶段无法输出，选择在内核态增加打印：

![image](./figures/f1269fd0-7fb4-462c-a21c-043778edace0.png)
![image](./figures/3c883235-950a-45d1-b10d-3ee4cbda5cd2.png)

根据日志能确认，是modprobe加载驱动时，在内核里失败了。

**分析结论**：`module_sig_check/setup_load_info`这两个函数其中一个失败了，返回-129。

### 3. 验签失败问题

返回-129，调用链为：

```txt
load_module->module_sig_check->mod_verify_sig->verify_pkcs7_signature->verify_pkcs7_message_sig->pkcs7_validate_trust->pkcs7_validate_trust_one->verify_signature->public_key_verify_signature ->crypto_akcipher_verify->pkcs1pad_verify->pkcs1pad_verify_complete
```

![image](./figures/7be4d825-680e-4dc0-989c-ae01843f90be.png)

![image](./figures/50c5f99c-f86d-4e23-bd4e-36b536837312.png)

从log中可以看出验签通过和验签失败时`out_buf`的数值都一致。而`req_ctx->out_buf + ctx->key_size`前后数值不一样。因此优先分析异常数值的来源。

从代码可以看出异常的数据由`sg_pcopy_to_buffer`进行获取发现异常数据为 签名数据中的digest部分，这数值是由签名数据的原始数据经过哈希算法生成digest，然后使用私钥对digest进行加密。

在验签的时候使用公钥对数据进行解密得到digest，并且对收到的原始数据重新计算digest，然后将计算得到的digest与解析得到的digest进行比较，以验证签名的有效性。异常的位置便发生在这个比较中，说明签名数据可能不完整或者被篡改。

通过添加log发现digest由`crypto_shash_digest(desc, pkcs7->data, pkcs7->data_len, sig->digest)`；验签通过和验签失败时的pkcs7->data进行打印，发现数据一致

![image](./figures/64f55b4f-466f-4e56-86bc-c8714c3a1e22.png)

将怀疑点放到了使用的算法驱动，于是将算法驱动的名字进行打印：

![image](./figures/c2b268e3-78f1-4d7c-b459-50b90e73c5b2.png)
结果发现所有返回-129时候使用的都是sha256-ce这个加密驱动。

该加密驱动有使用到armv8的加密扩展，使用了cpu的特殊指令进行加解密。使用同样的内核进行复现，未能复现出问题。怀疑是硬件问题。

在异常环境上，使用以下脚本反复加载卸载ko可以复现出-129的问题。

![image](./figures/ff7fd456-3f56-4e1c-9bd6-ace9565c271f.png)

**分析结论**：怀疑是CPU问题。

### 4. CPU 问题

在正常环境中则无法复现。

经过硬件排查，得到以下结论：

1. 是早期工程的芯片。

2. 这些指令里有测试异常的：

   ```txt
   AdvsimdLoadStore
   LCRTSveVectorMove
   ```

3. 使用测试套测试cpu130和131有问题。

根据硬件的结论，将反复加载ko的脚本绑定在130和131核心上运行，问题必现，绑定在129问题则不出现。

![image](./figures/22e3a767-9e64-4561-be56-76b92e3c17ad.png)

**至此，可以锁定为CPU故障。**

## 解决方案

**方案一：更换CPU**
由于cpu核130，131上的指令处理异常，硬件上替换正常CPU后可以解决该问题。

**方案二：隔离CPU核130，131**
软件上通过CPU核隔离设置，可以隔离出异常的CPU核，作为规避方案。

---

---
url: /zh/docs/common/faq/caselibrary/sssnic.md
---
# 制作内核热补丁，插入时dmesg提示缺少sssnic模块

## 问题背景

在使用openEuler-LTS-SP3版本制作内核热补丁时，补丁并没有修改sssnic驱动模块，但是制作出来的热补丁激活失败，使用dmesg命令后提示：livepatch: module 'sssdk' not  loaded。（sssnic是网卡驱动模块，源码位于内核drivers/net/ethernet/sssnic目录下）

### 版本信息

内核版本：5.10.0-182.0.0.95.oe2203sp3.aarch64

kpatch版本：kpatch-0.9.5-7.oe2203sp3.aarch64

### 现象描述

使用命令`./make_hotpatch -d .new -i procversion`制作内核热补丁，能够正常制作，但是制作完成后发现sssnic模块的函数有修改，导致热补丁依赖sssnic模块，实际补丁并未对该驱动模块源码进行修改；热补丁依赖sssnic模块会导致补丁激活失败，会报错未加载sssnic模块（sssnic模块为sss网卡驱动模块，默认不加载），制作出来的热补丁无法正常使用。

```shell
[166439.721426] klp_procversion: tainting kernel with TAINT_LIVEPATCH
[166439.760137] livepatch: module 'sssdk' not loaded
```

## 原因分析

1.根据日志可以发现，在提取新的修改后的elf段时，`sss_tool_nic_func.c`和`sss_tool_sdk.c`两个文件的确被识别出有差异，并被当做差异段。

2.进一步排查后发现，增量编译时sssnic模块不论是否被修改，sssnic模块都会重新被编译，且增量编译前后的二进制.o文件会被kpatch识别存在变化，被制作成热补丁。

## 解决方案

方案一：

若制作的热补丁并非sssnic模块，可以在识别差异段时尝试屏蔽该模块，修改`/usr/libexec/kpatch/kpatch-cc`文件，在忽略文件列表中增加sssnic源码路径，再次制作热补丁就可以看到不会再依赖sssnic模块，热补丁正常使用。

```shell
diff --git a/kpatch-build/kpatch-cc b/kpatch-build/kpatch-cc
index 80d310c...688d92b 100755
--- a/kpatch-build/kpatch-cc
+++ b/kpatch-build/kpatch-cc
@@ -49,7 +49,8 @@ if [[ "$TOOLCHAINCMD" =~ ^(.*-)?gcc$ ||
                arch/powerpc/kernel/prom_init.o|\
                lib/*|\
                .*.o|\
-               */.lib_exports.o)
+               */.lib_exports.o|\
+               drivers/net/ethernet/3snic/sssnic/*)
                    break
                    ;;
                *.o)
```

方案二：

有时未作出修改的函数被识别为变化函数可能是编译器优化后导致汇编结果发生了变化，这种场景下可以使用`KPATCH_IGNORE_FUNCTION`宏忽略该函数，在制作热补丁时就不会将该函数做到热补丁中。

查看热补丁制作时屏幕输出的日志可以发现sssnic模块对应有两个函数发生了变化：

```txt
Testing patch file(s)
Reading special section data
Building original source
Building patched source
Extracting new and modified ELF sections
sss_tool_nic_func.o: changed function: sss_tool_ioctl
sss_tool_sdk.o: changed function: sss_tool_get_hw_drv_version
version.o: changed function: version_proc_show
```

在sssnic模块显示变化的函数后，分别在对应函数发生修改的文件里增加以下两句（注意宏`KPATCH_IGNORE_FUNCTION`在函数声明后再使用，否则会报错找不到符号）：

```c
#include "/usr/share/kpatch/patch/kpatch-macros.h"
KPATCH_IGNORE_FUNCTION(sss_tool_ioctl);
```

```c
#include "/usr/share/kpatch/patch/kpatch-macros.h"
KPATCH_IGNORE_FUNCTION(sss_tool_get_hw_drv_version);
```

重新执行命令`./make_hotpatch -d .new -i procversion`制作热补丁就不会再报关于sssnic模块未加载的错误。

---

---
url: /zh/docs/common/faq/server/trusted_computing_faqs.md
---
# 可信计算常见问题与解决方法

## **问题1:开启IMA评估enforce模式并配置默认策略后，系统启动失败**

### 原因分析

IMA默认策略可能包含对应用程序执行、内核模块加载等关键文件访问流程的校验，如果关键文件访问失败，可能导致系统无法启动。通常原因有：

1. IMA校验证书未导入内核，导致摘要列表无法被正确校验；
2. 摘要列表文件未正确签名，导致摘要列表校验失败；
3. 摘要列表文件未导入initrd中，导致启动过程无法导入摘要列表；
4. 摘要列表文件和应用程序不匹配，导致应用程序匹配已导入的摘要列表失败。

### 解决方法

用户需要通过log模式进入系统进行问题定位和修复。重启系统，进入grub界面修改启动参数，采用log模式启动：

```sh
ima_appraise=log
```

系统启动后，可参考如下流程进行问题排查：

**步骤1：** 检查keyring中的IMA证书：

```sh
keyctl show %:.builtin_trusted_keys
```

对于openEuler LTS版本，至少应存在以下几本内核证书（其他未列出版本可根据发布时间前推参考）：

如果用户导入了其他内核根证书，也同样需要通过`keyctl`命令查询确认证书是否被成功导入。openEuler默认不使用IMA密钥环，如果用户存在使用的情况，则需要通过如下命令查询IMA密钥环中是否存在用户证书：

```sh
keyctl show %:.ima
```

如果排查结果为证书未正确导入，则用户需要根据*用户证书导入场景*章节进行流程排查。

**步骤2：** 检查摘要列表携带签名信息：

用户可通过如下命令查询当前系统中的摘要列表文件：

```sh
ls /etc/ima/digest_lists | grep '_list-compact-'
```

对于每个摘要列表文件，需要检查存在**以下三种之一**的签名信息：

(1) 检查该摘要列表文件存在对应的**RPM摘要列表文件**，且**RPM摘要列表文件**的ima扩展属性中包含签名值。以bash软件包的摘要列表为例，摘要列表文件路径为：

```sh
/etc/ima/digest_lists/0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64
```

RPM摘要列表路径为：

```sh
/etc/ima/digest_lists/0-metadata_list-rpm-bash-5.1.8-6.oe2203sp1.x86_64
```

检查RPM摘要列表签名，即文件的`security.ima`扩展属性不为空：

```sh
getfattr -n security.ima /etc/ima/digest_lists/0-metadata_list-rpm-bash-5.1.8-6.oe2203sp1.x86_64
```

(2) 检查摘要列表文件的`security.ima`扩展属性不为空：

```sh
getfattr -n security.ima /etc/ima/digest_lists/0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64
```

(3) 检查摘要列表文件的末尾包含了签名信息，可通过检查文件内容末尾是否包含`~Module signature appended~`魔鬼字符串进行判断（仅openEuler 24.03 LTS及之后版本支持的签名方式）：

```sh
tail -c 28 /etc/ima/digest_lists/0-metadata_list-compact-kernel-6.6.0-28.0.0.34.oe2403.x86_64
```

如果排查结果为摘要列表未包含签名信息，则用户需要根据*摘要列表签名机制说明*章节进行流程排查。

**步骤3：** 检查摘要列表的签名信息正确：

在确保摘要列表已携带签名信息的情况下，用户还需要确保摘要列表采用正确的私钥签名，即签名私钥和内核中的证书匹配。除用户自行进行私钥检查外，还可通过dmesg日志或audit日志（默认路径为`/var/log/audit/audit.log`）判断是否有签名校验失败的情况发生。典型的日志输出如下：

```sh
type=INTEGRITY_DATA msg=audit(1722578008.756:154): pid=3358 uid=0 auid=0 ses=1 subj=unconfined_u:unconfined_r:haikang_t:s0-s0:c0.c1023 op=appraise_data cause=invalid-signature comm="bash" name="/root/0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64" dev="dm-0" ino=785161 res=0 errno=0UID="root" AUID="root"
```

如果检查结果为签名信息错误，则用户需要根据*摘要列表签名机制说明*章节进行流程排查。

**步骤4：** 检查initrd中是否导入摘要列表文件：

用户需要通过如下命令查询当前initrd中是否存在摘要列表文件：

```sh
lsinitrd | grep 'etc/ima/digest_lists'
```

如果未查询到摘要列表文件，则用户需要重新制作initrd，并再次检查摘要列表导入成功：

```sh
dracut -f -e xattr
```

**步骤5：** 检查IMA摘要列表和应用程序是否匹配：

参考[问题2章节](#问题2开启IMA评估enforce模式后部分文件执行失败)。

## **问题2：开启IMA评估enforce模式后，部分文件执行失败**

### 原因分析

开启IMA评估enforce模式后，对于配置IMA策略的文件访问，如果文件的内容或扩展属性设置有误（和导入的摘要列表不匹配），则可能会导致文件访问被拒绝。通常原因有：

(1) 摘要列表未成功导入（可参考FAQ1）；

(2) 文件内容或属性被篡改。

### 解决方法

对于出现文件执行失败的场景，首先需要确定摘要列表文件已经成功导入内核，用户可以检查摘要列表数量判断导入情况：

```sh
cat /sys/kernel/security/ima/digests_count
```

然后用户可通过audit日志（默认路径为`/var/log/audit/audit.log`）判断具体哪个文件校验失败以及原因。典型的日志输出如下：

```sh
type=INTEGRITY_DATA msg=audit(1722811960.997:2967): pid=7613 uid=0 auid=0 ses=1 subj=unconfined_u:unconfined_r:haikang_t:s0-s0:c0.c1023 op=appraise_data cause=IMA-signature-required comm="bash" name="/root/test" dev="dm-0" ino=814424 res=0 errno=0UID="root" AUID="root"
```

在确定校验失败的文件后，可对比TLV摘要列表确定文件被篡改的原因。对于未开启扩展属性校验的场景，仅对比文件SHA256哈希值和TLV摘要列表中的`IMA digest`项即可，对于开启扩展属性校验的场景，则还需要对比文件当前的属性和TLV摘要列表中显示扩展属性的区别。

在确定问题原因后，可通过还原文件的内容及属性，或对当前文件再次生成摘要列表，签名并导入内核的方式解决问题。

## **问题3：开启IMA评估模式后，跨openEuler 22.03 LTS SP版本安装软件包时出现报错信息**

### 原因分析

开启IMA评估模式后，当安装不同版本的openEuler 22.03 LTS的软件包时，会自动触发IMA摘要列表的导入。其中包含对摘要列表的签名验证流程，即使用内核中的证书验证摘要列表的签名。由于openEuler在演进过程中，签名证书发生变化，因此部分跨版本安装场景存在后向兼容问题（无前向兼容问题，即新版本的内核可正常校验旧版本的IMA摘要列表文件）。

### 解决方法

建议用户确认当前内核中包含以下几本签名证书：

```sh
# keyctl show %:.builtin_trusted_keys
Keyring
 566488577 ---lswrv      0     0  keyring: .builtin_trusted_keys
 383580336 ---lswrv      0     0   \_ asymmetric: openeuler <openeuler@compass-ci.com> b675600b
 453794670 ---lswrv      0     0   \_ asymmetric: private OBS b25e7f66
 938520011 ---lswrv      0     0   \_ asymmetric: openeuler <openeuler@compass-ci.com> fb37bc6f
```

如缺少证书，建议将内核升级至最新版本。

```sh
yum update kernel
```

openEuler 24.03 LTS及之后版本已具备IMA专用证书，且支持证书链校验，证书生命周期可覆盖整个LTS版本。

## **问题4:开启IMA摘要列表评估模式后，IMA摘要列表文件签名正确，但是导入失败**

### 原因分析

IMA摘要列表导入存在检查机制，如果某次导入过程中，摘要列表的签名校验失败，则会关闭摘要列表导入功能，从而导致后续即使正确签名的摘要列表文件也无法被导入。用户可检查dmesg日志中是否存在如下打印确认是否为该原因导致：

```sh
# dmesg
ima: 0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64 not appraised, disabling digest lists lookup for appraisal
```

如上述日志，则说明在开启IMA摘要列表评估模式的情况下，已经导入了一个签名错误的摘要列表文件，从而导致功能关闭。

### 解决方法

用户需要重启系统，并修复错误的摘要列表签名信息。

## **问题5：openEuler 24.03 LTS及之后版本导入用户自定义的IMA证书失败**

Linux 6.6内核新增了对导入证书的字段校验限制，对于导入IMA密钥环的证书，需要满足如下约束（遵循X.509标准格式）：

* 为数字签名证书，即设置`keyUsage=digitalSignature`字段；
* 非CA证书，即不可设置`basicConstraints=CA:TRUE`字段；
* 非中间证书，即不可设置`keyUsage=keyCertSign`字段。

## **问题6:开启IMA评估模式后kdump服务启动失败**

开启IMA评估enforce模式后，如果IMA策略中配置了如下KEXEC\_KERNEL\_CHECK规则，可能导致kdump服务启动失败。

```shell
appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig
```

原因是在该场景下，所有通过KEXEC加载的文件都需要经过完整性校验，因此内核限制kdump加载内核映像文件时必须使用kexec\_file\_load系统调用。可通过修改/etc/sysconfig/kdump配置文件的KDUMP\_FILE\_LOAD开启kexec\_file\_load系统调用。

```shell
KDUMP_FILE_LOAD="on"
```

同时，kexec\_file\_load系统调用自身也会执行文件的签名校验，因此要求被加载的内核映像文件必须包含正确的安全启动签名，而且当前内核中必须包含对应的验签证书。

## **问题7:RAS安装后无法启动**

### 原因分析

因为在当前RAS的设计逻辑中，程序启动后需要从当前目录查找一份名为 `ecdsakey.pub` 的文件进行读取并作为之后访问该程序的身份验证码，若当前目录没有该文件，则RAS启动会报错。

### 解决方法

解决方法一：运行 `ras -T` 生成测试用token后会生成 `ecdsakey.pub` 。

解决方法二：自行部署oauth2认证服务后，将对应JWT token生成方对应的验证公钥保存为 `ecdsakey.pub` 。

## **问题8:RAS启动后，通过restapi无法访问**

因为RAS默认以https模式启动，开发者需要向RAS提供合法的证书才能正常访问，而http模式下启动的RAS则不需要提供证书。

---

---
url: /zh/docs/common/contribute/templates/feature_user_guide/security.md
---
# 安全管理

介绍使用特性时会遇到的安全管理方法，如安全加固等。

---

---
url: /zh/docs/common/contribute/templates/feature_user_guide/install.md
---
# 安装XXX

## 环境要求

说明部署/安装软件的硬件、软件要求。

### 硬件要求

### 软件要求

## （可选）获取软件

说明如何获取到软件，提供相应的下载包或跳转链接。

## 部署/安装XXX（软件或特性名称）

提供部署/安装软件的明确步骤。

## 配置/调测XXX（软件或特性名称）

提供配置与调测软件的详细指导。

---

---
url: /zh/docs/common/faq/caselibrary/efivars.md
---
# 安装出现efivars报错

## 场景1：无法增加efi boot标签

### 问题背景

安装界面出现告警，提示无法增加efi boot标签告警，进行忽略正常安装。
![image](./figures/安装_无法增加bootloader.png)

### 现象描述

1. CTRL+ALT+F2切换后台，查看/tmp下的日志信息。
2. 在storage.log日志中，错误提示无可用空间。
   ![image](./figures/安装_无bootloader空间.png)
3. 执行ls /sys/firmware/efi/efivars/命令，看到当前存在过多的efi boot。
   ![image](./figures/安装_ls_bootloader.png)

### 原因分析

bios缓存空间被占满，无可用空间导致无法增加新的efi boot标签。

### 解决方案

需要清理bios缓存数据。

## 场景2：mount 错误 32

### 问题背景

安装界面出现错误，界面报mount failed：32。
![image](./figures/安装_bootloader出错.png)

### 现象描述

在安装时出现，挂载目录/sys/firmware/efi/efivars时报错。

1. CTRL+ALT+F2切换后台，查看/tmp下的日志信息。
2. 在storage.log日志中，错误提示无可用空间。
   ![image](./figures/安装_bootloader_mount32.png)

报错里提示efivarfs里可能有bad superblock，需要BIOS进一步排查。

### 原因分析

bios相关的硬件异常，flash上bios的变量区数据异常，导致bios变量相关服务异常，os下mount映射uefi变量服务时报错，导致安装os失败，需要重新刷新bios固件。

### 解决方案

重新刷新bios固件。

---

---
url: /zh/docs/common/faq/server/applicationdev_faqs.md
---
# 应用开发常见问题与解决方法

## **问题1:部分依赖java-devel的应用程序自编译失败**

### 问题描述

部分依赖java-devel的应用程序会出现使用rpmbuild命令自编译失败的问题。

### 原因分析

为了提供更新的openjdk特性和对广大java应用程序的兼容，openEuler同时提供了openjdk-1.8.0、openjdk-11等多个版本的openjdk。部分应用程序在编译时需要依赖java-devel包，安装java-devel包时系统会默认安装更高版本的java-11-openjdk，从而导致这些应用的编译失败。

### 解决方法

用户需手动使用如下命令安装java-1.8.0-openjdk后再使用rpmbuild命令进行自编译。

```shell
# yum install java-1.8.0-openjdk
```

---

---
url: /zh/docs/common/contribute/contribution_process.md
---
# 快速入门

## 概述

openEuler 文档采用 Markdown 格式编写，通过 Git 进行版本控制，并托管在 AtomGit 平台。文档修改通过 Pull Request（PR）工作流进行审核与合并。openEuler 文档分仓存储，《发行说明》、《安装指南》、《升级指南》等基础特性文档存放在 openEuler/docs 仓，由 DOC SIG 负责维护。《A-Tune 用户指南》等增量特性文档存放在各特性代码仓，由对应 SIG 组维护。

文档中心将手册[按业务场景和工具模块分类](./directory_structure_introductory.md#文档存放地址)，常见的文档开发场景如下：

* 新增场景：需联系 DOC SIG maintainer 修改。
* 新增手册：新增功能特性时，需在对应场景下新增特性手册，除了需要在特性代码仓维护文档内容外，还需配置 openeuler/docs 仓对应场景的 `_toc.yaml` 文件。
* 修改手册：包括低错问题修复、章节增删等内容调整，直接在对应代码仓中更新文档即可。

## 快速开始

下面以 openEuler 24.03 LTS SP2 版本《oeAware用户指南》的修改为例，介绍文档开发流程。

### 准备仓库

需准备两个仓库：

* openeuler/docs：文档总仓库，通过引用机制，集成所有特性文档至文档中心。
* openeuler/oeAware-manager：oeAware 特性源码仓，存储《oeAware用户指南》源文档。

1. 准备 openeuler/docs 仓库

   (1) Fork openeuler/docs 仓库。

   访问 [Repository 首页](https://atomgit.com/openeuler/docs)。点击右上角的**Fork**按钮，按照指引，创建个人的云上 fork 仓库。
   ![image](figures/forkdocs.jpg)

   (2) 克隆 openeuler/docs 仓库。

   克隆 fork 仓库到本地，并关联本地与远程仓库。

   ```bash
   git clone https://gitee.com/wu-donger/docs.git

   cd docs
   git remote add upstream https://atomgit.com/openeuler/docs.git
   git fetch upstream
   ```

   (3) 切换分支。

   依据所需修改文档的版本，切换到对应的分支。通常情况下，分支命名规则为`stable-版本号`。此处以24.03 LTS SP2版本为例。

   ```bash
   git checkout -b work2403sp2 upstream/stable-24.03_LTS_SP2
   ```

2. 准备 openeuler/oeAware-manager 仓库

   访问 [Repository 首页](https://atomgit.com/openeuler/oeAware-manager)。fork 仓库并克隆到本地，切换目标分支。

   ```bash
   git clone https://gitee.com/wu-donger/oeAware-manager.git

   cd oeAware-manager
   git remote add upstream https://atomgit.com/openeuler/oeAware-manager.git
   git fetch upstream

   git checkout -b workmaster upstream/master
   ```

### 文档变更

#### 新增手册

1. 明确目标仓库

   首次添加特性指南时，需明确：

   * 源文档存放仓库：特性源码仓库。
   * 版本策略：通过分支或目录管理版本（推荐分支方式）。

   示例：《oeAware用户指南》的源文档存放在 oeAware 特性的源码仓库 openeuler/oeAware-manager，并通过目录区分 openEuler 版本。

   相关约束和要求详见[SIG 组文档目录结构规范](./directory_structure_introductory.md#sig-组文档目录结构规范)。

   此外，需为目标仓库配置文档 CI 和文档检视人员，可联系 DOC SIG maintainer 操作。

2. 创建文件夹

   openEuler 文档遵循特定的目录结构。新增手册时，需创建一个文件夹来存放实际内容文件和目录结构文件。《oeAware用户指南》的存放路径如下：

   ```text
   ├─openeuler/oeAware-manager 仓
   ├─docs
   |  ├─en
   |  └─zh
   |    └─2403_lts_sp2                                             # 版本：openEuler 24.03 LTS SP2
   |           ├─oeaware_user_guide.md                             # 文档内容文件
   |           └─_toc.yaml                                         # 文档目录结构文件 
   ```

   > \[!NOTE]说明
   > 在仓库根目录下创建 /docs 目录，并在其中创建 zh/ 和 en/ 目录存放需发布至官网的中英文文档。

3. 编辑目录结构文件`_toc.yaml`

   在步骤 1 创建的文件夹中，新建一个`_toc.yaml`文件，以维护手册内各章节的展示逻辑。

   ```yaml
   label: oeAware用户指南                                     # 手册名：《oeAware用户指南》
   isManual: true                                            # isManual：标识此文件是手册的目录结构文件
   description: 动态感知系统行为后，智能使能系统的调优特性       # 手册简介
   sections:
     - label: 使用oeAware                                    # 章节名：使用oeAware 
       href: ./oeaware_user_guide.md                         # 文档源文件地址
   ```

4. 关联手册至对应场景

   在 openeuler/docs 仓库服务器场景的 [\_toc.yaml](https://atomgit.com/openeuler/docs/tree/stable-24.03_LTS_SP2/docs/zh/server/_toc.yaml) 文件中，添加对《oeAware用户指南》的引用。

   ```yaml
   label: 服务器                                                                                                   # 场景名：服务器
   sections: 
     - label: 性能调优                                                                                             # 一级目录：性能调优 
       sections:
         - label: 概述                                                                                             # 二级目录：概述
           sections:
             - href: ./performance/overall/system_resource/_toc.yaml
         - label: 调优框架                                                                                          # 二级目录：调优框架 
           sections:
             - href:                                                                                               // [!code ++]
                 upstream: https://gitee.com/openeuler/oeAware-manager/blob/master/docs/zh/2403_lts_sp2/_toc.yaml  # 手册：《oeAware用户指南》  // [!code ++]
                 path: ./performance/tuning_framework/oeaware                                                      # 手册的url访问路径  // [!code ++]
   ```

#### 修改文档

以在《oeAware用户指南》中增加“认识oeAware”章节为例，首先在[《oeAware用户指南》的存放目录](https://atomgit.com/openeuler/oeAware-manager/blob/master/docs/zh/2403_lts_sp2/)下，新增文档内容文件`getting_to_know_oeaware.md`，并在《oeAware用户指南》的 [\_toc.yaml](https://atomgit.com/openeuler/oeAware-manager/blob/master/docs/zh/2403_lts_sp2/_toc.yaml) 下，增加对`getting_to_know_oeaware.md`的引用：

```yaml
label: oeAware用户指南                                     # 手册名：《oeAware用户指南》
isManual: true                                            # isManual：标识此文件是手册的目录结构文件
description: 动态感知系统行为后，智能使能系统的调优特性       # 手册简介
sections:
  - label: 认识oeAware                                    # 章节名：认识oeAware   // [!code ++]
    href: ./getting_to_know_oeaware.md                    # 新增的文档源文件地址  // [!code ++]
  - label: 使用oeAware                                    # 章节名：使用oeAware 
    href: ./oeaware_user_guide.md                         # 文档源文件地址
```

### 提交变更

1. 提交 openeuler/docs 仓库的文档变更。

   （1）提交变更并推送到远程仓库。

   ```bash
   git add .

   git commit -m "提交原因"

   git push origin work2403sp2
   ```

   （2）创建PR。

   在个人文档仓库的 Pull Requests 页面 `https://gitee.com/{your_org}/docs/pulls`，点击**新建Pull Request**创建PR。源分支选择 `{your_org}/docs/work2403sp2`，目的分支选择 `openeuler/docs/stable-24.03_LTS_SP2`。填写 PR 标题并简要说明修改内容，点击**创建Pull Request**。

   （3）合入PR。

   合入条件：文档流水线门禁通过，CLA 已签署，DOC SIG maintainer 检视通过。

   ![image](figures/approve.jpg)

2. 提交 openeuler/oeaware-manager 仓库的文档变更。

   （1）提交变更并推送到远程仓库，创建 PR。

   （2）合入 PR。

   合入条件：SIG 组代码仓涉及文档变更的 PR，除代码仓原有的合入条件外，还需要通过文档流水线门禁和 DOC SIG maintainer 审核。

   ![image](figures/sigdoc_approve.png)

   > \[!NOTE]说明
   > DOC SIG maintainer 审核通过前会审视此 PR 是否需要转测试验收，检视测试流程详见：[文档检视测试流程](./directory_structure_introductory.md#文档检视测试流程)。

3. 英文翻译

   合入中文文档 PR 后，系统将自动生成翻译 issue 并排入处理队列，翻译人员将按顺序完成英文翻译并提交 PR，需各 SIG 组 Maintainer 审核并合入。如需加急翻译，请将对应中文文档 PR 链接同步至 DOC SIG 协调优先处理。

## 更多

### 了解更多细节和进阶内容

* 了解 [openEuler 文档组织架构](./directory_structure_introductory.md)
* 了解 [markdown 写作规范](./documentation_writing_specifications.md)
* 了解 [开源社区贡献流程](https://atomgit.com/openeuler/docs/blob/stable-common/docs/zh/contribute/openeuler_contribution_guide.md)

---

---
url: /zh/docs/common/faq/caselibrary/rebranding.md
---
# 换标常见问题

## 场景1：安装失败，提示无法写入boot loader配置

### 问题背景

使用openEuler系统软件包，进行对应的换标替换，但在进行安装中出现安装失败。

### 现象描述

在进行镜像安装时，安装失败，提示无法写入boot loader配置信息。

![image](./figures/换标_config.png)

### 原因分析

CTRL+ALT+F2切换后台查看/tmp/anaconda.log日志。

![image](./figures/换标_config_日志.png)

从日志中看到并未找到对应的bootloader配置文件。
主要原因是安装中会去检查/etc/os-release文件对应的"ID"和"VARIANT\_ID"。

![image](./figures/换标_config_检查.png)

### 解决方案

修改anaconda中的openEuler.conf的"Profile Detection"下的"os\_id"和"variant\_id"与换标系统中的os-release文件保持一致。
![image](./figures/换标_config_修改.png)

## 场景2：安装成功，无法正常进入系统

### 问题背景

换标后能进行正常的镜像安装，但安装后无法正常启动。

### 现象描述

启动时提示无法正常找到efi启动文件。
![image](./figures/换标_未完全.png)

### 原因分析

存在部分软件包未完全换标，grub软件包换标存在遗漏情况。

### 解决方案

1.查看启动项是否存在不正确的情况。
![image](./figures/换标_未完全_配置.png)
2\. 参考[系统文件恢复问题](./sysfile.md)文档，可以进入系统后调整对应的启动配置文件。
3\. 完全解决换标问题，需要修改构建工程中的grub.cfg。

## 场景3：换标后构建kernel，安装依赖tk组件失败

### 问题背景

换标使用obs构建openEuler的同源包，出现tk安装出现错误。

![image](./figures/换标_构建kernel_报错.png)

### 现象描述

对tk软件包进行安装检查，单独安装同样存在报错现象。错误为组件的%post阶段。
![image](./figures/换标_tk_安装失败.png)

### 原因分析

spec中的%ldconfig\_post、%ldconfig\_postun未识别，导致安装软件中带入误认为脚本，并报错。
![image](./figures/换标_tk_报错.png)

### 解决方案

需要修改换标工程中的rpm-config宏定义。

## 场景4：安装中报POSTTRANS scriptlet错误

### 问题背景

换标或者自构建软件镜像，在安装时出现 POSTTRANS scriptlet错误。
![image](./figures/换标_script_报错.png)

### 现象描述

1. CTRL+ALT+F2切换后台，进入/mnt/sysroot目录，切根。
2. 执行dnf history info 1，可以看到post哪边执行出错。
   ![image](./figures/换标_script_post.png)

### 原因分析

软件安装后执行%POSTTRANS出错,未能找到相关的gz文件。

### 解决方案

分析软件包spec文件的%post脚本，修改后本地安装无问题后可以再次进行安装。

---

---
url: /zh/docs/common/contribute/ci_rules.md
---
# 文档开发流水线门禁

为提升文档质量，openEuler docs 仓引入了自动化动检视工具，对文档中低错问题进行排查。

开发者提交PR后，会自动触发门禁进行检查。当返回如下结果时表示已经通过了工具检查：

![](./figures/ci门禁检查结果.png)

通过门禁检查是PR合入的必要条件之一，检查项提示错误可在`Build Details`查看错误详细信息。

下面将对各个检查项进行介绍：

## Markdown Lint

Markdown Lint 对markdown文档的格式进行检视。markdownlint规则介绍及本仓规则设置，请参考[**检查规则**](https://gitee.com/openeuler/docs/blob/stable-common/docs/zh/contribute/markdownlint_rules.md)。可使用[**工具**](https://gitee.com/openeuler/docs/blob/stable-common/docs/zh/contribute/markdownlint_tools.md)对 markdownlint 进行批量修复。

## Tag Closed Check

Tag Closed Check将检查文档里HTML标签闭合问题。请注意，代码块里的HTML标签将不会扫描。

反例：

```html
  <table>
    <tr>
      <th>Header 1
      <th>Header 2</th>   <!-- 错误1：未闭合的<th> -->
    <tr>                  <!-- 错误2：未闭合的<tr> -->
      <td>Data 1<td>      <!-- 错误3：自闭合<td> -->
      <td>Data 2</td>
  </table>                <!-- 错误4：未闭合的<tr>导致结构错乱 -->
```

正例：

```html
  <table>
    <tr>
      <th>Header 1</th>
      <th>Header 2</th>
    </tr>
    <tr>
      <td>Data 1</td>
      <td>Data 2</td>
    </tr>
  </table>
```

## Link Validity Check

Link Validity Check 将检查文档中出现的所有链接是否有效。

反例：

```text
![错误官网](https://doc.openeuler.org/zh/)  <!-- 拼写错误导致 404 -->
```

正例：

```text
![正确官网](https://docs.openeuler.org/zh/)
```

## Resource Existence Check

此检查项将确认本地图片或者链接是否有效。

反例：

```text
![ci图片](./ci检查结果.jpg)  <!-- 图片相对路径错误 -->
```

正例：

```text
![ci图片](./figures/ci检查结果.jpg)
```

## Toc Check

* 新增文档为确保能在 openEuler 文档官网展示，需要在对应`_toc.yaml`文件增加所在章节位置，否则此检查项报错。新增文档场景写作流程可参考快速入门的[指导](./contribution_process.md#新增手册)，`toc.yaml`文件写作可参考[\_toc.yaml文件写作规范](./directory_structure_introductory.md#目录配置文件格式)。

* 变更文档名称时，File Exist Check会检查对应`_toc.yaml`文件是否同步修改文档名称。

* 在进行File Exist Check检查时，门禁会对文档进行全量检查，检查每个`_toc.yaml`文件中所有的文件是否存在于文档对应的路径下，若`_toc.yaml`文件中记录的文件不存在，则会报错。

## CodeSpell Check

Codespell 主要用于检查文档中的单词拼写错误，详细信息可参考<https://github.com/codespell-project/codespell>。

如果有特殊单词需要加入忽略清单，可联系[ECHO](https://gitee.com/echo10111111)和[wu-donger](https://gitee.com/wu-donger)。

---

---
url: /zh/docs/common/contribute/docs_decentralized.md
---
# 文档生产下沉至SIG组

## 重要事项

* 文档生产下沉至各 SIG 组后，各 SIG 组需重视文档质量，更新内容时请通知 doc-sig 成员检视（测试人员测试），通过后再合入。
* 文件名和文件夹名请使用英文小写字母、并用下划线连接（同一篇文档的中英文名保持一致），文档发布至官网后不可更改存放路径，如需更改需评审。

## 概述

之前所有文档都集中存于 docs 仓。改版后，基础特性文档依旧存放在 docs 仓，由 doc-sig 集中管理，增量特性文档在各特性 SIG 的代码仓维护。分散在各特性代码仓的文档，需通过在 docs 仓的配置文件（\_toc.yaml）中添加引用，实现发布至官网。

![image](figures/架构图.png)

## 约束

* 分版本写文档，openEuler 各版本均有独立配套文档。
* 文档发布后仍会持续更新，如修正低级错误、内容错漏等问题。
* 各 SIG 组负责维护特性文档，要求各 SIG 清晰掌握特性在各版本中的分布情况。
* 各 SIG 组自主规划文档，明确区分发布至官网的文档和仅在代码仓展示的文档。
* SIG 组维护文档以手册为最小单元（如服务器场景《A-Ops用户指南》），单本手册不得拆分存储，同一仓库可容纳多本手册。
* 所见即所得，在 gitee 仓库可直接查看文档内容，呈现效果与最终展示一致。

  ![image](figures/手册是最小粒度.png)

## 要求

无论各 SIG 如何组织文档，如果文档要发布至官网，需要满足如下条件：

* 在仓库根目录下创建 docs 目录，并在 docs 目录下的 zh/ 和 en/ 目录中存放需发布至官网的中英文文档，结构如下：

  ```text
  ├─docs/
  |  ├─zh/
  |  └─en/                                        
  ```

  注明：存放于 docs/zh 和 docs/en 下的文档均为发布至官网文档中心的文档，必须通过文档流水线，才能合入；若文档仅在仓库内展示、无需发布至官网，不建议存放于上述目录，可在 docs/ 下创建其他目录存放。

* 在 openeuler/docs 仓库中，找到对应场景的 \_toc.yaml 文件，并按照指定格式添加文档的索引地址：

  upstream: 填写手册的目录结构文件（\_toc.yaml）索引地址\
  path（可选，默认值是“./仓库名”）: 设置手册的url访问路径

  示例：\
  upstream: <https://atomgit.com/openeuler/openstack-docs/blob/openEuler-25.03/docs/zh/_toc.yaml>\
  path: ./openstack

* 文档须严格对应当前版本特性，禁止包含未发布的特性或功能。

* 支持文档的长期更新维护，确保 openEuler 新版本发布后，历史版本文档仍保持准确完整。

## 方案

为满足上述条件，建议的实现方案如下：

* 每个 SIG 组特性应该有自己的独立代码仓库，该仓库承载该SIG的特性文档、readme、contribute等内容。
* 在不同分支下维护对应版本的文档或创建目录维护不同版本的文档。
* 若选择创建目录维护不同版本的文档，其内容如下：

  ```text{7-9}
  ├─docs/
  |  ├─zh/
  |  |  ├─2409/     
  |  |  |   ├─_toc.yaml     
  |  |  |   ├─xxx.md
  |  |  |   └─xxx/    
  |  |  |      └─xxx.md 
  |  |  └─2503/     
  |  |      ├─_toc.yaml     
  |  |      ├─xxx.md
  |  |      └─xxx/    
  |  |         └─xxx.md  
  |  └─en/  
  |     ├─2409/ 
  |     └─2503/                                       
  ```

## 文档下沉流程

1. SIG 组确定方案，明确文档存放仓库及版本区分方式。
2. 为仓库配置文档流水线和检视人员，请联系doc-sig。
3. 将相关文档从 openeuler/docs 仓库迁移至目标仓库。
4. 待 doc-sig 成员检视通过后，由SIG组自行安排合入。

## 英文翻译

合入中文文档 PR 后，系统将自动生成翻译 issue 并排入处理队列，翻译人员会按顺序完成英文翻译并提交PR，需各 SIG 组 Maintainer 审核合入。如需加急翻译，请将对应中文文档 PR 链接同步至 doc-sig 协调优先处理。

---

---
url: /zh/docs/common/contribute/directory_structure_introductory.md
---
# 文档组织架构

## 介绍

本文介绍 openEuler 文档的生产发布流程与文档仓的组织架构，同时提供每本手册在文档仓中的具体存放位置。

![image](figures/架构图.png)

openEuler 文档的生产发布机制如上图所示。

* 文档中心将社区文档按业务场景和工具模块进行划分：
  * 业务场景：服务器、虚拟化、云原生、边缘计算、嵌入式、DevStation。
  * 创新场景与工具：超节点、社区工具、DevOps、AI、图形桌面使用、云原生工具、系统运维、安全。
* 发布机制：
  * 每个场景及工具模块均有对应目录配置文件（\_toc.yaml）。这些配置文件均存于 openEuler/docs 仓，由 DOC SIG 集中管理。
  * 各文档的责任 SIG 需将文档目录配置文件引用，添加到所属场景或工具模块的目录配置文件中，使文档可在对应模块下呈现。
* 文档生产：
  * openEuler的文档生产在 openEuler/docs 仓以及各 SIG 组的文档/源码仓中进行。
  * 基础特性文档，如发行说明、快速入门、安装、升级、管理员指南、配置与逻辑卷、配置网络、故障处理等，均存在 openEuler/docs 仓，由 DOC SIG 生产并维护。
  * 增量特性文档，如 A-Tune 用户指南、x2openEuler 特性指南、oeAware 用户指南等，责任主体为特性所属的 SIG 组，分别存放在各 SIG 组的文档/源码仓内。
  * 各 SIG 在文档/源码仓中维护的文件包括：内容文件和目录配置文件（\_toc.yaml）。其中，内容文件用来存放文档实际内容，目录配置文件用来维护文档章节呈现结构。

在仓库中根目录 /docs 下设有 /zh 和 /en 两个子目录，分别用以存放发布至官网的中文文档与英文文档，文档目录结构严格参照官网呈现的目录层级规划设置。此外，文档仓可设 /archive 目录，用来存放暂不适合推广，或尚不成熟的文档。待文档完善且满足发布需求时，再将其移至 /docs 目录，以便在官网展示。

```text
├─docs     <!-- 在文档中心发布的文档 -->
│  ├─en    <!-- 英文文档 -->
│  └─zh    <!-- 中文文档 -->
```

## 仓库目录结构说明

### openeuler/docs 仓库目录结构说明

#### 场景

文档中心有五个业务场景，服务器、虚拟化、云原生、边缘计算、嵌入式和 DevStation，分别对应 openeuler/docs 仓内 docs/zh 目录下的 server、virtualization、cloud、edge\_computing、embedded 和 devstation 子目录，工具模块对应 tools 子目录。

文档仓场景相关目录结构示例如下：

```text
├─openeuler/docs 仓
├─docs
│  ├─en
│  └─zh
│     ├─server          <!-- 场景：服务器 -->
│     ├─virtualization  <!-- 场景：虚拟化 -->
│     ├─cloud           <!-- 场景：云原生 -->
│     ├─edge_computing  <!-- 场景：边缘计算 -->
│     ├─embedded        <!-- 场景：嵌入式 -->
│     ├─devstation      <!-- 场景：DevStation -->
│     └─tools           <!-- 场景：工具 -->
```

创新场景与工具模块下的子模块包括超节点、社区工具、DevOps、AI、图形桌面使用、云原生工具、系统运维和安全，分别对应 tools 目录下的 unifiedbus、community\_tools、devops、ai、desktop、cloud、maintenance 和 security 子目录。

文档仓工具相关目录结构示例如下：

```text{10-17}
├─docs
│  ├─en
│  └─zh
│     ├─server              <!-- 场景：服务器 -->
│     ├─virtualization      <!-- 场景：虚拟化 -->
│     ├─cloud               <!-- 场景：云原生 -->
│     ├─edge_computing      <!-- 场景：边缘计算 -->
│     ├─embedded            <!-- 场景：嵌入式 -->
│     ├─devstation          <!-- 场景：DevStation -->
│     └─tools               <!-- 场景：创新场景与工具 -->
│        ├─unifiedbus       <!-- 工具模块：超节点 -->
│        ├─community_tools  <!-- 工具模块：社区工具 -->
│        ├─devops           <!-- 工具模块：devops（社区服务） -->
│        ├─ai               <!-- 工具模块：ai -->
│        ├─desktop          <!-- 工具模块：图形桌面使用 -->
│        ├─cloud            <!-- 工具模块：云原生工具 -->
│        ├─maintenance      <!-- 工具模块：系统运维 -->
│        └─security         <!-- 工具模块：安全 -->

```

#### 目录

各业务场景下均有具体的目录划分。以服务器场景为例，其进一步细分为发行说明、快速入门、安装升级、系统管理、系统运维等一级目录。

文档仓服务器场景目录结构示例如下：

```text{4-17}
├─docs
│  ├─en
│  └─zh
│      ├─server                     <!-- 场景：服务器 -->
│      │  ├─releasenotes            <!-- 一级目录：发行说明 -->
│      │  ├─quickstart              <!-- 一级目录：快速入门 -->
│      │  ├─installation_upgrade    <!-- 一级目录：安装升级 -->
│      │  ├─administration          <!-- 一级目录：系统管理 -->
│      │  ├─maintenance             <!-- 一级目录：系统运维 -->
|      |  ├─security                <!-- 一级目录：安全 -->
│      │  ├─memory_storage          <!-- 一级目录：内存与存储 -->
│      │  ├─network                 <!-- 一级目录：网络 -->
│      │  ├─performance             <!-- 一级目录：性能调优 -->
│      │  ├─development             <!-- 一级目录：应用开发 -->
│      │  ├─high_availability       <!-- 一级目录：HA高可用 -->
│      │  ├─diversified_computing   <!-- 一级目录：多样性算力 -->
│      │  └─_toc.yaml       
│      ├─virtualization             <!-- 场景：虚拟化 -->
│      ├─cloud                      <!-- 场景：云原生 -->
│      ├─edgecomputing              <!-- 场景：边缘计算 -->
│      ├─embedded                   <!-- 场景：嵌入式 -->   
│      ├─devstation                 <!-- 场景：DevStation -->      
│      └─tools                      <!-- 场景：创新场景与工具 -->
```

部分一级目录会进一步细分出二级目录，以服务器场景中的性能调优目录为例，其下进一步划分出二级目录，分别为概述、CPU调优、系统调优、调优框架。

文档仓服务器下性能调优的目录结构示例如下：

```text{13-22}
├─docs
│  ├─en
│  └─zh
│      ├─server                      <!-- 场景：服务器 -->
│      │  ├─releasenotes             <!-- 一级目录：发行说明 -->
│      │  ├─quickstart               <!-- 一级目录：快速入门 -->
│      │  ├─installation_upgrade     <!-- 一级目录：安装升级 -->
│      │  ├─administration           <!-- 一级目录：系统管理 -->
│      │  ├─maintenance              <!-- 一级目录：系统运维 -->
|      |  ├─security                 <!-- 一级目录：安全 -->
│      │  ├─memory_storage           <!-- 一级目录：内存与存储 -->
│      │  ├─network                  <!-- 一级目录：网络 -->
│      │  ├─performance              <!-- 一级目录：性能调优 -->
│      │  │  ├─overall               <!-- 二级目录：概述 -->
│      │  │  │  └─system_resource
│      │  │  ├─cpu_optimization      <!-- 二级目录：cpu调优 -->
│      │  │  │  ├─kae
│      │  │  │  └─sysboost
│      │  │  ├─system_optimization   <!-- 二级目录：系统调优 -->
│      │  │  │  └─atune
│      │  │  └─tuning_framework      <!-- 二级目录：调优框架 -->
│      │  │      └─oeaware
│      │  ├─development              <!-- 一级目录：应用开发 -->
│      │  ├─high_availability        <!-- 一级目录：HA高可用 -->
│      │  ├─diversified_computing    <!-- 一级目录：多样性算力 -->
│      │  └─_toc.yaml
│      ├─virtualization              <!-- 场景：虚拟化 -->
│      ├─cloud                       <!-- 场景：云原生 -->
│      ├─edge_computing              <!-- 场景：边缘计算 -->
│      ├─embedded                    <!-- 场景：嵌入式 -->
│      ├─devstation                  <!-- 场景：DevStation -->   
│      └─tools                       <!-- 场景：创新场景与工具 -->
```

#### 手册

目录下存放手册。以服务器场景中的系统运维目录为例，其中包含四本基础特性文档，每本文档分别对应文档仓的一个文件夹。

文档仓服务器下系统运维的目录结构示例如下：

```text{9-13}
├─docs
│   ├─en
│   └─zh
│      ├─server                     <!-- 场景：服务器 -->
│      │  ├─releasenotes            <!-- 一级目录：发行说明 -->
│      │  ├─quickstart              <!-- 一级目录：快速入门 -->
│      │  ├─installation_upgrade    <!-- 一级目录：安装升级 -->
│      │  ├─administration          <!-- 一级目录：系统管理 -->
│      │  ├─maintenance             <!-- 一级目录：系统运维 -->
│      │  │  ├─common_skills        <!-- 手册：《常用技能》 -->
│      │  │  ├─common_tools         <!-- 手册：《常用定位定界工具》 -->
│      │  │  ├─kernel_live_upgrade  <!-- 手册：《内核热升级指南》 -->
│      │  │  └─trouble_shooting     <!-- 手册：《故障应急处理》 -->
|      |  ├─security                <!-- 一级目录：安全 -->
│      │  ├─memory_storage          <!-- 一级目录：内存与存储 -->
│      │  ├─network                 <!-- 一级目录：网络 -->
│      │  ├─performance             <!-- 一级目录：性能调优 -->
│      │  ├─development             <!-- 一级目录：应用开发 -->
│      │  ├─high_availability       <!-- 一级目录：HA高可用 -->
│      │  ├─diversified_computing   <!-- 一级目录：多样性算力 -->
│      │  └─_toc.yaml
│      ├─virtualization             <!-- 场景：虚拟化 -->
│      ├─cloud                      <!-- 场景：云原生 -->
│      ├─edge_computing             <!-- 场景：边缘计算 -->
│      ├─embedded                   <!-- 场景：嵌入式 -->
│      ├─devstation                 <!-- 场景：DevStation -->   
│      └─tools                      <!-- 场景：创新场景与工具 -->
```

每本手册包含一个或多个文档内容文件（`.md`文件）对应一个或多个章节，及一个目录配置文件（`_toc.yaml`文件）。例如，《内核热升级指南》手册中包括三个章节，包括安装与部署、使用方法、常用问题与解决办法。

```text{12-16}
├─docs
│  ├─en
│  └─zh
│      ├─server                                        <!-- 场景：服务器 -->
│      │  ├─quickstart                                 <!-- 一级目录：快速入门 -->
│      │  ├─releasenotes                               <!-- 一级目录：发行说明 -->
│      │  ├─installation_upgrade                       <!-- 一级目录：安装升级 -->
│      │  ├─administration                             <!-- 一级目录：系统管理 -->
│      │  ├─maintenance                                <!-- 一级目录：系统运维 -->
│      │  │  ├─common_skills                           <!-- 手册：《常用技能》 -->
│      │  │  ├─common_tools                            <!-- 手册：《常用定位定界工具》 -->
│      │  │  ├─kernel_live_upgrade                     <!-- 手册：《内核热升级指南》 -->
│      |  │  │  ├─installation-and-deployment.md       <!-- 章节：安装与部署 -->
│      |  │  │  ├─how-to-run.md                        <!-- 章节：使用方法 -->
│      |  │  │  ├─common-problems-and-solutions.md     <!-- 章节：常用问题与解决办法 -->
│      |  │  │  └─_toc.yaml
│      │  │  └─trouble_shooting                        <!-- 手册：《故障应急处理》 -->
|      |  ├─security                                   <!-- 一级目录：安全 -->
│      │  ├─memory_storage                             <!-- 一级目录：内存与存储 -->
│      │  ├─network                                    <!-- 一级目录：网络 -->
│      │  ├─performance                                <!-- 一级目录：性能调优 -->
│      │  ├─development                                <!-- 一级目录：应用开发 -->
│      │  ├─high_availability                          <!-- 一级目录：HA高可用 -->
│      │  ├─diversified_computing                      <!-- 一级目录：多样性算力 -->
│      │  └─_toc.yaml
│      ├─virtualization                                <!-- 场景：虚拟化 -->
│      ├─cloud                                         <!-- 场景：云原生 -->
│      ├─edge_computing                                <!-- 场景：边缘计算 -->
│      ├─embedded                                      <!-- 场景：嵌入式 -->
│      ├─devstation                                    <!-- 场景：DevStation -->   
│      └─tools                                         <!-- 场景：创新场景与工具 -->
```

### SIG 组代码仓库目录结构说明

#### 手册

SIG 组文档/源码仓库仅存放特性手册，《oeAware用户指南》的目录结构示例如下：

```text
├─openeuler/oeAware-manager 仓
├─docs
|  ├─en
|  └─zh
|    └─2403_lts_sp2                          <!-- 版本：openEuler 24.03 LTS SP2 --> 
|           ├─oeawrae_user_guide.md          <!-- 章节：使用oeAware -->
|           └─_toc.yaml                              
```

#### SIG 组文档目录结构规范

##### 约束

* 分版本写文档，openEuler 各版本均有独立配套文档。
* 文档发布后仍会持续更新，如修正低级错误、内容错漏等问题。
* 各 SIG 组负责维护特性文档，要求各 SIG 清晰掌握特性在各版本中的分布情况。
* 各 SIG 组自主规划文档，明确区分发布至官网的文档和仅在代码仓展示的文档。
* SIG 组维护文档以手册为最小单元（如服务器场景《A-Ops用户指南》），单本手册不得拆分存储，同一仓库可容纳多本手册。
* 所见即所得，在 gitee 仓库可直接查看文档内容，呈现效果与最终展示一致。

##### 要求

无论各 SIG 如何组织文档，如果文档要发布至官网，需要满足如下条件：

* 在仓库根目录下创建 docs 目录，并在 docs 目录下的 zh/ 和 en/ 目录中存放需发布至官网的中英文文档，结构如下：

  ```text
  ├─docs/
  |  ├─zh/
  |  └─en/                                        
  ```

  > \[!NOTE]说明
  > 存放于 docs/zh 和 docs/en 下的文档均为发布至官网文档中心的文档，必须通过文档流水线，才能合入；若文档仅在仓库内展示、无需发布至官网，不可存放于上述目录，可在 docs/ 下创建其他目录存放。

* 在 openeuler/docs 仓库中，找到对应场景的 \_toc.yaml 文件，并按照指定格式添加文档的索引地址：

  upstream: 填写手册的目录配置文件（\_toc.yaml）索引地址\
  path（可选，默认值是“./仓库名”）: 设置手册的url访问路径

  示例：\
  upstream: <https://atomgit.com/openeuler/openstack-docs/blob/openEuler-25.03/docs/zh/_toc.yaml>\
  path: ./openstack

* 文档须严格对应当前版本特性，禁止包含未发布的特性或功能。

* 支持文档的长期更新维护，确保 openEuler 新版本发布后，历史版本文档仍保持准确完整。

##### 方案

为满足上述条件，建议的实现方案如下：

* 每个 SIG 组特性应该有自己的独立代码仓库，该仓库承载该SIG的特性文档、readme、contribute等内容。
* 在不同分支下维护对应版本的文档或创建目录维护不同版本的文档。
* 若选择创建目录维护不同版本的文档，其内容如下：

  ```text
  ├─docs/
  |  ├─zh/
  |  |  ├─2409/     
  |  |  |   ├─_toc.yaml     
  |  |  |   ├─xxx.md
  |  |  |   └─xxx/    
  |  |  |      └─xxx.md 
  |  |  └─2503/     
  |  |      ├─_toc.yaml     
  |  |      ├─xxx.md
  |  |      └─xxx/    
  |  |         └─xxx.md  
  |  └─en/  
  |     ├─2409/ 
  |     └─2503/                                       
  ```

## 目录配置文件格式

各个场景、每本手册均配置一个`_toc.yaml`文件，以维护目录结构。所有场景的`_toc.yaml`文件均存在 openeuler/docs 仓库，手册的`_toc.yaml`与手册源文件放在一起。
下面以服务器场景为例展示`_toc.yaml`的存放位置，其他场景的存放逻辑类似。

```text{7,9,10}
├─openeuler/docs 仓
├─docs
│  └─zh
│     ├─server                                       <!-- 场景：服务器 -->
│     │   ├─installation_upgrade                     <!-- 一级目录：安装升级 -->
│     │   |  ├─installation                          <!-- 手册：《安装指南》 -->
│     │   |  |   └─_toc.yaml 
│     │   |  ├─upgrade                               <!-- 手册：《升级指南》 -->
│     │   |  |   └─_toc.yaml 
│     │   └─_toc.yaml
```

### 手册的目录配置文件

每本手册都需要维护一个目录配置文件`_toc.yaml`来维护该本手册中各章节间的逻辑关系。

《内核热升级指南》手册的`_toc.yaml`文件示例如下：

```yaml
label: 内核热升级指南
isManual: true
description: 使用用户态自动化工具快速重启内核和程序热迁移实现内核热替换特性
sections:
  - label: 安装与部署
    href: ./installation-and-deployment.md
  - label: 使用方法
    href: ./how-to-run.md
  - label: 常见问题与解决方法
    href: ./common-problems-and-solutions.md
```

* label：手册名称。
* isManual：标识手册的目录配置文件，与场景的目录配置文件作区分。
* description：手册的简介说明。
* sections：
  * label：章节名称。
  * href：文档内容文件地址（建议使用相对路径）。

### 场景的目录配置文件

各业务场景下也要维护`_toc.yaml`文件，其中引用手册的`_toc.yaml`文件。以服务器场景为例，其`_toc.yaml`文件示例如下：

```yaml
label: 服务器
sections:
  - label: 从这里开始
    sections:
      - href: ./releasenotes/releasenotes/_toc.yaml
      - href: ./quickstart/quickstart/_toc.yaml
  - label: 安装升级
    sections:
      - href: ./installation_upgrade/installation/_toc.yaml
      - href: ./installation_upgrade/upgrade/_toc.yaml
  - label: 系统管理
    sections:
      - href: ./administration/administrator/_toc.yaml
      - href: ./administration/sysmaster/_toc.yaml
      - href: ./administration/compa_command/_toc.yaml
  - label: 系统运维
    sections:
      - href: 
          upstream: https://atomgit.com/openeuler/aops-zeus/blob/master/docs/zh/24.03_lts_sp2/_toc.yaml
          path: ./aops
      - href: ./maintenance/gala/_toc.yaml
      - href: ./maintenance/sysmonitor/_toc.yaml
      - href: ./maintenance/kernel_live_upgrade/_toc.yaml
      - href:
          upstream: https://atomgit.com/openeuler/syscare/blob/openEuler-24.03-LTS-SP2/docs/zh/_toc.yaml
          path: ./syscare
      - href: ./maintenance/common_skills/_toc.yaml
      - href: ./maintenance/common_tools/_toc.yaml
      - href: ./maintenance/troubleshooting/_toc.yaml
  - label: 安全
    sections:
      - href: ./security/secharden/_toc.yaml
      - href: ./security/trusted_computing/_toc.yaml
      - href: 
          upstream: https://atomgit.com/openeuler/secGear/blob/master/docs/zh/2403_LTS_SP2/_toc.yaml
          path: ./secgear
      - href: 
          upstream: https://atomgit.com/openeuler/cve-ease/blob/master/docs/zh/24.03_lts_sp2/_toc.yaml
          path: ./cve_ease
      - href: ./security/cert_signature/_toc.yaml
      - href: ./security/shangmi/_toc.yaml
      - href: 
          upstream: https://atomgit.com/openeuler/secDetector/blob/master/docs/zh/2403_LTS_SP2/_toc.yaml
          path: ./secdetector
  - label: 内存与存储
    sections:
      - href: ./memory_storage/lvm/_toc.yaml
      - href: ./memory_storage/etmem/_toc.yaml
      - href: ./memory_storage/gmem/_toc.yaml
      - href: ./memory_storage/hsak/_toc.yaml
  - label: 网络
    sections:
      - href: ./network/network_config/_toc.yaml
      - href: ./network/gazelle/_toc.yaml
  - label: 性能调优
    sections:
      - label: 概述
        sections:
          - href: ./performance/overall/system_resource/_toc.yaml
      - label: 调优框架
        sections:
          - href: 
              upstream: https://atomgit.com/openeuler/oeAware-manager/blob/master/docs/zh/2403_lts_sp2/_toc.yaml
              path: ./performance/tuning_framework/oeaware
      - label: CPU调优
        sections:
          - href: ./performance/cpu_optimization/sysboost/_toc.yaml
          - href: ./performance/cpu_optimization/kae/_toc.yaml
      - label: 系统调优
        sections:
          - href: 
              upstream: https://atomgit.com/openeuler/A-Tune/blob/master/docs/zh/24.03_LTS_SP2/_toc.yaml
              path: ./performance/system_optimization/atune
  - label: 应用开发
    sections:
      - href: ./development/application_dev/_toc.yaml
      - href: 
          upstream: https://atomgit.com/openeuler/compiler-docs/blob/openEuler-24.03-LTS-SP2/docs/zh/gcc/_toc.yaml
          path: ./compiler/gcc
      - href: 
          upstream: https://atomgit.com/openeuler/compiler-docs/blob/openEuler-24.03-LTS-SP2/docs/zh/llvm/_toc.yaml
          path: ./compiler/llvm
      - href: 
          upstream: https://atomgit.com/openeuler/compiler-docs/blob/openEuler-24.03-LTS-SP2/docs/zh/bisheng_autotuner/_toc.yaml
          path: ./compiler/bisheng_autotuner
      - href: ./development/ai4c/_toc.yaml
      - href: ./development/fangtian/_toc.yaml
      - href: ./development/annc/_toc.yaml
      - href: ./development/unt/_toc.yaml
  - label: HA高可用
    sections:
      - href: ./high_availability/ha/_toc.yaml
  - label: 多样性算力
    sections:
      - href: ./diversified_computing/dpu_offload/_toc.yaml
      - href: ./diversified_computing/dpu_os/_toc.yaml
```

* label：场景名称。
* description：场景的简介说明。
* sections：
  * label：一级目录名称。
  * sections：
    * href：手册的目录配置文件引用。
      * upstream（仅增量特性文档）：增量特性手册的目录配置文件引用。
      * path（仅增量特性文档）：手册的url访问路径（可选，默认值是“./仓库名”）。

## 文档检视测试流程

为保障文档质量，确保其得到充分检视与完备测试，建议各SIG组遵循以下流程。

![image](figures/检视合入流程.png)

**版本发布之前**：

1. 版本关键角色，根据版本需求，确定文档清单和交付计划，Doc SIG Maintainer 跟踪版本文档需求交付进展；
2. openeuler/docs 仓拉取新分支；
3. 代码仓拉取新分支；
4. 特性 owner 贡献文档，提 PR；
5. 特性 SIG Maintainer 检视文档；
6. 文档工程师检视文档，转测试；
7. 测试工程师检视文档；
8. 特性 owner 修改 PR 意见；
9. 特性 SIG Maintainer 合入文档。

**版本发布之后**：

1. 用户使用文档之后，发现文档问题；
2. 提交文档 Issue（基础特性文档提到 docs 仓，增量特性文档提到各个 SIG 组代码仓；如果提到 docs 仓，则由 Doc SIG maintainer 转到对应代码仓）；
3. 特性 owner 处理 Issue，提交文档 PR；
4. SIG Maintainer 检视文档；
5. 文档工程师检视文档，如仅低错修改，无需转测试；如涉及实际内容修改，则转测试（每周或双周固定时间转测试，与 update 版本 Issue 转测节奏保持一致）；
6. 测试工程师检视文档；
7. 特性 owner 修改 PR 意见；
8. SIG Maintainer 合入文档。

## 文档存放地址

openEuler 文档存储于 [openEuler/docs](https://atomgit.com/openeuler/docs) 仓和各 SIG 的文档/源码仓。以下为您提供每本手册在文档仓中的具体存放地址。

### 服务器

### 虚拟化

### 云原生

### 边缘计算

### 嵌入式

### 工具

---

---
url: /zh/docs/common/contribute/templates/feature_user_guide/readme.md
---
# 特性用户指南

本指南旨在帮助用户了解、安装和使用特性，通常命名为《XXX用户指南》（其中XXX为特性名称，如oeAware）。以下为指南的目录章节，开发者可按需调整，并可参考各章节的模板详情。

* [介绍](./description.md)
* [（可选）软件编译](./build.md)
* [安装XXX](./install.md)
* [使用XXX](./usage.md)
* [维护XXX](./maintain.md)
* [（可选）安全管理](./security.md)
* （可选）故障排除
* （可选）FAQ
* （可选）参考信息

`_toc.yaml`配置文件示例如下：

```yaml
label: XXX用户指南
isManual: true
sections:
  - label: 介绍
    href: ./description.md
  - label: 软件编译
    href: ./build.md
  - label: 安装XXX
    href: ./install.md
  - label: 使用XXX
    href: ./usage.md
  - label: 维护XXX
    href: ./maintain.md
...
```

> \[!NOTE]说明
> 建议将FAQ章节的内容统一收录至[常见问题](https://docs.openeuler.openatom.cn/zh/docs/common/faq/general/general_faq.html)，并在特性用户指南中添加指向该页面的链接，方便用户查阅。

---

---
url: /zh/docs/common/faq/general/project_intro_faq.md
---
# 特性通用问题

## openEuler的WSL应用场景有？

* 在 Windows 中快速部署和体验 openEuler LTS 版本。
* 利用 vs code 和 openEuler WSL 打造流畅跨平台开发体验。
* 在 openEuler WSL 中搭建 K8S 集群。
* 用你喜爱的 openEuler command-line 程序或脚本处理 Windows 或 WSL 中的文件和程序。

## openEuler中的HMDFS是什么 ？

HMDFS 是从 OH 社区迁移而来的在软总线生态之上的一个分布式文件系统，其在分布式软总线动态组网的基础上，为网络上各个设备结点提供一个全局一致的访问视图，支持开发者通过基础文件系统接口进行读写访问，具有高性能、低延时等优点。

## openEuler中的SysCare软件是什么？

SysCare 是一个系统级热修复软件，为操作系统提供安全补丁和系统错误热修复能力，主机无需重新启动即可修复该系统问题。SysCare 将内核态热补丁技术与用户态热补丁技术进行融合统一，用户仅需聚焦在自己核心业务中，系统修复问题交予 SysCare 进行处理。后期计划根据修复组件的不同，提供系统热升级技术，进一步解放运维用户提升运维效率。

## 什么是A-Ops？

A-Ops 是一款基于操作系统维度的故障运维平台，提供从数据采集、健康巡检、故障诊断、故障修复到智能运维解决方案。A-Ops 项目包括了若干子项目：覆盖故障发现（Gala）、故障定位支撑（X-diagnosis）、缺陷修复（Apollo） 等。

## secGear 主要提供哪三大能力？

* 架构兼容：屏蔽不同 SDK 接口差异，提供统一开发接口，实现不同架构共源码。
* 易开发：提供开发工具、通用安全组件等，帮助用户聚焦业务，开发效率显著提升。
* 高性能：提供零切换特性，在 REE-TEE 频繁交互、大数据交互等典型场景下提升 REE-TEE 交互性能 10 倍 +。

## AI for OS 安全主要有哪些技术？

* 漏洞挖掘：自动化漏洞挖掘是当前操作系统安全研究的热点，无论是基于代码分析的，还是模糊测试的，亦或两者结合的漏洞挖掘技术，都可以有效地识别出当前系统中存在的缺陷。传统的模糊测试工具在种子生成、选择、变异、测试、评估、反馈等多个环节都存在一定的盲目性和随机性，代码分析技术，无论是源码级，还是二进制级或者特定拓展成 DSL(Domain-  Specific Language)  的 IR(Intermediate Representation) 级，都十分依赖基于专家经验构建的缺陷模式库。结合人工智能技术，可以很好地挖掘缺陷代码数据集中的模式信息，从而指导模糊测试和代码分析技术的各个过程，有效地提升识别精度和效率。

* 入侵检测：以 APT 威胁为代表的现代安全问题持续出现且变幻莫测，攻击组织会使用一整套大型的攻击武器库，对目标系统进行自动化、持续的攻击尝试和自反馈，并且基于固定模式的安全防御技术难以抵御未知的威胁。因此，我们需要结合人工智能技术，自动化深度挖掘其中的关键特征，支持多种高维数据的联合特征提取。这在当前大数据时代是必须具备的能力，但是对于个人来说却又是困难的。另外，结合人工智能技术可以更有效地识别出系统中的异常行为或者状态，从而更准确、更及时地进行攻击阻断。比如，在异常流量检测、侧信道攻击检测领域中，通过结合人工智能技术的入侵检测技术，都取得了很好的效果。

## openEuler 提供的多级调度框架有什么优势？

openEuler 提供了多级调度框架，实现多种调度模型共存，业务可根据需要进行调度模型的选择。

* 相比于传统的进程/线程调度模型更为灵活，可移植性更好。
* 新增的协程等轻量级调度模型切换更快，调度时间占比更小。

## 根据openEuler操作系统安全机制的作用范围，可将这些安全机制分为哪三种类型？

真实性保护、完整性保护和机密性保护三种类型。

## 工业安全领域openEuler系统运用的安全隔离技术主要有哪两种范式？

* 隔离已知来源但可能存在漏洞的服务，以削减其受到攻击后对系统其他组成部分造成的危害。
* 限制不受信任来源的代码（可能是恶意代码或存在漏洞的组件）可能对系统其他组成部分造成的危害。

---

---
url: /zh/docs/common/faq/general/general_faq.md
---
# 社区通用问题

## openEuler 是什么？

OpenAtom openEuler（简称“openEuler”）是开放原子开源基金会孵化及运营的开源项目。
它是一个面向数字基础设施的开源操作系统，支持服务器、云计算、边缘计算、嵌入式等应用场景，支持多样性计算，致力于提供安全、稳定、易用的操作系统。通过为应用提供确定性保障能力，支持 OT 领域应用及 OT 与 ICT 的融合。

## openEuler社区是怎么样的？

2019 年 12 月 31 日，面向多样性计算的操作系统开源社区openEuler正式成立。
openEuler社区致力于与全球的开发者共同构建一个开放、多元和架构包容的软件生态体系，孵化支持多种处理器架构、覆盖数字基础设施全场景，推动企业数字基础设施软硬件、应用生态繁荣发展。openEuler社区和上下游生态紧密连接，构建多样性的社区合作伙伴和协作模式，共同推进版本演进。

## openEuler支持哪些架构？

openEuler社区当前已与多个设备厂商建立丰富的生态，包括Intel、AMD、兆芯、海光、鲲鹏、飞腾、龙芯、申威等主流芯片厂商，支持x86、Arm、SW64、RISC-V、LoongArch等多种处理器架构，逐步扩展PowerPC 等更多芯片架构；支持多款CPU芯片，包括龙芯3号、兆芯开先和开胜系列、Intel Sierra Forest和Granite Rapids、AMD EPYC Milan和Genoa等芯片系列；支持多个硬件厂商发布的多款整机型号、板卡型号，支持网卡、RAID、FC、GPU\&AI、DPU、SSD、安全卡七种类型的板卡，具备良好的兼容性。

## openEuler多久发布一次新版本？

openEuler发布两种社区版本，包括长期支持版本（即“LTS版本”）和创新版本。
LTS版本每两年发布一版，提供四年社区支持，包括两年的维护支持和两年的延长支持。LTS版本为企业级用户提供一个安全稳定可靠的操作系统。
创新版每六个月发布一版，提供六个月社区支持。创新版本快速集成openEuler的最新技术成果，将验证成熟的特性逐步回合到发行版中。这些新特性以单个开源项目的方式存在于社区，方便开发者获得源代码，也方便用户使用。
在单个版本生命周期结束前，用户会提前三个月在社区[邮件列表](https://www.openeuler.org/zh/community/mailing-list/)中收到公告等通知。

## openEuler有哪些SIG组？要如何加入？

目前openEuler社区共有超100个SIG组（special interest group），分别针对特定的技术主题或项目成立。每个SIG在Gitee上都会拥有一个或多个项目，这些项目会拥有一个或多个仓库，SIG的交付成果会保存在这些仓库内。SIG成员可以在仓库内提交Issue、解决问题、参与评审等，推动交付成果成为openEuler社区发行的一部分。
openEuler的SIG分为代码仓管理和社区运营治理两种类型，致力于推动工具链、架构、桌面、通用中间件、云原生基础设施等领域的技术创新，覆盖了AI、嵌入式、安全和合规等热门主题，助力openEuler社区生态构建。
您可以通过订阅SIG邮件、参加公开例会或直接联系SIG maintainer，加入心仪的SIG。SIG全景图请见[SIG中心](https://www.openeuler.org/zh/sig/sig-list/)。
如果现有的SIG中没有您感兴趣的，您可以在openEuler社区中寻找两到三个具有共同目标的人讨论决定成立SIG组，维护社区中的某个技术方向的软件包或发起孵化项目。具体如何成立SIG，请见[申请流程](https://www.openeuler.org/zh/sig/sig-guidance/)。

## 如何贡献openEuler社区？

openEuler社区欢迎代码类贡献和非代码类贡献。您可以按照以下步骤进行社区贡献：

1. 在参与社区贡献之前，您需要根据自身身份（个人、员工、或企业），签署对应的[贡献者许可协议](https://clasign.osinfra.cn/sign/gitee_openeuler-1611298811283968340)，即“CLA协议”。

2. 在[SIG中心](https://www.openeuler.org/zh/sig/sig-list/)，找到您感兴趣的SIG并加入。如果您对某个方向有浓厚的兴趣，且未找到对应的SIG组，那么您可以参考[SIG组申请流程](https://www.openeuler.org/zh/sig/sig-guidance/)来申请创建新的SIG进行维护和发展。

3. 贡献原创开源项目、进行代码类贡献和非代码类贡献。openEuler社区有两类代码仓库，包括存放源码类项目的[代码仓](https://atomgit.com/openeuler)和存放制作发布件所需的[软件包仓](https://gitee.com/src-openeuler)。

   * 可以直接在代码仓中创建原创项目，或者将您在其他社区开发的软件包加入到软件包仓。具体操作，请参考[新增代码包](https://atomgit.com/openeuler/community/blob/master/zh/contributors/create-package.md)。
   * 可以使用Gitee、GitHub或邮箱账号登录官网的[QuickIssue](https://quickissue.openeuler.org/zh/issues/)页面，快速提交issue。
   * 如果您对非代码贡献感兴趣，也可以在[非代码贡献指南](https://atomgit.com/openeuler/community/blob/master/zh/contributors/non-code-contributions.md)中找到合适的项目。

4. 参加社区活动：openEuler社区举办多种线上线下活动，包括主题峰会、社媒直播、meetup、SIG活动等，期待您的加入。

想要了解更多openEuler社区贡献途径，请访问[贡献攻略](https://www.openeuler.org/zh/community/contribution/detail.html)。

## 从哪些渠道可以获取openEuler的最新资讯？可以在哪些平台和其他openEuler用户交流？

您可以通过以下方式获取openEuler的最新资讯和进行互动交流：

* 官方网站：通过浏览openEuler官方网站，获取文档、白皮书和用户案例等相关信息。
* 课程中心：探索我们的[在线课程](https://www.openeuler.org/zh/learn/mooc/)，深入了解openEuler的技术细节。
* 社交媒体：关注我们的社交媒体账号，如openEuler微信公众号、B站、头条号等，获取开源行业和操作系统行业相关的最新事件、合作关系以及技术解决方案的最新资讯。
* 邮件列表：订阅我们的[邮件列表](https://www.openeuler.org/zh/community/mailing-list/)，获取openEuler各个SIG的最新动态。
* 官方论坛：访问[openEuler论坛](https://forum.openeuler.org/)或通过添加openEuler小助手（微信号：openeuler123）微信好友加入社区交流群，进行提问和参与讨论。

openEuler社区欢迎您的加入，期待您能在社区中提升技能、结交朋友！

## openEuler社区有哪些合作伙伴？应用于哪些行业？

openEuler目前已广泛应用于各行各业，包括政务、金融、运营商、互联网、电力、制造业、能源、教育、交通和医疗等行业。
openEuler 希望与广大生态伙伴、用户、开发者一起，通过联合创新、社区共建，不断增强场景化能力，最终实现统一操作系统支持多设备，应用一次开发覆盖全场景。

## openEuler操作系统噪声是指什么？

操作系统噪声是指业务运行中执行的非应用计算任务，包括：

* 系统/用户态守护进程。
* 中断处理。
* 用户态或内核中驻留的进程。
* 内存管理、调度开销。
* 业务应用中的非计算任务，如监控 log 线程通信等。
* 资源竞争带来的噪声，如由共享高速缓存导致的高速缓存不命中 (Cache    Miss), 以及由共享物理内存导致的页面错误 (Page  Fault)。

## openEuler常用repo源

为了方便大家快速找到openEuler所需版本的repo源，现将openEuler各版本的repo源进行了整理并归类，详情可查看：<https://forum.openeuler.org/t/topic/768>

---

---
url: /zh/docs/common/faq/server/installation_faq1.md
---
# 系统安装常见问题与解决方法-1

## **问题1：安装openEuler时选择第二盘位为安装目标，操作系统无法启动**

### 问题现象

安装操作系统时，直接将系统安装到第二块磁盘sdb，重启系统后启动失败。

### 原因分析

当安装系统到第二块磁盘时，MBR和GRUB会默认安装到第二块磁盘sdb。这样会有下面两种情况：

1. 如果第一块磁盘中有完整系统，则加载第一块磁盘中的系统启动。
2. 如果第一块磁盘中没有完好的操作系统，则会导致硬盘启动失败。

以上两种情况都是因为BIOS默认从第一块磁盘sda中加载引导程序启动系统，如果sda没有系统，则会导致启动失败。

### 解决方法

有以下两种解决方案：

* 当系统处于安装过程中，在选择磁盘（选择第一块或者两块都选择）后，指定引导程序安装到第一块盘sda中。
* 当系统已经安装完成，若BIOS支持选择从哪个磁盘启动，则可以通过修改BIOS中磁盘启动顺序，尝试重新启动系统。

## **问题2：openEuler开机后进入emergency模式**

### 问题现象

openEuler系统开机后进入emergency模式，如下图所示：

![](./figures/zh-cn_image_0229291264.jpg)

### 原因分析

操作系统文件系统损坏导致磁盘挂载失败，或者io压力过大导致磁盘挂载超时（超时时间为90秒）。

系统异常掉电、物理磁盘io性能低等情况都可能导致该问题。

### 解决方法

1. 用户直接输入root帐号的密码，登录系统。
2. 使用fsck工具，检测并修复文件系统，然后重启。

   > \[!NOTE]说明
   > fsck（file system check）用来检查和维护不一致的文件系统。若系统掉电或磁盘发生问题，可利用fsck命令对文件系统进行检查。 用户可以通过“fsck.ext3 -h”、“fsck.ext4 -h”命令查看fsck的使用方法。

另外，如果用户需要取消磁盘挂载超时时间，可以直接在“/etc/fstab”文件中添加“x-systemd.device-timeout=0”。如下：

```sh
# /etc/fstab
# Created by anaconda on Mon Sep 14 17:25:48 2015
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/openEuler-root / ext4 defaults,x-systemd.device-timeout=0 0 0
UUID=afcc811f-4b20-42fc-9d31-7307a8cfe0df /boot ext4 defaults,x-systemd.device-timeout=0 0 0
/dev/mapper/openEuler-home /home ext4 defaults 0 0
/dev/mapper/openEuler-swap swap swap defaults 0 0
```

## **问题3：系统中存在无法激活的逻辑卷组时，重装系统失败**

### 问题现象

由于磁盘故障，系统中存在无法激活的逻辑卷组，重装系统出现异常。

### 原因分析

安装时有激活逻辑卷组的操作，无法激活时会抛出异常。

### 解决方法

重装系统前如果系统中存在无法激活的逻辑卷组，为了避免重装系统过程出现异常，需在重装前将逻辑卷组恢复到正常状态或者清除这些逻辑卷组。举例如下：

* 恢复逻辑卷组状态
  1. 使用以下命令清除vg激活状态， 防止出现“Can't open /dev/sdc exclusively mounted filesystem”。

     ```sh
     # vgchange -a n testvg32947
     ```

  2. 根据备份文件重新创建pv。

     ```sh
     # pvcreate --uuid JT7zlL-K5G4-izjB-3i5L-e94f-7yuX-rhkLjL --restorefile /etc/lvm/backup/testvg32947 /dev/sdc
     ```

  3. 恢复vg信息。

     ```sh
     # vgcfgrestore testvg32947
     ```

  4. 重新激活vg。

     ```sh
     # vgchange -ay testvg32947
     ```

* 清除逻辑卷组

  ```sh
  # vgchange -a n testvg32947
  # vgremove -y testvg32947
  ```

## **问题4：选择安装源出现异常**

### 问题现象

选择安装源后出现："Error checking software selection"。

### 原因分析

这种现象是由于安装源中的软件包依赖存在问题。

### 解决方法

检查安装源是否存在异常。使用新的安装源。

## **问题5：如何手动开启kdump服务**

### 问题现象

执行systemctl status kdump命令，显示状态信息如下，提示无预留内存。

![](./figures/zh-cn_image_0229291280.png)

### 原因分析

kdump服务需要系统预留一段内存用于运行kdump内核，而当前系统没有为kdump服务预留内存，所以无法运行kdump服务。

### 解决方法

已安装操作系统的场景

1. 修改/boot/efi/EFI/openEuler/grub.cfg，添加crashkernel=1024M，high。
2. 重启系统使配置生效。
3. 执行如下命令，检查kdump状态：

   ```sh
   # systemctl status kdump
   ```

   若回显如下，即kdump的状态为active，说明kdump已使能，操作结束。

   ![](./figures/zh-cn_image_0229291272.png)

### 参数说明

kdump内核预留内存参数说明如下：

**表 1**  crashkernel参数说明

## **问题6：多块磁盘组成逻辑卷安装系统后，再次安装不能只选其中一块磁盘**

### 问题现象

在安装系统时，如果之前的系统选择多块磁盘组成逻辑卷进行安装，再次安装时，如果只选择了其中的一块或几块磁盘，没有全部选择，在保存配置时提示配置错误，如[图1](#fig115949762617)所示。

**图 1**  配置错误提示\
![](./figures/Configuration_error_prompt.png)

### 原因分析

之前的逻辑卷包含了多块磁盘，只在一块磁盘上安装会破坏逻辑卷。

### 解决方法

因为多块磁盘组成逻辑卷相当于一个整体，所以只需要删除对应的卷组即可。

1. 按“Ctrl+Alt+F2”可以切换到命令行，执行如下命令找到卷组。

   ```sh
   # vgs
   ```

   ![](./figures/zh-cn_image_0231657950.png)

2. 执行如下命令，删除卷组。

   ```sh
   # vgremove euleros
   ```

3. 执行如下命令，重启安装程序即可生效。

   ```sh
   # systemctl restart anaconda
   ```

   > \[!NOTE]说明
   > 图形模式下也可以按“Ctrl+Alt+F6”回到图形界面，点击[图1](#fig115949762617)右下角的“Refresh”刷新存储配置生效。

## **问题7：x86物理机UEFI模式由于secure boot安全选项问题无法安装**

### 问题现象

x86物理机安装系统时，由于设置了BIOS选项secure boot 为enable（默认是disable），导致系统一直停留在“No bootable device”提示界面，无法继续安装，如[图2](#fig115949762618)所示。

**图 2**  “No bootable device”提示界面\
![](./figures/No-bootable-device.png)

### 原因分析

开启secure boot后，主板会验证引导程序及操作系统 ，若没有用对应的私钥进行签名，则无法通过主板上内置公钥的认证。

### 解决方法

进入BIOS，设置secure boot为disable，重新安装即可。

1. 系统启动时，按“F11”，输入密码“Admin@9000”进入BIOS。

   ![](./figures/BIOS.png)

2. 选择进入Administer Secure Boot。

   ![](./figures/security.png)

3. 设置Enforce Secure Boot为Disabled。

   ![](./figures/select.png)

   > \[!NOTE]说明
   > 设置secure boot为disable之后，保存退出，重新安装即可。

## **问题8：安装openEuler时，软件选择页面选择“服务器-性能工具”，安装后messages日志有pmie\_check报错信息**

### 问题现象

安装系统时软件选择勾选服务器-性能工具，会安装pcp相关软件包，正常安装并重启后，/var/log/messages日志文件中会产生报错：pmie\_check failed in /usr/share/pcp/lib/pmie。

### 原因分析

anaconda不支持在chroot环境中安装selinux策略模块，当安装pcp-selinux时，postin脚本安装pcp相关selinux策略模块执行失败，从而导致重启后产生报错。

### 解决办法

完成安装并重启后，以下方法选择其一。

1. 执行如下命令，安装selinux策略模块pcpupstream

   ```sh
   # /usr/libexec/pcp/bin/selinux-setup /var/lib/pcp/selinux install "pcpupstream"
   ```

2. 重新安装pcp-selinux

   ```sh
   # sudo dnf reinstall pcp-selinux
   ```

## **问题9：在两块已经安装了系统的磁盘上进行重复选择，并自定义分区时，安装失败**

### 问题现象

用户在安装操作系统过程中，存在两块都已经安装过的磁盘，此时如果先选择一块盘，进行自定义分区，然后点击取消按钮，再选择第二块盘，并进行自定义分区时，会出现安装失败。

![](./figures/cancle_disk.png)
![](./figures/custom_paratition.png)

### 原因分析

用户存在两次选择磁盘的操作，当前点击取消后，再选择第二块磁盘，磁盘信息不正确，导致安装失败。

### 解决方法

直接选择目标磁盘进行自定义分区，请勿频繁取消操作，如果一定要进行取消重选建议重新安装。

### issue访问链接

<https://gitee.com/src-openeuler/anaconda/issues/I29P84?from=project-issue>

## **问题10：安装LSI MegaRAID卡的物理机kdump无法生成vmcore**

### 问题现象

部署好kdump服务后，手动执行`echo c > /proc/sysrq-trigger`命令或由于kernel故障导致kernel宕机，触发kdump启动second kernel过程中，MegaRAID驱动报错“BRCM Debug mfi stat 0x2d，data len requested/completed 0x200/0x0”，报错信息如下图，最终导致无法生成vmcore。

![](./figures/Megaraid_IO_Request_uncompleted.png)

### 原因分析

由于默认配置了reset\_devices启动参数，second kernel启动过程中会触发设备复位(reset\_devices)操作，设备复位操作导致MegaRAID控制器或磁盘状态故障，转储vmcore文件时访问MegaRAID卡的磁盘报错，进而无法生成vmcore。

### 解决方法

在物理机`etc/sysconfig/kdump`文件中将second kernel默认启动参数`reset_devices`删除，可以规避second kernel启动过程中由于MegaRAID卡驱动复位设备所致IO请求未完成问题，以成功生成vmcore。
![](./figures/reset_devices.png)

---

---
url: /zh/docs/common/faq/server/installation_faq2.md
---
# 系统安装常见问题与解决方法-2

## **问题1:树莓派启动失败**

### 问题现象

将 openEuler 发布的树莓派镜像刷写入 SD 卡后，树莓派启动失败。

### 原因分析

刷写 openEuler 发布的树莓派镜像后，树莓派启动失败，大致有以下几种情况：

1. 下载的镜像文件不完整，请确保该镜像通过完整性校验。
2. 镜像写入 SD 卡过程中出现问题，多出现在 Windows 环境下使用应用软件刷写镜像到 SD 卡的情况。

### 解决方法

将完整的镜像重新刷写入 SD 卡。

## **问题2:nmcli 命令连接 WIFI 失败**

### 问题现象

执行 `nmcli dev wifi connect SSID password PWD` 命令连接 WIFI 失败。例如提示 `Error: Connection activation failed: (7) Secrets were required, but not provided.` 等错误。

### 原因分析

执行的命令缺少密码。注意，如果密码中包含特殊字符，需要使用单引号将密码括起来。如果使用 nmcli 命令行连接 WIFI 失败，建议使用 nmtui 字符界面进行连接。

### 解决方法

执行 `nmtui` 命令进入到 nmtui 字符界面，按照以下步骤连接 WIFI。

1. 选择 `Edit a connection`，按 `Enter` 进入编辑网络连接窗口。

2. 按下键盘右方向键选择 `Add`，按 `Enter` 进入新建网络连接窗口。

3. 连接类型选择 `Wi-Fi` ，然后按下键盘右方向键选择 `Create`，按 `Enter` 进入 WIFI 编辑连接信息的界面。

4. WIFI 连接信息界面主要需要编辑以下内容，其他信息根据实际情况而定。编辑结束后选择 `OK`，按 `Enter` 完成编辑并回退到编辑网络连接窗口。
   1. `Profile name` 栏输入该 WIFI 连接的名称，这里可以使用默认名称，如 `Wi-Fi connection 1`；
   2. `Device` 栏输入要使用的无线网卡接口，这里输入 `wlan0`；
   3. `SSID` 栏输入要连接的 WIFI 的 SSID；
   4. `Security` 栏选择 WIFI 密码加密方式，这里根据实际情况选择，例如选择 `WPA & WPA2 Personal`；
   5. `Password` 栏输入 WIFI 密码。

5. 选择 `Back` 回退到最初的 nmtui 字符界面。

6. 选择 `Activate a connection`，按 `Enter` 进入激活网络连接窗口。

7. 查看添加的 WIFI 连接是否已激活（已激活的连接名称前有 `*` 标记）。如果未激活，选择该 WIFI 连接，然后按下键盘右方向键选择 `Activate`，按 `Enter` 激活该连接。待激活完成后，选择 `Back`，按 `Enter` 退出该激活界面，回退到最初的 nmtui 字符界面。

8. 选择 `Quit`，然后按下键盘右方向键选择 `OK`，按 `Enter` 退出 nmtui 字符界面。

## **问题3:tensorflow包及相关包安装失败**

### 问题现象

使用yum安装tensorflow及相关包时失败。

### 原因分析

tensorflow的依赖包暂时未升级至适配tensorflow==2.12.1的版本，因此需要通过pip手动安装其依赖软件。

### 解决方法

1. yumdownloader下载tensorflow的rpm包：yumdownloader python3-tensorflow。
2. 使用rpm --nodeps安装这个包：rpm -ivh --nodeps python3-tensorflow。
3. 安装tensorflow依赖包。
   1. 使用pip安装依赖：pip3 install tensorflow-estimator==2.12.0 keras==2.12.0 protobuf==3.20.3。
   2. 使用yum安装其他依赖软件：yum install python3-termcolor python3-future python3-numpy python3-six python3-astunparse python3-google-pasta python3-opt-einsum python3-typing-extensions python3-wrapt python3-h5py python3-grpcio python3-absl-py python3-flatbuffers python3-gast
4. 直接用yum下载相关包，例如python-keras-rl2，直接执行yum install python-keras-rl2。

---

---
url: /zh/docs/common/faq/server/system_management_faq.md
---
# 系统性能常见问题与解决方法

## 为什么在 openEuler 22.03 SP1 系统中，启动 NFS 服务后，尽管最初可以达到千兆网络速率，但经过不到一天的写入操作，客户端的响应速度会急剧下降到大约 2MB/s？

这个性能下降的主要原因是 NFS 服务端的缓存上涨至 50% 时，其性能会骤降。这种现象主要是由于内存分配和回收机制的问题。在申请内存的过程中，系统不会立即同步回收内存，而是依赖于较慢的后台内存回收机制。当系统无法及时申请到足够的内存时，会导致等待时间（如 500 毫秒的延迟）。这个问题在高负载下尤为明显，因为 NFS 服务端需要大量内存来处理客户端的请求。随着服务运行时间的增长，可用内存的减少会导致性能急剧下降，特别是在密集的写入操作中。

## 如何解决 openEuler 系统中由于 ext4 文件系统的 inode 错误而导致的文件创建失败问题？

为了解决这个问题，应该在处理 dx\_node 块的 rec\_len 字段时使用正确的方法。正确的做法是使用 ext4\_rec\_len\_from\_disk() 函数来转换 rec\_len 为 65536，然后进行比较。这样可以确保在添加新的 dx\_node 块时，正确地设置 rec\_len 字段，并为节点计算并设置正确的校验和。这个更正将避免由于校验和不正确导致的 inode 错误，从而允许系统正常创建和管理大量文件。

## 如何解决 openEuler 系统中业务进程因 glibc 线程缓存特性导致的内存占用过高问题？

解决这个问题的方法是关闭 glibc 的线程缓存特性。这可以通过在启动程序之前设置环境变量来实现。具体操作是在 bash\_profile 中添加 GLIBC\_TUNABLES=glibc.malloc.tcache\_count=0，这样可以关闭线程缓存。在进程启动后，还需要检查进程的环境变量（/proc/pid/environ）以确保成功添加。关闭 glibc 的 tcache 之后，进程的内存管理将与 glibc 2.17 版本一致，不会有其他副作用。根据客户反馈，实施这一修改方案后，内存占用明显降低，且相比于 CentOS，openEuler 的内存占用也变得更低。

## 为什么在 ARM 架构上进行 fio 压测多盘场景时，性能仅为 X86 架构的一半？

在 ARM 架构下进行 fio 压测时，性能低于 X86 架构的主要原因是中断处理机制的差异。在 ARM 环境中，8 盘同时压测时，所有盘的中断都集中在了 0、32、64 核心上，导致处理这些中断的 CPU 被压满。这个瓶颈主要是由于 ARM 架构下的 LPI (Large Payload Interface) 中断和 X86 架构下的 APIC 中断的实现机制不同。X86 架构通过 APIC 实现了中断负载均衡，而 ARM 架构的 LPI 中断主要由 ITS (Interrupt Translation Service) 驱动程序处理，它会默认将中断分配到最低编号的 CPU，导致单核处理中断出现瓶颈。

## 为什么在机器掉电后，系统启动时 xfs 文件系统会出现问题，执行 ls 命令时显示 input/output error？

机器掉电可以导致 xfs 文件系统损坏，因为掉电可能发生在文件系统写入操作的过程中，导致数据未能完全写入磁盘。当系统重新启动后，xfs文件系统可能处于不一致的状态，这种状态下执行文件系统操作，如 ls 命令，可能会遇到输入/输出错误。这类错误通常表明文件系统的某些部分未能正确加载或读取，可能是由于文件系统元数据损坏或者未完成的写入操作造成的。

## 如何解决系统启动时未使用新内核的问题？

要解决这个问题，用户需要将 /boot/grub2/grub.cfg 文件的内容替换为 /boot/efi/EFI/xxxx/grub.cfg 中的内容。这样做可以确保在启动时系统会读取包含新内核的正确配置文件。此外，检查并确认系统的引导模式（UEFI 或 legacy）也是解决此类问题的重要步骤。

---

---
url: /zh/docs/common/faq/caselibrary/sysfile.md
---
# 系统文件恢复问题

## 问题背景

在正常使用中常会出现误删系统文件或其他系统修改无法正常进入系统的问题，需要进行磁盘系统的恢复或修改，或者之前有重要数据需要备份保存。

## 现象描述

系统文件丢失或修改，导致无法正常进入系统，需要恢复系统文件。

## 解决方案

1. 挂载镜像，CTRL+ALT+F2切换后台。
   进入后台可以查看当前磁盘的状态，处于未激活状态，不能直接操作磁盘。
   ![image](./figures/系统恢复_未激活.png)

2. 配置网络，开启ssh服务。

   ```txt
   ifconfig eth0 xx.xx.xx.xx  netmask  255.255.255.0 up
   route add default gw xx.xx.xx.xx
   cp /etc/ssh/sshd_config.anaconda /etc/ssh/sshd_config
   systemctl restart sshd
   ```

3. 激活系统卷组。
   使用命令vgchange -ay对磁盘存在的系统进行激活操作。
   ![image](./figures/系统恢复_激活.png)

4. 通过挂载的方式挂载上各个分区，挂载后可进行系统操作。

   1. 创建临时目录test，将激活后的系统的进行挂载，挂载根/dev/oprnruler/root到临时目录test。
      ![image](./figures/系统恢复_挂载根.png)

   2. 挂载激活后的boot（/dev/sda2）到临时系统的boot上。
      ![image](./figures/系统恢复_挂载boot.png)

   3. 在此基础上就可以操作磁盘系统以及系统数据了。

---

---
url: /zh/docs/common/faq/server/administration_faqs.md
---
# 系统管理常见问题与解决方法

## **问题1：使用systemctl和top命令查询libvirtd服务占用内存不同**

### 问题描述

使用systemctl和systemd-cgtop命令查询libvirtd服务占用内存超1.5G，而使用top命令查询libvirtd服务占用内存仅70M左右。

### 原因分析

systemd管理的服务（包括systemctl和systemd-cgtop）中显示的内存通过查询CGroup对应的memory.usage\_in\_bytes得到。top是直接统计/proc下内存相关信息计算得出。两者的统计方法不同，不能直接比较。

一般来说，业务进程使用的内存主要有以下几种情况：

* anon\_rss：用户空间的匿名映射页（Anonymous pages in User Mode address spaces），比如调用malloc分配的内存，以及使用MAP\_ANONYMOUS的mmap。当系统内存不够时，内核可以将这部分内存交换出去。
* file\_rss：用户空间的文件映射页（Mapped pages in User Mode address spaces），包含map file和map tmpfs，前者比如指定文件的mmap，后者比如IPC共享内存。当系统内存不够时，内核可以回收这些页，但回收之前可能需要与文件同步数据。
* file\_cache：文件缓存（page in page cache of disk file），普通读写（read/write）文件时产生的文件缓存。当系统内存不够时，内核可以回收这些页，但回收之前可能需要与文件同步数据。
* buffer pages：属于page cache，比如读取块设备文件时的相关缓存。

其中anon\_rss和file\_rss属于进程的RSS，file\_cache和buffer pages属于page cache。简单来说：

top里的RSS = anon\_rss + file\_rss，SHR = file\_rss。

CGroup里的memory.usage\_in\_bytes = cache + RSS + swap。

由上可知，systemd相关命令和top命令的内存占用率含义不同，所以查询结果不同。

## **问题2：设置RAID0卷，参数stripsize设置为4时出错**

### 问题现象

设置RAID0卷，参数stripsize设置为4时出错。

### 原因分析

64K页表开启只能支持64K场景。

### 解决方法

不需要修改配置文件，openEuler执行lvcreate命令时，条带化规格支持的stripesize最小值为64KB，将参数stripesize设置为64。

## **问题3：使用rpmbuild编译mariadb失败**

### 问题描述

如果使用root帐号登录系统，并在该帐号下使用rpmbuild命令编译mariadb源代码，会出现编译失败现象，提示：

```shell
# echo 'mysql can'\''t run test as root'
mysql can't run test as root
# exit 1
```

### 原因分析

mariadb数据库不允许使用root权限的帐号进行测试用例执行，所以会阻止编译过程（编译过程中会自动执行测试用例）。

### 解决方案

使用vi等文本编辑工具，修改mariadb.spec文件中runtest变量的值。

修改前：

```text
%global runtest 1
```

修改后：

```text
%global runtest 0
```

该修改关闭了编译阶段执行测试用例的功能，但不会影响编译和编译后的RPM包内容。

## **问题4：使用默认配置启动SNTP服务失败**

### 问题现象

默认配置情况下SNTP服务启动失败。

### 原因分析

默认配置中未添加授时服务器域名。

### 解决方案

修改/etc/sysconfig/sntp文件 ，在文件中添加中国NTP快速授时服务器域名：0.generic.pool.ntp.org。

## **问题5：安装时出现软件包冲突、文件冲突或缺少软件包导致安装失败**

### 问题现象

安装软件包过程中，可能出现软件包冲突、文件冲突或缺少软件包，从而导致升安装被中断，最终安装失败。软件包冲突、文件冲突和缺少软件包的报错信息分别如下所示。

软件包冲突报错信息示例（以 libev-libevent-devel-4.24-11.oe1.aarch64与libevent-devel-2.1.11-2.oe1.aarch64冲突为例）：

```text
package libev-libevent-devel-4.24-11.oe1.aarch64 conflicts with libevent-devel provided by libevent-devel-2.1.11-2.oe1.aarch64  
 - cannot install the best candidate for the job  
 - conflicting requests
```

文件冲突报错信息示例（以/usr/bin/containerd文件冲突为例）：

```text
Error: Transaction test error:  
 file /usr/bin/containerd from install of containerd-1.2.0-101.oe1.aarch64 conflicts with file from package docker-engine-18.09.0-100.aarch64  
 file /usr/bin/containerd-shim from install of containerd-1.2.0-101.oe1.aarch64 conflicts with file from package docker-engine-18.09.0-100.aarch64
```

缺少软件包的报错信息示例（以缺失blivet-data软件包为例）：

```text
Error:  
  Problem: cannot install both blivet-data-1:3.1.1-6.oe1.noarch and blivet-data-1:3.1.1-5.noarch  
   - package python2-blivet-1:3.1.1-5.noarch requires blivet-data = 1:3.1.1-5, but none of the providers can be installed  
   - cannot install the best update candidate for package blivet-data-1:3.1.1-5.noarch  
   - problem with installed package python2-blivet-1:3.1.1-5.noarch(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
```

### 原因分析

* openEuler提供的软件包中，有些软件包虽然名称不同，但功能相同，导致两个软件包无法同时安装。
* openEuler提供的软件包中，有些软件包虽然名称不同，但功能相同，导致安装时安装后的文件相同，从而产生了文件冲突。
* 有些软件包，因在升级安装前被其他软件包所依赖，一旦该软件包升级后，可能导致依赖它的软件包因缺少软件包而不能安装。

### 解决方案

若为软件包冲突，则按如下步骤进行处理（以问题现象中示例的软件包冲突为例）：

1. 根据安装过程中的软件包冲突报错信息，确定与待安装的 libev-libevent-devel-4.24-11.oe1.aarch64软件包冲突的软件包为libevent-devel-2.1.11-2.oe1.aarch64。

2. 执行**dnf remove**命令将与待安装软件包冲突的软件包单独卸载。

   ```shell
   # dnf remove libevent-devel-2.1.11-2.oe1.aarch64
   ```

3. 重新进行安装操作。

若为文件冲突，则按如下步骤进行处理（以问题现象中示例的文件冲突为例）：

1. 根据安装过程中的文件冲突报错信息，确定导致文件冲突的软件包名称为containerd-1.2.0-101.oe1.aarch64和docker-engine-18.09.0-100.aarch64。

2. 将不需要安装的软件包名称记录下来，以不需要安装docker-engine-18.09.0-100.aarch64为例。

3. 执行**dnf remove**命令将不需要安装的软件包单独卸载。

   ```shell
   # dnf remove docker-engine-18.09.0-100.aarch64
   ```

4. 重新进行安装操作。

若为缺少软件包，则按如下步骤进行处理（以问题现象中示例的缺少软件包为例）：

1. 根据升级安装过程中的缺少软件包报错信息，确定待升级的软件包名称blivet-data-1:3.1.1-5.noarch及依赖它的软件包名称python2-blivet-1:3.1.1-5.noarch。

2. 执行dnf remove命令将依赖待升级包才能安装的软件包单独卸载或在升级软件包时加上--allowerasing参数。
   * 执行**dnf remove**命令将依赖blivet-data-1:3.1.1-5.noarch软件包才能安装的软件包单独卸载。

     ```shell
     # dnf remove python2-blivet-1:3.1.1-5.noarch
     ```

   * 升级软件包时加上--allowerasing参数。

     ```shell
     # yum update blivet-data-1:3.1.1-5.noarch -y --allowerasing
     ```

3. 重新进行升级操作。

### 安装冲突实例

* 文件冲突

  python3-edk2-devel.noarch 与 build.noarch 因文件名重复存在冲突。

  ```shell
  # yum install python3-edk2-devel.noarch build.noarch
  ...
  Error: Transaction test error:
  file /usr/bin/build conflicts between attempted installs of python3-edk2-devel-202002-3.oe1.noarch and build-20191114-324.4.oe1.noarch
  ```

## **问题6：libiscsi降级失败**

### 问题现象

libiscsi-1.19.0-4 版本及以上降级到 libiscsi-1.19.0-3 及以下版本时失败。

```text
Error:
Problem: problem with installed package libiscsi-utils-1.19.0-4.oe1.x86_64
- package libiscsi-utils-1.19.0-4.oe1.x86_64 requires libiscsi(x86-64) = 1.19.0-4.oe1, but none of the providers can be installed
- cannot install both libiscsi-1.19.0-3.oe1.x86_64 and libiscsi-1.19.0-4.oe1.x86_64
- cannot install both libiscsi-1.19.0-4.oe1.x86_64 and libiscsi-1.19.0-3.oe1.x86_64
- conflicting requests
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
```

### 原因分析

libiscsi-1.19.0-3 之前的版本把 iscsi-xxx 等二进制文件打包进了主包 libiscsi，而这些二进制文件引入了不合理的依赖 CUnit, 为了解决这种不合理的依赖，在 libiscsi-1.19.0-4 版本把这些二进制文件单独拆分出来一个子包 libiscsi-utils，主包弱依赖于子包，产品可以根据自己的需求在做镜像时是否集成该子包；不集成或卸载子包不会影响 libiscsi 主包的功能。
如果系统中安装了 libiscsi-utils 子包，libiscsi-1.19.0-4 及以上版本降级到 libiscsi-1.19.0-3 及以下版本时，由于 libiscsi-1.19.0-3 及以下版本无法提供对应的 libiscsi-utils，因此 libiscsi-utils 不会降级，但 libiscsi-utils 依赖于降级前的 libiscsi 主包，导致依赖问题无法解决，最终导致降级失败。

### 解决方案

执行以下命令，卸载 libiscsi-utils 子包，卸载成功后再进行降级操作。

```text
yum remove libiscsi-utils
```

## **问题7：xfsprogs降级失败**

### 问题现象

xfsprogs-5.6.0-2 及以上版本降级到 xfsprogs-5.6.0-1 及以下版本时失败。

```text
Error:
Problem: problem with installed package xfsprogs-xfs_scrub-5.6.0-2.oe1.x86_64
- package xfsprogs-xfs_scrub-5.6.0-2.oe1.x86_64 requires xfsprogs = 5.6.0-2.oe1, but none of the providers can be installed
- cannot install both xfsprogs-5.6.0-1.oe1.x86_64 and xfsprogs-5.6.0-2.oe1.x86_64
- cannot install both xfsprogs-5.6.0-2.oe1.x86_64 and xfsprogs-5.6.0-1.oe1.x86_64
- conflicting requests
```

### 原因分析

在 xfsprogs-5.6.0-2 版本中，为了减少 xfsprogs 主包的不合理依赖，同时将实验性质的命令从主包中分来，我们将 xfs\_scrub\* 命令拆分到单独的 xfsprogs-xfs\_scrub 子包中。而 xfsprogs 主包弱依赖于 xfsprogs-xfs\_scrub 子包，所以产品可以根据自己的需求在做镜像时是否集成该子包，或者是否卸载该子包。不集成或卸载该子包不会影响 xfsprogs 主包功能。

如果系统中安装了 xfsprogs-xfs\_scrub 子包，从 xfsprogs-5.6.0-2 及以上版本降级到 xfsprogs-5.6.0-1 及以下版本时，由于 xfsprogs-5.6.0-1 及以下版本无法提供对应的 xfsprogs-xfs\_scrub，因此 xfsprogs-xfs\_scrub 不会降级，但 xfsprogs-xfs\_scrub 依赖于降级前的 xfsprogs 主包，导致依赖问题无法解决，最终导致降级失败。

### 解决方案

执行以下命令，卸载 xfsprogs-xfs\_scrub 子包，卸载成功后再进行降级操作。

```shell
# yum remove xfsprogs-xfs_scrub
```

## **问题8：elfutils降级失败**

### 问题现象

elfutils降级缺少依赖，导致无法降级。

![](figures/1665628542704.png)

### 原因分析

22.03-LTS、22.03-LTS-Next分支：elfutils-0.185-12

master分支：elfutils-0.187-7

20.03-LTS-SP1分支：elfutils-0.180-9

如上版本，elfutils主包提供的eu-objdump、eu-readelf、eu-nm命令拆分到elfutils-extra子包中。当系统已安装elfutils-extra，且elfutils进行降级时，由于低版本（如上分支版本）无法提供对应的elfutils-extra包，因此elfutils-extra子包不会降级（elfutils-extra依赖于降级前的elfutils包），导致依赖问题无法解决，最终elfutils降级失败。

### 解决方案

执行以下命令，先卸载elfutils-extra包，再进行降级操作。

```shell
# yum remove -y elfutils-extra
```

## **问题9：cpython/Lib发现CVE-2019-9674:Zip炸弹漏洞**

### 问题现象

Python 3.7.2 及以下版本中的 Lib/zipfile.py 允许远程攻击者通过 zip 炸弹制造拒绝服务请求，从而导致资源消耗过大。

### 原因分析

远程攻击者通过 zip 炸弹导致拒绝服务，影响目标系统业务甚至达到使系统崩溃的结果。zip 炸弹就是一个高压缩比的 zip 文件，它本身可能只有几M或几十M的大小，但是解压缩之后会产生巨大的数据量，产生巨大的资源消耗。

### 解决方案

在 zipfile 文档中添加告警信息： <https://github.com/python/cpython/blob/3.7/Doc/library/zipfile.rst。>

## **问题10：不合理使用glibc正则表达式引起ReDoS攻击**

### 问题现象

使用glibc的regcomp/regexec接口编程，或者grep/sed等应用glibc正则表达式的shell命令，不合理的正则表达式或输入会造成ReDoS攻击（CVE-2019-9192/CVE-2018-28796）。
典型正则表达式pattern为“反向引用”（\1表示）与“\*”（匹配零次或多次）、“+”（匹配一次或多次）、“{m,n}”（最小匹配m次，最多匹配n次）的组合，或者配合超长字符串输入，示例如下：

```shell
# echo D | grep -E "$(printf '(\0|)(\\1\\1)*')"Segmentation fault (core dumped)
# grep -E "$(printf '(|)(\\1\\1)*')"
Segmentation fault (core dumped)
# echo A | sed '/\(\)\(\1\1\)*/p'
Segmentation fault (core dumped)
# time python -c 'print "a"*40000' | grep -E "a{1,32767}"
Segmentation fault (core dumped)
# time python -c 'print "a"*40900' | grep -E "(a)\\1"
Segmentation fault (core dumped)
```

### 原因分析

使用正则表达式的进程coredump。具体原因为glibc正则表达式的实现为NFA/DFA混合算法，内部原理是使用贪婪算法进行递归查找，目的是尽可能匹配更多的字符串，贪婪算法在处理递归正则表达式时会导致ReDoS。

### 解决方案

1. 需要对用户做严格的权限控制，减少攻击面。

2. 用户需保证正则表达式的正确性，不输入无效正则表达式，或者超长字符串配合正则的“引用” “\*”等容易触发无限递归的组合。

   ```shell
   # ()(\1\1)*
   # "a"*400000
   ```

3. 用户程序在检测到进程异常之后，通过重启进程等手段恢复业务，提升程序的可靠性。

## **问题11：安装卸载httpd-devel和apr-util-devel软件包，其中的依赖包gdbm-devel安装、卸载有报错**

### 问题现象

1. gdbm-devel-1.18.1-1包安装、卸载有报错；
2. 问题1修复后，gdbm和gdbm-devel包更新到1.18.1-2版本，但在安装httpd-devel、apr-util-devel等包（依赖关系中有gdbm-devel软件包）时，默认安装的gdbm-devel还是1.18.1-1旧版本，导致问题报错依然存在。

### 原因分析

1. gdbm-devel-1.18.1-1包中缺少提供info信息的help软件包，导致单独安装gdbm-devel并不能将help包引入进来，所以出现了如下告警信息。

   ```text
   install-info: 没有那个文件或目录 for /usr/share/info/gdbm.info.gz
   ```

2. 由于系统默认安装的gdbm主包是1.18.1-1版本，而没有安装gdbm-devel包。依赖gdbm-devel包的相关软件包在安装gdbm-devel包的过程中，仍会匹配gdbm的主包版本，故而依然安装了gdbm-devel的旧版本1.18.1-1,导致警告信息依然存在。

### 解决方案

1. 单包升级gdbm，安装使用gdbm-1.18.1-2版本相关软件包后，告警信息消失；
2. 在单包升级gdbm后，再进行安装依赖的gdbm-devel软件包安装，让其依赖高版本gdbm软件包，告警信息消失。

## **问题12：系统reboot后，执行yum/dnf等命令报错，提示rpmdb error**

### 问题现象

1. reboot系统，重启后，执行rpm相关命令(yum/dnf)提示：\
   error: db5 error(-30973) from dbenv->open: BDB0087 DB\_RUNRECOVERY: Fatal error, run database recovery\
   error: cannot open Packages index using db5 -  (-30973)\
   error: cannot open Packages database in /var/lib/rpm\
   Error: Error: rpmdb open failed

### 原因分析

1. 执行安装升级动作过程中，会对/var/lib/rpm/\_\_db.00\*文件进行读写操作，如果在运行中出现强制下电、磁盘空间满或者 ‘kill -9’ 等异常中断操作，会导致对应\_db文件损坏，后续执行rpm相关命令（dnf/yum）会发生报错

### 解决方案

步骤1 执行`kill -9`停止所有正在运行的rpm命令。
步骤2 执行`rm -rf /var/lib/rpm/__db.00*`删除所有db.00的文件。
步骤3 执行`rpmdb --rebuilddb`命令，重建rpm db后即可。

## **问题13：执行 rpmrebuild -d /home/test filesystem对filesystem包rebuild时，rebuild失败**

### 问题现象

执行 rpmrebuild --comment-missing=y --keep-perm -b -d /home/test filesystem-3.16-3.oe1.aarch64对filesystem包rebuild时，rebuild失败.
/usr/lib/rpmrebuild/rpmrebuild.sh:Error:(RpmBuild) Package 'filesystem-3.16-3.oe1.aarch64' build failed.
/usr/lib/rpmrebuild/rpmrebuild.sh:Error: RpmBuild

### 原因分析

软件包在%pretrans -p阶段创建目录，并在%ghost阶段对该目录进行修饰，如果用户在该目录下创建目录或文件，执行rpmrebuild对该包进行打包，发现创建的目录或文件也会打包到该包中。

上述问题的根本原因是因为filesystem在%pretrans阶段创建了/proc目录，并在%ghost阶段对该目录进行了修饰，但是该目录在系统运行时会动态的创建一些微量进程，这些进程非目录也非文件，在执行rpmrebuild的时无法对这些进程进行打包，所以rebuild失败。

### 解决方案

暂时不使用rpmrebuild命令对filesystem进行rebuild。

## **问题14：带参数f执行modprobe或insmod报错**

### 问题现象

执行`modprobe -f <mod>`或`insmod -f <mod>.ko.xz`报错，比如`insmod -f xfs.ko.xz`报错：`modprobe: ERROR: could not insert 'xfs': Key was rejected by service`。

迄今为止(2022.09.20, kmod v30)，该问题尚未被kmod社区修复。Linux v5.17 [b1ae6dc](https://github.com/torvalds/linux/commit/b1ae6dc41eaaa98bb75671e0f3665bfda248c3e7)添加了对压缩内核模块的支持，kmod尚未支持该特性。

### 原因分析

对于未经压缩的ko，`{modprobe, insmod}`使用`finit_module()`系统调用，而对于压缩的ko，解压由kmod完成，使用`init_module()`系统调用。系统调用`init_module()`无法传入ignore check的flag，导致内核执行路径中`mod_verify_sig()`始终进入，并且`{modprobe, insmod} -f`参数会更改ko有关的校验信息，导致`mod_verify_sig()`校验失败。

### 解决方案

对压缩了的ko不使用`{insmod, modprobe} -f`。

---

---
url: /zh/docs/common/contribute/templates/feature_user_guide/maintain.md
---
# 维护特性

## 升级软件

介绍如何升级软件。

## 卸载软件

介绍如何卸载软件。

## 查询命令

介绍使用特性时会用到的查询命令。

## 收集日志

介绍使用特性时如何收集日志。

---

---
url: /zh/docs/common/faq/virtualization/virt_faq.md
---
# 虚拟化常见问题与解决办法

## **问题1：使用libcareplus工具制作的qemu热补丁无法加载**

原因：qemu版本和热补丁版本不一致。

解决方法：下载qemu对应版本的源码，同时需保持制作热补丁的环境和制作qemu包环境一致，buildID可作为二者是否一致的判定标准。因用户无qemu版本的制作环境，故可以 **自行编包并安装** ，使用自编包中的/usr/libexec/qemu-kvm的buildID。

## **问题2：使用libcareplus工具制作的热补丁已加载但未生效**

原因：不支持死循环、不退出、递归的函数，不支持对初始化函数、inline 函数以及小于5字节的短函数。

解决方法：查看补丁所在函数是否在约束限制中。

## **问题3：使用kvmtop工具第一次显示的结果为间隔0.05秒的两次采样计算得到的结果，波动较大**

此为开源top框架缺陷导致，暂无解决方案。

---

---
url: /zh/docs/common/contribute/templates/feature_user_guide/build.md
---
# 软件编译

## 编译流程

提供编译流程。

## 配置编译环境

说明编译环境的软、硬件要求。

## 编译代码

描述如何编译软件代码。

---

---
url: /zh/docs/common/faq/server/migration_faqs.md
---
# 迁移常见问题与解决方法

## 迁移过程中可能需要用到的网站链接

openEuler迁移专区:

* x2openEuler官方文档<https://docs.openeuler.org/zh/docs/20.03_LTS_SP1/docs/x2openEuler/x2openEuler.html>

* x2openEuler工具下载<https://repo.oepkgs.net/openEuler/rpm/openEuler-22.03-LTS/contrib/x2openEuler/noarch/Packages>

* x2openEuler需要用到的openEuler的源<https://forum.openeuler.org/t/topic/768>

* x2openEuler兼容性分析比对数据库centos7/8—>openEuler22.03-LTS下载地址<https://repo.oepkgs.net/openEuler/rpm/openEuler-22.03-LTS/contrib/x2openEuler/noarch/Packages>

* x2openEuler迁移学习<https://hiss.shixizhi.huawei.com/center/privateCenter.htm?schoolId=1643780836745113602&type=studyCenter_LearningTask&sxz-lang=zh_CN&mapDetail=3&mapDetailId=1663089385654558721&freedomMapClickItemId=1681185900091936769&freedomMapClickItemId=1686217461215608834>

## 为什么在同样分配的物理内存下，openEuler 显示的可用内存比 CentOS 少？

这个问题是两个操作系统在分配给 crashkernel（内核崩溃时使用的内存区域）的内存大小不同导致的。在实验中，给两个操作系统都分配了 4G 的内存，但是 CentOS 可用内存为 3.7G，而 openEuler 可用内存只有 3.3G。通过查看系统的 dmesg 日志，发现在 CentOS 中为 crashkernel 预留的内存是 161MB，而在 openEuler 中预留的内存是 512MB。将 openEuler 的 crashkernel 内存预留修改为 256MB 后，可用内存与 CentOS 相同，从而证明了是由于 crashkernel 的内存预留差异导致的可用内存差异。

解决方法：在 openEuler 的 grub 配置文件（/boot/grub2/grub.cfg）中将 crashkernel 的预留内存从 512MB 修改为 256MB，可以解决这个问题。

## 如果在迁移软件包时遇到宏无法解析的问题，该如何解决？

这通常发生在从CentOS或Fedora迁移到其他系统时，因为不同系统的宏定义可能不同。解决方法有两种：一是查询宏的具体含义，并在spec文件中用其实际值替换宏；二是将提供宏定义的macros软件包引入到仓库中，并在BuildRequires中添加，以确保宏能够正确解析。

## 在进行虚拟机热迁移时，应该如何准备环境并检查迁移前的必要条件？

进行虚拟机热迁移前，需要准备两个物理机（源端和目的端）并进行一系列的检查来确保迁移可以顺利进行。这些检查包括：

权限检查：确保当前用户有执行热迁移的权限。
网络检查：检查源端和目的端主机之间的网络是否互通，并保证两个主机在相同网段。
存储资源检查：确认两端是否可以访问相同的存储资源，并确保目的端主机有足够的CPU、内存和存储资源。
CPU资源检查：确认两个主机的CPU资源情况。
内存检查：核实两个主机的内存情况。
存储检查：检查两个主机的存储配置。
虚拟机状态检查：确认被迁移的虚拟机处于运行状态。
此外，可根据需要设置热迁移参数，如最大停机时间和迁移过程中的最大带宽，以及确定存储方式是共享存储还是非共享存储。在非共享存储的情况下，可能还需要进行额外的配置，如通过NFS设置共享存储。

## 什么是虚拟机热迁移，它与虚拟机冷迁移有什么区别？

虚拟机热迁移是一种技术，它允许在不关闭虚拟机的情况下，将整个虚拟机的运行状态（包括内存中的数据和磁盘上的数据）完整地迁移到另一台物理服务器上。这种迁移过程对用户来说是透明的，即用户不会感受到任何服务中断或性能下降。热迁移通常用于硬件维护、升级，或是负载均衡等场景，确保关键业务连续性和服务的高可用性。

相比之下，虚拟机冷迁移（也称为静态迁移）涉及到在迁移前关闭虚拟机。这意味着在迁移过程中，该虚拟机上的服务是不可用的。冷迁移适用于非实时或可容忍停机时间的场景，例如批量处理作业或非关键业务的迁移。

总结两者的主要区别：

1. 热迁移允许在不停机的情况下迁移虚拟机，保证了业务的连续性；而冷迁移需要在迁移前关闭虚拟机，导致服务暂时不可用。
2. 热迁移对用户透明，用户体验不会受到影响；冷迁移则可能导致服务中断。
3. 热迁移技术复杂度较高，因为它需要同步迁移内存中的数据；而冷迁移相对简单，因为只涉及静态数据的迁移。

## 如何将SQL Server数据从Windows迁移到openEuler？

首先，在Windows上备份SQL Server数据库。可以使用SQL Server Management Studio (SSMS) 或使用SQL语句的方法进行备份。备份完成后，使用scp或其他方法将备份文件传输到openEuler系统。在openEuler上，创建一个新的备份目录并将备份文件移动到该目录。使用sqlcmd工具，执行SQL语句来还原数据库。如果数据库包含辅助文件，需要在RESTORE DATABASE命令中为这些文件添加MOVE子句。最后，通过列举所有数据库来验证数据迁移是否成功。

## 在进行CentOS到openEuler的操作系统迁移时，为什么需要考虑硬件兼容性检测？

硬件兼容性检测在CentOS到openEuler的迁移中至关重要，因为这不仅涉及到操作系统的更换，还包括对操作系统上运行的应用软件和业务系统的替代、适配、迁移和重构。确保硬件兼容性可以保障迁移过程中的系统稳定性和业务的连续性，防止迁移后出现硬件不兼容导致的应用故障或性能下降。

## openEuler社区提供的迁移工具x2openEuler具备哪些功能，它是如何应用于迁移评估？

openEuler社区提供的迁移工具x2openEuler主要用于迁移评估，具备以下功能：

1. 软件评估：通过扫描依赖的软件包清单信息，对各类应用（如rpm, tar, zip, gzip, jar, py, pyc, sh, bin等）进行评估，并生成HTML格式的评估报告。
2. 配置收集与评估：支持收集用户环境数据并生成JSON格式文件，包括硬件配置、配置接口、内核选项配置参数、系统配置参数（sysctl/proc/sys）、环境变量、服务、进程、端口、命令接口、系统调用项和设备驱动接口等信息，并完成配置信息分析评估。
3. 硬件评估：评估运行环境的整机和整机板卡（如RAID, NIC, FC, IB, GPU, SSD, TPM等）是否在openEuler的兼容性清单中。这些功能帮助用户在迁移之前识别潜在的兼容性问题，为顺利迁移提供支持。

---

---
url: /zh/docs/common/contribute/doc_tools_static_check.md
---
# 静态检查

## Markdown Lint

基于 `markdownlint v0.12.0` 版本实现，帮助用户规范 Markdown 文档格式。

![Markdown Lint](public_sys_resources/lint-markdown.gif)

### 功能介绍

* 自动检测 Markdown 文件中的[格式问题](./markdownlint_rules.md)，如标题格式、列表缩进、空行等；
* 可通过配置项灵活启用或禁用 lint 功能，并支持自定义 lint 规则，满足不同团队或个人的文档规范需求。
* 支持一键修复功能，帮助用户一键修复所有 markdownlint 问题。

### 使用方法

1. 安装并启用本插件，打开 Markdown 文件（`.md`），插件会自动对文件内容进行 lint 检查；
2. 检查结果会以警告（Warning）的形式在编辑器中高亮显示，可在底部问题面板，或将光标悬停在警告标记上，查看详细的规则说明和建议；
3. 可通过 VSCode 提供的 Quick Fix（快速修复）功能，点击灯泡图标或按下快捷键（通常为 `Cmd+.` 或 `Ctrl+.`），一键修复所有 markdownlint 问题。

### 配置说明

插件支持以下配置项（可在 VSCode 设置中搜索 `docTools.markdownlint`或通过`settings.json`进行配置）：

* `docTools.markdownlint`
  * 类型：`boolean`
  * 说明：是否启用 Markdown lint 功能
  * 默认：`true`
* `docTools.markdownlint.config`
  * 类型：`object`
  * 说明：自定义 markdownlint 配置对象。若未设置，则使用插件内置的默认规则。

#### 配置示例

```json
{
  "docTools.markdownlint": true, // 是否开启功能
  "docTools.markdownlint.config": {
    "MD013": false, // 禁用行长度限制
    "MD041": true // 启用标题必须为一级标题
  }
}
```

## Tag Closed Check

检查 Markdown 文件中的 HTML 标签是否正确闭合，帮助用户避免因标签未闭合导致的渲染或语法错误。

![tag closed check](public_sys_resources/check-tag-closed.gif)

### 功能介绍

* 自动检测 Markdown 文件中的 [HTML 标签闭合问题](./ci_rules.md#tag-closed-check)；
* 支持快速修复功能，帮助用户一键修正标签闭合和转义问题；
* 支持通过配置项灵活启用或禁用该功能。

### 使用方法

1. 安装并启用本插件，打开 Markdown 文件，自动检查文件内容有效性；
2. 检查结果会以错误（Error）的形式在编辑器中高亮显示，可在底部问题面板，或将光标悬停在错误标记处，查看错误详情；
3. 可通过 VSCode 提供的 Quick Fix（快速修复）功能，点击灯泡图标或按下快捷键（通常为 `Cmd+.` 或 `Ctrl+.`），一键修复标签问题。

### 快速修复说明

* \字符替换：适用于非 Html 标签嵌套的情况，会在对应的 Html 标签前加上 `\` 转义字符；
* <和>字符替换：适用于 Html 标签嵌套的情况，会将 `<` 和 `>` 替换为 `&lt;` 和 `&gt;`；
* 闭合标签：自动为未闭合的标签补全闭合部分。

### 注意事项

* 对于 Html 标签嵌套的情况，如`<table><errorLabel></table>`，请使用`&lt;和&gt;字符替换`方式进行修复，修复后为`<table>&lt;errorLabel&gt;</table>`。

### 配置说明

插件支持以下配置项（可在 VSCode 设置中搜索`docTools.check.tagClosed`或通过`settings.json`进行配置）：

* `docTools.check.tagClosed`
  * 类型：`boolean`
  * 说明：是否启用 HTML 标签闭合检查

#### 配置示例

```json
{
  "docTools.check.tagClosed": true
}
```

## Link Validity Check

检查 Markdown 文档中的链接有效性，帮助用户及时发现失效或错误的链接，提升文档质量。

![Link Validity Check](public_sys_resources/check-link-validity.gif)

### 功能介绍

* 链接识别\
  支持自动识别文档中以下三种格式的链接：
  1. `[文本](链接)`形式的标准 Markdown 链接；
  2. `<http://example.com>`形式的裸链接；
  3. `<a href="链接">`形式的 HTML 链接。
* 链接检查
  1. 支持 HTTP/HTTPS 链接检测，自动忽略链接中的锚点部分（如`#section`），仅校验主链接地址；
  2. 支持对相对路径的文件链接，进行本地文件存在性检查，并可检查[锚点的有效性](./doc_tools_functions.md#生成链接锚点并复制)。
* 白名单机制\
  支持配置 HTTP/HTTPS 链接检查白名单，避免对特定可信链接进行报错：
  * 默认从远程配置文件中获取：
    1. 内网地址，如`http://localhost`，`http://192.168.1.60`；
    2. 邮件协议，文件协议，FTP文件协议链接如`ftp://`，`file://`；
    3. 示例链接，如`https://repo.openeuler.org/openEuler-{version}/OS/x86_64/`。
  * 支持按需自定义添加新的白名单。
* 灵活配置
  1. 支持通过配置项灵活启用或禁用该功能；
  2. 支持通过配置项启用“仅在链接响应为404（无法访问）时提示”的模式。

### 使用方法

1. 安装并启用插件后，打开任意 Markdown 文件（`.md`），自动检测所有链接的有效性；
2. 无效链接会以错误（Error）的形式在编辑器中高亮显示，可在底部问题面板中，或将光标悬停在警告标记处，查看错误详情；
3. 访问超时的链接会以警告（Warning）的形式在编辑器中高亮显示，可在底部问题面板中，或将光标悬停在警告标记处，查看错误详情。

### 注意事项

* 插件仅检测链接的格式和可达性，不保证目标内容的正确性；
* 某些私有或受限网络下的链接，可能因网络原因被误判为无效；
* 对于本地文件链接，需确保路径正确且文件存在。

### 配置说明

插件支持以下配置项（可在 VSCode 设置中搜索 `docTools.check.linkValidity`或通过`settings.json`进行配置）：

* `docTools.check.linkValidity.enable`
  * 类型：`boolean`
  * 说明：是否启用链接有效性检查
  * 默认：`true`
* `docTools.check.linkValidity.only404.enable`
  * 类型：`boolean`
  * 说明：启用链接有效性检查只在链接无法访问时提示（可减少一些误报）
  * 默认：`true`
* `docTools.check.url.whiteList`
  * 类型：`array`
  * 说明：检测链接白名单（添加后忽略对该链接的检查）
  * 默认：`[]`

#### 配置示例

```json
{
  "docTools.check.linkValidity.enable": true,
  "docTools.check.linkValidity.only404.enable": true,
  "docTools.check.url.whiteList": []
}
```

## Resource Existence Check

检查 Markdown 文件中的资源链接（如图片、视频等）是否存在，帮助用户及时发现无效或丢失的资源引用。

![Resource Existence Check](public_sys_resources/resource_check.gif)

### 功能介绍

* 自动检测 Markdown 文件中的图片、视频等[资源链接是否有效](./ci_rules.md#resource-existence-check)，包括 Markdown 语法和 HTML 标签（如 `<img>`、`<image>`、`<video>`）中的资源路径；
* 支持多种资源引用方式，全面覆盖常见的资源链接格式；
* 支持通过配置项灵活启用或禁用该功能。

### 使用方法

1. 安装并启用本插件，打开 Markdown 文件（`.md`），自动检查文档中的资源链接有效性；
2. 检查结果会以警告（Warning）的形式在编辑器中高亮显示，可在“问题”面板，或将光标悬停在警告标记处，查看无效资源的具体路径和提示信息。

### 配置说明

插件支持以下配置项（可在 VSCode 设置中搜索 `docTools.check.resourceExistence` 或通过 `settings.json` 进行配置）：

* `docTools.check.resourceExistence`
  * 类型：`boolean`
  * 说明：是否启用资源存在性检查
  * 默认：`true`

#### 配置示例

```json
{
  "docTools.check.resourceExistence": true
}
```

## Toc Check

检查目录文件`_toc.yaml`中链接的有效性，同时检查目录下未被`_toc.yaml`引用的文档，帮助用户快速定位目录结构问题。

![toc Check](public_sys_resources/toc_check.gif)

### 功能介绍

* [`toc.yaml`链接有效性](./ci_rules.md#toc-check)：自动解析`_toc.yaml`，递归检查所有链接（如 href/upstream）对应的文档是否存在；
* [目录覆盖完整性](./ci_rules.md#toc-check)：自动检测当前打开的 Markdown 文档是否已被加入到项目的`_toc.yaml`目录结构中。

### 使用方法

1. 安装并启用本插件，打开任意目录文件`toc.yaml`，自动检查文件内容有效性，并检查文件是否被当前/上级目录的`_toc.yaml`引用；
2. `toc.yaml`链接有效性检查结果会以错误（Error）的形式在编辑器中高亮显示，可在“问题”面板，或将光标悬停在错误标记处，查看错误详情。
3. 若文档未加入目录文件，编辑器右下角会弹出提示信息，提醒用户将该文件加入`_toc.yaml`；

### 配置说明

插件支持以下配置项（可在 VSCode 设置中搜索 `docTools.check.toc` 或通过 `settings.json` 进行配置）：

* `docTools.check.toc`
  * 类型：`boolean`
  * 说明：是否启用 TOC 校验功能
  * 默认：`true`

#### 配置示例

```json
{
  "docTools.check.toc": true
}
```

## CodeSpell Check

检查 Markdown 文档中的单词拼写错误，帮助用户及时发现并修正。

### 功能介绍

* 自动扫描 Markdown 文档中的英文单词，检测拼写错误；
* 支持单词修正功能，用户可以根据提示选择正确的单词进行修正；
* 支持自定义忽略单词列表，用户可将特定单词加入白名单以避免误报；
* 支持通过配置项灵活启用或禁用该功能。

### 使用方法

1. 安装并启用本插件后，打开任意 Markdown 文件，自动检测文档中的单词拼写错误；
2. 检查结果会以提示（Info）的形式在编辑器中高亮显示，可在底部问题面板，或将光标悬停在错误标记处，查看错误详情。
3. 通过 Quick Fix（快速修复）功能，可选择候选单词进行修改，或将误报单词添加至白名单。

### 配置说明

插件支持以下配置项（可在 VSCode 设置中搜索 docTools.check.codespell 或通过 settings.json 进行配置）：

* `docTools.check.codespell.enable`
  * 类型：`boolean`
  * 说明：是否启用单词拼写检查
  * 默认：`true`
* `docTools.check.codespell.whiteList`
  * 类型：`array`
  * 说明：单词白名单（添加后忽略对该单词的检查）
  * 默认：`[]`

### 配置示例

```json
{
  "docTools.check.codespell.enable": true,
  "docTools.check.codespell.whiteList": ["xxx"]
}
```

### 注意事项

* 插件主要检测英文单词拼写，可能不适用于中文等非拉丁文字符；
* 某些专业术语或专有名词可能被误判为拼写错误，建议将其添加到白名单；
* 插件依赖于内置词典进行拼写检查，可能无法涵盖所有专业领域词汇。

## Filename Check

检查文件和目录的命名规范性，以及中英文文档名称一致性，提升文档管理效率。

### 功能介绍

* 支持检查 Markdown 文件名和目录名是否符合规范要求：使用小写英文字母并用下划线连接单词；
* 对于不符合规范的文件名，会在打开 markdown 文件时，在右下角弹出提示；
* 支持检查中英文文档文件名一致性，确保中英文文档一一对应；
* 当中英文文档文件名不一致时，会在打开 markdown 文件时，在右下角弹出提示；
* 支持通过配置项灵活启用或禁用相关功能。

### 使用方法

1. 安装并启用本插件，在设置中勾选启用相关功能；
2. 打开 Markdown 文件后，若文件名不符合规范或中英文文档文件名不一致，右下角会弹出提示。

### 配置说明

插件支持以下配置项（可在 VSCode 设置中搜索 `docTools.check.name` 或通过 `settings.json` 进行配置）：

* `docTools.check.name.enable`
  * 类型：`boolean`
  * 说明：是否启用文件/目录命名规范检查
  * 默认：`false`
* `docTools.check.name.whiteList`
  * 类型：`array`
  * 说明：文件/目录命名规范白名单（添加后忽略对该名称的检查）
  * 默认：`[]`
* `docTools.check.nameConsistency.enable`
  * 类型：`boolean`
  * 说明：是否启用中英文文档名称一致性检查
  * 默认：`false`

#### 配置示例

```json
{
  "docTools.check.name.enable": true,
  "docTools.check.name.whiteList": ["README.md", "README_en.md"],
  "docTools.check.nameConsistency.enable": true
}

```

### 注意事项

* 检查默认关闭，如需要相关功能请到设置中开启。

## Punctuation Check

检查 Markdown 文档中的标点符号使用规范，帮助用户发现并纠正标点符号错误，提升文档质量。

### 功能介绍

* 自动扫描 Markdown 文档中的文本内容，检测标点符号使用是否规范，包括以下检查项：
  1. 中文使用了英文符号：中文字段（代码样例等英文段落除外）使用英文标点符号；
  2. 多余标点符号：出现连续两个及以上标点符号；
  3. 多余空格：中文标点符号前后有空格；
  4. 多余空格：中文文字之间有空格；
  5. 成对标点符号不完整；
  6. 不符合链接规范：如手册类外链接的链接文字中缺少书名号；
  7. 多余空格：英文标点符号前有空格，或标点后有多于一个空格；
  8. 英文使用了中符号：英文字段使用中文标点符号。
* 支持通过配置项灵活启用或禁用相关功能。

### 使用方法

1. 安装并启用插件后，在设置中勾选启用相关功能；
2. 打开任意 Markdown 文件，插件将自动检测标点符号使用情况；
3. 检查结果会以提示（Info）的形式在编辑器中高亮显示，可在底部问题面板，或将光标悬停在错误标记处，查看错误详情。

### 配置说明

插件支持以下配置项（可在 VSCode 设置中搜索 `docTools.check.punctuation` 或通过 `settings.json` 进行配置）：

* `docTools.check.punctuationBlankSpace.enable`
  * 类型：`boolean`
  * 说明：启用中文标点符号前后无空格检查
  * 默认：`false`
* `docTools.check.punctuationMixing.enable`
  * 类型：`boolean`
  * 说明：启用中英文标点符号混用检查
  * 默认：`false`
* `docTools.check.punctuationConsecutive.enable`
  * 类型：`boolean`
  * 说明：启用连续标点符号检查
  * 默认：`false`
* `docTools.check.punctuationManualLink.enable`
  * 类型：`boolean`
  * 说明：启用手册/指南链接是否被书名号包裹检查
  * 默认：`false`

#### 配置示例

```json
{
  "docTools.check.punctuationBlankSpace.enable": true,
  "docTools.check.punctuationMixing.enable": true,
  "docTools.check.punctuationConsecutive.enable": true,
  "docTools.check.punctuationManualLink.enable": true
}
```

### 注意事项

* 检查默认关闭，如需要相关功能请到设置中开启；
* 因语句含义复杂，存在误报的情况。

---

---
url: /zh/docs/common/contribute/doc_tools_functions.md
---
# 高级功能

## 文档预览

![Doc Preview](public_sys_resources/preview.gif)

### 功能介绍

* 支持文档实时预览功能，可完整呈现单本手册内容，可将 Markdown 语法渲染为格式化文本，同时能够解析`_toc.yaml`文件，生成目录导航；
* 支持保存后刷新预览页面；
* 支持从预览页面打开对应的 Markdown 文件或 \_toc.yaml 文件。

### 使用方法

1. 安装并启用本插件，打开 Markdown 文件；
2. 点击编辑器右上角的**D**字图标以打开预览页面；
3. 保存对 Markdown 文件的修改后，预览页面将自动刷新；
4. 修改关联的 \_toc.yaml 文件后，预览页面将自动刷新。

## 目录生成

![toc generation](public_sys_resources/toc_generation.gif)

### 功能介绍

* 自动生成目录：根据实际存在的 Markdown 文件，自动生成 \_toc.yaml，避免手动维护目录结构；
* 同步标题：自动读取每个 Markdown 文件的一级标题（# ），作为目录项的 label；
* 去除无效项：如果 \_toc.yaml 中的某些 href 指向的文件不存在，会自动移除这些无效项。

### 使用方法

1. 安装并启用本插件；
2. 右键点击目标目录，选择**Doc Tools** > **生成手册 \_toc.yaml**；
3. 按需调整 label 值和章节顺序。

### 注意事项

* 只有带有一级标题（# 标题）的 Markdown 文件才会被收录。

## 批量检查链接可访问性

### 功能介绍

* 检测选中目录下的 Markdown 文档及 \_toc.yaml 文件中的所有链接是否可访问；
* 以页面形式展示检测结果。

### 使用方法

1. 安装并启用本插件；
2. 右键点击目标目录，选择**Doc Tools** > **检测链接可访问性**；
3. 在打开的页面中按需勾选配置，点击**开始检查**；
4. 根据检查结果，修复失效链接或将误报链接加入白名单。

### 注意事项

* 私有或受限网络下的链接可能因网络原因被误判为无效，请以实际访问为准。

## 批量检查文件命名规范

### 功能介绍

* 批量检查选中目录下所有文件和子目录的命名规范：使用小写英文字母并以下划线连接单词；
* 以页面形式展示检测结果，方便用户处理命名不规范的文件。

### 使用方法

1. 安装并启用本插件；
2. 右键点击目标目录，选择**Doc Tools** > **检查目录、文件是否符合命名规范**；
3. 在打开的页面中查看检查结果，根据提示信息修改不符合规范的文件名或目录名。

### 注意事项

* 批量检查将遍历选中目录下的所有文件和子目录，可能需要一些处理时间；
* 修改文件名或目录名后，请同步更改其他文档中的相关链接。

## 检查中英文文档名称一致性

### 功能介绍

* 批量检查选中目录下中英文文档的文件名一致性；
* 通过页面展示检查结果，清晰标识缺少对应语言版本的文档。

### 使用方法

1. 安装并启用本插件；
2. 右键点击目标目录，选择**Doc Tools** > **检查中英文文档名称一致性**；
3. 在打开的页面中查看检查结果，根据提示信息补充缺失的文档，或修正不一致的文件名。

### 注意事项

* 批量检查会遍历选中目录下的所有 Markdown 文件，可能需要一些处理时间；
* 修改文件名后请同步更改其他文档中的相关链接。

## 生成链接锚点并复制

### 功能介绍

* 将选中的文本生成锚点并复制到剪贴板，方便后续使用；
* 生成的锚点遵循以下规范：字母转换为小写，空格替换为中划线，并移除特殊符号（如`!`，`*`）。

### 使用方法

1. 安装并启用本插件；
2. 打开任意 Markdown 文件（`.md`）；
3. 选中标题文本；
4. 右击编辑区域，选择`Doc Tools` > `生成链接锚点并复制`。

### 注意事项

* 生成的锚点适用于 Vitepress 页面，VSCode 编辑器，以及 AtomGit 仓库的锚点跳转；
* 通常情况下和大多数 Markdown 预览（如 VSCode 编辑器自带的 Markdown 预览）的锚点跳转一致，但在内容有一些符号的情况下存在一些差异，可能导致锚点在非 Vitepress 构建出的页面下不能跳转。

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/diversified_computing/dpu_offload/container_management_plane_direct_connection_aggregation_environment_establishment.md
---
# **1** 容器管理面无感卸载

## **1.1** 简介

容器管理面，即dockerd、containerd、isulad等容器的管理工具，而容器管理面卸载，即是将容器管理面卸载到与容器所在机器（以下称为HOST）之外的另一台机器（以下称为DPU）上运行。

我们使用qtfs将HOST上一些与容器运行相关的目录挂载到DPU上，使得容器管理面工具可以访问到这些目录，为容器准备运行所需要的环境。此外，因为需要挂载远端的proc和sys文件系统，为避免对当前系统运行造成影响，可以创建一个专门的rootfs以作为dockerd、containerd的运行环境（以下称为`/another_rootfs`）。

通过rexec执行容器的拉起、删除等操作，可以将容器管理面和容器分离在不同的两台机器上，对容器进行远程管理。无感卸载的验证可以使用下述两种模式。

## 1.1.1 测试模式

需准备2台物理机（或虚拟机），两台机器之间网络互通。

其中一台作为DPU模拟，另一台作为HOST模拟。在本文档中用DPU和HOST指代这两台服务器。

> \[!NOTE]说明
> 测试模式因为会暴露网络端口且不做连接认证，存在网络安全风险，仅能用于内部测试验证，不要用于实际生产环境。
> 实际生产环境中应使用封闭通信方式，以防止外界连接风险，如下述vsock通信模式。

## 1.1.2 vsock模式

需要DPU和HOST通过virtio提供vsock通信方式。

文档当前仅描述基于测试模式的方法，下面的内容依然默认使用测试模式。如果验证环境支持vsock通信（虚拟化环境或支持vsock的DPU-HOST环境），下述测试步骤仍适用，只需要将测试中的IP网络地址修改为vsock cid即可（相关二进制编译过程也无需使用TEST\_MODE）。

# **2** 环境搭建

## **2.1** qtfs文件系统部署

可参考[qtfs主页](https://atomgit.com/openeuler/dpu-utilities/tree/master/qtfs)。

> \[!NOTE]说明
> 如果使用测试模式，需要在qtfs客户端及服务端ko编译时指定qtfs\_TEST\_MODE=1，vsock模式无需指定。

qtfs建联需要关闭DPU和HOST侧的防火墙，或在防火墙中开放相关网络端口号。

## **2.2** UDSPROXYD服务部署

### 2.2.1 简介

udsproxyd是一个跨主机的unix domain socket代理服务，需要分别部署在HOST和DPU上，在HOST和DPU上的udsproxyd组件是对等的关系，可以实现分布在HOST与DPU上的2个进程之间使用标准uds进行通信，通信进程是无感的，即如果这两个进程在同一主机内通过uds正常通信的功能，拉远到HOST和DPU之间也可以，不需要做代码适配。udsproxyd作为一个跨主机的unix socket服务，可以用`LD_PRELOAD=libudsproxy.so`动态库截获的方式对接使用，也可以通过提前进行udsconnect白名单配置无感应用，白名单配置具体有两种方式，将在后面详述。

### 2.2.2 部署方式

首先，在dpu-utilities工程内编译udsproxyd：

```bash
cd qtfs/ipc

make -j UDS_TEST_MODE=1 && make install
```

> \[!NOTE]说明
> 如果使用vsock模式，编译时不需要设置UDS\_TEST\_MODE

当前最新版本下，qtfs server侧的engine服务已经整合了udsproxyd的能力，所以server侧需要再额外启动udsproxyd。client侧则单独拉起udsproxyd服务：

```bash
nohup /usr/bin/udsproxyd <thread num> <addr> <port> <peer addr> <peer port> 2>&1 &
```

参数解释：

```bash
thread num: 线程数量，目前只支持单线程，填1

addr: 本机使用的ip，如果使用vsock通信模式则为cid

port：本机占用的port

peer addr: udsproxyd对端的ip，如果使用vsock模式则为cid

peer port: 对端port
```

示例：

```bash
nohup /usr/bin/udsproxyd 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 &
```

如果未拉起qtfs的engine服务，想单独测试udsproxyd，则在server端也对等拉起udsproxyd即可：

```bash
nohup /usr/bin/udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 &
```

### 2.2.3 应用方式

#### 2.2.3.1 独立使用udsproxyd服务

需要在使用uds服务的unix socket应用程序的client端进程启动时添加LD\_PRELOAD=libudsproxy.so环境变量，以接管glibc的connect api进行uds对接，或者通过qtcfg增加udsconnect白名单的方式指定系统接管特定目录下的uds连接。

#### 2.2.3.2 无感使用udsproxyd服务

首先为qtfs配置uds服务的白名单，这里说的白名单是unix socket的server端bind的sock文件地址，提供两种方式供选择：

* 通过配置工具qtcfg加载，进入qtfs/qtinfo目录编译工具：

在qtfs的client端执行

```bash
make role=client 
make install
```

在qtfs的server端执行

```bash
make role=server
make install
```

配置工具将会自动安装，然后使用qtcfg命令配置白名单，假设需要增加的白名单为"/var/lib/docker"，输入：

```bash
qtcfg -w udsconnect -x /var/lib/docker
```

查询白名单为：

```bash
qtcfg -w udsconnect -z
```

删除白名单为:

```bash
qtcfg -w udsconnect -y 0
```

删除白名单时，参数为查询白名单时列出来的index序号。

* 通过配置文件增加，这需要在qtfs或qtfs\_server内核模块加载前配置，通过内核模块初始化时读取该文件进行白名单配置。

在/etc/qtfs/whitelist文件中增加如下字段：

```bash
[Udsconnect]
/var/lib/docker
```

> \[!NOTE]说明
> 白名单是为了防止不相干的unix socket链接也进行远程连接产生错误，或者浪费不必要的资源，所以白名单尽量设置得精确一些，比如本文中针对容器场景设置为/var/lib/docker比较好，而直接将/var/lib/或/var/或直接将根目录加入的做法是有较大风险和系统影响的。

## **2.3** REXEC服务部署

### 2.3.1 简介

rexec是一个用c语言开发的远程执行组件，分为rexec client和rexec server。server端为一个常驻服务进程，client端为一个二进制文件，client端被执行后会基于udsproxyd服务与server端建立uds连接，并由server常驻进程在server端拉起指定程序。在容器管理面卸载中，dockerd卸载到DPU上，当它需要在HOST拉起容器业务进程时调用rexec client进行远程拉起。

### 2.3.2 部署方法

#### 2.3.2.1 配置环境变量与白名单

在HOST侧配置rexec server的白名单，将文件whitelist放置在/etc/rexec/目录下并修改权限为只读：

```bash
chmod 400 /etc/rexec/whitelist
```

下载dpu-utilities代码后，进入qtfs/rexec主目录下，执行：`make && make install`即可安装rexec所需全部二进制到/usr/bin目录下，包括了：`rexec、rexec_server`两个二进制可执行文件。

在server端启动rexec\_server服务之前，检查是否存在/var/run/rexec目录，没有则创建：

```bash
mkdir /var/run/rexec
```

rexec服务运行时底层通信使用unix socket，因此跨主机的rexec和rexec\_server通信依赖上述udsproxyd服务，并且需要在udsproxy中添加相关白名单：

```bash
qtcfg -w udsconnect -x /var/run/rexec
```

#### 2.3.2.2 服务方式

server端可以通过两种方式拉起rexec\_server服务。

* 方式1:

配置systemd服务

在/usr/lib/systemd/system/下增加rexec.service文件，内容如下：

[rexec.service](./config/rexec.service)

然后通过systemctl管理rexec服务。

首次配置服务时：

```bash
systemctl daemon-reload
systemctl enable --now rexec
```

后续重启新启动服务：

```bash
systemctl stop rexec
systemctl start rexec
```

* 方式2:

手动后台拉起

```bash
nohup /usr/bin/rexec_server 2>&1 &
```

## **3** 管理面组件改动

### **3.1** dockerd相关改动介绍

对dockerd的改动是基于18.09版本的。

在docker中，暂时绕过了网络部分，管理面卸载依赖容器网络组件做针对性适配，本测试方案暂不涉及。

对docker的改动可以参考[该目录](https://atomgit.com/openeuler/dpu-utilities/tree/master/usecases/transparent-offload/patches/docker)下的patch文件。

### **3.2** containerd相关改动介绍

对于containerd的改动是基于containerd-1.2-rc.1版本的。

对containerd的改动可以参考[该目录](https://atomgit.com/openeuler/dpu-utilities/tree/master/usecases/transparent-offload/patches/containerd)下的patch文件。

# **4** 容器管理面卸载操作指南

> \[!NOTE]说明
>
> 1. 在HOST端和DPU端，都要拉起rexec\_server；
> 2. HOST端拉起rexec\_server，主要是用于DPU创建容器时用rexec拉起containerd-shim；
> 3. DPU拉起rexec\_server，则是为了执行containerd-shim对dockerd和containerd的调用。

## **4.1** 准备dockerd和containerd运行的rootfs

> 注：本步骤仅需在DPU上执行的。

在下面的文档中，我们将这个rootfs称为`/another_rootfs`（具体的目录名称，可以根据自己的需求进行调整）。

### 4.1.1 使用openEuler官方qcow2镜像构造rootfs

建议用openEuler官方提供的qcow2镜像，来准备一个新的rootfs：

#### 4.1.1.1 工具安装

需要用yum安装xz、kpartx、qemu-img

```bash
yum install xz kpartx qemu-img
```

#### 4.1.1.2 下载qcow2镜像

在openEuler官网获取[22.03版本openEuler-x86虚拟机镜像](https://repo.openeuler.org/openEuler-22.03-LTS/virtual_machine_img/x86_64/openEuler-22.03-LTS-x86_64.qcow2.xz)(X86架构)，或者[22.03版本openEuler-arm64虚拟机镜像](https://repo.openeuler.org/openEuler-22.03-LTS/virtual_machine_img/aarch64/openEuler-22.03-LTS-aarch64.qcow2.xz)（ARM架构）。

#### 4.1.1.3 解压qcow2镜像

使用xz -d解压为openEuler-22.03-LTS-\<arch>.qcow2文件, 以x86镜像为例

```bash
xz -d openEuler-22.03-LTS-x86_64.qcow2.xz
```

#### 4.1.1.4 挂载qcow2镜像并拷贝文件

1. 使用`modprobe nbd maxpart=<任意数字>`来加载nbd模块
2. `qemu-nbd -c /dev/nbd0 <虚拟机镜像的路径>`
3. 创建任意文件夹`/random_dir`
4. 执行挂载`mount /dev/nbd0p2 /random_dir`
5. 拷贝文件

```bash
mkdir /another_rootfs
cp -r /random_dir/* /another_rootfs/
```

此时，虚拟机镜像遍已经挂载到当前文件夹中了。

#### 4.1.1.5 qcow2卸载

在准备好rootfs后，需要卸载qcow2文件，此时，需要执行如下指令：

```bash
umount /random_dir
qemu-nbd -d /dev/nbd0
```

### 4.1.2 向another\_rootfs中安装软件

1. 将根目录中的`/etc/resolv.conf`拷贝到`/another_rootfs/etc/resolv.conf`
2. 清空`/another_rootfs/etc/yum.repos.d`中的文件，并将`/etc/yum.repos.d/`中的文件拷贝到`/another_rootfs/etc/yum.repos.d`
3. 使用`yum install <软件包>  --installroot=/another_rootfs`来安装软件包

```bash
yum install --installroot=/another_rootfs iptables
```

## **4.2** 在Host侧拉起qtfs\_server

需要将rexec、containerd-shim、runc、engine拷贝到`/usr/bin`下面，这里要注意权限问题，rexec、engine已提供，docker相关二进制根据上述第三章相关patch编译生成。

### 4.2.1 插入qtfs\_server驱动

创建容器管理面所需要的文件夹，然后插入qtfs\_server.ko，并拉起engine进程。

可以使用[这个脚本](./config/server_start.sh)来执行此操作，如果执行错误，可能需要dos2unix来将此脚本的格式转换（如下所有脚本皆同理）。

> 注意，使用实际qtfs路径替换脚本中的模块及二进制路径。

此外在HOST端，还需要创建执行rexec指令的脚本`/usr/bin/dockerd`以及`/usr/bin/containerd`：

/usr/bin/dockerd：

```bash
#!/bin/bash
rexec /usr/bin/dockerd $*
```

/usr/bin/containerd：

```bash
#!/bin/bash
exec /usr/bin/containerd $*
```

在创建完成后，需要用chmod为这两个脚本赋予执行权限

```bash
chmod +x /usr/bin/containerd
chmod +x /usr/bin/dockerd
```

## **4.3** 挂载Host上依赖目录至DPU

### 4.3.1 安装软件包

#### 4.3.2.1 在根目录的安装

DPU根目录中（another\_rootfs之外）：安装iptables和libtool、libcgroup、tar,可以通过yum直接直接安装。

```bash
yum install iptables libtool libcgroup tar
```

也可以下载其所有的依赖包之后，用rpm指令安装，iptables以及libtool的包及依赖包链接如下：[iptables](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/iptables-1.8.7-5.oe2203.x86_64.rpm), [libtool](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/libtool-2.4.6-34.oe2203.x86_64.rpm), [emacs](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/emacs-27.2-3.oe2203.x86_64.rpm), [autoconf](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/autoconf-2.71-2.oe2203.noarch.rpm), [automake](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/automake-1.16.5-3.oe2203.noarch.rpm), [libtool-ltdl](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/libtool-ltdl-2.4.6-34.oe2203.x86_64.rpm), [m4](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/m4-1.4.19-2.oe2203.x86_64.rpm)， [tar](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/tar-1.34-1.oe2203.x86_64.rpm), [libcgroup](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/libcgroup-0.42.2-1.oe2203.x86_64.rpm)。

在下载上述软件包之后，执行命令

```bash
rpm -ivh iptables-1.8.7-5.oe2203.x86_64.rpm libtool-2.4.6-34.oe2203.x86_64.rpm emacs-27.2-3.oe2203.x86_64.rpm autoconf-2.71-2.oe2203.noarch.rpm automake-1.16.5-3.oe2203.noarch.rpm libtool-ltdl-2.4.6-34.oe2203.x86_64.rpm m4-1.4.19-2.oe2203.x86_64.rpm tar-1.34-1.oe2203.x86_64.rpm libcgroup-0.42.2-1.oe2203.x86_64.rpm
```

#### 4.3.2.2 another\_rootfs环境配置

1. 在`/another_rootfs`中，需要安装iptables，这个是dockerd启动所必须的依赖
   使用`yum install <软件包>  --installroot=/another_rootfs`来安装软件包

2. 并需要将rexec拷贝到`/another_rootfs/usr/bin`下面，并对其添加可执行权限

   ```bash
   cp rexec /another_rootfs/usr/bin
   chmod +x /another_rootfs/usr/bin/rexec
   ```

3. 另外，将根据社区docker源码与前述patch编译出来的containerd和dockerd拷贝到`/another_rootfs/usr/bin`下面, 将docker拷贝到`/usr/bin`下

   ```bash
   cp {YOUR_PATH}/dockerd /another_rootfs/usr/bin
   cp {YOUR_PATH}/containerd /another_rootfs/usr/bin
   cp {YOUR_PATH}/docker /usr/bin
   ```

4. 在`/another_rootfs`中删除/another\_rootfs/usr/sbin/modprobe

   ```bash
   rm -f /another_rootfs/usr/sbin/modprobe
   ```

5. 在`/another_rootfs`中创建如下三个脚本

   containerd-shim创建路径：`/another_rootfs/usr/local/bin/containerd-shim`

   ```bash
   #!/bin/bash
   /usr/bin/rexec /usr/bin/containerd-shim $*
   ```

   remote\_kill创建路径：`/another_rootfs/usr/bin/remote_kill`

   ```bash
   #!/bin/bash
   /usr/bin/rexec /usr/bin/kill $*
   ```

   modprobe创建路径：`/another_rootfs/usr/sbin/modprobe`

   ```bash
   #!/bin/bash
   /usr/bin/rexec /usr/sbin/modprobe $*
   ```

   在创建完成后，为其赋予执行权限

   ```bash
   chmod +x /another_rootfs/usr/local/bin/containerd-shim
   chmod +x /another_rootfs/usr/bin/remote_kill
   chmod +x /another_rootfs/usr/sbin/modprobe
   ```

#### 4.3.2.3 目录挂载

在DPU上执行此脚本[prepare.sh](./config/prepare.sh) ，将dockerd、containerd所需要的HOST目录挂载到DPU。

并且，需要确保在该脚本被挂载的远程目录在HOST和DPU都存在。

## **4.4** 拉起dockerd和containerd

在DPU上，打开两个窗口，并且都chroot到dockerd和containerd运行所需的/another\_rootfs。

```bash
chroot /another_rootfs
```

在两个窗口中用如下的命令先拉起containerd，后拉起dockerd。

**containerd**

```bash
#!/bin/bash
SHIM_HOST=${YOUR_SERVER_IP} containerd --config /var/run/docker/containerd/containerd.toml --address /var/run/containerd/containerd.sock
```

**dockerd**

```bash
#!/bin/bash
# this need to be done once
/usr/bin/rexec mount -t qtfs /var/lib/docker/overlay2 /another_rootfs/var/lib/docker/overlay2/
SHIM_HOST=${YOUR_SERVER_IP} /usr/bin/dockerd --containerd /var/run/containerd/containerd.sock -s overlay2 --iptables=false --debug 2>&1 | tee docker.log
```

因为我们已经将`/var/run/`和`/another_rootfs/var/run/`绑定在一起，可以在正常的rootfs下，通过docker来访问docker.sock接口，从而管理容器。

# **5** 环境恢复

如果需要卸载相关的目录，需要先删除已有容器，再关掉containerd、dockerd，并执行如下指令：

```bash
for i in `lsof | grep v1.linux | awk '{print $2}'`
do
        kill -9 $i
done
mount | grep qtfs | awk '{print $3}' | xargs umount
mount | grep another_rootfs | awk '{print $3}' | xargs umount

sleep 1

umount /another_rootfs/etc
umount /another_rootfs/sys
pkill udsproxyd
rmmod qtfs
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/diversified_computing/dpu_offload/libvirt_direct_connection_aggregation_environment_establishment.md
---
# **1** 硬件准备

## 测试模式

需准备2台物理机（虚拟机当前未试过），网络互通。

其中一台作为DPU模拟，另一台作为HOST模拟。在本文档中用DPU和HOST指代这两台服务器。

> \[!NOTE]说明
> 测试模式因为会暴露网络端口且不做连接认证，存在网络安全风险，仅能用于内部测试验证，不要用于实际生产环境。
> 实际生产环境中应使用封闭通信方式，以防止外界连接风险，如下述vsock通信模式。

## vsock模式

需要DPU加HOST，且DPU能支持通过virtio提供vsock通信方式。

文档当前仅描述基于测试模式的方法，下面的内容依然默认使用测试模式。如果验证环境支持vsock通信（虚拟化环境或支持vsock的DPU-HOST环境），下述测试步骤仍适用，只需要将测试中的IP网络地址修改为vsock cid即可（相关二进制编译过程也无需使用TEST\_MODE）。

# **2** libvirt卸载架构图

![arch](./figures/arch.png)

# **3** 环境搭建

## **3.1** qtfs文件系统部署

可参考qtfs主页：<https://atomgit.com/openeuler/dpu-utilities/tree/master/qtfs>

qtfs建联需要关闭DPU和HOST侧的防火墙，或在防火墙中开放相关网络端口号。

## **3.2** UDSPROXYD服务部署

### 3.2.1 简介

udsproxyd是一个跨主机的unix domain socket代理服务，需要分别部署在HOST和DPU上，在HOST和DPU上的udsproxyd组件是对等的关系，可以实现分布在HOST与DPU上的2个进程之间的uds通信，通信进程是无感的，也就是说如果这两个进程在同一主机内通过uds正常通信的功能，拉远到HOST和DPU之间也可以，不需要做代码适配，只需要作为client的一端加一个环境变量`LD_PRELOAD=libudsproxy.so`。udsproxyd作为一个跨主机的unix socket服务，本身可以用LD\_PRELOAD=libudsproxy.so的方式对接使用，在qtfs的支持下也可以无感应用，这需要提前做好白名单相关配置，具体有两种方式，将在后面详述。

### 3.2.2 部署方式

首先，在dpu-utilities工程内编译udsproxyd：

```bash
cd qtfs/ipc

make -j UDS_TEST_MODE=1 && make install
```

> 如果使用vsock模式，编译时不需要设置UDS\_TEST\_MODE。

当前最新版本下，qtfs server侧的engine服务已经整合了udsproxyd的能力，所以server侧若部署了qtfs后不需要再额外启动udsproxyd。client侧则单独拉起udsproxyd服务：

```bash
nohup /usr/bin/udsproxyd <thread num> <addr> <port> <peer addr> <peer port> 2>&1 &
```

参数解释：

```bash
thread num: 线程数量，目前只支持单线程，填1

addr: 本机使用的ip，如果使用vsock通信模式则为cid

port：本机占用的port

peer addr: udsproxyd对端的ip，如果使用vsock模式则为cid

peer port: 对端port
```

示例：

```bash
nohup /usr/bin/udsproxyd 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 &
```

如果未拉起qtfs的engine服务，想单独测试udsproxyd，则在server端也对等拉起udsproxyd即可：

```bash
nohup /usr/bin/udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 &
```

### 3.2.3 应用方式

#### 3.2.3.1 独立使用udsproxyd服务

需要在使用uds服务的unix socket应用程序的client端进程启动时添加LD\_PRELOAD=libudsproxy.so环境变量，以接管glibc的connect api进行uds对接，在libvirt卸载场景中可以将libudsproxy.so拷贝到libvirt的chroot目录下的/usr/lib64中以提供给libvirtd服务使用，这一步在后面介绍。

#### 3.2.3.2 无感使用udsproxyd服务

首先为qtfs配置uds服务的白名单，这里说的白名单是unix socket的server端bind的sock文件地址，例如libvirt的虚拟机建立的unix socket的服务端文件地址都在/var/lib/libvirt下，则我们需要增加一条白名单路径为/var/lib/libvirt/，提供两种方式供选择：

* 通过配置工具qtcfg加载，进入qtfs/qtinfo目录编译工具：

在qtfs的client端执行

```bash
make role=client 
make install
```

在qtfs的server端执行

```bash
make role=server
make install
```

配置工具将会自动安装，然后使用qtcfg命令配置白名单，假设需要增加的白名单为"/var/lib/libvirt/"，输入：

```bash
qtcfg -w udsconnect -x /var/lib/libvirt/
```

查询白名单为：

```bash
qtcfg -w udsconnect -z
```

删除白名单为:

```bash
qtcfg -w udsconnect -y 0
```

删除白名单时，参数为查询白名单时列出来的index序号。

* 通过配置文件增加，这需要在qtfs或qtfs\_server内核模块加载前配置，通过内核模块初始化时读取该文件进行白名单配置。

> \[!NOTE]说明
> 白名单是为了防止不相干的unix socket链接也进行远程连接产生错误，或者浪费不必要的资源，所以白名单尽量设置得精确一些，比如本文中针对libvirt场景设置为/var/lib/libvirt/比较好，而直接将/var/lib/或/var/或直接将根目录加入的做法是有较大风险的。

## **3.3** REXEC服务部署

### 3.3.1 简介

rexec是一个用c语言开发的远程执行组件，分为rexec client和rexec server。server端为一个常驻服务进程，client端为一个二进制文件，client端被执行后会基于udsproxyd服务与server端建立uds连接，并由server常驻进程在server端拉起指定程序。在libvirt虚拟化卸载中，libvirtd卸载到DPU上，当它需要在HOST拉起虚拟机qemu进程时调用rexec client进行远程拉起。

### 3.3.2 部署方法

#### 3.3.2.1 配置环境变量与白名单

在HOST侧配置rexec server的白名单，将文件whitelist放置在/etc/rexec/目录下并修改权限为只读：

```bash
chmod 400 /etc/rexec/whitelist
```

如果想仅用于测试，可以不进行白名单配置，删除此文件重启rexec\_server进程后则没有白名单限制。

下载dpu-utilities代码后，进入qtfs/rexec主目录下，执行：`make && make install`即可安装rexec所需全部二进制到/usr/bin目录下，包括了：`rexec、rexec_server`两个二进制可执行文件。

在server端启动rexec\_server服务之前，检查是否存在/var/run/rexec目录，没有则创建：

```bash
mkdir /var/run/rexec
```

rexec服务运行时底层通信使用unix socket，因此跨主机的rexec和rexec\_server通信依赖上述udsproxyd服务，并且需要在udsproxy中添加相关白名单：

```bash
qtcfg -w udsconnect -x /var/run/rexec
```

#### 3.3.2.2 服务方式

server端可以通过两种方式拉起rexec\_server服务。

* 方式1：

配置systemd服务

在/usr/lib/systemd/system/下增加rexec.service文件，内容如下：

[rexec.service](./config/rexec.service)

然后通过systemctl管理rexec服务。

首次配置服务时：

```bash
systemctl daemon-reload

systemctl enable --now rexec
```

后续重启新启动服务：

```bash
systemctl stop rexec

systemctl start rexec
```

* 方式2：

手动后台拉起

```bash
nohup /usr/bin/rexec_server 2>&1 &
```

## **3.4** libvirt服务部署

### 3.4.1 HOST侧部署

HOST无需额外部署，只需要安装虚拟机启动环境以及libvirt即可（安装libvirt主要是为了创建对应的目录）：

```bash
yum install -y qemu libvirt edk2-aarch64 #(arm环境虚拟机启动需要)
```

HOST需要放置虚拟机镜像，后面通过qtfs挂载到client端共享给libvirt。

### 3.4.2 DPU侧部署

#### 3.4.2.1 创建chroot环境

a) 从openEuler官网下载qcow镜像，例如openEuler 22.03 LTS版本：<https://repo.openeuler.org/openEuler-22.03-LTS/virtual_machine_img/>。

b) 将qcow2挂载出来：

```bash
cd /root/

mkdir p2 new_root_origin new_root

modprobe nbd maxport=8

qemu-nbd -c /dev/nbd0 xxx.qcow2

mount /dev/nbd0p2 /root/p2

cp -rf /root/p2/* /root/new_root_origin/

umount /root/p2

qemu-nbd -d /dev/nbd0
```

c) 此时new\_root\_origin有解压出来的镜像根目录，再将new\_root绑定挂载到该目录上，作为chroot的根目录挂载点：

```bash
mount --bind /root/new_root_origin /root/new_root
```

#### 3.4.2.2 安装libvirt

此处介绍patch方式源码编译，如果计算提供rpm包则参考计算提供的安装方法。

a) 进入chroot环境，安装编译环境和常用工具：

```bash
yum groupinstall "Development tools" -y
yum install -y vim meson qemu qemu-img strace edk2-aarch64 tar
```

其中edk2-aarch64是arm环境下虚拟机启动需要的。

b) 安装libvirt编译需要的依赖包：

```bash
yum install -y rpcgen python3-docutils glib2-devel gnutls-devel libxml2-devel libpciaccess-devel libtirpc-devel yajl-devel systemd-devel dmidecode glusterfs-api numactl
```

c) 下载libvirt-6.9.0源码包：<https://libvirt.org/sources/libvirt-6.9.0.tar.xz>。

d) 获取直连聚合libvirt patch：

<https://atomgit.com/openeuler/dpu-utilities/tree/master/usecases/transparent-offload/patches/libvirt>。

e) 将源码包解压到chroot环境下的目录，如/home。将patch打上。

f) 进入libvirt-6.9.0目录：

```bash
meson build --prefix=/usr -Ddriver_remote=enabled -Ddriver_network=enabled -Ddriver_qemu=enabled -Dtests=disabled -Ddocs=enabled -Ddriver_libxl=disabled -Ddriver_esx=disabled -Dsecdriver_selinux=disabled -Dselinux=disabled
```

g) 安装成功。

```bash
ninja -C build install
```

#### 3.4.2.3 启动libvirtd服务

libvirt直连聚合卸载模式，需要从chroot内启动libvirtd服务，首先需要把chroot之外的libvirtd服务停掉。

a) 放置虚拟机跳板脚本在chroot环境下的/usr/bin和/usr/libexec下：[qemu-kvm](./scripts/qemu-kvm)。替换原同名二进制，这个跳板脚本就是用于调用rexec拉起远端虚拟机。

> \[!NOTE]说明
> virsh使用的xml中，**\<devices>** 下面的 **\<emulator>** 需要填qemu-kvm，如果是填的其他，则需要修改为qemu-kvm，或者将跳板脚本替换 **\<emulator>** 指代的二进制，且跳板脚本内容需要对应地更改。

b) 将udsproxyd编译时附带产生的libudsproxy.so拷贝到本chroot目录下/usr/lib64下，如果配置qtfs的uds白名单方式使用udsproxyd服务，则不需要。

c) 将前面rexec编译产生的rexec二进制放置到本chroot的/usr/bin/目录下。

d) 配置chroot的挂载环境，需要挂载一些目录，使用如下配置脚本：

* [virt\_start.sh](./scripts/virt_start.sh)为配置脚本，virt\_start.sh脚本中需要手动修改qtfs ko dir为编译的ko位置，HOST ip address为正确的HOST地址。
* [virt\_umount.sh](./scripts/virt_umount.sh)为消除配置脚本。

e) 脚本中挂载目录位置都是按照本文档前文创建目录位置与名称为准，如果有修改需要同步适配修改脚本。

f) 配置好chroot环境后，进入chroot环境，手动拉起libvirtd。

未配置qtfs使用udsproxyd白名单的拉起方式:

```bash
LD_PRELOAD=/usr/lib64/libudsproxy.so virtlogd -d
LD_PRELOAD=/usr/lib64/libudsproxy.so libvirtd -d
```

如果已配置qtfs使用udsproxyd白名单，则不需要增加LD\_PRELOAD前缀：

```bash
virtlogd -d
libvirtd -d
```

查看是否已配置白名单的方式，在chroot之外的窗口，执行：

```bash
qtcfg -z
```

查看列举出来的白名单是否包含/var/lib/libvirt/。

## **3.5** 拉起虚拟机

服务部署完成后，即可以在DPU侧进行虚拟机的生命周期管理。

### 3.5.1 虚拟机define

a) 将虚拟机启动镜像放置在HOST侧某目录，例如：

```bash
/home/VMs/Domain_name
```

b) 使用qtfs将这个目录挂载到DPU侧：

```bash
mount -t qtfs /home/VMs /home/VMs
```

c) xml中使用`/home/VMs/Domain_name`作为启动镜像，这样在DPU和HOST侧看到的都是同一个镜像文件（Domain\_name是虚拟机domain的名字）。

d) 检查xml中\<emulator>是否指向了跳板脚本。

e) 执行。

```bash
virsh define xxx.xml
```

### 3.5.2 虚拟机start

```bash
virsh start domain
```

# **4** 环境重置

由于libvirt在DPU和HOST之间共享了部分目录，卸载环境时需要先将这部分目录全部umount。一般先停掉libvirtd和virtlogd进程，调用virt\_umount脚本即可。如果HOST还有虚拟机运行，也需要先杀掉才能umount。

# **5** 部分问题定位思路

1. libvirt编译失败：检查依赖包安装是否完全，如果chroot挂载了外部目录或者HOST目录，也可能导致编译失败，需先解除挂载。

2. qtfs挂载失败：可能server端engine进程没拉起、防火墙没关导致qtfs建联失败。

3. 虚拟机define失败：检查xml里的项目仿真器是否指向跳板脚本、虚拟机镜像是否已经通过qtfs挂载到DPU上可见，且路径与HOST一致。

4. 虚拟机启动失败：libvirtd和virtlogd服务是否拉起、rexec服务是否拉起、跳板进程是否拉起、是否qemu-kvm拉起时报错。

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/gala/using_gala_gopher.md
---
# **gala-gopher使用手册**

gala-gopher作为数据采集模块提供OS级的监控能力，支持动态加载 /卸载探针，可无侵入式地集成第三方探针，快速扩展监控范围。

本章介绍如何部署和使用gala-gopher服务。

## 安装

挂载repo源：

```basic
[oe-22.03-lts-SP4-everything] # openEuler 22.03-LTS-SP4 官方发布源
name=oe-2203-lts-SP4-everything
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-update] # openEuler 22.03-LTS-SP4 Update 官方发布源
name=oe-22.03-lts-SP4-epol-update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/update/main/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-main] # openEuler 22.03-LTS-SP4 EPOL 官方发布源
name=oe-22.03-lts-SP4-epol-main
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/x86_64/
enabled=1
gpgcheck=0
priority=1
```

安装gala-gopher：

```bash
# yum install gala-gopher
```

## 配置

### 配置介绍

gala-gopher配置文件为`/opt/gala-gopher/gala-gopher.conf`，该文件配置项说明如下(省略无需用户配置的部分)。

如下配置可以根据需要进行修改：

* global：gala-gopher全局配置信息。
  * log\_file\_name：gala-gopher日志文件名。
  * log\_level: gala-gopher日志级别（暂未开放此功能）。
  * pin\_path：ebpf探针共享map存放路径（建议维持默认配置）。
* metric：指标数据metrics输出方式配置。
  * out\_channel：metrics输出通道，支持配置web\_server|kafka，配置为空则输出通道关闭。
  * kafka\_topic：若输出通道为kafka，此为topic配置信息。
* event：异常事件event输出方式配置。
  * out\_channel：event输出通道，支持配置logs|kafka，配置为空则输出通道关闭。
  * kafka\_topic：若输出通道为kafka，此为topic配置信息。
* meta：元数据metadata输出方式配置。
  * out\_channel：metadata输出通道，支持logs|kafka，配置为空则输出通道关闭。
  * kafka\_topic：若输出通道为kafka，此为topic配置信息。
* imdb：cache缓存规格配置。
  * max\_tables\_num：最大的cache表个数，/opt/gala-gopher/meta目录下每个meta对应一个表。
  * max\_records\_num：每张cache表最大记录数，通常每个探针在一个观测周期内产生至少1条观测记录。
  * max\_metrics\_num：每条观测记录包含的最大的metric指标个数。
  * record\_timeout：cache表老化时间，若cache表中某条记录超过该时间未刷新则删除记录，单位为秒。
* web\_server：输出通道web\_server配置。
  * port：监听端口。
* kafka：输出通道kafka配置。
  * kafka\_broker：kafka服务器的IP和port。
* logs：输出通道logs配置。
  * metric\_dir：metrics指标数据日志路径。
  * event\_dir：异常事件数据日志路径。
  * meta\_dir：metadata元数据日志路径。
  * debug\_dir：gala-gopher运行日志路径。
* probes：native探针配置。
  * name：探针名称，要求与native探针名一致，如example.probe 探针名为example。
  * param ：探针启动参数，支持的参数详见[启动参数介绍表](#启动参数介绍)。
  * switch：探针是否启动，支持配置 on | off。
* extend\_probes ：第三方探针配置。
  * name：探针名称。
  * command：探针启动命令。
  * param：探针启动参数，支持的参数详见[启动参数介绍表](#启动参数介绍)。
  * start\_check：switch为auto时，需要根据start\_check执行结果判定探针是否需要启动。
  * switch：探针是否启动，支持配置on | off | auto，auto会根据start\_check判定结果决定是否启动探针。

### 启动参数介绍

| 参数项 | 含义                                                         |
| ------ | ------------------------------------------------------------ |
| -l     | 是否开启异常事件上报                                         |
| -t     | 采样周期，单位为秒，默认配置为探针5s上报一次数据             |
| -T     | 延迟时间阈值，单位为ms，默认配置为0ms                        |
| -J     | 抖动时间阈值，单位为ms，默认配置为0ms                        |
| -O     | 离线时间阈值，单位为ms，默认配置为0ms                        |
| -D     | 丢包阈值，默认配置为0(个)                                    |
| -F     | 配置为`task`表示按照`task_whitelist.conf`过滤，配置为具体进程的pid表示仅监控此进程 |
| -P     | 指定每个探针加载的探测程序范围，目前tcpprobe、taskprobe探针涉及 |
| -U     | 资源利用率阈值(上限)，默认为0%                               |
| -L     | 资源利用率阈值(下限)，默认为0%                               |
| -c     | 指示探针(tcp)是否标识client\_port，默认配置为0(否)            |
| -N     | 指定探针(ksliprobe)的观测进程名，默认配置为NULL              |
| -p     | 指定待观测进程的二进制文件路径，比如nginx\_probe，通过 -p /user/local/sbin/nginx指定nginx文件路径，默认配置为NULL |
| -w     | 筛选应用程序监控范围，如-w  /opt/gala-gopher/task\_whitelist.conf，用户可将需要监控的程序名写入task\_whitelist.conf中，默认配置为NULL表示不筛选 |
| -n     | 指定某个网卡挂载tc ebpf，默认配置为NULL表示所有网卡均挂载，示例： -n eth0 |

### 配置文件示例

* 配置选择数据输出通道：

  ```yaml
  metric =
  {
      out_channel = "web_server";
      kafka_topic = "gala_gopher";
  };

  event =
  {
      out_channel = "kafka";
      kafka_topic = "gala_gopher_event";
  };

  meta =
  {
      out_channel = "kafka";
      kafka_topic = "gala_gopher_metadata";
  };
  ```

* 配置kafka和webServer：

  ```yaml
  web_server =
  {
      port = 8888;
  };

  kafka =
  {
      kafka_broker = "<kafka服务器IP>:9092";
  };
  ```

* 选择开启的探针，示例如下：

  ```yaml
  probes =
  (
      {
          name = "system_infos";
          param = "-t 5 -w /opt/gala-gopher/task_whitelist.conf -l warn -U 80";
          switch = "on";
      },
  );
  extend_probes =
  (
      {
          name = "tcp";
          command = "/opt/gala-gopher/extend_probes/tcpprobe";
          param = "-l warn -c 1 -P 7";
          switch = "on";
      }
  );
  ```

## 启动

配置完成后，执行如下命令启动gala-gopher。

```bash
# systemctl start gala-gopher.service
```

查询gala-gopher服务状态。

```bash
# systemctl status gala-gopher.service
```

若显示结果如下，说明服务启动成功。需要关注开启的探针是否已启动，如果探针线程不存在，请检查配置文件及gala-gopher运行日志文件。

![gala-gopher成功启动状态](./figures/gala-gopher成功启动状态.png)

> 说明：gala-gopher部署和运行均需要root权限。

## 使用方法

### 外部依赖软件部署

![gopher软件架构图](./figures/gopher软件架构图.png)

如上图所示，绿色部分为gala-gopher的外部依赖组件。gala-gopher会将指标数据metrics输出到prometheus，将元数据metadata、异常事件event输出到kafka，灰色部分的gala-anteater和gala-spider会从prometheus和kafka获取数据。

> 说明：安装kafka、prometheus软件包时，需要从官网获取安装包进行部署。

### 输出数据

* **指标数据metrics**

  Prometheus Server内置了Express Browser UI，用户可以通过PromQL查询语句查询指标数据内容。详细教程参见官方文档：[Using the expression browser](https://prometheus.io/docs/prometheus/latest/getting_started/#using-the-expression-browser)。示例如下：

  指定指标名称为`gala_gopher_tcp_link_rcv_rtt`，UI显示的指标数据为：

  ```basic
  gala_gopher_tcp_link_rcv_rtt{client_ip="x.x.x.165",client_port="1234",hostname="openEuler",instance="x.x.x.172:8888",job="prometheus",machine_id="1fd3774xx",protocol="2",role="0",server_ip="x.x.x.172",server_port="3742",tgid="1516"} 1
  ```

* **元数据metadata**

  可以直接从kafka消费topic为`gala_gopher_metadata`的数据来看。示例如下：

  ```bash
  # 输入请求
  ./bin/kafka-console-consumer.sh --bootstrap-server x.x.x.165:9092 --topic gala_gopher_metadata
  # 输出数据
  {"timestamp": 1655888408000, "meta_name": "thread", "entity_name": "thread", "version": "1.0.0", "keys": ["machine_id", "pid"], "labels": ["hostname", "tgid", "comm", "major", "minor"], "metrics": ["fork_count", "task_io_wait_time_us", "task_io_count", "task_io_time_us", "task_hang_count"]}
  ```

* **异常事件event**

  可以直接从kafka消费topic为`gala_gopher_event`的数据来看。示例如下：

  ```bash
  # 输入请求
  ./bin/kafka-console-consumer.sh --bootstrap-server x.x.x.165:9092 --topic gala_gopher_event
  # 输出数据
  {"timestamp": 1655888408000, "meta_name": "thread", "entity_name": "thread", "version": "1.0.0", "keys": ["machine_id", "pid"], "labels": ["hostname", "tgid", "comm", "major", "minor"], "metrics": ["fork_count", "task_io_wait_time_us", "task_io_count", "task_io_time_us", "task_hang_count"]}
  ```

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/aops/deploying_aops.md
---
# 1 A-Ops服务介绍

A-Ops是用于提升主机整体安全性的服务，通过资产管理、漏洞管理、配置溯源等功能，识别并管理主机中的信息资产，监测主机中的软件漏洞、排查主机中遇到的系统故障，使得目标主机能够更加稳定和安全的运行。

下表是A-Ops服务涉及模块的说明：

| 模块 | 说明  |
| ---------- | ---------------------------------------------------- |
| aops-ceres | A-Ops服务的客户端，提供采集主机数据与管理其他数据采集器（如gala-gopher）的功能。响应管理中心下发的命令，处理管理中心的需求与操作。 |
| aops-zeus  | A-Ops基础应用管理中心，主要负责与其他模块的中转站，默认端口：11111对外提供基本主机管理服务，主机与主机组的添加、删除等功能依赖此模块实现。 |
| aops-hermes | A-Ops可视化操作界面，展示数据信息，提升服务易用性。 |
| aops-apollo | A-Ops漏洞管理模块相关功能依赖此服务实现，默认端口：11116识别客户机周期性获取openEuler社区发布的安全公告，并更新到漏洞库中。通过与漏洞库比对，检测出系统和软件存在的漏洞。 |
| aops-vulcanus | A-Ops工具库，**除aops-ceres与aops-hermes模块外，其余模块须与此模块共同安装使用**。 |
| aops-tools | 提供基础环境一键部署脚本、数据库表初始化，安装后在 **/opt/aops/scripts** 目录下可见。 |
| gala-ragdoll | A-Ops配置溯源模块，通过git监测并记录配置文件的改动，默认端口：11114 |
| dnf-hotpatch-plugin | dnf插件，使得dnf工具可识别热补丁信息，提供热补丁扫描及热补丁修复功能。 |

# 2 部署环境要求

建议采用4台openEuler 22.03-LTS-SP4 机器部署，其中3台用于配置服务端，1台用于纳管（aops服务纳管的主机），**且repo中需要配置update源**（[FAQ：配置update源](#q6-配置update源)），具体用途以及部署方案如下：

* 机器A：部署mysql、redis、elasticsearch等，主要提供数据服务支持，建议内存8G+。
* 机器B：部署A-Ops的资产管理zeus服务+前端展示服务，提供完整的业务功能支持，建议内存6G+。
* 机器C：部署A-Ops的漏洞管理配置溯源（gala-ragdoll），提供漏洞管理服务，建议内存4G+。
* 机器D：部署A-Ops的客户端，用作一个被AOps服务纳管监控的主机（需要监管的机器中都可以安装aops-ceres）。

| 机器编号 | 配置IP      | 部署模块                              |
| -------- | ----------- | ------------------------------------- |
| 机器A    | 192.168.1.1 | mysql，elasticsearch，redis          |
| 机器B    | 192.168.1.2 | aops-zeus，aops-hermes，aops-diana    |
| 机器C    | 192.168.1.3 | aops-apollo，gala-ragdoll，aops-diana |
| 机器D    | 192.168.1.4 | aops-ceres，dnf-hotpatch-plugin       |

> 每台机器在部署前，请先**关闭防火墙和SELinux**。

* 关闭防火墙

```shell
systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld
setenforce 0

```

* 禁用SELinux

```shell
# 修改/etc/selinux/config文件中SELINUX状态为disabled

vi /etc/selinux/config
SELINUX=disabled

# 更改之后，按下ESC键，键盘中输入 :wq 保存修改的内容
```

注：此SELINUX状态配置在系统重启后生效。

# 3 服务端部署

## 3.1 资产管理

使用资产管理功能需部署aops-zeus、aops-hermes、mysql、redis服务。

### 3.1.1 节点信息

| 机器编号 |  配置IP|部署模块|
| -------- | -------- | -------- |
| 机器A | 192.168.1.1 |mysql，redis|
| 机器B | 192.168.1.2 |aops-zeus，aops-hermes|

### 3.1.2 部署步骤

#### 3.1.2.1 部署mysql

* 安装mysql

```shell
yum install mysql-server -y
```

* 修改mysql配置文件

```bash
vim /etc/my.cnf
```

* 在mysqld配置节下新增bind-address，值为本机ip

```ini
[mysqld]
bind-address=192.168.1.1
```

* 重启mysql服务

```bash
systemctl restart mysqld
```

* 设置mysql数据库的root用户访问权限

```mysql
[root@localhost ~] mysql

mysql> show databases;
mysql> use mysql;
mysql> select user,host from user; -- 此处出现host为localhost时，说明mysql只允许本机连接，外网和本地软件客户端则无法连接。

+---------------+-----------+
| user          | host      |
+---------------+-----------+
| root          | localhost |
| mysql.session | localhost |
| mysql.sys     | localhost |
+---------------+-----------+
3 rows in set (0.00 sec)
```

```mysql
mysql> update user set host = '%' where user='root'; -- 设置允许root用户任意IP访问。
mysql> flush privileges; -- 刷新权限
mysql> exit
```

#### 3.1.2.2 部署redis

* 安装redis

```shell
yum install redis -y
```

* 修改配置文件

```shell
vim /etc/redis.conf
```

* 绑定IP

```ini
# It is possible to listen to just one or multiple selected interfaces using
# the "bind" configuration directive, followed by one or more IP addresses.
#
# Examples:
#
# bind 192.168.1.100 10.0.0.1
# bind 127.0.0.1 ::1
#
# ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the
# internet, binding to all the interfaces is dangerous and will expose the
# instance to everybody on the internet. So by default we uncomment the
# following bind directive, that will force Redis to listen only into
# the IPv4 lookback interface address (this means Redis will be able to
# accept connections only from clients running into the same computer it
# is running).
#
# IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES
# JUST COMMENT THE FOLLOWING LINE.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bind 127.0.0.1 192.168.1.1 # 此处添加机器A的真实IP
```

* 启动redis服务

```shell
systemctl start redis
```

#### 3.1.2.3 部署prometheus

* 安装prometheus：

```shell
yum install prometheus2 -y
```

* 修改配置文件

```shell
vim /etc/prometheus/prometheus.yml
```

* 被纳管的客户端**gala-gopher**地址添加至prometheus监控节点

  > 本指南中机器C用于部署客户端，故添加机器C的gala-gopher地址
  >
  > 修改**targets**配置项

```yml
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090', '192.168.1.3:8888']
```

* 启动prometheus服务

```shell
systemctl start prometheus
```

#### 3.1.2.4、 部署aops-zeus

* 安装aops-zeus

```shell
yum install aops-zeus -y
```

* 修改配置文件

```shell
vim /etc/aops/zeus.ini
```

* 将配置文件中各服务的地址修改为真实地址，本指南中aops-zeus部署于机器B，故需把IP地址配为机器B的ip地址

```ini
[zeus]
ip=192.168.1.2  // 此处ip修改为机器B真实ip
port=11111

[uwsgi]
wsgi-file=manage.py
daemonize=/var/log/aops/uwsgi/zeus.log
http-timeout=600
harakiri=600
processes=2  // 生成指定数目的worker/进程
gevent=100  // gevent异步核数

[mysql]
ip=192.168.1.1  // 此处ip修改为机器A真实ip
port=3306
database_name=aops
engine_format=mysql+pymysql://@%s:%s/%s
pool_size=100
pool_recycle=7200

[agent]
default_instance_port=8888

[redis]
ip=192.168.1.1  // 此处ip修改为机器A真实ip
port=6379

[apollo]
ip=192.168.1.3  // 此处ip修改为部署apollo服务的真实ip（建议apollo与zeus分开部署）。若不使用apollo的漏洞管理功能则可以不配置
port=11116
```

> **mysql数据库设置为密码模式**，请参阅[FAQ：密码模式下mysql服务配置链接字符串](#q5-mysql设置为密码模式)

* 启动aops-zeus服务

```shell
systemctl start aops-zeus
```

**注意：服务启动前请确保已 [初始化aops-zeus数据库](#3125-初始化aops-zeus数据库)**

> zeus服务启动失败，且报错内容包含mysql数据库连接失败，请排查是否设置mysql密码，如果是请参阅[FAQ：密码模式下mysql服务启动失败](#q5-mysql设置为密码模式)

#### 3.1.2.5 初始化aops-zeus数据库

* 执行数据库初始化

```shell
cd /opt/aops/scripts/deploy
bash aops-basedatabase.sh init zeus
```

**注意：在未安装aops-tools工具包时，也可获取sql脚本通过mysql加载的方式初始化（sql脚本路径：/opt/aops/database/zeus.sql）**

[FAQ：密码模式下mysql数据库初始化](#q5-mysql设置为密码模式)

[FAQ：/opt/aops/scripts/deploy目录不存在](#q7-optaopsscriptsdeploy目录不存在)

#### 3.1.2.6 部署aops-hermes

* 安装aops-hermes

```shell
yum install aops-hermes -y
```

* 修改配置文件

```shell
vim /etc/nginx/aops-nginx.conf
```

* 服务配置展示

  > 服务都部署在机器B，需将ngxin代理访问的各服务地址配置为机器B的真实ip

```ini
 # 保证前端路由变动时nginx仍以index.html作为入口
  location / {
      try_files $uri $uri/ /index.html;
      if (!-e $request_filename){
          rewrite ^(.*)$ /index.html last;
      }
  }
 # 此处修改为aops-zeus部署机器真实IP
  location /api/ {
      proxy_pass http://192.168.1.2:11111/;
  }
  # 此处IP对应gala-ragdoll的IP地址,涉及到端口为11114的IP地址都需要进行调整
  location /api/domain {
      proxy_pass http://192.168.1.3:11114/;
      rewrite ^/api/(.*) /$1 break;
  }
 # 此处IP对应aops-apollo的IP地址
  location /api/vulnerability {
      proxy_pass http://192.168.1.3:11116/;
      rewrite ^/api/(.*) /$1 break;
  }
```

* 开启aops-hermes服务

```shell
systemctl start aops-hermes
```

## 3.2 漏洞管理

CVE管理模块在[资产管理](#31-资产管理)模块的基础上实现，在部署CVE管理模块前须完成[资产管理](#31-资产管理)模块的部署，然后再部署aops-apollo。

数据服务部分aops-apollo服务的运行需要**mysql、elasticsearch、redis**数据库的支持。

### 3.2.1 节点信息

| 机器编号 | 配置IP      | 部署模块      |
| -------- | ----------- | ------------- |
| 机器A    | 192.168.1.1 | elasticsearch |
| 机器C    | 192.168.1.3 | aops-apollo   |

### 3.2.2 部署步骤

[部署步骤](#312-部署步骤)

#### 3.2.2.1 部署elasticsearch

* 生成elasticsearch的repo源：

```shell
echo "[aops_elasticsearch]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md" > "/etc/yum.repos.d/aops_elascticsearch.repo"
```

* 安装elasticsearch：

```shell
yum install elasticsearch-7.14.0-1 -y
```

* 修改elasticsearch配置文件：

```shell
vim /etc/elasticsearch/elasticsearch.yml
```

```yml
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
```

```yml
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
# 此处修改为机器A真实ip
network.host: 192.168.1.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
```

```yml
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1"]
# 跨域配置
http.cors.enabled: true
http.cors.allow-origin: "*"
```

* 重启elasticsearch服务

```shell
systemctl restart elasticsearch
```

#### 3.2.2.2 部署aops-apollo

* 安装aops-apollo

```shell
yum install aops-apollo -y
```

* 修改配置文件

```shell
vim /etc/aops/apollo.ini
```

* 将apollo.ini配置文件中各服务的地址修改为真实地址

```ini
[apollo]
ip=192.168.1.3//此处修改为机器C的真实IP
port=11116
host_vault_dir=/opt/aops
host_vars=/opt/aops/host_vars

[zeus]
ip=192.168.1.2 //此处修改为机器B的真实IP
port=11111

# hermes info is used to send mail.
[hermes]
ip=192.168.1.2 //此处修改为部署aops-hermes的真实IP,以机器B的IP地址为例
port=80  //此处改为hermes服务实际使用端口

[cve]
cve_fix_function=yum
# value between 0-23, for example, 2 means 2:00 in a day.
cve_scan_time=2

[mysql]
ip=192.168.1.1 //此处修改为机器A的真实IP
port=3306
database_name=aops
engine_format=mysql+pymysql://@%s:%s/%s
pool_size=100
pool_recycle=7200

[elasticsearch]
ip=192.168.1.1 //此处修改为机器A的真实IP
port=9200
max_es_query_num=10000000

[redis]
ip=192.168.1.1 //此处修改为机器A的真实IP
port=6379

[uwsgi]
wsgi-file=manage.py
daemonize=/var/log/aops/uwsgi/apollo.log
http-timeout=600
harakiri=600
processes=2
gevent=100

```

> **mysql数据库设置为密码模式**，请参阅[密码模式下mysql服务配置链接字符串](#q5-mysql设置为密码模式)

* 启动aops-apollo服务

```shell
systemctl start aops-apollo
```

**注意：服务启动前请确保已 [初始化aops-apollo数据库](#3223-初始化aops-apollo数据库)**

> apollo服务启动失败，且报错内容包含mysql数据库连接失败，请排查是否设置mysql密码，如果是请参阅[密码模式下mysql服务启动失败](#q5-mysql设置为密码模式)

#### 3.2.2.3 初始化aops-apollo数据库

* apollo数据库初始化

```shell
cd /opt/aops/scripts/deploy
bash aops-basedatabase.sh init apollo
```

**注意：在未安装aops-tools工具包时，也可获取sql脚本通过mysql加载的方式初始化（sql脚本路径：/opt/aops/database/apollo.sql）**

[FAQ：密码模式下mysql数据库初始化](#q5-mysql设置为密码模式)

[FAQ：/opt/aops/scripts/deploy目录不存在](#q7-optaopsscriptsdeploy目录不存在)

## 3.3 配置溯源

A-Ops配置溯源在机器管理的基础上依赖gala-ragdoll实现，同样在部署gala-ragdoll服务之前，须完成[资产管理](#31-资产管理)部分的部署。

### 3.3.1 节点信息

| 机器编号 | 配置IP      | 部署模块     |
| -------- | ----------- | ------------ |
| 机器C    | 192.168.1.3 | gala-ragdoll |

### 3.3.2 部署步骤

[部署步骤](#322-部署步骤)

#### 3.3.2.1 部署gala-ragdoll

* 安装gala-ragdoll

```shell
yum install gala-ragdoll python3-gala-ragdoll -y
```

* 修改配置文件

```shell
vim /etc/ragdoll/gala-ragdoll.conf
```

> **将collect节点collect\_address中IP地址修改为机器B的地址，collect\_api与collect\_port修改为实际接口地址**

```ini
[git]
git_dir = "/home/confTraceTest"
user_name = "user_name"
user_email = "user_email"

[collect]
collect_address = "http://192.168.1.2"    //此处修改为机器B的真实IP
collect_api = "/manage/config/collect"    //此处接口原为示例值，需修改为实际接口值/manage/config/collect
collect_port = 11111                      //此处修改为aops-zeus服务的实际端口

[sync]
sync_address = "http://192.168.1.2"
sync_api = "/manage/config/sync"          //此处接口原为示例值，需修改为实际接口值/manage/config/sync
sync_port = 11111

[objectFile]
object_file_address = "http://192.168.1.2"
object_file_api = "/manage/config/objectfile"   //此处接口原为示例值，需修改为实际接口值/manage/config/objectfile
object_file_port = 11111

[ragdoll]
port = 11114
```

* 启动gala-ragdoll服务：

```shell
systemctl start gala-ragdoll
```

## 3.4 异常检测

异常检测模块依赖[机器管理](#31-资产管理)服务，故在部署异常检测功能前须完成[机器管理](#31-资产管理)模块部署，然后再部署aops-diana。

基于分布式部署考虑，aops-diana服务需在机器A与机器B同时部署，分别扮演消息队列中的生产者与消费者角色。

数据服务部分aops-diana服务的运行需要**mysql、elasticsearch、kafka**以及**prometheus**的支持。

### 3.4.1 节点信息

| 机器编号 | 配置IP      | 部署模块   |
| -------- | ----------- | ---------- |
| 机器A    | 192.168.1.1 | kafka      |
| 机器B    | 192.168.1.2 | aops-diana |
| 机器C    | 192.168.1.3 | aops-diana |

### 3.4.2 部署步骤

[部署步骤](#312-部署步骤)

[部署elasticsearch](#3221-部署elasticsearch)

#### 3.4.2.1 部署kafka

kafka使用zooKeeper用于管理、协调代理，在应用**kafka**服务时需要同步部署**zookeeper**服务。

* 安装zookeeper

```shell
yum install zookeeper -y
```

* 启动zookeeper服务

```shell
systemctl start zookeeper
```

* 安装kafka

```shell
yum install kafka -y
```

* 修改kafka配置文件

```shell
vim /opt/kafka/config/server.properties
```

* 修改**listeners** 为本机ip

```yaml
############################# Socket Server Settings #############################

# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://192.168.1.1:9092
```

* 后台运行kafka服务：

```shell
cd /opt/kafka/bin
nohup ./kafka-server-start.sh ../config/server.properties &

# 查看nohup所有的输出出现A本机ip 以及kafka启动成功INFO
tail -f ./nohup.out
```

#### 3.4.2.2 部署diana

* 安装aops-diana

```shell
yum install aops-diana -y
```

* 修改配置文件

  > 机器A与机器B中aops-diana分别扮演不同的角色，通过**配置文件的差异来区分两者扮演角色的不同**

```shell
vim /etc/aops/diana.ini
```

（1）机器C中aops-diana以**executor**模式启动，**扮演kafka消息队列中的消费者角色**，配置文件需修改部分如下所示：

```ini
[diana]
ip=192.168.1.3  // 此处ip修改为机器C真实ip
port=11112
mode=executor  // 该模式为executor模式，用于常规诊断模式下的执行器，扮演kafka中消费者角色。
timing_check=on

[default_mode]
period=60
step=60

[elasticsearch]
ip=192.168.1.1  // 此处ip修改为机器A真实ip
port=9200
max_es_query_num=10000000

[mysql]
ip=192.168.1.1  // 此处ip修改为机器A真实ip
port=3306
database_name=aops
engine_format=mysql+pymysql://@%s:%s/%s
pool_size=100
pool_recycle=7200

[redis]
ip=192.168.1.1  // 此处ip修改为机器A真实ip
port=6379

[prometheus]
ip=192.168.1.1  // 此处ip修改为机器A真实ip
port=9090
query_range_step=15s

[agent]
default_instance_port=8888

[zeus]
ip=192.168.1.2  // 此处ip修改为机器B真实ip
port=11111

[consumer]
kafka_server_list=192.168.1.1:9092  // 此处ip修改为机器A真实ip
enable_auto_commit=False
auto_offset_reset=earliest
timeout_ms=5
max_records=3
task_name=CHECK_TASK
task_group_id=CHECK_TASK_GROUP_ID
result_name=CHECK_RESULT

[producer]
kafka_server_list = 192.168.1.1:9092  // 此处ip修改为机器A真实ip
api_version = 0.11.5
acks = 1
retries = 3
retry_backoff_ms = 100
task_name=CHECK_TASK
task_group_id=CHECK_TASK_GROUP_ID

[uwsgi]
wsgi-file=manage.py
daemonize=/var/log/aops/uwsgi/diana.log
http-timeout=600
harakiri=600
processes=2
threads=2
```

> **mysql数据库设置为密码模式**，请参阅[FAQ：密码模式下mysql服务配置链接字符串](#q5-mysql设置为密码模式)

（2）机器B中diana以**configurable**模式启动，**扮演kafka消息队列中的生产者角色**，aops-hermes中关于aops-diana的端口配置以该机器ip与端口为准，配置文件需修改部分如下所示：

```ini
[diana]
ip=192.168.1.2  // 此处ip修改为机器B真实ip
port=11112
mode=configurable  // 该模式为configurable模式，用于常规诊断模式下的调度器，充当生产者角色。
timing_check=on

[default_mode]
period=60
step=60

[elasticsearch]
ip=192.168.1.1  // 此处ip修改为机器A真实ip
port=9200
max_es_query_num=10000000

[mysql]
ip=192.168.1.1  // 此处ip修改为机器A真实ip
port=3306
database_name=aops
engine_format=mysql+pymysql://@%s:%s/%s
pool_size=100
pool_recycle=7200

[redis]
ip=192.168.1.1  // 此处ip修改为机器A真实ip
port=6379

[prometheus]
ip=192.168.1.1  // 此处ip修改为机器A真实ip
port=9090
query_range_step=15s

[agent]
default_instance_port=8888

[zeus]
ip=192.168.1.2  // 此处ip修改为机器B真实ip
port=11111

[consumer]
kafka_server_list=192.168.1.1:9092  // 此处ip修改为机器A真实ip
enable_auto_commit=False
auto_offset_reset=earliest
timeout_ms=5
max_records=3
task_name=CHECK_TASK
task_group_id=CHECK_TASK_GROUP_ID
result_name=CHECK_RESULT

[producer]
kafka_server_list = 192.168.1.1:9092  // 此处ip修改为机器A真实ip
api_version = 0.11.5
acks = 1
retries = 3
retry_backoff_ms = 100
task_name=CHECK_TASK
task_group_id=CHECK_TASK_GROUP_ID

[uwsgi]
wsgi-file=manage.py
daemonize=/var/log/aops/uwsgi/diana.log
http-timeout=600
harakiri=600
processes=2
threads=2
```

> **mysql数据库设置为密码模式**，请参阅[FAQ：密码模式下mysql服务配置链接字符串](#q5-mysql设置为密码模式)

* 启动aops-diana服务

```shell
systemctl start aops-diana
```

**注意：服务启动前请确保已 [初始化aops-diana数据库](#3423-初始化aops-diana数据库)**

> diana服务启动失败，且报错内容包含mysql数据库连接失败，请排查是否设置mysql密码，如果是请参阅[FAQ：密码模式下mysql服务启动失败](#q5-mysql设置为密码模式)

#### 3.4.2.3 初始化aops-diana数据库

* diana数据库初始化

```shell
cd /opt/aops/scripts/deploy
bash aops-basedatabase.sh init diana
```

**注意：在未安装aops-tools工具包时，也可获取sql脚本通过mysql加载的方式初始化（sql脚本路径：/opt/aops/database/diana.sql）**

[FAQ：密码模式下mysql数据库初始化](#q5-mysql设置为密码模式)

[FAQ：/opt/aops/scripts/deploy目录不存在](#q7-optaopsscriptsdeploy目录不存在)

## 3.5 客户端安装

aops-ceres作为A-Ops模块的客户端，通过ssh协议与AOps管理中心进行数据交互，提供采集主机信息、响应并处理中心命令等功能。

### 3.5.1 节点信息

| 机器编号 | 配置IP      | 部署模块   |
| -------- | ----------- | ---------- |
| 机器D    | 192.168.1.4 | aops-ceres |

### 3.5.2 部署客户端

```shell
yum install aops-ceres dnf-hotpatch-plugin -y
```

## FAQ

### Q1 最大连接数（MaxStartups）

批量添加主机接口服务执行过程中会受到aops-zeus安装所在主机sshd服务配置中最大连接数（MaxStartups）的限制，会出现部分主机不能连接的情况，如有大量添加主机的需求，可考虑临时调增该数值。关于该配置项的修改可参考[ssh文档](https://www.man7.org/linux/man-pages/man5/sshd_config.5.html)。

### Q2 504网关超时

部分http访问接口执行时间较长，web端可能返回504错误，可向nginx配置中添加proxy\_read\_timeout配置项，并适当调大该数值，可降低504问题出现概率。

### Q3 防火墙

若防火墙不方便关闭，请设置放行服务部署过程涉及的所有接口，否则会造成服务不可访问，影响A-Ops的正常使用。

### Q4 elasticasearch访问拒绝

elasticsearch分布式部署多节点时，需调整配置跨域部分，允许各节点访问。

### Q5 mysql设置为密码模式

* **服务配置mysql链接字符串**

mysql数据库链接设置密码模式（例如用户名为**root**，密码为**123456**），则需要调整\*\*\[mysql]配置节下engine\_format\*\*配置项（apollo、zeus同步调整），数据格式如下：

```ini
[mysql]
egine_format=mysql+pymysql://root:123456@%s:%s/%s
```

* **初始化脚本aops-basedatabase.sh修改**

aops-basedatabase.sh脚本需要调整145行代码实现

> aops-basedatabase.sh调整前内容如下：

```shell
database = pymysql.connect(host='$mysql_ip', port=$port, database='mysql', autocommit=True,client_flag=CLIENT.MULTI_STAT    EMENTS)
```

> aops-basedatabase.sh调整后内容如下：

```shell
database = pymysql.connect(host='$mysql_ip', port=$port, database='mysql', password='密码', user='用户名', autocommit=True, client_flag=CLIENT.MULTI_STATEMENTS)
```

* **服务启动时数据库连接错误**

**/usr/bin/aops-vulcanus**脚本需要调整178行代码实现

> /usr/bin/aops-vulcanus调整前内容如下：

```shell
connect = pymysql.connect(host='$mysql_ip', port=$port, database='$aops_database')
```

> /usr/bin/aops-vulcanus调整后内容如下：

```shell
connect = pymysql.connect(host='$mysql_ip', port=$port, database='$aops_database', password='密码', user='用户名')
```

**注意：当服务器不是以root用户登录时，需添加user="root"或mysql允许链接的用户名**

### Q6 配置update源

```shell
echo "[update]
name=update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/update/$basearch/
enabled=1
gpgcheck=0
[update-epol]
name=update-epol
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/update/main/$basearch/
enabled=1
gpgcheck=0" > /etc/yum.repos.d/openEuler-update.repo
```

> 注意：其中**openEuler-22.03-LTS-SP4** 根据部署的系统版本具体调整，或可直接参与openeuler官网中针对repo源配置介绍

### Q7 /opt/aops/scripts/deploy目录不存在

在执行数据库初始化时，提示不存在\*\*/opt/aops/scripts/deploy\*\*文件目录，执行安装aops-tools工具包

```shell
yum install aops-tools -y
```

---

---
url: /en/docs/22.03_LTS_SP4/server/development/gcc/gcc_user_guide.md
---
# 1 CFGO

## 1.1 Feature Description

* Concept: Continuous feature-guided optimization (CFGO) is a feature that continuously optimizes multimodal files (source code and binaries) and full lifecycle (covering compilation, linking, post-linking, runtimes, OSs, and libraries) of programs.
* Core idea: Assuming that the program functions remain unchanged, runtime profiling data is used to guide optimizations, resulting in a more refined program with better performance.
* Major optimizations include:
  * Code layout optimization: The binary layout of the target program is optimized by using techniques such as basic block reordering, function rearrangement, and hot/cold splitting to improve the I-cache and I-TLB hits.
  * Advanced compiler optimization: Optimization techniques such as inlining, loop unrolling, vectorization, and indirect calling enable the compiler to make more informed decisions on optimization.

## 1.2 BOLTUSE

### 1.2.1 Option Description

#### Option `-fbolt-use`

This option uses Binary Optimization and Layout Tool (BOLT) for profiling to complete post-linking optimization. Prepare the profiling file required for BOLT optimization in advance. You can obtain the profiling file either by using the AutoBOLT mode or by collecting data with `perf` and then converting it to a BOLT-compatible format with `perf2bolt`.

### 1.2.2 Common Application Enabling Methods

1. MySQL

   ```bash
   cmake .. -DCMAKE_INSTALL_PREFIX=/home/install/mysql-8.0.25 -DBUILD_CONFIG=mysql_release -DWITH_BOOST=../boost -DCMAKE_C_FLAGS="-Wl,-q" -DCMAKE_CXX_FLAGS="-Wl,-q" -DCMAKE_C_LINK_FLAGS="-Wl,-q" -DCMAKE_CXX_LINK_FLAGS="-Wl,-q -fbolt-use=/path/to/mysqld.profile -fbolt-target=mysqld" -DCMAKE_EXE_LINKER_FLAGS="-Wl,-q"
   ```

2. MongoDB

   ```bash
   # Build a baseline version with relocation information.
   python3 buildscripts/scons.py MONGO_VERSION=6.1.0 DESTDIR=/usr/local/mongo install-all-meta CFLAGS="-march=armv8-a+crc -mtune=generic -Wl,-q" CXXFLAGS="-march=armv8-a+crc -mtune=generic -Wl,-q" LINKFLAGS="-Wl,-q" -j 128 --disable-warnings-as-errors
   # Use perf and perf2bolt to generate a BOLT-compatible profiling file.
   # Use BOLTUSE to build an optimized version. Note that if the LLVM-BOLT version does not support PIEs, you need to manually delete the pie option from SConstruct.
   python3 buildscripts/scons.py MONGO_VERSION=6.1.0 DESTDIR=/usr/local/mongo install-devcore CFLAGS="-march=armv8-a+crc -mtune=generic -Wl,-q" CXXFLAGS="-march=armv8-a+crc -mtune=generic -Wl,-q" LINKFLAGS="-Wl,-q -fbolt-use=./mongod.profile -fbolt-target=mongod" -j 128 --disable-warnings-as-errors
   ```

3. Redis

   ```bash
   make CFLAGS="-Wl,-q" LDFLAGS="-fbolt-use=/path/to/redis.profile -fbolt-target=redis-server"
   ```

   **Note:**

   * By default, the profiling file named **data.fdata** in the current path is used for optimization. You can use `-fbolt-use=FILE` to specify the profiling file to be used, for example, `-fbolt-use=/tmp/a.fdata`.
   * It can be used together with `-fbolt-target` and `-fbolt-option=<options>`.

## 1.3 AutoBOLT

### 1.3.1 Option Description

In the instrumentation feedback-directed optimization scenario, two compilations are required. The first compilation is performed with compilation option `-fprofile-generate[=path]`. After a series of typical scenario tests, the profiling data is generated. The second compilation is performed with compilation option `-fprofile-use[=path]`, during which AutoBOLT converts the instrumentation data to a BOLT-compatible profiling file and automatically enables post-link optimization.

#### Application Scenarios

Scenarios that are insensitive to performance loss in the instrumentation phase, or those insensitive to the build and test time but pursue ultimate performance.

#### How to Use

```bash
# Perform instrumentation to generate gcno.
gcc -O3 -fprofile-generate[=path] -fprofile-update=atomic -o test test.c
# Perform tests to obtain gcda./test.
# Use the generated gcda to build a new executable file.
gcc -O3 -fprofile-use[=path] -fprofile-correction -Wno-error=coverage-mismatch -Wno-error=missing-profile -fauto-bolt -fbolt-target=test_pgoed -o test_pgoed test.c
```

### 1.3.2 Automatic (Sampling) Feedback-Directed Optimization

Different from instrumentation feedback-directed optimization, automatic feedback-directed optimization (AutoFDO) uses `perf` to collect program runtime information, `create_gcov` to resolve the sampling information from `perf` to the profiling data required by the compiler, and `-fauto-profile` to read the profiling data to complete optimization. The `-fprofile-correction` option is used to enable the MCF algorithm to smooth the basic block count imbalance caused by sampling.

#### Application Scenarios

Scenarios where versions are frequently released and the entire system needs to be deployed in the production environment.

#### How to Use

```bash
# Compile an executable file with debugging information.
gcc -O3 -g -o test_prof test.c
# Use perf to collect profiling data.
perf record -e br_retired -- ./test_prof
# Use create_gcov to resolve perf.data to a gcov file.
create_gcov --binary=./test_prof --profile=perf.data --gcov=test.gcov -gcov_version=2
# Use gcov to compile profiling data to generate a new executable file.
gcc -O3 -fauto-profile=test.gcov test.c -o test_autofdo
```

#### Additional Information

1. By default, the converted profiling file is saved in the current path. You can use `-fauto-bolt=PATH` to specify the path for saving the BOLT-compatible profiling file, for example, `-fauto-bolt=/tmp`.
2. In instrumentation feedback-directed optimization, the gcov-tool tool can be used to combine instrumentation feedback-directed optimization files to generate a high-performance kernel that improves performance in multiple scenarios.
3. When `-fprofile-generate[=path]` is used in instrumentation feedback-directed optimization for multi-thread applications, `-fprofile-update=atomic` can be added to generate more accurate data.
4. In scenarios where instrumentation feedback-directed optimization is used to optimize applications, `-fprofile-correction` can be used to automatically correct some profiling data, and `-Wno-error=coverage-mismatch -Wno-error=missing-profile` can be used to mask errors reported when the profiling file cannot be matched or does not exist.

#### Typical Application Enabling Methods

1. Applications built using CMake, such as MySQL and RocksDB

   ```bash
   # Build a baseline version with relocation information.
   cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DBUILD_CONFIG=mysql_release -DWITH_BOOST=../boost -DCMAKE_C_FLAGS="-Wl,-q" -DCMAKE_CXX_FLAGS="-Wl,-q" -DCMAKE_C_LINK_FLAGS="-Wl,-q" -DCMAKE_CXX_LINK_FLAGS="-Wl,-q" -DCMAKE_EXE_LINKER_FLAGS="-Wl,-q"
   # Use AutoBOLT to build an optimized version.
   ```

2. Applications built using Make, such as Redis and Memcached

   ```bash
   # Build a baseline version with relocation information.
   make CFLAGS="-Wl,-q" CXXFLAGS="-Wl,-q"
   # Build an optimized version.
   make CFLAGS="-fprofile-use=./profile -fauto-bolt -fbolt-target=redis-server -Wl,-q -Wno-missing-profile" LDFLAGS="-Wl,-q"
   ```

#### Performance Boosts for Typical Applications

| MySQL | openGauss | Nginx | Ceph | Redis | kernel |
| ----- | --------- | ----- | ---- | ----- | ------ |
| +15%  | +5%       | +15%  | +7%  | +5%   | +5%    |

#### FAQs

1. When `-fauto-bolt` and `-fbolt-target` are used, message "The linker\[2] is not for exec, just skip." displayed. In versions earlier than openEuler 22.03 LTS SP3, GCC does not support PIE or SO optimization. You can use openEuler 22.03 LTS SP4 or later.

#### Common Options for AutoBOLT and BOLTUSE

* -`fbolt-target=NAME` specifies the BOLT optimization object. If this option is used, the binaries and dynamic libraries except that specified by `NAME` will not be optimized. To use this option, `NAME` must be specified.
* -`fbolt-option=PARAM` specifies the BOLT optimization option. Different options must be separated by commas (,). For example, `-fbolt-option="-reorder-blocks=cache+,-reorder-functions=hfsort+"`. To use this option, `PARAM` must be specified. You can use `llvm-bolt --help` to query details about the optimization option.

## 1.4 Profile-Guided Optimization (PGO)

### Feature Description

PGO enhances program performance by leveraging runtime information (profiling files) gathered through instrumentation. The compiler utilizes these profiling files to make more informed optimization decisions during the compilation process, resulting in a more optimized executable file.

PGO primarily encompasses two optimization methods:

1. Edge profiling: Counters are inserted along the edges of the control flow graph (CFG) of the function to collect program execution count information.
2. Value profiling: This optimization inserts code during instrumentation to gather statistics on frequently occurring values of expressions and variables, providing the compiler with insights for better optimization decisions. After program execution, the generated profiling file is read back by the compiler to guide corresponding optimization passes. GCC currently supports various value profile counting functionalities, including frequently used variable values, identification of powers of two, common indirect call addresses, and function first run IDs.

### 1.4.1 Compilation Options

#### `-fprofile-generate[=path]`

This option enables both `-fprofile-arcs` and `-fprofile-values`.  `-fprofile-generate` optionally accepts a `path` argument, specifying the directory for storing profiling feedback data files. This function is equivalent to `-fprofile-dir`.

#### `-fprofile-correction`

This option employs heuristics for error correction or smoothing to mitigate inconsistencies in the profiling feedback data.

#### `-Wno-coverage-mismatch`

By default, profile mismatches with the source code are treated as errors. This option downgrades such errors to warnings, recommended only for situations with minimal code changes, particularly in cold code paths.

#### `-fprofile-use[=path]`

This option instructs the compiler to use profiling files stored at the specified `path` for optimization guidance.

### 1.4.2 Kernel Feedback Optimization

#### Background

As most servers today are dedicated to running a single or a limited set of applications, optimizing the OS based on the execution profile of these specific programs can yield significant performance gains.

#### Example

1. Download the kernel source code.

   ```bash
   yum install -y kernel-source
   ```

2. Compile the instrumented kernel.

   Go to the kernel source directory and configure the kernel options. This example uses kernel 5.10.

   ```bash
   cd /usr/src/linux-5.10.0***
   make openeuler_defconfig
   make menuconfig
   ```

   Within the menu:

   * Go to **General setup** > **Local Version-append to kernel release**, and provide a suitable suffix (such as **-test-going**).
   * Press **Esc** twice to return to the main menu.
   * Go to **General architecture-dependent options** > **GCOV-based kernel profiling**.
   * Press **Esc** twice to return to the main menu.
   * Go to **Kernel hacking** > **Compile-time checks and compiler options profiling** and disable **Compile the kernel with debug info**.
   * Save and exit.

3. Build the RPM package (replace 96 with the maximum core count).

   ```bash
   make binrpm-pkg -j 96
   ```

4. Deploy the instrumented kernel.

   Install the instrumented kernel on the server, set it as the default, and reboot.

   ```bash
   rpm -ivh kernel*** --forcegrub2-set-default 0 reboot
   ```

5. Execute target application workload for profiling. This example uses the wrk testing suite of Nginx.

   Start the Nginx server.

   ```bash
   nginx -c nginx.conf
   ```

   Run the Nginx wrk test suite on the client.

   ```bash
   wrk -c 2000 -d 60s -t 20 --latency --timeout 5s -H "Connection: close" http://192.168.1.10:10000/index.html
   ```

6. Execute the profile collection script to write the profiling information from memory to the drive.

7. (Optional) To optimize the kernel for multiple applications, you can merge multiple application profiling files. Here is an example with applications A and B:

   ```bash
   gcov-tool merge a/gcovdata b/gcovdata
   ```

8. Compile the optimized kernel. Go to the kernel source directory and configure the kernel options.

   ```bash
   make openeuler_defconfig
   make menuconfig
   ```

   Within the menu:

   * Go to **General setup** > **Local version-append to kernel release** and provide a suitable suffix (such as **-test-pgoed**).
   * Press **Esc** to return to the main menu.
   * Disable **Compile the kernel with debug info**.
   * Save and exit.

9. Build the RPM package (replace 96 with the maximum core count).

   ```bash
   make binrpm-pkg -j 96 KCFLAGS="-fprofile-use -fprofile-correction -Wno-error=coverage-mismatch -Wno-error=missing-profile -fprofile-dir=gcovdata"
   ```

   **Note:**

   The `-fprofile-dir` parameter should point to the absolute path of the **gcovdata** directory generated in step 6.

#### Common Issues

* Profiling files are not generated correctly.

  * Possible cause 1: The program does not have sufficient permissions to write in the directory specified by `-fprofile-generate` for profile generation. Run `chmod 777 [absolute path to the directory]` to grant permissions.
  * Possible cause 2: The program did not exit normally. The `__gcov_exit()` function is responsible for flushing profile information upon application exit. You can attach to the application using GDB and manually call `__gcov_exit()` to resolve this.

## 1.5 Link-Time Optimization (LTO)

### 1.5.1 Introduction

**Background**

Traditional compilation processes, like those using GCC, compile and optimize individual source files (compilation units) into object files (.o) containing assembly code. The linker then resolves symbols and relocates code from these object files to create an executable file. However, while the linker has access to cross-file function call information, it operates on assembly code, limiting its ability to perform optimizations. This traditional framework, while efficient for incremental compilation, misses optimization opportunities that span multiple files.

**LTO process**

LTO aims to overcome these limitations by enabling optimization across compilation units at link time. To achieve this, LTO preserves intermediate representation (IR) information needed for optimization. During linking, the linker invokes an LTO plugin which performs program-wide analysis, resulting in more effective optimization decisions. The optimized IR is then transformed back into object files containing assembly code, and finally, the linker performs the usual linking process.

**Partitioning and parallelism**

LTO often requires longer compilation times due to the need to analyze the global call graph. To speed up LTO compilation, partitioning is introduced. In partitioned mode, the global call graph is divided into partitions based on module dependencies. Each partition is optimized independently and in parallel, striking a balance between compilation time and optimization effectiveness.

### 1.5.2 Usage

#### `-flto=`

Enable LTO by adding this option to the compilation command. The number of parallel LTO jobs can be specified using `-flto=n`. Using `-flto=auto` automatically determines the appropriate number of jobs based on the environment (such as GNU Make configuration or available CPU cores).

#### `-ffat-lto-objects`

This option generates object files containing both assembly code and LTO information. When `-ffat-lto-objects` is enabled, `-fno-lto` can be used to perform regular linking using the assembly code of the object file, bypassing LTO.

#### `-flto-partion=1to1|balanced|max|one|none`

This option controls the partitioning strategy used to speed up compilation:

* `1to1`: corresponds each source file to a partition.
* `balanced`: makes partitions roughly equal in size.
* `max`: creates a partition for each symbol.
* `one`: attempts to create a single partition.
* `none`: disables partitioning.

#### `-flto-compresson-level=n`

This option controls the compression level of the IR information stored in the object files.

**Example**

```bash
# four.cint four() {  return 4;}
# five.cint five() {  return 5;}
# test.cextern int four();extern int five();int main() {  return four() + five();}
gcc -O2 -flto test.c four.c five.c
# Or
gcc -O2 -flto -c test.cgcc -O2 -flto -c four.cgcc -O2 -flto -c five.cgcc test.o four.o five.o
```

**Explanation**

In this example, LTO enables the compiler to perform inlining and constant folding across files. The expression `four() + five()` is optimized to the immediate value 9, eliminating the function calls and calculation overhead. This optimization would not be possible without LTO. Note that if only **four.c** and **test.c** are compiled with LTO, the optimization would be limited to those files, resulting in `4 + five()`.

### 1.5.3 Common Issues

* `plugin needed to handle lto object`

  This error occurs when the linker or related tools lack the necessary LTO plugin to process LTO object files. It commonly arises when using tools like `ar`, `ranlib`, or `nm` without LTO support. Switching to their GCC counterparts (`gcc-ar`, `gcc-ranlib`, `gcc-nm`) usually resolves the issue.

* `undefined symbol`

  This issue can have multiple causes, for example:

  * Symbol versioning: GCC does not support symbol versioning using `__asm__`. Use `__attribute__((.symver))` instead.

  * Incompatible tools: Tools like `objcopy` may not fully support LTO object files. This can lead to issues with symbol renaming and result in undefined symbol errors.

  * Link-time instrumentation: Using `-Wl,--wrap=foo` with LTO might lead to an undefined symbol error for `__wrap_foo`. This is a known issue being addressed upstream.

* `multi-defined symbol`

  This issue is often caused by tools like `objcopy` that lack full LTO support. Symbols intended to be hidden might not be properly handled, leading to multiple definitions.

## 1.6 Static Compilation Options

### 1.6.1 `-fipa-struct-reorg`

The `-fipa-struct-reorg` option enables structure reorganization optimizations to improve cache hit rate by re-arranging the layout of structure members in memory.

* Structure splitting: splits a structure into separate structures based on the access frequency (hot and cold) of its members.

  **Figure 1** Structure splitting optimization

  ![Structure splitting optimization](./figures/p1.png)

  **Example:**

  Original structure:

  ```c
  struct S
    {
      type1 field1; 
      // Hot field
      type2 field2;
    };S *v;
  ```

  Transformed structure:

  ```c
  struct S_hot 
    {
      type1 field1;
    };
  struct S_cold 
    {
      type2 field2;
    };
  S_hot *v_hot;
  S_cold *v_cold;
  ```

* Structure array optimization: transforms an array of structures into a structure of arrays.

  **Figure 1** Structure array optimization

  ![Structure array optimization](./figures/p2.png)

  **Example:**

  Original structure:

  ```bash
  struct 
    {
      type1 field1;
      type2 field2;
      type3 field3;
    } arr[N];
  ```

  Transformed structure:

  ```bash
  struct 
    {
      type1 field1[N];
      type2 field2[N];
      type3 field3[N];
    } arr;
  ```

#### How to Use

Add the following options during compilation:

```bash
-O3 -flto -flto-partition=one -fipa-struct-reorg
```

**Note:**

The `-fipa-struct-reorg` option requires the `-O3 -flto -flto-partition=one` options to be enabled globally.

Optimization effect: Up to 20% performance improvement is observed on the SPECCPU2017 intrate 505.mcf benchmark.

### 1.6.2 `-fipa-reorder-fields`

The `-fipa-reorder-fields` option reorders the fields within a structure based on their size (from largest to smallest). This minimizes padding due to alignment requirements, reduces the overall memory footprint of the structure, and improves cache hit rate.

Original structure:

```c
struct S
  {
    double a;
    int b;double c;
    double d;
    short e;
    double f;
    double g;
    double h;
    double i;
  };
```

Reordered structure:

```c
struct S.reorder
  {
    double a;
    double i;
    double c;
    double d;
    double f;
    double g;
    double h;
    int b;
    short e;
  }
```

#### How to Use

Add the following options during compilation:

```bash
-O3 -flto -flto-partition=one -fipa-reorder-fields
```

**Note:**

The `-fipa-reorder-fields` option requires the `-O3 -flto -flto-partition=one` options to be enabled globally.

Optimization effect: Up to 20% performance improvement is observed on the SPECCPU2017 intrate 505.mcf benchmark.

### 1.6.3 `-fipa-struct-reorg=n`

This option provides granular control over a series of structure layout optimizations. The value of *n* determines the level of optimization:

* `-fipa-struct-reorg=0`: disables all structure layout optimizations.

* `-fipa-struct-reorg=1`: enables structure splitting and structure array optimization (equivalent to using `-fipa-struct-reorg`).

* `-fipa-struct-reorg=2`: includes all optimizations from level 1, and additionally enables structure member reordering (`-fipa-reorder-fields`).

* `-fipa-struct-reorg=3`: includes all optimizations from level 2, and additionally enables **structure redundant member elimination**. This optimization removes unread structure members and their corresponding write statements.

  **Example:**

  Original structure:

  ```c
  struct S
    {
      type1 field1;
      // Never read in whole program
      type2 field2;};
  ```

  Transformed structure:

  ```c
  struct S.layout
    {
      type2 field2;
    };
  ```

* `-fipa-struct-reorg=4`: includes all optimizations from level 3, and additionally enables **safe structure pointer compression**.

  This optimization compresses structure pointers within structure fields to 8, 16, or 32 bits (configurable), reducing memory footprint and bandwidth pressure.

  * Safe structure pointer compression supports only scenarios where the size of the structure array is known at compile time.
  * Use `--param compressed-pointer-size=[8,16,32]` to control the compressed size (the default value is 32).

  **Example:**

  Original structure:

  ```c
  struct S 
    {
      struct S* field1;
      type2 field2;
    };
  ```

  Optimized structure:

  ```c
  struct S.layout 
    {
      uint32_t field1.pc;
      type2 field2;
    };
  ```

* `-fipa-struct-reorg=5`: includes all optimizations from level 4, and **relaxes the constraints for structure pointer compression**.

  This level supports scenarios where the size of the structure array is unknown at compile time. Users must ensure the legality of the compression level. The following table shows the maximum supported structure array size for different compression levels.

  | Pointer Size | Maximum Supported Array Range |
  | ------------ | ----------------------------- |
  | 8            | 254 (0xff-1)                  |
  | 16           | 65534 (0xffff-1)              |
  | 32           | 4294967294 (0xffffffff-1)     |

* `-fipa-struct-reorg=6`: includes all optimizations from level 5, and additionally enables **structure array semi-relayout optimization**.

  Semi-relayout packs and re-arranges members of structures within a certain range, improving data locality and enhancing performance.

  * Control the relayout scale with `--param semi-relayout-level=[11,12,13,14,15]` (the default value is 13). A value of 13 means semi-relayout will re-arrange every 1024 structures as a group. If there are fewer than 1024 structures, padding is added to reach 1024. Calculation: `(1 << semi-relayout-level) / 8`.

  * The figure below illustrates the transformation of structure array layout in memory from consecutive structures after applying semi-relayout.

  **Figure 1** Semi-relayout optimization principle diagram

  ![Semi-relayout](./figures/p3.png)

  ```c
  struct S 
    {
      long a,
      int b,
      short c
    };
  ```

#### How to Use

Add the following options during compilation:

```bash
-O3 -flto -flto-partition=one -fipa-struct-reorg=n
```

Where *n* can be between 0 and 6.

**Note:**

The `-fipa-struct-reorg=n` option requires the `-O3 -flto -flto-partition=one` options to be enabled globally.

### 1.6.4 `-ftree-slp-transpose-vectorize`

This option improves the analysis of loops with consecutive memory reads by inserting temporary arrays to split the loop during loop splitting, and adds analysis to transpose grouped stores in the superword level parallelism (SLP) vectorization stage.

**Example:**

Consider the following code:

```c
int foo (unsigned char *oxa, int ia, unsigned char *oxb, int ib)
  {
    unsigned tmp[4][4];
    unsigned a0, a1, a2, a3;int sum = 0;
    for (int i = 0; i < 4; i++, oxa += ia, oxb += ib)
      {
        a0 = (oxa[0] - oxb[0]) + ((oxa[4] - oxb[4]) << 16);
        a1 = (oxa[1] - oxb[1]) + ((oxa[5] - oxb[5]) << 16);
        a2 = (oxa[2] - oxb[2]) + ((oxa[6] - oxb[6]) << 16);
        a3 = (oxa[3] - oxb[3]) + ((oxa[7] - oxb[7]) << 16);
        int t0 = a0 + a1;int t1 = a0 - a1;int t2 = a2 + a3;
        int t3 = a2 - a3;tmp[i][0] = t0 + t2;
        tmp[i][2] = t0 - t2;tmp[i][1] = t1 + t3;
        tmp[i][3] = t1 - t3;
      }
    for (int i = 0; i < 4; i++)
      {
        int t0 = tmp[0][i] + tmp[1][i];
        int t1 = tmp[0][i] - tmp[1][i];
        int t2 = tmp[2][i] + tmp[3][i];
        int t3 = tmp[2][i] - tmp[3][i];
        a0 = t0 + t2;
        a2 = t0 - t2;
        a1 = t1 + t3;
        a3 = t1 - t3;
        sum += a0 + a1 + a2 + a3;
      }
    return sum;
  }
```

The first loop can be split as follows:

```c
for (int i = 0; i < 4; i++, oxa += ia, oxb += ib)
  {
    a00[i] = (oxa[0] - oxb[0]) + ((oxa[4] - oxb[4]) << 16);
    a11[i] = (oxa[1] - oxb[1]) + ((oxa[5] - oxb[5]) << 16);
    a22[i] = (oxa[2] - oxb[2]) + ((oxa[6] - oxb[6]) << 16);
    a33[i] = (oxa[3] - oxb[3]) + ((oxa[7] - oxb[7]) << 16);
  }
for (int i = 0; i < 4; i++)
  {
    int t0 = a00[i] + a11[i];
    int t1 = a00[i] - a11[i];
    int t2 = a22[i] + a33[i];
    int t3 = a22[i] - a33[i];
    tmp[i][0] = t0 + t2;
    tmp[i][2] = t0 - t2;
    tmp[i][1] = t1 + t3;
    tmp[i][3] = t1 - t3;
  }
```

The calculations on the right side of the first split loop are isomorphic and involve consecutive loads, making them suitable for vectorization. However, the memory addresses of `a00[i]`, `a11[i]`, `a22[i]`, and `a33[i]` on the left side are not consecutive, preventing them from being the root node of the vectorized SLP tree. Ideally, the register content after each iteration should be:

| **register** | **values**                  |
| ------------ | --------------------------- |
| vec0         | a00\[0] a00\[1] a00\[2] a00\[3] |
| vec1         | a11\[0] a11\[1] a11\[2] a11\[3] |
| vec2         | a22\[0] a22\[1] a22\[2] a22\[3] |
| vec3         | a33\[0] a33\[1] a33\[2] a33\[3] |

However, the actual content calculated within each iteration is:

| **register** | **values**                  |
| ------------ | --------------------------- |
| vec0         | a00\[0] a11\[0] a22\[0] a33\[0] |
| vec1         | a00\[1] a11\[1] a22\[1] a33\[1] |
| vec2         | a00\[2] a11\[2] a22\[2] a33\[2] |
| vec3         | a00\[3] a11\[3] a22\[3] a33\[3] |

By transposing the grouped stores, the desired SLP tree root node can be obtained. Subsequently, leveraging the existing capabilities of SLP, further vectorization analysis can be performed.

In addition, for scenarios like the second split loop and the last loop in the example, where the `tmp` two-dimensional array is written to memory and immediately read, this optimization transforms the memory access into permutation operations between registers. This optimization is enabled by default.

#### How to Use

Add the following option during compilation:

```bash
-O3 -ftree-slp-transpose-vectorize
```

**Note:**

The `-ftree-slp-transpose-vectorize` option requires `-O3` to be enabled.

### 1.6.5 `-fccmp2`

This option enhances the usage scenarios for the Arm CCMP instruction, streamlining instruction pipelining and improving performance.

For code like the following:

```bash
int f(int a, int b, int c) {  while(1) {    if (a--==0||b>=c){      return 1;    }  }}
```

Enabling the `-fccmp2` option can result in the following assembly code that leverages the CCMP instruction:

**Figure 1** Assembly code with the CCMP instruction

![Assembly code with the CCMP instruction](./figures/p4.png)

#### How to Use

Add the following option during compilation:

```bash
-fccmp2
```

**Note:**

Optimization effect: Approximately 1% performance improvement observed on SPECCPU2017 intrate 557.xz\_r benchmark.

### 1.6.6 `-farray-widen-compare`

This option enables array widening during comparison operations. It allows the compiler to use wider data types to dereference array pointers (where the array elements are of a narrower type), effectively comparing multiple elements at once and enhancing performance.

**Example:**

Original code:

```c
#define my_min(x, y) ((x) < (y) ? (x) : (y))
uint32_t func (uint32_t n0, uint32_t n1, const uint32_t limit, const uint8_t * a, const uint8_t * b)
  {
    uint32_t n = my_min(n0, n1);  
    while (++n != limit)
    if (a[n] != b[n])
      break;  
    return n;}
```

Optimized code with `-farray-widen-compare`:

```bash
#define my_min(x, y) ((x) < (y) ? (x) : (y))
uint32_t func (uint32_t n0, uint32_t n1, const uint32_t limit, const uint8_t * a, const uint8_t * b)
  {
    uint32_t n = my_min(n0, n1);  
    for (++n; n + sizeof(uint64_t) <= limit; n += sizeof(uint64_t))  
      {
        uint64_t k1 = *((uint64_t*)(a+n));
        uint64_t k2 = *((uint64_t*)(b+n));
        if(k1 != k2)
          {
            int lz = __builtin_ctzll(k1 ^ k2);       
            n += lz/8;       
            return n;     
           }    
       }    
     for (;n != limit; ++n)    
       {      
         if (a[n] != b[n])        
           break;     
       }     
    return n;
  }
```

#### How to Use

Add the following options during compilation:

```bash
-O3 -farray-widen-compare
```

# 2 AI for Compiler

## 2.1 Sampling Feedback-Directed Optimization Enhancement

### 2.1.1 Feature Description

Using relevant algorithms and a framework, AI for compiler can further enhance the sampling feedback-directed optimization, more accurately predict key values in the compilation optimization process, assist the compiler in making more reasonable optimization decisions, and improve the compilation optimization effect. Using AI for compiler algorithms such as code representation learning, AI for compiler trains a basic block precision correction model for BOLT sampling, builds a framework integrated with GCC for openEuler, and achieves performance improvement.

By optimizing the code and data layout in the binary file, BOLT reduces the CPU cache miss rate and branch prediction error, thus providing further optimization after the compiler has applied LTO and feedback-directed optimization (FDO). BOLT has two ways to use dynamic profiling data for program performance improvement: instrumentation and sampling. The instrumentation method has high performance overhead but high precision, while the sampling method has very low performance overhead but low precision. The basic block precision correction model of BOLT sampling aims to predict the Count value of a basic block by exploring multi-granularity code information such as the included instructions/variables and the function/file the basic block belongs, so as to improve the accuracy of CFG and make the sampling BOLT achieve the optimization effect close to that of the instrumentation BOLT.

### 2.1.2 Installation

1. Install rpmbuild and rpmdevtools using root privileges.

   ```bash
   ##  Install rpmbuild.
   yum install rpm-build
   ##  Install rpmdevtools.
   yum install rpmdevtools
   ```

2. Download code from the target repository.

   ```bash
   git clone https://atomgit.com/src-openeuler/AI4C.git
   ```

3. Generate the **rpmbuild** directory.

   ```bash
   rpmbuild-setuptree
   # Place AI4C-v0.2.0-alpha.tar.gz and related patch files in the SOURCES directory under the rpmbuild directory.
   # Place AI4C.spec to the SPECS directory.
   ```

4. Install AI4C.

   ```bash
   rpm -ivh <built RPM package>
   ```

   If a file conflict occurs due to an installed package of an older version, add the `--force` option to `rpm -ivh <built RPM package>` to force the installation of the new version. Alternatively, update the package by running `rpm -Uvh <built RPM package>`.

5. After the installation is complete, you can find the related dynamic library under **/usr/lib64** and the related model under the **/usr/lib64/AI4C** directory.

### 2.1.3 Usage

AI4C empowers users with two inference methods: utilizing custom models via API calls or leveraging preset models with sampling feedback optimization and tuning options provided by compilation flags.

To use custom models for inference:

1. Developers first need to build their models and save them as ONNX models using ONNX Runtime.

2. Users need to call the APIs provided by AI4C within the adaptation layer of their target optimization backend to perform model inference, as shown in the table below. The interfaces listed are available in the dynamic library, **libONNXRunner.so**, and are used for model inference configuration, inference execution, and resource cleanup after inference.

   | API Name                                                                                                                                                 | Description                                                                                          |
   | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
   | extern ONNXRunner*createONNXRunner(const char* model\_path)                                                                                               | Takes the model path as a parameter and creates a session to initialize and configure the inference. |
   | extern void deleteONNXRunner(ONNXRunner\* instance)                                                                                                       | Deletes the **ONNXRunner** object.                                                                   |
   | extern float runONNXModel(ONNXRunner\* instance, std::vector\<std::string> input\_string, std::vector\<int64\_t> input\_int64, std::vector\<float> input\_float) | Takes the model inputs as parameters and performa inference to obtain the corresponding results.     |

## 2.1.4 Performance Boosts in Common Applications

This test uses the MySQL as the test program. MySQL is a popular open source relational database management system (RDBMS), widely used in various applications. The AI for Compiler algorithm and framework enable more accurate estimations of critical values during the BOLT optimization phase, assisting the compiler in making more reasonable optimization decisions and enhancing the optimization effect. This ultimately leads to improved MySQL performance. Based on the tpmC throughput test results for MySQL, the AI for Compiler framework, combined with tuning options, can help achieve a 5% performance improvement in MySQL.

# 3 PIN

## 3.1 Feature Description

The Plug-IN (PIN) framework is a plug-in development platform that helps develop plug-ins once but apply to multiple compilers, thereby reducing development costs. It also supports and maintains common capabilities. PIN uses the proxy mode and consists of two key components: server and client. The PIN server bears the plug-in logic and enables developers to develop plug-ins based on the relatively neutral MLIR and a series of plug-in APIs, allowing developers to focus on designing the plug-in logic. The server connects to clients of different compilers, transfers intermediate representations (IRs) and operations through cross-process communication, converts and maps plug-in logic, and executes the logic on the client compiler. In this way, one piece of code can be implemented on multiple compilers.

Plug-in users only need to download the library file and verification file of the required plug-in and enable the plug-in on the compiler client through the PIN configuration file. The PIN client is loaded as a GCC/LLVM plug-in. New functions can be implemented without modifying the source code of the GCC compiler. In this way, users can use the plug-in more flexibly while enjoying various common capabilities supported and maintained by PIN.

## 3.2 Download and Installation

1. Install dependencies.

   ```bash
   ##  Install dependencies for the GCC client.
   yum install -y gityum install -y makeyum install -y cmakeyum install -y grpcyum install -y grpc-develyum install -y grpc-pluginsyum install -y protobuf-develyum install -y jsoncppyum install -y jsoncpp-develyum install -y gcc-plugin-develyum install -y llvm-mliryum install -y llvm-mlir-develyum install -y llvm-devel
   ## Install dependencies for the PIN server.
   yum install -y gityum install -y makeyum install -y cmakeyum install -y grpcyum install -y grpc-develyum install -y grpc-pluginsyum install -y protobuf-develyum install -y jsoncppyum install -y jsoncpp-develyum install -y llvm-mliryum install -y llvm-mlir-develyum install -y llvm-devel
   ```

2. Use either of the following methods to install or build PIN:

   1. Build

      ```bash
      ## Build the GCC client on PIN.
      git clone https://atomgit.com/openeuler/pin-gcc-client.gitcd pin-gcc-clientmkdir buildcd buildcmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH}make
      ## Build the PIN server.
      git clone https://atomgit.com/openeuler/pin-server.gitcd pin-servermkdir buildcd buildcmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH}make
      ```

   2. Install through Yum

      ```bash
      ## Install the GCC client.
      yum install -y pin-gcc-client
      ## Install the PIN server.
      yum install -y pin-server
      ```

# 4 GCC Toolset

## 4.1 Feature Description

To ensure the robustness of the OS, the foundational software selection policy prefers the version that has been verified and is relatively stable instead of the latest version. This policy helps avoid potential instability caused by version changes and ensures that the OS remains relatively stable throughout the long-term support (LTS) cycle. For this reason, GCC 12.3.1 is used as the baseline for openEuler 24.03 LTS development throughout its lifecycle.

Such a selection may cause the following problems: On one hand, many hardware features require the support of the basic GCC toolchain. If the GCC of a non-latest version is selected, new features cannot be enabled on the newly released OS in a timely manner. On the other hand, some users tend to use the latest compiler to enable the latest features for improved performance compared with earlier compilers.

To enable new features of various computing cases and meet different users' requirements for different hardware features, openEuler GCC Toolset comes with openEuler 24.09. This toolchain is designed to support compilation with multiple versions of GCC on openEuler. Typically, it provides a minor GCC that is later than the major GCC of the OS, enabling you to flexibly select the compilation environment. By using openEuler GCC Toolset, you can easily switch between GCC of different versions to make full use of new hardware features and enjoy the improved performance brought by the latest GCC optimization.

## 4.2 Restrictions

In openEuler 24.09, the default major GCC is GCC 12.3.1 and the minor GCC is gcc-toolset-14. This sets up a compilation environment that supports both GCC 12 and GCC 14.

* OS: openEuler 24.09
* Hardware architecture: AArch64/x86\_64

## 4.3 Download and Installation

### 4.3.1 Installing SCL

```bash
yum install scl-utils scl-utils-build
```

### 4.3.2 Installing GCC Toolset

Install gcc-toolset-14 in **/opt/openEuler/gcc-toolset-14/**.

```bash
yum install -y gcc-toolset-14-gcc*
yum install -y gcc-toolset-14-binutils*
```

## 4.4 How to Use

With Software Collections (SCL), you can manage multiple versions of compilation toolchains.

### 4.4.1 Registering gcc-toolset-14

```bash
## Register gcc-toolset-14.
scl register /opt/openEuler/gcc-toolset-14/
##Deregister gcc-toolset-14.
scl deregister gcc-toolset-14
```

Run `scl list-collections`. If gcc-toolset-14 is displayed, the toolset has been registered with SCL.

### 4.4.3 Switching to gcc-toolset-14

Start a new Bash shell session to use gcc-toolset-14 instead of the default GCC. In the new Bash shell session, you do not need to explicitly use the `scl` command.

```bash
scl enable gcc-toolset-14 bash
```

To exit the gcc-toolset-14 compilation environment, enter `exit` to exit the Bash shell session. GCC is switched to the default version.

SCL automatically sets environment variables for different GCC versions. For details, see the **/opt/openEuler/gcc-toolset-14/enable** file, in which the environment variables of gcc-toolset-14 are set. If SCL is unavailable in the OS, switch the toolchain version as follows:

```bash
## Solution 1: Use a script to switch the compilation toolchain if SCL is unavailable.
source /opt/openEuler/gcc-toolset-14/enable

## Solution 2: If SCL is available, use it to switch the compilation toolchain and activate the operating environment.
scl enable gcc-toolset-14 bash
```

## 4.5 Application Scenarios

Major GCC: The default GCC 12.3.1 is used for compilation.

Minor GCC: If the features of GCC 14 are required to build related applications, use SCL to switch the Bash environment to that of gcc-toolset-14.

---

---
url: /zh/docs/22.03_LTS_SP4/server/development/gcc/gcc_user_guide.md
---
# 1 CFGO反馈优化

## 1.1 特性描述

* 概念：CFGO （Continuous Feature Guided Optimization）是多模态（源代码、二进制）、全生命周期（编译、链接、链接后、运行时、OS、库）的持续反馈优化。
* 核心思想：在保证程序功能不变的前提下，通过收集程序运行时信息，指导各种编译优化技术进行更准确的，获得性能更优的目标程序。
* 主要优化点：
  * 代码布局优化：通过基本块重排、函数重排、冷热分区等技术，优化目标程序的二进制布局，提升i-cache和i-TLB命中率。
  * 高级编译器优化：内联、循环展开、向量化、间接调用等提升编译优化技术受益于反馈信息，能够使编译器执行更精确的优化决策。

## 1.2 BOLTUSE

### 1.2.1 选项介绍

#### 选项-fbolt-use

此选项用于直接使用指定的BOLT profile完成链接后优化。使用时需要提前准备好BOLT优化所需要的profile文件，可以使用AutoBOLT模式获取，也可以使用perf采样并执行perf2bolt转换生成。

### 1.2.2 常见应用使能方式

1. Mysql

   ```bash
   cmake .. -DCMAKE_INSTALL_PREFIX=/home/install/mysql-8.0.25 -DBUILD_CONFIG=mysql_release -DWITH_BOOST=../boost -DCMAKE_C_FLAGS="-Wl,-q" -DCMAKE_CXX_FLAGS="-Wl,-q" -DCMAKE_C_LINK_FLAGS="-Wl,-q" -DCMAKE_CXX_LINK_FLAGS="-Wl,-q -fbolt-use=/path/to/mysqld.profile -fbolt-target=mysqld" -DCMAKE_EXE_LINKER_FLAGS="-Wl,-q"
   ```

2. MongoDB

   ```bash
   # 带重定位信息的基线版本构建
   python3 buildscripts/scons.py MONGO_VERSION=6.1.0 DESTDIR=/usr/local/mongo install-all-meta CFLAGS="-march=armv8-a+crc -mtune=generic -Wl,-q" CXXFLAGS="-march=armv8-a+crc -mtune=generic -Wl,-q" LINKFLAGS="-Wl,-q" -j 128 --disable-warnings-as-errors
   # perf采样 + perf2bolt生成bolt profile
   # 使用boltuse构建优化版本，注意如果使用的llvm-bolt版本不支持pie，需要手动在SConstruct中删掉pie选项
   python3 buildscripts/scons.py MONGO_VERSION=6.1.0 DESTDIR=/usr/local/mongo install-devcore CFLAGS="-march=armv8-a+crc -mtune=generic -Wl,-q" CXXFLAGS="-march=armv8-a+crc -mtune=generic -Wl,-q" LINKFLAGS="-Wl,-q -fbolt-use=./mongod.profile -fbolt-target=mongod" -j 128 --disable-warnings-as-errors
   ```

3. Redis

   ```bash
   make CFLAGS="-Wl,-q" LDFLAGS="-fbolt-use=/path/to/redis.profile -fbolt-target=redis-server"
   ```

   **说明**

   * 默认使用当前路径下文件名为data.fdata的profile完成优化，可以使用-fbolt-use=FILE指定使用的profile，如-fbolt-use=/tmp/a.fdata。
   * 可以和-fbolt-target、-fbolt-option=\<options>共用。

## 1.3 AutoBOLT

### 1.3.1 选项介绍

插桩反馈优化场景下，需要经过两次编译，第一次带编译选项-fprofile-generate\[=path]，经过一系列典型场景测试后生成profile数据，第二次带编译选项-fprofile-use\[=path]编译优化版本，AutoBOLT会将插桩数据转换成BOLT profile，并自动使能链接后优化。

#### 适用场景

不介意插桩阶段性能损耗，对构建和测试时间不敏感，追求极致性能等情况

#### 使用方式

```bash
# 插桩，生成gcno
gcc -O3 -fprofile-generate[=path] -fprofile-update=atomic -o test test.c
# 执行测试，获得gcda./test# 使用生成的gcda数据编译生成新的可执行文件
gcc -O3 -fprofile-use[=path] -fprofile-correction -Wno-error=coverage-mismatch -Wno-error=missing-profile -fauto-bolt -fbolt-target=test_pgoed -o test_pgoed test.c
```

### 1.3.2 自动（采样）反馈优化场景

与插桩式反馈优化不同，自动反馈优化使用perf收集程序的运行信息，然后使用create\_gcov工具解析来自perf的采样信息为编译器所需profile，最后使用选项-fauto-profile读取profile完成优化。选项-fprofile-correction用于使能mcf算法，平滑由于采样导致的基本块计数不均衡。

#### 适用场景

频繁发布版本，需要在生产环境部署整套系统等情况。

#### 使用方式

```bash
# 编译带调试信息的可执行文件
gcc -O3 -g -o test_prof test.c
# 使用perf采集性能数据
perf record -e br_retired -- ./test_prof
# 使用create_gcov将perf.data解析为gcov文件
create_gcov --binary=./test_prof --profile=perf.data --gcov=test.gcov -gcov_version=2
# 利用gcov性能数据编译生成新的可执行文件
gcc -O3 -fauto-profile=test.gcov test.c -o test_autofdo
```

#### 补充说明

1. 转换后的profile默认保存在当前路径，可以使用-fauto-bolt=PATH指定BOLT profile的保存路径，如-fauto-bolt=/tmp。
2. 当前插桩反馈优化在内核反馈优化的场景下，支持使用gcov-tool工具合并插桩反馈优化文件以生成对多种场景具有性能提升的高性能内核。
3. 使用插桩反馈优化技术优化多线程应用场景下使用-fprofile-generate\[=path]时可以加上-fprofile-update=atomic，可以生成更精准的数据。
4. 使用插桩反馈优化技术优化应用的场景，使用-fprofile-correction可以自动修正部分profile数据，使用-Wno-error=coverage-mismatch -Wno-error=missing-profile可以屏蔽profile无法对应或不存在的情况下的报错。

#### 典型应用使能方式

1. MySQL/RocksDB等通过Cmake构建的应用

   ```bash
   # 带重定位信息的基线版本构建
   cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DBUILD_CONFIG=mysql_release -DWITH_BOOST=../boost -DCMAKE_C_FLAGS="-Wl,-q" -DCMAKE_CXX_FLAGS="-Wl,-q" -DCMAKE_C_LINK_FLAGS="-Wl,-q" -DCMAKE_CXX_LINK_FLAGS="-Wl,-q" -DCMAKE_EXE_LINKER_FLAGS="-Wl,-q"
   # 使用autobolt构建优化版本
   ```

2. Redis/Memcached等通过Make构建的应用

   ```bash
   # 带重定位信息的基线版本构建
   make CFLAGS="-Wl,-q" CXXFLAGS="-Wl,-q"
   # 优化版本构建
   make CFLAGS="-fprofile-use=./profile -fauto-bolt -fbolt-target=redis-server -Wl,-q -Wno-missing-profile" LDFLAGS="-Wl,-q"
   ```

#### 典型应用效果

| MySQL | openGauss | Nginx | Ceph | Redis | kernel |
| ----- | --------- | ----- | ---- | ----- | ------ |
| +15%  | +5%       | +15%  | +7%  | +5%   | +5%    |

#### 常见使用问题

1. 使用-fauto-bolt和-fbolt-target时出现打印信息: The linker\[2] is not for exec, just skip.22.03sp3，之前的gcc版本不支持对pie或so优化，可以使用sp4及之后版本。

#### AutoBOLT和BOLTUSE的通用选项

* -fbolt-target=NAME用于指定BOLT的优化对象，使用该选项后除了NAME之外的二进制和动态库都不会优化。使用该选项，必须指定NAME。
* -fbolt-option=PARAM用于指定BOLT的优化选项，不同选项以逗号分隔，如-fbolt-option="-reorder-blocks=cache+,-reorder-functions=hfsort+"。使用该选项，必须指定PARAM，优化选项详情可使用llvm-bolt --help查询。

## 1.4 PGO

### 特性说明

该优化通过插桩形式收集程序运行时信息（profile）进行优化决策，编译器更换这些运行时信息知道各种编译优化以进行更准确的优化决策，生成目标程序。

该有化主要包括两种优化方式：

1. Edge Profile：在函数CFG(Control Flow Graph)的边上插入计数器，获取程序的运行计数信息。
2. Value Profile：由于编译器的优化编希望知道一些表达式变量的取值，该优化在插桩过程中插入代码统计表达式变量经常的取值。程序运行后，将profile文件读回编译器，指导对应的优化编执行更佳的优化决策。当前GCC支持多种value profile计数功能，包括变量常用取值、是否是2的幂次、经常的间接调用地址、函数首次运行编号等。

### 1.4.1 选项介绍

#### 选项-fprofile-generate\[=path]

此选项打开了-fprofile-arcs和-fprofile-values。-fprofile-generate可以传入参数path，用来指定profile反馈数据文件的存储文件，同-fprofile-dir。

#### 选项-fprofile-correction

此选项会使用启发性纠错或者平滑得中和profile反馈数据不一致的情况。

#### 选项-Wno-coverage-mismatch

缺省情况下，当profile不匹配源代码时会被作为错误处理。此选项可以用来将错误转换为警告，仅建议在较少改动并且改动部分为冷代码的情况下适用。

#### 选项-fprofile-use\[=path]

此选项适用存储在path下的profile指导编译器进行优化。

### 1.4.2 内核反馈优化

#### 背景

当今大多数服务器只运行单个或少数的特定应用，因此根据特定的运行程序对操作系统进行优化，将获得更多的性能收益。

#### DEMO

1. 下载内核源码。

   ```bash
   yum install -y kernel-source
   ```

2. 编译插装代码。

   进入内核源码目录，进行内核选项配置，这里以openEuler 5.10内核为例。

   ```bash
   cd /usr/src/linux-5.10.0***
   make openeuler_defconfig
   make menuconfig
   在菜单中，进入General setup,选择Local Version-append to kernel release,填入合适的后缀名，例如-test-going双击esc回到上一级目录，进入General architecture-dependent options > GCOV-based kernel profiling,双击esc回到主目录，进入kernel hacking > Compile-time checks and compiler options profiling,关闭Compile the kernel debug，保存并退出。
   ```

3. 编译rpm包（96可替换为机器最大核数）。

   ```bash
   make binrpm-pkg -j 96
   ```

4. 使用插桩内核。

   在服务器端安装插桩内核，设置为默认内核并重启。

   ```bash
   rpm -ivh kernel*** --forcegrub2-set-default 0 reboot
   ```

5. 执行目标优化的应用测试，这里以Nginx的wrk测试套为例。

   在服务器端开Nginx 服务器。

   ```bash
   nginx -c nginx.conf
   ```

   在客户端运行Nginx wrk测试套。

   ```bash
   wrk -c 2000 -d 60s -t 20 --latency --timeout 5s -H "Connection: close" http://192.168.1.10:10000/index.html
   ```

6. 执行profile收集脚本，将profile信息从内存写入磁盘中。

7. （可选）如果需要针对多个应用编译优化内核，可以尝试合并多个应用的profile，这里以应用A和应用B为例。

   ```bash
   gcov-tool merge a/gcovdata b/gcovdata
   ```

8. 编译优化后的内核，进入内核源码目录，进行内核选项配置。

   ```bash
   make openeuler_defconfig
   make menuconfig
   在菜单中，进入General Setup.选在Local version-append to kernel release,填写合适的后缀名，例如-test-pgoed，双击回到主目录，关闭Compile the kernel with debug info，保存并退出。
   ```

9. 编译rpm包（96可替换为机器最大核数）。

   ```bash
   make binrpm-pkg -j 96 KCFLAGS="-fprofile-use -fprofile-correction -Wno-error=coverage-mismatch -Wno-error=missing-profile -fprofile-dir=gcovdata"
   ```

#### 常见使用问题

#### 须知

**Profile** **文件没有正常生成**

* 可能原因一：于-fprofile-generate生成profile的目录下没有写入权限。可以使用chmod 777 \[生成目录的绝对路径方式]解决。
* 可能原因二：程序没有正常退出。在应用退出时会调用\*\*\_\_gcov\_exit()\*\*刷新profile信息。可以使用gdb attach 应用，手动调用\_\_gcov\_exit()解决。

## 1.5 LTO

### 1.5.1 特性介绍

**背景**

在传统编译流程中，gcc 将单个源文件（称谓一个编译单元）直接进行编译优化生成包含汇编代码的.o目标对象文件，并由链接器对这些.o文件进行符号表解析与重定位，链接成可执行文件。在这个过程中，拥有跨文件函数调用信息的链接器由于操作的是汇编代码，难以进行编译优化，而可以执行编译优化的环节，却没有跨文件的全局信息。这样的编译框架，虽然提高了编译效率，每次重新编译只需要编译修改过的少量编译单元，但也丢失了许多跨文件的优化机会。

**LTO流程**

LTO 设计的初衷就是希望能够在链接时，拥有跨编译单元的调用信息的时候，进行编译优化，提供更多的优化机会。为了达到这个目的，LTO 需要将编译优化所需的 IR 信息保留到链接时。在链接时，链接器会调用 LTO 插件，执行全程序分析，生成更加有效的优化决策，再经由编译优化生成更高效的IR，进一步转成包含汇编代码的目标对象文件，最后由链接器完成常规的链接工作。

**分区与并行**

由于需要处理全局的调用图，在执行LTO时往往需要更长的编译时间，有些时候，LTO的编译时间会达到非LTO的数倍。为了使 LTO 能够并行，以加快编译速度，分区模式出现了。在分区模式下，全局调用图会根据模块之间的关联性被拆分成若干个分区，各分区之间独立并行优化，是一种在编译时间和优化效果之间的平衡。

### 1.5.2 使用方式

#### 选项-flto=

编译选项中加入此选项开启LTO。可通过-flto=n指定LTO的并发数，如使用auto可以指定与GNU make一致（如果可以获取）或是与当前机器线程数一致的并发数。

#### 选项-ffat-lto-objects

该选项将生成同时包含汇编信息和LTO信息的目标文件。并且在-ffat-lto-objects开启的情况下，允许通过-fno-lto选项，使用目标对象文件的汇编信息，来完成常规的链接，而不执行链接时优化。

#### 选项-flto-partion=1to1|balanced|max|one|none

该选项可以通过引入不同的分区策略从而加快编译速度。1to1意味着分区与源文件一一对应；balanced控制每个分区在规模上相等；max会为每一个符号创建一个分区；one会尽量生成一个分区；none会直接调过全程序分以及分区的环节。

#### 选项-flto-compresson-level=n

该选项用以控制LTO对性文件中IR信息的压缩等级。

**DEMO**

```bash
// four.cint four() {  return 4;}
// five.cint five() {  return 5;}
// test.cextern int four();extern int five();int main() {  return four() + five();}
gcc -O2 -flto test.c four.c five.c
# 或者
gcc -O2 -flto -c test.cgcc -O2 -flto -c four.cgcc -O2 -flto -c five.cgcc test.o four.o five.o
```

**说明**

对于上面这个案例，在使用 LTO 生成的可执行文件中，four() + five()会被内敛+常量折叠优化为立即数9，省略了函数调用与计算的步骤，这在非 LTO 模式下是无法实现的。 如果仅对four.c和test.c生成了 LTO 对象文件，最终结果便成了4 + five()，仅部分lto对象文件参与了局部范围内的链接时优化。

### 1.5.3 常见使用问题

* plugin needed to handle lto object

  链接器或者相关工具链在处理 lto 对象文件时，需要调用 lto 插件来解析对应 section 的信息。这类报错一般是由于 ar/ranlib/nm 没有调用 lto 插件导致的，可以通过更换为 gcc-ar/gcc-ranlib/gcc-nm 来解决。

* undefined symbol

  这类问题有许多个比较原因，以下举例比较常见的三种：

  一是符号版本控制相关的情况。由于 gcc 不支持 **asm** 形式的符号版本控制，需要在应用源码层面替换为\_\_attribute\_\_(.symver) 形式的符号版本控制。

  二是由于编译过程依赖了 objcopy 等工具，此类工具不支持处理lto对象文件，会导致需要重命名重定义的符号没有重命名，引起符号未定义的问题。

  三是涉及到了链接时插桩(-Wl,--wrap=foo)，在lto下会出现 \_\_wrap\_foo 的符号未定义的问题，该问题上游社区正在解决中。

* multi-defined symbol

  一般是由于编译过程依赖了 objcopy 等工具，此类工具不支持处理lto对象文件，会导致需要隐藏的符号没有隐藏，引起重复定义的问题。

## 1.6 静态编译

### 1.6.1 选项 -fipa-struct-reorg

内存空间布局优化，将结构体成员在内存中的排布进行新的排列组合，来提高cache的命中率。

* 结构体拆分：将结构体的冷热成员单独拆分成一个结构体类型

  **图1** 结构体拆分优化原理示意图

  ![Figure Name:unnaming.png CAD Name:zh-cn\_image\_0000002008223880.png](./figures/p1.png)

  将以下结构体：

  ```bash
  struct S
    {
      type1 field1; 
      // Hot field
      type2 field2;
    };S *v;
  ```

  转化为：

  ```bash
  struct S_hot 
    {
      type1 field1;
    };
  struct S_cold 
    {
      type2 field2;
    };
  S_hot *v_hot;
  S_cold *v_cold;
  ```

* 结构体数组优化：将结构体的数组转化为数组的结构体。

  **图1** 结构体数组优化原理示意图

  ![Figure Name:unnaming.png CAD Name:zh-cn\_image\_0000002008224188.png](./figures/p2.png)

  将以下结构体：

  ```bash
  struct 
    {
      type1 field1;
      type2 field2;
      type3 field3;
    } arr[N];
  ```

  转化为：

  ```bash
  struct 
    {
      type1 field1[N];
      type2 field2[N];
      type3 field3[N];
    } arr;
  ```

#### 使用方法

在编译选项中加入：

```bash
-O3 -flto -flto-partition=one -fipa-struct-reorg
```

**须知**

-fipa-struct-reorg选项，需要在-O3 -flto -flto-partition=one全局同时开启的基础上才使能。

SPEC性能提升效果：SPECCPU2017 intrate 505.mcf子项性能提升20%。

### 1.6.2 选项 -fipa-reorder-fields

内存空间布局优化之结构体重排优化，根据结构体中成员的占用空间大小，将成员从大到小排列，以减少边界对齐引入的padding，来减少结构体整体占用的内存大小，以提高cache的命中率。 将以下结构体

```bash
struct S
  {
    double a;
    int b;double c;
    double d;
    short e;
    double f;
    double g;
    double h;
    double i;
  };
```

经过重排后转化为：

```bash
struct S.reorder
  {
    double a;
    double i;
    double c;
    double d;
    double f;
    double g;
    double h;
    int b;
    short e;
  }
```

#### 适用方法

```bash
在选项中加入-O3 -flto -flto-partition=one -fipa-reorder-fields
```

**须知**

-fipa-reorder-fields选项，需要在-O3 -flto -flto-partition=one全局同时开启的基础上才使能。

SPEC性能提升效果：SPECCPU2017 intrate 505.mcf子项性能提升20%。

### 1.6.3 选项 -fipa-struct-reorg=n

使用该选项控制内存空间布局优化系列优化。

* -fipa-struct-reorg=**0** 不启用任何优化。

* -fipa-struct-reorg=**1** 启用结构体拆分和结构体数组优化，等同使用-fipa-struct-reorg

* -fipa-struct-reorg=**2** 在 -fipa-struct-reorg=1 的基础上，新增结构体成员重排-fipa-reorder-fields

* -fipa-struct-reorg=**3** 在-fipa-struct-reorg=2 的基础上，新增**结构体冗余成员消除优化**。结构体冗余成员消除，消除结构体中从不读取的结构体成员，同时删除冗余的写语句。 将以下结构体：

  ```bash
  struct S
    {
      type1 field1; 
      // Never read in whole program
      type2 field2;};
  ```

  转化为

  ```bash
  struct S.layout 
    {
      type2 field2;
    };
  ```

* -fipa-struct-reorg=**4** 在-fipa-struct-reorg=3的基础上，新增**安全结构体指针压缩优化**。

  结构体指针压缩将结构体域成员中的结构体指针压缩至可选的8、16和32 bits，缩小结构体占用内存大小，降低从内存中读写数据时的带宽压力，从而提升性能。

  * 安全的结构体指针压缩仅支持结构体数组大小在编译期间已知的场景。
  * 使用--param compressed-pointer-size=\[8,16,32]控制压缩目标大小，默认取值为32。

  将以下结构体：

  ```bash
  struct S 
    {
      struct S* field1;
      type2 field2;
    };
  ```

  转化为：

  ```bash
  struct S.layout 
    {
      uint32_t field1.pc;
      type2 field2;
    };
  ```

* -fipa-struct-reorg=**5** 在-fipa-struct-reorg=4的基础上，**放宽了结构体指针压缩优化的应用场景**。

  该等级支持结构体数组大小在编译期间未知的场景，用户需自行确认压缩等级合法，不同等级支持的最大结构体数组大小如下表所示。

  | **指针大小** | **最大支持数组范围**       |
  | ------------ | -------------------------- |
  | 8            | 254（0xff-1）              |
  | 16           | 65534（0xffff-1）          |
  | 32           | 4294967294（0xffffffff-1） |

* -fipa-struct-reorg=**6** 在-fipa-struct-reorg=5的基础上，新增**结构体数组semi-relayout优化**。\*

  semi-relayout在一定范围内，将结构体数组中各结构体的成员打包后重排，提高数据空间局部性，从而提升性能。

  * 通过参数--param semi-relayout-level=\[11,12,13,14,15]控制semi-relayout的重排规模，默认值为13，即semi-relayout将每1024个结构体作为一组进行重排，不足1024的情况下以padding的形式补齐1024个结构体空间；计算方式：(1 << semi-relayout-level) / 8。

  * 将以下结构体数组在内存中的排布形式从连续的结构体按照如下图形式转换。

  **图1** semi-relayout优化原理示意图

  ![Figure Name:unnaming.png CAD Name:zh-cn\_image\_0000002008224188.png](./figures/p3.png)

  ```bash
  struct S 
    {
      long a,
      int b,
      short c
    };
  ```

#### 使用方法

在选项中加入：

```bash
-O3 -flto -flto-partition=one -fipa-struct-reorg=n
```

其中n取值范围为\[0,6]。

**须知**

**-fipa-struct-reorg=n**选项，需要在-O3 -flto -flto-partition=one全局同时开启的基础上才使能。

### 1.6.4 选项 -ftree-slp-transpose-vectorize

该选项在循环拆分阶段，增强对存在连续访存读的循环的数据流分析能力，通过插入临时数组拆分循环；SLP矢量化阶段，新增对grouped\_stores进行转置的SLP分析。

```bash
int foo (unsigned char *oxa, int ia, unsigned char *oxb, int ib)
  {
    unsigned tmp[4][4];
    unsigned a0, a1, a2, a3;int sum = 0;
    for (int i = 0; i < 4; i++, oxa += ia, oxb += ib)
      {
        a0 = (oxa[0] - oxb[0]) + ((oxa[4] - oxb[4]) << 16);
        a1 = (oxa[1] - oxb[1]) + ((oxa[5] - oxb[5]) << 16);
        a2 = (oxa[2] - oxb[2]) + ((oxa[6] - oxb[6]) << 16);
        a3 = (oxa[3] - oxb[3]) + ((oxa[7] - oxb[7]) << 16);
        int t0 = a0 + a1;int t1 = a0 - a1;int t2 = a2 + a3;
        int t3 = a2 - a3;tmp[i][0] = t0 + t2;
        tmp[i][2] = t0 - t2;tmp[i][1] = t1 + t3;
        tmp[i][3] = t1 - t3;
      }
    for (int i = 0; i < 4; i++)
      {
        int t0 = tmp[0][i] + tmp[1][i];
        int t1 = tmp[0][i] - tmp[1][i];
        int t2 = tmp[2][i] + tmp[3][i];
        int t3 = tmp[2][i] - tmp[3][i];
        a0 = t0 + t2;
        a2 = t0 - t2;
        a1 = t1 + t3;
        a3 = t1 - t3;
        sum += a0 + a1 + a2 + a3;
      }
    return sum;
  }
```

对于如上所示的用例，针对第一个for循环，可以拆分为如下形式

```bash
for (int i = 0; i < 4; i++, oxa += ia, oxb += ib)
  {
    a00[i] = (oxa[0] - oxb[0]) + ((oxa[4] - oxb[4]) << 16);
    a11[i] = (oxa[1] - oxb[1]) + ((oxa[5] - oxb[5]) << 16);
    a22[i] = (oxa[2] - oxb[2]) + ((oxa[6] - oxb[6]) << 16);
    a33[i] = (oxa[3] - oxb[3]) + ((oxa[7] - oxb[7]) << 16);
  }
for (int i = 0; i < 4; i++)
  {
    int t0 = a00[i] + a11[i];
    int t1 = a00[i] - a11[i];
    int t2 = a22[i] + a33[i];
    int t3 = a22[i] - a33[i];
    tmp[i][0] = t0 + t2;
    tmp[i][2] = t0 - t2;
    tmp[i][1] = t1 + t3;
    tmp[i][3] = t1 - t3;
  }
```

再针对于拆分所得的第一个循环，等号右边的计算同构且为连续load可以矢量化，但是由于左边a00\[i]、a11\[i]、a22\[i]、a33\[i]的内存地址不连续，无法作为矢量化SLP树的根节点，因此失去这种场景的矢量化机会。 在a00\[i]、a11\[i]、a22\[i]、a33\[i]写入内存时，期望寄存器中的内容为：

| **register** | **values**                  |
| ------------ | --------------------------- |
| vec0         | a00\[0] a00\[1] a00\[2] a00\[3] |
| vec1         | a11\[0] a11\[1] a11\[2] a11\[3] |
| vec2         | a22\[0] a22\[1] a22\[2] a22\[3] |
| vec3         | a33\[0] a33\[1] a33\[2] a33\[3] |

而每次迭代内可以计算得到寄存器的内容为：

| **register** | **values**                  |
| ------------ | --------------------------- |
| vec0         | a00\[0] a11\[0] a22\[0] a33\[0] |
| vec1         | a00\[1] a11\[1] a22\[1] a33\[1] |
| vec2         | a00\[2] a11\[2] a22\[2] a33\[2] |
| vec3         | a00\[3] a11\[3] a22\[3] a33\[3] |

将grouped\_store进行转置，即可得到预期的SLP树根节点。再利用SLP原有的能力，进行后续的矢量化分析。

此外，针对拆分所得的第二个循环以及示例中的最后一个循环，tmp二维数组存在写入内存后立刻读取的行为，针对这种场景，将访存行为优化为寄存器之间的permutation行为。该访存优化默认开启。

#### 使用方法

在选项中加入：

```bash
-O3 -ftree-slp-transpose-vectorize
```

**须知**

**-ftree-slp-transpose-vectorize**选项，需要在-O3开启的基础上才使能。

### 1.6.5 选项 -fccmp2

该选项是Arm相关指令优化：增强ccmp指令使用场景，简化指令流水

针对如下代码场景：

```bash
int f(int a, int b, int c) {  while(1) {    if (a--==0||b>=c){      return 1;    }  }}
```

成功使能ccmp指令，可以生成如下汇编代码：

**图1** 汇编包含ccmp指令示意图

![Figure Name:unnaming.png CAD Name:zh-cn\_image\_0000002044503909.png](./figures/p4.png)

#### 使用方法

增加编译选项 -fccmp2

**须知**

SPEC性能提升效果：SPECCPU2017 intrate 557.xz\_r子项性能提升约1%

### 1.6.6 选项 -farray-widen-compare

数组宽比较优化：支持使用宽数据类型对原数组指针（指向的数组元素为窄类型）解引用，达到一次比较多个元素的效果，从而提高程序性能。

针对如下代码场景：

```bash
#define my_min(x, y) ((x) < (y) ? (x) : (y))
uint32_t func (uint32_t n0, uint32_t n1, const uint32_t limit, const uint8_t * a, const uint8_t * b)
  {
    uint32_t n = my_min(n0, n1);  
    while (++n != limit)
    if (a[n] != b[n])
      break;  
    return n;}
```

添加选项后，可以优化为如下代码：

```bash
#define my_min(x, y) ((x) < (y) ? (x) : (y))
uint32_t func (uint32_t n0, uint32_t n1, const uint32_t limit, const uint8_t * a, const uint8_t * b)
  {
    uint32_t n = my_min(n0, n1);  
    for (++n; n + sizeof(uint64_t) <= limit; n += sizeof(uint64_t))  
      {
        uint64_t k1 = *((uint64_t*)(a+n));
        uint64_t k2 = *((uint64_t*)(b+n));
        if(k1 != k2)
          {
            int lz = __builtin_ctzll(k1 ^ k2);       
            n += lz/8;       
            return n;     
           }    
       }    
     for (;n != limit; ++n)    
       {      
         if (a[n] != b[n])        
           break;     
       }     
    return n;
  }
```

#### 使用方法

增加编译选项-O3 -farray-widen-compare

# 2 AI For Compiler

## 2.1 采样反馈优化能力增强

### 2.1.1 特性描述

AI4Compiler通过相关算法及构建框架，可以在进一步增强采样反馈优化能力特性，更准确的预估编译优化过程中的关键数值，辅助编译器做出更合理的优化决策，提升编译优化效果。AI4Compiler通过代码表征学习等AI4Compiler算法，训练BOLT采样的基本块精度修正模型，构建框架集成至GCC for OpenEuler，实现性能提升。

BOLT通过改进二进制文件中的代码和数据布局来减少CPU的缓存未命中率和分支预测错误，从而在编译器在应用了链接时优化（LTO）以及反馈驱动优化（FDO）后提供进一步的优化。BOLT在利用动态profiling数据提高程序的运行性能时有两种不同的方式：即插桩和采样。插桩方式性能开销大但精度较高，采样方式性能开销微乎其微但精度不及插桩。而BOLT采样的基本块精度修正模型则力图通过挖掘一个BB块的内含指令/变量、所处函数/文件等多粒度代码信息来对其Count值进行预测，从而提高CFG的准确性，使采样BOLT达到接近插装BOLT的优化效果。

### 2.1.2 下载安装步骤

1. 使用root权限，安装rpmbuild、rpmdevtools

   ```bash
   ##  安装rpmbuild
   yum install rpm-build
   ##  安装rpmdevtools
   yum install rpmdevtools
   ```

2. 从目标仓库拉取代码

   ```bash
   git clone https://atomgit.com/src-openeuler/AI4C.git
   ```

3. 生成rpmbuild文件夹

   ```bash
   rpmbuild-setuptree将拉取代码中的AI4C-v0.2.0-alpha.tar.gz和相关patch文件放入rpmbuild中的SOURCES文件夹中，将AI4C.spec放入rpmbuild中的SPECS目录下。
   ```

4. 安装AI4C

   ```bash
   rpm -ivh \<成功构建的rpm>
   ```

   （若系统因存有旧版本安装包而导致的文件冲突，可以在rpm -ivh <成功构建的rpm>命令中添加一行--force选项，强制安装新版本；或者通过rpm -Uvh <成功构建的rpm>命令更新安装包）。

5. 待安装完成后，可在/usr/lib64下找到相关动态库，在/usr/lib64/AI4C目录下找到相关模型。

### 2.1.3 使用步骤

AI4C支持用户调用API使用自定义模型进行推理，也支持借助编译选项使用预设模型使用采样反馈优化能力增强特性和优化选项调优特性。

使用自定义模型推理：

1. 开发者首先需要构建自己的模型并借助onnxruntime保存成一个ONNX模型。

2. 用户需要在目标优化遍的适配层里调用AI4C提供的API，进行模型的推理，如下表所示。下表的接口存在动态库libONNXRunner.so中，用于模型的推理配置阶段、推理运行阶段和模型推理结束后的资源清理阶段。

   | API名                                                        | 说明                                                         |
   | ------------------------------------------------------------ | ------------------------------------------------------------ |
   | `extern ONNXRunner* createONNXRunner(const char* model_path)`  | 参数为模型存放的路径，并创建session对本次推理进行初始化和配置 |
   | `extern void deleteONNXRunner(ONNXRunner* instance)`           | 删除ONNXRunner对象                                           |
   | `extern float runONNXModel(ONNXRunner* instance, std::vector&lt;std::string&gt; input_string, std::vector&lt;int64_t&gt;input_int64, std::vector&lt;float&gt; input_float)` | 参数为模型的输入，将输入传入模型进行推理得到相应结果         |

### 2.1.4 常用应用提升效果

本次测试使用MySQL应用程序作为测试程序。MySQL是一种流行的开源关系型数据库管理系统（RDBMS），广泛应用于各种应用中。通过AI4Compiler算法及框架，可以在BOLT优化阶段更准确的预估编译优化过程中的关键数值，辅助编译器做出更合理的优化决策，提升编译优化效果，从而实现MySQL性能提升。根据对MySQL的运行tpmC吞吐量测试结果，AI4Compiler框架结合调优选项可以帮助MySQL性能提升5%。

# 3 插件框架

## 3.1 特性描述

编译器插件框架致力于让开发者减少开发成本，即基于编译能力的相关工具开发者只需要做一次开发，便可在多个不同编译器框架落地。同时编译器插件框架作为开发平台，也提供对公共能力的支持与维护。编译器插件框架采用代理模式，由服务端和客户端两个关键组件构成。插件服务端专注于承载插件逻辑，允许开发者基于相对中立的MLIR和我们提供的一系列插件API进行开发，使开发者能够将关注点聚焦于工具的设计逻辑上。服务端与不同编译器的客户端对接，通过跨进程通信传递IR数据与操作，将插件逻辑转换映射，并最终在客户端编译器上执行，从而实现了一份代码可以在多个编译器上落地的目的。

插件使用者只需要下载所需插件的库文件和校验文件，通过框架的配置文件即可与编译器客户端联合使能。插件客户端将作为GCC/LLVM Plugin进行加载，具有不需要修改GCC编译器源代码即可实现新功能的优势，使得用户能够更加灵活和便捷地使用插件，同时享受框架支持与维护的各种公共能力。

## 3.2 下载安装步骤

1. 安装依赖软件

   ```bash
   ##  安装GCC客户端依赖软件：
   yum install -y gityum install -y makeyum install -y cmakeyum install -y grpcyum install -y grpc-develyum install -y grpc-pluginsyum install -y protobuf-develyum install -y jsoncppyum install -y jsoncpp-develyum install -y gcc-plugin-develyum install -y llvm-mliryum install -y llvm-mlir-develyum install -y llvm-devel
   ## 安装插件框架服务端依赖软件
   yum install -y gityum install -y makeyum install -y cmakeyum install -y grpcyum install -y grpc-develyum install -y grpc-pluginsyum install -y protobuf-develyum install -y jsoncppyum install -y jsoncpp-develyum install -y llvm-mliryum install -y llvm-mlir-develyum install -y llvm-devel
   ```

2. 安装/构建插件框架，可以采用以下两种方式进行

   1. 编译构建

      ```bash
      ## 构建插件框架GCC客户端
      git clone https://atomgit.com/openeuler/pin-gcc-client.gitcd pin-gcc-clientmkdir buildcd buildcmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH}make 
      ## 构建插件框架服务端
      git clone https://atomgit.com/openeuler/pin-server.gitcd pin-servermkdir buildcd buildcmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH}make
      ```

   2. yum安装

      ```bash
      ## yum安装插件框架GCC客户端
      yum install -y pin-gcc-client
      ## yum安装插件框架服务端
      yum install -y pin-server
      ```

# 4 多版本GCC

## 4.1 特性描述

为确保操作系统的稳健性，基础软件的选型策略通常倾向于采用经过时间验证、相对稳定的版本，而非最新发布版本。这一策略旨在避免版本更迭带来的潜在不稳定因素，确保在整个长期支持（LTS）周期内，系统版本保持相对稳定。因此，当前 openEuler 在 24.03 LTS 版本整个生命周期都是选择使用 GCC 12.3.1 作为基线进行开发。

这样的选择会带来如下问题。首先，许多的硬件特性需要基础 GCC 工具链的支持，选择非最新版本的 GCC 会导致新特性无法及时在新发布的操作系统上使能。另外，某些用户倾向使用最新版本的编译器使能最新特性，这些特性相较于低版本编译器会带来部分性能提升。

因此，为了使能多样算例新特性，满足不同用户对不同硬件特性支持的需求，在 openEuler 24.09 版本推出 openEuler GCC Toolset 工具链，这是一个专为 openEuler 系统设计的 GCC 多版本编译工具链，该工具链提供一个高于系统主 GCC 版本的副版本 GCC 编译工具链，为用户提供了更加灵活且高效的编译环境选择。通过使用 openEuler GCC Toolset 多版本编译工具链，用户可以轻松地在不同版本的 GCC 之间进行切换，以便充分利用新硬件特性，同时享受到 GCC 最新优化所带来的性能提升。

## 4.2 使用约束

在默认 GCC 版本为 GCC 12.3.1 的 openEuler24.09 版本中，提供 gcc-toolset-14 的副版本编译工具链，形成主编译器版本为 GCC 12，副版本编译器为 GCC 14 的搭配；

* 操作系统：openEuler 24.09
* 硬件架构：Aarch64 / X86\_64

## 4.3 下载安装步骤

### 4.3.1 安装scl

```bash
yum install scl-utils scl-utils-build
```

### 4.3.2 安装多版本工具链

副版本编译工具链 gcc-toolset-14 安装路径为 /opt/openEuler/gcc-toolset-14/：

```bash
yum install -y gcc-toolset-14-gcc*
yum install -y gcc-toolset-14-binutils*
```

## 4.4 使用方式

本方案引入 SCL（Software Collections）工具进行不同版本编译工具链的管理。

### 4.4.1 注册 gcc-toolset-14

```bash
## 注册gcc-toolset-14
scl register /opt/openEuler/gcc-toolset-14/
##取消注册 gcc-toolset-14
scl deregister gcc-toolset-14
使用scl list-collections显示 gcc-toolset-14 表明已经在 scl 中注册成功；
```

### 4.4.2 切换 gcc-toolset-14

启动一个新的bash shell会话，其中使用 gcc-toolset-14 内的工具版本，而不是系统默认版本。在新的bash shell会话中，无需显式使用scl命令。

```bash
scl enable gcc-toolset-14 bash
```

如果需要退出 gcc-toolset-14 的编译环境，输入exit退出bash shell会话，此时gcc的版本切换成系统默认版本。

SCL工具的本质就是自动设置不同工具版本的环境变量，具体可以参考 /opt/openEuler/gcc-toolset-14/enable 文档，gcc-toolset-14 的环境变量均在该文件中设置。若用户系统没有 SCL 工具，则可以使用以下方式进行工具链版本切换：

```bash
## 方案一：无 SCL 工具，使用脚本切换编译工具链
source /opt/openEuler/gcc-toolset-14/enable

## 方案二：有 SCL 工具，使用 SCL 工具切换编译工具链并激活运行环境
scl enable gcc-toolset-14 bash
```

## 4.5 使用场景

主版本场景：正常编译使用系统默认的 gcc-12.3.1；

副版本场景：需要使用 GCC-14 高版本特性构建相关应用，使用 SCL 工具将 bash 环境切换为 gcc-toolset-14 编译工具链的编译环境。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/diversified_computing/dpu_offload/libvirt_direct_connection_aggregation_environment_establishment.md
---
# 1 Hardware Preparation

## Test Mode

Prepare two physical machines (VMs have not been tested) that can communicate with each other.

One physical machine functions as the DPU, and the other functions as the host. In this document, DPU and HOST refer to the two physical machines.

> \[!NOTE] **NOTE:**
> In the test mode, network ports are exposed without connection authentication, which is risky and should be used only for internal tests and verification. Do not use this mode in the production environment.
> In the production environment, use closed communication to prevent external connections, such as the vsock mode.

## vsock mode

The DPU and HOST are required. The DPU must be able to provide vsock communication through virtio.

This document describes only the test mode usage. If vsock communication is supported in the test environment (virtual environment or DPU-HOST environment that supports vsock), the following test procedure is applicable, except that you need to change the IP addresses to the vsock CIDs (**TEST\_MODE** is not required for binary file compilation).

# 2 libvirt offload architecture

![arch](./figures/arch.png)

# 3 Environment Setup

## 3.1 qtfs File System Deployment

For details, visit <https://atomgit.com/openeuler/dpu-utilities/tree/master/qtfs>.

To establish a qtfs connection, you need to disable the firewall between the DPU and HOST, or open related network ports on the firewall.

## 3.2 Deploying the udsproxyd Service

### 3.2.1 Introduction

udsproxyd is a cross-host Unix domain socket (UDS) proxy service, which needs to be deployed on both the host and DPU. The udsproxyd components on the host and dpu are peers. They implement seamless UDS communication between the host and DPU, which means that if two processes can communicate with each other through UDSs on the same host, they can do the same between the host and DPU. The code of the processes does not need to be modified, only that the client process needs to run with the **LD\_PRELOAD=libudsproxy.so** environment variable. As a cross-host Unix socket service, udsproxyd can be used by running with `LD_PRELOAD=libudsproxy.so`. With the support of qtfs, udsproxyd can also be used transparently. You need to configure the allowlist in advance. The specific operations are described later.

### 3.2.2 Deploying udsproxyd

Build udsproxyd in the dpu-utilities project:

```bash
cd qtfs/ipc
make -j UDS_TEST_MODE=1 && make install
```

> If the vsock mode is used, you do not need to set **UDS\_TEST\_MODE** during compilation.

The engine service on the qtfs server has incorporated the udsproxyd feature. You do not need to manually start udsproxyd if the qtfs server is deployed. However, you need to start udsproxyd on the client by running the following command:

```bash
nohup /usr/bin/udsproxyd <thread num> <addr> <port> <peer addr> <peer port> 2>&1 &
```

Parameters:

```bash
thread num: number of threads. Currently, only one thread is supported.
addr: IP address of the host. If the vsock communication mode is used, the value is the CID.
port: Port used on the host.
peer addr: IP address of the udsproxyd peer. If the vsock communication mode is used, the value is the CID.
peer port: port used on the udsproxyd peer.
```

Example:

```bash
nohup /usr/bin/udsproxyd 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 &
```

If the qtfs engine service is not started, you can start udsproxyd on the server to test udsproxyd separately. Run the following command:

```bash
nohup /usr/bin/udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 &
```

### 3.2.3 Using udsproxyd

#### 3.2.3.1 Using udsproxyd Independently

When starting the client process of the Unix socket application that uses the UDS service, add the **LD\_PRELOAD=libudsproxy.so** environment variable to intercept the **connect** API of glibc for UDS interconnection. In the libvirt offload scenario, you can copy **libudsproxy.so**, which will be used by the libvirtd service, to the **/usr/lib64** directory in the chroot directory of libvirt.

#### 3.2.3.2 Using the udsproxyd Service Transparently

Configure the UDS service allowlist for qtfs. The allowlist is the sock file address bound to the Unix socket server. For example, the files of the Unix socket server created by the libvirt VM are in the **/var/lib/libvirt** directory. In this case, add the directory path to the allowlist in either of the following ways:

* Load the allowlist by using the `qtcfg` utility. First compile the utility in **qtfs/qtinfo**.

Run the following command on the qtfs client:

```bash
make role=client 
make install
```

Run the following command on the qtfs server:

```bash
make role=server
make install
```

After `qtcfg` is installed automatically, run `qtcfg` to configure the allowlist. Assume that **/var/lib/libvirt** needs to be added to the allowlist:

```bash
qtcfg -w udsconnect -x /var/lib/libvirt/
```

Query the allowlist:

```bash
qtcfg -w udsconnect -z
```

Delete an allowlist entry:

```bash
qtcfg -w udsconnect -y 0
```

The parameter is the index number listed when you query the allowlist.

* Add an allowlist entry through the configuration file. The configuration file needs to be set before the qtfs or qtfs\_server kernel module is loaded. The allowlist is loaded when the kernel modules are initialized.

> \[!NOTE] **NOTE:**
> The allowlist prevents irrelevant Unix sockets from establishing remote connections, causing errors or wasting resources. You are advised to set the allowlist as precisely as possible. For example, in this document, **/var/lib/libvirt** is set in the libvirt scenario. It would be risky to directly add **/var/lib**, **/var**, or the root directory.

## 3.3 rexec Service Deployment

### 3.3.1 Introduction

rexec is a remote execution component developed using the C language. It consists of the rexec client and rexec server. The server is a daemon process, and the client is a binary file. After being started, the client establishes a UDS connection with the server using the udsproxyd service, and the server daemon process starts a specified program on the server machine. During libvirt virtualization offload, libvirtd is offloaded to the DPU. When libvirtd needs to start the QEMU process on the HOST, the rexec client is invoked to remotely start the process.

### 3.3.2 Deploying rexec

#### 3.3.2.1 Configuring the Environment Variables and Allowlist

Configure the rexec server allowlist on the host. Put the **whitelist** file in the **/etc/rexec** directory, and change the file permission to read-only.

```bash
chmod 400 /etc/rexec/whitelist
```

In the test environment, the allowlist is not mandatory. You can disable the allowlist by deleting the **whitelist** file and restarting the rexec\_server process.

After downloading the dpu-utilities code, go to the **qtfs/rexec** directory and run `make && make install` to install all binary files required by rexec (**rexec** and **rexec\_server**) to the **/usr/bin** directory.

Before starting the rexec\_server service on the server, check whether the **/var/run/rexec** directory exists. If not, create it.

```bash
mkdir /var/run/rexec
```

The underlying communication of the rexec service uses Unix sockets. Therefore, cross-host communication between rexec and rexec\_server depends on the udsproxyd service, and the related files need to be added to the udsproxy allowlist.

```bash
qtcfg -w udsconnect -x /var/run/rexec
```

#### 3.3.2.2 Starting the Service

You can start the rexec\_server service on the server in either of the following ways.

* Method 1:
  Configure rexec as a systemd service.

Add the **[rexec.service](./config/rexec.service)** file to **/usr/lib/systemd/system**.

Then, use `systemctl` to manage the rexec service.

Start the service for the first time:

```bash
systemctl daemon-reload

systemctl enable --now rexec
```

Restart the service:

```bash
systemctl stop rexec

systemctl start rexec
```

* Method 2:
  Manually start the service in the background.

```bash
nohup /usr/bin/rexec_server 2>&1 &
```

## 3.4 libvirt Service Deployment

### 3.4.1 Deploying on the HOST

Install the VM runtime and libvirt. (libvirt is installed to create related directories.)

```bash
yum install -y qemu libvirt edk2-aarch64 # (required for starting VMs in the Arm environment)
```

Put the VM image on the HOST. The VM image will be mounted to the client through qtfs and shared with libvirt.

### 3.4.2 Deploying on the DPU

#### 3.4.2.1 Creating the Chroot Environment

(a) Download the QCOW image from the openEuler official website, for example, openEuler 22.03 LTS: <https://repo.openeuler.org/openEuler-22.03-LTS/virtual_machine_img/>.

(b) Mount the QCOW2 image.

```bash
cd /root/

mkdir p2 new_root_origin new_root

modprobe nbd maxport=8

qemu-nbd -c /dev/nbd0 xxx.qcow2

mount /dev/nbd0p2 /root/p2

cp -rf /root/p2/* /root/new_root_origin/

umount /root/p2

qemu-nbd -d /dev/nbd0
```

(c) Now, the root directory of the image is decompressed in **new\_root\_origin**. Bind mount **new\_root** to **new\_root\_origin** as the mount point for chroot.

```bash
mount --bind /root/new_root_origin /root/new_root
```

#### 3.4.2.2 Installing libvirt

Compile the source code with a patch.

(a) Go to the chroot environment and install the compilation environment and common tools.

```bash
yum groupinstall "Development tools" -y
yum install -y vim meson qemu qemu-img strace edk2-aarch64 tar
```

**edk2-aarch64** is required for starting VMs in the Arm environment.

(b) Install the dependency packages required for libvirt compilation.

```bash
    yum install -y rpcgen python3-docutils glib2-devel gnutls-devel libxml2-devel libpciaccess-devel libtirpc-devel yajl-devel systemd-devel dmidecode glusterfs-api numactl
```

(c) Download the libvirt-6.9.0 source code package <https://libvirt.org/sources/libvirt-6.9.0.tar.xz>.

(d) Obtain the libvirt patch:

<https://atomgit.com/openeuler/dpu-utilities/tree/master/usecases/transparent-offload/patches/libvirt>.

(e) Decompress the source code package to a directory in the chroot environment, for example, **/home**, and apply the patch.

(f) Go to the **libvirt-6.9.0** directory and run the following command:

```bash
meson build --prefix=/usr -Ddriver_remote=enabled -Ddriver_network=enabled -Ddriver_qemu=enabled -Dtests=disabled -Ddocs=enabled -Ddriver_libxl=disabled -Ddriver_esx=disabled -Dsecdriver_selinux=disabled -Dselinux=disabled
```

(g) Complete the installation.

```bash
ninja -C build install
```

#### 3.4.2.3 Starting the libvirtd Service

To use libvirt direct connection aggregation, you need to start the libvirtd service in the chroot environment, which requires the libvirtd service outside the chroot environment to be stopped.

(a) Put the [VM jumper script](./scripts/qemu-kvm) in **/usr/bin** and **/usr/libexec** in the chroot environment to replace the **qemu-kvm** binary file. The jumper script will call rexec to start a remote VM.

> \[!NOTE] **NOTE:**
> In the XML file of virsh, set **\<emulator>** under **\<devices>** to **qemu-kvm**. If you set **\<emulator>** to another value, change it to **qemu-kvm** or replace the binary file specified by **\<emulator>** with the jumper script. The content of the jumper script also needs to be modified accordingly.

(b) Copy the **libudsproxy.so** file generated during udsproxyd compilation to the **/usr/lib64** directory in the chroot directory. If the udsproxyd service is used by configuring the UDS allowlist of qtfs, you do not need to copy the **libudsproxy.so** file.

(c) Save the **rexec** binary file generated during rexec compilation to the **/usr/bin** directory of the chroot environment.

(d) To configure the chroot mounting environment, you need to mount some directories. Use the following scripts:

* [virt\_start.sh](./scripts/virt_start.sh) is the configuration script. In the script, you need to manually change the **qtfs.ko** path to the path of the compiled **.ko** file and set the correct HOST IP address.
* [virt\_umount.sh](./scripts/virt_umount.sh) is the configuration revert script.

(e) The mount directories in the script are based on the examples in this document. You can modify the paths in the script as required.

(f) After the chroot environment is configured, enter the chroot environment and manually start libvirtd.

If qtfs is not configured to use the udsproxyd allowlist, run the following commands:

```bash
LD_PRELOAD=/usr/lib64/libudsproxy.so virtlogd -d
LD_PRELOAD=/usr/lib64/libudsproxy.so libvirtd -d
```

If qtfs is configured to use the udsproxyd allowlist, the LD\_PRELOAD prefix is not required:

```bash
virtlogd -d
libvirtd -d
```

To check whether the allowlist is configured, run the following command in another terminal that is not in the chroot environment:

```bash
qtcfg -z
```

Check whether the allowlist contains **/var/lib/libvirt**.

## 3.5 VM Startup

After the service is deployed, you can manage the VM life cycle from the DPU.

### 3.5.1 Defining the VM

(a) Place the VM boot image in a directory on the HOST, for example:

```bash
/home/VMs/Domain_name
```

(b) Use qtfs to mount the directory to the DPU.

```bash
mount -t qtfs /home/VMs /home/VMs
```

(c) In the XML file, **/home/VMs/Domain\_name** is used as the boot image. In this way, the same image file is presented to the DPU and HOST (**Domain\_name** is the VM **domain**).

(d) Check whether **\<emulator>** in the XML file points to the jumper script.

(e) Define the VM.

```bash
virsh define xxx.xml
```

### 3.5.2 Starting the VM

```bash
virsh start domain
```

# 4 Environment Reset

Some libvirt directories are shared between the DPU and the HOST. Therefore, you need to unmount these directories before uninstalling the environment. Generally, stop the libvirtd and virtlogd processes and run the **virt\_umount.sh** script. If a VM is running on the HOST, stop the VM before unmounting the directories.

# 5 Common Errors

1. libvirt compilation failure: Check whether the dependency packages are installed. If an external directory or HOST directory is mounted to the chroot environment, the compilation may fail. In this case, unmount the directory first.

2. qtfs mounting failure: The engine process on the server is not started or the firewall is not disabled. As a result, the qtfs connection fails.

3. VM definition failure: Check whether the emulator in the XML file points to the jumper script, whether the VM image has been mounted to the DPU through qtfs, and whether the path is the same as that on the HOST.

4. VM startup failure: Check whether the libvirtd and virtlogd services are started, whether the rexec service is started, whether the jumper process is started, and whether an error is reported when qemu-kvm is started.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/diversified_computing/dpu_offload/container_management_plane_direct_connection_aggregation_environment_establishment.md
---
# 1 Imperceptible Container Management Plane Offload

## 1.1 Overview

The container management plane refers to container management tools such as dockerd, containerd, and isulad. Container management plane offload means to offload the container management plane from the HOST where the container is located to the DPU.

We use qtfs to mount some directories related to container running on the HOST to the DPU so that the container management plane can access these directories to prepare the environment required for container running. In addition, the remote proc and sys file systems need to be mounted. To avoid impact on the current system, you can create a dedicated rootfs (referred to as **/another\_rootfs**) as the running environment of dockerd and containerd.

The `rexec` command is used to start or delete containers, allowing the management plane and containers to be separated on different machines for remote management. You can use either of the following modes to verify the offload.

## 1.1.1 Test Mode

Prepare two physical machines or VMs that can communicate with each other.

One physical machine functions as the DPU, and the other functions as the host. In this document, DPU and HOST refer to the two physical machines.

> \[!NOTE] **NOTE:**
> In the test mode, network ports are exposed without connection authentication, which is risky and should be used only for internal tests and verification. Do not use this mode in the production environment.
> In the production environment, use closed communication to prevent external connections, such as the vsock mode.

## 1.1.2 vsock Mode

The DPU and HOST are required and must be able to provide vsock communication through virtio.

This document describes only the test mode usage. If vsock communication is supported in the test environment (virtual environment or DPU-HOST environment that supports vsock), the following test procedure is applicable, except that you need to change the IP addresses to the vsock CIDs (**TEST\_MODE** is not required for binary file compilation).

# 2 Environment Setup

## 2.1 qtfs File System Deployment

For details, see [qtfs](https://atomgit.com/openeuler/dpu-utilities/tree/master/qtfs).

> \[!NOTE] **NOTE:**
> If the test mode is used, set **qtfs\_TEST\_MODE** to 1 when compiling the **.ko** files on the qtfs client and server. If the vsock mode is used, you do not need to set **qtfs\_TEST\_MODE**.

To establish a qtfs connection, you need to disable the firewall between the DPU and HOST, or open related network ports on the firewall.

## 2.2 udsproxyd Service Deployment

### 2.2.1 Overview

udsproxyd is a cross-host Unix domain socket (UDS) proxy service and needs to be deployed on the HOST and DPU. The udsproxyd components are in a peer relationship. Their respective processes on the host and DPU can communicate with each other transparently using the standard UDSs. That is, if the two processes communicate with each other through UDSs on the same host, they can also communicate with each other between the HOST and DPU without the need for modifying code. As a cross-host Unix socket service, udsproxyd can be used by running with `LD_PRELOAD=libudsproxy.so` or configuring the udsconnect allowlist in advance. The methods for configuring the allowlist are described later.

### 2.2.2 Deploying udsproxyd

Compile udsproxyd in the dpu-utilities project.

```bash
cd qtfs/ipc

make -j UDS_TEST_MODE=1 && make install
```

> \[!NOTE] **NOTE:**
> If the vsock mode is used, you do not need to set **UDS\_TEST\_MODE** during compilation.

The latest engine service on the qtfs server has integrated the udsproxyd capability. Therefore, you do not need to start the udsproxyd service on the server. Start the udsproxyd service on the client.

```bash
nohup /usr/bin/udsproxyd <thread num> <addr> <port> <peer addr> <peer port> 2>&1 &
```

Parameters:

```bash
thread num: number of threads. Currently, only one thread is supported.

addr: IP address of the host. If the vsock communication mode is used, the value is the CID.

port: Port used on the host.

peer addr: IP address of the udsproxyd peer. If the vsock communication mode is used, the value is the CID.

peer port: port used on the udsproxyd peer.
```

Example:

```bash
nohup /usr/bin/udsproxyd 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 &
```

If the qtfs engine service is not started and you want to test udsproxyd separately, start udsproxyd on the server.

```bash
nohup /usr/bin/udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 &
```

### 2.2.3 Using udsproxyd

#### 2.2.3.1 Using the udsproxyd Service Independently

When starting the client process of the Unix socket application that uses the UDS service, add the **LD\_PRELOAD=libudsproxy.so** environment variable to intercept the **connect** API of glibc for UDS interconnection. Alternatively, run the `qtcfg` command to configure the udsconnect allowlist to instruct the system to take over UDS connections in specified directories.

#### 2.2.3.2 Using the udsproxyd Service Transparently

Configure the allowlist of the UDS service for qtfs. The socket file bound to the Unix socket server needs to be added to the allowlist. You can use either of the following methods:

* Load the allowlist by using the `qtcfg` utility. First compile the utility in **qtfs/qtinfo**.

Run the following command on the qtfs client:

```bash
make role=client 
make install
```

Run the following command on the qtfs server:

```bash
make role=server
make install
```

After `qtcfg` is installed automatically, run `qtcfg` to configure the allowlist. Assume that **/var/lib/docker** needs to be added to the allowlist:

```bash
qtcfg -w udsconnect -x /var/lib/docker
```

Query the allowlist:

```bash
qtcfg -w udsconnect -z
```

Delete an allowlist entry:

```bash
qtcfg -w udsconnect -y 0
```

The parameter is the index number listed when you query the allowlist.

* Add an allowlist entry through the configuration file. The configuration file needs to be set before the qtfs or qtfs\_server kernel module is loaded. The allowlist is loaded when the kernel modules are initialized.

Add the following content to the **/etc/qtfs/whitelist** file.

```bash
[Udsconnect]
/var/lib/docker
```

> \[!NOTE] **NOTE:**
> The allowlist prevents irrelevant Unix sockets from establishing remote connections, causing errors or wasting resources. You are advised to set the allowlist as precisely as possible. For example, in this document, **/var/lib/docker** is set in the container scenario. It would be risky to directly add **/var/lib**, **/var**, or the root directory.

## 2.3 rexec Service Deployment

### 2.3.1 Overview

rexec is a remote execution component developed using the C language. It consists of the rexec client and rexec server. The server is a daemon process, and the client is a binary file. After being started, the client establishes a UDS connection with the server using the udsproxyd service, and the server daemon process starts a specified program on the server machine. During container management plane offload, dockerd is offloaded to the DPU. When dockerd needs to start a service container process on the HOST, the rexec client is invoked to remotely start the process.

### 2.3.2 Deploying rexec

#### 2.3.2.1 Configuring the Environment Variables and Allowlist

Configure the rexec server allowlist on the host. Put the **whitelist** file in the **/etc/rexec** directory, and change the file permission to read-only.

```bash
chmod 400 /etc/rexec/whitelist
```

After downloading the dpu-utilities code, go to the **qtfs/rexec** directory and run `make && make install` to install all binary files required by rexec (**rexec** and **rexec\_server**) to the **/usr/bin** directory.

Before starting the rexec\_server service on the server, check whether the **/var/run/rexec** directory exists. If not, create it.

```bash
mkdir /var/run/rexec
```

The underlying communication of the rexec service uses Unix sockets. Therefore, cross-host communication between rexec and rexec\_server depends on the udsproxyd service, and the related files need to be added to the udsproxy allowlist.

```bash
qtcfg -w udsconnect -x /var/run/rexec
```

#### 2.3.2.2 Starting the Service

You can start the rexec\_server service on the server in either of the following ways.

* Method 1:
  Configure rexec as a systemd service.

Add the **[rexec.service](./config/rexec.service)** file to **/usr/lib/systemd/system**.

[rexec.service](./config/rexec.service)

Then, use `systemctl` to manage the rexec service.

Start the service for the first time:

```bash
systemctl daemon-reload
systemctl enable --now rexec
```

Restart the service:

```bash
systemctl stop rexec
systemctl start rexec
```

* Method 2:
  Manually start the service in the background.

```bash
nohup /usr/bin/rexec_server 2>&1 &
```

## 3 Changes to Management Plane Components

### 3.1 dockerd

The changes to dockerd are based on version 18.09.

For details about the changes to Docker, see the patch file in [this directory](https://atomgit.com/openeuler/dpu-utilities/tree/master/usecases/transparent-offload/patches/docker).

### 3.2 containerd

The changes to containerd are based on containerd-1.2-rc.1.

For details about the changes to containerd, see the patch file in [this directory](https://atomgit.com/openeuler/dpu-utilities/tree/master/usecases/transparent-offload/patches/containerd).

# 4 Container Management Plane Offload Guide

> \[!NOTE] **NOTE:**
>
> 1. Start rexec\_server on both the HOST and DPU.
> 2. rexec\_server on the HOST is used to start containerd-shim by using rexec when the DPU creates a container.
> 3. rexec\_server on the DPU is used to execute the call to dockerd and containerd by containerd-shim.

## 4.1 Preparing the Rootfs for Running dockerd and containerd

> Note: Perform this step only on the DPU.

In the following document, the rootfs is called **/another\_rootfs** (the directory name can be changed as required).

### 4.1.1 Using the Official openEuler QCOW2 Image

You are advised to use the QCOW2 image provided by openEuler to prepare the new rootfs.

#### 4.1.1.1 Installing the Tools

Use `yum` to install xz, kpartx, and qemu-img.

```bash
yum install xz kpartx qemu-img
```

#### 4.1.1.2 Downloading the QCOW2 Image

Download the [openEuler 22.03 LTS VM image for x86](https://repo.openeuler.org/openEuler-22.03-LTS/virtual_machine_img/x86_64/openEuler-22.03-LTS-x86_64.qcow2.xz) or [openEuler 22.03 LTS VM image for Arm64](https://repo.openeuler.org/openEuler-22.03-LTS/virtual_machine_img/aarch64/openEuler-22.03-LTS-aarch64.qcow2.xz) from the openEuler official website.

#### 4.1.1.3 Decompressing the QCOW2 Image

Run `xz -d` to decompress the package and obtain the **openEuler-22.03-LTS-\<arch>.qcow2** file. The following uses the x86 image as an example.

```bash
xz -d openEuler-22.03-LTS-x86_64.qcow2.xz
```

#### 4.1.1.4 Mounting the QCOW2 Image and Copying Files

1. Run the `modprobe nbd maxpart=<any number>` command to load the nbd module.
2. `qemu-nbd -c /dev/nbd0 <VM image path>`
3. Create a folder, for example, `/random_dir`.
4. `mount /dev/nbd0p2 /random_dir`
5. Copy the files.

```bash
mkdir /another_rootfs
cp -r /random_dir/* /another_rootfs/
```

The VM image has been mounted to **/another\_rootfs**.

#### 4.1.1.5 Unmounting QCOW2

After the rootfs is prepared, run the following command to umount the QCOW2 file:

```bash
umount /random_dir
qemu-nbd -d /dev/nbd0
```

### 4.1.2 Installing Software in /another\_rootfs

1. Copy **/etc/resolv.conf** from the root directory to **/another\_rootfs/etc/resolv.conf**.
2. Remove the files in **/another\_rootfs/etc/yum.repos.d** and copy the files in **/etc/yum.repos.d/** to **/another\_rootfs/etc/yum.repos.d**.
3. Run `yum install <software package> --installroot=/another_rootfs` to install a software package.

```bash
yum install --installroot=/another_rootfs iptables
```

## 4.2 Starting qtfs\_server on the HOST

Copy **rexec**, **containerd-shim**, **runc**, and **engine** to the **/usr/bin** directory. Pay attention to the permissions. **rexec** and **engine** have been provided. Compile Docker binary files based on the patch described in "Changes to Management Plane Components."

### 4.2.1 Inserting the qtfs\_server Module

Create a folder required by the container management plane, insert **qtfs\_server.ko**, and start the engine process.

You can run [this script](./config/server_start.sh) to perform this operation. If an error occurs during the execution, try using dos2unix to convert the format of the script (the same applies to all the following scripts).

> Replace the paths of the module and binary file in the script with the actual qtfs path.

In addition, create the **/usr/bin/dockerd** and **/usr/bin/containerd** scripts for executing the `rexec` command on the HOST.

**/usr/bin/dockerd**:

```bash
#!/bin/bash
rexec /usr/bin/dockerd $*
```

**/usr/bin/containerd**:

```bash
#!/bin/bash
exec /usr/bin/containerd $*
```

After the two scripts are created, run the `chmod` command to grant execute permission on them.

```bash
chmod +x /usr/bin/containerd
chmod +x /usr/bin/dockerd
```

## 4.3 Mounting the Dependency Directories on the HOST to the DPU

### 4.3.1 Installing the Software Packages

#### 4.3.2.1 Installing in the Root Directory

In the DPU root directory (not **/another\_rootfs**), install iptables, libtool, libcgroup, and tar using `yum`.

```bash
yum install iptables libtool libcgroup tar
```

You can also download all dependency packages and run the `rpm` command to install them. The iptables and libtool packages and their dependency packages are: [iptables](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/iptables-1.8.7-5.oe2203.x86_64.rpm), [libtool](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/libtool-2.4.6-34.oe2203.x86_64.rpm), [emacs](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/emacs-27.2-3.oe2203.x86_64.rpm), [autoconf](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/autoconf-2.71-2.oe2203.noarch.rpm), [automake](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/automake-1.16.5-3.oe2203.noarch.rpm), [libtool-ltdl](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/libtool-ltdl-2.4.6-34.oe2203.x86_64.rpm), [m4](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/m4-1.4.19-2.oe2203.x86_64.rpm) and [tar](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/tar-1.34-1.oe2203.x86_64.rpm), [libcgroup](https://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/Packages/libcgroup-0.42.2-1.oe2203.x86_64.rpm).

After downloading the preceding software packages, run the following command:

```bash
rpm -ivh iptables-1.8.7-5.oe2203.x86_64.rpm libtool-2.4.6-34.oe2203.x86_64.rpm emacs-27.2-3.oe2203.x86_64.rpm autoconf-2.71-2.oe2203.noarch.rpm automake-1.16.5-3.oe2203.noarch.rpm libtool-ltdl-2.4.6-34.oe2203.x86_64.rpm m4-1.4.19-2.oe2203.x86_64.rpm tar-1.34-1.oe2203.x86_64.rpm libcgroup-0.42.2-1.oe2203.x86_64.rpm
```

#### 4.3.2.2 Configuring the /another\_rootfs Environment

1. Install iptables in **/another\_rootfs**, which is mandatory for dockerd startup.

   Run `yum install <software package > --installroot=/another_rootfs` to install.

2. Copy **rexec** to **/another\_rootfs/usr/bin** and grant execute permission.

   ```bash
   cp rexec /another_rootfs/usr/bin
   chmod +x /another_rootfs/usr/bin/rexec
   ```

3. In addition, copy **containerd** and **dockerd** compiled based on the community Docker source code and the preceding patch to **/another\_rootfs/usr/bin**, and copy **docker** to **/usr/bin**.

   ```bash
   cp {YOUR_PATH}/dockerd /another_rootfs/usr/bin
   cp {YOUR_PATH}/containerd /another_rootfs/usr/bin
   cp {YOUR_PATH}/docker /usr/bin
   ```

4. Delete **/another\_rootfs/usr/sbin/modprobe** from **/another\_rootfs**.

   ```bash
   rm -f /another_rootfs/usr/sbin/modprobe
   ```

5. Create the following scripts in **/another\_rootfs**:

   **/another\_rootfs/usr/local/bin/containerd-shim**:

   ```bash
   #!/bin/bash
   /usr/bin/rexec /usr/bin/containerd-shim $*
   ```

   **/another\_rootfs/usr/bin/remote\_kill**:

   ```bash
   #!/bin/bash
   /usr/bin/rexec /usr/bin/kill $*
   ```

   **/another\_rootfs/usr/sbin/modprobe**:

   ```bash
   #!/bin/bash
   /usr/bin/rexec /usr/sbin/modprobe $*
   ```

   After the creation is complete, grant execute permission to them.

   ```bash
   chmod +x /another_rootfs/usr/local/bin/containerd-shim
   chmod +x /another_rootfs/usr/bin/remote_kill
   chmod +x /another_rootfs/usr/sbin/modprobe
   ```

#### 4.3.2.3 Mounting Directories

Run the [prepare.sh](./config/prepare.sh) script on the DPU to mount the HOST directories required by dockerd and containerd to the DPU.

In addition, ensure that the remote directories mounted by the script exist on both the HOST and DPU.

## 4.4 dockerd and containerd Startup

On the DPU, open two sessions and chroot them to the **/another\_rootfs** required for running dockerd and containerd.

```bash
chroot /another_rootfs
```

Run the following commands in the two sessions to start containerd and then dockerd:

**containerd**

```bash
#!/bin/bash
SHIM_HOST=${YOUR_SERVER_IP} containerd --config /var/run/docker/containerd/containerd.toml --address /var/run/containerd/containerd.sock
```

**dockerd**

```bash
#!/bin/bash
# this need to be done once
/usr/bin/rexec mount -t qtfs /var/lib/docker/overlay2 /another_rootfs/var/lib/docker/overlay2/
SHIM_HOST=${YOUR_SERVER_IP} /usr/bin/dockerd --containerd /var/run/containerd/containerd.sock -s overlay2 --iptables=false --debug 2>&1 | tee docker.log
```

Because **/var/run/** and **/another\_rootfs/var/run/** have been bind-mounted, you can access the **docker.sock** interface through Docker in a normal rootfs to manage containers.

# 5 Environment Restoration

To unmount related directories, delete the existing containers, stop containerd and dockerd, and run the following commands:

```bash
for i in `lsof | grep v1.linux | awk '{print $2}'`
do
        kill -9 $i
done
mount | grep qtfs | awk '{print $3}' | xargs umount
mount | grep another_rootfs | awk '{print $3}' | xargs umount

sleep 1

umount /another_rootfs/etc
umount /another_rootfs/sys
pkill udsproxyd
rmmod qtfs
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/maintenance/aops/aops_asset_management_user_manual.md
---
# A-Ops Asset Management User Guide

This document is currently not available in English.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/maintenance/aops/aops_intelligent_positioning_framework_user_manual.md
---
# A-Ops Intelligent Location Framework User Guide

This document is currently not available in English.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/maintenance/aops/aops_vulnerability_management_module_user_manual.md
---
# A-Ops Vulnerability Management Module User Guide

This document is currently not available in English.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/kubeos/kubeos/about_kubeos.md
---
# About KubeOS

## Introduction

Containers and Kubernetes are widely used in cloud scenarios. However, a current manner of managing the containers and the OSs separately usually faces problems of function redundancy and difficult collaboration between scheduling systems. In addition, it is difficult to manage OS versions. Software packages are installed, updated, and deleted separately in OSs of the same version. After a period of time, the OS versions become inconsistent, causing version fragmentation. Besides, the OSs may be tightly coupled with services, making it difficult to upgrade major versions. To solve the preceding problems, openEuler provides KubeOS, a container OS upgrade tool based on openEuler.

Container OSs are lightweight OSs designed for scenarios where services run in containers. KubeOS connects container OSs as components to Kubernetes, so that the container OSs are in the same position as services. With KubeOS, a Kubernetes cluster manages containers and container OSs in a unified system.

KubeOS is a Kubernetes operator for controlling the container OS upgrade process and upgrading the container OSs as a whole to implement collaboration between the OS managers and services. Before the container OSs are upgraded, services are migrated to other nodes to reduce the impact on services during OS upgrade and configuration. In this upgrade pattern, the container OSs are upgraded atomically so that the OSs remain synchronized with the expected status. This ensures that the OS versions in the cluster are consistent, preventing version fragmentation.

## Architecture

### KubeOS Architecture

**Figure 1** KubeOS architecture

![](./figures/kubeos-architecture.png)

As shown in the preceding figure, KubeOS consists of three components: os-operator, os-proxy, and os-agent. The os-operator and os-proxy components run in containers and are deployed in the Kubernetes cluster. os-agent is not considered a cluster component. Its instances run on worker nodes as processes.

* os-operator: global container OS manager, which continuously checks the container OS versions of all nodes, controls the number of nodes to be upgraded concurrently based on the configured information, and marks the nodes to be upgraded.

* os-proxy: OS manager of a single node, which continuously checks the container OS version of the node. If a node is marked as the node to be upgraded by os-operator, the node is locked, the pod is evicted, and the upgrade information is forwarded to os-agent.

* os-agent: receives information from os-proxy, downloads the container OS image used for upgrade from the OS image server, upgrades the container OS, and restarts the node.

### File System of a Container OS

**Figure 2** File system layout of a container OS

![](./figures/file-system-layout-of-a-container-os.png)

As shown in the figure, a container OS comprises four partitions:

* boot partition: GRUB2 file partition.
* Persist partition: stores persistent user data. When the container OS is upgraded, the data in this partition is retained.
* Two root partitions: Container OSs use the dual-partition mode with two root partitions, rootA and rootB. Assume that the container runs the OS stored in the rootA partition after initialization. When the system is upgraded, the new system is downloaded to the rootB partition. GRUB has two boot options: A and B. The default boot option of GRUB is set to B and the node is restarted. After the node is started, the container runs the upgraded OS in the rootB partition.

The root file system of a container OS is read-only. Users' persistent data is stored in the Persist partition.

---

---
url: /en/docs/22.03_LTS_SP4/server/security/safeguard/about_safeguard.md
---
# About safeguard

safeguard helps protect the Linux kernel and the OS based on eBPF by intercepting and auditing security operations. It uses the libbpfgo library and the Go language to implement top-level control. Currently, the project has been open-sourced and is maintained by openEuler sig-ebpf at <https://atomgit.com/openeuler/safeguard>.

## KRSI (eBPF+LSM)

Extended Berkeley Packet Filter (eBPF) is a technology that can run sandboxed programs in kernel space. It is used to efficiently extend the capabilities of the kernel without requiring to change kernel source code or load kernel modules. This has led to a wave of eBPF-based projects covering a wide array of use cases, including networking, observability, and security functionality.

The Linux Security Module (LSM) framework provides a mechanism for various security checks to be hooked by new kernel extensions.

eBPF and LSM can be combined to form an eBPF-based LSM extension called kernel runtime security instrumentation (KRSI). It allows users to use eBPF programs to implement and enforce custom security policies and audit rules at runtime. The kernel does not need to be modified or recompiled, and existing LSM modules do not need to be configured. It attaches eBPF programs to LSM hooks and then executes those programs in the call path to check and control access to system resources.

## Features

* Audit: Records behavior within the scope of the configuration file and generates logs.
* Control: Secures access control for files, processes, and networks.
* Behavior analysis: Collects information and analyzes resources, hotspots, and exceptions.

## Application Scenarios

safeguard is a Linux security audit and control solution based on KRSI, which is a combination of eBPF and LSM. safeguard ensures OS-wide comprehensive protection and monitoring. While able to fit into a wide range of scenarios, it mainly streamlines the following areas:

* Container security: Audits and controls behavior inside a container. For example, safeguard records the container process, file, and network activities, limits resources or ports available to the container, and detects abnormal container behavior. In this way, safeguard effectively protects the container against attacks or abuse, thereby improving the security and stability of the container.
* Cloud service security: Audits and controls the clients of cloud service providers. For example, safeguard records OSs, applications, and users of the clients, limits the commands and system calls available to the clients, and detects malicious behavior or exploitation of vulnerabilities. This helps cloud service providers protect resources and reputations and prevents the clients from being intruded upon or damaged.
* Security compliance: Audits and controls system security compliance. For example, safeguard records information about system configurations, permissions, and logs, prevents the system from modifying specific settings or files, and detects violations and abnormal events in the system. This ensures compliance with security standards and regulations and improves the credibility and legitimacy of the system.

## Functions

### Audit and Control

File safeguarding:

* Traces file system activities, including file open, close, reads/writes, and delete.
* Modifies the behavior of file systems through the interception of certain file operations and custom security policies.
  Security policies:
  1. Operations on files can be intercepted or redirected through eBPF. For example, read and write operations on sensitive files can be intercepted, and access to certain files can be redirected.
  2. Access control can be customized. eBPF checks the identity, permissions, and environment of a user who requests access to a file, and allows or denies the request based on custom rules.
  3. Audit and monitoring can be customized. For example, eBPF records the information about operations on certain files, such as the operator, time, and action, and outputs the information to the logs.

Process safeguarding:

* Traces the process life cycles, such as process creation, termination, scheduling, and context switching.
* Modifies the behavior of processes, such as injecting or modifying some system calls or implementing custom scheduling policies.

Network safeguarding:

* Traces network activities, such as sending, receiving, forwarding, and discarding network packets.
* Modifies the behavior of networks through filtering and rewriting of network packets and custom routing policies.

## Feature List

### Audit and Control for Files, Networks, and Processes

* Files

  **Functions:**
  List of allowed files.
  List of denied files and related operations to be blocked.
  File open logs, including operation commands, host names, PIDs, and UIDs.

  **Affected operations:**
  File or directory open operations, including the create, read, write, and execute modes.
  File or directory permission or attribute modifications.
  File or directory mapping to the memory.

* Networks

  **Functions:**
  Allowed and denied CIDRs.
  Allowed and denied domains.
  Allowed and denied operation commands.
  Allowed and denied UIDs.
  Allowed and denied GIDs.
  Network connection logs, including cgroup IDS, PIDs, host names, and operation commands.

  **Affected operations:**
  Operations related to network connections, for example, socket creation in the following scenarios:
  \- A socket invokes the `connect()` or `sendto()` function, and the destination address is not NULL.
  \- A socket invokes the `accept()` or `recvfrom()` function, and the source address is not NULL.
  \- A socket invokes the `getpeername()` function, and the return address is not NULL.

* Processes

  **Functions:**

  Process creation logs, including PIDs, PPIDs, host names, and operation commands.

  **Affected operations:**

  Operations related to process creation, including fork, vfork, and clone.

### Logs

Set the log format to JSON or TXT.
Configure the log file output path.
Configure the maximum size to rotate a log file. If the size of a log file exceeds the maximum, a new log file is automatically created. The default size is 100 MB.
Configure the log file rotation time limit. After the time limit is exceeded, a new log file is automatically created. The default time limit is 365 days.
Configure log tags.
Configure log levels. The options are DEBUG (10), INFO (20), WARNING (30), ERROR (40), and CRITICAL (50).

### Configurations

Enable or disable the file, process, network, and log modules, and configure the corresponding sub-modules.
Configure the mode. Select **monitor** or **block**. Related operations (such as the list of denied files) are blocked only in **block** mode.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/security/secdetector/introduction_to_secdetector.md
---
# About secDetector

## Overview

secDetector is an intrusion detection system designed for OSs. It provides intrusion detection and response for critical information infrastructure and reduces development costs while enhancing detection and response for third-party security tools. Based on ATT\&CK attack patterns, secDetector provides real-time blocking and flexible responses in addition to a high volume of security primitives.

secDetector can be used in three modes:

1. Enabled by system users to generate alarms for and handle abnormal events.
2. Integrated with security awareness services to collect system information for analysis of complex security threats (such as APTs) and real-time blocking of critical events.
3. Used to build accurate, efficient, and timely intrusion detection and response capabilities based on the extensible framework under secondary development of security practitioners or security awareness service providers.

## Software Architecture

```text
||==APP===================================================================||
||                                                                        ||
||                     ----------------------------                       ||
||                     |           SDK            |                       ||
||                     ----------------------------                       ||
||                                 /^\                                    ||
||==================================|=====================================||
                                    |
                                    |
                                    |
||==OBSERVER========================|=====================================||
||                                  |                                     ||
||                     ----------------------------                       ||
||                     |         service          |                       ||
||                     ----------------------------                       ||
||                                 /^\                                    ||
||==================================|=====================================||
                                    |
||==DRIVER================================================================||
||                                                                        ||
||                     ----------------------------                       ||
||                     |     8 types of cases     |                       ||
||                     ----------------------------                       ||
||                                                                        ||
||------------------------------------------------------------------------||
||                                core                                    ||
||  -------------  ----------------  ----------------  -----------------  ||
||  | hook unit |  | collect unit |  | analyze unit |  | response unit |  ||
||  -------------  ----------------  ----------------  -----------------  ||
||                                                                        ||
||========================================================================||
```

secDetector consists of four parts: SDK, service, cases, and core.

* SDK

  The SDK is provided as a user-mode dynamic link library (DLL) deployed in the security awareness services that requires secDetector. The SDK communicates with the service to complete related operations (such as subscription, unsubscription, and message reading). The exception information provided by secDetector is defined as different cases. The security awareness services can subscribe to the cases as required.

* service

  The service is a user-mode service application. It manages and maintains the subscriptions of security awareness services and maintains the case running statuses. On a unified environment, it gathers and forwards the information collected by the core and cases to different security awareness services, and manages and forwards the configurations and management requirements of the security awareness services on the cases and core. Because multiple security awareness services may require the same case, the service finds and registers the intersection of cases of all security awareness services.

* Cases

  The cases correspond to a series of exception detection probes, which are in different forms. For example, each probe for detecting kernel exceptions is available in a kernel module (**.ko** file). A case represents a probe, which usually covers a type of exceptions or exception events. For example, the process probes are for creation, exit, and property modification events of all processes, whereas the memory modification probes collect information such as the kernel module list and security switches. A probe may monitor multiple events, but the monitoring logic may not be deployed in the same execution flow. Workflows are introduced to represent the scope of a probe in the same execution flow, whereby a probe contains one or more workflows. For example, the process probe manages the creation and property modification in different workflows.

* Core

  The core is the base framework for case management, and provides common functional units required by workflows. The kernel exception detection framework is carried by a kernel module (**.ko** file), in which a case can register itself with or deregister itself from the core. The core also provides specific interaction APIs to handle external dynamic requests. A workflow consists of four types of functional units: event generator, information collector, event analyzer, and response unit.

The cases and core are combined to be drivers, which provide the bottom-layer system-level implementation of secDetector functions.

The drivers are classified into kerneldriver and usrdriver. The kerneldriver is deployed in the kernel space as a kernel module. The usrdriver is deployed in the user space as a module in the service. Logically, the usrdriver is under the service. However, to reduce the communication cost, the usrdriver is directly integrated into the service program.

## Capabilities and Features

### Detection Capability

| Feature                           | Status   | Description                                                     |
| ------------------------------ | ------ | ------------------------------------------------------------ |
| Detection framework                       | Implemented | Unified, flexible, scalable, and efficient detection framework that supports different types of triggering, collection, analysis, and response units. |
| Process management probes                 | Implemented | Monitor process events such as creation, exit, and metadata modification.                        |
| File operation probes                 | Implemented | Monitor file events such as creation, deletion, read/write, and attribute modification.                     |
| Program behavior probes (API calls)      | Implemented | Monitor critical program behavior such as anonymous pipe creation, command execution, and ptrace system invoking.     |
| Memory modification probes (on critical kernel data) | Implemented | Monitor critical kernel data such as the kernel module list and hardware security function switches.            |

### Response Capability

| Feature     | Status   | Description                                               |
| -------- | ------ | -------------------------------------------------- |
| Response framework | Implemented | Unified, flexible, and scalable response framework that supports different types of response units. |
| Alarm reporting | Implemented | Response unit that reports exceptions.                     |

### Service Capability

| Feature     | Status   | Description                                                         |
| -------- | ------ | ------------------------------------------------------------ |
| Communication framework | Implemented | Applications use gRPC to communicate with the service. The functions are encapsulated in the dynamic library of the SDK. |
| Subscription management | Implemented | Applications can subscribe once and use secDetector for a long time to obtain information. secDetector manages the applications and distributes information about subscribed topics.|
| Configuration delivery | Implemented | The service can configure specific detection and blocking features in parameters to implement functions such as filtering and adjustment. Currently, it is not open to applications. |
| Instant detection | Implemented | secDetector provides real-time, accurate, and first-hand information.             |

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/account_list.md
---
# Account List

| User Name| Default Password     | Function          | User Status| Login Mode          | Remarks                                                        |
| ------ | ------------- | ------------------ | -------- | ------------------ | ------------------------------------------------------------ |
| root   | openEuler12#$ | Default user of the VM image| Enabled    | Remote login          | This account is used to log in to the VM installed using the openEuler VM image.                     |
| root   | openEuler#12  | GRUB2 login         | Enabled    | Local login and remote login| GRand UnifiedBootloader (GRUB) is used to boot different systems, such as Windows and Linux.GRUB2 is an upgraded version of GRUB. When the system is started, you can modify startup parameters on the GRUB2 GUI. To ensure that the system startup parameters are modified with authorization, you need to encrypt the GRUB2 GUI. The GRUB2 GUI can be modified only when you enter the correct GRUB2 password.|

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secharden/account_passwords.md
---
# Account Passwords

## Shielding System Accounts

### Description

Accounts excluding user accounts are system accounts. System accounts cannot be used for logins or performing other operations. Therefore, system accounts must be shielded.

### Implementation

Modify the shell of a system account to  **/sbin/nologin**.

```shell
usermod -L -s /sbin/nologin $systemaccount
```

> \[!NOTE] **NOTE:**
> *$systemaccount*  indicates the system account.

## Restricting Account Permissions on the su Command

### Description

The  **su**  command is used to switch user accounts. To improve system security, only the user  **root**  and users in the **wheel** group can use the  **su**  command.

### Implementation

Modify the  **/etc/pam.d/su**  file as follows:

```text
auth         required      pam_wheel.so use_uid
```

**Table  1**  Configuration item in pam\_wheel.so

## Setting Password Complexity

### Description

You can set the password complexity requirements by modifying the corresponding configuration file. You are advised to set the password complexity based on the site requirements.

### Implementation

The password complexity is implemented by the  **pam\_pwquality.so**  and  **pam\_pwhistory.so**  modules in the  **/etc/pam.d/password-auth**  and  **/etc/pam.d/system-auth**  files. You can modify the configuration items of the two modules to change the password complexity requirements.

### Example

This section provides an example for configuring password complexity.

**Password Complexity Requirements**

1. Contains at least eight characters.

2. Contains at least three types of the following characters:

   * At least one lowercase letter

   * At least one uppercase letter

   * At least one digit

   * At least one space or one of the following special characters: \` ~ ! @ # $ % ^ & \* ( ) - \_ = + \ | \[ { } ] ; : ' " , < . > / ?

3. Cannot be the same as an account name or the account name in reverse order.

4. Cannot be the last five passwords used.

**Implementation**

Add the following content to the first two lines of the **password** configuration item in the  **/etc/pam.d/password-auth**  and  **/etc/pam.d/system-auth**  files:

```text
password    requisite     pam_pwquality.so minlen=8 minclass=3 enforce_for_root try_first_pass local_users_only retry=3 dcredit=0 ucredit=0 lcredit=0 ocredit=0 
password    required      pam_pwhistory.so use_authtok remember=5 enforce_for_root
```

**Configuration Item Description**

For details about the configuration items of  **pam\_pwquality.so**  and  **pam\_pwhistory.so**, see  [Table 2](#table201221044172117)  and  [Table 3](#table1212544452120), respectively.

**Table  2**  Configuration items in pam\_pwquality.so

**Table  3**  Configuration items in pam\_pwhistory.so

## Setting the Password Validity Period

### Description

To ensure system security, you are advised to set the password validity period and notify users to change passwords before the passwords expire.

### Implementation

The password validity period is set by modifying the  **/etc/login.defs**  file.  [Table 4](#en-us_topic_0152100281_t77b5d0753721450c81911c18b74e82eb)  describes the hardening items. All hardening items in the table are in the  **/etc/login.defs**  file. You can directly modify the items in the configuration file.

**Table  4**  Configuration items in login.defs

> \[!NOTE] **NOTE:**
> The  **login.defs**  file is used to set restrictions on user accounts, such as setting the maximum password validity period and maximum length. The configuration in this file is invalid for the user  **root**. If the  **/etc/shadow**  file contains the same items, the  **/etc/shadow**  configuration takes precedence over the  **/etc/login.defs**  configuration. When a user attempts to log in after the password expires, the user will be informed of the password expiry and is required to change the password. If the user does not change the password, the user cannot access the system.

## Setting Password Encryption Algorithms

### Description

For system security, passwords cannot be stored in plaintext in the system and must be encrypted. The passwords that do not need to be restored must be encrypted using irreversible algorithms. Set the password encryption algorithm to SHA-512. This item has been set by default in openEuler. The preceding settings can effectively prevent password disclosure and ensure password security.

### Implementation

To set the password encryption algorithm, add the following configuration to the  **/etc/pam.d/password-auth**  and  **/etc/pam.d/system-auth**  files:

```text
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
```

**Table  5**  Configuration items in pam\_unix.so

## Locking an Account After Three Login Failures

### Description

To ensure user system security, you are advised to set the maximum number of failed login attempts (three attempts are recommended) and the automatic unlocking time (300 seconds are recommended) for a locked account.

If an account is locked, any input is invalid but does not reset the locking countdown timer. Records of the user's invalid inputs are cleared once unlocked. The preceding settings protect passwords from being forcibly cracked and improve system security.

> \[!NOTE] **NOTE:**
> By default, the maximum number of failed login attempts is 3 in openEuler. After an account is locked, the automatic unlock time is 60 seconds.

### Implementation

The password complexity is set by modifying the  **/etc/pam.d/password-auth**  and  **/etc/pam.d/system-auth**  files. The maximum number of failed login attempts is set to  **3**, and the unlocking time after an account is locked is set to  **300**  seconds. The configuration is as follows:

```text
auth        required      pam_faillock.so preauth audit deny=3 even_deny_root unlock_time=300
auth        [default=die] pam_faillock.so authfail audit deny=3 even_deny_root unlock_time=300
auth        sufficient    pam_faillock.so authsucc audit deny=3 even_deny_root unlock_time=300
```

**Table  6**  Configuration items in pam\_faillock.so

## Hardening the su Command

### Description

To enhance system security and prevent the environment variables of the current user from being brought into other environments when you run the  **su**  command to switch to another user, this item has been configured by default in openEuler. The  **PATH**  variable is always initialized when the  **su**  command is used to switch users.

### Implementation

Modify the  **/etc/login.defs**  file. The configuration is as follows:

```text
ALWAYS_SET_PATH=yes
```

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/acknowledgment.md
---
# Acknowledgment

We sincerely thank all the members who participated in and assisted in the openEuler project. It is your hard work to make the version released successfully and provide the possibility for the better development of openEuler.

---

---
url: /zh/docs/22.03_LTS_SP4/tools/ai.md
---


---

---
url: /en/docs/22.03_LTS_SP4/server/development/ai4c/ai4c_user_manual.md
---
# AI4C User Guide

This document is currently not available in English.

---

---
url: /zh/docs/22.03_LTS_SP4/server/development/ai4c/ai4c_user_manual.md
---
# AI4C使用手册

## 安装

1. 使用root权限，安装rpmbuild、rpmdevtools，具体命令如下

   ```bash
   ##  安装rpmbuild
   yum install rpm-build
   ##  安装rpmdevtools
   yum install rpmdevtools
   ```

2. 使用`git clone https://atomgit.com/src-openeuler/AI4C.git`，从目标仓库拉取代码后，使用`rpmdev-setuptree`在主目录下生成rpmbuild文件夹，将拉取代码中的AI4C-v0.1.0-alpha.tar.gz和相关patch文件放入rpmbuild中的SOURCES文件夹中，将AI4C.spec放入rpmbuild中的SPECS目录下。

3. 进入SPECS目录下，用户需先使用`yum-builddep --spec AI4C.spec`命令安装`AI4C`所需依赖。再使用`rpmbuild -ba AI4C.SPEC`，如果构建成功即可返回上一级文件夹，再进入RPM文件夹下，使用`rpm -ivh <成功构建的rpm>`（若系统因存有旧版本安装包而导致的文件冲突，可以在`rpm -ivh <成功构建的rpm>`命令中添加一行`--force`选项，强制安装新版本；或者通过`rpm -Uvh <成功构建的rpm>`命令更新安装包）。待安装完成后，相关动态库会安装在`/usr/lib64`下，模型会安装在`/usr/lib64/AI4C`目录下。

## 使用

1. 开发者在使用前可以通过`export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH`和`export INCLUDE_PATH=/usr/include:$INCLUDE_PATH`配置相关环境变量
2. 开发者需先构建自己的模型，再通过onnxruntime保存成一个**ONNX模型**，即可通过AI4C提供的API进行推理。

### API接口

以下接口存在动态库`libONNXRunner.so`中，用于模型的推理配置阶段、推理运行阶段和模型推理结束后的资源清理阶段，需要用户实现在目标优化遍的适配层内。

1. 参数为模型存放的路径，并创建session对本次推理进行初始化和配置

   ```cpp
   extern ONNXRunner* createONNXRunner(const char* model_path)
   ```

2. 删除ONNXRunner对象

   ```cpp
   extern void deleteONNXRunner(ONNXRunner* instance)
   ```

3. 参数为模型的输入，将输入传入模型进行推理得到相应结果

   ```cpp
   extern float runONNXModel(ONNXRunner* instance, std::vector<std::string> input_string, std::vector<int64_t> input_int64, std::vector<float> input_float)
   ```

### demo -- 以“创建session对本次推理进行初始化和配置”为例

这里建议使用`dlopen`链接动态库，避免对目标应用构建时增加增加额外依赖。使用时开发者需根据实际需求增加异常处理机制。

```cpp
//  挂载动态库
/*  dynamic_library_path 为动态库绝对路径，正常下载流程执行完成后，动态库将会在/usr/lib64下，这里可以直接使用动态库的名字作为输入*/
void *sample_dylib_handle = dlopen (dynamic_library_path, RTLD_LAZY | RTLD_GLOBAL);

//  获取动态库所需要调用的函数
/*  使用函数指针储存动态库需要使用的函数，不同的目标应用对于dlsym返回值的类型转换可能不一样，使用时需要以具体应用要求为标准  */
typedef void* (*sample_func)(const char *); //  参数和返回值以需要使用的函数为准

//  function_name为动态库中需要使用的函数名，这里以createONNXRunner为例
sample_func func = (sample_func) dlsym (smple_dylib_handle, "createONNXRunner");

//  使用所需函数，
func (model_path);

//  卸载动态库
dlclose （dynamic_library_path);
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/maintenance/aops/aops_intelligent_positioning_framework_user_manual.md
---
# AOps 智能定位框架使用手册

参照[《AOps部署指南》](./deploying_aops.md)部署AOps前后端服务，并参照[《AOps资产管理使用手册》](./aops_asset_management_user_manual.md)纳管了主机后，即可使用AOps智能定位框架。

智能定位框架包含了**智能定位**和**配置溯源**两部分，下文会从页面的维度进行AOps智能定位框架功能的介绍。

## 1. 智能定位

AOps项目的智能定位策略采用内置网络诊断应用作为模板，生成个性化工作流的策略进行检测和诊断。

“应用”作为工作流的模板，描述了检测中各步骤的串联情况，内置各步骤中使用的检测模型的推荐逻辑。用户在生成工作流时，可根据各主机的采集项、场景等信息，定制出工作流的详细信息。

### 1.1 工作流列表页面

![工作流](./figures/troubleshoot/工作流.jpg)

支持操作：

* 查看当前工作流列表，支持按照主机组、应用和状态进行筛选，并支持分页操作
* 查看当前应用列表

### 1.2 工作流详情页面

![工作流详情](./figures/troubleshoot/工作流详情.jpg)

支持操作：

* 查看工作流所属主机组、主机数量、状态等基础信息
* 查看单指标检测、多指标检测、集群故障诊断各步骤的详细算法模型信息
* 修改检测各步骤应用的模型
* 执行、暂停和删除工作流

修改某检测步骤的模型时，用户可根据模型名或标签搜索系统内置的模型库，选中模型后点击应用进行更改。

![修改模型](./figures/troubleshoot/修改模型.png)

### 1.3 应用详情页面

![app详情](./figures/troubleshoot/应用.png)

支持操作：

* 查看应用的整体流程
* 基于应用创建工作流

创建工作流时，点击右上角的创建工作流按钮，并在右侧弹出的窗口中输入工作流的名称和描述，选择要检测的主机组。选中主机组后，下方会列出该主机组的所有主机，用户可选中部分主机后移到右侧的列表，最后点击创建，即可在工作流列表中看到新创建的工作流。

![app详情](./figures/troubleshoot/app详情.jpg)

![创建工作流](./figures/troubleshoot/创建工作流.jpg)

### 1.4 告警

启动工作流后，会根据工作流的执行周期定时触发诊断，每次诊断若结果为异常，则会作为一条告警存入数据库，同时也会反映在前端告警页面中。

![告警](./figures/troubleshoot/告警.jpg)

支持操作：

* 查看当前告警总数
* 查看各主机组的告警数量
* 查看告警列表
* 告警确认
* 查看告警详情
* 下载诊断报告

告警确认后，将不在列表中显示

![告警确认](./figures/troubleshoot/告警确认.jpg)

点击异常详情后，可以根据主机维度查看告警详情，包括异常数据项的展示以及根因节点、根因异常的判断等。

![告警详情](./figures/troubleshoot/告警详情.jpg)

## 2. 配置溯源

AOps项目的配置溯源用于对目标主机配置文件内容的变动进行检测记录，对于文件配置错误类引发的故障起到很好的支撑作用。

### 2.1 创建配置域

![](./figures/chuangjianyewuyu.png)

### 2.2 添加配置域纳管node

![](./figures/tianjianode.png)

### 2.3 添加配置域配置

![](./figures/xinzengpeizhi.png)

### 2.4 查询预期配置

![](./figures/chakanyuqi.png)

### 2.5 删除配置

![](./figures/shanchupeizhi.png)

### 2.6 查询实际配置

![](./figures/chaxunshijipeizhi.png)

### 2.7 配置校验

![](./figures/zhuangtaichaxun.png)

### 2.8 配置同步

暂未提供

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/maintenance/aops/aops_vulnerability_management_module_user_manual.md
---
# AOps漏洞管理模块使用手册

参照[《AOps部署指南》](./deploying_aops.md)部署AOps前后端服务，并参照[《AOps资产管理使用手册》](./aops_asset_management_user_manual.md)纳管了主机后，即可使用AOps漏洞管理模块。

A-Ops智能运维工具的智能补丁管理模块(**apollo**)主要集成了**漏洞扫描、CVE修复、任务回退**、**热补丁移除**等核心功能：

* 支持对openEuler已修复并发布的漏洞进行手动/定时扫描。漏洞的详细信息通过**在线/离线**同步社区发布的安全公告进行获取。当前聚焦于内核漏洞的处理，后续支持用户态软件包漏洞。

* 支持漏洞批量修复。修复过程中，客户端会命令行调用基于dnf原生框架的dnf hotpatch插件，实现\*\*冷补丁（需重启）/热补丁（免重启）\*\*的修复。此插件将底层冷、热补丁的管理封装成统一的入口，方便单机用户的使用和集群的调用。

* 支持通过任务粒度回退或移除热补丁的形式，将系统恢复至原状态。

下文将按照漏洞修复的工作流来进行A-Ops智能补丁管理功能的介绍。

## 1. 配置repo源

openEuler的漏洞信息通过安全公告对外发布，同时在update源中发布修复所用的软件包及相应元数据。配置了update源后即可在命令行通过dnf updateinfo list cves命令或dnf hot-updateinfo list cves（需安装A-Ops的dnf热补丁插件）进行漏洞的扫描。

默认的openEuler系统安装后自带对应OS版本的冷补丁update源。对于自定义或离线场景，用户可以通过设置repo来自行配置冷/热补丁的update源。

### 1.1 Repo源添加

漏洞管理界面用于对目标主机存在的CVE进行监控与修复。

当前漏洞管理模块分为以下三个界面：

* 主机列表界面
* CVEs界面
* 任务列表界面

进入漏洞管理的主机管理子页面，可以从主机粒度看到当前纳管的所有主机的**已修复和未修复漏洞**情况：

![主机列表界面](./figures/vul_manage/主机列表界面.png)

点击下方CVE REPO的加号框，即可进行repo源的添加：

![添加REPO源](./figures/vul_manage/添加repo源.png)

若不清楚格式，可以点击下载模板按钮查看。注意baseurl和gpgkey要配置为客户端OS版本的对应地址。用户也可以直接上传编辑好的repo文件。

新建repo完毕后，即可在CVE REPO列表中进行查看或删除。

### 1.2 Repo设置

新建repo源后，点击右上角“设置repo”的按钮，可以创建一个任务，为勾选的主机进行批量的repo设置。

![设置repo](./figures/vul_manage/设置repo源.png)

点击“创建”或“立即执行”后会生成一个repo设置任务，执行完毕后即可在主机列表界面看到已设置好该repo源。

## 2. 漏洞扫描

确认好主机上已配置好repo源（或使用默认安装时自带的repo源）后，我们就可以为主机进行批量扫描了。直接点击右侧的漏洞扫描，默认扫描全部主机。用户也可以勾选部分主机进行扫描。

除了手动扫描，用户也可以配置后台定时任务，进行每日定时扫描。

![漏洞扫描](./figures/vul_manage/漏洞扫描.png)

扫描完毕后，若用户在创建用户时配置了邮箱信息，apollo会将漏洞情况邮件发送给用户。

![邮件通知](./figures/vul_manage/邮件通知.png)

## 3. 漏洞查看

### 3.1 主机详情信息界面

扫描完毕后，除了上文的主机列表可以看到每个主机的已修复和未修复的CVE数量，还可以点击某台主机查看详细的CVE信息：

![主机详情](./figures/vul_manage/主机详情.png)

支持如下操作：

* 查看主机基本信息与CVE个数
* 查看该主机未修复CVE和已修复CVE列表，支持导出。未修复CVE展开后可以看到受影响的RPM包及支持的修复方法，已修复CVE展开后可以看到修复使用的RPM。
* 生成CVE修复任务（切换至“未修复”时，才可支持CVE修复任务创建），可支持CVE粒度的任务创建，同时也可具体到特定的rpm包修复
* 生成热补丁移除任务（切换至“已修复”时，才可支持生成热补丁移除任务）
* 单机漏洞扫描

### 3.2 CVE列表界面

上文介绍了从主机维度查看漏洞情况，我们也可以从**漏洞维度**去查看我们重点关注的漏洞。

点击CVEs子页面，可以看到未修复和已修复两个页签，下方详细介绍了每个CVE的发布时间、影响软件包、严重性等信息，展开后则能看到描述信息及受影响的rpm包和支持的修复方式。

![CVEs列表](./figures/vul_manage/cve列表.png)

支持如下操作：

* 查看所有CVE信息（CVE严重性进行筛选、CVE ID、发布时间、CVSS分数、主机数量进行排序，可根据CVE ID或软件包名称进行检索）

* 切换至“未修复”列表
  * 展开某CVE可以看到受影响的RPM包和支持的修复方法，以及该组合对应的主机
  * 右侧按钮为“生成修复任务”，支持生成**CVE修复任务**

* 切换至“已修复”列表
  * 展开某CVE可以看到修复使用的RPM及对应的主机
  * 右侧按钮为“热补丁移除”，针对热补丁修复的CVE，可生成**热补丁移除任务**

* 上传安全公告

这里对安全公告的上传简单说明：

apollo支持定时从openEuler官网下载安全公告信息，针对无法连接外网的环境，提供了安全公告的手动上传功能。当前社区仅对社区软件包受影响的CVE发布了安全公告，用户可以从以下地址下载安全公告并上传压缩包：[https://repo.openeuler.org/security/data/cvrf/](https://gitee.com/link?target=https%3A%2F%2Frepo.openeuler.org%2Fsecurity%2Fdata%2Fcvrf%2F)

![上传安全公告](./figures/vul_manage/上传安全公告.png)

社区也提供了安全公告订阅，订阅后会收到邮件通知：<https://mailweb.openeuler.org/postorius/lists/sa-announce.openeuler.org/>

### 3.3 CVE详情信息

和主机详情界面类似，在CVE列表界面点击某一个CVE即可进入CVE详情界面。可以看到此漏洞影响的所有主机和已修复这个漏洞的主机。

![CVE详情](./figures/vul_manage/CVE详情界面.png)

支持如下操作：

* 查看CVE基本信息
* 查看关联CVE数量，即影响同样源码包（如kernel）的CVE
* 查看受此CVE影响的主机列表以及单个主机上受此CVE影响的rpm包列表
* 支持点击主机名称跳转至**主机详情页**
* 选择“未修复”列表，右侧按钮为“生成修复任务，支持**生成CVE修复任务**
* 选择“已修复”列表，右侧按钮为“热补丁移除任务”，支持**生成热补丁移除任务**

## 4. 漏洞修复

### 4.1 生成修复任务

在CVE列表、CVE详情、主机详情界面均可进行漏洞的批量修复。这里以CVE列表界面为示例，选中CVE点击“生成修复任务”按钮，右侧会出现弹窗。不选中CVE则默认修复全部CVE。

其中针对热补丁，有2个按钮：

* 是否accept：勾选后会在重启后自动激活此次修复使用的热补丁

* 冷补丁收编：勾选后，会同步生成热补丁对应的冷补丁的修复任务

![生成修复任务](./figures/vul_manage/生成修复任务.png)

需额外注意：

* 为了方便执行以及后续的任务回滚，生成任务时会自动将冷、热补丁的修复动作拆分成两个任务，可以通过任务名进行分辨。

### 4.2 执行修复任务

生成任务后可以点击立即跳转到该任务详情，或点击左侧的任务子页面，进入任务列表界面：

![任务列表](./figures/vul_manage/任务列表.png)

点击刚才生成的修复任务，可以看到此任务的基础信息，以及下方的主机以及该主机要修复的软件包信息。点击右侧的执行按钮即可执行。

![任务详情界面](./figures/vul_manage/任务详情.png)

**注意**：针对同一台主机，**热补丁任务应优先与冷补丁任务执行**。由于内核热补丁只能应用在指定版本内核，若先安装冷补丁再安装热补丁，aops客户端会报错，以防重启后内核切换、热补丁失效导致的漏洞重新暴露。而先安装热补丁再安装冷补丁时，客户端调用的dnf upgrade-en 命令会确保冷补丁包含了当前热补丁修复的漏洞。

### 4.3 任务报告查看

执行完毕后，可以看到任务的“上次执行时间”发生更新，并出现“查看报告”按钮。点击查看报告，即可查看各主机的执行情况，如执行结果、执行失败的原因等：

![修复任务报告](./figures/vul_manage/修复任务报告.png)

## 5. 修复任务回滚

进入修复任务详情，点击生成回滚任务，即可对该修复任务进行回退：

![生成回滚任务](./figures/vul_manage/生成回滚任务.png)

进入回滚任务详情，与修复任务相反，可以看见当前已安装的软件包（修复时安装的rpm），以及回退后的目标软件包（修复前的rpm）。执行时点击“执行”按钮即可。

![回滚任务详情](./figures/vul_manage/回滚任务详情.png)

## 6. 热补丁移除任务

若对已安装的热补丁不满意，可以在任意“已修复”的列表，勾选使用热补丁修复的CVE或主机，生成热补丁移除任务。

与回滚任务相比，热补丁移除任务只针对热补丁，且不支持对热补丁的升降级处理，只通过dnf操作将热补丁rpm进行移除。

![生成热补丁移除任务](./figures/vul_manage/生成热补丁移除任务.png)

## 7.定时任务配置

主体的漏洞处理过程在前台完成之后，用户还可以在apollo服务端针对后台的定时任务进行编辑，修改后`systemctl restart aops-apollo`重启服务生效。

定时任务主要包含3种类型任务，定时任务配置文件位于 /etc/aops/apollo\_crontab.ini，内容如下：

```ini
[cve_scan]
# timed task name
id = cve scan
# value between 0-6, for example, 0 means Monday, 0-6means everyday.
day_of_week = 0-6
# value between 0-23, for example, 2 means 2:00 in a day.
hour = 2
# value is true or false, for example, true means with service start.
auto_start = true

[download_sa]
id = download sa
day_of_week = 0-6
hour = 3
auto_start = true
cvrf_url = https://repo.openeuler.org/security/data/cvrf

[correct_data]
id = correct data
day_of_week = 0-6
hour = 4
auto_start = true
service_timeout_threshold_min = 15
```

### 7.1 定时巡检，执行漏洞扫描

**定时扫描cve任务的参数**

* id

  > 定时任务的名称，不能与其他定时任务名称重复，不建议修改。

* day\_of\_week

  > 定时任务在一周中的第几天启动，取值范围0-6，0-6表示每天，0表示周一，以此类推。

* hour

  > 任务启动的时间，取值范围0-23，与24小时制时间格式一致。

* auto\_start

  > 任务是否跟随服务启动，true表示同时启动，false表示不同时启动。

* 其他

  > 如果要精确到分钟，秒，需要添加minute（取值范围0-59）和second（取值范围0-59）。
  >
  > **示例**
  >
  > ```ini
  > minute = 0
  > second = 0
  > ```

**修改配置文件示例**

> 打开配置文件

```shell
vim /etc/aops/apollo_crontab.ini
```

> 修改定时任务执行时机

```ini
[cve_scan]
id = cve scan
day_of_week = 5
hour = 2
auto_start = true
```

### 7.2  定时下载安全公告

相同字段含义和使用与\[cve\_scan]一样。

* cvrf\_url

  > 获取安全公告详细信息的基础url，**暂不支持修改。**

### 7.3 定时校正异常数据

相同字段含义和使用与\[cve\_scan]一样。

* service\_timeout\_threshold\_min

  > 判断异常数据的阈值，取值为正整数，建议最小值为15。

**修改配置文件示例**

> 打开配置文件

```shell
vim /etc/aops/apollo_crontab.ini
```

> 设置异常数据阈值

```ini
service_timeout_threshold_min = 15
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/maintenance/aops/aops_asset_management_user_manual.md
---
# AOps资产管理使用手册

参照[AOps部署指南](./deploying_aops.md)部署AOps前后端服务后，即可使用AOps资产管理功能，纳管集群主机。

主机纳管是使用AOps进行智能运维的第一步，后续用户按需部署的漏洞管理、配置溯源及故障诊断服务均面向纳管的主机进行操作。

下文将为大家介绍如何使用资产管理功能，逐步纳管集群主机。

## 1. 登录

首先使用默认的admin账号进行登录，密码为changeme

![登陆界面](./figures/asset_manage/登陆界面.png)

这里也支持用户注册新账号，或是用gitee账号第三方登录。

登录后会直接切入数据看板界面，后续纳管主机并漏洞扫描后，可以在右侧页面主窗口右上角内看到CVE整体数量的分布统计。

![工作台详情](./figures/asset_manage/工作台.png)

## 2. 纳管集群主机

资产管理界面用于对集群中服务器的纳管（添加、编辑和删除），支持单台主机和批量主机的添加操作。

当前资产管理模块分为以下两个界面：

* 主机管理
* 主机组管理

纳管时需先创建主机组，再将主机添加至对应主机组中，便于后续对主机分组查看和管理。

### 2.1 添加主机组

进入资产管理的主机组管理子页面，点击右侧添加主机组按钮，即可添加主机组的名称和描述。

![主机组管理](./figures/asset_manage/主机组管理列表.png)

![添加主机组](./figures/asset_manage/添加主机组.png)

后续添加主机后，可以查看该主机组的主机列表：

![主机组内主机查看](./figures/asset_manage/主机组内主机查看.png)

### 2.2 添加主机

进入主机管理页面，可以看到当前所有纳管的主机。其中也可以看到各主机的在线状态。

![主机列表查看](./figures/asset_manage/主机列表.png)

此页面支持如下操作：

* 添加单台主机
* 批量添加主机
* 主机的批量删除
* 支持使用所属主机组、管理节点的主机过滤，同时可满足对主机名称的排序

#### 2.2.1 添加单台主机

点击“添加主机”按钮，即可对单台主机进行添加。

![主机管理-添加](./figures/asset_manage/主机管理-添加.png)

支持如下操作：

* 快捷添加主机组（同主机组管理中的添加功能）
* 登录认证方式支持主机密码模式（需提供账号和密码）和主机密钥模式（需提供可登录的密钥，**注意此处的密钥为私钥**）

注：管理节点/监控节点暂无本质区别，用户可按需指定。

#### 2.2.2 批量添加主机

针对大集群的场景，一个个添加主机过于麻烦。这里我们提供了上传excel的方式，将主机批量添加。

![主机管理-批量添加](./figures/asset_manage/批量添加主机.png)

下载模板并按照格式填写主机注册所需信息后，选择文件进行上传。

![主机管理-批量添加](./figures/asset_manage/批量添加-文件解析.png)

格式解析无误后，点击提交，即可看到添加结果。若添加失败会有相应提示。

![主机管理-批量添加](./figures/asset_manage/批量添加-添加结果.png)

支持如下操作：

* 在线下载批量导入主机模板，支持的类型有xls、xlsx、csv三种格式的文件
* 通过新增上传解析文件内容，数据回显至前端展示
* 支持单主机的数据调整或删除
* 点击提交后，可查看主机添加的结果

## 3 编辑主机

添加主机完毕后，若密码或密钥不对导致连接失败，或其他信息需要变更，可以点击主机列表中的编辑按钮进行编辑：

![主机管理-批量添加](./figures/asset_manage/主机编辑界面.png)

## 4 查看主机详情

客户端部署了aops-ceres命令行工具后，在主机列表点击主机可以查看该主机的一些基础信息。

![主机详情](./figures/asset_manage/主机详情.png)

若部署了prometheus服务以及客户端的采集器（node-exporter或gala-gopher等），可以在下方选择并展示主机的指标波形。

![指标波形](./figures/asset_manage/指标波形.png)

点击插件页签，可以看到node-exporter插件的各采集探针，按需开启或关闭。

点击场景识别后，系统会根据客户端的应用生成该主机的场景，并推荐检测该场景所需开启的插件以及采集项。

![插件开关](./figures/asset_manage/插件开关.png)

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secdetector/api_reference.md
---
# API Reference

secDetector provides an SDK for external systems. This section describes the APIs required for developing applications. There are only three APIs and two header files.

Header files:

* **secDetector/secDetector\_sdk.h**: contains API definitions.

* **secDetector/secDetector\_topic.h**: contains predefined macros required for calling APIs, such as the ID of a topic that can be subscribed to.

## secSub

API for subscribing to topics.

**Functions:**

This API is used by applications to subscribe to different topics by specifying topic IDs, for example, file opening exception probes. You can view the definitions of topic IDs provided by secDetector in **secDetector\_topic.h**. This API can be used to subscribe to multiple topics at a time. The topic IDs of multiple probes can be combined in bitmap format.

> \[!NOTE] **NOTE:**
> A subscription generates a message reader. An application needs to subscribe to all required topics in a subscription API call. In this way, a reader can be used to collect messages. To change the subscribed topics, you can unsubscribe from the topics and subscribe to them again.

**Function declaration:**

```c
void *secSub(const int topic);
```

**Parameters:**

* **topic**: input parameter, which specifies a set of topics to be subscribed to.

**Return values:**

* **NULL**: The subscription fails.
* **NOT NULL**: gRPC reader of the subscribed topics.

## secUnsub

API for unsubscribing from topics.

**Functions:**

This API is used by applications to unsubscribe from topics by entering the reader obtained after the subscription is successful. After the unsubscription, applications will not receive messages from the unsubscribed topics. If a topic is not subscribed to by any application, it will not be executed.

**Function declaration:**

```c
void secUnsub(void *reader);
```

**Parameters:**

* **reader**: input parameter, which specifies the message reader to be unsubscribed from.

**Return values:**

* None

## secReadFrom

API for reading messages from subscribed topics.

**Functions:**

After a topic is successfully subscribed to using the subscription API, you can use this API to receive messages of the subscribed topic from secDetector before unsubscribing from the topic. This API is in blocking mode. It is recommended that applications use an independent thread to call the API cyclically. This function can be resumed only when there are messages in the subscribed topic.

**Function declaration:**

```c
void secReadFrom(void *reader, char *data, int data_len);
```

**Parameters:**

* **reader**: input parameter, which specifies the message reader obtained after the subscription is successful.
* **data**: output parameter, which specifies the message buffer, a segment of memory provided by an application.
* **data\_len**: input parameter, which specifies the size of the message buffer.

**Return values:**

* None

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secgear/api_reference.md
---
# API Reference

The secGear unified programming framework for confidential computing provides basic APIs that shields the differences between TEE SDKs, as well as advanced features such as switchless, remote attestation, and secure channel.

## Basic APIs

### cc\_enclave\_create

Creates an enclave.

**Function:**

Initialization API. The function calls different TEE creation functions based on the type to initialize the enclave context in different TEE solutions. This API is called by the REE.

> \[!NOTE] **Note:**
> Due to Intel SGX restrictions, memory mapping contention exists when multiple thread invoke cc\_enclave\_create concurrently. As a result, the creation of the enclave API may fail. Avoid concurrent invocations of cc\_enclave\_create in your code.

**Function Declaration:**

```c
cc_enclave_result_t cc_enclave_create(const char* path, enclave_type_t type, uint32_t version,uint32_t flags,const enclave_features_t* features,uint32_t features_count,
 cc_enclave_t  ** enclave);
```

**Parameters:**

* **path**: input parameter, which specifies a path of the enclave to be loaded.
* **type**: input parameter, which specifies the TEE solution, for example, SGX\_ENCLAVE\_TYPE, GP\_ENCLAVE\_TYPE and AUTO\_ENCLAVE\_TYPE.
* **version**: input parameter, which specifies the enclave engine version. Currently, there is only one version, and the value is 0.
* **flags**: input parameter, which specifies the running status of the enclave. For example, SECGEAR\_DEBUG\_FLAG indicates the debugging status, and SECGEAR\_SIMULATE\_FLAG indicates the simulation status (not supported currently).
* **features**: input parameter, which specifies some features supported by the enclave, for example, PCL and switchless of the SGX. This parameter is not supported currently. Set it to NULL.
* **features\_count**: input parameter, which specifies the number of features. This parameter is not supported currently. Set it to 0.
* **enclave**: output parameter, which specifies the created enclave context.

**Return Values:**

* CE\_SUCCESS: The authentication information is verified successfully.
* CE\_ERROR\_INVALID\_PARAMETER: The input parameter is incorrect.
* CE\_ERROR\_OUT\_OF\_MEMORY: No memory is available.
* CC\_FAIL: Common failure.
* CC\_ERROR\_UNEXPECTED: Unexpected error.
* CC\_ERROR\_ENCLAVE\_MAXIMUM: The number of enclaves created by a single app reaches the maximum.
* CC\_ERROR\_INVALID\_PATH: The secure binary path is invalid.
* CC\_ERROR\_NO\_FIND\_REGFUNC: The enclave search fails.

### cc\_enclave\_destroy

Destroys an enclave.

**Function:**

This API is called by the REE to call the exit functions of different TEEs to release the created enclave entities.

**Function Declaration:**

```c
cc_enclave_result_t cc_enclave_destroy (cc_enclave_t ** enclave);
```

**Parameter:**

* **enclave**: input parameter, which specifies the context of the created enclave.

**Return Values:**

* CE\_SUCCESS: The authentication information is verified successfully.
* CE\_ERROR\_INVALID\_PARAMETER: The input parameter is incorrect.
* CE\_ERROR\_OUT\_OF\_MEMORY: No memory is available.
* CC\_ERROR\_NO\_FIND\_UNREGFUNC: The enclave search fails.
* CC\_FAIL: common failure.
* CC\_ERROR\_UNEXPECTED: unexpected error.

### cc\_malloc\_shared\_memory

Creates the shared memory.

**Functions**

After the switchless feature is enabled, this API is called by the REE to create the shared memory that can be accessed by both the TEE and REE.

**Function Declaration:**

```c
void *cc_malloc_shared_memory(cc_enclave_t *enclave, size_t size);
```

**Parameters:**

* **enclave**: input parameter, which indicates the context handle of the secure environment. Different platforms have different shared memory models. To ensure cross-platform interface consistency, this parameter is used only on the ARM platform and is ignored on the SGX platform.
* **size**: input parameter, which indicates the size of the shared memory.

**Return Values:**

* NULL: Failed to apply for the shared memory.
* Other values: start address of the created shared memory.

### cc\_free\_shared\_memory

Releases the shared memory.

**Functions**

This API is called by the REE to release the shared memory after the switchless feature is enabled.

**Function Declaration:**

```c
cc_enclave_result_t cc_free_shared_memory(cc_enclave_t *enclave, void *ptr);
```

**Parameters:**

* **enclave**: input parameter, which indicates the context handle of the secure environment. Different platforms have different shared memory models. To ensure cross-platform interface consistency, this parameter is used only on the ARM platform (the value of this parameter must be the same as the value of enclave passed when cc\_malloc\_shared\_memory is invoked). It is ignored on the SGX platform.
* **ptr**: input parameter, which indicates the shared memory address returned by cc\_malloc\_shared\_memory.

**Return Values:**

* CC\_ERROR\_BAD\_PARAMETERS: invalid input parameter.
* CC\_ERROR\_INVALID\_HANDLE: The enclave is invalid or the input enclave does not match the enclave corresponding to the ptr. (It takes effect only on the ARM platform. The SGX platform ignores the enclave and therefore does not check the enclave.)
* CC\_ERROR\_NOT\_IMPLEMENTED: The API is not implemented.
* CC\_ERROR\_SHARED\_MEMORY\_START\_ADDR\_INVALID: ptr is not the shared memory address returned by cc\_malloc\_shared\_memory (valid only on the ARM platform).
* CC\_ERROR\_OUT\_OF\_MEMORY: insufficient memory (valid only on the ARM platform).
* CC\_FAIL: common failure.
* CC\_SUCCESS: success

### cc\_enclave\_generate\_random

Generates random numbers.

**Function:**

Generate a secure random number for the password on the TEE.

**Function Declaration:**

```c
cc_enclave_result_t cc_enclave_generate_random(void *buffer, size_t size)
```

**Parameters:**

* **buffer**: input parameter, which specifies the buffer for generating random numbers.
* **size**: input parameter, which specifies the buffer length.

**Return Values:**

* CE\_OK: Authentication information is verified successfully.
* CE\_ERROR\_INVALID\_PARAMETER: incorrect input parameter.
* CE\_ERROR\_OUT\_OF\_MEMORY: no memory is available.

### cc\_enclave\_seal\_data

Ensures data persistence.

**Function:**

This API is called by the TEE to encrypt the internal data of the enclave so that the data can be persistently stored outside the enclave.

**Function Declaration:**

```c
cc_enclave_result_t cc_enclave_seal_data(uint8_t *seal_data, uint32_t seal_data_len,

    cc_enclave_sealed_data_t *sealed_data, uint32_t sealed_data_len,

    uint8_t *additional_text, uint32_t additional_text_len)
```

**Parameters:**

* **seal\_data**: input parameter, which specifies the data to be encrypted.
* **seal\_data\_len**: input parameter, which specifies the length of the data to be encrypted.
* **sealed\_data**: output parameter, which specifies the encrypted data processing handle.
* **sealed\_data\_len**: output parameter, which specifies the length of the encrypted ciphertext.
* **additional\_text**: input parameter, which specifies the additional message required for encryption.
* **additional\_text\_len**: input parameter, which specifies the additional message length.

**Return Values:**

* CE\_SUCCESS: Data encryption succeeds.
* CE\_ERROR\_INVALID\_PARAMETER: incorrect input parameter.
* CE\_ERROR\_OUT\_OF\_MEMORY: no memory is available.
* CC\_ERROR\_SHORT\_BUFFER: The input buffer is too small.
* CC\_ERROR\_GENERIC: Common bottom-layer hardware error.

### cc\_enclave\_unseal\_data

Decrypts data.

**Function:**

This API is called by the TEE to decrypt the data sealed by the enclave and import the external persistent data back to the enclave.

**Function Declaration:**

```c
cc_enclave_result_t cc_enclave_unseal_data(cc_enclave_sealed_data_t *sealed_data,

    uint8_t *decrypted_data, uint32_t *decrypted_data_len,

    uint8_t *additional_text, uint32_t *additional_text_len)
```

**Parameters:**

* **sealed\_data**: input parameter, which specifies the handle of the encrypted data.
* **decrypted\_data**: output parameter, which specifies the buffer of the decrypted ciphertext data.
* **decrypted\_data\_len**: output parameter, which specifies the length of the decrypted ciphertext.
* **additional\_text**: output parameter, which specifies an additional message after decryption.
* **additional\_text\_len**: output parameter, which specifies the length of the additional message after decryption.

**Return Values:**

* CE\_SUCCESS: Data decryption is successful.
* CE\_ERROR\_INVALID\_PARAMETER: incorrect input parameter.
* CE\_ERROR\_OUT\_OF\_MEMORY: no memory is available.
* CC\_ERROR\_SHORT\_BUFFER: The input buffer is too small.
* CC\_ERROR\_GENERIC: common bottom-layer hardware error.

### cc\_enclave\_get\_sealed\_data\_size

Obtains the size of the encrypted data.

**Function:**

Obtain the size of the sealed\_data data. This API can be called by the TEE and REE to allocate the decrypted data space.

**Function Declaration:**

```c
uint32_t cc_enclave_get_sealed_data_size(const uint32_t add_len, const uint32_t seal_data_len);
```

**Parameters:**

* **add\_len**: input parameter, which specifies the additional message length.
* **sealed\_data\_len**: input parameter, which specifies the length of the encrypted information.

**Return Values:**

* UINT32\_MAX: Parameter error or function execution error.
* others: The function is successfully executed, and the return value is the size of the sealed\_data structure.

### cc\_enclave\_get\_encrypted\_text\_size

Obtains the length of an encrypted message.

**Function:**

This API is called by the TEE to obtain the length of the encrypted message in the encrypted data.

**Function Declaration:**

```c
uint32_t cc_enclave_get_encrypted_text_size(const cc_enclave_sealed_data_t *sealed_data);
```

**Parameter:**

* **sealed\_data**: input parameter, which specifies the handle of the encrypted data

**Return Values:**

* UINT32\_MAX: Parameter error or function execution error.
* others: The function is executed successfully, and the return value is the length of the encrypted message in sealed\_data.

### cc\_enclave\_get\_add\_text\_size

Obtains the length of an additional message.

**Function:**

This API is called by the TEE to obtain the length of the additional message in the encrypted data.

**Function Declaration:**

```c
uint32_t cc_enclave_get_add_text_size(const cc_enclave_sealed_data_t *sealed_data);
```

**Parameter:**

* **sealed\_data**: input parameter, handle of the encrypted data.

**Return Values:**

* UINT32\_MAX: Parameter error or function execution error.
* others: The function is successfully executed, and the return value is the length of the additional message in sealed\_data.

### cc\_enclave\_memory\_in\_enclave

Performs security memory check.

**Function:**

This API is called by the TEE to check whether the memory addresses of the specified length belong to the TEE.

**Function Declaration:**

```c
bool cc_enclave_memory_in_enclave(const void *addr, size_t size)
```

**Parameters:**

* **\*addr**: input parameter, which specifies the memory address to be verified.
* **size**: input parameter, which specifies the length to be verified starting from the memory address.

**Return Values:**

* true: The memory in the specified zone is in the secure zone.
* false: Some or all memory in the specified area is not within the secure range.

### cc\_enclave\_memory\_out\_enclave

Performs security memory check.

**Function:**

This API is called by the TEE to check whether the memory addresses of the specified length belong to the REE.

**Function Declaration:**

```c
bool cc_enclave_memory_out_enclave(const void *addr, size_t size)
```

**Parameters:**

* **\*addr**: input parameter, which specifies the memory address to be verified.
* **size**: input parameter, length to be verified starting from the memory address.

**Return Values:**

* true: The memory of the specified area is in the non-secure area.
* false: Some or all of the memory in the specified zone is in the secure area.

### PrintInfo

Prints messages.

**Function:**

Print TEE logs. This API outputs the information that the TEE user wants to print. The input logs are stored in the REE /var/log/secgear/secgear.log.

**Function Declaration:**

```c
void PrintInfo(int level, const char *fmt, ...);
```

**Parameters:**

* **level**: log print level, which is an input parameter. The value can be PRINT\_ERROR, PRINT\_WARNING, PRINT\_STRACE, and PRINT\_DEBUG.
* **fmt**: Input parameter, and a character to be output.

**Return Value:**

* None

## Switchless

Switchless optimizes the performance of regular ecalls (for the REE to call the TEE). You can enable switchless in two steps as follows.

### Enabling Switchless

Use the **enclave\_features\_t** parameter of **cc\_enclave\_create** to enable the switchless feature. For details, see the [switchless example](https://atomgit.com/openeuler/secGear/tree/master/examples/switchless).
The configuration items are as follows:

```c
typedef struct {
uint32_t num_uworkers;
uint32_t num_tworkers;
uint32_t switchless_calls_pool_size;
uint32_t retries_before_fallback;
uint32_t retries_before_sleep;
uint32_t parameter_num;
uint32_t workers_policy;
uint32_t rollback_to_common;
} cc_sl_config_t;
```

| Item|   Description  |
| ------------ | ---- |
|       num\_uworkers       |   Number of proxy worker threads in the REE, which are used to make switchless OCALLs. Currently, this field takes effect only on the SGX platform and can be configured on the ARM platform. However, because the ARM platform does not support OCALLs, the configuration does not take effect on the ARM platform.Specifications:ARM: maximum value: **512**; minimum value: **1**; and default value: **8** (used when **num\_uworkers** is set to **0**).SGX: maximum value: **4294967295**; minimum value: **1**.|
|      num\_tworkers        |   Number of proxy worker threads in the TEE, which are used to make switchless ECALLs.Specifications:ARM: maximum value: **512**; minimum value: **1**; and default value: **8** (used when **num\_tworkers** is set to **0**).SGX: maximum value: **4294967295**; minimum value: **1**.|
|     switchless\_calls\_pool\_size         |    Size of the switchless call pool. The pool can contain **switchless\_calls\_pool\_size** x 64 switchless calls. For example, if **switchless\_calls\_pool\_size=1**, 64 switchless calls are contained in the pool.Specifications:ARM: maximum value: **8**; minimum value: **1**; default value: **1** (used when **switchless\_calls\_pool\_size** is set to **0**)SGX: maximum value: **8**; minimum value: **1**; and default value: **1** (used when **switchless\_calls\_pool\_size** is set to **0**).|
|        retries\_before\_fallback      |    After the **pause** assembly instruction is executed for **retries\_before\_fallback** times, if the switchless call is not made by the proxy worker thread on the other side, the system rolls back to the switch call mode. This field takes effect only on the SGX platform.Specifications:SGX: maximum value: **4294967295**; minimum value: **1**; and default value: **20000** (used when **retries\_before\_fallback** is set to **0**).|
|      retries\_before\_sleep        |   After the **pause** assembly instruction is executed for **retries\_before\_sleep** times, if the proxy worker thread does not receive any task, the proxy worker thread enters the sleep state. This field takes effect only on the SGX platform.Specifications:SGX: maximum value: **4294967295**; minimum value: **1**; and default value: **20000** (used when **retries\_before\_sleep** is set to **0**).|
|       parameter\_num       |   Maximum number of parameters supported by the switchless function. This parameter takes effect only on the ARM platform. There is no such restriction on the SGX platform.Specifications:ARM: maximum value: **16**; minimum value: **0**.|
|       workers\_policy       |   Running mode of the switchless worker thread. This parameter takes effect only on the ARM platform.Specifications:ARM:WORKERS\_POLICY\_BUSY: The worker thread always occupies CPU resources regardless of whether there are tasks to be processed. This mode is suitable for scenarios that require high performance and have rich system software and hardware resources. WORKERS\_POLICY\_WAKEUP:The worker thread is woken up only when there is a task. After the task is processed, the worker thread enters the sleep state and waits to be woken up by a new task.|
|       rollback\_to\_common       |   Whether to roll back to common invocation when the asynchronous invocation of switchless fails. This parameter takes effect only on the ARM platform.Specifications:ARM:0: No. An error code is returned upon failure. Other: Yes. Common invocation is used upon failure.|

### Enabling ecall

Use **transition\_using\_threads** to enable switchless for ecall.

```ocaml
enclave {
    include "secgear_urts.h"
    from "secgear_tstdc.edl" import *;
    from "secgear_tswitchless.edl" import *;
    trusted {
        public int get_string([out, size=32]char *buf);
        public int get_string_switchless([out, size=32]char *buf) transition_using_threads;
    };
};
```

## Remote Attestation

The following remote attestation APIs are provided. For details, see the [remote attestation example](https://atomgit.com/openeuler/secGear/tree/master/examples/remote_attest) and [local attestation example](https://atomgit.com/openeuler/secGear/tree/master/examples/local_attest).

| User   | API                | Function     | Header File                   | Library File                  |
|------|-------------------|----------|-----------------------|----------------------|
| Remote attestation service | cc\_prepare\_ra\_env | Attestation environment initialization  | sg\_ra\_report.h        | libsecgear\_ra.so     |
| Remote attestation service | cc\_get\_ra\_report  | Remote attestation report obtaining | sg\_ra\_report.h        | libsecgear\_ra.so     |
| TA   | cc\_local\_attest   | Local attestation     | sg\_local\_attest.h     | libsecgear\_la.a      |
| Verifier  | cc\_verify\_report  | Remote attestation report verification | sg\_ra\_report\_verify.h | libsecgear\_verify.so |

## Secure Channel

The secure channel is provided as a library and consists of the client, host, and enclave, which are icalled by the client, server client application (CA), and server trusted application (TA) of the service program respectively.

| Module         | Header File                      | Library File                   | Dependency      |
|------------|--------------------------|-----------------------|---------|
| Client        | secure\_channel\_client.h  | libcsecure\_channel.so | OpenSSL |
| Host    | secure\_channel\_host.h    | libusecure\_channel.so | OpenSSL |
| Enclave | secure\_channel\_enclave.h | libtsecure\_channel.so | TEE and TEE software stack     |

### APIs

| API                                                                                                                                         | Header File and Library                  | Function          | Remarks|
|----------------------------------------------------------------------------------------------------------------------------------------------|-----------------------|--------------|----|
| cc\_sec\_chl\_client\_init                                                 | secure\_channel\_client.h libcsecure\_channel.so | Initializes the secure channel on the client.  | Before calling this API, initialize the network connection and message sending hook function in the **ctx** parameter.  |
| cc\_sec\_chl\_client\_fini                                                                                         | secure\_channel\_client.h libcsecure\_channel.so | Destroys the secure channel on the client.   | Instructs the server to destroy the local client information and local secure channel information.  |
| cc\_sec\_chl\_client\_callback                                              | secure\_channel\_client.h libcsecure\_channel.so | Function for processing secure channel negotiation messages.| Processes messages sent from the server to the client during secure channel negotiation. This API is called when messages are received on the client.  |
| cc\_sec\_chl\_client\_encrypt | secure\_channel\_client.h libcsecure\_channel.so | Encryption API of the secure channel on the client.    |  None |
| cc\_sec\_chl\_client\_decrypt | secure\_channel\_client.h libcsecure\_channel.so | Decryption API of the secure channel on the client.    |  None  |
|  int (*cc\_conn\_opt\_funcptr\_t)(void*conn, void \*buf, size\_t count);                                                                                                                                            |    secure\_channel.h                    |    Prototype of the message sending hook function.         | Implemented by the client and server to specify the secure channel negotiation message type. It sends secure channel negotiation messages to the peer end.  |
|  cc\_sec\_chl\_svr\_init                                                                                                                                            |  secure\_channel\_host.h  libusecure\_channel.so                    |  Initializes the secure channel on the server.           | Before calling this API, initialize **enclave\_ctx** in **ctx**.  |
|  cc\_sec\_chl\_svr\_fini                                                                                                                                            |   secure\_channel\_host.h  libusecure\_channel.so                    |  Destroys the secure channel on the server.           |  Destroys information about the secure channel on the server and all clients. |
|  cc\_sec\_chl\_svr\_callback                                                                                                                                            |  secure\_channel\_host.h  libusecure\_channel.so                     |  Function for processing secure channel negotiation messages.           | Processes messages sent from the client to the server during security channel negotiation. This API is called when messages are received on the server. Before calling this API, you need to initialize the network connection to the client and the message sending hook function. For details, see [examples](https://atomgit.com/openeuler/secGear/blob/master/examples/secure_channel/host/server.c#:~:text=conn_ctx.conn_kit.send).  |
| cc\_sec\_chl\_enclave\_encrypt                                                                                                                                             |    secure\_channel\_enclave.h libtsecure\_channel.so                   | Encryption API of the secure channel on the enclave.            | None  |
|   cc\_sec\_chl\_enclave\_decrypt                                                                                                                                           |   secure\_channel\_enclave.h libtsecure\_channel.so                    | Decryption API of the secure channel on the enclave.            |  None|

### Precautions

* A secure channel encapsulates only the key negotiation process and encryption and decryption APIs, but does not establish any network connection. The negotiation process reuses the network connection of the service. The network connection between the client and server is established and maintained by the service. The message sending hook function and network connection pointer are transferred during the initialization of the secure channel on the client and the server.
* The read buffer of the network message receiving function on the client and server must be big enough to contain the secure channel initialization message of 12,320 bytes.
  For details, see [secure channel examples](https://atomgit.com/openeuler/secGear/tree/master/examples/secure_channel).

# Development Tools

secGear provides a tool set to facilitate application development. This document describes the tools and how to use them.

## Code Generation Tool

### Overview

secGear codegen is used to parse the EDL file to generate intermediate C code, that is, to assist in generating code that is called between the TEE and REE.

The EDL file format defined by secGear codegen is the same as that defined by Intel SGX SDK edger8r, but the complete syntax definition of Intel is not supported:

* The public can be used only in methods. Functions without public are declared as private by default.
* Switchless calls from the REE to the TEE and from the TEE to the REE are not supported.
* The Outside Call (OCALL) does not support some calling modes (such as cdecl, stdcall, and fastcall).

The EDL file syntax is similar to the C language syntax. The following describes parts different from  the C language syntax:

| Member                     | Description                                                         |
| ----------------------- | ------------------------------------------------------------ |
| include "my\_type.h"     | Uses the type defined in the external inclusion file. After the type is added, you need to use **--search-path** to specify the search path when running the **codegen** command.                                |
| trusted                 | Declares that secure functions are available on the trusted application (TA) side. |
| untrusted               | Declares that insecure functions are available on the TA side. |
| return\_type             | Defines the return value type.                                               |
| parameter\_type | Defines the parameter type.                                                 |
| \[in, size = len] | For the ECALL, this parameter indicates that data needs to be transferred from the REE to the TEE. For the OCALL, this parameter is required for the pointer type, and size indicates the buffer that is actually used. |
| \[out, size = len]       | For the ECALL, this parameter indicates that data needs to be transferred from the TEE to the REE. For the OCALL, this parameter needs to be used for the pointer type, and size indicates the buffer that is actually used.|

### Usage Instructions

#### Command Format

The format of the codegen command is as follows:

**codegen** < --trustzone | --sgx > \[--trusted-dir \<path> | **--untrusted-dir** \<path>| --trusted | --untrusted ]  edlfile

#### Parameter Description

The parameters are described as follows:

| **Parameter**               | Mandatory/Optional | Description                                                     |
| ---------------------- | -------- | ------------------------------------------------------------ |
| --trustzone | --sgx   | Mandatory     | Generates the API function corresponding to the confidential computing architecture only in the current command directory. If no parameter is specified, the SGX API function is generated by default. |
| --search-path \<path>   | Optional     | Specifies the search path of the file that the EDL file to be converted depends on.   |
| --use-prefix           | Optional     | Adds a prefix to the proxy function name. The prefix is the name of the EDL file.   |
| --header-only          | Optional     | Specifies that the code generation tool generates only header files.   |
| --trusted-dir \<path> | Optional     | Specifies the directory where the generated TEE auxiliary code is stored. If this parameter is not specified, the current path is used by default.   |
| --untrusted-dir \<path> | Optional     | Specifies the directory where the auxiliary code for generating insecure functions is located.                         |
| --trusted              | Optional     | Generates TEE auxiliary code.                                           |
| --untrusted            | Optional     | Generates REE auxiliary code.                                         |
| edlfile                | Mandatory     | EDL file to be converted, for example, hello.edl.                          |

#### Examples

* Convert *helloworld.edl* to generate TEE auxiliary code in *enclave-directory* and generate REE auxiliary code in *host-directory*. An example command is as follows:

```shell
codegen_x86_64 --sgx --trusted-dir enclave-directory --untrusted-dir host-directory helloworld.edl
```

* Convert *helloworld.edl* to generate TEE auxiliary code in the current directory. The following is a command example for not generating REE auxiliary code:

```shell
codegen_x86_64 --sgx --trusted helloworld.edl
```

* Convert *helloworld.edl* to generate REE auxiliary code in the current directory. The following is a command example that does not generate TEE auxiliary code:

```shell
codegen_x86_64 --sgx --untrusted  helloworld.edl
```

* Convert *helloworld.edl*. An example of the command for generating TEE and REE auxiliary code in the current directory is as follows:

```shell
codegen_x86_64 --sgx helloworld.edl
```

## Signing Tool

### Overview

secGear sign\_tool is a command line tool, including the compilation tool chain and signature tool, which are used for enclave signing. The sign\_tool has two signature modes:

* Single-step signature: applies only to the debugging mode.
* Two-step signature: applies to the commercial scenario. Obtain the signature private key from a third-party platform or an independent security device to sign the enclave.

### Usage Instructions

#### Command Format

The sign\_tool contains the `sign` command (for signing the enclave) and the `digest` command (for generating the digest value).

**sign\_tool.sh -d** \[sign | digest] **-x** \<parameter>  **-i** \<file>  **-p** \<file>  **-s** \<file>  \[OPTIONS] **–o** \<file>

#### Parameter Description

| sign Command Parameter | Description                                                       | Mandatory/Optional                                      |
| -------------- | -------------------------------------------------------------| --------------------------------------------  |
| -a \<parameter> | api\_level, which identifies the GP API version of the iTrustee TA. The default value is 1.        | Optional                                          |
| -c \<file>      | Configuration file                                                       | Optional                                       |
| -d \<parameter> | Specifies the operation (sign or digest) to be performed by the signature tool.                   | Only the sign operation is performed in single-step mode. In two-step mode, the digest operation must be performed before the sign operation.  |
| -e \<file>      | Public key certificate of the device, which is used to protect the AES key for encrypting rawdata (mandatory for iTrustee). | This parameter is mandatory only for the iTrustee type. |
| -f \<parameter> | OTRP\_FLAG, which determines whether to support the OTRP standard protocol. The default value is 0.                     | Optional                                         |
| -i \<file>      | Library file to be signed.                                                 | Mandatory                                         |
| -k \<file>      | Private key (PEM file) required for one-step signature.                                     | This parameter is mandatory only for the SGX type.                              |
| -m \<file>      | Security configuration file mainfest.txt, which is configured by users.                       | Only the iTrustee type is mandatory.                         |
| -o \<file>      | Output file.                                                      | Mandatory                                         |
| -p \<file>      | Public key certificate (PEM file) of the signature server required for two-step signing.                      | Mandatory                                         |
| -s \<file>      | Signed digest value required for two-step signing.                                      | Mandatory                              |
| -t \<parameter> | TA\_TYPA, which identifies TA binary format of the iTrustee. The default value is 1.               | Optional                                         |
| -x \<parameter> | enclave type (sgx or trustzone)                              | Mandatory                                         |
| -h             | Prints the help information.                                                   | Optional                                         |

#### Single-Step Signing

Set the enclave type is SGX, sign the test.enclave, and generate the signature file signed.enclave. The following is an example:

```shell
sign_tool.sh –d sign –x sgx –i test.enclave -k private_test.pem –o signed.enclave
```

#### Two-Step Signing

The following uses SGX as an example to describe the two-step signature procedure:

1. Generate digest value.

   Use the sign\_tool to generate the digest value digest.data and the temporary intermediate file signdata. The file is used when the signature file is generated and is automatically deleted after being signed. Example:

   ```shell
   sign_tool.sh –d digest –x sgx –i input –o digest.data
   ```

2. Send digest.data to the signature authority or platform and obtain the corresponding signature.

3. Use the obtained signature to generate the signed dynamic library signed.enclave.

   ```shell
   sign_tool.sh –d sign –x sgx–i input  –p pub.pem –s signature –o signed.enclave
   ```

Note: To release an official version of applications supported by Intel SGX, you need to apply for an Intel whitelist. For details about the process, see the Intel document at <https://software.intel.com/content/www/us/en/develop/download/overview-on-signing-and-whitelisting-for-intel-software-guard-extensions-enclaves.html>.

# Secure Application Development Guide

The following example describes how to use secGear to develop a helloworld program using the C language.

## Example Download

```shell
git clone https://atomgit.com/openeuler/secGear.git
```

## Directory Structure

```shell
cd examples/helloworld

#Directory structure:
├── helloworld
│   ├── CMakeLists.txt
│   ├── enclave
│   │   ├── CMakeLists.txt
│   │   ├── Enclave.config.xml
│   │   ├── Enclave.lds
│   │   ├── hello.c
│   │   ├── manifest.txt
│   │   └── config_cloud.ini
│   ├── helloworld.edl
│   └── host
│       ├── CMakeLists.txt
│       └── main.c
```

The code body consists of three parts:

* **main.c**: REE program
* **helloworld.edl**: header file of the APIs called by the REE and TEE
* **hello.c**: TEE program

## Preparations

In addition to the preceding three parts, there are compilation project file (**CMakeLists.txt**) and developer licenses (**Enclave.config.xml**/**Enclave.lds** of Intel SGX and **manifest.txt**/**config\_cloud.ini** of Kunpeng).

> \[!NOTE]NOTE:
>
> * The Kunpeng developer license needs to be [applied for from the Huawei service owner](https://www.hikunpeng.com/document/detail/en/kunpengcctrustzone/fg-tz/kunpengtrustzone_04_0009.html).
> * Because Intel SGX is debugged in debug mode, you do not need to apply for a developer license currently. If the remote attestation service of Intel is required for commercial use, you need to [apply for a license from Intel](https://www.intel.com/content/www/us/en/developer/tools/software-guard-extensions/request-license.html).

After the application is successful, the developer license file is obtained and needs to be stored in the corresponding code directory.

## Development Procedure

Reconstructing a confidential computing application based on secGear is similar to independently extracting functional modules. The procedure is as follows: Identify sensitive data processing logic, extract it into an independent library, deploy it in the TEE, and define APIs provided by the REE in the EDL file.

The following figure shows the development procedure.

1. Develop the main function and APIs in the REE, manage the enclave, and call functions in the TEE.
2. Develop the EDL file (similar to the C language header file that defines the interaction APIs between the REE and TEE).
3. Develop TEE APIs.
4. Call the code generation tool codegener to automatically generate the interaction source code between the REE and TEE based on the EDL file and compile the source code to the binary files of the REE and TEE. The REE logic directly calls the corresponding API of the TEE without considering the automatically generated interaction code, reducing the development cost.
5. Call the signing tool to sign binary files in the TEE to implement trusted boot of the TEE program.

![](./figures/develop_step.png)

## Build and Run

### Arm Environment

```shell
# Clone secGear repository
git clone https://atomgit.com/openeuler/secGear.git

# Build secGear and examples
cd secGear
source environment
mkdir debug && cd debug && cmake -DENCLAVE=GP .. && make && sudo make install

# Run helloworld
/vendor/bin/secgear_helloworld
```

### x86 Environment

```shell
# Clone secGear repository
git clone https://atomgit.com/openeuler/secGear.git

# Build secGear and examples
cd secGear
source /opt/intel/sgxsdk/environment && source environment
mkdir debug && cd debug && cmake .. && make && sudo make install

# Run helloworld
./examples/helloworld/host/secgear_helloworld
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/appendix.md
---
# Appendix

## Command Line Parameters

**Table  1**  login command parameters

**Table  2**  logout command parameters

**Table  3**  pull command parameters

**Table  4**  rmi command parameters

**Table  5**  load command parameters

**Table  6**  images command parameters

**Table  7**  inspect command parameters

**Table  8**  tag command parameters

**Table  9**  import command parameters

**Table  10**  export command parameters

## CNI Parameters

**Table  1**  CNI single network parameters

**Table  2**  CNI args parameters

**Table  3**  CNI multiple network parameters

---

---
url: /en/docs/22.03_LTS_SP4/cloud/container_form/secure_container/appendix_2.md
---
# Appendix

## configuration.toml

> \[!NOTE] **NOTE:**\
> The value of each field in the  **configuration.toml**  file is subject to the  **configuration.toml**  file in the  **kata-containers-<***version***>.rpm package**. You cannot set any field in the configuration file.

```text
[hypervisor.qemu]
path: specifies the execution path of the virtualization QEMU.
kernel: specifies the execution path of the guest kernel.
initrd: specifies the guest initrd execution path.
image: specifies the execution path of the guest image (not applicable).
machine_type: specifies the type of the analog chip. The value is virt for the ARM architecture and pc for the x86 architecture.
kernel_params: specifies the running parameters of the guest kernel.
firmware: specifies the firmware path. If this parameter is left blank, the default firmware is used.
machine_accelerators: specifies an accelerator.
default_vcpus: specifies the default number of vCPUs for each SB/VM.
default_maxvcpus: specifies the default maximum number of vCPUs for each SB/VM.
default_root_ports: specifies the default number of root ports for each SB/VM.
default_bridges: specifies the default number of bridges for each SB/VM.
default_memory: specifies the default memory size of each SB/VM. The default value is 1024 MiB.
memory_slots: specifies the number of memory slots for each SB/VM. The default value is 10.
memory_offset: specifies the memory offset. The default value is 0.
disable_block_device_use: disables the block device from being used by the rootfs of the container.
shared_fs: specifies the type of the shared file system. The default value is virtio-9p.
virtio_fs_daemon: specifies the path of the vhost-user-fs daemon process.
virtio_fs_cache_size: specifies the default size of the DAX cache.
virtio_fs_cache: specifies the cache mode.
block_device_driver: specifies the driver of a block device.
block_device_cache_set: specifies whether to set cache-related options for a block device. The default value is false.
block_device_cache_direct: specifies whether to enable O_DIRECT. The default value is false.
block_device_cache_noflush: specifies whether to ignore device update requests. The default value is false.
enable_iothreads: enables iothreads.
enable_mem_prealloc: enables VM RAM pre-allocation. The default value is false.
enable_hugepages: enables huge pages. The default value is false.
enable_swap: enables the swap function. The default value is false.
enable_debug: enables QEMU debugging. The default value is false.
disable_nesting_checks: disables nested check.
msize_9p = 8192: specifies the number of bytes transmitted in each 9p packet.
use_vsock: uses vsocks to directly communicate with the agent (the prerequisite is that vsocks is supported). The default value is false.
hotplug_vfio_on_root_bus: enables the hot swap of the VFIO device on the root bus. The default value is false.
disable_vhost_net: disables vhost_net. The default value is false.
entropy_source: specifies the default entropy source.
guest_hook_path: specifies the binary path of the guest hook.

[factory]
enable_template: enables the VM template. The default value is false.
template_path: specifies the template path.
vm_cache_number: specifies the number of VM caches. The default value is 0.
vm_cache_endpoint: specifies the address of the Unix socket used by the VMCache. The default value is /var/run/kata-containers/cache.sock.

[proxy.kata]
path: specifies the kata-proxy running path.
enable_debug: enables proxy debugging. The default value is false.

[shim.kata]
path: specifies the running path of kata-shim.
enable_debug: enables shim debugging. The default value is false.
enable_tracing: enables shim opentracing.

[agent.kata]
enable_debug: enables the agent debugging function. The default value is false.
enable_tracing: enables the agent tracing function.
trace_mode: specifies the trace mode.
trace_type: specifies the trace type.
enable_blk_mount: enables guest mounting of the block device.

[netmon]
enable_netmon: enables network monitoring. The default value is false.
path: specifies the kata-netmon running path.
enable_debug: enables netmon debugging. The default value is false.

[runtime]
enable_debug: enables runtime debugging. The default value is false.
enable_cpu_memory_hotplug: enables CPU and memory hot swap. The default value is false.
internetworking_model: specifies the network interconnection mode between VMs and containers.
disable_guest_seccomp: disables the seccemp security mechanism in the guest application. The default value is true.
enable_tracing: enables runtime opentracing. The default value is false.
disable_new_netns: disables network namespace creation for the shim and hypervisor processes. The default value is false.
experimental: enables the experimental feature, which does not support user-defined configurations.
```

## APIs

**Table  1**  Commands related to the kata-runtime network

**Table  2**  kata-ipvs command line interfaces

---

---
url: /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/appendix_1.md
---
# Appendix

## Command Line Interface List

This section lists commands in system containers, which are different from those in common containers. For details about other commands, refer to sections related to the iSulad container engine or run the  **isula *XXX* --help**  command.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/image_builder/isula_build/isula_build_appendix.md
---
# Appendix

## Command Line Parameters

**Table 1** Parameters of the `ctr-img build` command

| **Command** | **Parameter** | **Description** |
| ------------- | -------------- | ------------------------------------------------------------ |
| ctr-img build | --build-arg | String list, which contains variables required during the build. |
| | --build-static | Key value, which is used to build binary equivalence. Currently, the following key values are included: - build-time: string, which indicates that a fixed timestamp is used to build a container image. The timestamp format is YYYY-MM-DD HH-MM-SS. |
| | -f, --filename | String, which indicates the path of the Dockerfiles. If this parameter is not specified, the current path is used. |
| | --format | String, which indicates the image format **oci** or **docker** (**ISULABUILD\_CLI\_EXPERIMENTAL** needs to be enabled). |
| | --iidfile | String, which indicates the ID of the image output to a local file. |
| | -o, --output | String, which indicates the image export mode and path.|
| | --proxy | Boolean, which inherits the proxy environment variable on the host. The default value is true. |
| | --tag | String, which indicates the tag value of the image that is successfully built. |
| | --cap-add | String list, which contains permissions required by the **RUN** instruction during the build process.|

**Table 2** Parameters of the `ctr-img load` command

| **Command** | **Parameter** | **Description** |
| ------------ | ----------- | --------------------------------- |
| ctr-img load | -i, --input | String, path of the local .tar package to be imported.|

**Table 3** Parameters of the `ctr-img push` command

| **Command** | **Parameter** | **Description** |
| ------------ | ----------- | --------------------------------- |
| ctr-img push | -f, --format | String, which indicates the pushed image format **oci** or **docker** (**ISULABUILD\_CLI\_EXPERIMENTAL** needs to be enabled).|

**Table 4** Parameters of the `ctr-img rm` command

| **Command** | **Parameter** | **Description** |
| ---------- | ----------- | --------------------------------------------- |
| ctr-img rm | -a, --all | Boolean, which is used to delete all local persistent images. |
| | -p, --prune | Boolean, which is used to delete all images that are stored persistently on the local host and do not have tags. |

**Table 5** Parameters of the `ctr-img save` command

| **Command** | **Parameter** | **Description** |
| ------------ | ------------ | ---------------------------------- |
| ctr-img save | -o, --output | String, which indicates the local path for storing the exported images.|
| ctr-img save | -f, --format | String, which indicates the exported image format **oci** or **docker** (**ISULABUILD\_CLI\_EXPERIMENTAL** needs to be enabled).|

**Table 6** Parameters of the `login` command

| **Command** | **Parameter** | **Description** |
| -------- | -------------------- | ------------------------------------------------------- |
| login | -p, --password-stdin | Boolean, which indicates whether to read the password through stdin. or enter the password in interactive mode. |
| | -u, --username | String, which indicates the username for logging in to the image repository.|

**Table 7** Parameters of the `logout` command

| **Command** | **Parameter** | **Description** |
| -------- | --------- | ------------------------------------ |
| logout | -a, --all | Boolean, which indicates whether to log out of all logged-in image repositories. |

**Table 8** Parameters of the `manifest annotate` command

| **Command**       | **Parameter** | **Description**              |
| ----------------- | ------------- | ---------------------------- |
| manifest annotate | --arch        | Set architecture             |
|                   | --os          | Set operating system         |
|                   | --os-features | Set operating system feature |
|                   | --variant     | Set architecture variant     |

## Communication Matrix

The isula-build component processes communicate with each other through the Unix socket file. No port is used for communication.

## File and Permission

* All isula-build operations must be performed by the **root** user. To perform operations as a non-privileged user, you need to configure the `--group` option.

* The following table lists the file permissions involved in the running of isula-build.

| **File Path** | **File/Folder Permission** | **Description** |
| ------------------------------------------- | ------------------- | ------------------------------------------------------------ |
| /usr/bin/isula-build                        | 550                 | Binary file of the command line tool.                                       |
| /usr/bin/isula-builder                      | 550                 | Binary file of the isula-builder process.                          |
| /usr/lib/systemd/system/isula-build.service | 640                 | systemd configuration file, which is used to manage the isula-build service.                   |
| /usr/isula-build                            | 650                 | Root directory of the isula-builder configuration file. |
| /etc/isula-build/configuration.toml         | 600                 | General isula-builder configuration file, including the settings of the isula-builder log level, persistency directory, runtime directory, and OCI runtime. |
| /etc/isula-build/policy.json                | 600                 | Syntax file of the signature verification policy file.                                 |
| /etc/isula-build/registries.toml            | 600                 | Configuration file of each image repository, including the available image repository list and image repository blacklist. |
| /etc/isula-build/storage.toml               | 600                 | Configuration file of the local persistent storage, including the configuration of the used storage driver.       |
| /etc/isula-build/isula-build.pub            | 400                 | Asymmetric encryption public key file. |
| /var/run/isula\_build.sock                   | 660                 | Local socket of isula-builder.                            |
| /var/lib/isula-build                        | 700                 | Local persistency directory.                                             |
| /var/run/isula-build                        | 700                 | Local runtime directory.                                             |
| /var/lib/isula-build/tmp/\[build\_id]/isula-build-tmp-\*.tar              | 644                 | Local temporary directory for storing the images when they are exported to iSulad.                           |

---

---
url: /en/docs/22.03_LTS_SP4/edge_computing/ros/appendix.md
---
# Appendix

A more detailed introduction to ROS can be found at [ROS wiki](https://wiki.ros.org/), [ROS docs](http://docs.ros.org/).

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secharden/appendix.md
---
# Appendix

This chapter describes the file permissions and  **umask**  values.

## Permissions on Files and Directories

Permission on files and directories in Linux specifies the users who can access and perform operations on files and directories and the access and operation modes. Permissions on files and directories include read only, write only, and execute.

The following types of users can access files and directories:

* File creator
* Users in the same group as a file creator
* Users not in the same group as a file creator

An example of permission on files and directories is described as follows:

If the permission on  **/usr/src**  is set to  **755**  which is 111101101 in binary mode, permissions for each type of users are described as follows:

* The left-most  **111**  indicates that the file owner can read, write, and execute the file.
* The middle  **101**  indicates the group users can read and execute but cannot write the file.
* The right-most  **101**  indicates that other users can read and execute but cannot write the file.

## umask Values

When a user creates a file or directory, the file or directory has a default permission. The default permission is specified by the  **umask**  value.

The  **umask**  value is the complement of the permission value. The actual permission value is obtained by subtracting the  **umask**  value from the default maximum permission value. The default maximum permission of a file is readable and writable. The default maximum permission of a directory is readable, writable, and executable. The default permission of a file is 666 minus the  **umask**  value. The default permission of a directory is 777 minus the  **umask**  value.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/appendix.md
---
# Appendix

## Terminology & Acronyms and Abbreviations

For the terminology & acronyms and abbreviation used in this document, see  [Table 1](#table201236162279)  and  [Table 2](#table1423422319271).

**Table  1**  Terminology

**Table  2**  Acronyms and abbreviations

---

---
url: /en/docs/22.03_LTS_SP4/cloud/hybrid_deployment/rubik/appendix.md
---
# Appendixes

## DaemonSet Configuration Template

```yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: rubik
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["list", "watch"]
  - apiGroups: [""]
    resources: ["pods/eviction"]
    verbs: ["create"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: rubik
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: rubik
subjects:
  - kind: ServiceAccount
    name: rubik
    namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: rubik
  namespace: kube-system
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: rubik-config
  namespace: kube-system
data:
  config.json: |
    {
      "agent": {
        "logDriver": "stdio",
        "logDir": "/var/log/rubik",
        "logSize": 1024,
        "logLevel": "info",
        "cgroupRoot": "/sys/fs/cgroup",
        "enabledFeatures": [
          "preemption"
        ]
      },
      "preemption": {
        "resource": [
          "cpu"
        ]
      }
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: rubik-agent
  namespace: kube-system
  labels:
    k8s-app: rubik-agent
spec:
  selector:
    matchLabels:
      name: rubik-agent
  template:
    metadata:
      namespace: kube-system
      labels:
        name: rubik-agent
    spec:
      serviceAccountName: rubik
      hostPID: true
      containers:
      - name: rubik-agent
        image: hub.oepkgs.net/cloudnative/rubik:latest
        imagePullPolicy: IfNotPresent
        env:
          - name: RUBIK_NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
        securityContext:
          capabilities:
            add:
            - SYS_ADMIN
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: rubiklog
          mountPath: /var/log/rubik
          readOnly: false
        - name: runrubik
          mountPath: /run/rubik
          readOnly: false
        - name: sysfs
          mountPath: /sys/fs
          readOnly: false
        - name: devfs
          mountPath: /dev
          readOnly: false
        - name: config-volume
          mountPath: /var/lib/rubik
      terminationGracePeriodSeconds: 30
      volumes:
      - name: rubiklog
        hostPath:
          path: /var/log/rubik
      - name: runrubik
        hostPath:
          path: /run/rubik
      - name: sysfs
        hostPath:
          path: /sys/fs
      - name: devfs
        hostPath:
          path: /dev
      - name: config-volume
        configMap:
          name: rubik-config
          items:
          - key: config.json
            path: config.json
```

## Dockerfile Template

```dockerfile
FROM scratch
COPY ./build/rubik /rubik
ENTRYPOINT ["/rubik"]
```

## Image Build Script

```bash
#!/bin/bash
set -e

CURRENT_DIR=$(cd "$(dirname "$0")" && pwd)
BINARY_NAME="rubik"

RUBIK_FILE="${CURRENT_DIR}/build/rubik"
DOCKERFILE="${CURRENT_DIR}/Dockerfile"
YAML_FILE="${CURRENT_DIR}/rubik-daemonset.yaml"

# Get version and release number of rubik binary
VERSION=$(${RUBIK_FILE} -v | grep ^Version | awk '{print $NF}')
RELEASE=$(${RUBIK_FILE} -v | grep ^Release | awk '{print $NF}')
IMG_TAG="${VERSION}-${RELEASE}"

# Get rubik image name and tag
IMG_NAME_AND_TAG="${BINARY_NAME}:${IMG_TAG}"

# Build container image for rubik
docker build -f "${DOCKERFILE}" -t "${IMG_NAME_AND_TAG}" "${CURRENT_DIR}"

echo -e "\n"
# Check image existence
docker images | grep -E "REPOSITORY|${BINARY_NAME}"

# Modify rubik-daemonset.yaml file, set rubik image name
sed -i "/image:/s/:.*/: ${IMG_NAME_AND_TAG}/" "${YAML_FILE}"
```

## Communication Matrix

* The Rubik service process communicates with the Kubernetes API server as a client through the list-watch mechanism to obtain information about Pods.

|Source IP Address|Source Port|Destination IP Address|Destination Port|Protocol|Port Description|Listening Port Modifiable|Authentication Method|
|----|----|----|----|----|----|----|----|
|Rubik node|32768-61000|api-server node|443|TCP|Kubernetes external resource port |No|Token|

## File Permissions

* All Rubik operations require root permissions.

* Related file permissions are as follows:

|Path|Permissions|Description|
|----|----|----|
|/var/lib/rubik|750|Directory generated after the RPM package is installed, which stores Rubik-related files|
|/var/lib/rubik/build|550|Directory for storing the Rubik binary file|
|/var/lib/rubik/build/rubik|550|Rubik binary file|
|/var/lib/rubik/rubik-daemonset.yaml|550|Rubik DaemonSet configuration template to be used for Kubernetes deployment|
|/var/lib/rubik/Dockerfile|640|Dockerfile template|
|/var/lib/rubik/build\_rubik\_image.sh|550|Rubik container image build script.|
|/var/log/rubik|640|Directory for storing Rubik log files (requires logDriver=file)|
|/var/log/rubik/rubik.log\*|600|Rubik log files|

## Constraints

### Specifications

* Drive: More than 1 GB

* Memory: More than 100 MB

## Runtime

* Only one Rubik instance can exist on a Kubernetes node.

* Rubik cannot take any CLI parameters. Rubik will fail to be started if any CLI parameter is specified.

* When the Rubik process is in the T (TASK\_STOPPED or TASK\_TRACED) OR D (TASK\_UNINTERRUPTIBLE) state, the server is unavailable and does not respond. The service becomes available after the process recovers from the abnormal state.

### Pod Priorities

* Pod priorities cannot be raised. If the priority of service A is changed from -1 to 0, Rubik will report an error.

* Adding or modifying annotations or re-applying Pod YAML configuration file does not trigger Pod rebuild. Rubik senses changes in Pod annotations through the list-watch mechanism.

* After an online service is moved to the offline group, do not move it back to the online group, otherwise QoS exception may occur.

* Do not add important system services and kernel processes to the offline group. Otherwise, they cannot be scheduled timely, causing system errors.

* Online and offline configurations for the CPU and memory must be consistent to avoid QoS conflicts between the two subsystems.

* In the scenario of hybrid service deployment, the original CPU share mechanism is restricted:
  * When both online and offline services run on a CPU, the CPU share of the offline service does not take effect.
  * If only an online or offline service runs on a CPU, its CPU share takes effect.
  * You are advised to set the Pod priority of the offline service to BestEffort.

* Priority inversion of user-mode processes, SMT, cache, NUMA load balancing, and offline service load balancing are not supported.

### Other

To prevent data inconsistency, do not manually modify cgroup or resctrl parameters of the pods, including:

* CPU cgroup directory, such as **/sys/fs/cgroup/cpu/kubepods/burstable/\<PodUID>/\<container-longid>**
  * cpu.qos\_level
  * cpu.cfs\_burst\_us

* memory cgroup directory, such as **/sys/fs/cgroup/memory/kubepods/burstable/\<PodUID>/\<container-longid>**
  * memory.qos\_level
  * memory.soft\_limit\_in\_bytes
  * memory.force\_empty
  * memory.limit\_in\_bytes
  * memory.high

* blkio cgroup directory, such as **/sys/fs/cgroup/blkio/kubepods/burstable/\<PodUID>/\<container-longid>**
  * blkio.throttle.read\_bps\_device
  * blkio.throttle.read\_iops\_device
  * blkio.throttle.write\_bps\_device
  * blkio.throttle.write\_iops\_device

* RDT cgroup directory, such as **/sys/fs/resctrl**

---

---
url: /en/docs/22.03_LTS_SP4/server/performance/atune/appendix.md
---
# Appendixes

## Acronyms and Abbreviations

**Table  1**  Terminology

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/application_dev/application_development.md
---
# Application Development Guide

This document describes the common tools used for application development and guides users to develop applications based on openEuler.

## Overview

This document describes the following four parts to guide users to use openEuler and develop code based on openEuler.

* Install and use the GCC compiler on the openEuler operating system (OS), and complete the development, compilation, and execution of simple code.
* On the openEuler OS, use the JDK built-in tool to compile and execute code.
* Install IntelliJ IDEA on the openEuler OS for Java development.
* Create an RPM package locally or using the Open Build Service (OBS).

## Intended Audience

This document is intended for all users who use the openEuler OS for code development. You are expected to:

* Have basic knowledge of the Linux OS.
* Know how to use Linux command lines.

## Symbol Conventions

The symbols that may be found in this document are defined as follows.

|  Symbol   |Description  |
|:---  |:----  |
| ![](./figures/en-us_image_0229243712.png)|Indicates a potentially hazardous situation which, if not avoided, could result in equipment damage, data loss, performance deterioration, or unanticipated results.NOTICE is used to address practices not related to personal injury.|
| ![](./figures/en-us_image_0229243671.png)|Supplements the important information in the main text.NOTE is used to address information not related to personal injury, equipment damage, and environment deterioration.|

## Command Conventions

**Table  1**  Command conventions

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/application_scenarios.md
---
# Application Scenarios

This section describes how to use the iSula.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/secure_container/application_scenarios_2.md
---
# Application Scenarios

This section describes how to use a secure container.

---

---
url: /en/docs/22.03_LTS_SP4/server/performance/atune/usage_instructions.md
---
# Application Scenarios

You can use functions provided by A-Tune through the CLI client atune-adm. This chapter describes the functions and usage of the A-Tune client.

## Overview

* You can run the  **atune-adm help/--help/-h**  command to query commands supported by atune-adm.

* All example commands are used in single-node mode. For distributed mode, specify an IP address and port number. For example:

  ```sh
  #  atune-adm -a 192.168.3.196 -p 60001 list
  ```

* The  **define**,  **update**,  **undefine**,  **collection**,  **train**, and  **upgrade**commands do not support remote execution.

* In the command format, brackets (\[]) indicate that the parameter is optional, and angle brackets (<>) indicate that the parameter is mandatory. The actual parameters prevail.

## Querying Workload Types

### list

#### Function

Query the supported profiles, and the values of Active.

#### Format

**atune-adm list**

#### Example

```sh
# atune-adm list

Support profiles:
+------------------------------------------------+-----------+
| ProfileName                                    | Active    |
+================================================+===========+
| arm-native-android-container-robox             | false     |
+------------------------------------------------+-----------+
| basic-test-suite-euleros-baseline-fio          | false     |
+------------------------------------------------+-----------+
| basic-test-suite-euleros-baseline-lmbench      | false     |
+------------------------------------------------+-----------+
| basic-test-suite-euleros-baseline-netperf      | false     |
+------------------------------------------------+-----------+
| basic-test-suite-euleros-baseline-stream       | false     |
+------------------------------------------------+-----------+
| basic-test-suite-euleros-baseline-unixbench    | false     |
+------------------------------------------------+-----------+
| basic-test-suite-speccpu-speccpu2006           | false     |
+------------------------------------------------+-----------+
| basic-test-suite-specjbb-specjbb2015           | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-hdfs-dfsio-hdd                 | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-hdfs-dfsio-ssd                 | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-bayesian                 | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-kmeans                   | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql1                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql10                    | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql2                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql3                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql4                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql5                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql6                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql7                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql8                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql9                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-tersort                  | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-wordcount                | false     |
+------------------------------------------------+-----------+
| cloud-compute-kvm-host                         | false     |
+------------------------------------------------+-----------+
| database-mariadb-2p-tpcc-c3                    | false     |
+------------------------------------------------+-----------+
| database-mariadb-4p-tpcc-c3                    | false     |
+------------------------------------------------+-----------+
| database-mongodb-2p-sysbench                   | false     |
+------------------------------------------------+-----------+
| database-mysql-2p-sysbench-hdd                 | false     |
+------------------------------------------------+-----------+
| database-mysql-2p-sysbench-ssd                 | false     |
+------------------------------------------------+-----------+
| database-postgresql-2p-sysbench-hdd            | false     |
+------------------------------------------------+-----------+
| database-postgresql-2p-sysbench-ssd            | false     |
+------------------------------------------------+-----------+
| default-default                                | false     |
+------------------------------------------------+-----------+
| docker-mariadb-2p-tpcc-c3                      | false     |
+------------------------------------------------+-----------+
| docker-mariadb-4p-tpcc-c3                      | false     |
+------------------------------------------------+-----------+
| hpc-gatk4-human-genome                         | false     |
+------------------------------------------------+-----------+
| in-memory-database-redis-redis-benchmark       | false     |
+------------------------------------------------+-----------+
| middleware-dubbo-dubbo-benchmark               | false     |
+------------------------------------------------+-----------+
| storage-ceph-vdbench-hdd                       | false     |
+------------------------------------------------+-----------+
| storage-ceph-vdbench-ssd                       | false     |
+------------------------------------------------+-----------+
| virtualization-consumer-cloud-olc              | false     |
+------------------------------------------------+-----------+
| virtualization-mariadb-2p-tpcc-c3              | false     |
+------------------------------------------------+-----------+
| virtualization-mariadb-4p-tpcc-c3              | false     |
+------------------------------------------------+-----------+
| web-apache-traffic-server-spirent-pingpo       | false     |
+------------------------------------------------+-----------+
| web-nginx-http-long-connection                 | true      |
+------------------------------------------------+-----------+
| web-nginx-https-short-connection               | false     |
+------------------------------------------------+-----------+

```

> \[!NOTE]NOTE
> If the value of Active is **true**, the profile is activated. In the example, the profile of web-nginx-http-long-connection is activated.

## Workload Type Analysis and Auto Optimization

### analysis

#### Function

Collect real-time statistics from the system to identify and automatically optimize workload types.

#### Format

**atune-adm analysis**  \[OPTIONS]

#### Parameter Description

* OPTIONS

#### Example

* Use the default model for application identification.

  ```sh
  # atune-adm analysis --characterization
  ```

* Use the default model to identify applications and perform automatic tuning.

  ```sh
  # atune-adm analysis
  ```

* Use the user-defined training model for recognition.

  ```sh
  # atune-adm analysis --model /usr/libexec/atuned/analysis/models/new-model.m
  ```

## User-defined Model

A-Tune allows users to define and learn new models. To define a new model, perform the following steps:

1. Run the **define** command to define a new profile.
2. Run the **collection** command to collect the system data corresponding to the application.
3. Run the  **train**  command to train the model.

### define

#### Function

Add a user-defined application scenarios and the corresponding profile tuning items.

#### Format

**atune-adm define**  \<service\_type> \<application\_name> \<scenario\_name> \<profile\_path>

#### Example

Add a profile whose service\_type is **test\_service**, application\_name is **test\_app**, scenario\_name is **test\_scenario**, and tuning item configuration file is **example.conf**.

```sh
# atune-adm define test_service test_app test_scenario ./example.conf
```

The **example.conf** file can be written as follows (the following optimization items are optional and are for reference only). You can also run the **atune-adm info** command to view how the existing profile is written.

```sh
 [main]
 # list its parent profile
 [kernel_config]
 # to change the kernel config
 [bios]
 # to change the bios config
 [bootloader.grub2]
 # to change the grub2 config
 [sysfs]
 # to change the /sys/* config
 [systemctl]
 # to change the system service status
 [sysctl]
 # to change the /proc/sys/* config
 [script]
 # the script extension of cpi
 [ulimit]
 # to change the resources limit of user
 [schedule_policy]
 # to change the schedule policy
 [check]
 # check the environment
 [tip]
 # the recommended optimization, which should be performed manunaly
```

### collection

#### Function

Collect the global resource usage and OS status information during service running, and save the collected information to a CSV output file as the input dataset for model training.

> \[!NOTE]NOTE
>
> * This command depends on the sampling tools such as perf, mpstat, vmstat, iostat, and sar.
> * Currently, only the Kunpeng 920 CPU is supported. You can run the  **dmidecode -t processor**  command to check the CPU model.

#### Format

**atune-adm collection**  \<OPTIONS>

#### Parameter Description

* OPTIONS

#### Example

```sh
# atune-adm collection --filename name --interval 5 --duration 1200 --output_path /home/data --disk sda --network eth0 --app_type test_type 
```

### train

#### Function

Use the collected data to train the model. Collect data of at least two application types during training. Otherwise, an error is reported.

#### Format

**atune-adm train**  \<OPTIONS>

#### Parameter Description

* OPTIONS

  | Parameter         | Description                                            |
  | ----------------- | ------------------------------------------------------ |
  | --data\_path, -d   | Path for storing CSV files required for model training |
  | --output\_file, -o | Model generated through training                       |

#### Example

Use the CSV file in the  **data**  directory as the training input. The generated model  **new-model.m**  is stored in the  **model**  directory.

```sh
# atune-adm train --data_path /home/data --output_file /usr/libexec/atuned/analysis/models/new-model.m 
```

### undefine

#### Function

Delete a user-defined profile.

#### Format

**atune-adm undefine**  \<profile>

#### Example

Delete the user-defined profile.

```sh
# atune-adm undefine test_service-test_app-test_scenario
```

## Querying Profiles

### info

#### Function

View the profile content.

#### Format

**atune-adm info**  \<profile>

#### Example

View the profile content of web-nginx-http-long-connection.

```sh
# atune-adm info web-nginx-http-long-connection

*** web-nginx-http-long-connection:

#
# nginx http long connection A-Tune configuration
#
[main]
include = default-default

[kernel_config]
#TODO CONFIG

[bios]
#TODO CONFIG

[bootloader.grub2]
iommu.passthrough = 1

[sysfs]
#TODO CONFIG

[systemctl]
sysmonitor = stop
irqbalance = stop

[sysctl]
fs.file-max = 6553600
fs.suid_dumpable = 1
fs.aio-max-nr = 1048576
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_local_port_range = 1024     65500
net.ipv4.tcp_max_tw_buckets = 5000
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_mem =  362619      483495   725238
net.ipv4.tcp_rmem = 4096         87380   6291456
net.ipv4.tcp_wmem = 4096         16384   4194304
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

[script]
prefetch = off
ethtool =  -X {network} hfunc toeplitz

[ulimit]
{user}.hard.nofile = 102400
{user}.soft.nofile = 102400

[schedule_policy]
#TODO CONFIG

[check]
#TODO CONFIG

[tip]
SELinux provides extra control and security features to linux kernel. Disabling SELinux will improve the performance but may cause security risks. = kernel
disable the nginx log = application
```

## Updating a Profile

You can update the existing profile as required.

### update

#### Function

Update the original tuning items in the existing profile to the content in the **new.conf** file.

#### Format

**atune-adm update**  \<profile> \<profile\_path>

#### Example

Change the tuning item of the profile named **test\_service-test\_app-test\_scenario** to **new.conf**.

```sh
# atune-adm update test_service-test_app-test_scenario ./new.conf
```

## Activating a Profile

### profile

#### Function

Manually activate the profile to make it in the active state.

#### Format

**atune-adm profile** \<profile>

#### Parameter Description

For details about the profile name, see the query result of the list command.

#### Example

Activate the profile corresponding to the web-nginx-http-long-connection.

```sh
# atune-adm profile web-nginx-http-long-connection
```

## Rolling Back Profiles

### rollback

#### Functions

Roll back the current configuration to the initial configuration of the system.

#### Format

**atune-adm rollback**

#### Example

```sh
# atune-adm rollback
```

## Updating Database

### upgrade

#### Function

Update the system database.

#### Format

**atune-adm upgrade**  \<DB\_FILE>

#### Parameter Description

* DB\_FILE

  New database file path.

#### Example

The database is updated to  **new\_sqlite.db**.

```sh
# atune-adm upgrade ./new_sqlite.db
```

## Querying System Information

### check

#### Function

Check the CPU, BIOS, OS, and NIC information.

#### Format

**atune-adm check**

#### Example

```sh
# atune-adm check
 cpu information:
     cpu:0   version: Kunpeng 920-6426  speed: 2600000000 HZ   cores: 64
     cpu:1   version: Kunpeng 920-6426  speed: 2600000000 HZ   cores: 64
 system information:
     DMIBIOSVersion: 0.59
     OSRelease: 4.19.36-vhulk1906.3.0.h356.eulerosv2r8.aarch64
 network information:
     name: eth0              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth1              product: HNS GE/10GE/25GE Network Controller
     name: eth2              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth3              product: HNS GE/10GE/25GE Network Controller
     name: eth4              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth5              product: HNS GE/10GE/25GE Network Controller
     name: eth6              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth7              product: HNS GE/10GE/25GE Network Controller
     name: docker0           product:
```

## Automatic Parameter Optimization

A-Tune provides the automatic search capability with the optimal configuration, saving the trouble of manually configuring parameters and performance evaluation. This greatly improves the search efficiency of optimal configurations.

### Tuning

#### Function

Use the specified project file to search the dynamic space for parameters and find the optimal solution under the current environment configuration.

#### Format

**atune-adm tuning**  \[OPTIONS] \<PROJECT\_YAML>

> \[!NOTE]NOTE\
> Before running the command, ensure that the following conditions are met:
>
> 1. The YAML configuration file on the server has been edited and stored in the **/etc/atuned/tuning/** directory of the atuned service.
> 2. The YAML configuration file of the client has been edited and stored on the atuned client.

#### Parameter Description

* OPTIONS

  > \[!NOTE]NOTE\
  > If this parameter is used, the -p parameter must be followed by a specific project name and the YAML file of the project must be specified.

* **PROJECT\_YAML**: YAML configuration file of the client.

#### Configuration Description

**Table  1**  YAML file on the server

**Table  2**  Description of object configuration items

**Table  3**  Description of configuration items of a YAML file on the client

**Table  4**  Description of evaluations configuration item

#### Example

The following is an example of the YAML file configuration on a server:

```sh
project: "compress"
maxiterations: 500
startworkload: ""
stopworkload: ""
object :
  -
    name : "compressLevel"
    info :
        desc : "The compresslevel parameter is an integer from 1 to 9 controlling the level of compression"
        get : "cat /root/A-Tune/examples/tuning/compress/compress.py | grep 'compressLevel=' | awk -F '=' '{print $2}'"
        set : "sed -i 's/compressLevel=\\s*[0-9]*/compressLevel=$value/g' /root/A-Tune/examples/tuning/compress/compress.py"
        needrestart : "false"
        type : "continuous"
        scope :
          - 1
          - 9
        dtype : "int"
  -
    name : "compressMethod"
    info :
        desc : "The compressMethod parameter is a string controlling the compression method"
        get : "cat /root/A-Tune/examples/tuning/compress/compress.py | grep 'compressMethod=' | awk -F '=' '{print $2}' | sed 's/\"//g'"
        set : "sed -i 's/compressMethod=\\s*[0-9,a-z,\"]*/compressMethod=\"$value\"/g' /root/A-Tune/examples/tuning/compress/compress.py"
        needrestart : "false"
        type : "discrete"
        options :
          - "bz2"
          - "zlib"
          - "gzip"
        dtype : "string"
```

The following is an example of the YAML file configuration on a client:

```sh
project: "compress"
engine : "gbrt"
iterations : 20
random_starts : 10

benchmark : "python3 /root/A-Tune/examples/tuning/compress/compress.py"
evaluations :
  -
    name: "time"
    info:
        get: "echo '$out' | grep 'time' | awk '{print $3}'"
        type: "positive"
        weight: 20
  -
    name: "compress_ratio"
    info:
        get: "echo '$out' | grep 'compress_ratio' | awk '{print $3}'"
        type: "negative"
        weight: 80
```

#### Example

* Perform tuning.

  ```sh
  # atune-adm tuning --project compress --detail compress_client.yaml
  ```

* Restore the initial configuration before tuning. The compress is the project name in the YAML file.

  ```sh
  # atune-adm tuning --restore --project compress
  ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/security/secharden/authentication_and_authorization.md
---
# Authentication and Authorization

## Setting a Warning for Remote Network Access

### Description

A warning for remote network access is configured and displayed for users who attempt to remotely log in to the system. The warning indicates the penalty for authorized access and is used to threaten potential attackers. When the warning is displayed, system architecture and other system information are hidden to protect the system from being attacked.

### Implementation

This setting can be implemented by modifying the  **/etc/issue.net**  file. Replace the original content in the  **/etc/issue.net**  file with the following information (which has been set by default in openEuler):

```console
Authorized users only. All activities may be monitored and reported. 
```

## Forestalling Unauthorized System Restart by Pressing Ctrl+Alt+Delete

### Description

By default, you can restart the system by pressing **Ctrl**+**Alt**+**Delete**. You are advised to disable this function to prevent data loss due to misoperations.

### Implementation

To disable the feature of restarting the system by pressing  **Ctrl**+**Alt**+**Delete**, perform the following steps:

1. Run the following commands to delete the two  **ctrl-alt-del.target**  files:

   ```shell
   rm -f /etc/systemd/system/ctrl-alt-del.target
   rm -f /usr/lib/systemd/system/ctrl-alt-del.target
   ```

2. Change  **#CtrlAltDelBurstAction=reboot-force**  to  **CtrlAltDelBurstAction=none**  in the  **/etc/systemd/system.conf**  file.

3. Run the following command to restart systemd for the modification to take effect. Note that running the command may cause system services to be unavailable or restarted temporarily. In addition, you must be the **root** user or a user with the sudo permission to perform this operation.

   ```shell
   systemctl daemon-reexec
   ```

## Setting an Automatic Exit Interval for Shell

### Description

An unattended shell is prone to listening or attacks. Therefore, it is advised that a mechanism be configured to ensure that a shell can automatically exit when it does not run for a period.

### Implementation

At the end of file  **/etc/profile**, set the  **TMOUT**  field (unit: second) that specifies the interval for automatic exit as follows:

```text
export TMOUT=300
```

## Setting the Default umask Value for Users to 0077

### Description

The  **umask**  value is used to set default permission on files and directories. A smaller  **umask**  value indicates that group users or other users have incorrect permission, which brings system security risks. Therefore, the default  **umask**  value must be set to  **0077**  for all users, that is, the default permission on user directories is  **700**  and the permission on user files is  **600**. The  **umask**  value indicates the complement of a permission. For details about how to convert the  **umask**  value to a permission, see  [umask Values](./appendix.md#umask-values).

> \[!NOTE] **NOTE:**
> By default, the  **umask**  value of the openEuler user is set to  **0022**.

### Implementation

1. Add  **umask 0077**  to the  **/etc/bashrc**  file and all files in the  **/etc/profile.d/**  directory.

   ```text
   echo "umask 0077" >> $FILE
   ```

   > \[!NOTE] **NOTE:**
   > *$FILE*  indicates the file name, for example, echo "umask 0077" >> /etc/bashrc.

2. Set the ownership and group of the  **/etc/bashrc**  file and all files in the  **/etc/profile.d/**  directory to  **root**.

   ```text
   chown root.root $FILE
   ```

   > \[!NOTE] **NOTE:**
   > *$FILE*  indicates the file name, for example,  **chown root.root /etc/bashrc**.

## Setting the GRUB2 Encryption Password

### Description

GRand Unified Bootloader (GRUB) is an operating system boot manager used to boot different systems (such as Windows and Linux). GRUB2 is an upgraded version of GRUB.

When starting the system, you can modify the startup parameters of the system on the GRUB2 screen. To ensure that the system startup parameters are not modified randomly, you need to encrypt the GRUB2 screen. The startup parameters can be modified only when the correct GRUB2 password is entered.

> \[!NOTE] **NOTE:**
> The default password of GRUB2 is  **openEuler#12**. You are advised to change the default password upon the first login and periodically update the password. If the password is leaked, startup item configurations may be modified, causing the system startup failure.

### Implementation

1. Run the  **grub2-mkpasswd-pbkdf2**  command to generate an encrypted password.

   > \[!NOTE] **NOTE:**
   > SHA-512 is used as the GRUB2 encryption algorithm.

   ```shell
   $ grub2-mkpasswd-pbkdf2
   Enter password: 
   Reenter password: 
   PBKDF2 hash of your password is 
   grub.pbkdf2.sha512.10000.5A45748D892672FDA02DD3B6F7AE390AC6E6D532A600D4AC477D25C7D087644697D8A0894DFED9D86DC2A27F4E01D925C46417A225FC099C12DBD3D7D49A7425.2BD2F5BF4907DCC389CC5D165DB85CC3E2C94C8F9A30B01DACAA9CD552B731BA1DD3B7CC2C765704D55B8CD962D2AEF19A753CBE9B8464E2B1EB39A3BB4EAB08
   ```

   > \[!NOTE] **NOTE:**
   > Enter the same password in the  **Enter password**  and  **Reenter password**  lines.\
   > After  **openEuler#12**  is encrypted by  **grub2-mkpasswd-pbkdf2**, the output is  **grub.pbkdf2.sha512.10000.5A45748D892672FDA02DD3B6F7AE390AC6E6D532A600D4AC477D25C7D087644697D8A0894DFED9D86DC2A27F4E01D925C46417A225FC099C12DBD3D7D49A7425.2BD2F5BF4907DCC389CC5D165DB85CC3E2C94C8F9A30B01DACAA9CD552B731BA1DD3B7CC2C765704D55B8CD962D2AEF19A753CBE9B8464E2B1EB39A3BB4EAB08**. The ciphertext is different each time.

2. Open  **grub.cfg**  in a vi editor. Append the following fields to the beginning of  **grub.cfg**.

   ```text
   set superusers="root"
   password_pbkdf2 root grub.pbkdf2.sha512.10000.5A45748D892672FDA02DD3B6F7AE390AC6E6D532A600D4AC477D25C7D087644697D8A0894DFED9D86DC2A27F4E01D925C46417A225FC099C12DBD3D7D49A7425.2BD2F5BF4907DCC389CC5D165DB85CC3E2C94C8F9A30B01DACAA9CD552B731BA1DD3B7CC2C765704D55B8CD962D2AEF19A753CBE9B8464E2B1EB39A3BB4EAB08
   ```

   > \[!NOTE] **NOTE:**
   >
   > * In different modes, the paths of the **grub.cfg** file are different: In the UEFI mode of the x86 architecture, the path is **/boot/efi/EFI/openEuler/grub.cfg**. In the Legacy BIOS mode of the x86 architecture, the path is **/boot/grub2/grub.cfg**. In the aarch64 architecture, the path is **/boot/efi/EFI/openEuler/grub.cfg**.
   > * The  **superusers**  field is used to set the account name of the super GRUB2 administrator.
   > * The first parameter following the  **password\_pbkdf2**  field is the GRUB2 account name, and the second parameter is the encrypted password of the account.

## Setting the Secure Single-user Mode

### Description

When you log in to the system as user  **root**  in single-user mode, if the  **root**  password is not set, high security risks exist.

### Implementation

This setting can be implemented by modifying the  **/etc/sysconfig/init**  file. Set  **SINGLE**  to  **SINGLE=/sbin/sulogin**.

## Disabling Interactive Startup

### Description

With interactive guidance, console users can disable audit, firewall, or other services, which compromises system security. Users can disable interactive startup to improve security. This item is disabled by default in openEuler.

### Implementation

This setting can be implemented by modifying the  **/etc/sysconfig/init**  file. Set  **PROMPT**  to  **no**.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/eggo_automatic_deployment.md
---
# Automatic Deployment

Manual deployment of Kubernetes clusters requires manually deploying various components. This is both time- and labor-consuming, especially during large scale Kubernetes cluster deployment, as low efficiency and errors are likely to surface. To solve the problem, openEuler launched the Kubernetes cluster deployment tool in version 21.09. This highly flexible tool provides functions such as automatic deployment and deployment process tracking of large scale Kubernetes clusters.

The following describes the usage of the Kubernetes cluster automatic deployment tool.

## Architecture Overview

![](./figures/arch.png)

The overall architecture of automatic cluster deployment is shown in the figure above. The modules are described as follows:

* GitOps: Responsible for cluster configuration management, such as updating, creating, and deleting configurations.The cluster management function is not provided in version 21.09.
* InitCluster: The meta cluster, which functions as the central cluster to manage the other service clusters.
* eggops: Custom Resource Definitions (CRDs) and controllers used to abstract the Kubernetes clusters.
* master: The master node of Kubernetes, which provides the control plane of the cluster.
* worker: The load node of Kubernetes , which carries user services.
* ClusterA, ClusterB, and ClusterC: service clusters, which carry user services.

If you are interested in the Kubernetes cluster deployment tool provided by openEuler, visit <https://atomgit.com/openeuler/eggo>.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/administrator/basic_configuration.md
---
# Basic Configuration

## Setting the System Locale

System locale settings are stored in the **/etc/locale.conf** file and can be modified by the **localectl** command. These settings are read at system boot by the systemd daemon.

### Displaying the Current Locale Status

To display the current locale status, run the following command:

```bash
$ localectl status
```

For example, to display the current system settings, the command and output are as follows:

```bash
$ localectl status
   System Locale: LANG=zh_CN.UTF-8
       VC Keymap: cn
      X11 Layout: cn
```

### Listing Available Locales

To display available locales, run the following command:

```bash
$ localectl list-locales
```

You can check that by listing all Chinese locales with the following command:

```bash
$ localectl list-locales | grep zh
zh_CN.UTF-8
```

### Setting the Locale

To set the language environment, run the following command as the **root** user. In the command,  *locale*  indicates the language type to be set. Run the  **localectl list-locales**  command to obtain the value range. Change the value as required.

```bash
$ localectl set-locale LANG=locale
```

For example, if you want to use Simplified Chinese as the locale, run the following command as the **root** user:

```bash
$ localectl set-locale LANG=zh_CN.UTF-8
```

> \[!NOTE] **NOTE:**
> After the modification, log in again or run the `source /etc/locale.conf` command as the **root** user to update the configuration file for the modification to take effect.

## Setting the Keyboard Layout

Keyboard layout settings are stored in the **/etc/locale.conf** file and can be modified by the **localectl** command. These settings are read at early boot by the systemd daemon.

### Displaying the Current Settings

To display the current keyboard layout settings, run the following command:

```bash
$ localectl status
```

Example command output:

```bash
$ localectl status
   System Locale: LANG=zh_CN.UTF-8
       VC Keymap: cn
      X11 Layout: cn
```

### Listing Available Keyboard Layouts

To list all available keyboard layouts that can be configured on openEuler, run the following command:

```bash
$ localectl list-keymaps
```

For example, the command output of the Chinese keyboard layout is as follows:

```bash
$ localectl list-keymaps | grep cn
cn
```

### Setting the Keyboard Layout

To set the keyboard layout, run the following command as the **root** user. In the command,  *map*  indicates the keyboard layout to be set. Run the  **localectl list-keymaps**  command to obtain the value range. Change it as required.

```bash
$ localectl set-keymap map
```

The keyboard layout will be equally applied to graphical user interfaces.

Then you can verify if your setting was successful by checking the status:

```bash
$ localectl status
   System Locale: LANG=zh_CN.UTF-8
       VC Keymap: cn
      X11 Layout: us
```

## Setting the Date and Time

This section describes how to set the system date, time, and time zone by using **timedatectl**, **date**, and **hwclock** commands.

### Using the timedatectl Command

#### Displaying the Current Date and Time

To display the current date and time, run the following command:

```bash
$ timedatectl
```

Example command output:

```bash
$ timedatectl
               Local time: Mon 2023-05-20 04:05:00 EDT
           Universal time: Mon 2023-05-20 08:05:00 UTC
                 RTC time: Mon 2023-05-20 08:05:00
                Time zone: China Standard Time (CST), UTC +8
System clock synchronized: no
              NTP service: inactive
          RTC in local TZ: no
```

#### Synchronizing the System Clock with a Remote Server

Your system clock can be automatically synchronized with a remote server using the Network Time Protocol (NTP). Run the following command as the **root** user  to enable or disable NTP. The value of  *boolean*  is  **yes**  or  **no**, indicating that the NTP is enabled or disabled for automatic system clock synchronization. Change the value as required.

> \[!NOTE] **NOTE:**\
> If the remote NTP server is enabled to automatically synchronize the system clock, you cannot manually change the date and time. If you need to manually change the date or time, ensure that automatic NTP system clock synchronization is disabled. You can run the  `timedatectl set-ntp no`  command to disable the NTP service.

```bash
timedatectl set-ntp boolean
```

For example, to enable automatic remote time synchronization, run the following command:

```bash
$ timedatectl set-ntp yes
```

#### Changing the Current Date

> \[!NOTE] **NOTE:**\
> Before changing the date, ensure that automatic NTP system clock synchronization has been disabled.

Run the following command as the **root** user  to change the current date. In the command,  *YYYY*  indicates the year,  *MM*  indicates the month, and  *DD*  indicates the day. Change them as required.

```bash
$ timedatectl set-time YYYY-MM-DD
```

For example, to change the current date to August 14, 2019, run the following command as the **root** user:

```bash
$ timedatectl set-time '2019-08-14'
```

#### Changing the Current Time

> Before changing the time, ensure that automatic NTP system clock synchronization has been disabled. Run the following command to check whether the NTP service is running:

```shell
$ systemctl status ntp
```

Run the following command to disable the NTP service:

```shell
$ systemctl disable ntp
```

To change the current time, run the following command as the **root** user. In the command,  *HH*  indicates the hour,  *MM*  indicates the minute, and  *SS*  indicates the second. Change them as required.

```bash
$ timedatectl set-time HH:MM:SS
```

For example, to change the current time to 15:57:24, run the following command:

```bash
$ timedatectl set-time 15:57:24
```

#### Changing the Time Zone

To list all available time zones, run the following command:

```bash
$ timedatectl list-timezones
```

To change the current time zone, run the following command as the **root** user. In the command,  *time\_zone*  indicates the time zone to be set. Change it as required.

```bash
$ timedatectl set-timezone time_zone
```

Imagine you want to identify which time zone is closest to your present location while you are in Asia. You can check that by listing all available time zones in Asia with the following command:

```bash
$ timedatectl list-timezones | grep Asia
Asia/Aden
Asia/Almaty
Asia/Amman
Asia/Anadyr
Asia/Aqtau
Asia/Aqtobe
Asia/Ashgabat
Asia/Baghdad
Asia/Bahrain
......

Asia/Seoul
Asia/Shanghai
Asia/Singapore
Asia/Srednekolymsk
Asia/Taipei
Asia/Tashkent
Asia/Tbilisi
Asia/Tehran
Asia/Thimphu
Asia/Tokyo
```

To change the time zone to Asia/Shanghai, run the following command:

```bash
$ timedatectl set-timezone Asia/Shanghai
```

### Using the date Command

#### Displaying the Current Date and Time

To display the current date and time, run the following command:

```bash
$ date
```

By default, the  **date**  command displays the local time. To display the time in Coordinated Universal Time (UTC), run the command with the **--utc** or **-u** option:

```bash
$ date --utc
```

You can also customize the format of the displayed information by providing the **+ "format"** option on the command line:

```bash
$ date +"format"
```

**Table  1**  Formatting options

* To display the current date and time:

  ```bash
  $ date 
  Sat May  20 17:26:34 CST 2023
  ```

* To display the current date and time in UTC:

  ```bash
  $ date --utc
  Sat May  20 09:26:18 UTC 2023
  ```

* To customize the output of the **date** command:

  ```bash
  $ date +"%Y-%m-%d %H:%M"
  2023-05-20 17:24
  ```

#### Changing the Current Time

To change the current time, run the **date** command with the **--set** or **-s** option. Run the following command as the **root** user. In the command,  *HH*  indicates the hour,  *MM*  indicates the minute, and  *SS*  indicates the second. Change them as required.

```bash
$ date --set HH:MM:SS
```

By default, the **date** command sets the local time. To set the system clock in UTC instead, run the command with the **--utc** or **-u** option:

```bash
$ date --set HH:MM:SS --utc
```

For example, to change the current time to 23:26:00, run the following command as the **root** user:

```bash
$ date --set 23:26:00
```

#### Changing the Current Date

To change the current date, run the command with the **--set** or **-s** option. Run the following command as the **root** user. In the command,  *YYYY*  indicates the year,  *MM*  indicates the month, and  *DD*  indicates the day. Change them as required. Note that after the date is changed, the corresponding time is reset to 00:00:00.

```bash
$ date --set YYYY-MM-DD
```

For example, to change the current date to November 2, 2019, run the following command as the **root** user:

```bash
$ date --set 2019-11-02
```

### Using the hwclock Command

You can run the **hwclock** command to set the real time clock (RTC).

#### Real-Time Clock and System Clock

Linux divides clocks into the following types:

* System clock: clock of the current Linux kernel.
* Hardware clock RTC: hardware clock of the mainboard powered by the battery. This clock can be set in the  **Standard BIOS Feature**  option of the BIOS.

When Linux starts, it reads the RTC and sets the system clock time based on the RTC time.

#### Displaying the Current Date and Time

To display the current RTC date and time, run the following command as the **root** user:

```bash
$ hwclock
```

Example command output:

```bash
$ hwclock
2023-05-26 10:18:42.528948+08:00
```

#### Setting the Date and Time

Run the following command as the **root** user  to change the date and time of the current hardware. In the command,  *dd*  indicates the day,  *mm*  indicates the month,  *yyyy*  indicates the year,  *HH*  indicates the hour, and  *MM*  indicates the minute. Change them as required.

```bash
$ hwclock --set --date "dd mm yyyy HH:MM"
```

For example, to change the current time to 21:17 on May 21, 2023, run the following command:

```bash
$ hwclock --set --date "21 May 2023 21:17" --utc
```

## Setting kdump

This section describes how to set the memory reserved for kdump and modify parameters in the kdump configuration file.

### Setting the Memory Reserved for kdump

#### Parameter Formats of the Memory Reserved for kdump

The memory reserved for kdump must be added to the bootargs in the **/boot/efi/EFI/openEuler/grub.cfg** (UEFI boot mode) or **/boot/grub2/grub.cfg** (legacy boot mode). The memory reserved for kdump has been added to openEuler releases by default and can be adjusted as required. After adding or modifying the bootargs, restart the system for the settings to take effect. The parameter formats of the memory reserved for kdump are as follows:

| Bootarg| Description| Default Value| Remarks|
|----------|----------|----------|----------|
| crashkernel=x| If the physical memory size is less than 4 GB, x of the memory is reserved for kdump.| The default value is 512 MB for x86.| This configuration method is used only when the available memory size is less than 4 GB. In this case, ensure that the available contiguous memory is sufficient for reservation.|
| crashkernel=x@y| x of the memory is reserved at the start address of y for kdump.| Unused| Ensure that x of the memory at the start address of y is not reserved for other modules.|
| crashkernel=x,high| If the physical memory size is less than 4 GB, 256 MB memory is reserved. If the physical memory size is greater than 4 GB, x of the memory is reserved for kdump. | The default value is 1024M,high for ARM64.| Ensure that the available physical contiguous memory size is greater than or equal to 256 MB when the memory size is less than 4 GB, and is greater than or equal to x when the memory size is greater than 4 GB. The actual reserved memory size is 256 MB + x. |
| crashkernel=x,low crashkernel=y,high| x of the memory is reserved for kdump when the physical memory size is less than 4 GB, and y of the memory is reserved for kdump when the physical memory size is greater than 4 GB. | Unused| Ensure that the available physical contiguous memory size is greater than or equal to x when the physical memory size is less than 4 GB, and is greater than or equal to y when the physical memory size is greater than 4 GB.|

### Recommended Reserved Memory

| Recommended Solution| Reserved Parameter| Description|
|----------|----------|----------|
| General solution| crashkernel=2048M,high| If the memory size is less than 4 GB, 256 MB is reserved for kdump. If the memory size is greater than 4 GB, 2048 MB is reserved for kdump. 256 + 2048 MB in total.|
| Economical solution| crashkernel=1024M,high| If the memory size is less than 4 GB, 256 MB is reserved for kdump. If the memory size is greater than 4 GB, 1024 MB is reserved for kdump. 256 + 1024 MB in total. It is recommended that kdump files not be dumped using the network in scenarios where the system memory size is less than 512 MB. In VM scenarios, you can reduce the reserved memory. You are advised to set crashkernel to 512M or crashkernel to 256M,high.|

> \[!NOTE] **NOTE:**
>
> If kdump files are not dumped using the network, you need to set the kdump file system not to pack network drivers. Loading the network driver requires a large amount of memory. As a result, the memory reserved for kdump may be insufficient and kdump may fail. Therefore, you are advised to disable network drivers.

### Disabling Network Drivers

In the kdump configuration file **/etc/kdump.conf**, the dracut parameters can be used to set the tailored driver module. You can configure the network driver to the tailored driver list to prevent the kdump file system from loading the driver. After the configuration file is modified, restart the kdump service for the modification to take effect. Set the dracut parameters as follows:

```text
dracut_args --omit-drivers "mdio-gpi usb_8dev et1011c rt2x00usb bcm-phy-lib mac80211_hwsim rtl8723be rndis_host hns3_cae amd vrf rtl8192cu mt76x02-lib int51x1 ppp_deflate team_mode_loadbalance smsc911x aweth bonding mwifiex_usb hnae dnet rt2x00pci vaser_pci hdlc_ppp marvell rtl8xxxu mlxsw_i2c ath9k_htc rtl8150 smc91x cortina at803x rockchip cxgb4 spi_ks8995 mt76x2u smsc9420 mdio-cavium bnxt_en ch9200 dummy macsec ice mt7601u rtl8188ee ixgbevf net1080 liquidio_vf be2net mlxsw_switchx2 gl620a xilinx_gmii2rgmii ppp_generic rtl8192de sja1000_platform ath10k_core cc770_platform realte igb c_can_platform c_can ethoc dm9601 smsc95xx lg-vl600 ifb enic ath9 mdio-octeon ppp_mppe ath10k_pci cc770 team_mode_activebackup marvell10g hinic rt2x00lib mlx4_en iavf broadcom igc c_can_pci alx rtl8192se rtl8723ae microchip lan78xx atl1c rtl8192c-common almia ax88179_178a qed netxen_nic brcmsmac rt2800usb e1000 qla3xxx mdio-bitbang qsemi mdio-mscc-miim plx_pci ipvlan r8152 cx82310_eth slhc mt76x02-usb ems_pci xen-netfront usbnet pppoe mlxsw_minimal mlxsw_spectrum cdc_ncm rt2800lib rtl_usb hnae3 ath9k_common ath9k_hw catc mt76 hns_enet_drv ppp_async huawei_cdc_ncm i40e rtl8192ce dl2 qmi_wwan mii peak_usb plusb can-dev slcan amd-xgbe team_mode_roundrobin ste10Xp thunder_xcv pptp thunder_bgx ixgbe davicom icplus tap tun smsc75xx smsc dlci hns_dsaf mlxsw_core rt2800mmi softing uPD60620 vaser_usb dp83867 brcmfmac mwifiex_pcie mlx4_core micrel team macvlan bnx2 virtio_net rtl_pci zaurus hns_mdi libcxgb hv_netvsc nicvf mt76x0u teranetics mlxfw cdc_eem qcom-emac pppox mt76-usb sierra_net i40evf bcm87xx mwifiex pegasus rt2x00mmi sja1000 ena hclgevf cnic cxgb4vf ppp_synctty iwlmvm team_mode_broadcast vxlan vsockmon hdlc_cisc rtl8723-common bsd_comp fakelb dp83822 dp83tc811 cicada fm10 8139t sfc hs geneve hclge xgene-enet-v2 cdc_mbim hdlc asix netdevsim rt2800pci team_mode_random lxt ems_usb mlxsw_pci sr9700 mdio-thunder mlxsw_switchib macvtap atlantic cdc_ether mcs7830 nicpf mdi peak_pci atl1e cdc_subset ipvtap btcoexist mt76x0-common veth slip iwldvm bcm7xxx vitesse netconsole epic100 myri10ge r8169 qede microchip_t1 liquidi bnx2x brcmutil mwifiex_sdi mlx5_core rtlwifi vmxnet3 nlmon hns3 hdlc_raw esd_usb2 atl2 mt76x2-common iwlwifi mdio-bcm-unimac national ath rtwpci rtw88 nfp rtl8821ae fjes thunderbolt-net 8139cp atl1 mscc vcan dp83848 dp83640 hdlc_fr e1000e ipheth net_failover aquantia rtl8192ee igbvf rocker intel-xway tg3" --omit "ramdisk network ifcfg qemu-net" --install "chmod" --nofscks
```

## Setting the Drive Scheduling Algorithm

### Setting a Temporary Scheduling Policy

For example, run the following command to change all I/O scheduling algorithms to **mq-deadline**. The modification becomes invalid after a reboot.

```shell
$ echo mq-deadline > /sys/block/sd*/queue/scheduler
```

### Setting a Permanent Scheduling Policy

Add **elevator=mq-deadline** to the kernel boot configuration file **grub.cfg**. The modification takes effect after a reboot.

```text
linux   /vmlinuz-5.10.0-153.12.0.89.oe2203SP4.x86_64 root=/dev/mapper/openeuler-root ro resume=/dev/mapper/openeuler-swap rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap quiet crashkernel=512M elevator=mq-deadline
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/best_practices.md
---
# Best Practices

## Performance Best Practices

### Halt-Polling

#### Overview

If compute resources are sufficient, the halt-polling feature can be used to enable virtual machines (VMs) to obtain performance similar to that of physical machines. If the halt-polling feature is not enabled, the host allocates CPU resources to other processes when the vCPU exits due to idle timeout. When the halt-polling feature is enabled on the host, the vCPU of the VM performs polling when it is idle. The polling duration depends on the actual configuration. If the vCPU is woken up during the polling, the vCPU can continue to run without being scheduled from the host. This reduces the scheduling overhead and improves the VM system performance.

> \[!NOTE] **NOTE:**\
> The halt-polling mechanism ensures that the vCPU thread of the VM responds in a timely manner. However, when the VM has no load, the host also performs polling. As a result, the host detects that the CPU usage of the vCPU is high, but the actual CPU usage of the VM is not high.

#### Procedure

The halt-polling feature is enabled by default. You can dynamically change the halt-polling time of vCPU by modifying the  **halt\_poll\_ns**  file. The default value is  **500000**, in ns.

For example, to set the polling duration to 400,000 ns, run the following command:

```shell
# echo 400000 > /sys/module/kvm/parameters/halt_poll_ns
```

### I/O Thread Configuration

#### Overview

By default, QEMU main threads handle backend VM read and write operations on the KVM. This causes the following issues:

* VM I/O requests are processed by a QEMU main thread. Therefore, the single-thread CPU usage becomes the bottleneck of VM I/O performance.
* The QEMU global lock (qemu\_global\_mutex) is used when VM I/O requests are processed by the QEMU main thread. If the I/O processing takes a long time, the QEMU main thread will occupy the global lock for a long time. As a result, the VM vCPU cannot be scheduled properly, affecting the overall VM performance and user experience.

You can configure the I/O thread attribute for the virtio-blk disk or virtio-scsi controller. At the QEMU backend, an I/O thread is used to process read and write requests of a virtual disk. The mapping relationship between the I/O thread and the virtio-blk disk or virtio-scsi controller can be a one-to-one relationship to minimize the impact on the QEMU main thread, enhance the overall I/O performance of the VM, and improve user experience.

#### Configuration Description

To use I/O threads to process VM disk read and write requests, you need to modify VM configurations as follows:

* Configure the total number of high-performance virtual disks on the VM. For example, set  **\<iothreads>**  to  **4**  to control the total number of I/O threads.

  ```xml
  <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>   
       <name>VMName</name>
       <memory>4194304</memory>
       <currentMemory>4194304</currentMemory>
       <vcpu>4</vcpu>
       <iothreads>4</iothreads>
  ```

* Configure the I/O thread attribute for the virtio-blk disk.  **\<iothread>**  indicates I/O thread IDs. The IDs start from 1 and each ID must be unique. The maximum ID is the value of  **\<iothreads>**. For example, to allocate I/O thread 2 to the virtio-blk disk, set parameters as follows:

  ```xml
  <disk type='file' device='disk'>
        <driver name='qemu' type='raw' cache='none' io='native' iothread='2'/>
        <source file='/path/test.raw'/>
        <target dev='vdb' bus='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
  </disk>
  ```

* Configure the I/O thread attribute for the virtio-scsi controller. For example, to allocate I/O thread 2 to the virtio-scsi controller, set parameters as follows:

  ```xml
  <controller type='scsi' index='0' model='virtio-scsi'>
        <driver iothread='2'/>
        <alias name='scsi0'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
  </controller>
  ```

* Bind I/O threads to a physical CPU.

  Binding I/O threads to specified physical CPUs does not affect the resource usage of vCPU threads.  **\<iothread>**  indicates I/O thread IDs, and  **\<cpuset>**  indicates IDs of the bound physical CPUs.

  ```xml
  <cputune>
  <iothreadpin iothread='1' cpuset='1-3,5,7-12' />
  <iothreadpin iothread='2' cpuset='1-3,5,7-12' />
  </cputune>
  ```

### Raw Device Mapping

#### Overview

When configuring VM storage devices, you can use configuration files to configure virtual disks for VMs, or connect block devices (such as physical LUNs and LVs) to VMs for use to improve storage performance. The latter configuration method is called raw device mapping (RDM). Through RDM, a virtual disk is presented as a small computer system interface (SCSI) device to the VM and supports most SCSI commands.

RDM can be classified into virtual RDM and physical RDM based on backend implementation features. Compared with virtual RDM, physical RDM provides better performance and more SCSI commands. However, for physical RDM, the entire SCSI disk needs to be mounted to a VM for use. If partitions or logical volumes are used for configuration, the VM cannot identify the disk.

#### Configuration Example

VM configuration files need to be modified for RDM. The following is a configuration example.

* Virtual RDM

  The following is an example of mounting the SCSI disk  **/dev/sdc**  on the host to the VM as a virtual raw device:

  ```xml
  <domain type='kvm'>
   <devices>
      ...
      <controller type='scsi' model='virtio-scsi' index='0'/>
      <disk type='block' device='disk'>
          <driver name='qemu' type='raw' cache='none' io='native'/>
          <source dev='/dev/sdc'/>
          <target dev='sdc' bus='scsi'/>
          <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      </disk>
      ...
   </devices>
  </domain>
  ```

* Physical RDM

  The following is an example of mounting the SCSI disk  **/dev/sdc**  on the host to the VM as a physical raw device:

  ```xml
  <domain type='kvm'>
   <devices>
      ...
      <controller type='scsi' model='virtio-scsi' index='0'/>
      <disk type='block' device='lun' rawio='yes'>
          <driver name='qemu' type='raw' cache='none' io='native'/>
          <source dev='/dev/sdc'/>
          <target dev='sdc' bus='scsi'/>
          <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      </disk>
      ...
   </devices>
  </domain>
  ```

### kworker Isolation and Binding

#### Overview

kworker is a per-CPU thread implemented by the Linux kernel. It is used to execute workqueue requests in the system. kworker threads will compete for physical core resources with vCPU threads, resulting in virtualization service performance jitter. To ensure that the VM can run stably and reduce the interference of kworker threads on the VM, you can bind kworker threads on the host to a specific CPU.

#### Procedure

You can modify the  **/sys/devices/virtual/workqueue/cpumask**  file to bind tasks in the workqueue to the CPU specified by  **cpumasks**. Masks in  **cpumask**  are in hexadecimal format. For example, if you need to bind kworker to CPU0 to CPU7, run the following command to change the mask to  **ff**:

```shell
# echo ff > /sys/devices/virtual/workqueue/cpumask
```

### HugePage Memory

#### Overview

Compared with traditional 4 KB memory paging, openEuler also supports 2 MB/1 GB memory paging. HugePage memory can effectively reduce TLB misses and significantly improve the performance of memory-intensive services. openEuler uses two technologies to implement HugePage memory.

* Static HugePages

  The static HugePage requires that a static HugePage pool be reserved before the host OS is loaded. When creating a VM, you can modify the XML configuration file to specify that the VM memory is allocated from the static HugePage pool. The static HugePage ensures that all memory of a VM exists on the host as the HugePage to ensure physical continuity. However, the deployment difficulty is increased. After the page size of the static HugePage pool is changed, the host needs to be restarted for the change to take effect. The size of a static HugePage can be 2 MB or 1 GB.

* THP

  If the transparent HugePage (THP) mode is enabled, the VM automatically selects available 2 MB consecutive pages and automatically splits and combines HugePages when allocating memory. When no 2 MB consecutive pages are available, the VM selects available 64 KB (AArch64 architecture) or 4 KB (x86\_64 architecture) pages for allocation. By using THP, users do not need to be aware of it and 2 MB HugePages can be used to improve memory access performance.

If VMs use static HugePages, you can disable THP to reduce the overhead of the host OS and ensure stable VM performance.

#### Procedure

* Configure static HugePages.

  Before creating a VM, modify the XML file to configure a static HugePage for the VM.

  ```xml
    <memoryBacking>
      <hugepages>
        <page size='1' unit='GiB'/>
      </hugepages>
    </memoryBacking>
  ```

  The preceding XML segment indicates that a 1 GB static HugePage is configured for the VM.

  ```xml
    <memoryBacking>
      <hugepages>
        <page size='2' unit='MiB'/>
      </hugepages>
    </memoryBacking>
  ```

  The preceding XML segment indicates that a 2 MB static HugePage is configured for the VM.

* Configure transparent HugePage.

  Dynamically enable the THP through sysfs.

  ```shell
  # echo always > /sys/kernel/mm/transparent_hugepage/enabled
  ```

  Dynamically disable the THP.

  ```shell
  # echo never > /sys/kernel/mm/transparent_hugepage/enabled
  ```

### PV-qspinlock

#### Overview

PV-qspinlock optimizes the spin lock in the virtual scenario of CPU overcommitment. It allows the hypervisor to set the vCPU in the lock context to the block state and wake up the corresponding vCPU after the lock is released. In this way, pCPU resources can be better used in the overcommitment scenario, and the compilation application scenario is optimized to reduce the compilation duration.

#### Procedure

Modify the /boot/efi/EFI/openEuler/grub.cfg configuration file of the VM, add arm\_pvspin to the startup parameter in the command line, and restart the VM for the modification to take effect. After PV-qspinlock takes effect, run the `dmesg` command on the VM. The following information is displayed:

```text
[    0.000000] arm-pv: PV qspinlocks enabled
```

> \[!NOTE] **Note:**\
> PV-qspinlock is supported only when the operating systems of the host machine and VM are both openEuler 20.09 or later and the VM kernel compilation option CONFIG\_PARAVIRT\_SPINLOCKS is set to y (default value for openEuler).

### Guest-Idle-Haltpoll

#### Overview

To ensure fairness and reduce power consumption, when the vCPU of the VM is idle, the VM executes the WFx/HLT instruction to exit to the host machine and triggers context switchover. The host machine determines whether to schedule other processes or vCPUs on the physical CPU or enter the energy saving mode. However, overheads of switching between a VM and a host machine, additional context switching, and IPI wakeup are relatively high, and this problem is particularly prominent in services where sleep and wakeup are frequently performed. The Guest-Idle-Haltpoll technology indicates that when the vCPU of a VM is idle, the WFx/HLT is not executed immediately and VM-exit occurs. Instead, polling is performed on the VM for a period of time. During this period, the tasks of other vCPUs that share the LLC on the vCPU are woken up without sending IPI interrupts. This reduces the overhead of sending and receiving IPI interrupts and the overhead of VM-exit, thereby reducing the task wakeup latency.

> !\[!NOTE] **Note:**
> The execution of the `idle-haltpoll` command by the vCPU on the VM increases the CPU overhead of the vCPU on the host machine. Therefore, it is recommended that the vCPU exclusively occupy physical cores on the host machine when this feature is enabled.

#### Procedure

The Guest-Idle-Haltpoll feature is disabled by default. The following describes how to enable this feature.

1. Enable the Guest-Idle-Haltpoll feature.
   * If the processor architecture of the host machine is x86, you can configure hint-dedicated in the XML file of the VM on the host machine to enable this feature. In this way, the status that the vCPU exclusively occupies the physical core can be transferred to the VM through the VM XML configuration. The host machine ensures the status of the physical core exclusively occupied by the vCPU.

     ```xml
     <domain type='kvm'>
      ...
      <features>
        <kvm>
          ...
          <hint-dedicated state='on'/>
        </kvm>
      </features>
       ...
     </domain>
     ```

     Alternatively, log into the VM to perform online configuration at the VM granularity. This method does not rely on the host to configure the vCPU to exclusively occupy the physical core.

     ```shell
     echo Y > /sys/module/cpuidle_haltpoll/parameters/force
     ```

   * If the processor architecture of the host machine is AArch64, this feature can be enabled only by logging into the VM to perform online configuration at the VM granularity. This method does not rely on the host to configure the vCPU to exclusively occupy the physical core.

     ```shell
     echo Y > /sys/module/cpuidle_haltpoll/parameters/force
     ```

2. Check whether the Guest-Idle-Haltpoll feature takes effect. Run the following command on the VM. If haltpoll is returned, the feature has taken effect.

   ```shell
   # cat /sys/devices/system/cpu/cpuidle/current_driver
   ```

3. (Optional) Set the Guest-Idle-Haltpoll parameter.

   The following configuration files are provided in the /sys/module/haltpoll/parameters/ directory of the VM. You can adjust the configuration parameters based on service characteristics.

   * guest\_halt\_poll\_ns: a global parameter that specifies the maximum polling duration after the vCPU is idle. The default value is 200000 (unit: ns).
   * guest\_halt\_poll\_shrink:  a divisor that is used to shrink the current vCPU guest\_halt\_poll\_ns when the wakeup event occurs after the global guest\_halt\_poll\_ns time. The default value is 2.
   * guest\_halt\_poll\_grow:  a multiplier that is used to extend the current vCPU guest\_halt\_poll\_ns when the wakeup event occurs after the current vCPU guest\_halt\_poll\_ns and before the global guest\_halt\_poll\_ns. The default value is 2.
   * guest\_halt\_poll\_grow\_start: When the system is idle, the guest\_halt\_poll\_ns of each vCPU reaches 0. This parameter is used to set the initial value of the current vCPU guest\_halt\_poll\_ns to facilitate scaling in and scaling out of the vCPU polling duration. The default value is 50000 (unit: ns).
   * guest\_halt\_poll\_allow\_shrink: a switch that is used to enable vCPU guest\_halt\_poll\_ns scale-in. The default value is Y. (Y indicates enabling the scale-in; N indicates disabling the scale-in.)

   You can run the following command as the user root to change the parameter values: In the preceding command, *value* indicates the parameter value to be set, and *configFile* indicates the corresponding configuration file.

   ```shell
   # echo value > /sys/module/haltpoll/parameters/configFile
   ```

   For example, to set the global guest\_halt\_poll\_ns to 200000 ns, run the following command:

   ```shell
   # echo 200000 > /sys/module/haltpoll/parameters/guest_halt_poll_ns
   ```

### NVMe Drive Passthrough

#### Overview

The device passthrough technology is a hardware-based virtualization solution. With this technology, VMs can be directly connected to specified physical passthrough devices. To improve VM storage performance, you can use the PCI passthrough technology to pass through NVMe drives to VMs.

#### Procedure

1. Make preparations.

   * Ensure that the driver provided by the NVMe drive vendor is installed in the guest OS. Otherwise, the NVMe drive cannot work properly.
   * Ensure that the VT-d and VT-x support of the CPU is enabled on the host OS.
   * Ensure that the IOMMU function of the kernel is enabled on the host OS.
   * Ensure that the interrupt remapping function of the kernel is enabled on the host OS.

2. Obtain the PCI BDF information of an NVMe drive.

   Run the **lspci** command on the host to obtain the resource list of PCI devices on the host.

   ```shell
   # lspci -vmm
   Slot: 81:00.1
   Class: Non-Volatile memory controller
   ...
   ```

   In the command output, **Slot** indicates the PCI BDF number of the NVMe drive, **81** indicates the bus number, **00** indicates the slot number, and **1** indicates the function number.

3. Mount a PCI passthrough NVMe drive to a VM.

   When creating a VM, add the PCI NVMe drive passthrough configuration option to the corresponding XML configuration file. The configuration file is as follows:

   ```xml
   <hostdev mode='subsystem' type='pci' managed='yes'>
       <source>
           <address domain='0x0000' bus='0x81' slot='0x00' function='0x1' />
       </source>
   </hostdev>
   ```

   * **hostdev.source.address.domain**: domain number of the PCI device on the host OS.
   * **hostdev.source.address.bus**: bus number of the PCI device on the host OS.
   * **hostdev.source.address.slot**: slot number of the PCI device on the host OS.
   * **hostdev.source.address.function**: function number of the PCI device on the host OS.

4. Specify a PCI BAR of the NVMe drive.

   To further maximize the performance of the NVMe drive, you need to specify a BAR for PCI MSI-X interrupts of the passthrough NVMe drive in the guest OS. The configuration is as follows:

   ```xml
   <hostdev mode='subsystem' type='pci' managed='yes'>
       <source>
           <address domain='0x0000' bus='0x01' slot='0x00' function='0x0' />
       </source>
       <alias name='ua-sm2262'/>
           <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
   </hostdev>
   <qemu:commandline>
       <qemu:arg value='-set'/>
       <qemu:arg value='device.ua-sm2262.x-msix-relocation=bar2'/>
   </qemu:commandline>
   ```

   In the preceding XML configuration, the interrupt information of the passthrough NVMe drive is processed on BAR 2. After this configuration is added, the performance of the NVMe drive in the guest OS is almost the same as that of that in the host OS.

## security Best Practices

### Libvirt Authentication

#### Overview

When a user uses libvirt remote invocation but no authentication is performed, any third-party program that connects to the host's network can operate VMs through the libvirt remote invocation mechanism. This poses security risks. To improve system security, openEuler provides the libvirt authentication function. That is, users can remotely invoke a VM through libvirt only after identity authentication. Only specified users can access the VM, thereby protecting VMs on the network.

#### Enabling Libvirt Authentication

By default, the libvirt remote invocation function is disabled on openEuler. This following describes how to enable the libvirt remote invocation and libvirt authentication functions.

1. Log in to the host.

2. Modify the libvirt service configuration file  **/etc/libvirt/libvirtd.conf**  to enable the libvirt remote invocation and libvirt authentication functions. For example, to enable the TCP remote invocation that is based on the Simple Authentication and Security Layer (SASL) framework, configure parameters by referring to the following:

   ```conf
   #Transport layer security protocol. The value 0 indicates that the protocol is disabled, and the value 1 indicates that the protocol is enabled. You can set the value as needed.
   listen_tls = 0
   #Enable the TCP remote invocation. To enable the libvirt remote invocation and libvirt authentication functions, set the value to 1.
   listen_tcp = 1
   #User-defined protocol configuration for TCP remote invocation. The following uses sasl as an example.
   auth_tcp = "sasl" 
   ```

3. Modify the  **/etc/sasl2/libvirt.conf**  configuration file to set the SASL mechanism and SASLDB.

   ```conf
   #Authentication mechanism of the SASL framework.
   mech_list: digest-md5
   #Database for storing usernames and passwords
   sasldb_path: /etc/libvirt/passwd.db
   ```

4. Add the user for SASL authentication and set the password. Take the user  **userName**  as an example. The command is as follows:

   ```shell
   # saslpasswd2 -a libvirt userName
   Password:
   Again (for verification):
   ```

5. Modify the  **/etc/sysconfig/libvirtd**  configuration file to enable the libvirt listening option.

   ```conf
   LIBVIRTD_ARGS="--listen"
   ```

6. Restart the libvirtd service to make the modification to take effect.

   ```shell
   # systemctl restart libvirtd
   ```

7. Check whether the authentication function for libvirt remote invocation takes effect. Enter the username and password as prompted. If the libvirt service is successfully connected, the function is successfully enabled.

   ```shell
   # virsh -c qemu+tcp://192.168.0.1/system
   Please enter your authentication name: openeuler
   Please enter your password:
   Welcome to virsh, the virtualization interactive terminal.

   Type:  'help' for help with commands
          'quit' to quit

   virsh #
   ```

#### Managing SASL

The following describes how to manage SASL users.

* Query an existing user in the database.

  ```shell
  # sasldblistusers2 -f /etc/libvirt/passwd.db
  user@localhost.localdomain: userPassword
  ```

* Delete a user from the database.

  ```shell
  # saslpasswd2 -a libvirt -d user
  ```

### qemu-ga

#### Overview

QEMU guest agent (qemu-ga) is a daemon running within VMs. It allows users on a host OS to perform various management operations on the guest OS through outband channels provided by QEMU. The operations include file operations (open, read, write, close, seek, and flush), internal shutdown, VM suspend (suspend-disk, suspend-ram, and suspend-hybrid), and obtaining of VM internal information (including the memory, CPU, NIC, and OS information).

In some scenarios with high security requirements, qemu-ga provides the blacklist function to prevent internal information leakage of VMs. You can use a blacklist to selectively shield some functions provided by qemu-ga.

> \[!NOTE] **NOTE:**\
> The qemu-ga installation package is  **qemu-guest-agent-***xx***.rpm**. It is not installed on openEuler by default.  *xx*  indicates the actual version number.

#### Procedure

To add a qemu-ga blacklist, perform the following steps:

1. Log in to the VM and ensure that the qemu-guest-agent service exists and is running.

   ```shell
   # systemctl status qemu-guest-agent |grep Active
      Active: active (running) since Wed 2018-03-28 08:17:33 CST; 9h ago
   ```

2. Query which  **qemu-ga**  commands can be added to the blacklist:

   ```shell
   # qemu-ga --blacklist ?
   guest-sync-delimited
   guest-sync
   guest-ping
   guest-get-time
   guest-set-time
   guest-info
   ...
   ```

3. Set the blacklist. Add the commands to be shielded to  **--blacklist**  in the  **/usr/lib/systemd/system/qemu-guest-agent.service**  file. Use spaces to separate different commands. For example, to add the  `guest-file-open`  and  `guest-file-close`  commands to the blacklist, configure the file by referring to the following:

   ```text
   [Service]
   ExecStart=-/usr/bin/qemu-ga \
         --blacklist=guest-file-open guest-file-close
   ```

4. Restart the qemu-guest-agent service.

   ```shell
   # systemctl daemon-reload
   # systemctl restart qemu-guest-agent
   ```

5. Check whether the qemu-ga blacklist function takes effect on the VM, that is, whether the  **--blacklist**  parameter configured for the qemu-ga process is correct.

   ```shell
   # ps -ef|grep qemu-ga|grep -E "blacklist=|b="
   root       727     1  0 08:17 ?        00:00:00 /usr/bin/qemu-ga --method=virtio-serial --path=/dev/virtio-ports/org.qemu.guest_agent.0 --blacklist=guest-file-open guest-file-close guest-file-read guest-file-write guest-file-seek guest-file-flush -F/etc/qemu-ga/fsfreeze-hook
   ```

   > \[!NOTE] **NOTE:**\
   > For more information about qemu-ga, visit  <https://wiki.qemu.org/Features/GuestAgent>.

### sVirt Protection

#### Overview

In a virtualization environment that uses the discretionary access control (DAC) policy only, malicious VMs running on hosts may attack the hypervisor or other VMs. To improve security in virtualization scenarios, openEuler uses sVirt for protection. sVirt is a security protection technology based on SELinux. It is applicable to KVM virtualization scenarios. A VM is a common process on the host OS. In the hypervisor, the sVirt mechanism labels QEMU processes corresponding to VMs with SELinux labels. In addition to types which are used to label virtualization processes and files, different categories are used to label different VMs. Each VM can access only file devices of the same category. This prevents VMs from accessing files and devices on unauthorized hosts or other VMs, thereby preventing VM escape and improving host and VM security.

#### Enabling sVirt Protection

**Enabling SELinux on the Host**

1. Log in to the host.

2. Enable the SELinux function on the host.
   1. Modify the system startup parameter file  **grub.cfg**  to set  **selinux**  to  **1**.

      ```text
      selinux=1
      ```

   2. Modify  **/etc/selinux/config**  to set the  **SELINUX**  to  **enforcing**.

      ```shell
      SELINUX=enforcing
      ```

3. Restart the host.

   ```shell
   # reboot
   ```

**Creating a VM Where the sVirt Function Is Enabled**

1. Add the following information to the VM configuration file:

   ```xml
   <seclabel type='dynamic' model='selinux' relabel='yes'/>
   ```

   Or check whether the following configuration exists in the file:

   ```xml
   <seclabel type='none' model='selinux'/>
   ```

2. Create a VM.

   ```shell
   # virsh define openEulerVM.xml
   ```

**Checking Whether sVirt Is Enabled**

Run the following command to check whether sVirt protection has been enabled for the QEMU process of the running VM. If  **svirt\_t:s0:c**  exists, sVirt protection has been enabled.

```shell
# ps -eZ|grep qemu |grep "svirt_t:s0:c"
system_u:system_r:svirt_t:s0:c200,c947 11359 ? 00:03:59 qemu-kvm
system_u:system_r:svirt_t:s0:c427,c670 13790 ? 19:02:07 qemu-kvm
```

### VM Trusted Boot

#### Overview

Trusted boot includes measure boot and remote attestation. The measure boot function is mainly provided by virtualization component. The remote attestation function is enabled by users who install related software (RA client) on VMs and set up the RA server.

The two basic elements for measure boot are the root of trust (RoT) and chain of trust. The basic idea is to establish a RoT in the computer system. The trustworthiness of the RoT is ensured by physical security, technical security, and management security, that is, CRTM (Core Root of Trust for Measurement). A chain of trust is established, starting from the RoT to the BIOS/BootLoader, operating system, and then to the application. The measure boot and trust is performed by one level to the previous level. Finally, the trust is extended to the entire system. The preceding process looks like a chain, so it is called a chain of trust.

The CRTM is the root of the measure boot and the first component of the system startup. No other code is used to check the integrity of the CRTM. Therefore, as the starting point of the chain of trust, it must be an absolutely trusted source of trust. The CRTM needs to be technically designed as a segment of read-only or strictly restricted code to defend against BIOS attacks and prevent remote injection of malicious code or modification of startup code at the upper layer of the operating system. In a physical host, the CPU microcode is used as the CRTM. In a virtualization environment, the sec part of the vBIOS is generally the CRTM.

During startup, the previous component measures (calculates the hash value) the next component, and then extends the measurement value to the trusted storage area, for example, the PCR of the TPM. The CRTM measurement BootLoader extends the measurement value to the PCR, and the BootLoader measurement OS extends the measurement value to the PCR.

#### Configuring the vTPM Device to Enable Measurement Startup

**Installing the swtpm and libtpms Software**

swtpm provides a TPM emulator (TPM 1.2 and TPM 2.0) that can be integrated into a virtualization environment. So far, it has been integrated into QEMU and serves as a prototype system in RunC. swtpm uses libtpms to provide TPM 1.2 and TPM 2.0 simulation functions.
Currently, openEuler 22.03 LTS provides the libtpms and swtpm sources. You can run the `yum` command to install them.

```shell
# yum install libtpms swtpm swtpm-devel swtpm-tools
```

**Configuring the vTPM Device for the VM**

1. Add the following information to the VM configuration file:

   ```xml
   <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
       ...
       <devices>
           ...
           <tpm model='tpm-tis'>
               <backend type='emulator' version='2.0'/>
           </tpm>
           ...
       </devices>
       ...
   </domain>
   ```

   > !\[!NOTE] **Note:**\
   > Do not configure the ACPI feature for a VM running openEuler 20.09 in the AArch64 architecture, because the VM trusted boot  does not support the ACPI feature. Otherwise, the VM cannot recognize the vTPM device after startup. For openEuler earlier than version 22.03 in the AArch64 architecture, set the value of **tpm model** to **\<tpm model='tpm-tis-device'>**.

2. Create the VM.

   ```shell
   # virsh define MeasuredBoot.xml
   ```

3. Start the VM.

   Before starting the VM, run the `chmod` command to grant the following permission to the /var/lib/swtpm-localca/ directory. Otherwise, the libvirt cannot start the swtpm.

   ```shell
   chmod -R 777 /var/lib/swtpm-localca/
   virsh start MeasuredbootVM
   ```

**Confirming that the Measure Boot Is Successfully Enabled**

The vBIOS determines whether to enable the measure boot function. Currently, the vBIOS in openEuler 22.03 LTS has the measure boot capability. If the host machine uses the edk2 component of another version, check whether the edk2 component supports the measure boot function.

Log in to the VM as user root and check whether the TPM driver, tpm2-tss protocol stack, and tpm2-tools are installed on the VM.
By default, the tpm driver (tpm\_tis.ko), tpm2-tss protocol stack, and tpm2-tools are installed in openEuler 22.03 LTS. If another OS is used, run the following command to check whether the driver and related tools are installed:

```shell
# lsmod |grep tpm
# tpm_tis          16384   0
#
# yum list installed | grep -E 'tpm2-tss|tpm2-tools'
#
# yum install tpm2-tss tpm2-tools
```

You can run the `tpm2_pcrread` (`tpm2_pcrlist` in tpm2\_tools of earlier versions) command to list all PCR values.

```shell
# tpm2_pcrread
sha1 :
  0  : fffdcae7cef57d93c5f64d1f9b7f1879275cff55
  1  : 5387ba1d17bba5fdadb77621376250c2396c5413
  2  : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236
  3  : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236
  4  : e5d40ace8bb38eb170c61682eb36a3020226d2c0
  5  : 367f6ea79688062a6df5f4737ac17b69cd37fd61
  6  : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236
  7  : 518bd167271fbb64589c61e43d8c0165861431d8
  8  : af65222affd33ff779780c51fa8077485aca46d9
  9  : 5905ec9fb508b0f30b2abf8787093f16ca608a5a
  10 : 0000000000000000000000000000000000000000
  11 : 0000000000000000000000000000000000000000
  12 : 0000000000000000000000000000000000000000
  13 : 0000000000000000000000000000000000000000
  14 : 0000000000000000000000000000000000000000
  15 : 0000000000000000000000000000000000000000
  16 : 0000000000000000000000000000000000000000
  17 : ffffffffffffffffffffffffffffffffffffffff
  18 : ffffffffffffffffffffffffffffffffffffffff
  19 : ffffffffffffffffffffffffffffffffffffffff
  20 : ffffffffffffffffffffffffffffffffffffffff
  21 : ffffffffffffffffffffffffffffffffffffffff
  22 : ffffffffffffffffffffffffffffffffffffffff
  23 : 0000000000000000000000000000000000000000
sha256 :
  0  : d020873038268904688cfe5b8ccf8b8d84c1a2892fc866847355f86f8066ea2d
  1  : 13cebccdb194dd916f2c0c41ec6832dfb15b41a9eb5229d33a25acb5ebc3f016
  2  : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969
  3  : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969
  4  : 07f9074ccd4513ef1cafd7660f9afede422b679fd8ad99d25c0659eba07cc045
  5  : ba34c80668f84407cd7f498e310cc4ac12ec6ec43ea8c93cebb2a688cf226aff
  6  : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969
  7  : 65caf8dd1e0ea7a6347b635d2b379c93b9a1351edc2afc3ecda700e534eb3068
  8  : f440af381b644231e7322babfd393808e8ebb3a692af57c0b3a5d162a6e2c118
  9  : 54c08c8ba4706273f53f90085592f7b2e4eaafb8d433295b66b78d9754145cfc
  10 : 0000000000000000000000000000000000000000000000000000000000000000
  11 : 0000000000000000000000000000000000000000000000000000000000000000
  12 : 0000000000000000000000000000000000000000000000000000000000000000
  13 : 0000000000000000000000000000000000000000000000000000000000000000
  14 : 0000000000000000000000000000000000000000000000000000000000000000
  15 : 0000000000000000000000000000000000000000000000000000000000000000
  16 : 0000000000000000000000000000000000000000000000000000000000000000
  17 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  18 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  19 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  20 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  21 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  22 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  23 : 0000000000000000000000000000000000000000000000000000000000000000
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/application_dev/building_an_rpm_package.md
---
# Building an RPM Package

This section describes how to build an RPM software package on a local PC or using OBS. For details, see the [openEuler Packaging Guide](https://atomgit.com/openeuler/community/blob/master/en/contributors/packaging.md).

## Packaging Description

### Principles

During RPM packaging, the source code is compiled. The composed configuration files and binary command files need to be placed in proper positions. The RPM package needs to be tested as required. A workspace is required for these operations.

After installing rpmdevtools by running `dnf install rpmdevtools*`, you can run `rpmdev-setuptree` to generate a set of standard workspacesin the **/root** directory (or **/home/*user*** directory if the command is run by a non-root user). The directory structure is as follows:

```shell
$ tree rpmbuild
rpmbuild
├── BUILD
├── RPMS
├── SOURCES
├── SPECS
└── SRPMS
```

The content is described as follows:

The  **~/rpmbuild/SPECS**  directory contains the configuration file of the RPM package, which is the drawing of the RPM package. This file tells the  **rpmbuild**  command how to build the RPM package. The  **Macro Code**  column contains the corresponding directories in the .spec file, which is similar to the macro or global variable in the programming language.

### Packaging Process

The packaging process is as follows:

1. Place the source code in  **%\_sourcedir**.
2. Compile the source code in  **%\_builddir**. Generally, the source code is compressed and needs to be decompressed first.
3. Install the RPM package. The installation is similar to pre-assembling the software package. Copy the contents (such as binary files, configuration files, and man files) that should be contained in the software package to  **%\_buildrootdir**  and assemble the contents based on the actual directory structure after installation. For example, if binary commands are stored in  **/usr/bin**, copy the directory structure to  **%\_buildrootdir**.
4. Perform necessary configurations, such as preparations before installation and cleanup after installation. These are configured in the SPEC file to tell the  **rpmbuild**  command how to build.
5. Check whether the software is running properly.
6. The generated RPM package is stored in  **%\_rpmdir**, and the source code package is stored in  **%\_srcrpmdir**.

In the SPEC file, each phase is described as follows:

|  Phase              | Directory to Read    |   Directory to Write      | Action             |
|-------------------|--------------|-----------------|-------------------------------------------|
| %prep | %\_sourcedir | %\_builddir | Read the source code and patches in the **%\_sourcedir** directory. Then, decompress the source code to the **%\_builddir** subdirectory and apply all patches. |
| %build | %\_builddir | %\_builddir |Compile files in the **%\_builddir** build directory. Run a command similar to `./configure && make`.|
| %install | %\_builddir | %\_buildrootdir |Read files in the **%\_builddir** build directory and install them to the **%\_buildrootdir** directory. These files are generated after the RPM is installed.|
| %check | %\_builddir | %\_builddir | Check whether the software is running properly. Run a command similar to `make test`.|
| bin | %\_buildrootdir | %\_rpmdir| Read files in the **%\_buildrootdir** final installation directory to create RPM packages in the **%\_rpmdir** directory. In this directory, RPM packages of different architectures are stored in different subdirectories. The **noarch** directory stores RPM packages applicable to all architectures. These RPM files are the RPM packages that are finally installed by users.  |
| src | %\_sourcedir | %\_srcrpmdir | Create the source code RPM package (SRPM for short, with the file name extension **.src.rpm**) and save it to the **%\_srcrpmdir** directory. The SRPM package is usually used to review and upgrade software packages. |

### Packaging Options

Run the  **rpmbuild**  command to build the software package. The  **rpmbuild**  command can be used to build software packages by building .spec, .tar, and source files.

The format of the  **rpmbuild**  command is rpmbuild \[*option*...]

The common rpmbuild packaging options are described as follows.

**Table  1**  rpmbuild Packaging Options

| *option* value | Description |
|----------|--------------|
|-bp *specfile* |Starts build from the **%prep** phase of the *specfile* (decompress the source code package and install the patch).|
|-bc *specfile* |Starts build from the **%install** phase of the *specfile*.|
|-bi *specfile* |Starts build from the **%build** phase of the *specfile*.|
|-bl *specfile* |Uses the *specfile* to build the source code package and binary package.|
|-ba *specfile* |Starts check from the **%files** phase of the *specfile*.|
|-bb *specfile* |Uses the *specfile* to build the source code package.|
|-bs *specfile* |Uses the *specfile* to build the binary package.|
|-rp *sourcefile* |Starts build from the **%build** phase of the *sourcefile*.|
|-rc *sourcefile* |Starts build from the **%prep** phase of the *sourcefile* (decompress the source code package and install the patch).|
|-ri *sourcefile* |Starts build from the **%files** phase of the *sourcefile*.|
|-rl *sourcefile* |Starts build from the **%install** phase of the *sourcefile*.|
|-ra *sourcefile* |Uses the *sourcefile* to build the binary package.|
|-rb *sourcefile* |Uses the *sourcefile* to build the source code package and binary package.|
|-rs *sourcefile* |Starts build from the **%prep** phase of the *tarfile* (decompress the source code package and install the patch).|
|-tp *tarfile* |Uses the *sourcefile* to build the source code package.|
|-tc *tarfile* |Starts build from the **%install** phase of the *tarfile*.|
|-ti *tarfile* |Starts build from the **%build** phase of the *tarfile*.|
|-ta *tarfile* |Uses the *tarfile* to build the binary package.|
|-tb *tarfile* |Uses the *tarfile* to build the source code package and binary package.|
|-ts *tarfile* |During the build, uses *DIRECTORY* to overwrite the default **/root** directory.|
|--buildroot=*DIRECTORY* |Uses the *tarfile* to build the source code package.|
|--clean |No actual build steps are performed. It can be used to test the SPEC file.|
|--nobuild |Deletes the files in the BUILD directory.|
|--noclean |Skips the **%check** phase of the SPEC file (even if it does exist).|
|--nocheck |Skips the **%clean** phase of the SPEC file (even if it does exist).|
|--dbpath *DIRECTORY* |Sets *DIRECTORY* to the highest level. The default value is **/**, indicating the highest level.|
|--root *DIRECTORY* |Uses the database in *DIRECTORY* instead of the default directory **/var/lib/rpm**.|
|--rebuild *sourcefile* |Builds a new binary package based on `--recompile`. When the build is complete, the build directory, source code, and SPEC file are deleted. The deletion effect is the same as that of `--clean`.|
|--recompile *sourcefile* |Installs the specified source code package *sourcefile*, that is, start preparation, compilation, and installation of the source code package.|
|-?，--help |Displays detailed version information.|
|--version |Displays detailed help information.|

## Building an RPM Package Locally

This section uses an example to describe how to build an RPM software package locally.

### Setting Up the Development Environment

#### Prerequisites

You have obtained the  **root**  permission, and have configured a repo source for openEuler.

#### Procedure

You can use the DNF tool to install rpmdevtools, including the  **rpm-build**  command and related dependencies (such as make and gdb). Run the following command:

```shell
dnf install rpmdevtools*
```

### Creating a Hello World RPM Package

The following uses the packaging process of the GNU Hello World project as an example. The package contains the most common peripheral components related to the typical Free and Open Source Software (FOSS) project, including the configuration, compilation, and installation environments, documents, and internationalization (i18n) information.

#### Obtaining the Source Code

Run the following command to download the source code of the official example:

```shell
rpmdev-setuptree
cd ~/rpmbuild/SOURCES
wget http://ftp.gnu.org/gnu/hello/hello-2.10.tar.gz
```

#### Editing the SPEC File

Run the following command to create the .spec file in the **~/rpmbuild/SPECS** directory:

```shell
cd ~/rpmbuild/SPECS
vi hello.spec 
```

Write the corresponding content to the file and save the file. The following is an example of the file content. Modify the corresponding fields based on the actual requirements.

```text
Name:     hello
Version:  2.10
Release:  1%{?dist}
Summary:  The "Hello World" program from GNU
Summary(zh_CN): GNU Hello World program
License:  GPLv3+
URL:      http://ftp.gnu.org/gnu/hello
Source0:  http://ftp.gnu.org/gnu/hello/%{name}-%{version}.tar.gz

BuildRequires:  gettext
Requires(post): info
Requires(preun): info

%description
The "Hello World" program, done with all bells and whistles of a proper FOSS
project, including configuration, build, internationalization, help files, etc.

%description -l zh_CN
The Hello World program contains all parts required by the FOSS project, including configuration, build, i18n, and help files.

%prep
%setup -q

%build
%configure
make %{?_smp_mflags}

%install
make install DESTDIR=%{buildroot}
%find_lang %{name}
rm -f %{buildroot}/%{_infodir}/dir

%post
/sbin/install-info %{_infodir}/%{name}.info %{_infodir}/dir || :

%preun
if [ $1 = 0 ] ; then
/sbin/install-info --delete %{_infodir}/%{name}.info %{_infodir}/dir || :
fi

%files -f %{name}.lang
%doc AUTHORS ChangeLog NEWS README THANKS TODO
%license COPYING
%{_mandir}/man1/hello.1.*
%{_infodir}/hello.info.*
%{_bindir}/hello

%changelog
* Thu Dec 26 2019 Your Name <youremail@xxx.xxx> - 2.10-1
- Update to 2.10
* Sat Dec 3 2016 Your Name <youremail@xxx.xxx> - 2.9-1
- Update to 2.9
```

* The  **Name**  tag indicates the software name, the  **Version**  tag indicates the version number, and the  **Release**  tag indicates the release number.
* The  **Summary**  tag is a brief description. The first letter of the tag must be capitalized to prevent the rpmlint tool (packaging check tool) from generating alarms.
* The  **License**  tag describes the protocol version of the software package. The packager is responsible for checking the license status of the software, which can be implemented by checking the source code or license file or communicating with the author.
* The  **Group**  tag is used to classify software packages by  **/usr/share/doc/rpm/GROUPS**. Currently, this tag has been discarded. However, the VIM template still has this tag. You can delete it. However, adding this tag does not affect the system. The  **%changelog**  tag should contain the log of changes made for each release, especially the description of the upstream security/vulnerability patches. The  **%changelog**  tag should contain the version string to avoid the rpmlint tool from generating alarms.
* If multiple lines are involved, such as %changelog or %description, start from the next line of the instruction and end with a blank line.
* Some unnecessary lines (such as BuildRequires and Requires) can be commented out with a number sign (#) at the beginning of the lines.
* The default values of  **%prep**,  **%build**,  **%install**, and  **%files**  are retained.

#### Building an RPM Package

Run the following command in the directory where the .spec file is located to build the source code, binary files, and software packages that contain debugging information:

```shell
rpmbuild -ba hello.spec 
```

Run the following command to view the execution result:

```shell
$ tree ~/rpmbuild/*RPMS

/home/testUser/rpmbuild/RPMS
└── aarch64
    ├── hello-2.10-1.aarch64.rpm
    ├── hello-debuginfo-2.10-1.aarch64.rpm
    └── hello-debugsource-2.10-1.aarch64.rpm
/home/testUser/rpmbuild/SRPMS
└── hello-2.10-1.src.rpm
```

## Building an RPM Package Using the OBS

This section describes how to build RPM software packages using the OBS on the web page or with OSC. There are two methods:

* Modifying an existing software package: Modify the source code of an existing software package and build the modified source code into an RPM software package.
* Adding a software package: A new software source file is developed from scratch, and the newly developed source file is used to build an RPM software package.

### OBS Overview

OBS is a general compilation framework based on the openSUSE distribution. It is used to build source code packages into RPM software packages or Linux images. OBS uses the automatic distributed compilation mode and supports the compilation of images and installation packages of multiple Linux OS distributions (such as openEuler, SUSE, and Debian) on multiple architecture platforms (such as x86 and ARM64).

OBS consists of the backend and frontend. The backend implements all core functions. The frontend provides web applications and APIs for interaction with the backend. In addition, OBS provides an API command line client OSC, which is developed in an independent repository.

OBS uses the project organization software package. Basic permission control, related repository, and build targets (OS and architecture) can be defined in the project. A project can contain multiple subprojects. Each subproject can be configured independently to complete a task.

### Building an RPM Software Package Online

This section describes how to build an RPM software package online on OBS.

#### Building an Existing Software Package

> \[!NOTE] **NOTE:**
>
> * If you use OBS for the first time, register an individual account on the OBS web page.
> * With this method, you must copy the modified code and commit it to the code directory before performing the following operations. The code directory is specified in the  **\_service**  file.

To modify the source code of the existing software and build the modified source file into an RPM software package on the OBS web client, perform the following steps:

1. Log in to OBS at <https://build.openeuler.openatom.cn>.

2. Click  **All Projects**. The  **All Projects**  page is displayed.

3. Click the project to be modified. The project details page is displayed. For example, click  **openEuler:Mainline**.

4. On the project details page, search for the software package to be modified and click the software package name. The software package details page is displayed.

5. Click  **Branch package**. In the displayed dialog box, click  **Accept**, as shown in  [Figure 1](#fig77646143214).

   **Figure  1** **Branch Confirmation**  page\
   ![](./figures/branch-confirmation-page.png)

6. Click the  **\_service**  file to go to the editing page, modify the file content, and click  **Save**. An example of the  **\_service**  file content is as follows.  *userCodeURL*  and  *userCommitID*  indicate the user code path and commission version number or branch, respectively.

   ```xml
   <services>
       <service name="tar_scm_kernel">
           <param name="scm">git</param>
           <param name="url">userCodeURL</param>
           <param name="revision">userCommitID</param>
       </service>
       <service name="recompress">
           <param name="compression">bz2</param>
           <param name="file">*.tar</param>
        </service>
   </services>
   ```

   > \[!NOTE] **NOTE:**
   > Click  **Save**  to save the  **\_service**  file. OBS downloads the source code from the specified URL to the software directory of the corresponding OBS project based on the  **\_service**  file description and replaces the original file. For example, the  **kernel**  directory of the  **openEuler:Mainline**  project in the preceding example.

7. After the files are copied and replaced, OBS automatically starts to build the RPM software package. Wait until the build is complete and view the build status in the status bar on the right.
   * **succeeded**: The build is successful. You can click  **succeeded**  to view the build logs, as shown in  [Figure 2](#fig10319114217337).

     **Figure  2** **Succeeded**  page\
     ![](./figures/succeeded-page.png)

   * **failed**: The build failed. Click  **failed**  to view error logs, locate the fault, and rebuild again.

   * **unresolvable**: The build is not performed. The possible cause is that the dependency is missing.

   * **disabled**: The build is manually closed or is queuing for build.

   * **excluded**: The build is prohibited. The possible cause is that the .spec file is missing or the compilation of the target architecture is prohibited in the .spec file.

#### Adding a Software Package

To add a new software package on the OBS web page, perform the following steps:

1. Log in to the OBS console.

2. Select a project based on the dependency of the new software package. That is, click  **All Projects**  and select the corresponding project, for example,  **openEuler:Mainline**.

3. Click a software package in the project. The software package details page is displayed.

4. Click  **Branch package**. On the confirmation page that is displayed, click  **Accept**.

5. Click  **Delete package**  to delete the software package in the new subproject, as shown in  [Figure 3](#fig18306181103615).

   **Figure  3**  Deleting a software package from a subproject\
   ![](./figures/deleting-a-software-package-from-a-subproject.png)

   > \[!NOTE] **NOTE:**
   > The purpose of creating a project by using existing software is to inherit the dependency such as the environment. Therefore, you need to delete these files.

6. Click  **Create Package**. On the page that is displayed, enter the software package name, title, and description, and click  **Create**  to create a software package, as shown in  [Figure 4](#fig6762111693811)  and  [Figure 5](#fig18351153518389).

   **Figure  4** **Create Package**  page\
   ![](./figures/create-package-page.png)

   **Figure  5**  Creating a software package\
   ![](./figures/creating-a-software-package.png)

7. Click  **Add file**  to upload the .spec file and the file to be compiled (specified in the .spec file), as shown in  [Figure 6](#fig1475845284011).

   **Figure  6** **Add file**  page\
   ![](./figures/add-file-page.png)

8. After the file is uploaded, OBS automatically starts to build the RPM software package. Wait until the build is complete and view the build status in the status bar on the right.
   * **succeeded**: The build is successful. You can click  **succeeded**  to view the build logs.
   * **failed**: The build failed. Click  **failed**  to view error logs, locate the fault, and rebuild again.
   * **unresolvable**: The build is not performed. The possible cause is that the dependency is missing.
   * **disabled**: The build is manually closed or is queuing for build.
   * **excluded**: The build is prohibited. The possible cause is that the .spec file is missing or the compilation of the target architecture is prohibited in the .spec file.

#### Obtaining the Software Package

After the RPM software package is built, perform the following operations to obtain the RPM software package on the web page:

1. Log in to the OBS console.

2. Click  **All Projects**  and find the project corresponding to the required software package, for example,  **openEuler:Mainline**.

3. Click the name of the required software package in the project. The software package details page is displayed, for example, the  **kernel**  page in the preceding example.

4. Click the  **Repositories**  tab. On the software repository management page that is displayed, click  **Enable**  in  **Publish Flag**  to enable the RPM software package download function (the status changes from  ![](./figures/en-us_image_0229243704.png)  to  ![](./figures/en-us_image_0229243702.png)), as shown in  [Figure 7](#fig17480830144217).

   **Figure  7** **Repositories**  page\
   ![](./figures/repositories-page.png)

5. Click the project name in the  **Repository**  column. On the RPM software package download page that is displayed, click  **Download**  on the right of the RPM software package to download the RPM software package, as shown in  [Figure 8](#fig12152145615438).

   **Figure  8**  RPM software package download page\
   ![](./figures/rpm-software-package-download-page.png)

### Building a Software Package Using OSC

This section describes how to use the OBS command line tool OSC to create a project and build an RPM software package.

#### Installing and Configuring the OSC

##### Prerequisites

You have obtained the  **root**  permission, and have configured a repo source for openEuler.

##### Procedure

1. Install the OSC command line tool and its dependency as the  **root**  user.

   ```shell
   dnf install osc build
   ```

   > \[!NOTE] **NOTE:**
   > The compilation of RPM software packages depends on build.

2. Configure the OSC.
   1. Run the following command to open the  **~/.oscrc**  file:

      ```shell
      vi ~/.oscrc
      ```

   2. Add the  **user**  and  **pass**  fields to  **~/.oscrc**. The values of  *userName*  and  *passWord*  are the account and password registered on the OBS website (<https://build.openeuler.openatom.cn\\>).

      ```text
      [general]
      apiurl = https://build.openeuler.openatom.cn
      [https://build.openeuler.openatom.cn]
      user=userName
      pass=passWord
      ```

#### Building an Existing Software Package

**Creating a Project**

1. You can copy an existing project to create a subproject of your own. For example, to copy the  **zlib**  software package in the  **openEuler:Mainline**  project to the new branch, run the following command:

   ```shell
   osc branch openEuler:Mainline zlib
   ```

   If the following information is displayed, a new branch project  **home:testUser:branches:openEuler:Mainline**  is created for user  **testUser**.

   ```console
   A working copy of the branched package can be checked out with:
   osc co home:testUser:branches:openEuler:Mainline/zlib
   ```

2. Download the configuration file (for example,  **\_service**) of the software package to be modified to the local directory. In the preceding command,  *testUser*  indicates the account name configured in the  **~/.oscrc**  configuration file. Change it based on the actual requirements.

   ```shell
   osc co home:testUser:branches:openEuler:Mainline/zlib
   ```

   Information similar to the following is displayed:

   ```console
   A    home:testUser:branches:openEuler:Mainline
   A    home:testUser:branches:openEuler:Mainline/zlib
   A    home:testUser:branches:openEuler:Mainline/zlib/_service
   ```

3. Go to the local subproject directory and synchronize the remote code of the software package to the local host.

   ```shell
   cd home:testUser:branches:openEuler:Mainline/zlib
   osc up -S
   ```

   Information similar to the following is displayed:

   ```console
   A    _service:tar_scm_kernel_repo:0001-Neon-Optimized-hash-chain-rebase.patch
   A    _service:tar_scm_kernel_repo:0002-Porting-optimized-longest_match.patch
   A    _service:tar_scm_kernel_repo:0003-arm64-specific-build-patch.patch
   A    _service:tar_scm_kernel_repo:zlib-1.2.11-optimized-s390.patch
   A    _service:tar_scm_kernel_repo:zlib-1.2.11.tar.xz
   A    _service:tar_scm_kernel_repo:zlib-1.2.5-minizip-fixuncrypt.patch
   A    _service:tar_scm_kernel_repo:zlib.spec
   ```

**Building an RPM Package**

1. Rename the source file and add the renamed source file to the temporary storage of OBS.

   ```shell
   rm -f _service;for file in `ls | grep -v .osc`;do new_file=${file##*:};mv $file $new_file;done
   osc addremove *
   ```

2. Modify the source code and .spec file, and run the following command to update the file.

   ```shell
   osc up
   ```

3. Synchronize all modifications of the corresponding software package to the OBS server. The following is an example of command. The information after the **-m** parameter indicates the submmission record.

   ```shell
   osc ci -m "commit log"
   ```

4. Run the following command to obtain the repository name and architecture of the current project:

   ```shell
   osc repos home:testUser:branches:openEuler:Mainline
   ```

5. After the modification is committed, OBS automatically compiles the software package. You can run the following command to view the compilation logs of the corresponding repository. In the command, *standard\_aarch64*  and  *aarch64*  indicate the repository name and architecture obtained in the command output.

   ```shell
   osc buildlog standard_aarch64  aarch64
   ```

   > \[!NOTE] **NOTE:**
   > You can also open the created project on the web client to view the build logs.

#### Adding a Software Package

To use the OSC tool of OBS to add a new software package, perform the following steps:

**Creating a Project**

1. Create a project based on the dependency of the new software package and a proper project. For example, to create a project based on  **zlib**  of the  **openEuler:Mainline**  project, run the following command (**zlib**  is any software package in the project):

   ```shell
   osc branch openEuler:Mainline zlib
   ```

2. Delete unnecessary software packages added during project creation. For example, to delete the  **zlib**  software package, run the following command:

   ```shell
   cd home:testUser:branches:openEuler:Mainline
   osc rm zlib
   osc commit -m "commit log"
   ```

3. Create a software package in your own project. For example, to add the  **my-first-obs-package**  software package, run the following command:

   ```shell
   mkdir my-first-obs-package 
   cd my-first-obs-package
   ```

**Building an RPM Package**

1. Add the prepared source file and .spec file to the software package directory.

2. Modify the source code and .spec file, and upload all files of the corresponding software package to the OBS server. The following is a command example. The information after the  **-m**  parameter is the commission record.

   ```shell
   cd home:testUser:branches:openEuler:Mainline
   osc add my-first-obs-package
   osc ci -m "commit log"
   ```

3. Run the following command to obtain the repository name and architecture of the current project:

   ```shell
   osc repos home:testUser:branches:openEuler:Mainline
   ```

4. After the modification is committed, OBS automatically compiles the software package. You can run the following command to view the compilation logs of the corresponding repository. In the command,  *standard\_aarch64*  and  *aarch64*  indicate the repository name and architecture obtained in the command output.

   ```shell
   cd home:testUser:branches:openEuler:Mainline/my-first-obs-package
   osc buildlog standard_aarch64  aarch64
   ```

   > \[!NOTE] **NOTE:**
   > You can also open the created project on the web client to view the build logs.

#### Obtaining the Software Package

After the RPM software package is built, run the following command to obtain the RPM software package using the OSC:

```shell
osc getbinaries home:testUser:branches:openEuler:Mainline my-first-obs-package standard_aarch64 aarch64
```

The parameters in the command are described as follows. You can modify the parameters according to the actual situation.

* *home:testUser:branches:openEuler:Mainline*: name of the project to which the software package belongs.
* *my-first-obs-package*: name of the software package.
* *standard\_aarch64*: repository name.
* *aarch64*: repository architecture name.

> \[!NOTE] **NOTE:**
> You can also obtain the software package built using OSC from the web page. For details, see  [Obtaining the Software Package](#obtaining-the-software-package).

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/system_container/configurable_cgroup_path.md
---
# cgroup路径可配置

## 功能描述

系统容器提供在宿主机上进行容器资源隔离和预留的能力。通过--cgroup-parent参数，可以将容器使用的cgroup目录指定到某个特定目录下，从而达到灵活分配宿主机资源的目的。例如可以设置容器a、b、c的cgroup父路径为/lxc/cgroup1，容器d、e、f的cgroup父路径为/lxc/cgroup2，这样通过cgroup路径将容器分为两个group，实现容器cgroup组层面的资源隔离。

## 参数说明

除了通过命令行指定单个系统容器对应的cgroup父路径外，还可通过修改iSulad容器引擎启动配置文件，指定所有容器的cgroup路径。

## 约束限制

* 如果daemon端和客户端都设置了cgroup parent参数，最终以客户端指定的--cgroup-parent生效。
* 如果已启动容器A，然后启动容器B，容器B的cgroup父路径指定为容器A的cgroup路径，在删除容器的时候需要先删除容器B再删除容器A，否则会导致cgroup资源残留。

## 使用示例

启动系统容器，指定--cgroup-parent参数：

```sh
[root@localhost ~]# isula run -tid --cgroup-parent /lxc/cgroup123 --system-container --external-rootfs /root/myrootfs none init
115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
```

查看容器init进程的cgroup信息：

```sh
[root@localhost ~]# isula inspect -f "{{json .State.Pid}}" 11
22167
[root@localhost ~]# cat /proc/22167/cgroup
13:blkio:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
12:perf_event:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
11:cpuset:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
10:pids:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
9:rdma:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
8:devices:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
7:hugetlb:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
6:memory:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
5:net_cls,net_prio:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
4:cpu,cpuacct:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
3:files:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
2:freezer:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
1:name=systemd:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e/init.scope
0::/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
```

可以看到容器的cgroup父路径被设置为/sys/fs/cgroup/\<controller>/lxc/cgroup123

同时，对于所有容器cgroup父路径的设置可以配置一下容器daemon文件，例如：

```conf
{
         "cgroup-parent": "/lxc/cgroup123",
}
```

然后重启容器引擎，配置生效。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/checking_the_container_health_status.md
---
# Checking the Container Health Status

## Scenarios

In the production environment, bugs are inevitable in applications provided by developers or services provided by platforms. Therefore, a management system is indispensable for periodically checking and repairing applications. The container health check mechanism adds a user-defined health check function for containers. When a container is created, the  **--health-cmd**  option is configured so that commands are periodically executed in the container to monitor the health status of the container based on return values.

## Configuration Methods

Configurations during container startup:

```sh
isula run -itd --health-cmd "echo iSulad >> /tmp/health_check_file || exit 1" --health-interval 5m --health-timeout 3s --health-exit-on-unhealthy  busybox bash
```

The configurable options are as follows:

* **--health-cmd**: This option is mandatory. If  **0**  is returned after a command is run in a container, the command execution succeeds. If a value other than  **0**  is returned, the command execution fails.
* **--health-interval**: interval between two consecutive command executions. The default value is  **30s**. The value ranges from  **1s**  to the maximum value of Int64 (unit: nanosecond). If the input parameter is set to  **0s**, the default value is used.
* **--health-timeout**: maximum duration for executing a single check command. If the execution times out, the command execution fails. The default value is  **30s**. The value ranges from  **1s**  to the maximum value of Int64 (unit: nanosecond). If the input parameter is set to  **0s**, the default value is used.
* **--health-start-period**: container initialization time. The default value is  **0s**. The value ranges from  **1s**  to the maximum value of Int64 (unit: nanosecond).
* **--health-retries**: maximum number of retries for the health check. The default value is  **3**. The maximum value is the maximum value of Int32.
* **--health-exit-on-unhealthy**: specifies whether to kill a container when it is unhealthy. The default value is  **false**.

## Check Rules

1. After a container is started, the container status is  **health:starting**.
2. After the period specified by  **start-period**, the  **cmd**  command is periodically executed in the container at the interval specified by  **interval**. That is, after the command is executed, the command will be executed again after the specified period.
3. If the  **cmd**  command is successfully executed within the time specified by  **timeout**  and the return value is  **0**, the check is successful. Otherwise, the check fails. If the check is successful, the container status changes to  **health:healthy**.
4. If the  **cmd**  command fails to be executed for the number of times specified by  **retries**, the container status changes to  **health:unhealthy**, and the container continues the health check.
5. When the container status is  **health:unhealthy**, the container status changes to  **health:healthy**  if a check succeeds.
6. If  **--exit-on-unhealthy**  is set, and the container exits due to reasons other than being killed (the returned exit code is  **137**), the health check takes effect only after the container is restarted.
7. When the  **cmd**  command execution is complete or times out, the iSulad daemon will record the start time, return value, and standard output of the check to the configuration file of the container. A maximum of five records can be recorded. In addition, the configuration file of the container stores health check parameters.
8. When the container is running, the health check status is written into the container configurations. You can run the  **isula inspect**  command to view the status.

```conf
"Health": {
    "Status": "healthy",
    "FailingStreak": 0,
    "Log": [
        {
            "Start": "2018-03-07T07:44:15.481414707-05:00",
            "End": "2018-03-07T07:44:15.556908311-05:00",
            "ExitCode": 0,
            "Output": ""
        },
        {
            "Start": "2018-03-07T07:44:18.557297462-05:00",
            "End": "2018-03-07T07:44:18.63035891-05:00",
            "ExitCode": 0,
            "Output": ""
        },
        ......
}
```

## Usage Restrictions

* A maximum of five health check status records can be stored in a container. The last five records are saved.
* If health check parameters are set to  **0**  during container startup, the default values are used.
* After a container with configured health check parameters is started, if iSulad daemon exits, the health check is not executed. After iSulad daemon is restarted, the health status of the running container changes to  **starting**. Afterwards, the check rules are the same as above.
* If the health check fails for the first time, the health check status will not change from  **starting**  to  **unhealthy**  until the specified number of retries (**--health-retries**) is reached, or to  **healthy**  until the health check succeeds.

---

---
url: /en/docs/22.03_LTS_SP4/cloud.md
---


---

---
url: >-
  /en/docs/22.03_LTS_SP4/tools/community_tools/oepkgs/co_construction_and_future_of_oepkgs.md
---
# Co-construction and Future of oepkgs

## Contribution as an Individual

![](./public_sys_resources/contrib-oepkgs.png)

1. Submit a PR to create a repository.

   Submit a PR in the [oepkgs-management](https://gitee.com/oepkgs/oepkgs-management) repository and fill in two configuration files. After the PR is merged, the repository creation robot ci-robot automatically creates a repository under [src-oepkgs](https://gitee.com/src-oepkgs).

   > \[!NOTE] **NOTE:**
   >
   > * The oepkgs repository classifies software packages by domain and category. Software packages of different domains and categories are maintained by different **SIGs**.
   > * Open source software can be introduced to **an existing SIG of oepkgs** by submitting a PR and modifying the **sig-info.yaml** file of the SIG. It is not mandatory to create a SIG when a software package is introduced.

   For example, if Nginx is to be introduced, the configuration files in the oepkgs-management repository would be **sig-info.yaml** and **nginx.yaml**.

   Fields in **sig-info.yaml**:

   | Field| Description| Mandatory|
   |---|---|---|
   | name | SIG name, which is related to the software package domain.| √ |
   | description | Description of the SIG.| √ |
   | mailing\_list | Subscribed email addresses of the SIG| × |
   | meeting\_url | SIG meeting link| × |
   | maintainers | Manager of the SIG, responsible for review and merging of PRs of the SIG repositories.| √ |
   | repositories | SIG repositories| √ |
   | committers | Committers of SIG repositories are responsible for review and merging of PRs of the corresponding repositories.| √ |

   Fields in **nginx.yaml**:

   | Field| Description| Mandatory|
   |---|---|---|
   | name | Package name (repository name)| √ |
   | description | Software package description| √ |
   | upstream | Upstream repository address of software package| √ |
   | branches | Repository branches| √ |

2. Add source code files.

   * After step 1 is complete, the <https://gitee.com/src-oepkgs/nginx> repository is generated within 5 minutes. Add source code files to the repository through PRs. The source code files include **nginx.spec** that can be used to build RPM packages and **nginx-2.12.0.tar.bz2**. For details, see <https://gitee.com/src-oepkgs/nginx>.

   > \[!NOTE] **NOTE:**
   >
   > * After a PR is submitted, the PR quality gate build test will be performed within 5 to 30 minutes. The test result will be submitted as a comment under the PR. It is recommended that the PR be merged after **Build\_Result** is displayed as **SUCCESS**.
   > * The maintainer specified in the **oepkgs-management/sig/virtual/sig-info.yaml** configuration file can merge the PR by commenting **/lgtm** and **/approve** under the PR.

3. Build the software package.

   oepkgs provides a mature CI/CD system to support software package source code building, binary scanning, and basic function verification, ensuring reliable quality and continuous evolution of the software repository.

## Future plan

Services for more users and developers

* More services will be opened to users and developers for wider participation, promoting the improvement of the oepkgs service.

Software package patch management

* A software package patch management system has been planned to enhance the display of binary package security hardening information. The query platform uses software package patch management to provide more comprehensive binary package information.

Continuous construction

* The openEuler expansion repository will be continuously developed to include more software. oepkgs works with the official openEuler repository to promote the development of the openEuler ecosystem.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/maintenance/common_skills/common_configurations.md
---
# Common Skills

## Configuring the Network

1. Configure the IP address.

   Run the **ip** command to configure an address for the interface. **interface-name** indicates the name of the NIC.

   ```shell
   ip addr [ add | del ] address dev interface-name
   ```

2. Configure a static IP address.

   ```shell
   $ Configure the static IP address.
   ip address add 192.168.0.10/24 dev enp3s0

   # Run the following command as the root user to query the configuration result:
   ip addr show dev enp3s0

   # The result is as follows:
   2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
   link/ether 52:54:00:aa:ad:4a brd ff:ff:ff:ff:ff:ff
   inet 192.168.202.248/16 brd 192.168.255.255 scope global dynamic noprefixroute enp3s0
   valid_lft 9547sec preferred_lft 9547sec
   inet 192.168.0.10/24 scope global enp3s0
   valid_lft forever preferred_lft forever
   inet6 fe80::32e8:cc22:9db2:f4d4/64 scope link noprefixroute
   valid_lft forever preferred_lft forever
   ```

3. Configure a static route.

   Run the **ip route add** command to add a static route to the routing table and run the **ip route del** command to delete a static route. The common format of the **ip route** command is as follows:

   ```shell
   ip route [ add | del | change | append | replace ] destination-address
   ```

   * To add a static route to the host address, run the following command as the **root** user:

   ```shell
   ip route add 192.168.2.1 via 10.0.0.1 [dev interface-name]
   ```

   * To add a static route to the network, run the following command as the **root** user:

   ```shell
   ip route add 192.168.2.0/24 via 10.0.0.1 [dev interface-name]
   ```

4. Configure the network using the ifcfg file.

   Modify the **ifcfg-enp4s0** file generated in the **/etc/sysconfig/network-scripts/ directory** as the **root** user. The following is an example:

   ```text
   TYPE=Ethernet
   PROXY_METHOD=none
   BROWSER_ONLY=no
   BOOTPROTO=none
   IPADDR=192.168.0.10
   PREFIX=24
   DEFROUTE=yes
   IPV4_FAILURE_FATAL=no
   IPV6INIT=yes
   IPV6_AUTOCONF=yes
   IPV6_DEFROUTE=yes
   IPV6_FAILURE_FATAL=no
   IPV6_ADDR_GEN_MODE=stable-privacy
   NAME=enp4s0static
   UUID=xx
   DEVICE=enp4s0
   ONBOOT=yes
   ```

## Managing RPM Packages

The full name of RPM is RPM Package Manager, which is intended to manage Red Hat software packages. It is used in mainstream distributions such as openEuler, Fedora, Red Hat, Mandriva, SUSE and YellowDog, and distributions developed based on these distributions.

RPM installs the required software to a set of management programs on the Linux host in database record mode. The software to be installed is compiled and packaged, and the default database record in the packaged software records the dependencies required for the software installation. When a user installs the software on a Linux host, RPM checks whether the dependencies on the Linux host meets the requirements based on the data recorded in it.

* If yes, install the software.
* If no, do not install the software.

During the installation, all software information is written into the RPM database for subsequent query, verification, and uninstallation.

![en-us\_other\_0000001337581224](./images/en-us_other_0000001337581224.jpeg)

1. Default installation path of the RPM packages

   Generally, RPM uses the default installation path. (The default installation path can be queried by running a command and will be described in detail in subsequent sections.) All installation files are distributed to the directories listed in the following table by type.

   Table 1 RPM installation paths and their meanings

   |Installation Path|Description|
   |--|--|
   |/etc/|Configuration file installation directory|
   |/usr/bin/|Installation directory of the executable commands|
   |/usr/lib/|Path for storing the function library used by the program|
   |/usr/share/doc|Location where the basic software user manual is saved|
   |/usr/share/man/|Path for saving the help file|

   Note: You can manually specify the installation path of RPM, but this method is not recommended. After the installation path is manually specified, all installation files are installed in the specified path, and the command for querying the installation path in the system cannot be used. The command can be identified by the system only after being manually configured.

2. rpm command options

   * **Checking the RPM Signature of the Software Package**

   Before installing the RPM package on a Linux host, check the GPG signature. After ensuring that the signature integrity and source are correct, run the **rpm --checksig** command to verify the validity:

   ```shell
   rpm --checksig nano-2.3.1-10.el7.x86_64.rpm
   ```

   * **Installing RPM Packages**

   To install RPM packages in Linux, use the **-i** option in the **rpm** command.

   ```shell
   rpm -ivh nano-2.3.1-10.el7.x86_64.rpm
   ```

   * **-i**: installs the software package.

   * **-v**: displays detailed information.

   * **-h**: lists flags during suite installation.

   * **Querying an Installed RPM Package**

   To query an RPM package (dnf) installed in the Linux system, use the **-q** option in the **rpm** command.

   ```shell
   rpm -q dnf
   ```

   * **-q**: query operation

   If the specified package is not installed, the following error message is displayed:

   ```text
   package dnf is not installed
   ```

   * **Querying All Installed RPM Packages**

   To query all RPM packages installed in Linux, use the **-qa** option in the **rpm** command.

   ```shell
   $ rpm -qa 
   dracut-config-rescue-055-7.oe2203SP3.x86_64
   parted-3.5-1.oe2203SP3.x86_64
   irqbalance-1.8.0-9.oe2203SP3.x86_64
   ......
   ```

   Note: When using the **-qa** option, use the pipe character (|) together to improve the search accuracy.

   * **Querying Details About an Installed RPM Package**

   Use the **-qi** option in the **rpm** command to query the details of an RPM package installed in the system.

   ```shell
   $ rpm -qi python3
   Name        : python3
   Version     : 3.9.9
   Release     : 24.oe2203SP3
   Architecture: x86_64
   Install Date: Wed 05 Jul 2023 08:30:23 PM CST
   Group       : Unspecified
   Size        : 35916839
   License     : Python-2.0
   Signature   : RSA/SHA1, Wed 28 Jun 2023 01:11:59 PM CST, Key ID d557065eb25e7f66
   Source RPM  : python3-3.9.9-24.oe2203SP3.x86_64.rpm
   Build Date  : Wed 28 Jun 2023 01:11:59 PM CST
   Build Host  : obs-worker1639015616-x86-0001
   Packager    : http://openeuler.org
   Vendor      : http://openeuler.org
   URL         : https://www.python.org/
   Summary     : Interpreter of the Python3 programming language
   Description :
   Python combines remarkable power with very clear syntax. It has modules,
   classes, exceptions, very high level dynamic data types, and dynamic
   typing. There are interfaces to many system calls and libraries, as well
   as to various windowing systems. New built-in modules are easily written
   in C or C++ (or other languages, depending on the chosen implementation).
   Python is also usable as an extension language for applications written
   in other languages that need easy-to-use scripting or automation interfaces.

   This package Provides python version 3.
   ```

   * **Querying All Files in an RPM Package**

   To query the file list of an RPM package that is not installed, use the **-qlp** option in the **rpm** command.

   ```shell
   $ rpm -qlp pkgship-2.2.0-10.oe2203SP3.noarch.rpm
   /etc/ima/digest_lists.tlv/0-metadata_list-compact_tlv-pkgship-2.2.0-10.oe2203SP3.noarch
   /etc/ima/digest_lists/0-metadata_list-compact-pkgship-2.2.0-10.oe2203SP3.noarch
   /etc/pkgship/auto_install_pkgship_requires.sh
   /etc/pkgship/conf.yaml
   /etc/pkgship/package.ini
   ......
   ```

   * **Querying RPM Package Dependencies**

   To query the list of dependency packages compiled by a specified RPM package that is not installed, use the **-qRp** option in the **rpm** command.

   ```shell
   $ rpm -qRp pkgship-2.2.0-10.oe2203SP3.noarch.rpm
   /bin/bash
   /bin/sh
   /usr/bin/python3
   config(pkgship) = 2.2.0-10.oe2203SP3
   python3
   python3-Flask-Limiter
   ......
   ```

   * **Verifying All Installed RPM Packages**

   To verify an installed RPM package, use the **-Va** option in the **rpm** command to compare the information about the files installed in the package with the information about the files obtained from the package metadata stored in the RPM database.

   ```shell
   $ rpm -Va
   S.5....T.  c /root/.bashrc
   .......T.  c /etc/yum.repos.d/openEuler.repo
   S.5....T.  c /etc/issue
   S.5....T.  c /etc/issue.net
   S.5....T.  c /etc/csh.login
   S.5....T.  c /etc/profile
   .M....G..  g /var/log/lastlog
   .M.......  c /boot/grub2/grubenv
   ......
   ```

   Table 2 Output fields of the **rpm -Va** command and their meanings

   |Field|Description|
   |--|--|
   |S|The file length changes.|
   |M|The access permission or type of a file changes.|
   |5|The MD5 checksum changes.|
   |D|The attributes of a device node change.|
   |L|The symbolic link of a file changes.|
   |U|The owner of a file, subdirectory, or device node changes.|
   |G|The group of a file, subdirectory, or device node changes.|
   |T|The last modification time of a file changes.|

   * **Querying the RPM Package of a Specific File**

   To query an RPM package that provides a specific binary file on Linux, use the **-qf** option in the **rpm** command.

   ```shell
   $ rpm -qf /usr/share/doc/pkgship
   pkgship-2.2.0-10.oe2203SP3.noarch.rpm
   ```

   * **Querying Files in an Installed RPM Package**

   To query the list of installation files of an RPM package, use the **-ql** option in the **rpm** command.

   ```shell
   $ rpm -ql dnf
   /etc/bash_completion.d/dnf
   /etc/ima/digest_lists.tlv/0-metadata_list-compact_tlv-dnf-4.14.0-14.oe2203SP3.noarch
   /etc/ima/digest_lists/0-metadata_list-compact-dnf-dnf-4.14.0-14.oe2203SP3.noarch
   /usr/bin/dnf
   /usr/lib/systemd/system/dnf-makecache.service
   /usr/lib/systemd/system/dnf-makecache.timer
   /usr/share/doc/dnf
   /usr/share/doc/dnf/AUTHORS
   /usr/share/doc/dnf/README.rst
   /usr/share/licenses/dnf
   /usr/share/licenses/dnf/COPYING
   /usr/share/licenses/dnf/PACKAGE-LICENSING
   /var/cache/dnf
   ```

   * **Querying the Recently Installed RPM Packages**

   Linux is a multi-user OS. During the use of Linux, other users may have installed some software packages. To query the recently installed packages in the system, use the **-qa --last** options in the **rpm** command.

   ```shell
   $ rpm -qa --last
   ntp-4.2.8p15-11.oe2203SP3.x86_64
   ntpstat-0.6-4.oe2203SP3.noarch
   ntp-help-4.2.8p15-11.oe2203SP3.noarch
   ```

   * **Querying Only the Documents of the Installed RPM Packages**

   You can obtain the help information of any command from the **Linux Man** page (path for storing **/usr/share/doc/Package\_Name-Version\_Number/docs\*** documents). To query the list of documents associated with the installed RPM packages, use the **-qdf** option in the **rpm** command and enter the binary file path.

   ```shell
   $ rpm -qdf /usr/bin/grep
   /usr/share/doc/grep/NEWS
   /usr/share/doc/grep/README
   /usr/share/doc/grep/THANKS
   /usr/share/doc/grep/TODO
   /usr/share/info/grep.info.gz
   /usr/share/man/man1/egrep.1.gz
   /usr/share/man/man1/fgrep.1.gz
   /usr/share/man/man1/grep.1.gz
   ```

   * **Upgrading an Installed RPM Package**

   You can easily upgrade the installed RPM package to the latest version by using the **-Uvh** option and the **rpm** command.

   ```shell
   $ rpm -Uvh pkgship-2.2.0-10.oe2203SP3.noarch.rpm
   Preparing...        ################################# [100%]
   ```

   Note: When the installed RPM package is upgraded, the old RPM package is deleted and the new RPM package is installed.

   * **Removing an Installed RPM Package**

   To remove an RPM package installed on the system, use the **-ev** or **-e** option in the **rpm** command.

   ```shell
   rpm -ev pkgship
   ```

   * **Rebuilding the Damaged RPM Database**

   When you try to update the system using the **yum update** command, you may receive an error message indicating that the RPM database is damaged. If you receive this message, use the **--rebuilddb** option in the **rmp** command to rebuild the database.

   ```shell
   rm /var/lib/rpm/__db*
   rpm --rebuilddb
   ```

   * **Checking Whether Vulnerabilities in Specific Packages Have Been Fixed**

   You can use the **--changelog** option in the **rpm** command and enter the corresponding CVE ID.

   ```shell
   rpm -q --changelog python-2.6.6 | grep -i "CVE-2019-9636"
   ```

   * **Importing the RPM GPG Key**

   By default, when a new repository is added to the Linux system, the GPG key is automatically imported. You can also use **--import** in the **rpm** command to manually import the RPM GPG key to check the integrity of a package when downloading it from the repository.

   ```shell
   rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenEuler-22.03-LTS-SP4
   ```

3. DNF commands

   Table 3 DNF commands

   |Command|Description|
   |--|--|
   |repolist|Displays the configured software repository source.|
   |install|Installs one or more software packages on Linux.|
   |upgrade|Upgrades one or more software packages on Linux.|
   |list|Lists a software package or a group of software packages.|
   |info|Displays detailed information about a package or package group.|
   |updateinfo|Displays the bulletin information about a package.|
   |search|Searches for the specified character string in the software package details.|
   |check-update|Checks for software package update.|
   |remove|Removes one or more software packages from the system.|
   |reinstall|Reinstalls a package.|
   |downgrade|Downgrades a software package.|
   |autoremove|Removes all unnecessary software packages that are installed due to dependency relationships.|
   |distro-sync|Synchronizes the installed software package to the latest available version.|
   |makecache|Creates a metadata cache.|
   |repository-package|Runs commands on all software packages in a specified repository.|
   |provides|Searches for the software package that provides the specified content.|
   |group|Displays or uses group information.|
   |history|Displays or uses transaction history.|
   |clean|Deletes cached data.|

   * **Displaying Configured Software Repositories**

   By default, the **--enabled** option is added to display the enabled software repositories.

   ```shell
   $ dnf repolist --enabled
   repo id                                                                   repo name
   EPOL                                                                      EPOL
   OS                                                                        OS
   debuginfo                                                                 debuginfo
   everything                                                                everything
   pkgship_elasticsearch                                                     Elasticsearch repository
   source                                                                    source
   update                                                                    update
   ```

   * **--all**: displays all software repositories.
   * **--disabled**: displays disabled software repositories.
   * **--enabled**: displays enabled repositories (default).

   Installing One or More Software Packages

   You can run the **install** command to install RPM packages.

   ```shell
   dnf install software_package
   ```

   Conflicting packages or packages that cannot be installed may exist during software package installation. You can add **--allowerasing** to the command to replace the conflicting packages or **--skip-broken** to skip the packages that cannot be installed.

   ```shell
   dnf install software_package [software_package ...] --allowerasing --skip-broken
   ```

   When dnf is used to install a software package, add **--installroot** to set the root directory for installing the software package.

   ```shell
   dnf install software_package --installroot software_package_root_directory
   ```

   If you need to temporarily specify a repository source for installation, you can add the **--setopt=reposdir=** option to specify the loading directory of the repository source.

   ```shell
   dnf install software_package --setopt=reposdir=repo_source_directory
   ```

   If interactive confirmation is not required during installation, you can add **-y** or **--assumeyes** to enable all software packages to be installed to automatically answer **Yes**.

   ```shell
   dnf install software_package -y
   ```

   To install an RPM package by specifying a specific repository source, you can specify the **--repo** or **--enablerepo** option. To achieve the same effect, you can also use the **--disablerepo** option to disable the matched repository source. You are advised to use the **--repo** option to install the RPM package.

   ```shell
   dnf install software_package --repo=repo_source_
   ```

   * **Reinstalling a Software Package**

   You can run the **reinstall** command to reinstall a software package in the system.

   ```shell
   dnf reinstall software_package
   ```

   * **Upgrading One or More Software Packages**

   * You can use the **upgrade** command to upgrade one or more software packages on Linux.

   ```shell
   dnf upgrade software_package [software_package ...]
   ```

   * You can also run the **update** command to upgrade one or more software packages.

   ```shell
   dnf update software_package [software_package ...]
   ```

   * **Downgrading a Software Package**

   If a compatibility problem occurs because the version of a software package is too late, you can downgrade the software package.

   ```shell
   dnf downgrade software_package
   ```

   * **Listing a Package or a Group of Packages**

   You can run the **list** command to list the software packages installed in the system and the software packages in the configured repository.

   ```shell
   dnf list
   ```

   You can add options to filter the displayed package list.

   * **--all**: displays all software packages (default).

   * **--available**: displays only available software packages.

   * **-- installed**: displays only installed software packages.

   * **--extras**: displays only additional software packages.

   * **--updates**: displays only the software packages to be upgraded.

   * -**-upgrades**: displays only the software packages to be upgraded.

   * **--autoremove**: displays only the software packages to be removed.

   * **--recent**: displays the software packages that have been changed recently.

   * **Querying Details About a Software Package**

   You can run the **info** command to query details about a software package.

   ```shell
   dnf info software_package
   ```

   * **Searching for a Software Package**

   If you need to install a software package in the system but you are not sure about the full name of the software package, you can run the **search** command to search for the matched package.

   ```shell
   dnf search software_package
   ```

   * **Uninstalling One or More Software Packages**

   You can run the **remove** command to remove an expired or duplicate software package.

   ```shell
   dnf remove software_package
   ```

   * **--duplicates**: removes installed (duplicate) software packages.

   * **--oldinstallonly**: removes expired installation-only software packages.

   * **Automatically Removing Software Packages Installed Due to Dependency Relationships**

   You can run the **autoremove** command to remove unnecessary software packages that are installed due to dependency relationships.

   ```shell
   dnf autoremove software_package
   ```

## Configuring SSH

1. Introduction to the SSH service

   Secure Shell (SSH) is a reliable protocol that ensures the security of remote login sessions and other network services. The SSH protocol can effectively prevent information leakage during remote management. SSH encrypts transferred data to prevent domain name server (DNS) spoofing and IP spoofing. OpenSSH was created as an open source alternative to the proprietary SSH protocol.

2. Configuring the SSH Service

   ```shell
   # Open and modify the /etc/ssh/sshd_config file.
   vi /etc/ssh/sshd_config

   # Restart the SSH service.
   systemctl restart sshd

   # Check the SSH service status.
   systemctl status sshd
   ```

3. Main options in the SSH service configuration file

   ```text
   $ Specify the SSH protocol version.
   Protocol 2

   # Allowed users
   AllowUsers xxx

   # Denied users
   DenyUser root

   # Configure session timeout.
   ClientAliveInterval 120

   # Disable SSH root login.
   PermitRootLogin no

   # Configure or change the SSH port number.
   Port 1234

   # Disable SSH password authentication.
   PasswordAuthentication no
   ```

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/cve.md
---
# Common Vulnerabilities and Exposures (CVEs)

For details about the CVEs involved in the version, see the [CVE list](https://www.openeuler.org/en/security/cve/).

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/common_tools/commonly_used_tools.md
---
# Commonly Used Tools

* [Commonly Used Tools](#commonly-used-tools)
  * [ftrace](#ftrace)
  * [strace](#strace)
  * [kdump](#kdump)

## ftrace

1. ftrace: a debug tool for the Linux kernel space. The kernel provides trace events for you to trace . ftrace can capture events so that you can intuitively view these events and trace kernel functions.
2. Configuration and usage of ftrace: To use ftrace, you need to compile its dependencies into the kernel. By default, openEuler compiles the ftrace option. If the ftrace option is not enabled, you can enable it by choosing **Kernel hacking** > **Tracers** > **Trace syscalls** in **menuconfig**. In addition, you need to compile the debugfs by choosing **Kernel hacking** > **Generic Kernel Debugging Instruments** > **Debug Filesystem**.

* **Configuring the ftrace function**

ftrace provides access interfaces for user space through the debugfs. After the debugfs is configured in the kernel, the **/sys/kernel/debug** directory is created. The debugfs is mounted to this directory. If the kernel supports ftrace-related configuration items, a **tracing** directory is created in the debugfs. The debugfs is mounted to this directory. The following figure shows the content of this directory.

![](./images/en-us_image_0000001322372918.png)

* **Introduction to the ftrace debugfs interface**

  You can view some control and output files provided by ftrace through the debugfs. The common files are described as follows:

  available\_tracers: available tracers

  current\_tracer: running tracer

  available\_events: lists all available trace events in the OS

  events: This directory differentiates events by module.

  set\_event: lists the events to be traced.

  tracing\_on: enables or disables tracing. echo 0 > tracing\_on indicates that tracing is disabled, and 1 indicates that tracing is enabled.

  trace: queries trace data.

* **Available tracers**

![en-us\_image\_0000001373373585](./images/en-us_image_0000001373373585.png)

function: a function call tracing program that does not require parameters
function\_graph: a function call tracer that uses subcalls

* **Trace events**

```shell
# Specify the arm_event of the RAS to be traced.
echo ras:arm_event > /sys/kernel/debug/tracing/set_event

# This file contains the event format and fields to be printed.
cat /sys/kernel/debug/tracing/events/ras/arm_event/format

# Start tracing.
echo 1 > /sys/kernel/debug/tracing/tracing_on

#  Observe the trace output.
tail -f /sys/kernel/debug/tracing/trace
```

![c50cb9df64f4659787c810167c89feb4\_1884x257](./images/c50cb9df64f4659787c810167c89feb4_1884x257.png)

* **Tracing input parameters of kernel functions**

Trace mmap, which corresponds to the system call **do\_mmap**. Output the **addr** input parameter.

![en-us\_image\_0000001373379529](./images/en-us_image_0000001373379529.png)

```shell
# Trace through the kprobe.
echo 'p:probe1 do_mmap addr=%x1' > kprobe_events

# Enable kprobe.
echo 1 > events/kprobes/probe1/enable

# Start tracing.
echo 1 > tracing_on

# View trace data.
```

![en-us\_image\_0000001322379488](./images/en-us_image_0000001322379488.png)

* **Tracing function calls**

```shell
# Select a tracing type.
echo function_graph > current_tracer

# Set the PID of the process to be filtered.
echo <pid> set_ftrace_pid

# Start tracing.
echo 1 > tracing_on

# View trace data.
```

![en-us\_image\_0000001322219840](./images/en-us_image_0000001322219840.png)

## strace

The `strace` command is a diagnosis and debugging tool. You can use the `strace` command to analyze system calls and signal transmission of applications to solve problems or understand the application execution process.

You can run the `strace -h` command to view the functions provided by strace.

![en-us\_image\_0000001322112990](./images/en-us_image_0000001322112990.png)

The most common usage is to trace the *xx* command, trace the forks, print the time, and output the result to the **output** file.

```shell
strace -f -tt -o output xx
```

## kdump

1. crash/kdump Principles

   kdump is a snapshot of the memory status of the OS running at a certain time point. It helps O\&M personnel debug and analyze the cause of system breakdown. kdump is usually used when system breakdown and panic happen.

   The process is as follows.

   ![en-us\_image\_0000001321685172](./images/en-us_image_0000001321685172.png)

2. Installing and configuring related tools

   ```shell
   # Use Yum to install the corresponding software package.
   yum install kernel-debuginfo-$(uname -r) kexec-tools crash -y

   # Set the reserved memory size for crashkernel.
   vim /etc/default/grub
   ```

   ![en-us\_image\_0000001372821865](./images/en-us_image_0000001372821865.png)

   ```shell
   # Regenerate the grub configuration file.
   grub2-mkconfig -o /boot/efi/EFI/openEuler/grub.cfg
   reboot

   # Start the kdump service.
   systemctl start kdump #Start kdump.
   systemctl enable kdump #Set the kdump to start upon system startup.
   ```

3. Triggering a crash

   Operation 1 Retain the default settings of the kernel. When a hard lock or oops occurs, a panic is triggered.

   ![en-us\_image\_0000001372824637](./images/en-us_image_0000001372824637.png)

   Operation 2 Modify the settings. The following commands cam make the settings take effect only once and become invalid after the system is restarted.

   ```shell
   # Set a soft lock to trigger a panic.
   echo 1 > /proc/sys/kernel/softlockup_panic

   # Trigger a kernel panic when an out of memory (OOM) error occurs.
   echo 1 > /proc/sys/vm/panic_on_oom

   # A panic occurs when a process is hung.
   echo 1 > /proc/sys/kernel/hung_task_panic

   # Set the timeout interval of the hung task mechanism.
   echo 60 > /proc/sys/kernel/kernel.hung_task_timeout_secs
   ```

   Operation 3 To make the configuration take effect permanently, write the following parameters to the **/etc/sysctl.conf** file and run the `sysctl -p` command:

   ```shell
   kernel.hung_task_panic=1
   kernel.hung_task_timeout_secs=60
   kernel.softlockup_panic=1
   vm.panic_on_oom=1
   ```

4. Analyzing the crash

   Operation 1 Enable crash debugging.

   Operation 2 Find the generated **vmcore** file generated in the **/var/crash/*IP\_address*-*time*** directory.

   Operation 3 Run the following command to start crash debugging:

   ```shell
   crash {vmcore file} {debug kernel vmlinux}
   ```

   ![en-us\_image\_0000001372748125](./images/en-us_image_0000001372748125.png)

   The format of the **crash** debugging command is *command args*. *command* indicates the command to be executed, and *args* indicates the parameters required by some debugging commands.

   |Command|Description|
   |--|--|
   |help|Prints the help information of a command. You can view the supported commands or the help information of a specific command. For example, run `help bt`.|
   |bt|Prints the function call stack information.|
   |log|Prints the system message buffer. Parameters can be appended, for example, **log**.|
   |ps|Displays the process status. **>** indicates that the process is active.|
   |dis|Disassembles a specified function or address. Example: `dis -l \[func\]`|
   |mount|Displays information about the current file system.|

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/maintenance/aops/community_hotpatch_creation_and_release_process.md
---
# Community Hot Patch Creation and Release Process

This document is currently not available in English.

---

---
url: /en/docs/22.03_LTS_SP4/tools/devops.md
---


---

---
url: /en/docs/22.03_LTS_SP4/tools/community_tools.md
---


---

---
url: /en/docs/22.03_LTS_SP4/server/administration/compa_command/overview.md
---
# Compatibility Commands

This document describes the shell and Linux commands re-written in the Rust language. These commands can be used on openEuler and is compatible with native Linux commands.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/configurable_cgroup_path.md
---
# Configurable Cgroup Path

## Function Description

System containers provide the capabilities of isolating and reserving container resources on hosts. You can use the  **--cgroup-parent**  parameter to specify the cgroup directory used by a container to another directory, thereby flexibly allocating host resources. For example, if the cgroup parent path of containers A, B, and C is set to  **/lxc/cgroup1**, and the cgroup parent path of containers D, E, and F is set to  **/lxc/cgroup2**, the containers are divided into two groups through the cgroup paths, implementing resource isolation at the cgroup level.

## Parameter Description

In addition to specifying the cgroup parent path for a system container using commands, you can also specify the cgroup paths of all containers by modifying the startup configuration files of the iSulad container engine.

## Constraints

* If the  **cgroup parent**  parameter is set on both the daemon and client, the value specified on the client takes effect.
* If container A is started before container B, the cgroup parent path of container B is specified as the cgroup path of container A. When deleting a container, you need to delete container B and then container A. Otherwise, residual cgroup resources exist.

## Example

Start a system container and specify the  **--cgroup-parent**  parameter.

```shell
[root@localhost ~]# isula run -tid --cgroup-parent /lxc/cgroup123 --system-container --external-rootfs /root/myrootfs none init
115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
```

Check the cgroup information of the init process in the container.

```shell
[root@localhost ~]# isula inspect -f "{{json .State.Pid}}" 11
22167
[root@localhost ~]# cat /proc/22167/cgroup
13:blkio:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
12:perf_event:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
11:cpuset:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
10:pids:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
9:rdma:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
8:devices:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
7:hugetlb:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
6:memory:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
5:net_cls,net_prio:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
4:cpu,cpuacct:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
3:files:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
2:freezer:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
1:name=systemd:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e/init.scope
0::/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e
```

The cgroup parent path of the container is set to  **/sys/fs/cgroup/***\<controller>***/lxc/cgroup123**.

In addition, you can configure the container daemon file to set the cgroup parent paths for all containers. For example:

```text
{
         "cgroup-parent": "/lxc/cgroup123",
}
```

Restart the container engine for the configuration to take effect.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/secure_container/configuring_network_for_a_secure_container.md
---
# Configuring Networking for a Secure Container

## TAP-based Network Support

The secure container technology is implemented based on QEMU VMs. For a physical machine system, a secure container is equivalent to a VM. Therefore, the secure container may connect the VM to an external network in the Neutron network by using the test access point (TAP) technology. You do not need to pay attention to TAP device creation and bridging. You only need to hot add the specified TAP device (with an existing host) to the VM in the pause container and update the NIC information.

Related commands are as follows:

1. **Run the following command to add a TAP NIC for a started container:**

   ```shell
   cat ./test-iface.json | kata-runtime kata-network add-iface 6ec7a98 -
   ```

   In the preceding command,  **6ec7a98**  is the truncated container ID, and  **test-iface.json**  is the file that describes the NIC information. The following is an example:

   ```json
   {
       "device": "tap-test", 
       "name": "eth-test", 
       "IPAddresses": [
           {
               "address": "172.16.0.3", 
               "mask": "16"
           }
       ], 
       "hwAddr":"02:42:20:6f:a3:69",
       "mtu": 1500,
       "vhostUserSocket":"/usr/local/var/run/openvswitch/vhost-user1",
       "queues":5
   }
   ```

   The fields in the JSON file are described as follows:

   The following describes the output of the  **kata-runtime kata-network add-iface**  command for adding NICs:

   * If the command is successfully executed, the NIC information in JSON format is returned from  **standard output (stdout)**. The content in JSON format is the same as the input NIC information.

     Example:

     ```shell
     $ kata-runtime kata-network add-iface <container-id> net.json 
     {"device":"tap_test","name":"eth-test","IPAddresses":[{"Family":2,"Address":"173.85.100.1","Mask":"24"}],"mtu":1500,"hwAddr":"02:42:20:6e:03:01","pciAddr":"01.0/00"}
     ```

   * If the command fails to be executed, null is returned from  **stdout**.

     Example:

     ```shell
     $ kata-runtime kata-network add-iface <container-id> netbad.json 2>/dev/null
     null
     ```

   > \[!NOTE] **NOTE:**\
   > If an IP address is specified for an NIC that is successfully added, Kata adds a default route whose destination is in the same network segment as the IP address of the NIC. In the preceding example, after the NIC is added, the following route is added to the container:
   >
   > ```shell
   > [root@6ec7a98 /]# ip route  
   > 172.16.0.0/16 dev eth-test proto kernel scope link src 172.16.0.3  
   > ```

2. **Run the following command to view the added NICs:**

   ```shell
   $ kata-runtime kata-network list-ifaces 6ec7a98
   [{"name":"eth-test","mac":"02:42:20:6f:a3:69","ip":["172.16.0.3/16"],"mtu":1500}]
   ```

   The information about the added NICs is displayed.

   The following describes the output of the  **kata-runtime kata-network list-ifaces**command for listing added NICs:

   * If the command is executed successfully, information about all NICs inserted into the pod in JSON format is returned from  **stdout**.

     If multiple NICs are inserted into the pod, the NIC information in JSON array format is returned.

     ```shell
     $ kata-runtime kata-network list-ifaces <container-id>
     [{"name":"container_eth","mac":"02:42:20:6e:a2:59","ip":["172.17.25.23/8"],"mtu":1500},{"name":"container_eth_2","mac":"02:90:50:6b:a2:29","ip":["192.168.0.34/24"],"mtu":1500}]
     ```

     If no NIC is inserted into the pod, null is returned from  **stdout**.

     ```shell
     $ kata-runtime kata-network list-ifaces <container-id>
     null
     ```

   * If the command fails to be executed, null is returned from  **stdout**, and error description is returned from  **standard error (stderr)**.

     Example:

     ```shell
     $ kata-runtime kata-network list-ifaces <container-id>
     null
     ```

3. **Add a route for a specified NIC.**

   ```shell
   $ cat ./test-route.json | kata-runtime kata-network add-route 6ec7a98 -
   [{"dest":"default","gateway":"172.16.0.1","device":"eth-test"}]
   ```

   The following describes the output of the  **kata-runtime kata-network add-route**  command for adding a route to a specified NIC:

   * If the command is executed successfully, the added route information in JSON format is returned from  **stdout**.

     Example:

     ```shell
     $ kata-runtime kata-network add-route <container-id> route.json 
     [{"dest":"177.17.0.0/24","gateway":"177.17.25.1","device":"netport_test_1"}]
     ```

   * If the command fails to be executed, null is returned from  **stdout**, and error description is returned from  **standard error (stderr)**.

     Example:

     ```shell
     $ kata-runtime kata-network add-route <container-id> routebad.json 2>/dev/null
     null
     ```

   Key fields are described as follows:

   * **dest**: Network segment corresponding to the route. The value is in the format of <*ip*>/<*mask*>. <*ip*> is mandatory. There are three cases:
     1. Both IP address and mask are configured.
     2. If only an IP address is configured, the default mask is 32.
     3. If  **"dest":"default"**  is configured, there is no destination by default. In this case, the gateway needs to be configured.

   * **gateway**: Next-hop gateway of the route. When  **"dest":"default"**  is configured, the gateway is mandatory. In other cases, this parameter is optional.

   * **device**: Name of the NIC corresponding to the route, which is mandatory. The value contains a maximum of 15 characters.

   > \[!NOTE] **NOTE:**\
   > If a route is added for the loopback device  **lo**  in the container, the device name corresponding to the  **device**  field in the route configuration file is  **lo**.

4. **Run the following command to delete a specified route:**

   ```shell
   cat ./test-route.json | kata-runtime kata-network del-route 6ec7a98 -
   ```

   The fields in the  **test-route.json**  file are the same as those in the JSON file for adding a route.

   The following describes the output of the**kata-runtime kata-network del-route**  command for deleting a specified route:

   * If the command is executed successfully, the added route information in JSON format is returned from  **stdout**.

     Example:

     ```shell
     $ kata-runtime kata-network del-route <container-id> route.json 
     [{"dest":"177.17.0.0/24","gateway":"177.17.25.1","device":"netport_test_1"}]
     ```

   * If the command fails to be executed, null is returned from  **stdout**, and error description is returned from  **standard error (stderr)**.

     Example:

     ```shell
     $ kata-runtime kata-network del-route <container-id> routebad.json 2>/dev/null
     null
     ```

   > \[!NOTE] **NOTE:**
   >
   > * In the input fields,  **dest**  is mandatory, and both  **device**  and  **gateway**  are optional. Kata performs fuzzy match based on different fields and deletes the corresponding routing rules. For example, if  **dest**  is set to an IP address, all rules of this IP address will be deleted.
   > * If the route of the loopback device  **lo**  in the container is deleted, the device name corresponding to the  **device**  field in the route configuration file is  **lo**.

5. **Run the following command to delete an NIC:**

   ```shell
   cat ./test-iface.json | kata-runtime kata-network del-iface 6ec7a98 -
   ```

   > \[!NOTE] **NOTE:**\
   > When deleting an NIC, you can only delete it based on the  **name**  field in the NIC container. Kata does not identify other fields.

   The following describes the output of the  **kata-runtime kata-network del-iface**command for deleting NICs:

   * If the command is executed successfully, null is returned from  **stdout**.

     Example:

     ```shell
     $ kata-runtime kata-network del-iface <container-id> net.json
     null
     ```

   * If the command fails to be executed, the information about NICs that fail to be deleted in JSON format is returned from  **stdout**, and error description is returned from  **stderr**.

     Example:

     ```shell
     $ kata-runtime kata-network del-iface <container-id> net.json
     {"device":"tapname_fun_012","name":"netport_test_1","IPAddresses":[{"Family":0,"Address":"177.17.0.1","Mask":"8"}],"mtu":1500,"hwAddr":"02:42:20:6e:a2:59","linkType":"tap"}
     ```

The preceding are common commands. For details about the command line interfaces, see  [APIs](appendix_2.md#apis).

## Kata IPVS Subsystem

The secure container provides an API for adding the  **ipvs**  command and setting the IPVS rule for the container. The functions include adding, editing, and deleting virtual services, adding, editing, and deleting real servers, querying IPVS service information, setting connection timeout, clearing the system connection cache, and importing rules in batches.

1. Add a virtual service address for the container.

   ```shell
   kata-runtime kata-ipvs ipvsadm --parameters "--add-service --tcp-service 172.17.0.7:80 --scheduler rr --persistent 3000" <container-id>
   ```

2. Modify virtual service parameters of a container.

   ```shell
   kata-runtime kata-ipvs ipvsadm --parameters "--edit-service --tcp-service 172.17.0.7:80 --scheduler rr --persistent 5000" <container-id>
   ```

3. Delete the virtual service address of a container.

   ```shell
   kata-runtime kata-ipvs ipvsadm --parameters "--delete-service --tcp-service 172.17.0.7:80" <container-id>
   ```

4. Add a real server for the virtual service address.

   ```shell
   kata-runtime kata-ipvs ipvsadm --parameters "--add-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80 --weight 100" <container-id>
   ```

5. Modify real server parameters of a container.

   ```shell
   kata-runtime kata-ipvs ipvsadm --parameters "--edit-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80 --weight 200" <container-id>
   ```

6. Delete a real server from a container.

   ```shell
   kata-runtime kata-ipvs ipvsadm --parameters "--delete-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80" <container-id>
   ```

7. Query service information.

   ```shell
   kata-runtime kata-ipvs ipvsadm --parameters "--list" <container-id>
   ```

8. It takes a long time to import rules one by one. You can write rules into a file and import them in batches.

   ```shell
   kata-runtime kata-ipvs ipvsadm --restore - < <rule file path> <container-id>
   ```

   > \[!NOTE] **NOTE:**\
   > By default, the NAT mode is used for adding a single real server. To add real servers in batches, you need to manually add the  **-m**  option to use the NAT mode.\
   > The following is an example of the rule file content:\
   > -A -t 10.10.11.12:100 -s rr -p  3000\
   > -a -t 10.10.11.12:100 -r  172.16.0.1:80 -m\
   > -a -t 10.10.11.12:100 -r  172.16.0.1:81 -m\
   > -a -t 10.10.11.12:100 -r 172.16.0.1:82  -m

9. Clear the system connection cache.

   ```shell
   kata-runtime kata-ipvs cleanup --parameters "--orig-dst 172.17.0.4 --protonum tcp" <container-id>
   ```

10. Set timeout interval for TCP, TCP FIN, or UDP connections.

    ```shell
    kata-runtime kata-ipvs ipvsadm --parameters "--set 100 100 200" <container-id>
    ```

    > \[!NOTE] **NOTE:**
    >
    > 1. Each container supports a maximum of 20000 iptables rules (5000 services and three servers/services). Both add-service and add-server are rules.
    > 2. Before importing rules in batches, you need to clear existing rules.
    > 3. No concurrent test scenario exists.
    > 4. The preceding are common commands. For details about the command line interfaces, see  [APIs](appendix_2.md#apis).

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/secure_container/configuring_resources_for_a_secure_container.md
---
# Configuring Resources for a Secure Container

The secure container runs on a virtualized and isolated lightweight VM. Therefore, resource configuration is divided into two parts: resource configuration for the lightweight VM, that is, host resource configuration; resource configuration for containers in the VM, that is, guest container resource configuration. The following describes resource configuration for the two parts in detail.

## Sharing Resources

Because the secure container runs on a virtualized and isolated lightweight VM, resources in some namespaces on the host cannot be accessed. Therefore,  `--net host`,  `--ipc host`,  `--pid host`, and  `--uts host`  are not supported during startup.

When a pod is started, all containers in the pod share the same net namespace and ipc namespace by default. If containers in the same pod need to share the pid namespace, you can use Kubernetes to configure the pid namespace. In Kubernetes 1.11, the pid namespace is disabled by default.

## Limiting CPU Resources

1. Configure CPU resources for running a lightweight VM.

   Configuring CPU resources of a lightweight VM is to configure the vCPUs for running the VM. The secure container uses  `--annotation com.github.containers.virtcontainers.sandbox\_cpu`  to configure the CPU resources for running the lightweight VM. This option can be configured only on the pause container.

   ```shell
   docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=podsandbox --annotation com.github.containers.virtcontainers.sandbox_cpu=<cpu-nums> <pause-image> <command>
   ```

   Example:

   ```shell
   # Start a pause container.
   docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=podsandbox --annotation com.github.containers.virtcontainers.sandbox_cpu=4 busybox sleep 999999
   be3255a3f66a35508efe419bc52eccd3b000032b9d8c9c62df611d5bdc115954

   # Access the container and check whether the number of CPUs is the same as that configured in the com.github.containers.virtcontainers.sandbox_cpu file.
   docker exec be32 lscpu
   Architecture:        aarch64
   Byte Order:          Little Endian
   CPU(s):              4
   On-line CPU(s) list: 0-3
   Thread(s) per core:  1
   Core(s) per socket:  1
   Socket(s):           4
   ```

   > \[!NOTE] **NOTE:**
   > The maximum number of CPUs that can be configured is the number of CPUs (excluding isolated cores) that can run on the OS. The minimum number of CPUs is 0.5.

2. Configure CPU resources for running a container.

   The method of configuring CPU resources for a container is the same as that for an open-source Docker container. You can configure CPU resources by setting the following parameters in the  `docker run`  command:

3. Configure CPU hot swap.

   > \[!NOTE] **NOTE:**
   > The CPU hot swap function of the secure container requires the virtualization component QEMU.

   The  **enable\_cpu\_memory\_hotplug**  option in the kata-runtime configuration file  **config.toml**  is used to enable or disable CPU and memory hot swap. The default value is  **false**, indicating that CPU and memory hot swap is disabled. If the value is  **true**, CPU and memory hot swap is enabled.

   The  `--cpus`  option is reused in kata-runtime to implement the CPU hot swap function. The total number of  `--cpus`  options of all containers in a pod is calculated to determine the number of CPUs to be hot added to the lightweight VM.

   Example:

   ```shell
   # Start a pause container. By default, one vCPU is allocated to a lightweight VM.
   docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=podsandbox busybox sleep 999999
   77b40fb72f63b11dd3fcab2f6dabfc7768295fced042af8c7ad9c0286b17d24f

   # View the number of CPUs in the lightweight VM after the pause container is started.
   docker exec 77b40fb72f6 lscpu
   Architecture:          x86_64
   CPU op-mode(s):        32-bit, 64-bit
   Byte Order:            Little Endian
   CPU(s):                1
   On-line CPU(s) list:   0
   Thread(s) per core:    1
   Core(s) per socket:    1
   Socket(s):             1

   # Start a new container in the same pod and run the --cpus command to set the number of CPUs required by the container to 4.
   docker run -tid --runtime kata-runtime --network none --cpus 4 --annotation io.kubernetes.docker.type=container --annotation io.kubernetes.sandbox.id=77b40fb72f63b11dd3fcab2f6dabfc7768295fced042af8c7ad9c0286b17d24f busybox sleep 999999
   7234d666851d43cbdc41da356bf62488b89cd826361bb71d585a049b6cedafd3

   # View the number of CPUs in the current lightweight VM.
   docker exec 7234d6668 lscpu
   Architecture:          x86_64
   CPU op-mode(s):        32-bit, 64-bit
   Byte Order:            Little Endian
   CPU(s):                4
   On-line CPU(s) list:   0-3
   Thread(s) per core:    1
   Core(s) per socket:    1
   Socket(s):             4

   # View the number of CPUs in the lightweight VM after deleting the container where CPUs are hot added.
   docker rm -f 7234d666851d
   7234d666851d

   docker exec 77b40fb72f6  lscpu
   Architecture:          x86_64
   CPU op-mode(s):        32-bit, 64-bit
   Byte Order:            Little Endian
   CPU(s):                1
   On-line CPU(s) list:   0
   Thread(s) per core:    1
   Core(s) per socket:    1
   Socket(s):             1
   ```

   > \[!NOTE] **NOTE:**
   > The pause container is only a placeholder container and does not have any workload. Therefore, when a lightweight VM is started, the CPU allocated by default can be shared by other containers. Therefore, you only need to hot add three CPUs to the lightweight VM for the new container started in the preceding example.

   * After the container where the CPU is hot added is stopped, the CPU is removed when the container is started.

## Limiting Memory Resources

1. Configure memory resources for running a lightweight VM.

   Configuring the memory resources of a lightweight VM is to configure the memory for running the VM. The secure container uses  `--annotation com.github.containers.virtcontainers.sandbox\_mem`  to configure the memory resources for running the lightweight VM. This option can be configured only on the pause container.

   ```shell
   docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=podsandbox --annotation com.github.containers.virtcontainers.sandbox_mem=<memory-size> <pause-image> <command>
   ```

   Example:

   ```shell
   # Start a pause container and use --annotation com.github.containers.virtcontainers.sandbox_mem=4G to allocate 4 GB memory to the lightweight VM.
   docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=podsandbox --annotation com.github.containers.virtcontainers.sandbox_mem=4G busybox sleep 999999
   1532c3e59e7a45cd6b419aa1db07dd0069b0cdd93097f8944177a25e457e4297

   # View the memory information of the lightweight VM and check whether the memory size is the same as that configured in the com.github.containers.virtcontainers.sandbox_mem file.
   docker exec 1532c3e free -m
                 total        used        free      shared  buff/cache   available
   Mem:           3950          20        3874          41          55        3858
   Swap:             0           0           0
   ```

   > \[!NOTE] **NOTE:**
   >
   > * If the memory size of a lightweight VM is not set using  `--annotation com.github.containers.virtcontainers.sandbox\_mem`, the lightweight VM uses 1 GB memory by default.
   > * The minimum memory size of a pod in a secure container is 1 GB, and the maximum memory size is 256 GB. If the memory size allocated to a user exceeds 256 GB, an undefined error may occur. Currently, secure containers do not support the scenario where the memory size exceeds 256 GB.

2. Configure memory resources for running a container.

   The method of configuring memory resources for running a container is the same as that for the open-source Docker container. You can configure memory resource limitation parameters in the  `docker run`  command.

3. Configure memory hot add.

   The memory hot add function is also configured by the  **enable\_cpu\_memory\_hotplug**  option in the kata-runtime configuration file  **config.toml**. For details, see  [3](#limiting-cpu-resources).

   > \[!NOTE] **NOTE:**
   > Currently, memory resources support hot add only.

   The  `-m`  option is reused in kata-runtime to implement the memory hot add function. The sum of the  `-m`  options of all containers in a pod is collected to determine the number of memories to be hot added to a lightweight VM.

   Example:

   ```shell
   # Start a pause container. By default, 1 GB memory is allocated to the lightweight VM.
   docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=podsandbox busybox sleep 999999
   99b78508ada3fa7dcbac457bb0f6e3784e64e7f7131809344c5496957931119f

   # View the memory size of the lightweight VM after the pause container is started.
   docker exec 99b78508ada free -m
                 total        used        free      shared  buff/cache   available
   Mem:            983          18         914          36          50         908
   Swap:             0           0           0

   # Start a new container in the same pod and run the -m command to set the memory size required by the container to 4 GB.
   docker run -tid --runtime kata-runtime --network none -m 4G --annotation io.kubernetes.docker.type=container --annotation io.kubernetes.sandbox.id=99b78508ada3fa7dcbac457bb0f6e3784e64e7f7131809344c5496957931119f busybox sleep 999999
   c49461745a712b2ef3127fdf43b2cbb034b7614e6060b13db12b7a5ff3c830c8

   # View the memory size of the lightweight VM.
   docker exec c49461745 free -m
                 total        used        free      shared  buff/cache   available
   Mem:           4055          69        3928          36          57        3891
   Swap:             0           0           0

   # After deleting the container where the CPU is hot added, check the memory size of the lightweight VM.
   docker rm -f c49461745
   c49461745

   # The hot added memory does not support the hot add function. Therefore, after the hot added memory container is deleted from the lightweight VM, the memory is still 4 GB.
   docker exec 99b78508ada free -m
                 total        used        free      shared  buff/cache   available
   Mem:           4055          69        3934          36          52        3894
   Swap:             0           0           0
   ```

   > \[!NOTE] **NOTE:**
   > The pause container is only a placeholder container and does not have any workload. Therefore, the memory allocated to the lightweight VM during startup can be shared by other containers. You only need to hot add 3 GB memory to the lightweight VM for the new container started in the preceding example.

## Limiting Block I/O Resources

1. Configure the block I/O resources for running a lightweight VM.

   To configure block I/O resources for running a lightweight VM of secure containers, use  `--annotation com.github.containers.virtcontainers.blkio\_cgroup`. This option can be configured only on the pause container.

   ```shell
   docker run -tid --runtime --network none --annotation io.kubernetes.docker.type=podsandbox --annotation com.github.containers.virtcontainers.blkio_cgroup=<blkio json string<pause-image> <command>
   ```

   The value of  `--annotation com.github.containers.virtcontainers.blkio\_cgroup`  must comply with the definition of the BlkioCgroup structure.

   ```go
   // BlkioCgroup for Linux cgroup 'blkio' data exchange
   type BlkioCgroup struct {
    // Items specifies per cgroup values
    Items []BlockIOCgroupItem `json:"blkiocgroup,omitempty"`
   }

   type BlockIOCgroupItem struct {
    // Path represent path of blkio device
    Path string `json:"path,omitempty"`
    // Limits specifies the blkio type and value
    Limits []IOLimit `json:"limits,omitempty"`
   }

   type IOLimit struct {
    // Type specifies IO type
    Type string `json:"type,omitempty"`
    // Value specifies rate or weight value
    Value uint64 `json:"value,omitempty"`
   }
   ```

   The values of the  **Type**  field in the  **IOLimit**  structure body are as follows:

   ```go
   // BlkioThrottleReadBps is the key to fetch throttle_read_bps
   BlkioThrottleReadBps = "throttle_read_bps"

   // BlkioThrottleWriteBps is the key to fetch throttle_write_bps
   BlkioThrottleWriteBps = "throttle_write_bps"

   // BlkioThrottleReadIOPS is the key to fetch throttle_read_iops
   BlkioThrottleReadIOPS = "throttle_read_iops"

   // BlkioThrottleWriteIOPS is the key to fetch throttle_write_iops
   BlkioThrottleWriteIOPS = "throttle_write_iops"

   // BlkioWeight is the key to fetch blkio_weight
   BlkioWeight = "blkio_weight"

   // BlkioLeafWeight is the key to fetch blkio_leaf_weight
   BlkioLeafWeight = "blkio_leaf_weight"
   ```

   Example:

   ```shell
   docker run -tid --runtime kata-runtime --network none --annotation com.github.containers.virtcontainers.blkio_cgroup='{"blkiocgroup":[{"path":"/dev/sda","limits":[{"type":"throttle_read_bps","value":400},{"type":"throttle_write_bps","value":400},{"type":"throttle_read_iops","value":700},{"type":"throttle_write_iops","value":699}]},{"limits":[{"type":"blkio_weight","value":78}]}]}' busybox sleep 999999
   ```

   The preceding command is used to limit the block I/O traffic of the  **/dev/sda**  disk used by the started secure container by setting  **throttle\_read\_bps**  to 400 bit/s,  **throttle\_write\_bps**  to 400 bit/s,  **throttle\_read\_iops**  to 700 times/s,  **throttle\_write\_iops**  to 699 times/s, and the weight of the block I/O cgroup to 78.

## Limiting File Descriptor Resources

To prevent the file descriptor resources on the host from being exhausted when a large number of files in the 9p shared directory are opened in the container, the secure container can customize the maximum number of file descriptors that can be opened by the QEMU process of the secure container.

The secure container reuses the  `--files-limit`  option in the  `docker run`  command to set the maximum number of file descriptors that can be opened by the QEMU process of the secure container. This parameter can be configured only on the pause container. The usage method is as follows:

```shell
docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=podsandbox --files-limit <max-open-files> <pause-image> bash
```

> \[!NOTE] **NOTE:**
>
> * If the value of  `--files-limit`  is less than the default minimum value  **1024**  and is not  **0**, the maximum number of file descriptors that can be opened by the QEMU process of the secure container is set to the minimum value  **1024**.
> * If the value of  `--files-limit`  is 0, the maximum number of file descriptors that can be opened by the QEMU process of the secure container is the default value obtained by dividing the maximum number of file descriptors that can be opened by the system (**/proc/sys/fs/file-max**) by 400.
> * If the maximum number of file descriptors that can be opened by the QEMU process of the secure container is not displayed when the secure container is started, the maximum number of file descriptors that can be opened by the QEMU process of the secure container is the same as the system default value.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/administrator/configuring_the_ftp_server.md
---
# Configuring the FTP Server

## General Introduction

### FTP Overview

File Transfer Protocol (FTP) is one of the earliest transmission protocols on the Internet. It is used to transfer files between the server and client. FTP allows users to access files on a remote system using a set of standard commands without logging in to the remote system. In addition, the FTP server provides the following functions:

* Subscriber classification

  By default, the FTP server classifies users into real users, guest users, and anonymous users based on the login status. The three types of users have different access permissions. Real users have complete access permissions, while anonymous users have only the permission to downloading resources.

* Command records and log file records

  FTP can use the syslogd to record data, including historical commands and user transmission data (such as the transmission time and file size). Users can obtain log information from the /var/log/ directory.

* Restricting the access scope of users

  FTP can limit the work scope of a user to the home directory of the user. After a user logs in to the system through FTP, the root directory displayed by the system is the home directory of the user. This environment is called change root (chroot for short). In this way, users can access only the main directory, but not important directories such as /etc, /home, and /usr/local. This protects the system and keeps the system secure.

### Port Used by the FTP Server

The FTP service requires multiple network ports. The server uses the following ports:

* Command channel. The default port number is 21.
* Data channel. The default port number is 20.

Port 21 is used to receive connection requests from the FTP client, and port 20 is used by the FTP server to proactively connect to the FTP client.

### Introduction to vsftpd

FTP has a long history and uses the unencrypted transmission mode, and is therefore considered insecure. This section describes the Very Secure FTP Daemon (vsftpd), to use FTP in a more secure way.

The vsftpd is introduced to build a security-centric FTP server. The vsftpd is designed with the following features:

* The startup user of the vsftpd service is a common user who has low system permission. In addition, the vsftpd service uses chroot to change the root directory, preventing the risk of misusing system tools.
* Any vsftpd command that requires high execution permission is controlled by a special upper-layer program. The upper-layer program has low permission and does not affect the system.
* vsftpd integrates most of the extra commands (such as dir, ls, and cd) used by FTP. Generally, the system does not need to provide extra commands, which are secure for the system.

## Using vsftpd

### Installing vsftpd

To use the vsftpd service, you need to install the vsftpd software. If the yum source has been configured, run the following command as the root user to install the vsftpd service:

```shell
dnf install vsftpd
```

### Service Management

To start, stop, or restart the vsftpd service, run the corresponding command as the root user.

* Starting vsftpd services

  ```shell
  systemctl start vsftpd
  ```

  You can run the netstat command to check whether communication port 21 is enabled. If the following information is displayed, the vsftpd service has been enabled.

  ```shell
  $ netstat -tulnp | grep 21
  tcp6       0      0 :::21                   :::*                    LISTEN      19716/vsftpd
  ```

  > \[!NOTE] **NOTE:**
  > If the **netstat** command does not exist, run the **dnf install net-tools** command to install the **net-tools** software and then run the **netstat** command.

* Stopping the vsftpd services

  ```shell
  systemctl stop vsftpd
  ```

* Restarting the vsftpd service

  ```shell
  systemctl restart vsftpd
  ```

## Configuring vsftpd

### vsftpd Configuration Files

You can modify the vsftpd configuration file to control user permissions.  [Table 1](#table1541615718372)  describes the vsftpd configuration files. You can modify the configuration files as required. You can run the man command to view more parameter meanings.

**Table  1**  vsftpd configuration files

### Default Configuration Description

> \[!NOTE] **NOTE:**
> The configuration content in this document is for reference only. You can modify the content based on the site requirements (for example, security hardening requirements).

In the openEuler system, vsftpd does not open to anonymous users by default. Run the vim command to view the main configuration file. The content is as follows:

```shell
$ vim /etc/vsftpd/vsftpd.conf
anonymous_enable=NO
local_enable=YES
write_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
connect_from_port_20=YES
xferlog_std_format=YES
listen=NO
listen_ipv6=YES
pam_service_name=vsftpd
userlist_enable=YES
```

[Table 2](#table18185162512499)  describes the parameters.

**Table  2**  Parameter description

### Setting the Local Time

#### Overview

In the openEuler system, vsftpd uses the Greenwich Mean Time (GMT) time by default, which may be different from the local time. For example, the GMT time is 8 hours later than the Beijing time. You need to change the GMT time to the local time. Otherwise, the server time and client time are inconsistent, which may cause errors during file upload and download.

#### Setting Method

To set the vsftpd time to the local time, perform the following steps as the **root** user:

1. Open the vsftpd.conf file and change the value of use\_localtime to  **YES**. Run the following command:

   ```shell
   vim /etc/vsftpd/vsftpd.conf
   ```

   Modify the file contents as follows:

   ```shell
   use_localtime=YES
   ```

2. Restart the vsftpd service.

   ```shell
   systemctl restart vsftpd
   ```

3. Set the vsftpd service to start automatically upon power-on.

   ```shell
   systemctl enable vsftpd
   ```

### Configuring Welcome Information

To use the vsftpd service normally, the welcome information file must exist. To configure the **welcome.txt** file of the vsftpd service, perform the following steps as the **root** user:

1. Open the vsftpd.conf configuration file, add the welcome information to the file, save the file, and exit.

   ```shell
   vim /etc/vsftpd/vsftpd.conf
   ```

   The following configuration lines need to be added:

   ```text
   banner_file=/etc/vsftpd/welcome.txt
   ```

2. Create welcome information. Specifically, open the welcome.txt file, write the welcome information, save the file, and exit.

   ```shell
   vim /etc/vsftpd/welcome.txt
   ```

   The following is an example:

   ```text
   Welcome to this FTP server!
   ```

### Configuring the Login Permission of a System Account

Generally, users need to restrict the login permission of some accounts. You can set the restriction as required.

Two files are used to restrict the login of system accounts. The default files are as follows:

* /etc/vsftpd/ftpusers: This file is managed by the PAM module and is determined by the settings of the /etc/pam.d/vsftpd file.
* /etc/vsftpd/user\_list: This file is set by userlist\_file in vsftpd.conf and is provided by vsftpd.

Both files must exist and have the same content. You can write the accounts whose UIDs are smaller than 500 to the two files by referring to the /etc/passwd. Each line indicates an account.

To restrict the login of system accounts, add the accounts to /etc/vsftpd/ftpusers and /etc/vsftpd/user\_list as the **root** user.

Open the user\_list file to view the account information in the current file. The command and output are as follows:

```shell
$ vim /etc/vsftpd/user_list
root
bin
daemon
adm
lp
sync
shutdown
halt
mail
news
uucp
operator
games
nobody
```

## Verifying Whether the FTP Service Is Successfully Set Up

You can use the FTP client provided by openEuler for verification. The command and output are as follows. Enter the user name (an existing user in the system) and password as prompted. If the message "Login successful" is displayed, the FTP server is successfully set up.

```shell
$ ftp localhost
Trying 127.0.0.1...
Connected to localhost (127.0.0.1).
220-Welcome to this FTP server!
220
Name (localhost:root): USERNAME
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> bye
221 Goodbye.
```

> \[!NOTE] **NOTE:**
> If the **ftp** command does not exist, run the **dnf install ftp** command as the **root** user to install the **ftp** software and then run the **ftp** command.

## Configuring a Firewall

To open the FTP service to the Internet, you need to configure the firewall and SElinux as the **root** user.

```shell
$ firewall-cmd --add-service=ftp --permanent
success
$ firewall-cmd --reload
success
$ setsebool -P ftpd_full_access on
```

## File Transmission

### Overview

This section describes how to transfer files after the vsftpd service is started.

### Connecting to the Server

**Command Format**

**ftp**  \[*hostname*  |  *ip-address*]

**hostname**  indicates the name of the server, and  **ip-address**  indicates the IP address of the server.

**Requirements**

Run the following command on the command-line interface (CLI) of the openEuler OS:

```shell
ftp ip-address
```

Enter the user name and password as prompted. If the following information is displayed after the authentication is successful, the FTP connection is successful. In this case, you have accessed the directory of the connected server.

```shell
ftp>
```

At this prompt, you can enter different commands to perform related operations.

* Display the current path of the server.

  ```shell
  ftp>pwd
  ```

* Display the local path. You can upload the files in this path to the corresponding location on the FTP server.

  ```shell
  ftp>lcd
  ```

* Exit the current window and return to the local Linux terminal.

  ```shell
  ftp>!
  ```

### Downloading a File

Generally, the get or mget command is used to download files.

**How to use get**

* Function description: Transfers files from a remote host to a local host.

* Command format:  **get**  \[*remote-file*] \[*local-file*]

  *remote-file*  indicates a remote file, and  *local-file*  indicates a local file.

* For example, run the following command to obtain the /home/openEuler/openEuler.htm file on the remote server to the local directory /home/myopenEuler/ and change the file name to myopenEuler.htm

  ```shell
  ftp> get /home/openEuler/openEuler.htm /home/myopenEuler/myopenEuler.htm
  ```

**How to use mget**

* Function description: Receives a batch of files from the remote host to the local host.

* Command format:  **mget**  \[*remote-file*]

  *remote-file*  indicates a remote file.

* For example, to obtain all files in the /home/openEuler/ directory on the server, run the following command:

  ```shell
  ftp> cd /home/openEuler/
  ftp> mget *.*
  ```

  > \[!NOTE] **NOTE:**
  >
  > * In this case, a message is displayed each time a file is downloaded. To block the prompt information, run the  **prompt off**  command before running the  **mget \*.\***  command.
  > * The files are downloaded to the current directory on the Linux host. For example, if you run the ftp command in /home/myopenEuler/, all files are downloaded to /home/myopenEuler/.

### Uploading a file

Generally, the put or mput command is used to upload files.

**How to use put**

* Function: Transfers a local file to a remote host.

* Command format:  **put**  \[*local-file*] \[*remote-file*]

  *remote-file*  indicates a remote file, and  *local-file*  indicates a local file.

* For example, run the following command to transfer the local myopenEuler.htm file to the remote host /home/openEuler/ and change the file name to openEuler.htm:

  ```shell
  ftp> put myopenEuler.htm /home/openEuler/openEuler.htm
  ```

**How to use mput**

* Function: Transfers a batch of files from the local host to a remote host.

* Command format:  **mput**  \[*local-file*]

  *local-file*  indicates a local file.

* For example, run the following command to upload all HTM files in the local directory to the /home/openEuler/ directory on the server:

  ```shell
  ftp> cd /home/openEuler/
  ftp> mput *.htm
  ```

### Deleting a File

Generally, the  **delete**  or  **mdelete**  command is used to delete a file.

**How to use delete**

* Function description: Deletes one or more files from the remote server.

* Command format:  **delete**  \[*remote-file*]

  *remote-file*  indicates a remote file.

* For example, to delete the /home/openEuler/openEuler.htm from the remote server, run the following command:

  ```shell
  ftp> cd /home/openEuler/
  ftp> delete openEuler.htm
  ```

**How to use mdelete**

* Function description: Deletes files from a remote server. This function is used to delete files in batches.

* Command format:  **mdelete**  \[*remote-file*]

  *remote-file*  indicates a remote file.

* For example, to delete all files whose names start with  **a** from the /home/openEuler/ directory on the remote server, run the following command:

  ```shell
  ftp> cd /home/openEuler/
  ftp> mdelete a*
  ```

### Disconnecting from the Server

Run the bye command to disconnect from the server.

```shell
ftp> bye 
```

---

---
url: /en/docs/22.03_LTS_SP4/server/network/network_config/network_configuration.md
---
# Configuring the Network

## Configuring an IP Address

### Using the nmcli Command

> \[!NOTE]NOTE\
> The network configuration configured by running the  **nmcli**  command takes effect immediately and will not be lost after the system restarts.

#### Introduction to nmcli

**nmcli**  (NetworkManager Command Line Interface) is the command-line utility to configure networking through NetworkManager. The basic format of using  **nmcli**  is as follows:

```shell
nmcli [OPTIONS] OBJECT { COMMAND | help }
```

In the preceding command,  **OBJECT**  can be one of the following options:  **general**,  **networking**,  **radio**,  **connection**, and  **device**.  **OPTIONS**  can be optional options, such as  **-t**,  **--terse**  (for script processing),**-p**,  **--pretty**  (for human-readable output),  **-h**, and  **--help**. For more information, run the  **nmcli help**  command.

```shell
nmcli help
```

Common commands are listed as follows:

* To display the general status of NetworkManager, run the following command:

  ```shell
  nmcli general status
  ```

* To display all connections, run the following command:

  ```shell
  nmcli connection show
  ```

* To display the current active connections only, add the  **-a**  or  **--active**  option as follows:

  ```shell
  nmcli connection show --active
  ```

* To display the device identified by NetworkManager and its connection status, run the following command:

  ```shell
  nmcli device status
  ```

* To start or stop network interfaces, for example, run the nmcli commands as the **root** user:

  ```shell
  nmcli connection up id enp3s0
  nmcli device disconnect enp3s0
  ```

#### Device Management

##### Connecting to a Device

Run the following command to connect NetworkManager to the corresponding network device. Try to find the proper connection configuration and activate it.

```shell
nmcli device connect "$IFNAME"   
```

> If the corresponding connection configuration does not exist, NetworkManager creates and activates a configuration file with default settings.

##### Disconnecting to a Device

Run the following command to disconnect NetworkManager with the network device and prevent the device from being automatically activated.

```shell
nmcli device disconnect "$IFNAME"    
```

#### Setting Network Connections

Run the following command to display all the available network connections:

```shell
$ nmcli con show

NAME    UUID                                  TYPE      DEVICE
enp4s0  5afce939-400e-42fd-91ee-55ff5b65deab  ethernet  enp4s0
enp3s0  c88d7b69-f529-35ca-81ab-aa729ac542fd  ethernet  enp3s0
virbr0  ba552da6-f014-49e3-91fa-ec9c388864fa  bridge    virbr0
```

> \[!NOTE]NOTE\
> In the command output,  **NAME**  indicates the connection ID (name).

After a network connection is added, the corresponding configuration file is generated and associated with the corresponding device. To check for available devices, run the following command:

```shell
$ nmcli dev status

DEVICE      TYPE      STATE      CONNECTION
enp3s0      ethernet  connected  enp3s0
enp4s0      ethernet  connected  enp4s0
virbr0      bridge    connected  virbr0
lo          loopback  unmanaged  --
virbr0-nic  tun       unmanaged  --
```

##### Configuring Dynamic IP Connections

###### Configuring IP Addresses

When DHCP is used to allocate a network, run the following command to add a network configuration file:

```shell
nmcli connection add type ethernet con-name connection-name ifname interface-name
```

For example, to create a dynamic connection configuration file named  **net-test**, run the following command as the **root** user:

```shell
$ nmcli connection add type ethernet con-name net-test ifname enp3s0
Connection 'net-test' (a771baa0-5064-4296-ac40-5dc8973967ab) successfully added.
```

The NetworkManager sets  **connection.autoconnect**  to  **yes**  and saves the setting to the  **/etc/sysconfig/network-scripts/ifcfg-net-test**  file. In the  **/etc/sysconfig/network-scripts/ifcfg-net-test**  file,  **ONBOOT**  is set to  **yes**.

###### Activating a Connection and Checking Device Connection Status

Run the following command as the **root** user to activate a network connection:

```shell
$ nmcli con up net-test 
Connection successfully activated (D-Bus active path:/org/freedesktop/NetworkManager/ActiveConnection/5)
```

Run the following command to check the connection status of devices:

```shell
$ nmcli device status

DEVICE      TYPE      STATE      CONNECTION
enp4s0      ethernet  connected  enp4s0
enp3s0      ethernet  connected  net-test
virbr0      bridge    connected  virbr0
lo          loopback  unmanaged  --
virbr0-nic  tun       unmanaged  --
```

##### Configuring Static IP Connections

###### Configuring IP Addresses

To add a static IPv4 network connection, run the following command:

```shell
nmcli connection add type ethernet con-name connection-name ifname interface-name ip4 address gw4 address
```

> \[!NOTE]NOTE\
> To add an IPv6 address and related gateway information, use the  **ip6**  and  **gw6**  options.

For example, to create a static connection configuration file named  **net-static**, run the following command as the **root** user:

```shell
nmcli con add type ethernet con-name net-static ifname enp3s0 ip4 192.168.0.10/24 gw4 192.168.0.254
```

You can also specify the IPv6 address and gateway for the device. The following is an example:

```shell
$ nmcli con add type ethernet con-name test-lab ifname enp3s0 ip4 192.168.0.10/24 gw4 192.168.0.254 ip6 abbe::**** gw6 2001:***::*
Connection 'net-static' (63aa2036-8665-f54d-9a92-c3035bad03f7) successfully added.
```

The NetworkManager sets the internal parameter  **ipv4.method**  to  **manual**,  **connection.autoconnect**  to  **yes**, and writes the setting to the  **/etc/sysconfig/network-scripts/ifcfg-my-office**  file. In the file,  **BOOTPROTO**  is set to  **none**, and  **ONBOOT**  is set to  **yes**.

Run the following command as the **root** user to set IPv4 addresses of two DNS servers:

```shell
nmcli con mod net-static ipv4.dns "*.*.*.* *.*.*.*"
```

Run the following command as the **root** user to set IPv6 addresses of two DNS servers:

```shell
nmcli con mod net-static ipv6.dns "2001:4860:4860::**** 2001:4860:4860::****"
```

###### Activating a Connection and Checking Device Connection Status

Run the following command as the **root** user to activate a network connection:

```shell
$ nmcli con up net-static ifname enp3s0
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
```

Run the following command to check the connection status of devices:

```shell
$ nmcli device status

DEVICE      TYPE      STATE      CONNECTION
enp4s0      ethernet  connected  enp4s0
enp3s0      ethernet  connected  net-static
virbr0      bridge    connected  virbr0
lo          loopback  unmanaged  --
virbr0-nic  tun       unmanaged  --
```

Run the following command to view the connection details (with the  **-p**  and  **--pretty**  options to add the title and segment to the output):

```shell
$ nmcli -p con show net-static 
===============================================================================
Connection profile details (net-static )
===============================================================================
connection.id:                          net-static
connection.uuid:                        b9f18801-6084-4aee-af28-c8f0598ff5e1
connection.stable-id:                   --
connection.type:                        802-3-ethernet
connection.interface-name:              enp3s0
connection.autoconnect:                 yes
connection.autoconnect-priority:        0
connection.autoconnect-retries:         -1 (default)
connection.multi-connect:               0 (default)
connection.auth-retries:                -1
connection.timestamp:                   1578988781
connection.read-only:                   no
connection.permissions:                 --
connection.zone:                        --
connection.master:                      --
connection.slave-type:                  --
connection.autoconnect-slaves:          -1 (default)
connection.secondaries:                 --
connection.gateway-ping-timeout:        0
connection.metered:                     unknown
connection.lldp:                        default
connection.mdns:                        -1 (default)
connection.llmnr:                       -1 (default)
```

##### Adding a Wi-Fi Connection

You can add the Wi-Fi connection using either of the following methods:

**Method 1: Connect to the Wi-Fi network using a network port.**

Connect to the Wi-Fi network specified by the SSID or BSSID. Run the following command to find a matching connection or create a connection, and then activate the connection on the device.

```shell
nmcli device wifi connect "$SSID" password "$PASSWORD" ifname "$IFNAME"  
nmcli --ask device wifi connect "$SSID" 
```

**Method 2: Connect to the Wi-Fi network using the configuration file.**

1. Run the following command to check for available Wi-Fi access points:

   ```shell
   nmcli dev wifi list
   ```

2. Run the following command to generate a static IP address configuration that allows Wi-Fi connections automatically allocated by the DNS:

   ```shell
   nmcli con add con-name Wifi ifname wlan0 type wifi ssid MyWifi ip4 192.168.100.101/24 gw4 192.168.100.1
   ```

3. Run the following command to set a WPA2 password, for example,  **answer**:

   ```shell
   nmcli con modify Wifi wifi-sec.key-mgmt wpa-psk
   nmcli con modify Wifi wifi-sec.psk answer
   ```

4. Run the following command to change the Wi-Fi status:

   ```shell
   nmcli radio wifi [ on | off ]
   ```

##### Modifying Attributes

Run the following command to check a specific attribute, for example, mtu:

```shell
$ nmcli connection show id 'Wifi ' | grep mtu
802-11-wireless.mtu: auto
```

Run the following command to modify the attribute:

```shell
nmcli connection modify id 'Wifi ' 802-11-wireless.mtu 1350
```

Run the following command to confirm the modification:

```shell
$ nmcli connection show id 'Wifi ' | grep mtu
802-11-wireless.mtu: 1350
```

#### Configuring a Static Route

* Run the nmcli command to configure a static route for a network connection:

  ```shell
  nmcli connection modify enp3s0 +ipv4.routes "192.168.122.0/24 10.10.10.1"
  ```

* Run the following command to configure the static route using the editor:

  ```shell
  $ nmcli con edit type ethernet con-name enp3s0
  ===| nmcli interactive connection editor |===
  Adding a new '802-3-ethernet' connection
  Type 'help' or '?' for available commands.
  Type 'describe [<setting>.<prop>]' for detailed property description.
  You may edit the following settings: connection, 802-3-ethernet (ethernet), 802-1x, ipv4, ipv6, dcb
  nmcli> set ipv4.routes 192.168.122.0/24 10.10.10.1
  nmcli>
  nmcli> save persistent
  Saving the connection with 'autoconnect=yes'. That might result in an immediate activation of the connection.
  Do you still want to save? [yes] yes
  Connection 'enp3s0' (1464ddb4-102a-4e79-874a-0a42e15cc3c0) successfully saved.
  nmcli> quit
  ```

### Using the ip Command

> \[!NOTE]NOTE\
> The network configuration configured using the  **ip**  command takes effect immediately, but the configuration will be lost after the system restarts.

#### Configuring IP Addresses

Run the  **ip**  command to configure an IP address for the interface. The command format is as follows, where  *interface-name*  indicates the NIC name.

```shell
ip addr [ add | del ] address dev interface-name
```

##### Configuring a Static IP Address

Run the following command as the **root** user to configure an IP address:

```shell
ip address add 192.168.0.10/24 dev enp3s0
```

Run the following command as the **root** user to view the configuration result:

```shell
$ ip addr show dev enp3s0
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:aa:ad:4a brd ff:ff:ff:ff:ff:ff
    inet 192.168.202.248/16 brd 192.168.255.255 scope global dynamic noprefixroute enp3s0
       valid_lft 9547sec preferred_lft 9547sec
    inet 192.168.0.10/24 scope global enp3s0
       valid_lft forever preferred_lft forever
    inet6 fe80::32e8:cc22:9db2:f4d4/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
```

##### Configuring Multiple IP Addresses

The  **ip**  command can be used to assign multiple IP addresses to an interface. You can run the  **ip**  command multiple times as the **root** user to assign IP addresses to an interface. The following is an example:

```shell
$ ip address add 192.168.2.223/24 dev enp4s0
$ ip address add 192.168.4.223/24 dev enp4s0
$ ip addr

3: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:aa:da:e2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.203.12/16 brd 192.168.255.255 scope global dynamic noprefixroute enp4s0
       valid_lft 8389sec preferred_lft 8389sec
    inet 192.168.2.223/24 scope global enp4s0
       valid_lft forever preferred_lft forever
    inet 192.168.4.223/24 scope global enp4s0
       valid_lft forever preferred_lft forever
    inet6 fe80::1eef:5e24:4b67:f07f/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
```

#### Configuring a Static Route

To add a static route to the routing table, run the **ip route add**  command. To delete a route, run the  **ip route del**  command. The following shows the common format of the  **ip route**  command:

```shell
ip route [ add | del | change | append | replace ] destination-address
```

To display the current IP routing table, run the  **ip route**  command as the **root** user. The following is an example:

```shell
$ ip route

default via 192.168.0.1 dev enp3s0 proto dhcp metric 100
default via 192.168.0.1 dev enp4s0 proto dhcp metric 101
192.168.0.0/16 dev enp3s0 proto kernel scope link src 192.168.202.248 metric 100
192.168.0.0/16 dev enp4s0 proto kernel scope link src 192.168.203.12 metric 101
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown
```

To add a static route to the host address, run the following command as the **root** user:

```shell
ip route add 192.168.2.1 via 10.0.0.1 [dev interface-name]
```

In the preceding command,  **192.168.2.1**  is the IP address in the dot-decimal notation,  **10.0.0.1**  is the next hop, and  *interface-name*  is the exit interface for entering the next hop.

To add a static route to the network, that is, an IP address that represents an IP address range, run the following command as the **root** user:

```shell
ip route add 192.168.2.0/24 via 10.0.0.1 [dev interface-name]
```

In the preceding command,  **192.168.2.1**  is the IP address of the target network,  *10.0.0.1*  is the network prefix, and  *interface-name*  is the NIC name.

### Configuring the Network Through the ifcfg File

> \[!NOTE]NOTE\
> The network configured in the **ifcfg** file does not take effect immediately. After modifying the file (for example, **ifcfg-enp3s0**), you need to run the **nmcli con reload;nmcli con up enp3s0** command as the **root** user to reload the configuration file and activate the connection for the modification to take effect.

#### Configuring a Static Network

The following uses the  **enp4s0**  network interface as an example to describe how to configure a static network by modifying the  **ifcfg**  file as the **root** user. The  **ifcfg-enp4s0**  file is generated in the  **/etc/sysconfig/network-scripts/**  directory. Modify the following parameters in the file:

```text
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.0.10
PREFIX=24
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=enp4s0static
UUID=08c3a30e-c5e2-4d7b-831f-26c3cdc29293
DEVICE=enp4s0
ONBOOT=yes
```

#### Configuring a Dynamic Network

The following uses the  **em1**  network interface as an example to describe how to configure a dynamic network by modifying the  **ifcfg**  file. The  **ifcfg-em1**  file is generated in the  **/etc/sysconfig/network-scripts/**  directory. Modify the following parameters in the file:

```text
DEVICE=em1
BOOTPROTO=dhcp
ONBOOT=yes
```

To configure an interface to send different host names to the DHCP server, add the following content to the  **ifcfg**  file:

```text
DHCP_HOSTNAME=hostname
```

To configure an interface to ignore the routes sent by the DHCP server to prevent network services from updating the /etc/resolv.conf file using the DNS server received from the DHCP server, add the following content to the  **ifcfg**  file:

```text
PEERDNS=no
```

To configure an interface to use a specific DNS server, set the  **PEERDNS**  parameter to  **no**  and add the following content to the  **ifcfg**  file:

```text
DNS1=ip-address
DNS2=ip-address
```

**ip-address**  is the IP address of the DNS server. This allows the network service to update the **/etc/resolv.conf**  file using the specified DNS server.

#### Default Gateway Configuration

When determining the default gateway, parse the  **/etc/sysconfig/network**  file and then the  **ifcfg**  file, and uses the value of  **GATEWAY**  that is read last as the default route in the routing table.

In a dynamic network environment, when the NetworkManager is used to manage hosts, you are advised to set the default gateway to DHCP assignment.

## Configuring a Host Name

### Introduction

There are three types of host names:  **static**,  **transient**, and  **pretty**.

* **static**: Static host name, which can be set by users and saved in the  **/etc/hostname**  file.
* **transient**: Dynamic host name, which is maintained by the kernel. The initial value is a static host name. The default value is  **localhost**. The value can be changed when the DHCP or mDNS server is running.
* **pretty**: Flexible host name, which can be set in any form (including special characters/blanks). Static and transient host names are subject to the general domain name restrictions.

> \[!NOTE]NOTE\
> Static and transient host names can contain only letters (a to z and A to Z), digits (0 to 9), hyphens (-), and periods (.). The host names cannot start or end with a period (.) or contain two consecutive periods (.). The host name can contain a maximum of 64 characters.

### Configuring a Host Name by Running the hostnamectl Command

#### Viewing All Host Names

Run the following command to view the current host name:

```shell
hostnamectl status
```

> \[!NOTE]NOTE\
> If no option is specified in the command, the  **status**  option is used by default.

#### Setting All Host Names

Run the following command as the **root** user to set all host names:

```shell
hostnamectl set-hostname name
```

#### Setting a Specific Host Name

Run the following command as the **root** user to set a specific host name:

```shell
hostnamectl set-hostname name [option...]
```

The option may be one or more of  **--pretty**,  **--static**, and  **--transient**.

If  **--static**  or  **--transient**  is used together with  **--pretty**, the host names of the  **static**  or  **transient**  type will be simplified to the host names of the  **pretty**  type with spaces replaced with hyphens (-) and special characters deleted.

When setting a host name of the  **pretty**  type, use double quotation marks if the host name contains spaces or single quotation marks. An example is as follows:

```shell
hostnamectl set-hostname "Stephen's notebook" --pretty
```

#### Clearing a Specific Host Name

To clear a specific host name and restore it to the default format, run the following command as the **root** user:

```shell
hostnamectl set-hostname "" [option...]
```

In the preceding command,  **""**  is a blank character string, and the  *option*  may be one or more of  **--pretty**,  **--static**, and  **--transient**.

#### Remotely Changing a Host Name

To change the host name in a remote system, run the  **hostnamectl**  command as the **root** user with the  **-H**  or  **--host**  option.

```shell
hostnamectl set-hostname -H [username]@hostname new_hostname
```

In the preceding command,  *hostname*  indicates the name of the remote host to be configured,  *username* indicates the user-defined name, and  *new\_hostname*  indicates the new host name.  **hostnamectl**  is used to connect to the remote system through SSH.

### Configuring a Host Name by Running the nmcli Command

To query a static host name, run the following command:

```shell
nmcli general hostname
```

To name a static host as  **host-server**, run the following command as **root** user:

```shell
nmcli general hostname host-server
```

To enable the system to detect the change of the static host name, run the following command as the **root** user to restart the hostnamed service:

```shell
systemctl restart systemd-hostnamed
```

## Configuring Network Bonding

### Running the nmcli Command

* To create a bond named  **mybond0**, run the following command:

  ```shell
  nmcli con add type bond con-name mybond0 ifname mybond0 mode active-backup
  ```

* To add a slave interface, run the following command:

  ```shell
  nmcli con add type bond-slave ifname enp3s0 master mybond0
  ```

  To add another slave interface, repeat the preceding command with the new interface name:

  ```shell
  $ nmcli con add type bond-slave ifname enp4s0 master mybond0
  Connection 'bond-slave-enp4s0' (05e56afc-b953-41a9-b3f9-0791eb49f7d3) successfully added.
  ```

* To enable a bond, run the following command to enable the slave interface first:

  ```shell
  $ nmcli con up bond-slave-enp3s0
  Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/14)
  ```

  ```shell
  $ nmcli con up bond-slave-enp4s0
  Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/15)
  ```

  Then, run the following command to enable the bond:

  ```shell
  $ nmcli con up mybond0
  Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/16)
  ```

### Configuring Network Bonding by Using a Command Line

#### Checking Whether the Bonding Kernel Module Is Installed

By default, the bonding kernel module is loaded. To load this module, run the following command as the **root** user:

```shell
modprobe --first-time bonding
```

Run the following command as the **root** user to display the information about the module:

```shell
modinfo bonding
```

For more commands, run the modprobe --help command as the **root** user.

#### Creating a Channel Bonding Interface

To create a channel bonding interface, you can create a file named  **ifcfg-bondN**  in the  **/etc/sysconfig/network-scripts/**  directory as the **root** user (replacing N with the actual interface number, for example, 0).

Write the corresponding content to the configuration file according to the type of the interface to be bonded, for example, network interface. An example of the interface configuration file is as follows:

```text
DEVICE=bond0
NAME=bond0
TYPE=Bond
BONDING_MASTER=yes
IPADDR=192.168.1.1
PREFIX=24
ONBOOT=yes
BOOTPROTO=none
BONDING_OPTS="bonding parameters separated by spaces"
```

#### Creating a Slave Interface

After creating a channel bonding interface, you must add the  **MASTER**  and  **SLAVE**  instructions to the configuration file of the slave interface.

For example, to bind the two network interfaces enp3s0 and enp4s0 in channel mode, the configuration files are as follows:

```text
TYPE=Ethernet
NAME=bond-slave-enp3s0
UUID=3b7601d1-b373-4fdf-a996-9d267d1cac40
DEVICE=enp3s0
ONBOOT=yes
MASTER=bond0
SLAVE=yes
```

```text
TYPE=Ethernet
NAME=bond-slave-enp4s0
UUID=00f0482c-824f-478f-9479-abf947f01c4a
DEVICE=enp4s0
ONBOOT=yes
MASTER=bond0
SLAVE=yes
```

#### Activating Channel Bonding

To activate channel bonding, you need to enable all the slave interfaces. Run the following command as the  **root** user:

```shell
$ ifup enp3s0
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7)
```

```shell
$ ifup enp4s0
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/8)
```

> \[!NOTE]NOTE\
> If an interface is in  **up**  state, run the  **ifdown** *enp3s0*  command to change the state to  **down**. In the command,  *enp3s0*  indicates the actual NIC name.

After that, enable all the slave interfaces to enable the bonding (do not set them to  **Down**).

To enable the NetworkManager to detect the modifications made by the system, run the following command as the **root**  user after each modification:

```shell
nmcli con load /etc/sysconfig/network-scripts/ifcfg-device
```

Run the following command as the **root** user to check the status of the bonded interface:

```shell
$ ip link show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 52:54:00:aa:ad:4a brd ff:ff:ff:ff:ff:ff
3: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 52:54:00:aa:da:e2 brd ff:ff:ff:ff:ff:ff
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether 86:a1:10:fb:ef:07 brd ff:ff:ff:ff:ff:ff
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN mode DEFAULT group default qlen 1000
    link/ether 52:54:00:29:35:4c brd ff:ff:ff:ff:ff:ff
```

#### Creating Multiple Bondings

The system creates a channel bonding interface for each bonding, including the  **BONDING\_OPTS**  instruction. This configuration method allows multiple bonded devices to use different configurations. Perform the following operations to create multiple channel bonding interfaces:

* Create multiple  **ifcfg-bondN**  files that contain the  **BONDING\_OPTS**  instruction so that network scripts can create bonding interfaces as required.
* Create or edit the existing interface configuration file to be bonded, and add the  **SLAVE**  instruction.
* Use the MASTER instruction to assign the interface to be bonded, that is, the slave interface, to the channel bonding interface.

The following is an example of the configuration file of a channel bonding interface:

```text
DEVICE=bondN
NAME=bondN
TYPE=Bond
BONDING_MASTER=yes
IPADDR=192.168.1.1
PREFIX=24
ONBOOT=yes
BOOTPROTO=none
BONDING_OPTS="bonding parameters separated by spaces"
```

In this example, replace N with the number of the bonded interface. For example, to create two interfaces, you need to create two configuration files  **ifcfg-bond0**  and  **ifcfg-bond1**  with correct IP addresses.

## IPv6 Differences (vs IPv4)

### Restrictions

* chrony supports global addresses but not link-local addresses.
* Firefox supports the access to the global address through HTTP or HTTPS, but does not support the access to the link-local address.

### Configuration Description

#### Setting the MTU of an Interface Device

##### Overview

In an IPv6 scenario, the minimum MTU value of the entire routing path is used as the PMTU value of the current link. The source end determines whether to fragment packets based on the PMTU value. Other devices on the entire path do not need to fragment packets. This reduces the load of intermediate routing devices. The minimum value of IPv6 PMTU is 1280.

##### Setting the MTU of the Interface Device

If the MTU of an interface configured with an IPv6 address is set to a value smaller than  **1280**  (the minimum value of the IPv6 PMTU), the IPv6 address of the interface will be deleted and cannot be added again. Therefore, in IPv6 scenarios, the MTU of the interface device must be greater than or equal to 1280. Run the following commands as the **root** user to view the details:

```shell
$ ip addr show enp3s0
3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx
    inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0
       valid_lft 38663sec preferred_lft 38663sec
    inet6 2001:222::2/64 scope global
       valid_lft forever preferred_lft forever
```

```shell
$ ip link set dev enp3s0 mtu 1200
$ ip addr show enp3s0
3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1200 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx
    inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0
       valid_lft 38642sec preferred_lft 38642sec
```

```shell
$ ip addr add 2001:222::2/64 dev enp3s0
RTNETLINK answers: No buffer space available
```

```shell
$ ip link set dev enp3s0 mtu 1500
$ ip addr show enp3s0
3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx
    inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0
       valid_lft 38538sec preferred_lft 38538sec
```

```shell
$ ip addr add 2001:222::2/64 dev enp3s0
$ ip addr show enp3s0
3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx
    inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0
       valid_lft 38531sec preferred_lft 38531sec
    inet6 2001:222::2/64 scope global
       valid_lft forever preferred_lft forever
```

#### Stateful IPv6 Address Autoconfiguration

##### Overview

Both IPv6 and IPv4 addresses can be obtained through DHCP as the **root** user. There are configuration methods for IPv6 address: stateless autoconfiguration and stateful autoconfiguration.

* Stateless autoconfiguration

  The DHCP server is not required for management. The device obtains the network prefix according to the router advertisement (RA), or the prefix of a link-local address is fixed to fe80::. The interface ID is automatically obtained based on the value of IPV6\_ADDR\_GEN\_MODE in the ifcfg file.

  1. If the value of IPv6\_ADDR\_GEN\_MODE is stable-privacy, the device determines a random interface ID based on the device and network environment.
  2. If the value of IPv6\_ADDR\_GEN\_MODE is EUI64, the device determines the interface ID based on the device MAC address.

* Stateful autoconfiguration: The DHCP server manages and leases IPv6 addresses from the DHCPv6 server base on the DHCPv6 protocol.

  In stateful autoconfiguration, the DHCPv6 server can classify clients based on the vendor class configured on the clients and assign IPv6 addresses in different address segments to different types of clients. In IPv4 scenarios, the client can use the -V option of the dhclient command to set the vendor-class-identifier field. The DHCP server classifies clients based on the vendor-class-identifier field in the configuration file. In IPv6 scenarios, if the same method is used to classify clients, the classification does not take effect.

  ```shell
  dhclient -6 <interface> -V <vendor-class-identifier string> <interface>
  ```

  This is because DHCPv6 differs greatly from DHCP. The vendor-class-option in DHCPv6 replaces the vendor-class-identifier in DHCP. However, the -V option of dhclient cannot be set to vendor-class-option.

##### Setting the vendor class for dhclient in Stateful IPv6 Address Autoconfiguration

* On the client, add the setting of vendor class by using the configuration file.

  Client configuration file (/etc/dhcp/dhclient6.conf): The file location can be customized. You need to specify the configuration file using the dhclient -cf option.

  ```text
  option dhcp6.vendor-class code 16 = {integer 32, integer 16, string};
  interface "enp3s0" {
          send dhcp6.vendor-class <Enterprise-ID number> <vendor class string length> <vendor class string>;
  }
  ```

  > \[!NOTE]NOTE

  * \<Enterprise-ID number>: a 32-digit integer, indicating the enterprise ID. The enterprise is registered through the IANA.
  * \<vendor class string length>: a 16-digit integer, indicating the length of the vendor class string.
  * \<vendor class string>: character string of the vendor class to be set, for example, HWHW.

  On the client:

  ```shell
  dhclient -6 <interface> -cf /etc/dhcp/dhclient6.conf
  ```

* The DHCPv6 server configuration file (/etc/dhcp/dhcpd6.conf) needs to be specified by the dhcpd -cf option.

  ```text
  option dhcp6.vendor-class code 16 = {integer 32, integer 16, string};
  subnet6 fc00:4:12:ffff::/64 {
          class "hw" {
                  match if substring ( option dhcp6.vendor-class, 6, 10 ) = "HWHW";
          }
          pool6 {
                  allow members of "hw";
                  range6 fc00:4:12:ffff::ff10 fc00:4:12:ffff::ff20;
          }
          pool6 {
                  allow unknown clients;
                  range6 fc00:4:12:ffff::100 fc00:4:12:ffff::120;
          }
  }
  ```

  > \[!NOTE]NOTE\
  > In substring (option dhcp6.vendor-class, 6, 10), the start position of the substring is 6, because the substring contains four bytes of \<Enterprise-ID number> and two bytes of \<string length>. The end position of the substring is 6+\<vendor class string length>. In this example, the vendor class string is HWHW, and the length of the string is 4. Therefore, the end position of the substring is 6 + 4 = 10. You can specify \<vendor class string> and \<vendor class string length> as required.

  On the server:

  ```shell
  dhcpd -6 -cf /etc/dhcp/dhcpd6.conf <interface>
  ```

#### Kernel Supporting Socket-Related System Calls

##### Overview

The length of an IPv6 address is extended to 128 bits, indicating that there are sufficient IPv6 addresses for allocation. Compared with the IPv4 header, the IPv6 header is simplified, and the IPv6 automatic configuration function is enhanced. IPv6 addresses are classified into unicast addresses, multicast addresses, and anycast addresses. Common unicast addresses include link-local addresses, unique local addresses, and global addresses. As there are sufficient global IPv6 addresses, unique local addresses are not used. (formerly known as site-local addresses, which were discarded in 2004.) Currently, the mainstream unicast addresses are link-local address and global address. The current kernel supports socket system invoking. The link-local address and global address using unicast addresses are different.

##### Differences Between the link-local Address and global Address During Socket Invoking

RFC 2553: Basic Socket Interface Extensions for IPv6 defines the sockaddr\_in6 data structure as follows:

```c
struct sockaddr_in6 {     
    uint8_t         sin6_len;       /* length of this struct */     
    sa_family_t     sin6_family;    /* AF_INET6 */     
    in_port_t       sin6_port;      /* transport layer port # */     
    uint32_t        sin6_flowinfo;  /* IPv6 flow information */     
    struct in6_addr sin6_addr;      /* IPv6 address */     
    uint32_t        sin6_scope_id;  /* set of interfaces for a scope */ 
}; 
```

> \[!NOTE]NOTE\
> sin6\_scope\_id: a 32-bit integer. For the link-local address, it identifies the index of the specified interface. For the link-range sin6\_addr, it identifies the index of the specified interface. For the site-range sin6\_addr, it is used as the site identifier (the site-local address has been discarded).

When the link-local address is used for socket communication, the interface index corresponding to the address needs to be specified when the destination address is constructed. Generally, you can use the if\_nametoindex function to convert an interface name into an interface index number. Details are as follows:

```c
int port = 1234;
int sk_fd;
int iff_index = 0;
char iff_name[100] = "enp3s0";
char * ll_addr[100] = "fe80::123:456:789";
struct sockaddr_in6 server_addr;

memset(&server_addr,0,sizeof(structsockaddr_in6));
iff_index=if_nametoindex(iff_name);

server_addr.sin6_family=AF_INET6;
server_addr.sin6_port=htons(port);
server_addr.sin6_scope_id=iff_index;
inet_pton(AF_INET6, ll_addr, &(server_addr.sin6_addr));

sk_fd=socket(AF_INET6, SOCK_STREAM, IPPROTO_TCP);
connect(sk_fd, (struct sockaddr *)&server_addr, sizeof(struct sockaddr_in6));
```

#### Persistency Configuration of the IPv4 dhclient Daemon Process

##### Overview

When the NetworkManager service is used to manage network services, if the ifcfg-\<interface-name> configuration file of an interface is configured to obtain an IP address in DHCP mode, the NetworkManager service starts the dhclient daemon process to obtain an IP address from the DHCP server.

The dhclient provides the -1 option to determine whether the dhclient process persistently attempts to request an IP address or exits after the request times out before receiving a response from the DHCP server. For the IPv4 dhclient daemon process, you can set PERSISTENT\_DHCLIENT in the ifcfg-\<interface-name> configuration file to determine whether to set the persistence of the IPv4 dhclient process.

##### Restrictions

1. If the ongoing dhclient process is killed, the network service cannot automatically start it. Therefore, you need to ensure the reliability.
2. If PERSISTENT\_DHCLIENT is configured, ensure that the corresponding DHCP server exists. If no DHCP server is available when the network service is started and the dhclient process continuously attempts to send request packets but does not receive any response, the network service is suspended until the network service times out. The network service starts the IPv4 dhclient processes of multiple NICs in serial mode. If persistency is configured for a NIC but the DHCP server is not ready, the network service will be suspended when obtaining an IPv4 address for the NIC. As a result, the NIC cannot obtain an IPv4 or IPv6 address.

The preceding restrictions apply to special scenarios. You need to ensure reliability.

##### Configuration Differences Between IPv4 DHCP and IPv6 DHCPv6

You can configure the ifcfg-\<interface-name> parameter on an interface to enable IPv4 and IPv6 to dynamically obtain IP addresses using DHCP or DHCPv6. The configuration is as follows:

```text
BOOTPROTO=none|bootp|dhcp
DHCPV6C=yes|no
PERSISTENT_DHCLIENT=yes|no|1|0
```

* BOOTPROTO:  **none**  indicates that an IPv4 address is statically configured. **bootp|dhcp** enables DHCP dhclient to dynamically obtain an IPv4 address.
* DHCPV6C:  **no**  indicates that an IPv6 address is statically configured, and  **yes**  indicates that the DHCPv6 dhclient is enabled to dynamically obtain the IPv6 address.
* PERSISTENT\_DHCLIENT:  **no|0**  indicates that the IPv4 dhclient process is configured as nonpersistent. If the dhclient sends a request packet to the DHCP server but does not receive any response, the dhclient exits after a period of time and the exit value is 2.  **yes|1**  indicates that the IPv4 dhclient process is configured to be persistent. The dhclient process repeatedly sends request packets to the DHCP server.  **If PERSISTENT\_DHCLIENT is not configured, dhclient of IPv4 is set to yes|1 by default.**

  > \[!NOTE]NOTE\
  > The PERSISTENT\_DHCLIENT configuration takes effect only for IPv4 and does not take effect for IPv6-related dhclient -6 processes. By default, the persistence configuration is not performed for IPv6.

#### Differences Between IPv4 and IPv6 Configuration Using the iproute Command

##### Overview

IPv4 and IPv6 are two different protocol standards. Therefore, the iproute commands are different in usage. This section describes the differences between IPv4 and IPv6 commands in the iproute package.

To run the iproute commands, you must have the root permission.

##### Lifecycle of an IPv6 Address

Remarks:

* preferred\_lft: preferred lifetime. The preferred\_lft address has not expired and can be used for normal communication. If there are multiple preferred addresses, the address is selected based on the kernel mechanism.
* valid\_lft: valid lifetime. The address cannot be used for creating new connections within the period of \[preferred\_lft, valid\_lft]. The existing connections are still valid.

##### Command ip link

The commands are as follows:

```shell
ip link set IFNAME mtu MTU
```

The minimum PMTU of IPv6 is 1280. If the MTU is set to a value smaller than 1280, IPv6 addresses will be lost. Other devices cannot ping the IPv6 address.

##### Command ip addr

1. The commands are as follows:

   ```shell
   ip [-6] addr add IFADDR dev IFNAME
   ```

   You can choose to add the -6 option or not to add the IPv6 address. The ip addr command determines whether the address is an IPv4 address or an IPv6 address based on the address type.

   If the -6 option is specified but IFADDR is an IPv4 address, an error message is returned.

2. The commands are as follows:

   ```shell
   ip [-6] addr add IFADDR  dev IFNAME [home|nodad]
   ```

   \[home|nodad] is valid only for IPv6 addresses.

   * home: specifies the home address defined in RFC 6275. (This address is obtained by the mobile node from the home link, and is a permanent address of the mobile node. If the mobile node remains in the same home link, communication between various entities is performed normally.)
   * nodad: indicates that DAD is not performed when this IPv6 address is added. (RFC 4862) If multiple interfaces on a device are configured with the same IPv6 address through nodad, the IPv6 address is used in the interface sequence. An IPv6 address with both nodad and non-nodad cannot be added the same interface because the two IP addresses are the same. Otherwise, the message "RTNETLINK answers: File exists" is displayed.

3. The commands are as follows:

   ```shell
   ip [-6] addr del IFADDR dev IFNAME
   ```

   You can choose to add the -6 option or not to delete an IPv6 address. The ip addr del command determines whether an IPv4 address or an IPv6 address is used based on the address type.

4. The commands are as follows:

   ```shell
   ip [-6] addr show dev IFNAME [tentative|-tentative|deprecated|-deprecated|dadfailed|-dadfailed|temporary]
   ```

   * If the -6 option is not specified, both IPv4 and IPv6 addresses are displayed. If the -6 option is specified, only IPv6 addresses are displayed.
   * \[tentative|-tentative|deprecated|-deprecated|dadfailed|-dadfailed|temporary]. These options are only for IPv6. You can filter and view addresses based on the IPv6 address status.
     1. tentative: (only for IPv6) lists only the addresses that have not passed duplicate address detection (DAD).
     2. -tentative: (only for IPv6) lists only the addresses that are not in the DAD process.
     3. deprecated: (only for IPv6) lists only the deprecated addresses.
     4. -deprecated: (only for IPv6) lists only the addresses that are not deprecated.
     5. dadfailed: (only for IPv6) lists only the addresses that fail the DAD.
     6. -dadfailed: (only for IPv6) lists only the addresses that do not encounter DAD failures.
     7. temporary: (only for IPv6) lists only the temporary addresses.

##### Command ip route

1. The commands are as follows:

   ```shell
   ip [-6] route add ROUTE [mtu lock MTU]
   ```

   * -6 option: You can add the -6 option or not when adding an IPv6 route. The ip route command determines whether an IPv4 or IPv6 address is used based on the address type.

   * mtu lock MTU: specifies the MTU of the locked route. If the MTU is not locked, the MTU value may be changed by the kernel during the PMTUD process. If the MTU is locked, PMTUD is not attempted. All IPv4 packets are not set with the DF bit and IPv6 packets are segmented based on the MTU.

2. The commands are as follows:

   ```shell
   ip [-6] route del ROUTE
   ```

   You can choose whether to add the -6 option when deleting an IPv6 route. The ip route command determines whether an IPv4 address or an IPv6 address is used based on the address type.

##### Command ip rule

1. The commands are as follows:

   ```shell
   ip [-6] rule list
   ```

   -6 option: If the -6 option is set, IPv6 policy-based routes are printed. If the -6 option is not set, IPv4 policy-based routes are printed. Therefore, you need to configure the -6 option according to the specific protocol type.

2. The commands are as follows:

   ```shell
   ip [-6] rule [add|del] [from|to] ADDR table TABLE pref PREF
   ```

   -6 option: IPv6-related policy routing entries need to be configured with the -6 option. Otherwise, the error message "Error: Invalid source address." is displayed. Accordingly, the -6 option cannot be set for IPv4-related policy routing entries. Otherwise, the error message "Error: Invalid source address." is displayed.

#### Configuration Differences of the NetworkManager Service

##### Overview

The NetworkManager service uses the ifup/ifdown logical interface definition to perform advanced network settings. Most of the parameters are set in the /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg-\<interface-name> configuration files. The former is a global setting, and the latter is a setting of a specified NIC. When the two settings conflict, the latter takes effect.

##### Configuration Differences

The configuration differences in /etc/sysconfig/network are as follows:

The differences in /etc/sysconfig/network-scripts/ifcfg-\<interface-name> are as follows:

### FAQs

#### The iscsi-initiator-utils Does Not Support the fe80 IPv6 Address

##### Symptom

When a client uses an IPv6 address to log in to the iSCSI server, run the iscsiadm -m node -p ipv6address -l command. If the global address is used, replace ipv6address in the command example with the global address. However, the link-local address (IPv6 address starting with fe80) cannot be used because the current mechanism of iscsi-initiator-utils does not support the link-local address to log in to the iSCSI server.

##### Possible Cause

If you log in to the system using the iscsiadm -m node -p fe80::xxxx -l format, a login timeout error is returned. This is because you must specify an interface when using the link-local address. Otherwise, the iscsi\_io\_tcp\_connect function fails to invoke the connect function, and the standard error code 22 is generated.

If you use the iscsiadm -m node -p fe80::xxxx%enp3s0 -l format for login, the iscsi\_addr\_match function will compare the address fe80::xxxx%enp3s0 with the address fe80::xxxx in the node information returned by the server. The comparison result does not match, causing the login failure.

Therefore,  **the current mechanism of iscsi-initiator-utils does not support login to the iSCSI server using a link-local address.**

#### The IPv6 Address Is Lost After the NIC Is Down

##### Symptom

Run the ip link down+up NIC or ifconfig down+up NIC command to disable the NIC and then enable it to go online. Check the IP address configured on the NIC. It is found that the IPv4 address is not lost but the configured IPv6 address is lost.

##### Possible Cause

According to the processing logic in the kernel, if the NIC is set to the down state, all IPv4 and IPv6 addresses will be cleared. After the NIC is set to the up state, the IPv4 address is automatically restored, and the automatically configured IPv6 link-local address on the NIC is also restored. However, other IPv6 addresses are lost by default. To retain these IPv6 addresses, run the  **sysctl -w net.ipv6.conf.< *NIC name* >.keep\_addr\_on\_down=1**  command.

#### Taking a Long Time to Add or Delete an IPv6 Address for a Bond Interface with Multiple IPv6 Addresses

##### Symptom

When users run the following command to add or delete (including flush) an IPv6 address, the waiting time increases linearly along with the number of IPv6 addresses configured on a bond interface.  **X**  is the least significant 16 bits that dynamically change. For example, it takes about five minutes to add 3000 IPv6 address to or delete them from a bond interface that already has four physical NICs using a single thread, while for a common physical NIC, it takes less than 10 seconds.

```shell
ip a add/del 192:168::18:X/64 dev DEVICE
```

##### Possible Cause

When an IPv6 address is added to a bond interface, the IPv6 multicast address is generated and synchronized to all physical NICs. The time required increases with the number of IPv6 addresses. As a result, it takes a too long time.

##### Solution

The IPv6 multicast address is generated by combining the least significant 24 bits of the IPv6 address and 33-33-ff. If there are too many multicast addresses, it takes a long time to add or delete the address. If there are a few multicast addresses, the time required is not affected.

It is recommended that you set the least significant 24 bits of the IPv6 address to be the same as the most significant 24 bits of the IPv6 address. In this way, a single NIC can communicate with external devices using only one IP address in a network segment.

#### Rsyslog Log Transmission Is Delayed in the Scenario Where Both IPv4 and IPv6 Are Used

##### Symptom

When both IPv4 and IPv6 addresses are configured in the configuration file of the rsyslog client and the port configurations are the same, there is a possibility that log output is delayed when the server collects logs.

##### Possible Cause

The delay is caused by the buffer queue mechanism of rsyslog. By default, rsyslog writes data to a file only when the number of buffer queues reaches a specified value.

##### Solution

You can disable the buffer queue mechanism by configuring the Direct mode as the **root** user. Add the following information at the beginning of the new remote transmission configuration file in the /etc/rsyslog.d directory on the rsyslog remote transmission server:

```text
$ActionQueueType Direct
$MainMsgQueueType Direct
```

> \[!NOTE]NOTE

* In direct mode, the queue size is reduced by 1. Therefore, one log is reserved in the queue for the next log output.
* The direct mode degrades the rsyslog performance of the server.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/administrator/configuring_the_repo_server.md
---
# Configuring the Repo Server

> \[!NOTE] **NOTE:**
> openEuler provides multiple repo sources for online usage. For details about the repo sources, see [Installing the OS](./../../releasenotes/os_installation.md). If you cannot obtain the openEuler repo source online, you can use the ISO release package provided by openEuler to create a local openEuler repo source. This section uses the  **openEuler-22.03-LTS-SP4-aarch64-dvd.iso**  file as an example. Modify the ISO file as required.

## Overview

Create the  **openEuler-22.03-LTS-SP4-aarch64-dvd.iso**  file provided by openEuler as the repo source. The following uses Nginx as an example to describe how to deploy the repo source and provide the HTTP service.

## Creating or Updating a Local Repo Source

Mount the openEuler ISO file  **openEuler-22.03-LTS-SP4-aarch64-dvd.iso**  to create and update a repo source.

### Obtaining the ISO File

Obtain the openEuler ISO file from the following website:

<https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/>

### Mounting an ISO File to Create a Repo Source

Run the  **mount**  command as the **root** user to mount the ISO file.

The following is an example:

```shell
mount /home/openEuler/openEuler-22.03-LTS-SP4-aarch64-dvd.iso /mnt/
```

The mounted mnt directory is as follows:

```text
.
│── boot.catalog
│── docs
│── EFI
│── images
│── Packages
│── repodata
│── TRANS.TBL
└── RPM-GPG-KEY-openEuler
```

In the preceding directory,  **Packages**  indicates the directory where the RPM package is stored,  **repodata**  indicates the directory where the repo source metadata is stored, and  **RPM-GPG-KEY-openEuler**  indicates the public key for signing openEuler.

### Creating a Local Repo Source

You can copy related files in the ISO file to a local directory to create a local repo source. The following is an example:

```shell
mount /home/openEuler/openEuler-22.03-LTS-SP4-aarch64-dvd.iso /mnt/
mkdir -p ~/srv/repo/
cp -r /mnt/Packages ~/srv/repo/
cp -r /mnt/repodata ~/srv/repo/
cp -r /mnt/RPM-GPG-KEY-openEuler ~/srv/repo/
```

The local Repo directory is as follows:

```text
.
│── Packages
│── repodata
└── RPM-GPG-KEY-openEuler
```

**Packages**  indicates the directory where the RPM package is stored,  **repodata**  indicates the directory where the repo source metadata is stored, and  **RPM-GPG-KEY-openEuler**  indicates the public key for signing openEuler.

### Updating the Repo Source

You can update the repo source in either of the following ways:

* Use the latest ISO file to update the existing repo source. The method is the same as that for creating a repo source. That is, mount the ISO file or copy the ISO file to the local directory.

* Add a RPM package to the  **Packages**  directory of the repo source and run the  **createrepo**  command to update the repo source.

  ```shell
  createrepo --update --workers=10 ~/srv/repo
  ```

In this command,  **--update**  indicates the update, and  **--workers**  indicates the number of threads, which can be customized.

> \[!NOTE] **NOTE:**\
> If the command output contains "createrepo: command not found", run the **dnf install createrepo** command as the **root** user to install the **createrepo** softeware.

## Deploying the Remote Repo Source

Install openEuler OS and deploy the repo source using Nginx on openEuler OS.

### Installing and Configuring Nginx

1. Download the Nginx tool and install it as the **root** user.

2. After Nginx is installed, configure /etc/nginx/nginx.conf as the **root** user.

   > \[!NOTE] **NOTE:**\
   > The configuration content in this document is for reference only. You can configure the content based on the site requirements (for example, security hardening requirements).

   ```text
   user  nginx;
   worker_processes auto;                             # You are advised to set this parameter to  **core-1** .
   error_log /var/log/nginx/error.log warn;            # Log storage location
   pid        /var/run/nginx.pid;

   events {
       worker_connections  1024;
   }

   http {
       include       /etc/nginx/mime.types;
       default_type  application/octet-stream;

       log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                         '$status $body_bytes_sent "$http_referer" '
                         '"$http_user_agent" "$http_x_forwarded_for"';

       access_log  /var/log/nginx/access.log  main;
       sendfile        on;
       keepalive_timeout  65;

       server {
           listen       80;
     server_name localhost;                         # Server name (URL)
           client_max_body_size 4G;
     root /usr/share/nginx/repo;                 # Default service directory

           location / {
      autoindex on; # Enable the access to lower-layer files in the directory.
               autoindex_exact_size on;
               autoindex_localtime  on; 
           }

       }

   }
   ```

### Starting Nginx

1. Run the following commands to start the Nginx service as the **root** user using systemd.

   ```shell
   systemctl enable nginx
   systemctl start nginx
   ```

2. You can run the following command to check whether Nginx is started successfully:

   ```shell
   systemctl status nginx
   ```

   * [Figure 1](#en-us_topic_0151920971_fd25e3f1d664b4087ae26631719990a71)  indicates that the Nginx service is started successfully.

     **Figure  1**  The Nginx service is successfully started.\
     ![](./figures/the-nginx-service-is-successfully-started.png)

   * If the Nginx service fails to be started, view the error information.

   ```shell
   systemctl status nginx.service --full
   ```

   **Figure  2**  The Nginx service startup fails\
   ![](./figures/nginx-startup-failure.png)

   As shown in  [Figure 2](#en-us_topic_0151920971_f1f9f3d086e454b9cba29a7cae96a4c54), the Nginx service fails to be created because the /var/spool/nginx/tmp/client\_body directory fails to be created. You need to manually create the directory as the **root** user. Solve similar problems as follows:

   ```shell
   mkdir -p /var/spool/nginx/tmp/client_body
   mkdir -p /var/spool/nginx/tmp/proxy
   mkdir -p /var/spool/nginx/tmp/fastcgi
   mkdir -p /usr/share/nginx/uwsgi_temp
   mkdir -p /usr/share/nginx/scgi_temp
   ```

### Deploying the Repo Source

1. Run the following command as the **root** user to create the /usr/share/nginx/repo directory specified in the Nginx configuration file /etc/nginx/nginx.conf:

   ```shell
   mkdir -p /usr/share/nginx/repo
   ```

2. Run the following command as the **root** user to modify the /usr/share/nginx/repo directory permission:

   ```shell
   chmod -R 755 /usr/share/nginx/repo
   ```

3. Configure firewall rules as the **root** user to enable the port (port 80) configured for Nginx.

   ```shell
   firewall-cmd --add-port=80/tcp --permanent
   firewall-cmd --reload
   ```

   Check whether port 80 is enabled as the **root** user. If the output is **yes**, port 80 is enabled.

   ```shell
   firewall-cmd --query-port=80/tcp
   ```

   You can also enable port 80 using iptables as the **root** user.

   ```shell
   iptables -I INPUT -p tcp --dport 80 -j ACCEPT
   ```

4. After the Nginx service is configured, you can use the IP address to access the web page, as shown in  [Figure 3](#en-us_topic_0151921017_fig1880404110396).

   **Figure  3**  Nginx deployment succeeded\
   ![](./figures/nginx-deployment-succeeded.png)

5. Use either of the following methods to add the repo source to the **/usr/share/nginx/repo** directory:

   * Copy related files in the image to the /usr/share/nginx/repo directory as the **root** user.

     ```shell
     mount /home/openEuler/openEuler-22.03-LTS-SP4-aarch64-dvd.iso  /mnt/
     cp -r /mnt/Packages /usr/share/nginx/repo/
     cp -r /mnt/repodata /usr/share/nginx/repo/
     cp -r /mnt/RPM-GPG-KEY-openEuler /usr/share/nginx/repo/
     chmod -R 755 /usr/share/nginx/repo
     ```

     The  **openEuler-22.03-LTS-SP4-aarch64-dvd.iso**  file is stored in the  **/home/openEuler**  directory.

   * Create a soft link for the repo source in the /usr/share/nginx/repo directory as the **root** user.

     ```shell
     ln -s /mnt /usr/share/nginx/repo/os
     ```

     **/mnt**  is the created repo source, and  **/usr/share/nginx/repo/os**  points to  **/mnt** .

## Using the repo Source

The repo source can be configured as a yum source, which is a shell front-end software package manager. Based on the Redhat package manager (RPM), YUM can automatically download the RPM package from the specified server, install the package, and process dependent relationship. It supports one-off installation for all dependent software packages.

### Configuring Repo as the Yum Source

You can configure the built repo as the yum source and create the \*\*\*.repo configuration file (the extension .repo is mandatory) in the /etc/yum.repos.d/ directory as the **root** user. You can configure the yum source on the local host or HTTP server.

* Configuring the local yum source.

  Create the  **openEuler.repo**  file in the  **/etc/yum.repos.d**  directory and use the local repository as the yum source. The content of the  **openEuler.repo**  file is as follows:

  ```text
  [base]
  name=base
  baseurl=file:///home/openEuler/srv/repo
  enabled=1
  gpgcheck=1
  gpgkey=file:///home/openEuler/srv/repo/RPM-GPG-KEY-openEuler
  ```

  > \[!NOTE] **NOTE:**
  >
  > * **repoid** indicates the ID of the software repository. Repoids in all .repo configuration files must be unique. In the example,  **repoid**  is set to **base**.
  > * **name** indicates the string that the software repository describes.
  > * **baseurl** indicates the address of the software repository.
  > * **enabled** indicates whether to enable the software source repository. The value can be **1** or **0**. The default value is **1**, indicating that the software source repository is enabled.
  > * **gpgcheck** indicates whether to enable the GNU privacy guard (GPG) to check the validity and security of sources of RPM packages.  **1**  indicates GPG check is enabled.  **0**  indicates the GPG check is disabled.
  > * **gpgkey** indicates the public key used to verify the signature.

* Configuring the yum source for the HTTP server

  Create the **openEuler.repo** file in the **/etc/yum.repos.d** directory.

  * If the repo source of the HTTP server deployed by the user is used as the yum source, the content of **openEuler.repo** is as follows:

  ```text
  [base]
  name=base
  baseurl=http://192.168.139.209/
  enabled=1
  gpgcheck=1
  gpgkey=http://192.168.139.209/RPM-GPG-KEY-openEuler
  ```

  > \[!NOTE] **NOTE:**\
  > 192.168.139.209 is an example. Replace it with the actual IP address.

  * If the openEuler repo source provided by openEuler is used as the yum source, the content of **openEuler.repo** is as follows (the AArch64-based OS repo source is used as an example):

  ```text
  [base]
  name=base
  baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/aarch64/
  enabled=1
  gpgcheck=1
  gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/aarch64/RPM-GPG-KEY-openEuler
  ```

### repo Priority

If there are multiple repo sources, you can set the repo priority in the .repo file. If the priority is not set, the default priority is  **99** . If the same RPM package exists in the sources with the same priority, the latest version is installed.  **1**  indicates the highest priority and  **99**  indicates the lowest priority. The following shows how to set the priority of  **openEuler.repo**  to  **2**.

```text
[base]
name=base
baseurl=http://192.168.139.209/
enabled=1
priority=2
gpgcheck=1
gpgkey=http://192.168.139.209/RPM-GPG-KEY-openEuler
```

### Related Commands of dnf

The  **dnf**  command can automatically parse the dependency between packages during installation and upgrade. The common usage method is as follows:

```shell
dnf <command> <packages name>
```

Common commands are as follows:

* Installation

  Run the following command as the **root** user.

  ```shell
  dnf install <packages name>
  ```

* Upgrade

  Run the following command as the **root** user.

  ```shell
  dnf update <packages name>
  ```

* Rollback

  Run the following command as the **root** user.

  ```shell
  dnf downgrade <packages name>
  ```

* Update check

  ```shell
  dnf check-update
  ```

* Uninstallation

  Run the following command as the **root** user.

  ```shell
  dnf remove <packages name>
  ```

* Query

  ```shell
  dnf search <packages name>
  ```

* Local installation

  Run the following command as the **root** user.

  ```shell
  dnf localinstall <absolute path to package name>
  ```

* Historical records check

  ```shell
  dnf history
  ```

* Cache records clearing

  ```shell
  dnf clean all
  ```

* Cache update

  ```shell
  dnf makecache
  ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/administrator/configuring_the_web_server.md
---
# Configuring the Web Server

## Apache Server

### Overview

World Wide Web (Web) is one of the most commonly used Internet protocols. At present, the web server in the Unix-Like system is mainly implemented through the Apache server software. To operate dynamic websites, LAMP (Linux + Apache + MySQL + PHP) is developed. Web services can be combined with multimedia such as text, graphics, images, and audio, and support information transmission through hyperlinks.

The web server version in the openEuler system is Apache HTTP server 2.4, that is, httpd, which is an open-source web server developed by the Apache Software Foundation.

### Managing httpd

#### Overview

You can use the systemctl tool to manage the httpd service, including starting, stopping, and restarting the service, and viewing the service status. This section describes how to manage the Apache HTTP service.

#### Prerequisites

* To use the Apache HTTP service, ensure that the rpm package of the httpd service has been installed in your system. Run the following command as the **root** user to install the rpm package:

  ```shell
  # dnf install httpd
  ```

  For more information about service management, see  [Service Management](./service_management.md).

* To start, stop, and restart the httpd service, you must have the root permission.

#### Starting a Service

* Run the following command to start and run the httpd service:

  ```shell
  # systemctl start httpd
  ```

* If you want the httpd service to automatically start when the system starts, the command and output are as follows:

  ```shell
  # systemctl enable httpd
  Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service.
  ```

> \[!NOTE] **NOTE:**\
> If the running Apache HTTP server functions as a secure server, a password is required after the system is started. The password is an encrypted private SSL key.

#### Stopping the Service

* Run the following command to stop the httpd service:

  ```shell
  # systemctl stop httpd
  ```

* If you want to prevent the service from automatically starting during system startup, the command and output are as follows:

  ```shell
  # systemctl disable httpd
  Removed /etc/systemd/system/multi-user.target.wants/httpd.service.
  ```

#### Restarting a Service

You can restart the service in any of the following ways:

* Restart the service by running the restart command:

  ```shell
  # systemctl restart httpd
  ```

  This command stops the ongoing httpd service and restarts it immediately. This command is generally used after a service is installed or when a dynamically loaded module (such as PHP) is removed.

* Reload the configuration.

  ```shell
  # systemctl reload httpd
  ```

  This command causes the running httpd service to reload its configuration file. Any requests that are currently being processed will be interrupted, causing the client browser to display an error message or re-render some pages.

* Re-load the configuration without affecting the activation request.

  ```shell
  # apachectl graceful
  ```

  This command causes the running httpd service to reload its configuration file. Any requests that are currently being processed will continue to use the old configuration file.

#### Verifying the Service Status

Check whether the httpd service is running.

```shell
$ systemctl is-active httpd
```

If active is displayed in the command output, the service is running.

### Configuration File Description

After the httpd service is started, it reads the configuration file shown in  [Table 1](#table24341012096)  by default.

**Table  1**  Configuration file description

Although the default configuration can be used in most cases, you need to be familiar with some important configuration items. After the configuration file is modified, run the following command as the **root** user to check the syntax errors that may occur in the configuration file:

```shell
# apachectl configtest
```

If the following information is displayed, the syntax of the configuration file is correct:

```shell
Syntax OK
```

> \[!NOTE] **NOTE:**
>
> * Before modifying the configuration file, back up the original file so that the configuration file can be quickly restored if a fault occurs.
> * The modified configuration file takes effect only after the web service is restarted.

### Management Module and SSL

#### Overview

The httpd service is a modular application that is distributed with many Dynamic Shared Objects (DSOs). DSOs can be dynamically loaded or unloaded when running if necessary. These modules are located in the /usr/lib64/httpd/modules/ directory of the server operating system. This section describes how to load and write a module.

#### Loading a Module

To load a special DSO module, you can use the load module indication in the configuration file. The modules provided by the independent software package have their own configuration files in the /etc/httpd/conf.modules.d directory.

For example, to load the asis DSO module, perform the following steps:

1. In the /etc/httpd/conf.modules.d/00-optional.conf file, uncomment the following configuration line as the **root** user:

   ```shell
   LoadModule asis_module modules/mod_asis.so
   ```

2. After the loading is complete, restart the httpd service as the **root** user to reload the configuration file.

   ```shell
   # systemctl restart httpd
   ```

3. After the loading is complete, run the httpd -M command as the **root** user to check whether the asis DSO module is loaded.

   ```shell
   # httpd -M | grep asis
   ```

   If the following information is displayed, the asis DSO module is successfully loaded:

   ```shell
   asis_module (shared)
   ```

> \[!NOTE] **NOTE:**\
> **Common httpd commands**
>
> * httpd -v: views the httpd version number.
> * httpd -l: views the static modules compiled into the httpd program.
> * httpd -M: views the static modules and loaded dynamic modules that have been compiled into the httpd program.

#### Introduction to SSL

Secure Sockets Layer (SSL) is an encryption protocol that allows secure communication between the server and client. The Transport Layer Security (TLS) protocol ensures security and data integrity for network communication. openEuler supports Mozilla Network Security Services (NSS) as the security protocol TLS. To load the SSL, perform the following steps:

1. Install the  **mod\_ssl**  RPM package as the **root** user.

   ```shell
   # dnf install mod_ssl
   ```

2. After the loading is complete, restart the httpd service as the **root** user to reload the configuration file.

   ```shell
   # systemctl restart httpd
   ```

3. After the loading is complete, run the  **httpd -M**  command as the **root** user to check whether the SSL is loaded.

   ```shell
   # httpd -M | grep ssl
   ```

   If the following information is displayed, the SSL has been loaded successfully.

   ```shell
   ssl_module (shared)
   ```

### Verifying Whether the Web Service Is Successfully Set Up

After the web server is set up, perform the following operations to check whether the web server is set up successfully:

1. Run the following command as the **root** user to check the IP address of the server:

   ```shell
   # ifconfig
   ```

   If the following information is displayed, the IP address of the server is 192.168.1.60.

   ```shell
   enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
   inet 192.168.1.60  netmask 255.255.255.0  broadcast 192.168.1.255
   inet6 fe80::5054:ff:fe95:499f  prefixlen 64  scopeid 0x20<link>
   ether 52:54:00:95:49:9f  txqueuelen 1000  (Ethernet)
   RX packets 150713207  bytes 49333673733 (45.9 GiB)
   RX errors 0  dropped 43  overruns 0  frame 0
   TX packets 2246438  bytes 203186675 (193.7 MiB)
   TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

   enp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
   ether 52:54:00:7d:80:9e  txqueuelen 1000  (Ethernet)
   RX packets 149937274  bytes 44652889185 (41.5 GiB)
   RX errors 0  dropped 1102561  overruns 0  frame 0
   TX packets 0  bytes 0 (0.0 B)
   TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

   lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
   inet 127.0.0.1  netmask 255.0.0.0
   inet6 ::1  prefixlen 128  scopeid 0x10<host>
   loop  txqueuelen 1000  (Local Loopback)
   RX packets 37096  bytes 3447369 (3.2 MiB)
   RX errors 0  dropped 0  overruns 0  frame 0
   TX packets 37096  bytes 3447369 (3.2 MiB)
   TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
   ```

2. Configure the firewall as the **root** user.

   ```shell
   # firewall-cmd --add-service=http --permanent
   success
   # firewall-cmd --reload
   success
   ```

3. Verify whether the web server is successfully set up. You can select the Linux or Windows operating system for verification.
   * Using the Linux OS

     Run the following command to check whether the web page can be accessed. If the service is successfully set up, the web page can be accessed.

     ```shell
     $ curl http://192.168.1.60
     ```

     Run the following command to check whether the command output is 0. If the command output is 0, the httpd server is successfully set up.

     ```shell
     $ echo $?
     ```

   * Using the Windows OS

     Open the browser and enter the following address in the address box. If the web page can be accessed, the httpd server is successfully set up.

     <http://192.168.1.60>

     If the port number is changed, enter the address in the following format:

     <http://192.168.1.60>: port number

## Nginx Server

### Overview

Nginx is a lightweight web server which also acts as a reverse proxy server and email (IMAP/POP3) proxy server. It features low memory usage and strong concurrency capability. Nginx supports FastCGI, SSL, virtual hosts, URL rewrite, Gzip, and extension of many third-party modules.

### Installing Nginx

1. Configure the local yum source. For details, see  [Configuring the Repo Server](./configuring_the_repo_server.md).

2. Clear the cache.

   ```shell
   $ dnf clean all
   ```

3. Create a cache.

   ```shell
   $ dnf makecache
   ```

4. Install the Nginx server as the **root** user.

   ```shell
   # dnf install nginx
   ```

5. Check the installed RPM package.

   ```shell
   $ dnf list all | grep nginx
   ```

### Managing Nginx

#### Overview

You can use the systemctl tool to manage the Nginx service, including starting, stopping, and restarting the service, and viewing the service status. This section describes how to manage the Nginx service.

#### Prerequisites

* Ensure that the Nginx service has been installed. If not, install it by referring to  [Installing Nginx](#installing-nginx).

  For more information about service management, see  [Service Management](./service_management.md).

* To start, stop, and restart the Nginx service, you must have the  **root**  permission.

#### Starting a Service

* Run the following command to start and run the Nginx service:

  ```shell
  # systemctl start nginx
  ```

* If you want the Nginx service to automatically start when the system starts, the command and output are as follows:

  ```shell
  # systemctl enable nginx
  Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.
  ```

> \[!NOTE] **NOTE:**\
> If the running Nginx server functions as a secure server, a password is required after the system is started. The password is an encrypted private SSL key.

#### Stopping the Service

* Run the following command to stop the Nginx service:

  ```shell
  # systemctl stop nginx
  ```

* If you want to prevent the service from automatically starting during system startup, the command and output are as follows:

  ```shell
  # systemctl disable nginx
  Removed /etc/systemd/system/multi-user.target.wants/nginx.service.
  ```

#### Restarting a Service

You can restart the service in any of the following ways:

* Restart the service.

  ```shell
  # systemctl restart nginx
  ```

  This command stops the ongoing Nginx service and restarts it immediately. This command is generally used after a service is installed or when a dynamically loaded module (such as PHP) is removed.

* Reload the configuration.

  ```shell
  # systemctl reload nginx
  ```

  This command causes the running Nginx service to reload its configuration file. Any requests that are currently being processed will be interrupted, causing the client browser to display an error message or re-render some pages.

* Smoothly restart Nginx.

  ```shell
  # kill -HUP PID
  ```

  This command causes the running Nginx service to reload its configuration file. Any requests that are currently being processed will continue to use the old configuration file.

#### Verifying the Service Status

Check whether the Nginx service is running.

```shell
$ systemctl is-active nginx
```

If  **active** is displayed in the command output, the service is running.

### Configuration File Description

After the Nginx service is started, it reads the configuration file shown in  [Table 2](#table24341012096)  by default.

**Table  2**  Configuration file description

Although the default configuration can be used in most cases, you need to be familiar with some important configuration items. After the configuration file is modified, run the following command as the **root** user to check the syntax errors that may occur in the configuration file:

```shell
# nginx -t
```

If the command output contains  **syntax is ok**, the syntax of the configuration file is correct.

> \[!NOTE] **NOTE:**
>
> * Before modifying the configuration file, back up the original file so that the configuration file can be quickly restored if a fault occurs.
> * The modified configuration file takes effect only after the web service is restarted.

### Management Modules

#### Overview

The Nginx service is a modular application that is distributed with many Dynamic Shared Objects (DSOs). DSOs can be dynamically loaded or unloaded when running if necessary. These modules are located in the  **/usr/lib64/nginx/modules/**  directory of the server operating system. This section describes how to load and write a module.

#### Loading a Module

To load a special DSO module, you can use the load module indication in the configuration file. Generally, the modules provided by independent software packages have their own configuration files in the  **/usr/share/nginx/modules**  directory.

The DSO is automatically loaded when the  **dnf install nginx**  command is used to install the Nginx in the openEuler operating system.

### Verifying Whether the Web Service Is Successfully Set Up

After the web server is set up, perform the following operations to check whether the web server is set up successfully:

1. Run the following command as the **root** user to check the IP address of the server:

   ```shell
   # ifconfig
   ```

   If the following information is displayed, the IP address of the server is  **192.168.1.60**.

   ```shell
   enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
   inet 192.168.1.60  netmask 255.255.255.0  broadcast 192.168.1.255
   inet6 fe80::5054:ff:fe95:499f  prefixlen 64  scopeid 0x20<link>
   ether 52:54:00:95:49:9f  txqueuelen 1000  (Ethernet)
   RX packets 150713207  bytes 49333673733 (45.9 GiB)
   RX errors 0  dropped 43  overruns 0  frame 0
   TX packets 2246438  bytes 203186675 (193.7 MiB)
   TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

   enp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
   ether 52:54:00:7d:80:9e  txqueuelen 1000  (Ethernet)
   RX packets 149937274  bytes 44652889185 (41.5 GiB)
   RX errors 0  dropped 1102561  overruns 0  frame 0
   TX packets 0  bytes 0 (0.0 B)
   TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

   lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
   inet 127.0.0.1  netmask 255.0.0.0
   inet6 ::1  prefixlen 128  scopeid 0x10<host>
   loop  txqueuelen 1000  (Local Loopback)
   RX packets 37096  bytes 3447369 (3.2 MiB)
   RX errors 0  dropped 0  overruns 0  frame 0
   TX packets 37096  bytes 3447369 (3.2 MiB)
   TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
   ```

2. Configure the firewall as the **root** user.

   ```shell
   # firewall-cmd --add-service=http --permanent
   success
   # firewall-cmd --reload
   success
   ```

3. Verify whether the web server is successfully set up. You can select the Linux or Windows operating system for verification.
   * Using the Linux OS

     Run the following command to check whether the web page can be accessed. If the service is successfully set up, the web page can be accessed.

     ```shell
     $ curl http://192.168.1.60
     ```

     Run the following command to check whether the command output is  **0**. If the command output is  **0**, the Nginx server is successfully set up.

     ```shell
     $ echo $?
     ```

   * Using the Windows OS

     Open the browser and enter the following address in the address box. If the web page can be accessed, the Nginx server is successfully set up.

     <http://192.168.1.60>

     If the port number is changed, enter the address in the following format:

     <http://192.168.1.60>: port number

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/vm_configuration.md
---
# Configuring VMs

## Overview

With StratoVirt, you can use command line parameters to specify VM configurations. Alternatively, you can interconnect StratoVirt with libvirt and use XML files to configure VMs. This chapter describes the command-line configuration mode.

> \[!NOTE]**NOTE**
>
> In this document, **/path/to/socket** indicates the socket file in the user-defined path.
>
> In openEuler 22.03 LTS SP4 and later versions, JSON files are not supported.

## Specifications

StratoVirt supports lightweight and standard VMs.

* Lightweight VMs use the lightweight microVM mainboard and the MMIO bus.
* Standard VMs support standard startup. They use the Q35 mainboard on x86 platforms, and the virt mainboard and PCI bus on AArch64 platforms.

### Lightweight VMs

* Number of VM CPUs: \[1, 254]
* VM memory size: \[256 MiB, 512 GiB]
* Number of VM disks (including hot plugged-in disks): \[0, 6]
* Number of VM NICs (including hot plugged-in NICs): \[0, 2]
* The VM console device supports only single way connection.
* If the host CPU architecture is x86\_64, a maximum of 11 MMIO devices can be configured. However, you are advised to configure a maximum of two other devices except disks and NICs. On the AArch64 platform, a maximum of 160 MMIO devices can be configured. You are advised to configure a maximum of 12 other devices except disks and NICs.

### Standard VMs

* Number of VM CPUs: \[1, 254]
* VM memory size: \[256 MiB, 512 GiB]
* The VM console device supports only single way connection.
* Only one console device is supported.
* A maximum of 32 PCI devices are supported.
* PCI bus to which the PCI device is mounted: slot ID \[0, 32); function ID \[0, 8).

## Minimal Configuration

The minimum configuration for running StratoVirt is as follows:

* Use the Linux kernel image in PE or bzImage format (x86\_64 only).
* Set the rootfs image as the virtio-blk device and add it to kernel parameters.
* Use api-channel to control StratoVirt.
* To use a serial port for login, add one to the kernel startup command line. The standard model on the AArch64 platform is ttyAMA0, and the model used in other scenarios is ttyS0.

## Configuration Description

### Command Format

The format of the command configured by running cmdline is as follows:

**/path/to/stratovirt** *- \[Parameter 1] \[Option]-\[Parameter 2] \[Option]...*

### Usage Instructions

1. To ensure that the socket required by the api-channel can be created, run the following command to clear the environment:

   ```shell
   rm [parameter] *[user-defined socket file path]*
   ```

2. Run the cmdline command.

   ```shell
   /path/to/stratovirt - [Parameter 1] [Parameter option] - [Parameter 2] [Parameter option]...
   ```

### Basic Parameters

The following table lists the basic configuration information.

| Parameter| Option| Description|
| ---------------- | ----------------------------------------------- | ------------------------------------------------------------ |
| -name            | *VMname*                                        | Configures the VM name (a string of 1 to 255 characters).|
| -kernel          | /path/to/vmlinux.bin| Configures the kernel image.|
| -append          | console=ttyS0 root=/dev/vda reboot=k panic=1 rw | Configures the kernel command line parameter. For the standard x86\_64 virtualization platform, **console** is default to **ttyS0**. For the AArch64 platform, **console** is default to **ttyAMA0**. If the virtio-console device is configured but the serial port device is not configured, set **console** to **hvc0** (irrelevant to the architecture).|
| -initrd          | /path/to/initrd.img                             | Configures the initrd file.|
| -smp             | \[cpus=] CPU count| Configures the number of CPUs. The value range is \[1, 254].|
| -m               | Memory size (MiB/GiB). The default unit is MiB.| Configures the memory size. The value range is \[256 MiB, 512 GiB].|
| -qmp             | unix:/path/to/socket,server,nowait              | Configures api-channel. Before running api-channel, ensure that the socket file does not exist.|
| -D               | /path/to/logfile                                | Configures the log file.|
| -pidfile         | /path/to/pidfile                                | Configures the pid file. This parameter must be used together with **-daemonize**. Ensure that the pid file does not exist before running the script.|
| -disable-seccomp | N/A| Disables Seccomp. Seccomp is enabled by default.|
| -daemonize       | N/A| Enables daemon processes.|

### VM Types

You can run the **-machine** parameter to specify the type of the VM to be started.

Parameters:

* **type**: VM startup type. The value is **MicroVm** for lightweight virtualization, **q35** for standard virtualization on the x86\_64 platform, and **virt** for standard virtualization on the AArch64 platform.
* **dump-guest-core** (optional): whether to dump the VM memory when a process panics.
* **mem-share** (optional): whether to share memory with other processes.

### Disk Configuration

VM disk configuration includes the following configuration items:

* **drive\_id**: disk ID.
* **path\_on\_host**: disk path.
* **serial\_num** (optional): serial number of the disk.
* **read\_only** (optional): whether the disk is read-only.
* **direct** (optional): whether to open the disk in O\_DIRECT mode.
* **iothread** (optional): iothread attribute.
* **throttling.iops-total** (optional): disk QoS for limiting disk I/O operations.
* **if** (optional): driver type. The default value is **none**. The block device is **none**.
* **bus**: bus to which the device is to be mounted.
* **addr**: IDs of the slot and function to which the device is to be mounted.
* **multifunction** (optional): whether to enable PCI multi-function.

#### Disk Configuration Modes

Disk configuration consists of two steps: driver configuration and block device configuration.

The lightweight VM configuration format is as follows:

```shell
-drive id=drive_id,file=path_on_host[,readonly=off][,direct=off][,throttling.iops-total=200][,if=none]
-device virtio-blk-device,drive=drive_id,id=blkid[,iothread=iothread1][,serial=serial_num]
```

The standard VM configuration format is as follows:

```shell
-drive id=drive_id,file=path_on_host[,readonly=off][,direct=off][,throttling.iops-total=200][,if=none]
-device virtio-blk-pci,drive=drive_id,bus=pcie.0,addr=0x3.0x0,id=blkid[,iothread=iothread1,][serial=serial_num][,multifunction=on]
```

The following describes the **throttling.iops-total** and **iothread** configuration items:

#### Disk QoS

##### Introduction

QoS is short for quality of service. In cloud scenarios, multiple VMs are started on a single host. Because the total disk access bandwidth of the host is limited, when a VM has heavy disk access pressure, it will occupy the access bandwidth of other VMs. As a result, the I/O performance of other VMs will be affected. To reduce the impact between VMs, you can configure QoS to limit the disk access rate of the VMs.

##### Precautions

* Currently, QoS supports the configuration of disk IOPS.
* The value range of IOPS is \[0, 1000000]. The value **0** indicates that the IOPS is not limited. The actual IOPS does not exceed the preset value or the upper limit of the actual backend disk performance.
* Only the average IOPS can be limited. Instantaneous burst traffic cannot be limited.

##### Configuration Methods

Usage:

**CLI**

```shell
-drive xxx,throttling.iops-total=200
```

Parameters:

* **throttling.iops-total**: I/O delivery speed of the disk on a VM after IOPS is configured. It does not exceed the value of this parameter.
* *xxx*: other settings of the disk.

#### iothread

For details about the iothread configuration, see [iothread Configuration](#iothread-configuration).

### NIC Configuration

VM NIC configuration includes the following configuration items:

* **idv**: unique device ID.
* **tap**: tap device.
* **ifname**: name of the tap device on the host.
* **mac** (optional): MAC address of the VM.
* **iothread** (optional): iothread attribute of the disk. For details about the iothread configuration of the NIC, see [iothread Configuration](#iothread-configuration).

#### Configuration Methods

> \[!NOTE]**NOTE**:
>
> Before using the network, run the following commands to configure the host bridge and tap device:
>
> ```shell
> yum install -y bridge-utils iproute net-tools
> brctl addbr qbr0
> ip tuntap add tap0 mode tap
> brctl addif qbr0 tap0
> ifconfig qbr0 up; ifconfig tap0 up
> ifconfig qbr0 192.168.0.1
> ```

1. Configure virtio-net. (\[] indicates an optional parameter.)

   Lightweight VMs:

   ```shell
   -netdev tap,id=netdevid,ifname=host_dev_name[,vhostfd=2]
   -device virtio-net-device,netdev=netdevid,id=netid[,iothread=iothread1,mac=12:34:56:78:9A:BC]
   ```

   Standard VMs:

   ```shell
   -netdev tap,id=netdevid,ifname=host_dev_name[,vhostfd=2]
   -device virtio-net-pci,netdev=netdevid,id=netid,bus=pcie.0,addr=0x2.0x0[,multifunction=on,iothread=iothread1,mac=12:34:56:78:9A:BC]
   ```

2. Configure vhost-net.

   Lightweight VMs:

   ```shell
   -netdev tap,id=netdevid,ifname=host_dev_name,vhost=on[,vhostfd=2]
   -device virtio-net-device,netdev=netdevid,id=netid[,iothread=iothread1,mac=12:34:56:78:9A:BC]
   ```

   Standard VMs:

   ```shell
   -netdev tap,id=netdevid,ifname=host_dev_name,vhost=on[,vhostfd=2]
   -device virtio-net-pci,netdev=netdevid,id=netid,bus=pcie.0,addr=0x2.0x0[,multifunction=on,iothread=iothread1,mac=12:34:56:78:9A:BC]
   ```

### chardev Configuration

Redirect I/Os from the Guest to chardev on the host. The chardev backend type can be **stdio**, **pty**, **socket**, or **file**.**file** can be set only during output. The configuration items are as follows:

* **id**: unique device ID.
* **backend**: redirection type.
* **path**: path of the device redirection file. This parameter is required only for **socket** and **file** devices.
* **server**: uses chardev as a server. This parameter is required only for **socket** devices.
* **nowait**: The expected status is disconnected. This parameter is required only for **socket** devices.

When chardev is used, a console file is created and used. Therefore, ensure that the console file does not exist before starting StratoVirt.

#### Configuration Methods

```shell
-chardev backend,id=chardev_id[,path=path,server,nowait]
```

### Serial Port Configuration

A serial port is a VM device used to transmit data between hosts and VMs. To use a serial port, configure **console** to **ttyS0** in the kernel command line, and to **ttyAMA0** for standard startup on the AArch64 platform. The configuration items are as follows:

* **chardev**: redirected chardev device.
* **backend**, **path**, **server**, and **nowait**: The meanings of these parameters are the same as those in **chardev**.

#### Configuration Methods

```shell
-serial chardev:chardev_id
```

Or:

```shell
-chardev backend[,path=path,server,nowait]
```

### Console Device Configuration

virtio-console is a universal serial port device used for data transmission between hosts and VMs. If only the console device is configured and I/O operations are performed through the console device, set **console** to **hvc0** in the kernel startup parameters. The console device has the following configuration items:

* **id**: device ID.
* **path**: path of virtio console files.
* **socket**: redirection in socket mode.
* **chardev**: redirected chardev device.

#### Configuration Methods

The console configuration consists of three steps: specify virtio-serial, create a character device, and then create a virtconsole device.

Lightweight VMs:

```shell
-device virtio-serial-device[,id=virtio-serial0]
-chardev socket,path=socket_path,id=virtioconsole1,server,nowait
-device virtconsole,chardev=virtioconsole1,id=console_id
```

Standard VMs:

```shell
-device virtio-serial-pci,bus=pcie.0,addr=0x1.0x0[,multifunction=on,id=virtio-serial0]
-chardev socket,path=socket_path,id=virtioconsole1,server,nowait
-device virtconsole,chardev=virtioconsole1,id=console_id
```

### vsock Device Configuration

The vsock is also a device for communication between hosts and VMs. It is similar to the console but has better performance. The configuration items are as follows:

* **id**: unique device ID.
* **guest\_cid**: unique context ID.

#### Configuration Methods

Lightweight VMs:

```shell
-device vhost-vsock-device,id=vsock_id,guest-cid=3
```

Standard VMs:

```shell
-device vhost-vsock-pci,id=vsock_id,guest-cid=3,bus=pcie.0,addr=0x1.0x0[,multifunction=on]
```

### Memory Huge Page Configuration

#### Introduction

StratoVirt supports the configuration of huge pages for VMs. Compared with the traditional 4 KB memory page mode, huge page memory can effectively reduce the number of TLB misses and page fault interrupts, significantly improving the performance of memory-intensive services.

#### Precautions

* The directory to which the huge pages are mounted must be an absolute path.
* Memory huge pages can be configured only during startup.
* Only static huge pages are supported.
* Configure huge pages on the host before use.
* To use the huge page feature, ensure that the VM memory size is an integer multiple of *huge page size*.

#### Mutually Exclusive Features

* If the huge page feature is configured, the balloon feature does not take effect.

#### Configuration Methods

##### Configuring Huge Pages on the Host

###### Mounting

Mount the huge page file system to a specified directory. `/path/to/hugepages` is the user-defined empty directory.

```shell
mount -t hugetlbfs hugetlbfs /path/to/hugepages
```

###### Setting the Number of Huge Pages

* Set the number of static huge pages. `num` indicates the specified number.

  ```shell
  sysctl vm.nr_hugepages=num
  ```

* Query huge page statistics.

  ```shell
  cat /proc/meminfo | grep Hugepages
  ```

  To view statistics about huge pages of other sizes, view the related information in the `/sys/kernel/mm/hugepages/hugepages-*/` directory.

> \[!NOTE]**NOTE**:
>
> Configure the StratoVirt memory specifications and huge pages based on the huge page usage. If the huge page resources are insufficient, the VM fails to be started.

#### Adding Huge Page Configuration When Starting StratoVirt

* CLI

  ```shell
  -mem-path /page/to/hugepages
  ```

  In the preceding command, `/page/to/hugepages` indicates the directory to which the huge page file system is mounted. Only absolute paths are supported.

> \[!NOTE]**NOTE**:
>
> **Typical configuration**: Set **mem-path** in the StratoVirt command line to the *huge page file system mount directory*. The StratoVirt huge page feature is recommended for the typical configuration.

### iothread Configuration

#### Introduction

After a VM with the iothread configuration is started on StratoVirt, threads independent of the main thread are started on the host. These independent threads can be used to process I/O requests of devices, improving the device I/O performance and reducing the impact on message processing on the management plane.

#### Precautions

* A maximum of eight iothreads can be configured.
* The iothread attribute can be configured for disks and NICs.
* iothreads occupy CPU resources of the host. When the I/O pressure is high in a VM, the CPU resources occupied by a single iothread depend on the disk access speed. For example, a common SATA disk occupies less than 20% CPU resources.

#### Creating an iothread

**CLI**

```shell
-object iothread,id=iothread1 -object iothread,id=iothread2
```

Parameters:

* **id**: identifies an iothread. This ID can be set to the iothread attribute of the disk or NIC. If iothread is configured in the startup parameter, the thread with the specified ID is started on the host after the VM is started.

#### Configuring the iothread Attribute for a Disk or NIC

**CLI-based configurations**

Lightweight VMs:

Disks

```shell
-device virtio-blk-device xxx,iothread=iothread1
```

NICs

```shell
-device virtio-net-device xxx,iothread=iothread2
```

Standard VMs:

Disks

```shell
-device virtio-blk-pci xxx,iothread=iothread1
```

NICs

```shell
-device virtio-net-pci xxx,iothread=iothread2
```

Parameters:

1. **iothread**: Set this parameter to the iothread ID, indicating the thread that processes the I/O of the device.
2. *xxx*: other configurations of the disk or NIC.

### Balloon Device Configuration

#### Introduction

During running of a VM, the balloon driver in it occupies or releases memory to dynamically adjust the VM's available memory, achieving memory elasticity.

#### Precautions

* Before enabling balloon, ensure that the page size of the guest is the same as that of the host.
* The balloon feature must be enabled for the guest kernel.
* When memory elastic scaling is enabled, slight frame freezing may occur in the VM and the memory performance may deteriorate.

#### Mutually Exclusive Features

* This feature is mutually exclusive with huge page memory.
* In the x86 architecture, the number of interrupts is limited. Therefore, the total number of balloon devices and other virtio devices cannot exceed 11. By default, six block devices, two net devices, and one serial port device are used.

#### Specifications

* Each VM can be configured with only one balloon device.

#### Configuration Methods

Lightweight VMs:

```shell
-device virtio-balloon-device,deflate-on-oom=true
```

Standard VMs:

```shell
-device virtio-balloon-pci,bus=pcie.0,addr=0x4.0x0,deflate-on-oom=true[,multifunction=on]
```

\[!NOTE]**NOTE**

> 1. The value of **deflate-on-oom** is of the Boolean type, indicating whether to enable the auto deflate feature. When this feature is enabled, if the balloon device has reclaimed some memory, it automatically releases the memory to the guest when the guest requires the memory. If this feature is disabled, the memory is not automatically returned.
> 2. When running the QMP command to reclaim the VM memory, ensure that the VM has sufficient memory to keep basic running. Otherwise, some operations may time out and the VM cannot apply for idle memory.
> 3. If the huge page feature is enabled in the VM, the balloon device cannot reclaim the memory occupied by the huge pages.
>
> If **deflate-on-oom** is set to **false**, when the guest memory is insufficient, the balloon device does not automatically release the memory. As a result, the guest OOM may occur, the processes may be killed, and even the VM cannot run properly.

### RNG Configuration

#### Introduction

Virtio RNG is a paravirtualized random number generator that generates hardware random numbers for the guest.

#### Configuration Methods

Virtio RNG can be configured as the Virtio MMIO device or Virtio PCI device. To configure the Virtio RNG device as a Virtio MMIO device, run the following command:

```shell
-object rng-random,id=objrng0,filename=/path/to/random_file
-device virtio-rng-device,rng=objrng0,max-bytes=1234,period=1000
```

To configure the Virtio RNG device as a Virtio PCI device, run the following command:

```shell
-object rng-random,id=objrng0,filename=/path/to/random_file
-device virtio-rng-pci,rng=objrng0,max-bytes=1234,period=1000,bus=pcie.0,addr=0x1.0x0,id=rng-id[,multifunction=on]
```

Parameters:

* **filename**: path of the character device used to generate random numbers on the host, for example, **/dev/random**.
* **period**: period for limiting the read rate of random number characters, in milliseconds.
* **max-bytes**: maximum number of bytes of a random number generated by a character device within a period.
* **bus**: name of the bus to which the Virtio RNG device is mounted.
* **addr**: address of the Virtio RNG device. The parameter format is **addr=***\[slot].\[function]*, where *slot* and *function* indicate the slot number and function number of the device respectively. The slot number and function number are hexadecimal numbers. The function number of the Virtio RNG device is **0x0**.

#### Precautions

* If **period** and **max-bytes** are not configured, the read rate of random number characters is not limited.
* Otherwise, the value range of **max-bytes/period\*1000** is \[64, 1000000000]. It is recommended that the value be not too small to prevent the rate of obtaining random number characters from being too slow.
* Only the average number of random number characters can be limited, and the burst traffic cannot be limited.
* If the guest needs to use the Virtio RNG device, the guest kernel requires the following configurations: **CONFIG\_HW\_RANDOM=y**, **CONFIG\_HW\_RANDOM\_VIA=y**, and **CONFIG\_HW\_RANDOM\_VIRTIO=y**.
* When configuring the Virtio RNG device, check whether the entropy pool is sufficient to avoid VM freezing. For example, if the character device path is **/dev/random**, you can check **/proc/sys/kernel/random/entropy\_avail** to view the current entropy pool size. When the entropy pool is full, the entropy pool size is **4096**. Generally, the value is greater than 1000.

## Configuration Examples

### Lightweight VMs

This section provides an example of the minimum configuration for creating a lightweight VM.

1. Log in to the host and delete the socket file to ensure that the QMP can be created.

   ```shell
   rm -f /tmp/stratovirt.socket
   ```

2. Run StratoVirt.

   ```shell
   $ /path/to/stratovirt \
       -kernel /path/to/vmlinux.bin \
       -append console=ttyS0 root=/dev/vda rw reboot=k panic=1 \
       -drive file=/home/rootfs.ext4,id=rootfs,readonly=false \
       -device virtio-blk-device,drive=rootfs \
       -qmp unix:/tmp/stratovirt.socket,server,nowait \
       -serial stdio
   ```

   After the running is successful, the VM is created and started based on the specified configuration parameters.

### Standard VMs

This section provides an example of the minimum configuration for creating a standard VM on the ARM platform.

1. Delete the socket file to ensure that QMP can be created.

   ```shell
   rm -f /tmp/stratovirt.socket
   ```

2. Run StratoVirt.

   ```shell
   $ /path/to/stratovirt \
       -kernel /path/to/vmlinux.bin \
       -append console=ttyAMA0 root=/dev/vda rw reboot=k panic=1 \
       -drive file=/path/to/code_storage_file,if=pflash,unit=0[,readonly=true] \
       -drive file=/path/to/data_storage_file,if=pflash,unit=1, \
       -drive file=/home/rootfs.ext4,id=rootfs,readonly=false \
       -device virtio-blk-device,drive=rootfs,bus=pcie.0,addr=0x1 \
       -qmp unix:/tmp/stratovirt.socket,server,nowait \
       -serial stdio
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/interconnect_isula.md
---
# Connecting to the iSula Secure Container

## Overview

To provide a better isolation environment for containers and improve system security, you can interconnect StratoVirt with iSula secure containers.

## Connecting to the iSula Secure Container

### Prerequisites

iSulad and kata-containers have been installed, and iSulad supports the containerd-kata-shim-v2 container runtime and devicemapper storage driver.

The following describes how to install and configure iSulad and kata-containers.

1. Configure the Yum source and install iSulad and kata-containers as the **root** user.

   ```shell
   yum install iSulad
   yum install kata-containers
   ```

2. Create and configure a storage device.

   You need to plan the drive, for example, **/dev/sdxx**, which will be formatted.

   ```shell
   pvcreate /dev/sdxx
   vgcreate isulaVG0 /dev/sdxx
   lvcreate --wipesignatures y -n thinpool isulaVG0 -l 95%VG
   lvcreate --wipesignatures y -n thinpoolmeta isulaVG0 -l 1%VG
   lvconvert -y --zero n -c 512K --thinpool isulaVG0/thinpool --poolmetadata isulaVG0/thinpoolmeta
   ```

   Add the following information to the **/etc/lvm/profile/isulaVG0-thinpool.profile** configuration file:

   ```text
   activation {
       thin_pool_autoextend_threshold=80
       thin_pool_autoextend_percent=20
   }
   ```

   Modify **storage-driver** and **storage-opts** in the **/etc/isulad/daemon.json** configuration file as follows. Set the default storage driver type **overlay** to **devicemapper**.

   ```JSON
   "storage-driver": "devicemapper",
   "storage-opts": [
    "dm.thinpooldev=/dev/mapper/isulaVG0-thinpool",
    "dm.fs=ext4",
    "dm.min_free_space=10%"
   ],
   ```

3. Restart **isulad**.

   ```shell
   systemctl daemon-reload
   systemctl restart isulad
   ```

4. Check whether the iSula storage driver is successfully configured.

   ```shell
   isula info
   ```

   If the following information is displayed, the configuration is successful:

   ```text
   Storage Driver: devicemapper
   ```

### Interconnection Guide

This section describes how to interconnect StratoVirt with kata-containers to access the iSula container ecosystem.

#### Connecting to a Lightweight VM

1. Modify the kata configuration file. Its default path is **/usr/share/defaults/kata-containers/configuration.toml**. You can also configure the file by referring to **configuration-stratovirt.toml** in the same directory. Modify the **hypervisor** type of the secure container to **stratovirt**, **kernel** to the absolute path of the kernel image of kata-containers, and **initrd** to the **initrd** image file of kata-containers. (If you use Yum to install kata-containers, the two image files are downloaded and stored in the **/var/lib/kata/** directory by default. You can also use other images during the configuration.)

   The modified configurations are as follows:

   ```shell
   [hypervisor.stratovirt]
   path = "/usr/bin/stratovirt"
   kernel = "/var/lib/kata/kernel"
   initrd = "/var/lib/kata/kata-containers-initrd.img"
   machine_type = "microvm"
   block_device_driver = "virtio-mmio"
   use_vsock = true
   enable_netmon = true
   internetworking_model="tcfilter"
   sandbox_cgroup_with_emulator = false
   disable_new_netns = false
   disable_block_device_use = false
   disable_vhost_net = true
   ```

2. Run the `isula` command with **root** permissions to start the BusyBox secure container and interconnect StratoVirt with it.

   ```shell
   isula run -tid --runtime "io.containerd.kata.v2" --net=none --name test busybox:latest sh
   ```

3. Run the `isula ps` command to check whether the secure container **test** is running properly. Then run the following command to access the container:

   ```shell
   isula exec –ti test sh
   ```

4. Use a VM snapshot to accelerate startup of the secure container and reduce the VM memory overhead.

   Modify the kata configuration file **configuration.toml** and set **enable\_template** to **true** to allow the VM to start by creating a snapshot.

   ```shell
   [factory]
   # VM templating support. Once enabled, new VMs are created from template
   # using vm cloning. They will share the same initial kernel, initramfs and
   # agent memory by mapping it readonly. It helps speeding up new container
   # creation and saves a lot of memory if there are many kata containers running
   # on the same host.
   #
   # When disabled, new VMs are created from scratch.
   #
   # Note: Requires "initrd=" to be set ("image=" is not supported).
   #
   # Default false
   enable_template = true
   ```

   After the **enable\_template** configuration item is set to **true**, kata-containers checks whether a snapshot file exists in the default path (**/run/vc/vm/template**) during secure container creation. If yes, kata-containers starts the VM using the snapshot file. If no, kata-containers creates a VM snapshot and start the VM using the snapshot file.

5. Use the security component Ozone to further enhance the isolation of secure containers.

   Modify the kata configuration file **configuration.toml** and set the configuration item **ozone\_path** to the path of the Ozone executable file. (If StratoVirt is installed using Yum, the Ozone executable file is stored in the **/usr/bin** directory by default.) After this item is configured, the Ozone security sandbox function is enabled to protect the VM against attacks after the virtualization layer isolation is broken and further enhance the isolation of StratoVirt secure containers.

   ```toml
   # Path for the ozone specific to stratovirt
   # If the ozone path is set, stratovirt will be launched in
   # ozone secure environment. It is disabled by default.
   ozone_path = "/usr/bin/ozone"
   ```

You can now run container commands in the **test** container.

#### Connecting to a Standard VM

To use a StratoVirt standard VM as the sandbox of a secure container, you need to modify some other configurations.

1. The configurations are as follows:

   ```text
   [hypervisor.stratovirt]
   path = "/usr/bin/stratovirt"
   kernel = "/var/lib/kata/kernel"
   initrd = "/var/lib/kata/kata-containers-initrd.img"
   # x86_64 architecture
   machine_type = "q35"
   # AArch64 architecture
   machine_type = "virt"
   block_device_driver = "virtio-blk"
   pcie_root_port = 2
   use_vsock = true
   enable_netmon = true
   internetworking_model = "tcfilter"
   sandbox_cgroup_with_emulator = false
   disable_new_netns = false
   disable_block_device_use = false
   disable_vhost_net = true
   ```

   In the configurations above, modify the VM type according to the architecture of the host machine. Change the value of **block\_device\_driver** to **virtio-blk**. StratoVirt supports only devices hot-plugged to the root port. Set a proper value of **pcie\_root\_port** based on the number of devices to be hot-plugged.

2. Install the firmware required for starting a standard VM.

   x86\_64 architecture:

   ```shell
   yum install -y edk2-ovmf
   ```

   AArch64 architecture:

   ```shell
   yum install -y edk2-aarch64
   ```

3. Build and replace the binary file of kata-containers 2.x.

   Currently, a StratoVirt standard VMs can only be used as the sandbox of a kata-containers 2.x container (corresponding to the openEuler-21.09 branch in the kata-containers repository). You need to download and compile the kata-containers source code and replace the **containerd-shim-kata-v2** binary file in the **/usr/bin** directory.

   ```shell
   mkdir -p /root/go/src/github.com/
   cd /root/go/src/github.com/
   git clone https://atomgit.com/src-openeuler/kata-containers.git
   cd kata-containers
   git checkout openEuler-21.09
   ./apply-patches
   cd src/runtime
   make
   ```

   Back up the kata binary file in the **/usr/bin/** directory and replace it with the compiled binary file **containerd-shim-kata-v2**.

   ```shell
   cp /usr/bin/containerd-shim-kata-v2 /usr/bin/containerd-shim-kata-v2.bk
   cp containerd-shim-kata-v2 /usr/bin/containerd-shim-kata-v2
   ```

4. Run the `isula` command with **root** permissions to start the BusyBox secure container and interconnect StratoVirt with it.

   ```shell
   isula run -tid --runtime "io.containerd.kata.v2" --net=none --name test busybox:latest sh
   ```

5. Run the `isula ps` command to check whether the secure container **test** is running properly. Then run the following command to access the container:

   ```shell
   isula exec -ti test sh
   ```

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/syscare/constraints.md
---
# Constraints

## Version Constraints

OS version: openEuler 22.03 LTS SP4
Architecture: x86 or AArch64

## Application Constraints

Currently, user-mode patches support only Redis and Nginx.

Note:

1. Currently, each software needs to be adapted to process the LINE macro. Currently, only Redis and Nginx are adapted. Other software that is not adapted may cause the patch size to be too large. (Parameters will be introduced in the future to support user adaptation.)
2. Each user-mode live patch can contain only one ELF file. To fix multiple bugs, you can pass the patch files of multiple bug fixes to the patch making parameters to make a live patch for multiple bugs.

## Language Constraints

Theoretically, patches are compared at the object file level, which is irrelevant to the programming language. Currently, only the C and C++ languages are tested.

## Others

* Only 64-bit OSs are supported.
* Only the ELF format can be hot-patched. Interpreted languages are not supported.
* Only GCC and G++ compilers are supported.
* The compiler must support the `-gdwarf`, `-ffunction-sections`, and `-fdata-sections` parameters.
* The debug information must be in the DWARF format.
* Cross compilation is not supported.
* Source files that are in different paths but have the same file name, same global variables, and same functions cannot be recognized.
* Assembly code, including **.S** files and inline assembly code, cannot be modified.
* External symbols (dynamic library dependencies) cannot be added.
* Multiple patches cannot be applied to the same binary file.
* Mixed compilation of C and C++ is not supported.
* C++ exceptions cannot be modified.
* The `-g3` group section compilation option, specific compilation optimization options, and specific GCC plugins are not supported.
* ifunc cannot be added by using `__attribute__((ifunc("foo")))`.
* TLS variables cannot be added by using `__thread int foo`.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/container_engine.md
---
# Container Engine

Docker daemon is a system process that resides in the background. Before you run a docker subcommand, start Docker daemon.

If the Docker daemon is installed using the RPM package or system package management tool, you can run the  **systemctl start docker**  command to start the Docker daemon.

The  **docker**  command supports the following parameters:

1. To combine parameters of a single character, run the following command:

   ```shell
   docker run -t -i busybox /bin/sh
   ```

   The command can be written as follows:

   ```shell
   docker run -ti busybox /bin/sh
   ```

2. **bool**  command parameters such as  **--icc=true**, are displayed in the command help. If this parameter is not used, the default value displayed in the command help is used. If this parameter is used, the opposite value of the value displayed in the command help is used. In addition, if  **--icc**  is not added when Docker daemon is started,  **--icc=true**  is used by default. Otherwise,  **--icc=false**  is used.

3. Parameters such as  **--attach=\[]**  in the command help indicate that these parameters can be set for multiple times. For example:

   ```shell
   docker run --attach=stdin --attach=stdout -i -t busybox /bin/sh
   ```

4. Parameters such as  **-a**  and  **--attach=\[]**  in the command help indicate that the parameter can be specified using either  **-a** *value*  or  **--attach=***value*. For example:

   ```shell
   docker run -a stdin --attach=stdout -i -t busybox /bin/sh
   ```

5. Parameters such as  **--name=""**  can be configured with a character string and can be configured only once. Parameters such as  **-c=**  can be configured with an integer and can be configured only once.

**Table  1**  Parameters specified during the Docker daemon startup

---

---
url: /en/docs/22.03_LTS_SP4/cloud/image_builder/isula_build/isula_build.md
---
# Container Image Building

## Overview

isula-build is a container image build tool developed by the iSula container team. It allows you to quickly build container images using Dockerfiles.

The isula-build uses the server/client mode. The isula-build functions as a client and provides a group of command line tools for image build and management. The isula-builder functions as the server to process client management requests, and runs as a daemon process in the background.

![isula-build architecture](./figures/isula-build_arch.png)

> \[!NOTE] **Note:**
>
> * Currently, isula-build supports OCI image format ([OCI Image Format Specification](https://github.com/opencontainers/image-spec/blob/main/spec.md)) and Docker image format ([Image Manifest Version 2, Schema 2](https://docs.docker.com/registry/spec/manifest-v2-2/)). Use the `export ISULABUILD_CLI_EXPERIMENTAL=enabled` command to enable the experimental feature for supporting OCI image format. When the experimental feature is disabled, isula-build will take Docker image format as the default image format. Otherwise, isula-build will take OCI image format as the default image format.

## Installation

### Preparations

To ensure that isula-build can be successfully installed, the following software and hardware requirements must be met:

* Supported architectures: x86\_64 and AArch64
* Supported OS: openEuler
* You have the permissions of the root user.

#### Installing isula-build

Before using isula-build to build a container image, you need to install the following software packages:

**(Recommended) Method 1: Using Yum**

1. Configure the openEuler Yum source.

2. Log in to the target server as the root user and install isula-build.

   ```sh
   sudo yum install -y isula-build
   ```

**Method 2: Using the RPM Package**

1. Obtain an **isula-build-\*.rpm** installation package from the openEuler Yum source, for example, **isula-build-0.9.6-4.oe1.x86\_64.rpm**.

2. Upload the obtained RPM software package to any directory on the target server, for example, **/home/**.

3. Log in to the target server as the root user and run the following command to install isula-build:

   ```sh
   sudo rpm -ivh /home/isula-build-*.rpm
   ```

> \[!NOTE] **Note:**
>
> * After the installation is complete, you need to manually start the isula-build service. For details about how to start the service, see [Managing the isula-build Service](#managing-the-isula-build-service).

## Configuring and Managing the isula-build Service

### Configuring the isula-build Service

After the isula-build software package is installed, the systemd starts the isula-build service based on the default configuration contained in the isula-build software package on the isula-build server. If the default configuration file on the isula-build server cannot meet your requirements, perform the following operations to customize the configuration file: After the default configuration is modified, restart the isula-build server for the new configuration to take effect. For details, see [Managing the isula-build Service](#managing-the-isula-build-service).

Currently, the isula-build server contains the following configuration file:

* **/etc/isula-build/configuration.toml**: general isula-builder configuration file, which is used to set the isula-builder log level, persistency directory, runtime directory, and OCI runtime. Parameters in the configuration file are described as follows:

| Configuration Item    | Mandatory or Optional | Description                         | Value                         |
| --------- | -------- | --------------------------------- | ----------------------------------------------- |
| debug | Optional | Indicates whether to enable the debug log function. | **true**: Enables the debug log function. **false**: Disables the debug log function. |
| loglevel | Optional | Sets the log level.          | debuginfowarnerror                |
| run\_root | Mandatory | Sets the root directory of runtime data. | For example, **/var/run/isula-build/** |
| data\_root | Mandatory | Sets the local persistency directory. | For example, **/var/lib/isula-build/** |
| runtime | Optional | Sets the runtime type. Currently, only **runc** is supported. | runc                                            |
| group | Optional | Sets the owner group for the local socket file **isula\_build.sock** so that non-privileged users in the group can use isula-build. | isula |
| experimental | Optional | Indicates whether to enable experimental features. | **true**: Enables experimental features. **false**: Disables experimental features. |

* **/etc/isula-build/storage.toml**: configuration file for local persistent storage, including the configuration of the storage driver in use.

| Configuration Item    | Mandatory or Optional | Description                         |
| ------ | -------- | ------------------------------ |
| driver | Optional | Storage driver type. Currently, **overlay2** is supported. |

For more settings, see [containers-storage.conf.5](https://github.com/containers/storage/blob/main/docs/containers-storage.conf.5.md).

* **/etc/isula-build/registries.toml**: configuration file for each image repository.

| Configuration Item    | Mandatory or Optional | Description                         |
| ------------------- | -------- | ------------------------------------------------------------ |
| registries.search | Optional | Search domain of the image repository. Only listed image repositories can be found. |
| registries.insecure | Optional | Accessible insecure image repositories. Listed image repositories cannot pass the authentication and are not recommended. |

For more settings, see [containers-registries.conf.5](https://github.com/containers/image/blob/main/docs/containers-registries.conf.5.md).

* **/etc/isula-build/policy.json**: image pull/push policy file. Note: Currently, this parameter cannot be configured.

> \[!NOTE] **Note:**
>
> * isula-build supports the preceding configuration file with the maximum size of 1 MB.
> * The persistent working directory dataroot cannot be configured on the memory disk, for example, tmpfs.
> * Currently, only overlay2 can be used as the underlying storage driver.
> * Before setting the `--group` option, ensure that the corresponding user group has been created on a local OS and non-privileged users have been added to the group. After isula-builder is restarted, non-privileged users in the group can use the isula-build function. In addition, to ensure permission consistency, the owner group of the isula-build configuration file directory **/etc/isula-build** is set to the group specified by `--group`.

### Managing the isula-build Service

Currently, openEuler uses systemd to manage the isula-build service. The isula-build software package contains the systemd service files. After installing the isula-build software package, you can use the systemd tool to start or stop the isula-build service. You can also manually start the isula-build software.

> \[!NOTE] **Note:**
>
> * Only one isula-build process can be started on a node at a time.

#### (Recommended) Using systemd for Management

You can run the following systemd commands to start, stop, and restart the isula-build service:

* Run the following command to start the isula-build service:

  ```sh
  sudo systemctl start isula-build.service
  ```

* Run the following command to stop the isula-build service:

  ```sh
  sudo systemctl stop isula-build.service
  ```

* Run the following command to restart the isula-build service:

  ```sh
  sudo systemctl restart isula-build.service
  ```

The systemd service file of the isula-build software installation package is stored in the `/usr/lib/systemd/system/isula-build.service` directory. If you need to modify the systemd configuration of the isula-build service, modify the file and run the following command to make the modification take effect. Then restart the isula-build service based on the systemd management command.

```sh
sudo systemctl daemon-reload
```

#### Directly Running isula-builder

You can also run the `isula-builder` command on the server to start the service. The `isula-builder` command can contain flags for service startup. The following flags are supported:

* `-D, --debug`: whether to enable the debugging mode.
* `--log-level`: log level. The options are **debug**, **info**, **warn**, and **error**. The default value is **info**.
* `--dataroot`: local persistency directory. The default value is **/var/lib/isula-build/**.
* `--runroot`: runtime directory. The default value is **/var/run/isula-build/**.
* `--storage-driver`: underlying storage driver type.
* `--storage-opt`: underlying storage driver configuration.
* `--group`: sets the owner group for the local socket file **isula\_build.sock** so that non-privileged users in the group can use isula-build. The default owner group is **isula**.
* `--experimental`: whether to enable experimental features.

> \[!NOTE] **Note:**
>
> If the command line parameters contain the same configuration items as those in the configuration file, the command line parameters are preferentially used for startup.

Start the isula-build service. For example, to specify the local persistency directory **/var/lib/isula-build** and disable debugging, run the following command:

```sh
sudo isula-builder --dataroot "/var/lib/isula-build" --debug=false
```

## Usage Guidelines

### Prerequisites

isula-build depends on the executable file **runc** to build the **RUN** instruction in the Dockerfile. Therefore,  runc must be pre-installed in the running environment of isula-build. The installation method depends on the application scenario. If you do not need to use the complete docker-engine tool chain, you can install only the docker-runc RPM package.

```sh
sudo yum install -y docker-runc
```

If you need to use a complete docker-engine tool chain, install the docker-engine RPM package, which contains the executable file **runc** by default.

```sh
sudo yum install -y docker-engine
```

> \[!NOTE] **Note:**
>
> Ensure the security of OCI runtime (runc) executable files to prevent malicious replacement.

### Overview

The isula-build client provides a series of commands for building and managing container images. Currently, the isula-build client provides the following commands:

* `ctr-img`: manages container images. The `ctr-img` command contains the following subcommands:
  * `build`: builds a container image based on the specified Dockerfile.
  * `images`: lists local container images.
  * `import`: imports a basic container image.
  * `load`: imports a cascade image.
  * `rm`: deletes a local container image.
  * `save`: exports a cascade image to a local disk.
  * `tag`: adds a tag to a local container image.
  * `pull`: pulls an image to a local host.
  * `push`: pushes a local image to a remote repository.
* `info`: displays the running environment and system information of isula-build.
* `login`: logs in to the remote container image repository.
* `logout`: logs out of the remote container image repository.
* `version`: displays the versions of isula-build and isula-builder.
* `manifest` (experimental): manages the manifest list.

> \[!NOTE] **Note:**
>
> * The `isula-build completion` and `isula-builder completion` commands are used to generate the bash command completion script. These commands are implicitly provided by the command line framework and is not displayed in the help information.
> * isula-build client does not have any configuration file. To use isula-build experimental features, enable the environment variable **ISULABUILD\_CLI\_EXPERIMENTAL** on the client using the `export ISULABUILD_CLI_EXPERIMENTAL=enabled` command.

The following describes how to use these commands in detail.

### ctr-img: Container Image Management

The isula-build command groups all container image management commands into the `ctr-img` command. The command format is as follows:

```sh
isula-build ctr-img [command]
```

#### build: Container Image Build

The subcommand build of the `ctr-img` command is used to build container images. The command format is as follows:

```sh
isula-build ctr-img build [flags]
```

The `build` command contains the following flags:

* `--build-arg`: string list containing variables required during the build process.
* `--build-static`: key value, which is used to build binary equivalence. Currently, the following key values are included:
  `- build-time`: string indicating that a container image is built at a specified timestamp. The timestamp format is *YYYY-MM-DD HH-MM-SS*.
* `-f, --filename`: string indicating the path of the Dockerfiles. If this parameter is not specified, the current path is used.
* `--format`: string indicating the image format **oci** or **docker** (**ISULABUILD\_CLI\_EXPERIMENTAL** needs to be enabled).
* `--iidfile`: string indicating a local file to which the ID of the image is output.
* `-o, --output`: string indicating the image export mode and path.
* `--proxy`: boolean, which inherits the proxy environment variable on the host. The default value is **true**.
* `--tag`: string indicating the tag value of the image that is successfully built.
* `--cap-add`: string list containing permissions required by the **RUN** instruction during the build process.

**The following describes the flags in detail.**

**--build-arg**

Parameters in the Dockerfile are inherited from the commands. The usage is as follows:

```sh
$ echo "This is bar file" > bar.txt
$ cat Dockerfile_arg
FROM busybox
ARG foo
ADD ${foo}.txt .
RUN cat ${foo}.txt
$ sudo isula-build ctr-img build --build-arg foo=bar -f Dockerfile_arg
STEP  1: FROM busybox
Getting image source signatures
Copying blob sha256:8f52abd3da461b2c0c11fda7a1b53413f1a92320eb96525ddf92c0b5cde781ad
Copying config sha256:e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b
Writing manifest to image destination
Storing signatures
STEP  2: ARG foo
STEP  3: ADD ${foo}.txt .
STEP  4: RUN cat ${foo}.txt
This is bar file
Getting image source signatures
Copying blob sha256:6194458b07fcf01f1483d96cd6c34302ffff7f382bb151a6d023c4e80ba3050a
Copying blob sha256:6bb56e4a46f563b20542171b998cb4556af4745efc9516820eabee7a08b7b869
Copying config sha256:39b62a3342eed40b41a1bcd9cd455d77466550dfa0f0109af7a708c3e895f9a2
Writing manifest to image destination
Storing signatures
Build success with image id: 39b62a3342eed40b41a1bcd9cd455d77466550dfa0f0109af7a708c3e895f9a2
```

**--build-static**

Specifies a static build. That is, when isula-build is used to build a container image, differences between all timestamps and other build factors (such as the container ID and hostname) are eliminated. Finally, a container image that meets the static requirements is built.

When isula-build is used to build a container image, assume that a fixed timestamp is given to the build subcommand and the following conditions are met:

* The build environment is consistent before and after the upgrade.
* The Dockerfile is consistent before and after the build.
* The intermediate data generated before and after the build is consistent.
* The build commands are the same.
* The versions of the third-party libraries are the same.

For container image build, isula-build supports the same Dockerfile. If the build environments are the same, the image content and image ID generated in multiple builds are the same.

`--build-static` supports the key-value pair option in the *key=value* format. Currently, the following options are supported:

* build-time: string, which indicates the fixed timestamp for creating a static image. The value is in the format of *YYYY-MM-DD HH-MM-SS*. The timestamp affects the attribute of the file for creating and modifying the time at the diff layer.

  Example:

  ```sh
  sudo isula-build ctr-img build -f Dockerfile --build-static='build-time=2020-05-23 10:55:33' .
  ```

  In this way, the container images and image IDs built in the same environment for multiple times are the same.

**--format**

This option can be used when the experiment feature is enabled. The default image format is **oci**. You can specify the image format to build. For example, the following commands are used to build an OCI image and a Docker image, respectively.

```sh
export ISULABUILD_CLI_EXPERIMENTAL=enabled; sudo isula-build ctr-img build -f Dockerfile --format oci .
```

```sh
export ISULABUILD_CLI_EXPERIMENTAL=enabled; sudo isula-build ctr-img build -f Dockerfile --format docker .
```

**--iidfile**

Run the following command to output the ID of the built image to a file:

```sh
isula-build ctr-img build --iidfile filename
```

For example, to export the container image ID to the **testfile** file, run the following command:

```sh
sudo isula-build ctr-img build -f Dockerfile_arg --iidfile testfile
```

Check the container image ID in the **testfile** file.

```sh
$ cat testfile
76cbeed38a8e716e22b68988a76410eaf83327963c3b29ff648296d5cd15ce7b
```

**-o, --output**

Currently, `-o` and `--output` support the following formats:

* `isulad:image:tag`: directly pushes the image that is successfully built to iSulad, for example, `-o isulad:busybox:latest`. The following restrictions apply:

  * isula-build and iSulad must be on the same node.
  * The tag must be configured.
  * On the isula-build client, you need to temporarily save the successfully built image as **/var/tmp/isula-build-tmp-%v.tar** and then import it to iSulad. Ensure that the **/var/tmp/** directory has sufficient disk space.

* `docker-daemon:image:tag`: directly pushes the successfully built image to Docker daemon, for example, `-o docker-daemon:busybox:latest`. The following restrictions apply:

* isula-build and Docker must be on the same node.
  * The tag must be configured.

* `docker://registry.example.com/repository:tag`: directly pushes the successfully built image to the remote image repository in Docker image format, for example, `-o docker://localhost:5000/library/busybox:latest`.

* `docker-archive:<path>/<filename>:image:tag`: saves the successfully built image to the local host in Docker image format, for example, `-o docker-archive:/root/image.tar:busybox:latest`.

When experiment feature is enabled, you can build image in OCI image format with:

* `oci://registry.example.com/repository:tag`: directly pushes the successfully built image to the remote image repository in OCI image format(OCI image format should be supported by the remote repository), for example, `-o oci://localhost:5000/library/busybox:latest`.

* `oci-archive:<path>/<filename>:image:tag`: saves the successfully built image to the local host in OCI image format, for example, `-o oci-archive:/root/image.tar:busybox:latest`.

In addition to the flags, the `build` subcommand also supports an argument whose type is string and meaning is context, that is, the context of the Dockerfile build environment. The default value of this parameter is the current path where isula-build is executed. This path affects the path retrieved by the **ADD** and **COPY** instructions of the .dockerignore file and Dockerfile.

**--proxy**

Specifies whether the container started by the **RUN** instruction inherits the proxy-related environment variables **http\_proxy**, **https\_proxy**, **ftp\_proxy**, **no\_proxy**, **HTTP\_PROXY**, **HTTPS\_PROXY**, and **FTP\_PROXY**. The default value is **true**.

When a user configures proxy-related **ARG** or **ENV** in the Dockerfile, the inherited environment variables will be overwritten.

> \[!NOTE] **Note:**
>
> * If the client and daemon are running on different terminals, the environment variables of the terminal where the daemon is running are inherited.

**--tag**

Specifies the tag of the image stored on the local disk after the image is successfully built.

**--cap-add**

Run the following command to add the permission required by the **RUN** instruction during the build process:

```sh
isula-build ctr-img build --cap-add ${CAP}
```

Example:

```sh
sudo isula-build ctr-img build --cap-add CAP_SYS_ADMIN --cap-add CAP_SYS_PTRACE -f Dockerfile
```

> **Note:**
>
> * A maximum of 100 container images can be concurrently built.
> * isula-build supports Dockerfiles with a maximum size of 1 MB.
> * isula-build supports a .dockerignore file with a maximum size of 1 MB.
> * Ensure that only the current user has the read and write permissions on the Dockerfiles to prevent other users from tampering with the files.
> * During the build, the **RUN** instruction starts the container to build in the container. Currently, isula-build supports the host network only.
> * isula-build only supports the tar compression format.
> * isula-build commits once after each image build stage is complete, instead of each time a Dockerfile line is executed.
> * isula-build does not support cache build.
> * isula-build starts the build container only when the **RUN** instruction is built.
> * Currently, the history function of Docker images is not supported.
> * The stage name can start with a digit.
> * The stage name can contain a maximum of 64 characters.
> * isula-build does not support resource restriction on a single Dockerfile build. If resource restriction is required, you can configure a resource limit on isula-builder.
> * Currently, isula-build does not support a remote URL as the data source of the **ADD** instruction in the Dockerfile.
> * The local tar package exported using the **docker-archive** and **oci-archive** types are not compressed, you can manually compress the file as required.

#### image: Viewing Local Persistent Build Images

You can run the `images` command to view the images in the local persistent storage.

```sh
$ sudo isula-build ctr-img images
---------------------------------------  -----------  -----------------  ------------------------  ------------
REPOSITORY                               TAG          IMAGE ID           CREATED                   SIZE
---------------------------------------  -----------  -----------------  ------------------------  ------------
localhost:5000/library/alpine            latest       a24bb4013296       2022-01-17 10:02:19       5.85 MB
<none>                                   <none>       39b62a3342ee       2022-01-17 10:01:12       1.45 MB
---------------------------------------  -----------  -----------------  ------------------------  ------------
```

> \[!NOTE] **Note:**
>
> * The image size displayed by running the `isula-build ctr-img images` command may be different from that displayed by running the `docker images` command. When calculating the image size, `isula-build` directly calculates the total size of .tar packages at each layer, while `docker` calculates the total size of files by decompressing the .tar packages and traversing the diff directory. Therefore, the statistics are different.

#### import: Importing a Basic Container Image

A tar file in rootfs form can be imported into isula-build via the `ctr-img import` command.

The command format is as follows:

```sh
isula-build ctr-img import [flags]
```

Example:

```sh
$ sudo isula-build ctr-img import busybox.tar mybusybox:latest
Getting image source signatures
Copying blob sha256:7b8667757578df68ec57bfc9fb7754801ec87df7de389a24a26a7bf2ebc04d8d
Copying config sha256:173b3cf612f8e1dc34e78772fcf190559533a3b04743287a32d549e3c7d1c1d1
Writing manifest to image destination
Storing signatures
Import success with image id: "173b3cf612f8e1dc34e78772fcf190559533a3b04743287a32d549e3c7d1c1d1"
$ sudo isula-build ctr-img images
---------------------------------------  -----------  -----------------  ------------------------  ------------
REPOSITORY                               TAG          IMAGE ID           CREATED                   SIZE
---------------------------------------  -----------  -----------------  ------------------------  ------------
mybusybox                                latest       173b3cf612f8       2022-01-12 16:02:31       1.47 MB
---------------------------------------  -----------  -----------------  ------------------------  ------------
```

> \[!NOTE] **Note**
>
> * isula-build supports the import of container basic images with a maximum size of 1 GB.

#### load: Importing Cascade Images

Cascade images are images that are saved to the local computer by running the `docker save` or `isula-build ctr-img save` command. The compressed image package contains a layer-by-layer image package named **layer.tar**. You can run the `ctr-img load` command to import the image to isula-build.

The command format is as follows:

```sh
isula-build ctr-img load [flags]
```

Currently, the following flags are supported:

* `-i, --input`: path of the local .tar package.

Example:

```sh
$ sudo isula-build ctr-img load -i ubuntu.tar
Getting image source signatures
Copying blob sha256:cf612f747e0fbcc1674f88712b7bc1cd8b91cf0be8f9e9771235169f139d507c
Copying blob sha256:f934e33a54a60630267df295a5c232ceb15b2938ebb0476364192b1537449093
Copying blob sha256:943edb549a8300092a714190dfe633341c0ffb483784c4fdfe884b9019f6a0b4
Copying blob sha256:e7ebc6e16708285bee3917ae12bf8d172ee0d7684a7830751ab9a1c070e7a125
Copying blob sha256:bf6751561805be7d07d66f6acb2a33e99cf0cc0a20f5fd5d94a3c7f8ae55c2a1
Copying blob sha256:c1bd37d01c89de343d68867518b1155cb297d8e03942066ecb44ae8f46b608a3
Copying blob sha256:a84e57b779297b72428fc7308e63d13b4df99140f78565be92fc9dbe03fc6e69
Copying blob sha256:14dd68f4c7e23d6a2363c2320747ab88986dfd43ba0489d139eeac3ac75323b2
Copying blob sha256:a2092d776649ea2301f60265f378a02405539a2a68093b2612792cc65d00d161
Copying blob sha256:879119e879f682c04d0784c9ae7bc6f421e206b95d20b32ce1cb8a49bfdef202
Copying blob sha256:e615448af51b848ecec00caeaffd1e30e8bf5cffd464747d159f80e346b7a150
Copying blob sha256:f610bd1e9ac6aa9326d61713d552eeefef47d2bd49fc16140aa9bf3db38c30a4
Copying blob sha256:bfe0a1336d031bf5ff3ce381e354be7b2bf310574cc0cd1949ad94dda020cd27
Copying blob sha256:f0f15db85788c1260c6aa8ad225823f45c89700781c4c793361ac5fa58d204c7
Copying config sha256:c07ddb44daa97e9e8d2d68316b296cc9343ab5f3d2babc5e6e03b80cd580478e
Writing manifest to image destination
Storing signatures
Loaded image as c07ddb44daa97e9e8d2d68316b296cc9343ab5f3d2babc5e6e03b80cd580478e
```

> \[!NOTE] **Note:**
>
> * isula-build allows you to import a container image with a maximum size of 50 GB.
> * isula-build automatically recognizes the image format and loads it from the cascade image file.

#### rm: Deleting a Local Persistent Image

You can run the `rm` command to delete an image from the local persistent storage. The command format is as follows:

```sh
isula-build ctr-img rm IMAGE [IMAGE...] [FLAGS]
```

Currently, the following flags are supported:

* `-a, --all`: deletes all images stored locally.
* `-p, --prune`: deletes all images that are stored locally and do not have tags.

Example:

```sh
$ sudo isula-build ctr-img rm -p
Deleted: sha256:78731c1dde25361f539555edaf8f0b24132085b7cab6ecb90de63d72fa00c01d
Deleted: sha256:eeba1bfe9fca569a894d525ed291bdaef389d28a88c288914c1a9db7261ad12c
```

#### save: Exporting Cascade Images

You can run the `save` command to export the cascade images to the local disk. The command format is as follows:

```sh
isula-build ctr-img save [REPOSITORY:TAG]|imageID -o xx.tar
```

Currently, the following flags are supported:

* `-f, --format`: which indicates the exported image format: **oci** or **docker** (**ISULABUILD\_CLI\_EXPERIMENTAL** needs to be enabled)
* `-o, --output`: which indicates the local path for storing the exported images.

The following example shows how to export an image using *image/tag*:

```sh
$ sudo isula-build ctr-img save busybox:latest -o busybox.tar
Getting image source signatures
Copying blob sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a
Copying blob sha256:824082a6864774d5527bda0d3c7ebd5ddc349daadf2aa8f5f305b7a2e439806f
Copying blob sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef
Copying config sha256:21c3e96ac411242a0e876af269c0cbe9d071626bdfb7cc79bfa2ddb9f7a82db6
Writing manifest to image destination
Storing signatures
Save success with image: busybox:latest
```

The following example shows how to export an image using *ImageID*:

```sh
$ sudo isula-build ctr-img save 21c3e96ac411 -o busybox.tar
Getting image source signatures
Copying blob sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a
Copying blob sha256:824082a6864774d5527bda0d3c7ebd5ddc349daadf2aa8f5f305b7a2e439806f
Copying blob sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef
Copying config sha256:21c3e96ac411242a0e876af269c0cbe9d071626bdfb7cc79bfa2ddb9f7a82db6
Writing manifest to image destination
Storing signatures
Save success with image: 21c3e96ac411
```

The following example shows how to export multiple images to the same tarball:

```sh
$ sudo isula-build ctr-img save busybox:latest nginx:latest -o all.tar
Getting image source signatures
Copying blob sha256:eb78099fbf7fdc70c65f286f4edc6659fcda510b3d1cfe1caa6452cc671427bf
Copying blob sha256:29f11c413898c5aad8ed89ad5446e89e439e8cfa217cbb404ef2dbd6e1e8d6a5
Copying blob sha256:af5bd3938f60ece203cd76358d8bde91968e56491daf3030f6415f103de26820
Copying config sha256:b8efb18f159bd948486f18bd8940b56fd2298b438229f5bd2bcf4cedcf037448
Writing manifest to image destination
Storing signatures
Getting image source signatures
Copying blob sha256:e2d6930974a28887b15367769d9666116027c411b7e6c4025f7c850df1e45038
Copying config sha256:a33de3c85292c9e65681c2e19b8298d12087749b71a504a23c576090891eedd6
Writing manifest to image destination
Storing signatures
Save success with image: [busybox:latest nginx:latest]
```

> \[!NOTE] **NOTE:**
>
> * Save exports an image in .tar format by default. If necessary, you can save the image and then manually compress it.
> * When exporting an image using image name, specify the entire image name in the *REPOSITORY:TAG* format.

#### tag: Tagging Local Persistent Images

You can run the `tag` command to add a tag to a local persistent container image. The command format is as follows:

```sh
isula-build ctr-img tag <imageID>/<imageName> busybox:latest
```

Example:

```sh
$ sudo isula-build ctr-img images
---------------------------------------  -----------  -----------------  --------------------------  ------------
REPOSITORY                               TAG          IMAGE ID           CREATED                     SIZE
---------------------------------------  -----------  -----------------  --------------------------  ------------
alpine                                   latest       a24bb4013296       2020-05-29 21:19:46         5.85 MB
---------------------------------------  -----------  -----------------  --------------------------  ------------
$ sudo isula-build ctr-img tag a24bb4013296 alpine:v1
$ sudo isula-build ctr-img images
---------------------------------------  -----------  -----------------  ------------------------  ------------
REPOSITORY                               TAG          IMAGE ID           CREATED                   SIZE
---------------------------------------  -----------  -----------------  ------------------------  ------------
alpine                                   latest       a24bb4013296       2020-05-29 21:19:46       5.85 MB
alpine                                   v1           a24bb4013296       2020-05-29 21:19:46       5.85 MB
---------------------------------------  -----------  -----------------  ------------------------  ------------
```

#### pull: Pulling an Image To a Local Host

Run the `pull` command to pull an image from a remote image repository to a local host. Command format:

```sh
isula-build ctr-img pull REPOSITORY[:TAG]
```

Example:

```sh
$ sudo isula-build ctr-img pull example-registry/library/alpine:latest
Getting image source signatures
Copying blob sha256:8f52abd3da461b2c0c11fda7a1b53413f1a92320eb96525ddf92c0b5cde781ad
Copying config sha256:e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b
Writing manifest to image destination
Storing signatures
Pull success with image: example-registry/library/alpine:latest
```

#### push: Pushing a Local Image to a Remote Repository

Run the `push` command to push a local image to a remote repository. Command format:

```sh
isula-build ctr-img push REPOSITORY[:TAG]
```

Currently, the following flags are supported:

* `-f, --format`: indicates the pushed image format **oci** or **docker** (**ISULABUILD\_CLI\_EXPERIMENTAL** needs to be enabled)

Example:

```sh
$ sudo isula-build ctr-img push example-registry/library/mybusybox:latest
Getting image source signatures
Copying blob sha256:d2421964bad195c959ba147ad21626ccddc73a4f2638664ad1c07bd9df48a675
Copying config sha256:f0b02e9d092d905d0d87a8455a1ae3e9bb47b4aa3dc125125ca5cd10d6441c9f
Writing manifest to image destination
Storing signatures
Push success with image: example-registry/library/mybusybox:latest
```

> \[!NOTE] **NOTE:**
>
> Before pushing an image, log in to the corresponding image repository.

### info: Viewing the Operating Environment and System Information

You can run the `isula-build info` command to view the running environment and system information of isula-build. The command format is as follows:

```sh
 isula-build info [flags]
```

The following flags are supported:

* `-H, --human-readable`: Boolean. The memory information is printed in the common memory format. The value is 1000 power.
* `-V, --verbose`: Boolean. The memory usage is displayed during system running.

Example:

```sh
$ sudo isula-build info -H
   General:
     MemTotal:     7.63 GB
     MemFree:      757 MB
     SwapTotal:    8.3 GB
     SwapFree:     8.25 GB
     OCI Runtime:  runc
     DataRoot:     /var/lib/isula-build/
     RunRoot:      /var/run/isula-build/
     Builders:     0
     Goroutines:   12
   Store:
     Storage Driver:     overlay
     Backing Filesystem: extfs
   Registry:
     Search Registries:
       oepkgs.net
     Insecure Registries:
       localhost:5000
       oepkgs.net
   Runtime:
    MemSys:           68.4 MB
     HeapSys:          63.3 MB
     HeapAlloc:        7.41 MB
     MemHeapInUse:     8.98 MB
     MemHeapIdle:      54.4 MB
     MemHeapReleased:  52.1 MB
```

### login: Logging In to the Remote Image Repository

You can run the `login` command to log in to the remote image repository. The command format is as follows:

```sh
isula-build login SERVER [FLAGS]
```

Currently, the following flags are supported:

```text
Flags:
  -p, --password-stdin    Read password from stdin
  -u, --username string   Username to access registry
```

Enter the password through the standard input. In the following example, the password in **creds.txt** is transferred to the standard input of isula-build through a pipe for input.

```sh
 $ cat creds.txt | sudo isula-build login -u cooper -p mydockerhub.io
 Login Succeeded
```

Enter the password in interactive mode.

```sh
 $ sudo isula-build login mydockerhub.io -u cooper
 Password:
 Login Succeeded
```

### logout: Logging Out of the Remote Image Repository

You can run the `logout` command to log out of the remote image repository. The command format is as follows:

```sh
isula-build logout [SERVER] [FLAGS]
```

Currently, the following flags are supported:

```text
Flags:
   -a, --all   Logout all registries
```

Example:

```sh
 $ sudo isula-build logout -a
   Removed authentications
```

### version: Querying the isula-build Version

You can run the `version` command to view the current version information.

```sh
$ sudo isula-build version
Client:
  Version:       0.9.6-18
  Go Version:    go1.17.3
  Git Commit:    37aa419
  Built:         Mon Jun 26 15:32:55 2023
  OS/Arch:       linux/arm64

Server:
  Version:       0.9.6-18
  Go Version:    go1.17.3
  Git Commit:    37aa419
  Built:         Mon Jun 26 15:32:55 2023
  OS/Arch:       linux/arm64
```

### manifest: Manifest List Management

The manifest list contains the image information corresponding to different system architectures. You can use the same manifest (for example, **openeuler:latest**) in different architectures to obtain the image of the corresponding architecture. The manifest contains the create, annotate, inspect, and push subcommands.

> \[!NOTE] **NOTE:**
>
> manifest is an experiment feature. When using this feature, you need to enable the experiment options on the client and server. For details, see Client Overview and Configuring Services.

#### create: Manifest List Creation

The create subcommand of the `manifest` command is used to create a manifest list. The command format is as follows:

```sh
isula-build manifest create MANIFEST_LIST MANIFEST [MANIFEST...]
```

You can specify the name of the manifest list and the remote images to be added to the list. If no remote image is specified, an empty manifest list is created.

Example:

```sh
sudo isula-build manifest create openeuler localhost:5000/openeuler_x86:latest localhost:5000/openeuler_aarch64:latest
```

#### annotate: Manifest List Update

The `annotate` subcommand of the `manifest` command is used to update the manifest list. The command format is as follows:

```sh
isula-build manifest annotate MANIFEST_LIST MANIFEST [flags]
```

You can specify the manifest list to be updated and the images in the manifest list, and use flags to specify the options to be updated. This command can also be used to add new images to the manifest list.

Currently, the following flags are supported:

* \--arch: Applicable architecture of the rewritten image. The value is a string.
* \--os: Indicates the applicable system of the image. The value is a string.
* \--os-features: Specifies the OS features required by the image. This parameter is a string and rarely used.
* \--variant: Variable of the image recorded in the list. The value is a string.

Example:

```sh
sudo isula-build manifest annotate --os linux --arch arm64 openeuler:latest localhost:5000/openeuler_aarch64:latest
```

#### inspect: Manifest List Inspect

The `inspect` subcommand of the `manifest` command is used to query the manifest list. The command format is as follows:

```sh
isula-build manifest inspect MANIFEST_LIST
```

Example:

```sh
$ sudo isula-build manifest inspect openeuler:latest
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 527,
            "digest": "sha256:bf510723d2cd2d4e3f5ce7e93bf1e52c8fd76831995ac3bd3f90ecc866643aff",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        },
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 527,
            "digest": "sha256:f814888b4bb6149bd39ba8375a1932fb15071b4dbffc7f76c7b602b06abbb820",
            "platform": {
                "architecture": "arm64",
                "os": "linux"
            }
        }
    ]
}
```

#### push: Manifest List Push to the Remote Repository

The manifest subcommand `push` is used to push the manifest list to the remote repository. The command format is as follows:

```sh
isula-build manifest push MANIFEST_LIST DESTINATION
```

Example:

```sh
sudo isula-build manifest push openeuler:latest localhost:5000/openeuler:latest
```

## Directly Integrating a Container Engine

isula-build can be integrated with iSulad or Docker to import the built container image to the local storage of the container engine.

### Integration with iSulad

Images that are successfully built can be directly exported to the iSulad.

Example:

```sh
sudo isula-build ctr-img build -f Dockerfile -o isulad:busybox:2.0
```

Specify iSulad in the -o parameter to export the built container image to iSulad. You can query the image using isula images.

```sh
$ sudo isula images
isula images
REPOSITORY                     TAG        IMAGE ID             CREATED              SIZE
busybox                        2.0        2d414a5cad6d         2020-08-01 06:41:36  5.577 MB
```

> \[!NOTE] **Note:**
>
> * It is required that isula-build and iSulad be on the same node.
> * When an image is directly exported to the iSulad, the isula-build client needs to temporarily store the successfully built image as `/var/lib/isula-build/tmp/[buildid]/isula-build-tmp-%v.tar` and then import it to the iSulad. Ensure that the /var/tmp/ directory has sufficient disk space. If the isula-build client process is killed or Ctrl+C is pressed during the export, you need to manually clear the `/var/lib/isula-build/tmp/[buildid]/isula-build-tmp-%v.tar` file.

### Integration with Docker

Images that are successfully built can be directly exported to the Docker daemon.

Example:

```sh
sudo isula-build ctr-img build -f Dockerfile -o docker-daemon:busybox:2.0
```

Specify docker-daemon in the -o parameter to export the built container image to Docker. You can run the `docker images` command to query the image.

```sh
$ sudo docker images
REPOSITORY                                          TAG                 IMAGE ID            CREATED             SIZE
busybox                                             2.0                 2d414a5cad6d        2 months ago        5.22MB
```

> \[!NOTE] **Note:**
>
> isula-build and Docker must be on the same node.

## Precautions

This chapter is something about constraints, limitations and differences with `docker build` when you use isula-builder to build images.

### Constraints or Limitations

1. When export an image to iSulad, a tag is necessary.
2. Because the OCI runtime, for example, **runc**, will be called by isula-builder when executing the **RUN** instruction, the integrity of the runtime binary should be guaranteed by the user.
3. DataRoot should not be set to **tmpfs**.
4. **Overlay2** is the only storage driver supported by isula-builder currently.
5. Docker image is the only image format supported by isula-builder currently.
6. You are advised to set file permission of the Dockerfile to **0600** to avoid tampering by other users.
7. Only host network is supported by the **RUN** instruction currently.
8. When export image to a tar package, only tar compression format is supported by isula-builder currently.
9. The base image size is limited to 1 GB when importing a base image using `import`.

### Differences with `docker build`

The `isula-build` complies with [Dockerfile specification](https://docs.docker.com/engine/reference/builder/), but there are also some subtle differences between `isula-builder` and `docker build` as follows:

1. isula-builder commits after each build stage, but not every line.
2. Build cache is not supported by isula-builder.
3. Only **RUN** instruction will be executed in the build container.
4. Build history is not supported currently.
5. Stage name can be start with a number.
6. The length of the stage name is limited to 64 in `isula-builder`.
7. **ADD** instruction source can not be a remote URL currently.
8. Resource restriction on a single build is not supported. If resource restriction is required, you can configure a resource limit on isula-builder.
9. `isula-builder` add each origin layer tar size to get the image size, but docker only uses the diff content of each layer. So the image size listed by `isula-builder images` is different.
10. Image name should be in the *NAME:TAG* format. For example **busybox:latest**, where **latest** must not be omitted.

## Appendix

### Command Line Parameters

**Table 1** Parameters of the `ctr-img build` command

| **Command** | **Parameter** | **Description** |
| ------------- | -------------- | ------------------------------------------------------------ |
| ctr-img build | --build-arg | String list, which contains variables required during the build. |
| | --build-static | Key value, which is used to build binary equivalence. Currently, the following key values are included: - build-time: string, which indicates that a fixed timestamp is used to build a container image. The timestamp format is YYYY-MM-DD HH-MM-SS. |
| | -f, --filename | String, which indicates the path of the Dockerfiles. If this parameter is not specified, the current path is used. |
| | --format | String, which indicates the image format **oci** or **docker** (**ISULABUILD\_CLI\_EXPERIMENTAL** needs to be enabled). |
| | --iidfile | String, which indicates the ID of the image output to a local file. |
| | -o, --output | String, which indicates the image export mode and path.|
| | --proxy | Boolean, which inherits the proxy environment variable on the host. The default value is true. |
| | --tag | String, which indicates the tag value of the image that is successfully built. |
| | --cap-add | String list, which contains permissions required by the **RUN** instruction during the build process.|

**Table 2** Parameters of the `ctr-img load` command

| **Command** | **Parameter** | **Description** |
| ------------ | ----------- | --------------------------------- |
| ctr-img load | -i, --input | String, path of the local .tar package to be imported.|

**Table 3** Parameters of the `ctr-img push` command

| **Command** | **Parameter** | **Description** |
| ------------ | ----------- | --------------------------------- |
| ctr-img push | -f, --format | String, which indicates the pushed image format **oci** or **docker** (**ISULABUILD\_CLI\_EXPERIMENTAL** needs to be enabled).|

**Table 4** Parameters of the `ctr-img rm` command

| **Command** | **Parameter** | **Description** |
| ---------- | ----------- | --------------------------------------------- |
| ctr-img rm | -a, --all | Boolean, which is used to delete all local persistent images. |
| | -p, --prune | Boolean, which is used to delete all images that are stored persistently on the local host and do not have tags. |

**Table 5** Parameters of the `ctr-img save` command

| **Command** | **Parameter** | **Description** |
| ------------ | ------------ | ---------------------------------- |
| ctr-img save | -o, --output | String, which indicates the local path for storing the exported images.|
| ctr-img save | -f, --format | String, which indicates the exported image format **oci** or **docker** (**ISULABUILD\_CLI\_EXPERIMENTAL** needs to be enabled).|

**Table 6** Parameters of the `login` command

| **Command** | **Parameter** | **Description** |
| -------- | -------------------- | ------------------------------------------------------- |
| login | -p, --password-stdin | Boolean, which indicates whether to read the password through the standard input. or enter the password in interactive mode. |
| | -u, --username | String, which indicates the username for logging in to the image repository.|

**Table 7** Parameters of the `logout` command

| **Command** | **Parameter** | **Description** |
| -------- | --------- | ------------------------------------ |
| logout | -a, --all | Boolean, which indicates whether to log out of all logged-in image repositories. |

**Table 8** Parameters of the `manifest annotate` command

| **Command**       | **Parameter** | **Description**              |
| ----------------- | ------------- | ---------------------------- |
| manifest annotate | --arch        | Set architecture             |
|                   | --os          | Set operating system         |
|                   | --os-features | Set operating system feature |
|                   | --variant     | Set architecture variant     |

### Communication Matrix

The isula-build component processes communicate with each other through the Unix socket file. No port is used for communication.

### File and Permission

* All isula-build operations must be performed by the **root** user. To perform operations as a non-privileged user, you need to configure the `--group` option.

* The following table lists the file permissions involved in the running of isula-build.

| **File Path** | **File/Folder Permission** | **Description** |
| ------------------------------------------- | ------------------- | ------------------------------------------------------------ |
| /usr/bin/isula-build                        | 550                 | Binary file of the command line tool.                                       |
| /usr/bin/isula-builder                      | 550                 | Binary file of the isula-builder process.                          |
| /usr/lib/systemd/system/isula-build.service | 640                 | systemd configuration file, which is used to manage the isula-build service.                   |
| /usr/isula-build                            | 650                 | Root directory of the isula-builder configuration file. |
| /etc/isula-build/configuration.toml         | 600                 | General isula-builder configuration file, including the settings of the isula-builder log level, persistency directory, runtime directory, and OCI runtime. |
| /etc/isula-build/policy.json                | 600                 | Syntax file of the signature verification policy file.                                 |
| /etc/isula-build/registries.toml            | 600                 | Configuration file of each image repository, including the available image repository list and image repository blacklist. |
| /etc/isula-build/storage.toml               | 600                 | Configuration file of the local persistent storage, including the configuration of the used storage driver.       |
| /etc/isula-build/isula-build.pub            | 400                 | Asymmetric encryption public key file. |
| /var/run/isula\_build.sock                   | 660                 | Local socket of isula-builder.                            |
| /var/lib/isula-build                        | 700                 | Local persistency directory.                                             |
| /var/run/isula-build                        | 700                 | Local runtime directory.                                             |
| /var/lib/isula-build/tmp/\[buildid]/isula-build-tmp-\*.tar              | 644                 | Local temporary directory for storing the images when they are exported to iSulad.                           |

---

---
url: /en/docs/22.03_LTS_SP4/cloud/image_builder/isula_build/overview.md
---
# Container Image Building

## Overview

isula-build is a container image build tool developed by the iSula container team. It allows you to quickly build container images using Dockerfiles.

The isula-build uses the server/client mode. The isula-build functions as a client and provides a group of command line tools for image build and management. The isula-builder functions as the server to process client management requests, and runs as a daemon process in the background.

![isula-build architecture](./figures/isula-build_arch.png)

> \[!NOTE]Note
>
> Currently, isula-build supports OCI image format ([OCI Image Format Specification](https://github.com/opencontainers/image-spec/blob/main/spec.md/)) and Docker image format ([Image Manifest Version 2, Schema 2](https://docs.docker.com/registry/spec/manifest-v2-2/)). Use the `export ISULABUILD_CLI_EXPERIMENTAL=enabled` command to enable the experimental feature for supporting OCI image format. When the experimental feature is disabled, isula-build will take Docker image format as the default image format. Otherwise, isula-build will take OCI image format as the default image format.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/container_management_1.md
---
# Container Management

## Creating a Container

### Downloading Images

Only user  **root**  can run the  **docker**  command. If you log in as a common user, you need to use the  **sudo**  command before running the  **docker**  command.

```console
[root@localhost ~]# docker pull busybox
```

This command is used to download the  **busybox:latest**  image from the official Docker registry. (If no tag is specified in the command, the default tag name  **latest**  is used.) During the image download, the system checks whether the dependent layer exists locally. If yes, the image download is skipped. When downloading images from a private registry, specify the registry description. For example, if a private registry containing some common images is created and its IP address is  **192.168.1.110:5000**, you can run the following command to download the image from the private registry:

```console
[root@localhost ~]# docker pull 192.168.1.110:5000/busybox
```

The name of the image downloaded from the private registry contains the registry address information, which may be too long. Run the  **docker tag**  command to generate an image with a shorter name.

```console
[root@localhost ~]# docker tag 192.168.1.110:5000/busybox busybox
```

Run the  **docker images**  command to view the local image list.

### Running a Simple Application

```console
[root@localhost ~]# docker run busybox /bin/echo "Hello world"
Hello world
```

This command uses the  **busybox:latest**  image to create a container, and executes the  **echo "Hello world"**  command in the container. Run the following command to view the created container:

```console
[root@localhost ~]# docker ps -l
CONTAINER ID        IMAGE               COMMAND                   CREATED             STATUS                     PORTS               NAMES
d8c0a3315bc0        busybox"/bin/echo 'Hello wo..."   5 seconds ago       Exited (0) 3 seconds ago                       practical_franklin
```

### Creating an Interactive Container

```console
[root@localhost ~]# docker run -it busybox /bin/bash
root@bf22919af2cf:/# ls 
bin  boot  dev  etc  home  lib  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var 
root@bf22919af2cf:/# pwd 
/
```

The  **-ti**  option allocates a pseudo terminal to the container and uses standard input (STDIN) for interaction. You can run commands in the container. In this case, the container is an independent Linux VM. Run the  **exit**  command to exit the container.

### Running a Container in the Background

Run the following command.  **-d**  indicates that the container is running in the background.  **--name=container1**  indicates that the container name is  **container1**.

```console
[root@localhost ~]# docker run -d --name=container1 busybox /bin/sh -c "while true;do echo hello world;sleep 1;done"
7804d3e16d69b41aac5f9bf20d5f263e2da081b1de50044105b1e3f536b6db1c
```

The command output contains the container ID but does not contain  **hello world**. In this case, the container is running in the background. You can run the  **docker ps**  command to view the running container.

```console
[root@localhost ~]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
7804d3e16d69        busybox "/bin/sh -c 'while tr"   11 seconds ago      Up 10 seconds                           container1
```

Run the following  **docker logs**  command to view the output during container running:

```console
[root@localhost ~]# docker logs container1
hello world
hello world
hello world
...
```

### Container Network Connection

By default, a container can access an external network, while port mapping is required when an external network accesses a container. The following uses how to run the private registry service in Docker as an example. In the following command,  **-P**  is used to expose open ports in the registry to the host.

```console
[root@localhost ~]# docker run --name=container_registry -d -P registry 
cb883f6216c2b08a8c439b3957fb396c847a99079448ca741cc90724de4e4731 
```

The container\_registry container has been started, but the mapping between services in the container and ports on the host is not clear. You need to run the  **docker port**  command to view the port mapping.

```console
[root@localhost ~]# docker port container_registry 
5000/tcp -> 0.0.0.0:49155 
```

The command output shows that port 5000 in the container is mapped to port 49155 on the host. You can access the registry service by using the host IP address  **49155**. Enter  **<http://localhost:49155>**  in the address box of the browser and press  **Enter**. The registry version information is displayed.

When running registry images, you can directly specify the port mapping, as shown in the following:

```shell
docker run --name=container_registry -d -p 5000:5000 registry 
```

**-p 5000:5000**  is used to map port 5000 in the container to port 5000 on the host.

### Precautions

* **Do Not Add -a stdin Independently During Container Startup**

  When starting a container, you must add  **-a stdout**  or  **-a stderr**  together with  **-a stdin**  instead of  **-a stdin**  only. Otherwise, the device stops responding even after the container exits.

* **Do Not Use the Long Or Short ID of an Existing Container As the Name of a New Container**

  When creating a container, do not use the long or short ID of the existing container A as the name of the new container B. If the long ID of container A is used as the name of container B, Docker will match container A even though the name of container B is used as the specified target container for operations. If the short ID of container A is used as the name of container B, Docker will match container B even though the short ID of container A is used as the specified target container for operations. This is because Docker matches the long IDs of all containers first. If the matching fails, the system performs exact matching using the value of  **container\_name**. If matching failure persists, the container ID is directly matched in fuzzy mode.

* **Containers That Depend on Standard Input and Output, Such As sh/bash, Must Use the -ti Parameter to Avoid Exceptions**

  Normal case: If you do not use the  **-ti**  parameter to start a process container such as sh/bash, the container exits immediately.

  The cause of this problem is that Docker creates a stdin that matches services in the container first. If the interactive parameters such as  **-ti**  are not set, Docker closes pipe after the container is started and the service container process sh/bash exits after stdin is closed.

  Exception: If Docker daemon is forcibly killed in a specific phase (before pipe is closed), daemon of the pipe is not closed in time. In this case, the sh/bash process does not exit even without  **-ti**. As a result, an exception occurs. You need to manually clear the container.

  After being restarted, daemon takes over the original container stream. Containers without the  **-ti**  parameter may not be able to process the stream because these containers do not have streams to be taken over in normal cases. In actual services, sh/bash without the  **-ti**  parameter does not take effect and is seldom used. To avoid this problem, the  **-ti**  parameter is used to restrict interactive containers.

* **Container Storage Volumes**

  If you use the  **-v**  parameter to mount files on the host to a container when the container is started, the inodes of the files may be changed when you run the  **vi**  or  **sed**  command to modify the files on the host or in the container. As a result, files on the host and in the container are not synchronized. Do not mount files in the container in this mode (or do not use together with the  **vi**  and  **sed**  commands). You can also mount the upper-layer directories of the files to avoid exceptions. The  **nocopy**  option can be used to prevent original files in the mount point directory of a container from being copied to the source directory of the host when Docker mounts volumes. However, this option can be used only when an anonymous volume is mounted and cannot be used in the bind mount scenario.

* **Do Not Use Options That May Affect the Host**

  The  **--privileged**  option enables all permissions for a container. On the container, mounting operations can be performed and directories such as  **/proc**  and  **/sys**  can be modified, which may affect the host. Therefore, do not use this option for common containers.

  A host-shared namespace, such as the  **--pid host**,  **--ipc host**, or  **--net host**  option, can enable a container to share the namespace with the host, which will also affect the host. Therefore, do not use this option.

* **Do Not Use the Unstable Kernel Memory Cgroup**

  Kernel memory cgroup on the Linux kernel earlier than 4.0 is still in the experimental phase and runs unstably. Therefore, do not use kernel memory cgroup.

  When the  **docker run --kernel-memory**  command is executed, the following alarm is generated:

  ```console
  WARNING: You specified a kernel memory limit on a kernel older than 4.0. Kernel memory limits are experimental on older kernels, it won't work as expected as expected and can cause your system to be unstable.
  ```

* **blkio-weight Parameter Is Unavailable in the Kernel That Supports blkio Precise Control**

  **--blkio-weight-device**  can implement more accurate blkio control in a container. The control requires a specified disk device, which can be implemented through the  **--blkio-weight-device**  parameter of Docker. In this kernel, Docker does not provide the  **--blkio-weight**  mode to limit the container blkio. If you use this parameter to create a container, the following error is reported:

  ```console
  docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:398: container init caused \"process_linux.go:369: setting cgroup config for ready process caused \\\"blkio.weight not supported, use weight_device instead\\\"\""
  ```

* **Using --blkio-weight-device in CFQ Scheduling Policy**

  The  **--blkio-weight-device**  parameter works only when the disk works in the Completely Fair Queuing (CFQ) policy.

  You can view the scheduler file (**/sys/block/***disk***/queue/scheduler**) to obtain the policies supported by the disk and the current policy. For example, you can run the following command to view  **sda**.

  ```shell
  cat /sys/block/sda/queue/scheduler noop [deadline] cfq 
  ```

  **sda**  supports the following scheduling policies:  **noop**,  **deadline**, and  **cfq**, and the  **deadline**  policy is being used. You can run the  **echo**  command to change the policy to  **cfq**.

  ```shell
  echo cfq > /sys/block/sda/queue/scheduler
  ```

* **systemd Usage Restrictions in Basic Container Images**

  When containers created from basic images are used, systemd in basic images is used only for system containers.

### Concurrent Performance

* There is an upper limit for the message buffer in Docker. If the number of messages exceeds the upper limit, the messages are discarded. Therefore, it is recommended that the number of commands executed concurrently should not exceed 1000. Otherwise, the internal messages in Docker may be lost and the container may fail to be started.
* When containers are concurrently created and restarted, the error message"oci runtime error: container init still running" is occasionally reported. This is because containerd optimizes the performance of the event waiting queue. When a container is stopped, the  **runc delete**  command is executed to kill the init processes in the container within 1s. If the init processes are not killed within 1s, runC returns this error message. The garbage collection (GC) mechanism of containerd reclaims residual resources after  **runc delete**  is executed at an interval of 10s. Therefore, operations on the container are not affected. If the preceding error occurs, wait for 4 or 5s and restart the container.

### Security Feature Interpretation

1. The following describes default permission configuration analysis of Docker.

   In the default configuration of a native Docker, capabilities carried by each default process are as follows:

   ```text
   "CAP_CHOWN", 
   "CAP_DAC_OVERRIDE", 
   "CAP_FSETID", 
   "CAP_FOWNER", 
   "CAP_MKNOD", 
   "CAP_NET_RAW", 
   "CAP_SETGID", 
   "CAP_SETUID", 
   "CAP_SETFCAP", 
   "CAP_SETPCAP", 
   "CAP_NET_BIND_SERVICE", 
   "CAP_SYS_CHROOT", 
   "CAP_KILL", 
   "CAP_AUDIT_WRITE",
   ```

   The default seccomp configuration is a whitelist. If any syscall is not in the whitelist,  **SCMP\_ACT\_ERRNO**  is returned by default. Different system invoking is enabled for different caps of Docker. If a capability is not in the whitelist, Docker will not assign it to the container by default.

2. CAP\_SYS\_MODULE

   CAP\_SYS\_MODULE allows a container to insert or remove ko modules. Adding this capability allows the container to escape or even damage the kernel. Namespace provides the maximum isolation for a container. In the ko module, you only need to point its namespace to  **init\_nsproxy**.

3. CAP\_SYS\_ADMIN

   The sys\_admin permission provides the following capabilities for a container:

   * For file system:  **mount**,  **umount**, and  **quotactl**
   * For namespace setting:  **setns**,  **unshare**, and  **clone new namespace**
   * driver ioctl
   * For PCI control:  **pciconfig\_read**,  **pciconfig\_write**, and  **pciconfig\_iobase**
   * **sethostname**

4. CAP\_NET\_ADMIN

   CAP\_NET\_ADMIN allows a container to access network interfaces and sniff network traffic. The container can obtain the network traffic of all containers including the host, which greatly damages network isolation.

5. CAP\_DAC\_READ\_SEARCH

   CAP\_DAC\_READ\_SEARCH calls the open\_by\_handle\_at and name\_to\_handle\_at system calls. If the host is not protected by SELinux, the container can perform brute-force search for the inode number of the file\_handle structure to open any file on the host, which affects the isolation of the file system.

6. CAP\_SYS\_RAWIO

   CAP\_SYS\_RAWIO allows a container to write I/O ports to the host, which may cause the host kernel to crash.

7. CAP\_SYS\_PTRACE

   The ptrace permission for a container provides ptrace process debugging in the container. RunC has fixed this vulnerability. However, some tools, such as nsenter and docker-enter, are not protected. In a container, processes executed by these tools can be debugged to obtain resource information (such as namespace and fd) brought by these tools. In addition, ptrace can bypass seccomp, greatly increasing attack risks of the kernel.

8. Docker capability interface: --cap-add all

   \--cap-add all grants all permissions to a container, including the dangerous permissions mentioned in this section, which allows the container to escape.

9. Do not disable the seccomp feature of Docker.

   Docker has a default seccomp configuration with a whitelist.  **sys\_call**  that is not in the whitelist is disabled by seccomp. You can disable the seccomp feature by running  **--security-opt 'seccomp:unconfined'**. If seccomp is disabled or the user-defined seccomp configuration is used but the filtering list is incomplete, attack risks of the kernel in the container are increased.

10. Do not set the  **/sys**  and  **/proc**  directories to writable.

    The  **/sys**  and  **/proc**  directories contain Linux kernel maintenance parameters and device management interfaces. If the write permission is configured for the directories in a container, the container may escape.

11. Docker open capability: --CAP\_AUDIT\_CONTROL

    The permission allows a container to control the audit system and run the  **AUDIT\_TTY\_GET**  and  **AUDIT\_TTY\_SET**  commands to obtain the TTY execution records (including the  **root**  password) recorded in the audit system.

12. CAP\_BLOCK\_SUSPEND and CAP\_WAKE\_ALARM

    The permission provides a container the capability to block the system from suspending (epoll).

13. CAP\_IPC\_LOCK

    With this permission, a container can break the max locked memory limit in  **ulimit**  and use any mlock large memory block to cause DoS attacks.

14. CAP\_SYS\_LOG

    In a container with this permission, system kernel logs can be read by using dmesg to break through kernel kaslr protection.

15. CAP\_SYS\_NICE

    In a container with this permission, the scheduling policy and priority of a process can be changed, causing DoS attacks.

16. CAP\_SYS\_RESOURCE

    With this permission, a container can bypass resource restrictions, such as disk space resource restriction, keymaps quantity restriction, and  **pipe-size-max**  restriction, causing DoS attacks.

17. CAP\_SYS\_TIME

    In a container with this capability, the time on the host can be changed.

18. Risk analysis of Docker default capabilities

    The default capabilities of Docker include CAP\_SETUID and CAP\_FSETID. If the host and a container share a directory, the container can set permissions for the binary file in the shared directory. Common users on the host can use this method to elevate privileges. With the CAP\_AUDIT\_WRITE capability, a container can write logs to the host, and the host must be configured with log anti-explosion measures.

19. Docker and host share namespace parameters, such as  **--pid**,  **--ipc**, and  **--uts**.

    This parameter indicates that the container and host share the namespace. The container can attack the host as the namespace of the container is not isolated from that of the host. For example, if you use  **--pid**  to share PID namespace with the host, the PID on the host can be viewed in the container, and processes on the host can be killed at will.

20. **--device**  is used to map the sensitive directories or devices of the host to the container.

    The Docker management plane provides interfaces for mapping directories or devices on a host to the container, such as  **--device**  and  **-v**. Do not map sensitive directories or devices on the host to the container.

## Creating Containers Using hook-spec

### Principles and Application Scenarios

Docker supports the extended features of hooks. The execution of hook applications and underlying runC complies with the [OCI standards](https://github.com/opencontainers/runtime-spec/blob/main/config.md/#hooks).

There are three types of hooks: prestart, poststart, and poststop. They are respectively used before applications in the container are started, after the applications are started, and after the applications are stopped.

### API Reference

The  **--hook-spec**  parameter is added to the  **docker run**  and  **create**  commands and is followed by the absolute path of the  **spec**  file. You can specify the hooks to be added during container startup. These hooks will be automatically appended after the hooks that are dynamically created by Docker (currently only libnetwork prestart hook) to execute programs specified by users during the container startup or destruction.

The structure of  **spec**  is defined as follows:

```text
// Hook specifies a command that is run at a particular event in the lifecycle of a container
type Hook struct{       
               Path    string   `json:"path"`    
               Args    []string `json:"args,omitempty"`    
               Env     []string `json:"env,omitempty"`      
               Timeout *int     `json:"timeout,omitempty"`
}
// Hooks for container setup and teardown
type  Hooks struct{
               // Prestart is a list of hooks to be run before the container process is executed.
               // On Linux, they are run after the container namespaces are created.         
               Prestart []Hook `json:"prestart,omitempty"`
               // Poststart is a list of hooks to be run after the container process is started.         
               Poststart []Hook `json:"poststart,omitempty"`
               // Poststop is a list of hooks to be run after the container process exits.         
               Poststop []Hook `json:"poststop,omitempty"`
}
```

* The  **Path**,  **Args**, and  **Env**  parameters are mandatory.
* **Timeout**  is optional, while you are advised to set this parameter to a value ranging from 1 to 120. The parameter type is int. Floating point numbers are not allowed.
* The content of the  **spec**  file must be in JSON format as shown in the preceding example. If the format is incorrect, an error is reported.
* Both  **docker run --hook-spec /tmp/hookspec.json***xxx*, and  **docker create --hook-spec /tmp/hookspec.json***xxx***&& docker start***xxx*  can be used.

### Customizing Hooks for a Container

Take adding a NIC during the startup as an example. The content of the  **hook spec**  file is as follows:

```json
{
    "prestart": [
         {
             "path": "/var/lib/docker/hooks/network-hook",             
             "args": ["network-hook", "tap0", "myTap"],             
             "env": [],
             "timeout": 5
         }
     ],
     "poststart":[],     
     "poststop":[]
}
```

Specify prestart hook to add the configuration of a network hook. The path is  **/var/lib/docker/hooks/network-hook**.  **args**  indicates the program parameters. Generally, the first parameter is the program name, and the second parameter is the parameter accepted by the program. For the network-hook program, two parameters are required. One is the name of the NIC on the host, and the other is the name of the NIC in the container.

* Precautions
  1. The  **hook**  path must be in the**hooks**  folder in the  **graph**  directory (**--graph**) of Docker. Its default value is  **/var/lib/docker/hooks**. You can run the  **docker info**  command to view the root path.

     ```console
     [root@localhost ~]# docker info 
     ... 
     Docker Root Dir: /var/lib/docker 
     ...
     ```

     This path may change due to the user's manual configuration and the use of user namespaces (**daemon --userns-remap**). After the symbolic link of the path is parsed, the parsed path must start with  *Docker Root Dir***/hooks**  (for example,  **/var/lib/docker/hooks**). Otherwise, an error message is displayed.

  2. The  **hook**  path must be an absolute path because daemon cannot properly process a relative path. In addition, an absolute path meets security requirements.

  3. The information output by the hook program to stderr is output to the client and affects the container lifecycle (for example, the container may fail to be started). The information output to stdout is ignored.

  4. Do not reversely call Docker instructions in hooks.

  5. The execute permission must have been granted on the configured hook execution file. Otherwise, an error is reported during hook execution.

  6. The execution time of the hook operation must be as short as possible. If the prestart period is too long (more than 2 minutes), the container startup times out. If the poststop period is too long (more than 2 minutes), the container is abnormal.

     The known exceptions are as follows: When the  **docker stop**  command is executed to stop a container and the clearing operation is performed after 2 minutes, the hook operation is not complete. Therefore, the system waits until the hook operation is complete (the process holds a lock). As a result, all operations related to the container stop responding. The operations can be recovered only after the hook operation is complete. In addition, the two-minute timeout processing of the  **docker stop**  command is an asynchronous process. Therefore, even if the  **docker stop**  command is successfully executed, the container status is still  **up**. The container status is changed to  **exited**  only after the hook operation is completed.

* Suggestions
  1. You are advised to set the hook timeout threshold to a value less than 5s.
  2. You are advised to configure only one prestart hook, one poststart hook, and one poststop hook for each container. If too many hooks are configured, the container startup may take a long time.
  3. You are advised to identify the dependencies between multiple hooks. If required, you need to adjust the sequence of the hook configuration files according to the dependencies. The execution sequence of hooks is based on the sequence in the configured  **spec**  file.

### Multiple  **hook-spec**

If multiple hook configuration files are available and you need to run multiple hooks, you must manually combine these files into a configuration file and specify the new configuration file by using the  **--hook-spec**  parameter. Then all hooks can take effect. If multiple  **--hook-spec**  parameters are configured, only the last one takes effect.

Configuration examples:

The content of the  **hook1.json**  file is as follows:

```shell
$ cat /var/lib/docker/hooks/hookspec.json 
{
    "prestart": [
        {
            "path": "/var/lib/docker/hooks/lxcfs-hook",             
            "args": ["lxcfs-hook", "--log", "/var/log/lxcfs-hook.log"],             
            "env": []
        }
     ],     
     "poststart":[],     
     "poststop":[]
}
```

The content of the  **hook2.json**  file is as follows:

```shell
$ cat /etc/isulad-tools/hookspec.json 
{
      "prestart": [
         {
               "path": "/docker-root/hooks/docker-hooks",             
               "args": ["docker-hooks", "--state", "prestart"],             
               "env": []
         }
       ],     
       "poststart":[],     
       "poststop":[
          {
               "path": "/docker-root/hooks/docker-hooks",             
               "args": ["docker-hooks", "--state", "poststop"],             
               "env": []
          }
        ]
}
```

The content in JSON format after manual combination is as follows:

```json
{
       "prestart":[
          {
                "path": "/var/lib/docker/hooks/lxcfs-hook",             
                "args": ["lxcfs-hook", "--log", "/var/log/lxcfs-hook.log"],             
                "env": []
           },         
           {
                "path": "/docker-root/hooks/docker-hooks",             
                "args": ["docker-hooks", "--state", "prestart"],             
                "env": []
           }
        ],     
        "poststart":[],     
        "poststop":[
            {
                "path": "/docker-root/hooks/docker-hooks",             
                "args": ["docker-hooks", "--state", "poststop"],             
                "env": []
            }
         ]
}
```

Docker daemon reads the binary values of hook in actions such as prestart in the hook configuration files in sequence based on the array sequence and executes the actions. Therefore, you need to identify the dependencies between multiple hooks. If required, you need to adjust the sequence of the hook configuration files according to the dependencies.

### Customizing Default Hooks for All Containers

Docker daemon can receive the  **--hook-spec**  parameter. The semantics of  **--hook-spec**  is the same as that of  **--hook-spec**  in  **docker create**  or  **docker run**. You can also add hook configurations to the  **/etc/docker/daemon.json**  file.

```json
{
     "hook-spec": "/tmp/hookspec.json"
}
```

When a container is running, hooks specified in  **--hook-spec**  defined by daemon are executed first, and then hooks customized for each container are executed.

## Configuring Health Check During Container Creation

Docker provides the user-defined health check function for containers. You can configure the  **HEALTHCHECK CMD**  option in the Dockerfile, or configure the  **--health-cmd**  option when a container is created so that commands are periodically executed in the container to monitor the health status of the container based on return values.

### Configuration Methods

* Add the following configurations to the Dockerfile file:

  ```text
  HEALTHCHECK --interval=5m --timeout=3s --health-exit-on-unhealthy=true \
     CMD curl -f http://localhost/ || exit 1
  ```

  The configurable options are as follows:

  1. **--interval=DURATION**: interval between two consecutive command executions. The default value is  **30s**. After a container is started, the first check is performed after the interval time.
  2. **--timeout=DURATION**: maximum duration for executing a single check command. If the execution times out, the command execution fails. The default value is  **30s**.
  3. **--start-period=DURATION**: container initialization period. The default value is  **0s**. During the initialization, the health check is also performed, while the health check failure is not counted into the maximum number of retries. However, if the health check is successful during initialization, the container is considered as started. All subsequent consecutive check failures are counted in the maximum number of retries.
  4. **--retries=N**. maximum number of retries for the health check. The default value is  **3**.
  5. **--health-exit-on-unhealthy=BOOLEAN**: whether to kill a container when it is unhealthy. The default value is  **false**.
  6. **CMD**: This option is mandatory. If  **0**  is returned after a command is run in a container, the command execution succeeds. If a value other than  **0**  is returned, the command execution fails.

     After  **HEALTHCHECK**  is configured, related configurations are written into the image configurations during image creation. You can run the  **docker inspect**  command to view the configurations. For example:

     ```json
     "Healthcheck": {
         "Test": [
             "CMD-SHELL",
             "/test.sh"
         ]
     },
     ```

* Configurations during container creation:

  ```shell
  docker run -itd --health-cmd "curl -f http://localhost/ || exit 1" --health-interval 5m --health-timeout 3s --health-exit-on-unhealthy centos bash
  ```

  The configurable options are as follows:

  1. **--health-cmd**: This option is mandatory. If  **0**  is returned after a command is run in a container, the command execution succeeds. If a value other than  **0**  is returned, the command execution fails.
  2. **--health-interval**: interval between two consecutive command executions. The default value is  **30s**. The upper limit of the value is the maximum value of Int64 (unit: nanosecond).
  3. **--health-timeout**: maximum duration for executing a single check command. If the execution times out, the command execution fails. The default value is  **30s**. The upper limit of the value is the maximum value of Int64 (unit: nanosecond).
  4. **--health-start-period**: container initialization time. The default value is  **0s**. The upper limit of the value is the maximum value of Int64 (unit: nanosecond).
  5. **--health-retries**: maximum number of retries for the health check. The default value is  **3**. The maximum value is the maximum value of Int32.
  6. **--health-exit-on-unhealthy**: specifies whether to kill a container when it is unhealthy. The default value is  **false**.

     After the container is started, the  **HEALTHCHECK**  configurations are written into the container configurations. You can run the  **docker inspect**  command to view the configurations. For example:

     ```json
     "Healthcheck": {
         "Test": [
             "CMD-SHELL",
             "/test.sh"
         ]
     },
     ```

### Check Rules

1. After a container is started, the container status is  **health:starting**.
2. After the period specified by  **start-period**, the  **cmd**  command is periodically executed in the container at the interval specified by  **interval**. That is, after the command is executed, the command will be executed again after the specified period.
3. If the  **cmd**  command is successfully executed within the time specified by  **timeout**  and the return value is  **0**, the check is successful. Otherwise, the check fails. If the check is successful, the container status changes to  **health:healthy**.
4. If the  **cmd**  command fails to be executed for the number of times specified by  **retries**, the container status changes to  **health:unhealthy**, and the container continues the health check.
5. When the container status is  **health:unhealthy**, the container status changes to  **health:healthy**  if a check succeeds.
6. If  **--health-exit-on-unhealthy**  is set, and the container exits due to reasons other than being killed (the returned exit code is  **137**), the health check takes effect only after the container is restarted.
7. When the  **cmd**  command execution is complete or times out, Docker daemon will record the start time, return value, and standard output of the check to the configuration file of the container. A maximum of five latest records can be recorded. In addition, the configuration file of the container stores health check parameters.

Run the  **docker ps**  command to view the container status.

```console
[root@bac shm]# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                         PORTS               NAMES
7de2228674a2        testimg             "bash"              About an hour ago   Up About an hour (unhealthy)                       cocky_davinci
```

When the container is running, the health check status is written into the container configurations. You can run the  **docker inspect**  command to view the configurations.

```json
"Health": {
    "Status": "healthy",
    "FailingStreak": 0,
    "Log": [
        {
            "Start": "2018-03-07T07:44:15.481414707-05:00",
            "End": "2018-03-07T07:44:15.556908311-05:00",
            "ExitCode": 0,
            "Output": ""
        },
        {
            "Start": "2018-03-07T07:44:18.557297462-05:00",
            "End": "2018-03-07T07:44:18.63035891-05:00",
            "ExitCode": 0,
            "Output": ""
        },
        ......
}
```

> \[!NOTE] **NOTE:**
>
> * A maximum of five health check status records can be stored in a container. The last five records are saved.
> * Only one health check configuration item can take effect in a container at a time. The later items configured in the Dockerfile will overwrite the earlier ones. Configurations during container creation will overwrite those in images.
> * In the Dockerfile, you can set  **HEALTHCHECK NONE**  to cancel the health check configuration in a referenced image. When a container is running, you can set  **--no-healthcheck**  to cancel the health check configuration in an image. Do not configure the health check and  **--no-healthcheck**  parameters at the same time during the startup.
> * After a container with configured health check parameters is started, if Docker daemon exits, the health check is not executed. After Docker daemon is restarted, the container health status changes to  **starting**. Afterwards, the check rules are the same as above.
> * If health check parameters are set to  **0**  during container image creation, the default values are used.
> * If health check parameters are set to  **0**  during container startup, the default values are used.

## Stopping and Deleting a Container

Run the  **docker stop**  command to stop the container named  **container1**.

```console
[root@localhost ~]# docker stop container1
```

Or run the  **docker kill**  command to kill and stop the container.

```console
[root@localhost ~]# docker kill container1
```

After the container is stopped, run the  **docker rm**  command to delete the container.

```console
[root@localhost ~]# docker rm container1
```

Or run the  **docker rm -f**  command to forcibly delete the container.

```console
[root@localhost ~]# docker rm -f container1
```

### Precautions

* Do not run the  **docker rm -f***XXX*  command to delete a container. If you forcibly delete a container, the  **docker rm**  command ignores errors during the process, which may cause residual metadata of the container. If you delete an image in common mode and an error occurs during the deletion process, the deletion fails and no metadata remains.
* Do not run the  **docker kill**  command. The  **docker kill**  command sends related signals to service processes in a container. Depending on the signal processing policies of service processes in the container may cause the result that the signal execution cannot be performed as expected.
* A container in the restarting state may not stop immediately when you run the  **docker stop**  command. If a container uses the restart rules, when the container is in the restarting state, there is a low probability that the  **docker stop**  command on the container returns immediately. The container will still be restarted with the impact of the restart rule.
* Do not run the  **docker restart**  command to restart a container with the  **--rm**  parameter. When a container with the  **--rm**  parameter exits, the container is automatically deleted. If the container with the  **--rm**  parameter is restarted, exceptions may occur. For example, if both the  **--rm**  and  **-ti**  parameters are added when the container is started, the restart operation cannot be performed on the container, otherwise, the container may stop responding and cannot exit.

### When Using docker stop/restart to Specify -t and t<0, Ensure That Applications in the Container Can Process Stop Signal

Stop Principle: (The stop process is called by  **Restart**.)

1. The SIGTERM (15) signal can be sent to a container for the first time.
2. Wait for a period of time (**t**  entered by the user).
3. If the container process still exists, send the SIGKILL (9) signal to forcibly kill the process.

The meaning of the input parameter  **t**  (unit: s) is as follows:

* **t**  < 0: Wait for graceful stop. This setting is preferred when users are assured that their applications have a proper stop signal processing mechanism.
* **t**  = 0: Do not wait and send  **kill -9**  to the container immediately.
* **t**  > 0: Wait for a specified period and send  **kill -9**  to the container if the container does not stop within the specified period.

Therefore, if  **t**  is set to a value less than 0 (for example,  **t**  =  **-1**), ensure that the container application correctly processes the SIGTERM signal. If the container ignores this signal, the container will be suspended when the  **docker stop**  command is run.

### Manually Deleting Containers in the Dead State As the Underlying File System May Be Busy

When Docker deletes a container, it stops related processes of the container, changes the container status to Dead, and then deletes the container rootfs. When the file system or devicemapper is busy, the last step of deleting rootfs fails. Run the  **docker ps -a**  command. The command output shows that the container is in the Dead state. Containers in the Dead state cannot be started again. Wait until the file system is not busy and run the  **docker rm**  command again to delete the containers.

### In PID namespace Shared Containers, If Child Container Is in pause State, Parent Container Stops Responding and the docker run Command Cannot Be Executed

When the  **--pid**  parameter is used to create the parent and child containers that share PID namespace, if any process in the child container cannot exit (for example, it is in the D or pause state) when the  **docker stop**  command is executed, the  **docker stop**  command of the parent container is waiting. You need to manually recover the process so that the command can be executed normally.

In this case, run the  **docker inspect**  command on the container in the pause state to check whether the parent container corresponding to  **PidMode**  is the container that requires  **docker stop**. For the required container, run the  **docker unpause**  command to cancel the pause state of the child container. Then, proceed to the next step.

Generally, the possible cause is that the PID namespace corresponding to the container cannot be destroyed due to residual processes. If the problem persists, use Linux tools to obtain the residual processes and locate the cause of the process exit failure in PID namespace. After the problem is solved, the container can exit.

* Obtain PID namespace ID in a container.

  ```shell
  docker inspect --format={{.State.Pid}} CONTAINERID | awk '{print  "/proc/"$1"/ns/pid"}' |xargs readlink
  ```

* Obtain threads in the namespace.

  ```shell
   ls -l /proc/*/task/*/ns/pid |grep -F PIDNAMESPACE_ID |awk '{print $9}' |awk -F  \/ '{print $5}'
  ```

## Querying Container Information

In any case, the container status should not be determined based on whether the  **docker**  command is successfully returned. To view the container status, you are advised to use the following command:

```shell
docker inspect <NAME|ID>
```

## Modification Operations

### Precautions for Starting Multiple Processes in Container Using docker exec

When the first  **docker exec**  command executed in a container is the  **bash**  command, ensure that all processes started by  **exec**  are stopped before you run the  **exit**  command. Otherwise, the device may stop responding when you run the  **exit**  command. To ensure that the process started by  **exec**  is still running in the background when the  **exit**  command is run, add  **nohup**  when starting the process.

### Usage Conflict Between docker rename and docker stats  *container\_name*

If you run the  **docker stats***container\_name*  command to monitor a container in real time, after the container is renamed by using  **docker rename**, the name displayed after  **docker stats**  is executed is the original name instead of the renamed one.

### Failed to Perform docker rename Operation on Container in restarting State

When the rename operation is performed on a container in the restarting state, Docker modifies the container network configuration accordingly. The container in the restarting state may not be started and the network does not exist. As a result, the rename operation reports an error indicating that the sandbox does not exist. You are advised to rename only containers that are not in the restarting state.

### docker cp

1. When you run  **docker cp**  to copy files to a container, all operations on the container can be performed only after the  **docker cp**  command is executed.
2. When a container runs as a non-**root**  user, and you run the  **docker cp**  command to copy a non-**root**  file on the host to the container, the permission role of the file in the container changes to  **root**. Different from the  **cp**  command, the  **docker cp**  command changes UIDs and GIDs of the files copied to the container to  **root**.

### docker login

After the  **docker login**  command is executed,  **user/passwd**  encrypted by AES (256-bit) is saved in  **/root/.docker/config.json**. At the same time,  *root***.docker/aeskey**  (permission 0600) is generated to decrypt  **user/passwd**  in  **/root/.docker/config.json**. Currently, AES key cannot be updated periodically. You need to manually delete the AES key for updating. After AES key is updated, you need to log in to Docker daemon again to push the AES key no matter whether Docker daemon is restarted. For example:

```console
root@hello:~/workspace/dockerfile# docker login 
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. 
Username: example Password: 
Login Succeeded 
root@hello:~/workspace/dockerfile# docker push example/empty 
The push refers to a repository [docker.io/example/empty] 
547b6288eb33: Layer already exists 
latest: digest: sha256:99d4fb4ce6c6f850f3b39f54f8eca0bbd9e92bd326761a61f106a10454b8900b size: 524 
root@hello:~/workspace/dockerfile# rm /root/.docker/aeskey 
root@hello:~/workspace/dockerfile# docker push example/empty 
WARNING: Error loading config file:/root/.docker/config.json - illegal base64 data at input byte 0 
The push refers to a repository [docker.io/example/empty] 
547b6288eb33: Layer already exists 
errors: 
denied: requested access to the resource is denied 
unauthorized: authentication required 
root@hello:~/workspace/dockerfile# docker login 
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. 
Username: example 
Password: 
Login Succeeded 
root@hello:~/workspace/dockerfile# docker push example/empty 
The push refers to a repository [docker.io/example/empty] 
547b6288eb33: Layer already exists 
latest: digest: sha256:99d4fb4ce6c6f850f3b39f54f8eca0bbd9e92bd326761a61f106a10454b8900b size: 524
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/container_management_2.md
---
# Container Management

## Overall Description

Subcommands supported by Docker are classified into the following groups by function:

Some subcommands have some parameters, such as  **docker run**. You can run the  **docker***command***--help**  command to view the help information of the command. For details about the command parameters, see the preceding command parameter description. The following sections describe how to use each command.

## attach

Syntax:  **docker attach \[***options***]** *container*

Function: Attaches an option to a running container.

Parameter description:

**--no-stdin=false**: Does not attach any STDIN.

**--sig-proxy=true**: Proxies all signals of the container, except SIGCHLD, SIGKILL, and SIGSTOP.

Example:

```shell
$ sudo docker attach attach_test
root@2988b8658669:/# ls bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
```

## commit

Syntax:  **docker commit \[***options***]***container***\[***repository\[:tag]***]**

Function: creates an image from a container.

Parameter description:

**-a**,  **--author=""**: specifies an author.

**-m**,  **--message=""**: specifies the submitted information.

**-p**,  **--pause=true**: pauses the container during submission.

Example:

Run the following command to start a container and submit the container as a new image:

```shell
$ sudo docker commit test busybox:test
sha256:be4672959e8bd8a4291fbdd9e99be932912fe80b062fba3c9b16ee83720c33e1

$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             latest              e02e811dd08f        2 years ago         1.09MB
```

## cp

Syntax:  **docker cp \[***options***]***container***:***src\_path* *dest\_path***|-**

**docker cp \[***options***]** *src\_path***|-** *container***:***dest\_path*

Function: Copies a file or folder from a path in a container to a path on the host or copies a file or folder from the host to the container:

Precautions: The  **docker cp**  command does not support the copy of files in virtual file systems such as  **/proc**,  **/sys**,  **/dev**, and  **/tmp**  in the container and files in the file systems mounted by users in the container.

Parameter description:

**-a**,  **--archive**: Sets the owner of the file copied to the container to the  **container**  user (**--user**).

**-L**,  **--follow-link**: Parses and traces the symbolic link of a file.

Example:

Run the following command to copy the  **/test**  directory in the registry container to the  **/home/***aaa*  directory on the host:

```shell
sudo docker cp registry:/test /home/aaa
```

## create

Syntax:  **docker create \[***options***]** *image* **\[***command***] \[***arg***...]**

Function: Creates a container using an image file and return the ID of the container. After the container is created, run the  **docker start**  command to start the container.  *options*  are used to configure the container during container creation. Some parameters will overwrite the container configuration in the image file.  *command*  indicates the command to be executed during container startup.

Parameter description:

**Table  1**  Parameter description

Example:

Run the following command to create a container named  **busybox**  and run the  **docker start**  command to start the container.

```shell
sudo docker create -ti --name=busybox busybox /bin/bash
```

## diff

Syntax:  **docker diff** *container*

Function: Checks the differences between containers and determines the changes have been made compared with the container creation.

Parameter description: none.

Example:

```shell
$ sudo docker diff registry
C /root
A /root/.bash_history
A /test
```

## exec

Syntax:  **docker exec \[***options***]** *container* *command* **\[***arg...***]**

Function: Runs a command in the container.

Parameter description:

**-d**  and  **--detach=false**: Run in the background.

**-i**  and  **--interactive=false**: Keep the STDIN of the container enabled.

**-t**  and  **--tty=false**: Allocate a virtual terminal.

**--privileged**: Executes commands in privilege mode.

**-u**  and  **--user**: Specifies the user name or UID.

Example:

```shell
$ sudo docker exec -ti exec_test ls
bin  etc   lib   media  opt   root  sbin  sys  tmp  var
dev  home  lib64  mnt  proc  run   srv   test  usr
```

## export

Syntax:  **docker export** *container*

Function: Exports the file system content of a container to STDOUT in .tar format.

Parameter description: none.

Example:

Run the following commands to export the contents of the container named  **busybox**  to the  **busybox.tar**  package:

```shell
$ sudo docker export busybox > busybox.tar
$ ls
busybox.tar
```

## inspect

Syntax:  **docker inspect \[***options***]***container***|***image***\[***container*|*image...***]**

Function: Returns the underlying information about a container or image.

Parameter description:

**-f**  and  **--format=""**: Output information in a specified format.

**-s**  and  **--size**: Display the total file size of the container when the query type is container.

**--type**: Returns the JSON format of the specified type.

**-t**  and  **--time=120**: Timeout interval, in seconds. If the  **docker inspect**  command fails to be executed within the timeout interval, the system stops waiting and immediately reports an error. The default value is  **120**.

Example:

1. Run the following command to return information about a container:

   ```shell
   $ sudo docker inspect busybox_test
   [
       {
           "Id": "9fbb8649d5a8b6ae106bb0ac7686c40b3cbd67ec2fd1ab03e0c419a70d755577",
           "Created": "2019-08-28T07:43:51.27745746Z",
           "Path": "bash",
           "Args": [],
           "State": {
               "Status": "running",
               "Running": true,
               "Paused": false,
               "Restarting": false,
               "OOMKilled": false,
               "Dead": false,
               "Pid": 64177,
               "ExitCode": 0,
               "Error": "",
               "StartedAt": "2019-08-28T07:43:53.021226383Z",
               "FinishedAt": "0001-01-01T00:00:00Z"
           },
   ......
   ```

2. Run the following command to return the specified information of a container in a specified format. The following uses the IP address of the busybox\_test container as an example.

   ```shell
   $ sudo docker inspect -f {{.NetworkSettings.IPAddress}} busybox_test
   172.17.0.91
   ```

## logs

Syntax:  **docker logs \[***options***]** *container*

Function: Captures logs in a container that is in the  **running**  or  **stopped**  state.

Parameter description:

**-f**  and  **--follow=false**: Print logs in real time.

**-t**  and  **--timestamps=false**: Display the log timestamp.

**--since**: Displays logs generated after the specified time.

**--tail="all"**: Sets the number of lines to be displayed. By default, all lines are displayed.

Example:

1. Run the following command to check the logs of the jaegertracing container where a jaegertracing service runs:

   ```shell
   $ sudo docker logs jaegertracing
   {"level":"info","ts":1566979103.3696961,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":14269,"status":"unavailable"}
   {"level":"info","ts":1566979103.3820567,"caller":"memory/factory.go:55","msg":"Memory storage configuration","configuration":{"MaxTraces":0}}
   {"level":"info","ts":1566979103.390773,"caller":"tchannel/builder.go:94","msg":"Enabling service discovery","service":"jaeger-collector"}
   {"level":"info","ts":1566979103.3908608,"caller":"peerlistmgr/peer_list_mgr.go:111","msg":"Registering active peer","peer":"127.0.0.1:14267"}
   {"level":"info","ts":1566979103.3922884,"caller":"all-in-one/main.go:186","msg":"Starting agent"}
   {"level":"info","ts":1566979103.4047635,"caller":"all-in-one/main.go:226","msg":"Starting jaeger-collector TChannel server","port":14267}
   {"level":"info","ts":1566979103.404901,"caller":"all-in-one/main.go:236","msg":"Starting jaeger-collector HTTP server","http-port":14268}
   {"level":"info","ts":1566979103.4577134,"caller":"all-in-one/main.go:256","msg":"Listening for Zipkin HTTP traffic","zipkin.http-port":9411}
   ```

2. Add  **-f**  to the command to output the logs of the jaegertracing container in real time.

   ```shell
   $ sudo docker logs -f jaegertracing
   {"level":"info","ts":1566979103.3696961,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":14269,"status":"unavailable"}
   {"level":"info","ts":1566979103.3820567,"caller":"memory/factory.go:55","msg":"Memory storage configuration","configuration":{"MaxTraces":0}}
   {"level":"info","ts":1566979103.390773,"caller":"tchannel/builder.go:94","msg":"Enabling service discovery","service":"jaeger-collector"}
   {"level":"info","ts":1566979103.3908608,"caller":"peerlistmgr/peer_list_mgr.go:111","msg":"Registering active peer","peer":"127.0.0.1:14267"}
   {"level":"info","ts":1566979103.3922884,"caller":"all-in-one/main.go:186","msg":"Starting agent"}
   ```

## pause/unpause

Syntax:  **docker pause** *container*

**docker unpause** *container*

Function: The two commands are used in pairs. The  **docker pause**  command suspends all processes in a container, and the  **docker unpause**  command resumes the suspended processes.

Parameter description: none.

Example:

The following uses a container where the docker registry service runs as an example. After the  **docker pause**  command is executed to pause the process of the container, access of the registry service by running the  **curl**  command is blocked. You can run the  **docker unpause**  command to resume the suspended registry service. The registry service can be accessed by running the  **curl**  command.

1. Run the following command to start a registry container:

   ```shell
   sudo docker run -d --name pause_test -p 5000:5000 registry
   ```

   Run the  **curl**  command to access the service. Check whether the status code  **200 OK**  is returned.

   ```shell
   sudo curl -v 127.0.0.1:5000
   ```

2. Run the following command to stop the processes in the container:

   ```shell
   sudo docker pause pause_test
   ```

   Run the  **curl**  command to access the service to check whether it is blocked and wait until the service starts.

3. Run the following command to resume the processes in the container:

   ```shell
   sudo docker unpause pause_test
   ```

   The cURL access in step 2 is restored and the request status code  **200 OK**  is returned.

## port

Syntax:  **docker port***container* **\[***private\_port\[/proto]***]**

Function: Lists the port mapping of a container or queries the host port where a specified port resides.

Parameter description: none.

Example:

1. Run the following command to list all port mappings of a container:

   ```shell
   $ sudo docker port registry
   5000/tcp -> 0.0.0.0.:5000
   ```

2. Run the following command to query the mapping of a specified container port:

   ```shell
   $ sudo docker port registry 5000
   0.0.0.0.:5000
   ```

## ps

Syntax:  **docker ps \[***options***]**

Function: Lists containers in different states based on different parameters. If no parameter is added, all running containers are listed.

Parameter description:

**-a**  and  **--all=false**: Display the container.

**-f**  and  **--filter=\[]**: Filter values. The available options are:  **exited=***int*  (exit code of the container)  **status=***restarting|running|paused|exited*  (status code of the container), for example,  **-f status=running**: lists the running containers.

**-l**  and  **--latest=false**: List the latest created container.

**-n=-1**: Lists the latest created  *n*  containers.

**--no-trunc=false**: Displays all 64-bit container IDs. By default, 12-bit container IDs are displayed.

**-q**  and  **--quiet=false**: Display the container ID.

**-s**  and  **--size=false**: Display the container size.

Example:

1. Run the following command to lists running containers:

   ```shell
   sudo docker ps
   ```

2. Run the following command to display all containers:

   ```shell
   sudo docker ps -a
   ```

## rename

Syntax:  **docker rename OLD\_NAME NEW\_NAME**

Function: Renames a container.

Example:

Run the  **docker run**  command to create and start a container, run the  **docker rename**  command to rename the container, and check whether the container name is changed.

```shell
$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
b15976967abb        busybox:latest        "bash"              3 seconds ago       Up 2 seconds                            festive_morse
$ sudo docker rename festive_morse new_name
$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
b15976967abb        busybox:latest        "bash"              34 seconds ago      Up 33 seconds                           new_name
```

## restart

Syntax:  **docker restart \[***options***]** *container* **\[***container...***]**

Function: Restarts a running container.

Parameter description:

**-t**  and  **--time=10**: Number of seconds to wait for the container to stop before the container is killed. If the container has stopped, restart the container. The default value is  **10**.

Example:

```shell
sudo docker restart busybox
```

> \[!NOTE] **NOTE:**
> During the container restart, if a process in the  **D**  or  **Z**  state exists in the container, the container may fail to be restarted. In this case, you need to analyze the cause of the  **D**  or  **Z**  state of the process in the container. Restart the container after the  **D**  or  **Z**  state of the process in the container is released.

## rm

Syntax:  **docker rm \[***options***]***container* **\[***container...***]**

Function: Deletes one or more containers.

Parameter description:

**-f**  and  **--force=false**: Forcibly delete a running container.

**-l**  and  **--link=false**: Remove the specified link and do not remove the underlying container.

**-v**  and  **--volumes=false**: Remove the volumes associated with the container.

Example:

1. Run the following command to delete a stopped container:

   ```shell
   sudo docker rm test
   ```

2. Run the following command to delete a running container:

   ```shell
   sudo docker rm -f rm_test
   ```

## run

Syntax:  **docker run \[***options***]***image* **\[***command***] \[***arg***...]**

Function: Creates a container from a specified image (if the specified image does not exist, an image is downloaded from the official image registry), starts the container, and runs the specified command in the container. This command integrates the  **docker create**,  **docker start**, and  **docker exec**  commands.

Parameter description: (The parameters of this command are the same as those of the  **docker create**  command. For details, see the parameter description of the  **docker create**  command. Only the following two parameters are different.)

**--rm=false**: Specifies the container to be automatically deleted when it exits.

**-v**: Mounts a local directory or an anonymous volume to the container. Note: When a local directory is mounted to a container with a SELinux security label, do not add or delete the local directory at the same time. Otherwise, the security label may not take effect.

**--sig-proxy=true**: Receives proxy of the process signal. SIGCHLD, SIGSTOP, and SIGKILL do not use the proxy.

Example:

Run the busybox image to start a container and run the  **/bin/sh**  command after the container is started:

```shell
sudo docker run -ti busybox /bin/sh
```

## start

Syntax:  **docker start \[***options***]** *container* **\[***container***...]**

Function: Starts one or more containers that are not running.

Parameter description:

**-a**  and  **--attach=false**: Attach the standard output and error output of a container to STDOUT and STDERR of the host.

**-i**  and  **--interactive=false**: Attach the standard input of the container to the STDIN of the host.

Example:

Run the following command to start a container named  **busybox**  and add the  **-i -a**  to the command to add standard input and output. After the container is started, directly enter the container. You can exist the container by entering  **exit**.

If  **-i -a**  is not added to the command when the container is started, the container is started in the background.

```shell
sudo docker start -i -a busybox
```

## stats

Syntax:  **docker stats \[***options***] \[***container***...]**

Function: Continuously monitors and displays the resource usage of a specified container. (If no container is specified, the resource usage of all containers is displayed by default.)

Parameter description:

**-a**, and  **--all**: Display information about all containers. By default, only running containers are displayed.

**--no-stream**: Displays only the first result and does not continuously monitor the result.

Example:

Run the  **docker run**  command to start and create a container, and run the  **docker stats**  command to display the resource usage of the container:

```shell
$ sudo docker stats
CONTAINER ID        NAME                    CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
2e242bcdd682        jaeger                  0.00%               77.08MiB / 125.8GiB   0.06%               42B / 1.23kB        97.9MB / 0B         38
02a06be42b2c        relaxed_chandrasekhar   0.01%               8.609MiB / 125.8GiB   0.01%               0B / 0B             0B / 0B             10
deb9e49fdef1        hardcore_montalcini     0.01%               12.79MiB / 125.8GiB   0.01%               0B / 0B             0B / 0B             9
```

## stop

Syntax:  **docker stop \[***options***]** *container* **\[***container***...]**

Function: Sends a SIGTERM signal to a container and then sends a SIGKILL signal to stop the container after a certain period.

Parameter description:

**-t**  and  **--time=10**: Number of seconds that the system waits for the container to exit before the container is killed. The default value is  **10**.

Example:

```shell
sudo docker stop -t=15 busybox
```

## top

Syntax:  **docker top** *container* **\[***ps options***]**

Function: Displays the processes running in a container.

Parameter description: none.

Example:

Run the top\_test container and run the  **top**  command in the container.

```shell
$ sudo docker top top_test
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
root                70045               70028               0                   15:52               pts/0               00:00:00            bash
```

The value of  **PID**  is the PID of the process in the container on the host.

## update

Syntax:  **docker update \[***options***]** *container* **\[***container***...]**

Function: Hot changes one or more container configurations.

Parameter description:

**Table  1**  Parameter description

Example:

Run the following command to change the CPU and memory configurations of the container named  **busybox**, including changing the relative weight of the host CPU obtained by the container to  **512**, the CPU cores that can be run by processes in the container to  **0,1,2,3**, and the memory limit for running the container to  **512 m**.

```shell
sudo docker update  --cpu-shares 512  --cpuset-cpus=0,3 --memory 512m ubuntu
```

## wait

Syntax:  **docker wait** *container* **\[***container...***]**

Function: Waits for a container to stop and print the exit code of the container:

Parameter description: none.

Example:

Run the following command to start a container named  **busybox**:

```shell
sudo docker start -i -a busybox
```

Run the  **docker wait**  command:

```shell
$ sudo docker wait busybox
0
```

Wait until the busybox container exits. After the busybox container exits, the exit code  **0**  is displayed.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/container_management.md
---
# Container Management

## Creating a Container

### Description

To create a container, run the  **isula create**  command. The container engine will use the specified container image to create a read/write layer, or use the specified local rootfs as the running environment of the container. After the creation is complete, the container ID is output as standard output. You can run the  **isula start**  command to start the container. The new container is in the  **inited**  state.

### Usage

```shell
isula create [OPTIONS] IMAGE [COMMAND] [ARG...]
```

### Parameters

The following table lists the parameters supported by the  **create**  command.

**Table  1**  Parameter description

### Constraints

* When the  **--user**  or  **--group-add**  parameter is used to verify the user or group during container startup, if the container uses an OCI image, the verification is performed in the  **etc/passwd**  and  **etc/group**  files of the actual rootfs of the image. If a folder or block device is used as the rootfs of the container, the  **etc/passwd**  and  **etc/group**  files in the host are verified. The rootfs ignores mounting parameters such as  **-v**  and  **--mount**. That is, when these parameters are used to attempt to overwrite the  **etc/passwd**  and  **etc/group**  files, the parameters do not take effect during the search and take effect only when the container is started. The generated configuration is saved in the  **iSulad root directory/engine/container ID/start\_generate\_config.json**  file. The file format is as follows:

  ```json
  {
      "uid": 0,
      "gid": 8,
      "additionalGids": [
          1234,
          8
      ]
  }
  ```

### Example

Create a container.

```shell
# isula create busybox
fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
# isula ps -a
STATUS PID IMAGE   COMMAND EXIT_CODE RESTART_COUNT STARTAT FINISHAT RUNTIME ID           NAMES                
inited -   busybox "sh"    0         0             -       -        runc     fd7376591a9c fd7376591a9c4521...
```

## Starting a Container

### Description

To start one or more containers, run the  **isula start**  command.

### Usage

```shell
isula start [OPTIONS] CONTAINER [CONTAINER...]
```

### Parameters

The following table lists the parameters supported by the  **start**  command.

**Table  1**  Parameter description

### Example

Start a new container.

```shell
isula start fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
```

## Running a Container

### Description

To create and start a container, run the  **isula run**  command. You can use a specified container image to create a container read/write layer and prepare for running the specified command. After the container is created, run the specified command to start the container. The  **run**  command is equivalent to creating and starting a container.

### Usage

```shell
isula run [OPTIONS] ROOTFS|IMAGE [COMMAND] [ARG...]
```

### Parameters

The following table lists the parameters supported by the  **run**  command.

**Table  1**  Parameter description

### Constraints

* When the parent process of a container exits, the corresponding container automatically exits.

* When a common container is created, the parent process cannot be initiated because the permission of common containers is insufficient. As a result, the container does not respond when you run the  **attach**  command though it is created successfully.

* If  **--net**  is not specified when the container is running, the default host name is  **localhost**.

* If the  **--files-limit**  parameter is to transfer a small value, for example, 1, when the container is started, iSulad creates a cgroup, sets the files.limit value, and writes the PID of the container process to the  **cgroup.procs**  file of the cgroup. At this time, the container process has opened more than one handle. As a result, a write error is reported, and the container fails to be started.

* If both\*\*--mount\*\*  and  **--volume**  exist and their destination paths conflict,  **--mount**  will be run after  **--volume**  (that is, the mount point in  **--volume**  will be overwritten).

  Note: The value of the  **type**  parameter of lightweight containers can be  **bind**  or  **squashfs**. When  **type**  is set to  **squashfs**,  **src**  is the image path. The value of the  **type**  parameter of the native Docker can be  **bind**,  **volume**, and  **tmpfs**.

* The restart policy does not support  **unless-stopped**.

* The values returned for Docker and lightweight containers are 127 and 125 respectively in the following three scenarios:

  The host device specified by  **--device**  does not exist.

  The hook JSON file specified by  **--hook-spec**  does not exist.

  The entry point specified by  **--entrypoint**  does not exist.

* When the  **--volume**  parameter is used, /dev/ptmx will be deleted and recreated during container startup. Therefore, do not mount the  **/dev**  directory to that of the container. Use  **--device**  to mount the devices in  **/dev**  of the container.

* When the **-it** parameter is used, the **/dev/ptmx** device will be deleted and rebuilt when the container is started. Therefore, do not mount the **/dev** directory to the **/dev** directory of the container. Instead, use **--device** to mount the devices in the **/dev** directory to the container.

* Do not use the echo option to input data to the standard input of the  **run**  command. Otherwise, the client will be suspended. The echo value should be directly transferred to the container as a command line parameter.

  ```shell
  # echo ls | isula run -i busybox /bin/sh

  ^C
  #
  ```

  The client is suspended when the preceding command is executed because the preceding command is equivalent to input  **ls**  to  **stdin**. Then EOF is read and the client does not send data and waits for the server to exit. However, the server cannot determine whether the client needs to continue sending data. As a result, the server is suspended in reading data, and both parties are suspended.

  The correct execution method is as follows:

  ```shell
  # isula run -i busybox ls
  bin
  dev
  etc
  home
  proc
  root
  sys
  tmp
  usr
  var
  #
  ```

* If the root directory (/) of the host is used as the file system of the container, the following situations may occur during the mounting:

  **Table  2**  Mounting scenarios

  > \[!TIP] **NOTICE:**
  > Scenario 1: Mount  **/home/test1**  and then  **/home/test2**. In this case, the content in  **/home/test1**  overwrites the content in  **/mnt**. As a result, the  **abc**  directory does not exist in  **/mnt**, and mounting\*\*/home/test2\*\*  to  **/mnt/abc**  fails.
  > Scenario 2: Mount  **/home/test2**  and then  **/home/test1**. In this case, the content of  **/mnt**  is replaced with the content of  **/home/test1**  during the second mounting. In this way, the content mounted during the first mounting from  **/home/test2**  to  **/mnt/abc**  is overwritten.
  > The first scenario is not supported. For the second scenario, users need to understand the risk of data access failures.

* Exercise caution when configuring the **/sys** and **/proc** directories to be writable.

  The **/sys** and **/proc** directories contain the APIs for maintaining Linux kernel parameters and managing devices. If the directories are writable in a container, container escape may occur.

* Exercise caution when configuring containers to share namespaces with hosts.

  For example, if you use **--pid**, **--ipc**, **--uts**, or **--net** to configure namespace sharing between the container and the host, the namespace isolation between the container and the host is lost, and the host can be attacked from the container. For example, if you use **--pid** to configure PID namespace sharing between the container and the host, the PID of the process on the host can be viewed in the container and the process can be killed in the container.

* Exercise caution when configuring parameters that can be used to mount host resources, such as **--device** and **-v**. Do not map sensitive directories or devices of the host to containers to prevent leakage of sensitive information.

* Exercise caution when using the **--privileged** option to start a container. If the **--privileged** option is used, the container will have excessive permissions, affecting the host configuration.

  > \[!TIP] **NOTICE:**
  >
  > * In high concurrency scenarios (200 containers are concurrently started), the memory management mechanism of Glibc may cause memory holes and large virtual memory (for example, 10 GB). This problem is caused by the restriction of the Glibc memory management mechanism in the high concurrency scenario, but not by memory leakage. Therefore, the memory consumption does not increase infinitely. You can set the  **MALLOC\_ARENA\_MAX**  environment variable to reduce the virtual memory and increase the probability of reducing the physical memory. However, this environment variable will cause the iSulad concurrency performance to deteriorate. Set this environment variable based on the site requirements.
  >
  >   ```text
  >   To balance performance and memory usage, set MALLOC_ARENA_MAX to 4. (The iSulad performance deterioration on the ARM64 server is controlled by less than 10%.)
  >   Configuration method:
  >   1. To manually start iSulad, run the export MALLOC_ARENA_MAX=4 command and then start the iSulad.
  >   2. If systemd manages iSulad, you can modify the /etc/sysconfig/iSulad file by adding MALLOC_ARENA_MAX=4.
  >   ```

### Example

Run a new container.

```shell
# isula run -itd busybox
9c2c13b6c35f132f49fb7ffad24f9e673a07b7fe9918f97c0591f0d7014c713b
```

## Stopping a Container

### Description

To stop a container, run the  **isula stop**  command. The SIGTERM signal is sent to the first process in the container. If the container is not stopped within the specified time (10s by default), the SIGKILL signal is sent.

### Usage

```shell
isula stop [OPTIONS] CONTAINER [CONTAINER...]
```

### Parameters

The following table lists the parameters supported by the  **stop**  command.

**Table  1**  Parameter description

### Constraints

* If the  **t**  parameter is specified and the value of  **t**  is less than 0, ensure that the application in the container can process the stop signal.

  Principle of the Stop command: Send the SIGTERM signal to the container, and then wait for a period of time (**t**  entered by the user). If the container is still running after the period of time, the SIGKILL signal is sent to forcibly kill the container.

* The meaning of the input parameter  **t**  is as follows:

  **t**  < 0: Wait for graceful stop. This setting is preferred when users are assured that their applications have a proper stop signal processing mechanism.

  **t**  = 0: Do not wait and send  **kill -9**  to the container immediately.

  **t**  > 0: Wait for a specified period and send  **kill -9**  to the container if the container does not stop within the specified period.

  Therefore, if  **t**  is set to a value less than 0 (for example,  **t**  = -1), ensure that the container application correctly processes the SIGTERM signal. If the container ignores this signal, the container will be suspended when the  **isula stop**  command is run.

### Example

Stop a container.

```shell
# isula stop fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
```

## Forcibly Stopping a Container

### Description

To forcibly stop one or more running containers, run the  **isula kill**  command.

### Usage

```shell
isula kill [OPTIONS] CONTAINER [CONTAINER...]
```

### Parameters

The following table lists the parameters supported by the  **kill**  command.

**Table  1**  Parameter description

### Example

Kill a container.

```shell
# isula kill fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
```

## Removing a Container

### Description

To remove a container, run the  **isula rm**  command.

### Usage

```shell
isula rm [OPTIONS] CONTAINER [CONTAINER...]
```

### Parameters

The following table lists the parameters supported by the  **rm**  command.

**Table  1**  Parameter description

### Constraints

* In normal I/O scenarios, it takes T1 to delete a running container in an empty environment (with only one container). In an environment with 200 containers (without a large number of I/O operations and with normal host I/O), it takes T2 to delete a running container. The specification of T2 is as follows: T2 = max {T1 x 3, 5}s.

### Example

Delete a stopped container.

```shell
# isula rm fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
```

## Attaching to a Container

### Description

To attach standard input, standard output, and standard error of the current terminal to a running container, run the  **isula attach**  command.

### Usage

```shell
isula attach [OPTIONS] CONTAINER
```

### Parameters

The following table lists the parameters supported by the  **attach**  command.

**Table  1**  Parameter description

### Constraints

* For the native Docker, running the  **attach**  command will directly enter the container. For the iSulad container, you have to run the  **attach**  command and press  **Enter**  to enter the container.

### Example

Attach to a running container.

```shell
# isula attach fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
/ #
/ #
```

## Renaming a Container

### Description

To rename a container, run the  **isula rename**  command.

### Usage

```shell
isula rename [OPTIONS] OLD_NAME NEW_NAME
```

### Parameters

The following table lists the parameters supported by the  **rename**  command.

**Table  1**  Parameter description

### Example

Rename a container.

```shell
isula rename my_container my_new_container
```

## Executing a Command in a Running Container

### Description

To execute a command in a running container, run the  **isula exec**  command. This command is executed in the default directory of the container. If a user-defined directory is specified for the basic image, the user-defined directory is used.

### Usage

```shell
isula exec [OPTIONS] CONTAINER COMMAND [ARG...]
```

### Parameters

The following table lists the parameters supported by the  **exec**  command.

**Table  1**  Parameter description

### Constraints

* If no parameter is specified in the  **isula exec**  command, the  **-it**  parameter is used by default, indicating that a pseudo terminal is allocated and the container is accessed in interactive mode.

* When you run the  **isula exec**  command to execute a script and run a background process in the script, you need to use the  **nohup**  flag to ignore the  **SIGHUP**  signal.

  When you run the  **isula exec**  command to execute a script and run a background process in the script, you need to use the  **nohup**  flag. Otherwise, the kernel sends the  **SIGHUP**  signal to the process executed in the background when the process (first process of the session) exits. As a result, the background process exits and zombie processes occur.

* After running the  **isula exec**  command to access the container process, do not run background programs. Otherwise, the system will be suspended.

  To run the  **isula exec**  command to execute a background process, perform the following steps:

  1. Run the  **isula exec container\_name bash**  command to access the container.
  2. After entering the container, run the  **script &**  command.
  3. Run the  **exit**  command. The terminal stops responding.

  > After the isula exec command is executed to enter the container, the background program stops responding because the isula exec command is executed to enter the container and run the background while1 program. When the bash command is run to exit the process, the while1 program does not exit and becomes an orphan process, which is taken over by process 1.
  > The while1 process is executed by the initial bash process fork \&exec of the container. The while1 process copies the file handle of the bash process. As a result, the handle is not completely closed when the bash process exits.
  > The console process cannot receive the handle closing event, epoll\_wait stops responding, and the process does not exit.

* Do not run the  **isula exec**  command in the background. Otherwise, the system may be suspended.

  Run the  **isula exec**  command in the background as follows:

  Run the  **isula exec script &**  command in the background, for example,  **isula exec container\_name script &,isula exec**. The command is executed in the background. The script continuously displays a file by running the  **cat**  command. Normally, there is output on the current terminal. If you press  **Enter**  on the current terminal, the client exits the stdout read operation due to the I/O read failure. As a result, the terminal does not output data. The server continues to write data to the buffer of the FIFO because the process is still displaying files by running the  **cat**  command. When the buffer is full, the process in the container is suspended in the write operation.

* When a lightweight container uses the  **exec**  command to execute commands with pipe operations, you are advised to run the  **/bin/bash -c**  command.

  Typical application scenarios:

  Run the  **isula exec container\_name -it ls /test | grep "xx" | wc -l**  command to count the number of xx files in the test directory. The output is processed by  **grep**  and  **wc**  through the pipe because  **ls /test**  is executed with  **exec**. The output of  **ls /test**  executed by  **exec**  contains line breaks. When the output is processed, the result is incorrect.

  Cause: Run the  **ls /test**  command using  **exec**. The command output contains a line feed character. Run the\*\*| grep "xx" | wc -l\*\*  command for the output. The processing result is 2 (two lines).

  ```shell
  # isula exec  -it container ls /test
  xx    xx10  xx12  xx14  xx3   xx5   xx7   xx9
  xx1   xx11  xx13  xx2   xx4   xx6   xx8
  #
  ```

  Suggestion: When running the  **run/exec**  command to perform pipe operations, run the  **/bin/bash -c**  command to perform pipe operations in the container.

  ```shell
  # isula exec  -it container  /bin/sh -c "ls /test | grep "xx" | wc -l"
  15
  #
  ```

* Do not use the  **echo**  option to input data to the standard input of the  **exec**  command. Otherwise, the client will be suspended. The echo value should be directly transferred to the container as a command line parameter.

  ```shell
  # echo ls | isula exec 38 /bin/sh

  ^C
  #
  ```

  The client is suspended when the preceding command is executed because the preceding command is equivalent to input  **ls**  to  **stdin**. Then EOF is read and the client does not send data and waits for the server to exit. However, the server cannot determine whether the client needs to continue sending data. As a result, the server is suspended in reading data, and both parties are suspended.

  The correct execution method is as follows:

  ```shell
  # isula exec 38 ls
  bin   dev   etc   home  proc  root  sys   tmp   usr   var
  ```

### Example

Run the echo command in a running container.

```shell
# isula exec c75284634bee echo "hello,world"
hello,world
```

## Querying Information About a Single Container

### Description

To query information about a single container, run the  **isula inspect**  command.

### Usage

```shell
isula inspect [OPTIONS] CONTAINER|IMAGE [CONTAINER|IMAGE...]
```

### Parameters

The following table lists the parameters supported by the  **inspect**  command.

**Table  1**  Parameter description

### Example

Query information about a container.

```shell
# isula inspect -f '{{.State.Status} {{.State.Running}}' c75284634bee
running
true

# isula inspect c75284634bee
[
    {
        "Id": "c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a",
        "Created": "2019-08-01T22:48:13.993304927-04:00",
        "Path": "sh",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "Pid": 21164,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2019-08-02T06:09:25.535049168-04:00",
            "FinishedAt": "2019-08-02T04:28:09.479766839-04:00",
            "Health": {
                "Status": "",
                "FailingStreak": 0,
                "Log": []
            }
        },
        "Image": "busybox",
        "ResolvConfPath": "",
        "HostnamePath": "",
        "HostsPath": "",
        "LogPath": "none",
        "Name": "c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a",
        "RestartCount": 0,
        "HostConfig": {
            "Binds": [],
            "NetworkMode": "",
            "GroupAdd": [],
            "IpcMode": "",
            "PidMode": "",
            "Privileged": false,
            "SystemContainer": false,
            "NsChangeFiles": [],
            "UserRemap": "",
            "ShmSize": 67108864,
            "AutoRemove": false,
            "AutoRemoveBak": false,
            "ReadonlyRootfs": false,
            "UTSMode": "",
            "UsernsMode": "",
            "Sysctls": {},
            "Runtime": "runc",
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "CapAdd": [],
            "CapDrop": [],
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": [],
            "HookSpec": "",
            "CPUShares": 0,
            "Memory": 0,
            "OomScoreAdj": 0,
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "CPUPeriod": 0,
            "CPUQuota": 0,
            "CPURealtimePeriod": 0,
            "CPURealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "SecurityOpt": [],
            "StorageOpt": {},
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "FilesLimit": 0,
            "Ulimits": [],
            "Hugetlbs": [],
            "HostChannel": {
                "PathOnHost": "",
                "PathInContainer": "",
                "Permissions": "",
                "Size": 0
            },
            "EnvTargetFile": "",
            "ExternalRootfs": ""
        },
        "Mounts": [],
        "Config": {
            "Hostname": "localhost",
            "User": "",
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "TERM=xterm",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Tty": true,
            "Cmd": [
                "sh"
            ],
            "Entrypoint": [],
            "Labels": {},
            "Annotations": {
                "log.console.file": "none",
                "log.console.filerotate": "7",
                "log.console.filesize": "1MB",
                "rootfs.mount": "/var/lib/isulad/mnt/rootfs",
                "native.umask": "secure"
            },
            "HealthCheck": {
                "Test": [],
                "Interval": 0,
                "Timeout": 0,
                "StartPeriod": 0,
                "Retries": 0,
                "ExitOnUnhealthy": false
            }
        },
        "NetworkSettings": {
            "IPAddress": ""
        }
    }
]
```

## Querying Information About All Containers

### Description

To query information about all containers, run the  **isula ps**  command.

### Usage

```shell
isula ps [OPTIONS]
```

### Parameters

The following table lists the parameters supported by the  **ps**  command.

**Table  1**  Parameter description

### Example

Query information about all containers.

```shell
# isula ps -a

ID           IMAGE                                     STATUS  PID    COMMAND EXIT_CODE RESTART_COUNT STARTAT        FINISHAT    RUNTIME NAMES
e84660aa059c rnd-dockerhub.huawei.com/official/busybox running 304765 "sh"    0         0             13 minutes ago -           runc     e84660aa059cafb0a77a4002e65cc9186949132b8e57b7f4d76aa22f28fde016
# isula ps -a --format "table {{.ID}} {{.Image}}" --no-trunc
ID                                                               IMAGE
e84660aa059cafb0a77a4002e65cc9186949132b8e57b7f4d76aa22f28fde016 rnd-dockerhub.huawei.com/official/busybox
```

## Restarting a Container

### Description

To restart one or more containers, run the  **isula restart**  command.

### Usage

```shell
isula restart [OPTIONS] CONTAINER [CONTAINER...]
```

### Parameters

The following table lists the parameters supported by the  **restart**  command.

**Table  1**  Parameter description

### Constraints

* If the  **t**  parameter is specified and the value of  **t**  is less than 0, ensure that the application in the container can process the stop signal.

  The restart command first calls the stop command to stop the container. Send the SIGTERM signal to the container, and then wait for a period of time (**t**  entered by the user). If the container is still running after the period of time, the SIGKILL signal is sent to forcibly kill the container.

* The meaning of the input parameter  **t**  is as follows:

  **t**  < 0: Wait for graceful stop. This setting is preferred when users are assured that their applications have a proper stop signal processing mechanism.

  **t**  = 0: Do not wait and send  **kill -9**  to the container immediately.

  **t**  > 0: Wait for a specified period and send  **kill -9**  to the container if the container does not stop within the specified period.

  Therefore, if  **t**  is set to a value less than 0 (for example,  **t**  = -1), ensure that the container application correctly processes the SIGTERM signal. If the container ignores this signal, the container will be suspended when the  **isula stop**  command is run.

### Example

Restart a container.

```shell
# isula restart c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a
 c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a
```

## Waiting for a Container to Exit

### Description

To wait for one or more containers to exit, run the  **isula wait**  command. Only containers whose runtime is of the LCR type are supported.

### Usage

```shell
isula wait [OPTIONS] CONTAINER [CONTAINER...]
```

### Parameters

The following table lists the parameters supported by the  **wait**  command.

**Table  1**  Parameter description

### Example

Wait for a single container to exit.

```shell
$ isula wait c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a
 137
```

## Viewing Process Information in a Container

### Description

To view process information in a container, run the  **isula top**  command. Only containers whose runtime is of the LCR type are supported.

### Usage

```shell
isula top [OPTIONS] container [ps options]
```

### Parameters

The following table lists the parameters supported by the  **top**  command.

**Table  1**  Parameter description

### Example

Query process information in a container.

```shell
# isula top 21fac8bb9ea8e0be4313c8acea765c8b4798b7d06e043bbab99fc20efa72629c
UID        PID  PPID  C STIME TTY          TIME CMD
root     22166 22163  0 23:04 pts/1    00:00:00 sh
```

## Displaying Resource Usage Statistics of a Container

### Description

To display resource usage statistics in real time, run the  **isula stats**  command. Only containers whose runtime is of the LCR type are supported.

### Usage

```shell
isula stats [OPTIONS] [CONTAINER...]
```

### Parameters

The following table lists the parameters supported by the  **stats**  command.

**Table  1**  Parameter description

### Example

Display resource usage statistics.

```shell
# isula stats --no-stream 21fac8bb9ea8e0be4313c8acea765c8b4798b7d06e043bbab99fc20efa72629c                                                                                 CONTAINER        CPU %      MEM USAGE / LIMIT          MEM %      BLOCK I / O                PIDS
21fac8bb9ea8     0.00       56.00 KiB / 7.45 GiB       0.00       0.00 B / 0.00 B            1
```

## Obtaining Container Logs

### Description

To obtain container logs, run the  **isula logs**  command.

### Usage

```shell
isula logs [OPTIONS] [CONTAINER...]
```

### Parameters

The following table lists the parameters supported by the  **logs**  command.

**Table  1**  Parameter description

### Constraints

* By default, the container log function is enabled. To disable this function, run the  **isula create --log-opt disable-log=true**  or  **isula run --log-opt disable-log=true**  command.

### Example

Obtain container logs.

```shell
# isula logs 6a144695f5dae81e22700a8a78fac28b19f8bf40e8827568b3329c7d4f742406
hello, world
hello, world
hello, world
```

## Copying Data Between a Container and a Host

### Description

To copy data between a host and a container, run the  **isula cp**  command. Only containers whose runtime is of the LCR type are supported.

### Usage

```shell
isula cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH
isula cp [OPTIONS] SRC_PATH CONTAINER:DEST_PATH
```

### Parameters

The following table lists the parameters supported by the  **cp**  command.

**Table  1**  Parameter description

### Constraints

* When iSulad copies files, note that the  **/etc/hostname**,  **/etc/resolv.conf**, and  **/etc/hosts**  files are not mounted to the host, neither the  **--volume**  and  **--mount**  parameters. Therefore, the original files in the image instead of the files in the real container are copied.

  ```shell
  # isula cp b330e9be717a:/etc/hostname /tmp/hostname
  # cat /tmp/hostname
  #
  ```

* When decompressing a file, iSulad does not check the type of the file or folder to be overwritten in the file system. Instead, iSulad directly overwrites the file or folder. Therefore, if the source is a folder, the file with the same name is forcibly overwritten as a folder. If the source file is a file, the folder with the same name will be forcibly overwritten as a file.

  ```shell
  # rm -rf /tmp/test_file_to_dir && mkdir /tmp/test_file_to_dir
  # isula exec b330e9be717a /bin/sh -c "rm -rf /tmp/test_file_to_dir && touch /tmp/test_file_to_dir"
  # isula cp b330e9be717a:/tmp/test_file_to_dir /tmp
  # ls -al /tmp | grep test_file_to_dir
  -rw-r-----    1 root     root             0 Apr 26 09:59 test_file_to_dir
  ```

* The **cp** command is used only for maintenance and fault locating. Do not use the **cp** command in the production environment.

### Example

Copy the  **/test/host**  directory on the host to the  **/test**  directory on container 21fac8bb9ea8.

```shell
isula cp /test/host 21fac8bb9ea8:/test
```

Copy the  **/www**  directory on container 21fac8bb9ea8 to the  **/tmp**  directory on the host.

```shell
isula cp 21fac8bb9ea8:/www /tmp/
```

## Pausing All Processes in a Container

### Description

The **isula pause** command is used to pause all processes in one or more containers.

### Usage

```shell
isula pause [OPTIONS] CONTAINER [CONTAINER...]
```

### Parameters

### Constraints

* Only containers in the running state can be paused.
* After a container is paused, other lifecycle management operations (such as  **restart**,  **exec**,  **attach**,  **kill**,  **stop**, and  **rm**) cannot be performed.
* After a container with health check configurations is paused, the container status changes to unhealthy.

### Example

Pause a running container.

```shell
# isula pause 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac
 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac
```

## Resuming All Processes in a Container

### Description

The **isula unpause** command is used to resume all processes in one or more containers. It is a reversible process of **isula pause**.

### Usage

```shell
isula unpause [OPTIONS] CONTAINER [CONTAINER...]
```

### Parameters

### Constraints

* Only containers in the paused state can be unpaused.

### Example

Resume a paused container.

```shell
# isula unpause 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac
 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac
```

## Obtaining Event Messages from the Server in Real Time

### **Description**

The **isula events** command is used to obtain real-time events from the server.

### Usage

```shell
isula events [OPTIONS]
```

### Parameter

### Constraints

* Support container-related events: create, start, restart, stop, exec\_create, exec\_die, attach, kill, top, rename, archive-path, extract-to-dir, update, pause, unpause, export, and resize.
* Supported image-related events: load, remove, pull, login, and logout.

### Example

Run the following command to obtain event messages from the server in real time:

```shell
# isula events
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/container_resource_management.md
---
# Container Resource Management

## Description

You can use namespaces and cgroups to manage container resources. iSula can use cgroup v1 and cgroup v2 to restrict resources. cgroup v2 is an experimental feature and cannot be put into commercial use. When the system is configured to support only cgroup v2 and cgroup v2 is mounted to the **/sys/fs/cgroup** directory, iSula uses cgroup v2 for resource management. Whether cgroup v1 or cgroup v2 is used to manage container resources, iSula provides the same interface for users to implement resource restriction.

## Sharing Resources

### Description

Containers or containers and hosts can share namespace information mutually, including PID, network, IPC, and UTS information.

> \[!NOTE] **NOTE:**
> When namespace information is shared with a host, the namespace isolation mechanism is unavailable. As a result, information on the host can be queried and operated in a container, causing security risks. For example, if **--pid=host** is used to share the PID namespace of a host, information about other processes on the host can be viewed, causing information leakage or even killing the host process. Exercise caution when using the shared host namespace function to ensure security.

### Usage

When running the  **isula create/run**  command, you can set the namespace parameters to share resources. For details, see the following parameter description table.

### Parameters

You can specify the following parameters when running the  **isula create/run**  command:

### Example

If two containers need to share the same PID namespace, add  **--pid container:\<containerID>**  when running the container. For example:

```shell
isula run -tid --name test_pid busybox sh
isula run -tid --name test --pid container:test_pid busybox sh
```

## Restricting CPU Resources of a Running Container

### Description

You can set parameters to restrict the CPU resources of a container.

### Usage

When running the  **isula create/run**  command, you can set CPU-related parameters to limit the CPU resources of a container. For details about the parameters and values, see the following table.

### Parameters

You can specify the following parameters when running the  **isula create/run**  command:

### Example

To restrict a container to use a specific CPU, add  **--cpuset-cpus number**  when running the container. For example:

```shell
isula run -tid --cpuset-cpus 0,2-3 busybox sh
```

> \[!NOTE] **NOTE:**
> You can check whether the configuration is successful. For details, see "Querying Information About a Single Container."

## Restricting the Memory Usage of a Running Container

### Description

You can set parameters to restrict the memory usage of a container.

### Usage

When running the  **isula create/run**  command, you can set memory-related parameters to restrict memory usage of containers. For details about the parameters and values, see the following table.

### Parameters

You can specify the following parameters when running the  **isula create/run**  command:

### Example

To set the upper limit of the memory of a container, add  **--memory \<number>\[\<unit>]**  when running the container. For example:

```shell
isula run -tid --memory 1G busybox sh
```

## Restricting I/O Resources of a Running Container

### Description

You can set parameters to limit the read/write speed of devices in the container.

### Usage

When running the  **isula create/run**  command, you can set  **--device-read-bps/--device-write-bps \<device-path>:\<number>\[\<unit>]**  to limit the read/write speed of devices in the container.

### Parameters

When running the  **isula create/run**  command, set  **--device-read/write-bps**.

### Example

To limit the read/write speed of devices in the container, add  **--device-write-bps/--device-read-bps \<device-path>:\<number>\[\<unit>]**  when running the container. For example, to limit the read speed of the device  **/dev/sda**  in the container  **busybox**  to 1 Mbit/s, run the following command:

```shell
isula run -tid --device-read-bps /dev/sda:1mb busybox sh
```

To limit the write speed, run the following command:

```shell
isula run -tid --device-write-bps /dev/sda:1mb busybox sh
```

## Restricting the Rootfs Storage Space of a Container

### Description

When the overlay2 storage driver is used on the EXT4 file system, the file system quota of a single container can be set. For example, the quota of container A is set to 5 GB, and the quota of container B is set to 10 GB.

This feature is implemented by the project quota function of the EXT4 file system. If the kernel supports this function, use the syscall SYS\_IOCTL to set the project ID of a directory, and then use the syscall SYS\_QUOTACTL to set the hard limit and soft limit of the corresponding project ID.

### Usage

1. Prepare the environment.

   Ensure that the file system supports the  **Project ID**  and  **Project Quota**  attributes, the kernel version is 4.19 or later, and the version of the peripheral package e2fsprogs is 1.43.4-2 or later.

2. Before mounting overlayfs to a container, set different project IDs for the upper and work directories of different containers and set inheritance options. After overlayfs is mounted to a container, the project IDs and inherited attributes cannot be modified.

3. Set the quota as a privileged user outside the container.

4. Add the following configuration to daemon:

   ```shell
   -s overlay2 --storage-opt overlay2.override_kernel_check=true
   ```

5. Daemon supports the following options for setting default restrictions for containers:

   **--storage-opt overlay2.basesize=128M**  specifies the default limit. If  **--storage-opt size**  is also specified when you run the  **isula run**  command, the value of this parameter takes effect. If no size is specified during the daemon process or when you run the  **isula run**  command, the size is not limited.

6. Enable the  **Project ID**  and  **Project Quota**  attributes of the file system.
   * Format and mount the file system.

     ```shell
     mkfs.ext4 -O quota,project /dev/sdb
     mount -o prjquota /dev/sdb /var/lib/isulad
     ```

### Parameters

When running the  **create/run**  command, set  **--storage-opt**.

### Example

In the  **isula run/create**  command, use the existing parameter  **--storage-opt size=***value*  to set the quota. The value is a positive number in the unit of  **\[kKmMgGtTpP]?\[iI]?\[bB]?**. If the value does not contain a unit, the default unit is byte.

```console
$ [root@localhost ~]# isula run -ti --storage-opt size=10M busybox
/ # df -h
Filesystem                Size      Used Available Use% Mounted on
overlay                  10.0M     48.0K     10.0M   0% /
none                     64.0M         0     64.0M   0% /dev
none                     10.0M         0     10.0M   0% /sys/fs/cgroup
tmpfs                    64.0M         0     64.0M   0% /dev
shm                      64.0M         0     64.0M   0% /dev/shm
/dev/mapper/vg--data-ext41
                          9.8G     51.5M      9.2G   1% /etc/hostname
/dev/mapper/vg--data-ext41
                          9.8G     51.5M      9.2G   1% /etc/resolv.conf
/dev/mapper/vg--data-ext41
                          9.8G     51.5M      9.2G   1% /etc/hosts
tmpfs                     3.9G         0      3.9G   0% /proc/acpi
tmpfs                    64.0M         0     64.0M   0% /proc/kcore
tmpfs                    64.0M         0     64.0M   0% /proc/keys
tmpfs                    64.0M         0     64.0M   0% /proc/timer_list
tmpfs                    64.0M         0     64.0M   0% /proc/sched_debug
tmpfs                     3.9G         0      3.9G   0% /proc/scsi
tmpfs                    64.0M         0     64.0M   0% /proc/fdthreshold
tmpfs                    64.0M         0     64.0M   0% /proc/fdenable
tmpfs                     3.9G         0      3.9G   0% /sys/firmware
/ # 
/ # dd if=/dev/zero of=/home/img bs=1M count=12 && sync
dm-4: write failed, project block limit reached.
10+0 records in
9+0 records out
10432512 bytes (9.9MB) copied, 0.011782 seconds, 844.4MB/s
/ # df -h | grep overlay
overlay                  10.0M     10.0M         0 100% /
/ # 
```

### Constraints

1. The quota applies only to the rw layer.

   The quota of overlay2 is for the rw layer of the container. The image size is not included.

2. The kernel supports and enables this function.

   The kernel must support the EXT4 project quota function. When running  **mkfs**, add  **-O quota,project**. When mounting the file system, add  **-o prjquota**. If any of the preceding conditions is not met, an error is reported when  **--storage-opt size=***value*  is used.

   ```console
   $ [root@localhost ~]# isula run -it --storage-opt size=10Mb busybox df -h
   Error response from daemon: Failed to prepare rootfs with error: time="2019-04-09T05:13:52-04:00" level=fatal msg="error creating read-
   write layer with ID "a4c0e55e82c55e4ee4b0f4ee07f80cc2261cf31b2c2dfd628fa1fb00db97270f": --storage-opt is supported only for overlay over
   xfs or ext4 with 'pquota' mount option"
   ```

3. Description of the limit of quota:
   1. If the quota is greater than the size of the partition where user  **root**  of iSulad is located, the file system quota displayed by running the  **df**  command in the container is the size of the partition where user  **root**  of iSulad is located, not the specified quota.
   2. **--storage-opt size=0**  indicates that the size is not limited and the value cannot be less than 4096. The precision of size is one byte. If the specified precision contains decimal bytes, the decimal part is ignored. For example, if size is set to  **0.1**, the size is not limited. (The value is restricted by the precision of the floating point number stored on the computer. That is, 0.999999999999999999999999999 is equal to 1. The number of digits 9 may vary according to computers. Therefore, 4095.999999999999999999999999999 is equal to 4096.) Note that running  **isula inspect**  displays the original command line specified format. If the value contains decimal bytes, you need to ignore the decimal part.
   3. If the quota is too small, for example,**--storage-opt size=4k**, the container may fail to be started because some files need to be created for starting the container.
   4. The  **-o prjquota**  option is added to the root partition of iSulad when iSulad is started last time. If this option is not added during this startup, the setting of the container with quota created during the last startup does not take effect.
   5. The value range of the daemon quota  **--storage-opt overlay2.basesize**  is the same as that of  **--storage-opt size**.

4. When  **storage-opt**  is set to 4 KB, the lightweight container startup is different from that of Docker.

   Use the  **storage-opt size=4k**  and image  **rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest**  to run the container.

   Docker fails to be started.

   ```console
   [root@localhost ~]# docker run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest
   docker: Error response from daemon: symlink /proc/mounts /var/lib/docker/overlay2/e6e12701db1a488636c881b44109a807e187b8db51a50015db34a131294fcf70-init/merged/etc/mtab: disk quota exceeded.
   See 'docker run --help'.
   ```

   The lightweight container is started properly and no error is reported.

   ```console
   [root@localhost ~]# isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest
   636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728
   [root@localhost ~]# isula ps
   STATUS  PID   IMAGE                                                 COMMAND   EXIT_CODE RESTART_COUNT STARTAT       FINISHAT RUNTIME ID           NAMES                                                            
   running 17609 rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest /bin/bash 0         0             2 seconds ago -        runc     636480b1fc2c 636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728 
   ```

   During container startup, if you need to create a file in the  **rootfs**  directory of the container, the image size exceeds 4 KB, and the quota is set to 4 KB, the file creation will fail.

   When Docker starts the container, it creates more mount points than iSulad to mount some directories on the host to the container, such as  **/proc/mounts**  and  **/dev/shm**. If these files do not exist in the image, the creation will fail, therefore, the container fails to be started.

   When a lightweight container uses the default configuration during container startup, there are few mount points. The lightweight container is created only when the directory like  **/proc**  or  **/sys**  does not exist. The image  **rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest**  in the test case contains  **/proc**  and  **/sys**. Therefore, no new file or directory is generated during the container startup. As a result, no error is reported during the lightweight container startup. To verify this process, when the image is replaced with  **rnd-dockerhub.huawei.com/official/busybox-aarch64:latest**, an error is reported when the lightweight container is started because  **/proc**  does not exist in the image.

   ```console
   [root@localhost ~]# isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/busybox-aarch64:latest
   8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4
   Error response from daemon: Start container error: runtime error: 8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4:tools/lxc_start.c:main:404 starting container process caused "Failed to setup lxc,
   please check the config file."
   ```

5. Other description:

   When using iSulad with the quota function to switch data disks, ensure that the data disks to be switched are mounted using the  **prjquota**  option and the mounting mode of the  **/var/lib/isulad/storage/overlay2**  directory is the same as that of the  **/var/lib/isulad**  directory.

   > \[!NOTE] **NOTE:**
   > Before switching the data disk, ensure that the mount point of  **/var/lib/isulad/storage/overlay2**  is unmounted.

## Restricting the Number of File Handles in a Container

### Description

You can set parameters to limit the number of file handles that can be opened in a container.

### Usage

When running the  **isula create/run**  command, set the  **--files-limit**  parameter to limit the number of file handles that can be opened in a container.

### Parameters

Set the  **--files-limit**  parameter when running the  **isula create/run**  command.

### Example

When running the container, add  **--files-limit n**. For example:

```shell
isula run -ti --files-limit 1024 busybox bash
```

### Constraints

1. If the  **--files-limit**  parameter is set to a small value, for example, 1, the container may fail to be started.

   ```console
   [root@localhost ~]# isula run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64
   004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4
   Error response from daemon: Start container error: Container is not running:004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4
   ```

   Docker will be started successfully, and the value of  **files.limit cgroup**  is  **max**.

   ```console
   [root@localhost ~]# docker run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64
   ef9694bf4d8e803a1c7de5c17f5d829db409e41a530a245edc2e5367708dbbab
   [root@localhost ~]# docker exec -it ef96 cat /sys/fs/cgroup/files/files.limit
   max
   ```

   The root cause is that the startup principles of the lxc and runc processes are different. After the lxc process creates the cgroup, the files.limit value is set, and then the PID of the container process is written into the cgroup.procs file of the cgroup. At this time, the process has opened more than one handle. As a result, an error is reported, and the startup fails. After you create a cgroup by running the  **runc**  command, the PID of the container process is written to the cgroup.procs file of the cgroup, and then the files.limit value is set. Because more than one handle is opened by the process in the cgroup, the file.limit value does not take effect, the kernel does not report any error, and the container is started successfully.

## Restricting the Number of Processes or Threads that Can Be Created in a Container

### Description

You can set parameters to limit the number of processes or threads that can be created in a container.

### Usage

When creating or running a container, use the  **--pids-limit**  parameter to limit the number of processes or threads that can be created in the container.

### Parameters

When running the  **create/run**  command, set the  **--pids-limit**  parameter.

### Example

When running the container, add  **--pids-limit n**. For example:

```shell
isula run -ti --pids-limit 1024 busybox bash
```

### Constraints

During container creation, some processes are created temporarily. Therefore, the value cannot be too small. Otherwise, the container may fail to be started. It is recommended that the value be greater than 10.

## Configuring the ulimit Value in a Container

### Description

You can use parameters to control the resources for executed programs.

### Usage

Set the  **--ulimit**  parameter when creating or running a container, or configure the parameter on the daemon to control the resources for executed programs in the container.

### Parameters

Use either of the following methods to configure ulimit:

1. When running the  **isula create/run**  command, use  **--ulimit \<type>=\<soft>\[:\<hard>]**  to control the resources of the executed shell program.

2. Use daemon parameters or configuration files.

   For details, see **--default-ulimits** in  [Configuration Mode](./installation_configuration.md#configuration-mode).

   **--ulimit**  can limit the following types of resources:

### Example

When creating or running a container, add  **--ulimit \<type>=\<soft>\[:\<hard>]**. For example:

```shell
isula create/run -tid --ulimit nofile=1024:2048 busybox sh
```

### Constraints

The ulimit cannot be configured in the  **daemon.json**  and  **/etc/sysconfig/iSulad**  files (or the iSulad command line). Otherwise, an error is reported when iSulad is started.

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/contribution.md
---
# Contribution

As an openEuler user, you can contribute to the openEuler community in multiple ways. For details about how to contribute to the community, see  [How to Contribute](https://www.openeuler.org/en/community/contribution/). Here, some methods are listed for reference.

## Special Interest Groups (SIGs)

openEuler brings together people of common interest to form different special interest groups (SIGs). For details about existing SIGs, see the  [SIG list](https://www.openeuler.org/en/sig/sig-list/).

You are welcome to join an existing SIG or create a SIG. For details about how to create a SIG, see the  [SIG Management Procedure](https://atomgit.com/openeuler/community/blob/master/en/technical-committee/governance/).

## Mail List and Tasks

You are welcome to actively help users solve problems raised in the  [mail list](https://www.openeuler.org/en/community/mailing-list/)  and issues (including  [code repository issues](https://gitee.com/organizations/openeuler/issues)  and  [software package repository issues](https://gitee.com/organizations/src-openeuler/issues)). In addition, you can submit an issue. All these will help the openEuler community to develop better.

## Documents

You can contribute to the community by submitting code. We also welcome your feedback on problems and difficulties, or suggestions on improving the usability and integrity of documents. For example, problems in obtaining software or documents and difficulties in using the system. Welcome to pay attention to and improve the documentation module of the  [openEuler community](https://openeuler.org/en/).

## IRC

openEuler has also opened a channel in IRC as an additional channel to provide community support and interaction. For details, see  [openEuler IRC](https://atomgit.com/openeuler/community/tree/master/en/communication).

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/maintenance/syssentry/cpu_fault_inspection_plugin.md
---
# CPU故障巡检插件

## 硬件规格要求

* 仅支持aarch64架构

## 安装插件

### 前置条件

已通过《[安装和使用](./installation_and_usage.md)》安装sysSentry巡检框架。

### 安装软件包

```sh
yum install cpu_sentry ipmitool libxalarm -y
```

### 加载内核模块

```sh
modprobe cpu_inspect
modprobe inspector-atf
```

## CPU巡检参数配置

cpu故障巡检任务的配置保存在/etc/sysSentry/plugins/cpu\_sentry.ini中。

* 配置项说明

| 插件配置参数  | 默认值  | 取值范围                    | 配置项说明                      |
| ------------- | ------- | --------------------------- | ------------------------------- |
| cpu\_list      | default | 测试环境CPU核ID列表         | 进行故障巡检的CPU列表           |
| patrol\_second | 60      | 大于0的整数                 | 巡检超时时间，单位：s           |
| cpu\_utility   | 100     | 1-100之间的整数（含1和100） | 允许巡检程序运行的CPU最大利用率 |

* 配置示例

```ini
[args]
cpu_list = default
patrol_second = 60
cpu_utility = 100
```

## 加载CPU巡检任务

启动CPU巡检任务之前，需要先加载CPU巡检任务。执行 `sentryctl reload cpu_sentry` 加载CPU巡检任务。
![输入图片说明](figures/load_cpu_sentry.png)

## 启动CPU巡检任务

执行 `sentryctl start cpu_sentry` 启动CPU巡检任务。

## 查看CPU巡检状态

执行 `sentryctl status cpu_sentry` 查看CPU巡检任务状态。

## 停止CPU巡检任务

执行 `sentryctl stop cpu_sentry` 停止正在运行的CPU巡检任务。

## 查看CPU巡检结果

巡检结果获取命令：`sentryctl get_result cpu_sentry`， 命令回显信息参考get\_result接口说明，details信息如下：

```json
{
    ... ...
    "details": {
        "code":0,
        "msg":"xxx",
        "isolated_cpulist":"xxx"
    }
}
```

details各个字段含义如下：

| key              | 含义                                                         |
| ---------------- | ------------------------------------------------------------ |
| code             | cpu巡检任务返回错误码，整型。错误码及可能的原因如下：  0：所有CPU未发现问题    1001：0号CPU有问题，无法隔离    1002： 部分CPU有问题，故障核隔离成功    1003：无效的配置参数    1004：巡检程序执行出错    1005：巡检程序被杀死 |
| msg              | 字符串，错误描述信息                                         |
| isolated\_cpulist | 字符串，故障隔离核列表，例如："10-13,15-18"                  |

比如配置patrol\_second值为-1，启动巡检任务，得到结果如下：
![输入图片说明](figures/result-invalid-params.png)

## 查看CPU巡检日志

CPU巡检任务的日志记录在 /var/log/sysSentry/cpu\_sentry.log 文件中，主要记录错误信息，比如配置文件中配置非法的参数值，会得到如下的日志内容：

```sh
[root@localhost ~]# cat /var/log/sysSentry/cpu_sentry.log
ERROR:root:config 'cpu_list' (value [1--3]) is invalid in cpu_sentry.ini !
[root@localhost ~]#
```

## 常见问题Q\&A

* 在配置无效参数时，调用`sentryctl start cpu_sentry`命令，未执行cpu巡检，但是该命令的退出状态为0，说明命令执行成功。这是否正常，为什么？

  sentryctl命令用于管理巡检任务，它的退出状态与巡检任务执行的执行状态无关。sentryctl进程与巡检任务进程是异步执行的，互不影响。用户执行`sentryctl start cpu_sentry`是下发cpu巡检任务，该命令退出状态为0表示任务下发成功，但是不能说明CPU巡检任务执行成功。在执行CPU故障巡检之前，巡检任务会检测任务配置参数是否合法。如果CPU巡检任务配置文件中有非法参数，巡检任务会直接退出，检查结果体现在巡检任务执行结果中，我们可以执行`sentryctl get_result cpu_sentry`获取巡检任务执行结果，如下图所示：\
  ![输入图片说明](figures/return-code-invalid-params.png)

* 插入inspector-atf模块失败，提示`Operation not supported`，这是为什么？

  这是因为当前BIOS版本不支持CPU故障巡检，查看dmesg日志可以得到相关日志信息，比如：

![输入图片说明](figures/bios-not-support-cpu-inspec.PNG)

* 巡检结果中的end\_time - start\_time 的差值会超过1s，这是为什么？

  CPU巡检只允许同时运行一个任务，因此在拉起CPU巡检任务之前会调用pgrep命令检测测试环境中是否有已经运行的CPU巡检进程，而不同环境中pgrep命令耗时有差异，end\_time - start\_time 的超时时间主要为pgrep命令的耗时。

---

---
url: /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/cri.md
---
# CRI

## Description

The Container Runtime Interface (CRI) provided by Kubernetes defines container and image service APIs. iSulad uses the CRI to interconnect with Kubernetes.

Since the container runtime is isolated from the image lifecycle, two services need to be defined. This API is defined by using  [Protocol Buffer](https://developers.google.com/protocol-buffers/)  based on  [gRPC](https://grpc.io/).

The current CRI version is v1alpha1. For official API description, access the following link:

<https://github.com/kubernetes/kubernetes/blob/release-1.14/pkg/kubelet/apis/cri/runtime/v1alpha2/api.proto>

iSulad uses the API description file of version 1.14 used by Pass, which is slightly different from the official API description file. API description in this document prevails.

> \[!NOTE] **NOTE:**\
> The listening IP address of the CRI WebSocket streaming service is  **127.0.0.1**  and the port number is  **10350**. The port number can be configured in the  **--websocket-server-listening-port**  command or in the  **daemon.json**  configuration file.

## APIs

The following tables list the parameters that may be used in each API. Some parameters do not take effect now, which have been noted in the corresponding parameter description.

### API Parameters

* **DNSConfig**

  This API is used to configure DNS servers and search domains of a sandbox.

* **Protocol**

  This API is used to specify enum values of protocols.

* **PortMapping**

  This API is used to configure the port mapping for a sandbox.

  | **参数成员**         | **描述**           |
  |----------------------|--------------------|
  | Protocol protocol    | Protocol used for port mapping. |
  | int32 container\_port | Port number in the container.    |
  | int32 host\_port      | Port number on the host.    |
  | string host\_ip       | Host IP address.         |

* **MountPropagation**

  This API is used to specify enums of mount propagation attributes.

* **Mount**

  This API is used to mount a volume on the host to a container. (Only files and folders are supported.)

  | **Parameter**                 | **Description**                                                                        |
  |------------------------------|---------------------------------------------------------------------------------|
  | string container\_path        | Path in the container.                                                                    |
  | string host\_path             | Path on the host.                                                                    |
  | bool readonly                | Whether the configuration is read-only in the container. Default value: **false**                                       |
  | bool selinux\_relabel         | Whether to set the SELinux label. This parameter does not take effect now.                                               |
  | MountPropagation propagation | Mount propagation attribute. The value can be **0**, **1**, or **2**, corresponding to the **private**, **rslave**, or **rshared** propagation attributes, respectively. The default value i **0**. |

* **NamespaceOption**

* **Capability**

  This API is used to specify the capabilities to be added and deleted.

* **Int64Value**

  This API is used to encapsulate data of the signed 64-bit integer type.

* **UInt64Value**

  This API is used to encapsulate data of the unsigned 64-bit integer type.

* **LinuxSandboxSecurityContext**

  This API is used to configure the Linux security options of a sandbox.

  Note that these security options are not applied to containers in the sandbox, and may not be applied to the sandbox without any running process.

  | **参数成员**                       | **描述**                                                                                                                                                                                       |
  |------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | NamespaceOption namespace\_options  | Sandbox namespace options.                                                                                                                                                                      |
  | SELinuxOption selinux\_options      | SELinux options. This parameter does not take effect now.                                                                                                                                                                     |
  | Int64Value run\_as\_user             | Process UID in the sandbox.                                                                                                                                                                         |
  | bool readonly\_rootfs               | Whether the root file system of the sandbox is read-only.                                                                                                                                                                |
  | repeated int64 supplemental\_groups | Information of the user group of the init process in the sandbox (except the primary GID).                                                                                                                                                    |
  | bool privileged                    | Whether the sandbox is a privileged container.                                                                                                                                                                     |
  | string seccomp\_profile\_path        | Path to the seccomp configuration file. Valid values are as follows: **// unconfined**: seccomp is not configured.   **// localhost/**\<Full path to the configuration file> // \<Full path to the configuration file> // **// unconfined** is the default value. |

* **LinuxPodSandboxConfig**

  This API is used to configure information related to the Linux host and containers.

  | **参数成员**                                 | **描述**                                                                                |
  |----------------------------------------------|-----------------------------------------------------------------------------------------|
  | string cgroup\_parent                         | Parent path of the cgroup of the sandbox. The runtime can use the cgroupfs or systemd syntax based on site requirements. This parameter does not take effect now. |
  | LinuxSandboxSecurityContext security\_context | Security attribute of the sandbox.                                                                       |
  | map\<string, string> sysctls                  | Linux sysctls configuration of the sandbox.                                                              |

* **PodSandboxMetadata**

  Sandbox metadata contains all information that constructs a sandbox name. It is recommended that the metadata be displayed on the user interface during container running to improve user experience. For example, a unique sandbox name can be generated based on the metadata during running.

* **PodSandboxConfig**

  This API is used to specify all mandatory and optional configurations for creating a sandbox.

  | **参数成员**                       | **描述**                                                                                                                                            |
  |------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
  | PodSandboxMetadata metadata        | Sandbox metadata, which uniquely identifies a sandbox. The runtime must use the information to ensure that operations are correctly performed, and to improve user experience, for example, construct a readable sandbox name. |
  | string hostname                    | Host name of the sandbox.                                                                                                                                  |
  | string log\_directory               | Folder for storing container log files in the sandbox.                                                                                                       |
  | DNSConfig dns\_config               | Sandbox DNS configuration.                                                                                                                                   |
  | repeated PortMapping port\_mappings | Sandbox port mapping.                                                                                                                                   |
  | map\<string, string> labels         | Key-value pair that can be used to identify a sandbox or a series of sandboxes.                                                                                                               |
  | map\<string, string> annotations    | /a>Key-value pair that stores any information, whose values cannot be changed and can be queried by using the **PodSandboxStatus** API.                                                                        |
  | LinuxPodSandboxConfig linux        | Options related to the Linux host.                                                                                                                             |

* **PodSandboxNetworkStatus**

  This API is used to describe the network status of a sandbox.

* **Namespace**

  This API is used to set namespace options.

  | **参数成员**            | **描述**           |
  |-------------------------|--------------------|
  | NamespaceOption options | Linux namespace options. |

* **LinuxPodSandboxStatus**

  This API is used to describe the status of a Linux sandbox.

  | **参数成员**         | **描述**        |
  |----------------------|-----------------|
  | Namespace **namespaces** | Sandbox namespace. |

* **PodSandboxState**

  This API is used to specify enum data of the sandbox status values.

* **PodSandboxStatus**

  This API is used to describe the PodSandbox status.

  | **Parameter**                              | **Description**                                          |
  |-------------------------------------------|---------------------------------------------------|
  | string id                                 | Sandbox ID.                                       |
  | PodSandboxMetadata metadata               | Sandbox metadata.                                   |
  | PodSandboxState state                     | Sandbox status value.                                   |
  | int64 created\_at                          | Sandbox creation timestamp (unit: ns).                     |
  | repeated PodSandboxNetworkStatus networks | Multi-plane network status of the sandbox.                           |
  | LinuxPodSandboxStatus linux               | Sandbox status complying with the Linux specifications.                            |
  | map\<string, string> labels                | Key-value pair that can be used to identify a sandbox or a series of sandboxes.             |
  | map\<string, string> annotations           | Key-value pair that stores any information, whose values cannot be changed by the runtime. |

* **PodSandboxStateValue**

  This API is used to encapsulate **PodSandboxState**.

  | **Parameter**          | **Description**        |
  |-----------------------|-----------------|
  | PodSandboxState state | Sandbox status value. |

* **PodSandboxFilter**

  This API is used to add filter criteria for the sandbox list. The intersection of multiple filter criteria is displayed.

  | **Parameter**                       | **Description**                                             |
  |------------------------------------|------------------------------------------------------|
  | string id                          | Sandbox ID.                                          |
  | PodSandboxStateValue state         | Sandbox status value.                                        |
  | map\<string, string> label\_selector | /a>Sandbox label, which does not support regular expressions and must be fully matched. |

* **PodSandbox**

  This API is used to provide a minimum description of a sandbox.

  | **Parameter**                    | **Description**                                          |
  |---------------------------------|---------------------------------------------------|
  | string id                       | Sandbox ID.                                       |
  | PodSandboxMetadata metadata     | Sandbox metadata.                                   |
  | PodSandboxState state           | Sandbox status value.                                   |
  | int64 created\_at                | Sandbox creation timestamp (unit: ns).                    |
  | map\<string, string> labels      | Key-value pair that can be used to identify a sandbox or a series of sandboxes.             |
  | map\<string, string> annotations | Key-value pair that stores any information, whose values cannot be changed by the runtime. |

* **KeyValue**

  This API is used to encapsulate key-value pairs.

* **SELinuxOption**

  This API is used to specify the SELinux label of a container.

* **ContainerMetadata**

  Container metadata contains all information that constructs a container name. It is recommended that the metadata be displayed on the user interface during container running to improve user experience. For example, a unique container name can be generated based on the metadata during running.

* **ContainerState**

  This API is used to specify enums of container status values.

* **ContainerStateValue**

  This API is used to encapsulate the data structure of **ContainerState**.

  | **参数成员**         | **描述**   |
  |----------------------|------------|
  | ContainerState **state** | Container status value. |

* **ContainerFilter**

  This API is used to add filter criteria for the container list. The intersection of multiple filter criteria is displayed.

  | **参数成员**                       | **描述**                                               |
  |------------------------------------|--------------------------------------------------------|
  | string id                          | Container ID.                                         |
  | PodSandboxStateValue state         | Container status.                                        |
  | string pod\_sandbox\_id              | Sandbox ID.                                            |
  | map\<string, string> label\_selector | Container label, which does not support regular expressions and must be fully matched. |

* **LinuxContainerSecurityContext**

  This API is used to specify container security configurations.

  | **参数成员**                       | **描述**                                                                                                                           |
  |------------------------------------|------------------------------------------------------------------------------------------------------------------------------------|
  | Capability capabilities            | Added or removed capabilities.                                                                                                                   |
  | bool privileged                    | Whether the container is in privileged mode. Default value: **false**                                                                                             |
  | NamespaceOption namespace\_options  | Container namespace options.                                                                                                            |
  | SELinuxOption selinux\_options      | SELinux context, which is optional. This parameter does not take effect now.                                                                                               |
  | Int64Value run\_as\_user             | UID for running container processes. Only **run\_as\_user** or **run\_as\_username** can be specified at a time.  **run\_as\_username** is preferred.                                      |
  | string run\_as\_username             | Username for running container processes. If specified, the user must exist in **/etc/passwd** in the container image and be parsed by the runtime. Otherwise, an error must occur during running. |
  | bool readonly\_rootfs               | Whether the root file system in a container is read-only. The default value is configured in **config.json**.                                                                             |
  | repeated int64 supplemental\_groups | List of user groups of the init process running in the container (except the primary GID).                                                                                                  |
  | string apparmor\_profile            | AppArmor configuration file of the container. This parameter does not take effect now.                                                                                                    |
  | string seccomp\_profile\_path        | Path to the seccomp configuration file of the container.                                                                                                         |
  | bool no\_new\_privs                  | Whether to set the **no\_new\_privs** flag in the container.                                                                                                |

* **LinuxContainerResources**

  This API is used to specify configurations of Linux container resources.

* **Image**

  This API is used to describe the basic information about an image.

  | **参数成员**                 | **描述**               |
  |------------------------------|------------------------|
  | string id                    | Image ID.                |
  | repeated string repo\_tags    | Image tag **repo\_tags**. |
  | repeated string repo\_digests | Image digest information.         |
  | uint64 size                  | Image size.               |
  | Int64Value uid               | Default image UID.        |
  | string username              | Default image user name.       |

* **ImageSpec**

  This API is used to represent the internal data structure of an image. Currently, ImageSpec encapsulates only the container image name.

* **StorageIdentifier**

  This API is used to specify the unique identifier for defining the storage.

* **FilesystemUsage**

  | **参数成员**                 | **描述**                   |
  |------------------------------|----------------------------|
  | int64 timestamp              | Timestamp when file system information is collected. |
  | StorageIdentifier storage\_id | UUID of the file system that stores images.     |
  | UInt64Value used\_bytes       | Size of the metadata that stores images.      |
  | UInt64Value inodes\_used      | Number of inodes of the metadata that stores images. |

* **AuthConfig**

* **Container**

  This API is used to describe container information, such as the ID and status.

  | **参数成员**                    | **描述**                                                    |
  |---------------------------------|-------------------------------------------------------------|
  | string id                       | Container ID.                                               |
  | string pod\_sandbox\_id           | ID of the sandbox to which the container belongs.                                     |
  | ContainerMetadata metadata      | Container metadata.                                           |
  | ImageSpec image                 | Image specifications.                                                    |
  | string image\_ref                | Image used by the container. This parameter is an image ID for most runtime. |
  | ContainerState state            | Container status.                                             |
  | int64 created\_at                | Container creation timestamp (unit: ns).                           |
  | map\<string, string> labels      | Key-value pair that can be used to identify a container or a series of containers.                     |
  | map\<string, string> annotations | Key-value pair that stores any information, whose values cannot be changed by the runtime.           |

* **ContainerStatus**

  This API is used to describe the container status information.

  | **参数成员**                    | **描述**                                                                  |
  |---------------------------------|---------------------------------------------------------------------------|
  | string id                       | Container ID.                                                             |
  | ContainerMetadata metadata      | Container metadata.                                                         |
  | ContainerState state            | Container status.                                                           |
  | int64 created\_at                | Container creation timestamp (unit: ns).                                         |
  | int64 started\_at                | Container start timestamp (unit: ns).                                       |
  | int64 finished\_at               | Container exit timestamp (unit: ns).                                       |
  | int32 exit\_code                 | Container exit code.                                                                |
  | ImageSpec image                 | Image specifications.                                                                 |
  | string image\_ref                | Image used by the container. This parameter is an image ID for most runtime.               |
  | string reason                   | Brief description of the reason why the container is in the current status.                                            |
  | string message                  | Information that is easy to read and indicates the reason why the container is in the current status.                        |
  | map\<string, string> labels      | Key-value pair that can be used to identify a container or a series of containers.                                   |
  | map\<string, string> annotations | Key-value pair that stores any information, whose values cannot be changed by the runtime.                         |
  | repeated Mount mounts           | Information about the container mount point.                                                         |
  | string log\_path                 | Path to the container log file in the **log\_directory** folder configured in **PodSandboxConfig**. |

* **ContainerStatsFilter**

  This API is used to add filter criteria for the container stats list. The intersection of multiple filter criteria is displayed.

* **ContainerStats**

  This API is used to add filter criteria for the container stats list. The intersection of multiple filter criteria is displayed.

  | **参数成员**                   | **描述**       |
  |--------------------------------|----------------|
  | ContainerAttributes attributes | Container information.     |
  | CpuUsage cpu                   | CPU usage information.   |
  | MemoryUsage memory             | Memory usage information.  |
  | FilesystemUsage writable\_layer | Information about the writable layer usage. |

* **ContainerAttributes**

  This API is used to list basic container information.

  | **参数成员**                   | **描述**                                          |
  |--------------------------------|---------------------------------------------------|
  | string id                      | Container ID.                                          |
  | ContainerMetadata metadata     | Container metadata.                                    |
  | map\<string,string> labels      | Key-value pair that can be used to identify a container or a series of containers.           |
  | map\<string,string> annotations | Key-value pair that stores any information, whose values cannot be changed by the runtime. |

* **CpuUsage**

  This API is used to list the CPU usage information of a container.

* **MemoryUsage**

  This API is used to list the memory usage information of a container.

* **FilesystemUsage**

  This API is used to list the read/write layer information of a container.

* **Device**

  This API is used to specify the host volume to be mounted to a container.

* **LinuxContainerConfig**

  This API is used to specify Linux configurations.

  | **Parameter**                                   | **Description**                |
  |------------------------------------------------|-------------------------|
  | LinuxContainerResources resources              | Container resource specifications.          |
  | LinuxContainerSecurityContext security\_context | Linux container security configuration. |

* **ContainerConfig**

  This API is used to specify all mandatory and optional fields for creating a container.

  | **Parameter**                    | **Description**                                                                                                                                             |
  |---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
  | ContainerMetadata metadata      | Container metadata. The information will uniquely identify a container and should be used at runtime to ensure correct operations. The information can also be used at runtime to optimize the user experience (UX) design, for example, construct a readable name. This parameter is mandatory. |
  | ImageSpec image                 | Image used by the container. This parameter is mandatory.                                                                                                                               |
  | repeated string command         | Command to be executed. Default value: **/bin/sh**                                                                                                                      |
  | repeated string args            | Parameters of the command to be executed.                                                                                                                                    |
  | string working\_dir              | Current working directory of the command.                                                                                                                               |
  | repeated KeyValue envs          | Environment variables configured in the container.                                                                                                                               |
  | repeated Mount mounts           | Information about the mount point to be mounted in the container.                                                                                                                           |
  | repeated Device devices         | Information about the device to be mapped in the container.                                                                                                                             |
  | map\<string, string> labels      | Key-value pair that can be used to index and select a resource.                                                                                                                   |
  | map\<string, string> annotations | Unstructured key-value mappings that can be used to store and retrieve any metadata.                                                                                                     |
  | string log\_path                 | Relative path to **PodSandboxConfig.LogDirectory**, which is used to store logs (STDOUT and STDERR) on the container host.                                                              |
  | bool stdin                      | Whether to open **stdin** of the container.                                                                                                                                  |
  | bool stdin\_once                 | Whether to immediately disconnect other data flows connected with **stdin** when a data flow connected with **stdin** is disconnected. This parameter does not take effect now.                                                                      |
  | bool tty                        | Whether to use a pseudo terminal to connect to **stdio** of the container.                                                                                                                        |
  | LinuxContainerConfig linux      | lContainer configuration information in the Linux system.                                                                                                                       |

* **RuntimeConfig**

  This API is used to specify runtime network configurations.

  | **Parameter**                 | **Description**          |
  |------------------------------|-------------------|
  | NetworkConfig network\_config | Runtime network configurations. |

* **RuntimeCondition**

  This API is used to describe runtime status information.

* **RuntimeStatus**

  This API is used to describe runtime status.

### Runtime Service

The runtime service provides APIs for operating pods and containers, and APIs for querying the configuration and status information of the runtime service.

#### RunPodSandbox

#### Prototype

```text
rpc RunPodSandbox(RunPodSandboxRequest) returns (RunPodSandboxResponse) {}
```

#### Description

This API is used to create and start a PodSandbox. If the PodSandbox is successfully run, the sandbox is in the ready state.

#### Precautions

1. The default image for starting a sandbox is  **rnd-dockerhub.huawei.com/library/pause-${***machine***}:3.0**  where  **${***machine***}**  indicates the architecture. On x86\_64, the value of  *machine*  is  **amd64**. On ARM64, the value of  *machine*  is  **aarch64**. Currently, only the  **amd64**  or  **aarch64**  image can be downloaded from the rnd-dockerhub registry. If the image does not exist on the host, ensure that the host can download the image from the rnd-dockerhub registry. If you want to use another image, refer to  **pod-sandbox-image**  in the  *iSulad Deployment Configuration*.
2. The container name is obtained from fields in **PodSandboxMetadata** and separated by underscores (\_). Therefore, the metadata cannot contain underscores (\_). Otherwise, the **ListPodSandbox** API cannot be used for query even when the sandbox is running successfully.

#### Parameters

| **Parameter**            | **Description**                                                              |
|-------------------------|-----------------------------------------------------------------------|
| PodSandboxConfig config | Sandbox configuration.                                                         |
| string runtime\_handler  | Runtime for the created sandbox. Currently, lcr and kata-runtime are supported. |

#### Return Values

#### StopPodSandbox

#### Prototype

```text
rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse) {}
```

#### Description

This API is used to stop PodSandboxes and sandbox containers, and reclaim the network resources (such as IP addresses) allocated to a sandbox. If any running container belongs to the sandbox, the container must be forcibly stopped.

#### Parameters

#### Return Values

#### RemovePodSandbox

#### Prototype

```text
rpc RemovePodSandbox(RemovePodSandboxRequest) returns (RemovePodSandboxResponse) {}
```

#### Description

This API is used to delete a sandbox. If any running container belongs to the sandbox, the container must be forcibly stopped and deleted. If the sandbox has been deleted, no errors will be returned.

#### Precautions

1. When a sandbox is deleted, network resources of the sandbox are not deleted. Before deleting a pod, you must call StopPodSandbox to clear network resources. Ensure that StopPodSandbox is called at least once before deleting the sandbox.
2. If the container in a sandbox fails to be deleted when the sandbox is deleted, the sandbox is deleted but the container remains. In this case, you need to manually delete the residual container.

#### Parameters

#### Return Values

#### PodSandboxStatus

#### Prototype

```text
rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse) {}
```

#### Description

This API is used to query the sandbox status. If the sandbox does not exist, an error is returned.

#### Parameters

#### Return Values

| **Return Value**               | **Description**                                                                                                                                 |
|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------|
| PodSandboxStatus status  | Status of the sandbox.                                                                                                                        |
| map\<string, string> info | Additional information about the sandbox. The key can be any string, and the value is a JSON character string. The information can be any debugging content. When **verbose** is set to **true**, **info**  cannot be empty. This parameter does not take effect now. |

#### ListPodSandbox

#### Prototype

```text
rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse) {}
```

#### Description

This API is used to return the sandbox information list. Filtering based on criteria is supported.

#### Parameters

| **Parameter**            | **Description**     |
|-------------------------|--------------|
| PodSandboxFilter filter | Filter criteria. |

#### Return Values

| **Return Value**                | **Description**          |
|---------------------------|-------------------|
| repeated PodSandbox items | Sandbox information list. |

#### CreateContainer

```text
rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse) {}
```

#### Description

This API is used to create a container in the PodSandbox.

#### Precautions

* **sandbox\_config**  in**CreateContainerRequest**  is the same as the configuration transferred to  **RunPodSandboxRequest**  to create a PodSandbox. It is transferred again for reference only. PodSandboxConfig must remain unchanged throughout the lifecycle of a pod.
* The container name is obtained from fields in  **\[ContainerMetadata** and separated by underscores (\_). Therefore, the metadata cannot contain underscores (\_). Otherwise, the **ListContainers** API cannot be used for query even when the sandbox is running successfully.
* **CreateContainerRequest**  does not contain the  **runtime\_handler**  field. The runtime type of the container is the same as that of the corresponding sandbox.

#### Parameters

| **Parameter**                    | **Description**                           |
|---------------------------------|------------------------------------|
| string pod\_sandbox\_id           | ID of the PodSandbox where a container is to be created. |
| ContainerConfig config          | Container configuration information.                    |
| PodSandboxConfig sandbox\_config | PodSandbox configuration information.               |

#### Supplement

Unstructured key-value mappings that can be used to store and retrieve any metadata. The field can be used to transfer parameters for the fields for which the CRI does not provide specific parameters.

* Customize the field:

#### Return Values

#### StartContainer

#### Prototype

```text
rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {}
```

#### Description

This API is used to start a container.

#### Parameters

#### Return Values

#### StopContainer

#### Prototype

```text
rpc StopContainer(StopContainerRequest) returns (StopContainerResponse) {}
```

#### Description

This API is used to stop a running container. You can set a graceful timeout time. If the container has been stopped, no errors will be returned.

#### Parameters

#### Return Values

None

#### RemoveContainer

#### Prototype

```text
rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse) {}
```

#### Description

This API is used to delete a container. If the container is running, it must be forcibly stopped. If the container has been deleted, no errors will be returned.

#### Parameters

#### Return Values

None

#### ListContainers

#### Prototype

```text
rpc ListContainers(ListContainersRequest) returns (ListContainersResponse) {}
```

#### Description

This API is used to return the container information list. Filtering based on criteria is supported.

#### Parameters

| **Parameter**           | **Description**     |
|------------------------|--------------|
| ContainerFilter filter | Filter criteria. |

#### Return Values

| **Return Value**                    | **Description**       |
|-------------------------------|----------------|
| repeated Container containers | Container information list. |

#### ContainerStatus

#### Prototype

```text
rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse) {}
```

#### Description

This API is used to return the container status information. If the container does not exist, an error will be returned.

#### Parameters

#### Return Values

| **Return Value**               | **Description**                                                                                                                                 |
|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------|
| ContainerStatus status   | Container status information.                                                                                                                          |
| map\<string, string> info | Additional information about the sandbox. The key can be any string, and the value is a JSON character string. The information can be any debugging content. When **verbose** is set to **true**, **info** cannot be empty. This parameter does not take effect now. |

#### UpdateContainerResources

#### Prototype

```text
rpc UpdateContainerResources(UpdateContainerResourcesRequest) returns (UpdateContainerResourcesResponse) {}
```

#### Description

This API is used to update container resource configurations.

#### Precautions

* This API cannot be used to update the pod resource configurations.
* The value of  **oom\_score\_adj**  of any container cannot be updated.

#### Parameters

| **Parameter**                  | **Description**          |
|-------------------------------|-------------------|
| string container\_id           | Container ID.            |
| LinuxContainerResources linux | Linux resource configuration information. |

#### Return Values

None

#### ExecSync

#### Prototype

```text
rpc ExecSync(ExecSyncRequest) returns (ExecSyncResponse) {}
```

#### Description

This API is used to run a command in containers in synchronization mode through the gRPC communication method.

#### Precautions

The interaction between the terminal and the containers must be disabled when a single command is executed.

#### Parameters

#### Return Values

#### Exec

#### Prototype

```text
rpc Exec(ExecRequest) returns (ExecResponse) {}
```

#### Description

This API is used to run commands in a container through the gRPC communication method, that is, obtain URLs from the CRI server, and then use the obtained URLs to establish a long connection to the WebSocket server, implementing the interaction with the container.

#### Precautions

The interaction between the terminal and the container can be enabled when a single command is executed. One of  **stdin**,  **stdout**, and  **stderr** must be true. If  **tty**  is true,  **stderr**  must be false. Multiplexing is not supported. In this case, the output of  **stdout**  and  **stderr**  will be combined to a stream.

#### Parameters

#### Return Values

#### Attach

#### Prototype

```text
rpc Attach(AttachRequest) returns (AttachResponse) {}
```

#### Description

This API is used to take over the init process of a container through the gRPC communication method, that is, obtain URLs from the CRI server, and then use the obtained URLs to establish a long connection to the WebSocket server, implementing the interaction with the container.

#### Parameters

#### Return Values

#### ContainerStats

#### Prototype

```text
rpc ContainerStats(ContainerStatsRequest) returns (ContainerStatsResponse) {}
```

#### Description

This API is used to return information about resources occupied by a container. Only containers whose runtime is of the LCR type are supported.

#### Parameters

#### Return Values

| **Return Value**           | **Description**                                                |
|----------------------|---------------------------------------------------------|
| ContainerStats stats | Container information.Note: Disks and inodes support only the query of containers started by OCI images. |

#### ListContainerStats

#### Prototype

```text
rpc ListContainerStats(ListContainerStatsRequest) returns (ListContainerStatsResponse) {}
```

#### Description

This API is used to return the information about resources occupied by multiple containers. Filtering based on criteria is supported.

#### Parameters

| **Parameter**                | **Description**     |
|-----------------------------|--------------|
| ContainerStatsFilter filter | Filter criteria. |

#### Return Values

| **Return Value**                    | **Description**                                                        |
|-------------------------------|-----------------------------------------------------------------|
| repeated ContainerStats stats | Container information list. Note: Disks and inodes support only the query of containers started by OCI images. |

#### UpdateRuntimeConfig

#### Prototype

```text
rpc UpdateRuntimeConfig(UpdateRuntimeConfigRequest) returns (UpdateRuntimeConfigResponse);
```

#### Description

This API is used as a standard CRI to update the pod CIDR of the network plug-in. Currently, the CNI network plug-in does not need to update the pod CIDR. Therefore, this API records only access logs.

#### Precautions

API operations will not modify the system management information, but only record a log.

#### Parameters

| **Parameter**                 | **Description**                |
|------------------------------|-------------------------|
| RuntimeConfig runtime\_config | Information to be configured for the runtime. |

#### Return Values

None

#### Status

#### Prototype

```text
rpc Status(StatusRequest) returns (StatusResponse) {};
```

#### Description

This API is used to obtain the network status of the runtime and pod. Obtaining the network status will trigger the update of network configuration.

#### Precautions

If the network configuration fails to be updated, the original configuration is not affected. The original configuration is overwritten only when the update is successful.

#### Parameters

#### Return Values

| **Return Value**               | **Description**                                                                                                    |
|--------------------------|-------------------------------------------------------------------------------------------------------------|
| RuntimeStatus status     | Runtime status.                                                                                               |
| map\<string, string> info | Additional information about the runtime. The key of **info**can be any value. The value must be in JSON format and can contain any debugging information. When **verbose** is set to **true**, **info** cannot be empty. |

### Image Service

The service provides the gRPC API for pulling, viewing, and removing images from the registry.

#### ListImages

#### Prototype

```text
rpc ListImages(ListImagesRequest) returns (ListImagesResponse) {}
```

#### Description

This API is used to list existing image information.

#### Precautions

This is a unified API. You can run the  **cri images**  command to query embedded images. However, embedded images are not standard OCI images. Therefore, query results have the following restrictions:

* An embedded image does not have an image ID. Therefore, the value of  **image ID**  is the config digest of the image.
* An embedded image has only config digest, and it does not comply with the OCI image specifications. Therefore, the value of  **digest**  cannot be displayed.

#### Parameters

| **Parameter**     | **Description**       |
|------------------|----------------|
| ImageSpec filter | Name of the image to be filtered. |

#### Return Values

| **Return Value**            | **Description**     |
|-----------------------|--------------|
| repeated Image images | Image information list. |

#### ImageStatus

#### Prototype

```text
rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse) {}
```

#### Description

This API is used to query the information about a specified image.

#### Precautions

1. If the image to be queried does not exist,  **ImageStatusResponse**  is returned and  **Image**  is set to  **nil**  in the return value.
2. This is a unified API. Since embedded images do not comply with the OCI image specifications and do not contain required fields, the images cannot be queried by using this API.

#### Parameters

| **Parameter**    | **Description**                               |
|-----------------|----------------------------------------|
| ImageSpec image | Image name.                               |
| bool verbose    | Whether to query additional information. This parameter does not take effect now. No additional information is returned. |

#### Return Values

| **Return Value**               | **Description**                               |
|--------------------------|----------------------------------------|
| Image image              | Image information.                              |
| map\<string, string> info | Additional image information. This parameter does not take effect now. No additional information is returned. |

#### PullImage

#### Prototype

```text
rpc PullImage(PullImageRequest) returns (PullImageResponse) {}
```

#### Description

This API is used to download images.

#### Precautions

Currently, you can download public images, and use the username, password, and auth information to download private images. The  **server\_address**,  **identity\_token**, and  **registry\_token**  fields in  **authconfig**  cannot be configured.

#### Parameters

| **Parameter**                    | **Description**                          |
|---------------------------------|-----------------------------------|
| ImageSpec image                 | Name of the image to be downloaded.                  |
| AuthConfig auth                 | Verification information for downloading a private image.          |
| PodSandboxConfig sandbox\_config | Whether to download an image in the pod context. This parameter does not take effect now. |

#### Return Values

#### RemoveImage

#### Prototype

```text
rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse) {}
```

#### Description

This API is used to delete specified images.

#### Precautions

This is a unified API. Since embedded images do not comply with the OCI image specifications and do not contain required fields, you cannot delete embedded images by using this API and the image ID.

#### Parameters

| **Parameter**    | **Description**               |
|-----------------|------------------------|
| ImageSpec image | Name or ID of the image to be deleted. |

#### Return Values

None

#### ImageFsInfo

#### Prototype

```text
rpc ImageFsInfo(ImageFsInfoRequest) returns (ImageFsInfoResponse) {}
```

#### Description

This API is used to query the information about the file system that stores images.

#### Precautions

Queried results are the file system information in the image metadata.

#### Parameters

None

#### Return Values

| **Return Value**                                 | **Description**             |
|--------------------------------------------|----------------------|
| repeated FilesystemUsage image\_filesystems | Information about the file system that stores images. |

### Constraints

1. If  **log\_directory**  is configured in the  **PodSandboxConfig**  parameter when a sandbox is created,  **log\_path**  must be specified in  **ContainerConfig**  when all containers that belong to the sandbox are created. Otherwise, the containers may not be started or deleted by using the CRI.

   The actual value of  **LOGPATH**  of containers is  **log\_directory/log\_path**. If  **log\_path**  is not set, the final value of  **LOGPATH**  is changed to  **log\_directory**.

   * If the path does not exist, iSulad will create a soft link pointing to the actual path of container logs when starting a container. Then  **log\_directory**  becomes a soft link. There are two cases:
     1. In the first case, if  **log\_path**  is not configured for other containers in the sandbox,  **log\_directory**  will be deleted and point to  **log\_path**  of the newly started container. As a result, logs of the first started container point to logs of the later started container.
     2. In the second case, if  **log\_path**  is configured for other containers in the sandbox, the value of  **LOGPATH**  of the container is  **log\_directory/log\_path**. Because  **log\_directory**  is a soft link, the creation fails when  **log\_directory/log\_path**  is used as the soft link to point to the actual path of container logs.

   * If the path exists, iSulad will attempt to delete the path (non-recursive) when starting a container. If the path is a folder path containing content, the deletion fails. As a result, the soft link fails to be created, the container fails to be started, and the same error occurs when the container is going to be deleted.

2. If  **log\_directory**  is configured in the  **PodSandboxConfig**  parameter when a sandbox is created, and  **log\_path**  is specified in  **ContainerConfig**  when a container is created, the final value of  **LOGPATH**  is  **log\_directory/log\_path**. iSulad does not recursively create  **LOGPATH**, therefore, you must ensure that  **dirname(LOGPATH)**  exists, that is, the upper-level path of the final log file path exists.

3. If  **log\_directory**  is configured in the  **PodSandboxConfig**  parameter when a sandbox is created, and the same  **log\_path**  is specified in  **ContainerConfig**  when multiple containers are created, or if containers in different sandboxes point to the same  **LOGPATH**, the latest container log path will overwrite the previous path after the containers are started successfully.

4. If the image content in the remote registry changes and the original image is stored in the local host, the name and tag of the original image are changed to  **none**  when you call the CRI Pull image API to download the image again.

   An example is as follows:

   Locally stored images:

   ```text
   IMAGE                                        TAG                 IMAGE ID            SIZE
   rnd-dockerhub.huawei.com/pproxyisulad/test   latest              99e59f495ffaa       753kB
   ```

   After the  **rnd-dockerhub.huawei.com/pproxyisulad/test:latest**  image in the remote registry is updated and downloaded again:

   ```text
   IMAGE                                        TAG                 IMAGE ID            SIZE
   <none>                                       <none>              99e59f495ffaa       753kB
   rnd-dockerhub.huawei.com/pproxyisulad/test   latest              d8233ab899d41       1.42MB
   ```

   Run the  **isula images**  command. The value of  **REF**  is displayed as  **-**.

   ```text
   REF                                               IMAGE ID               CREATED              SIZE       
   rnd-dockerhub.huawei.com/pproxyisulad/test:latest d8233ab899d41          2019-02-14 19:19:37  1.42MB     
   -                                                 99e59f495ffaa          2016-05-04 02:26:41  753kB
   ```

5. The iSulad CRI API exec/attach is implemented using the WebSocket protocol. Clients interact with the iSulad using the same protocol. When using the exec/attach API, do not transfer a large amount of data or files over the serial port. The exec/attach API is used only for basic command interaction. If the user does not process the data or files in a timely manner, data may be lost. In addition, do not use the exec/attach API to transfer binary data or files.

6. The iSulad CRI API exec/attach depends on libwebsockets (LWS). It is recommended that the streaming API be used only for persistent connection interaction but not in high-concurrency scenarios, because the connection may fail due to insufficient host resources. It is recommended that the number of concurrent connections be less than or equal to 100.

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/cri.md
---
# CRI接口

## 描述

CRI API接口是由kubernetes推出的容器运行时接口，CRI定义了容器和镜像的服务接口。iSulad使用CRI接口，实现和kubernetes的对接。

因为容器运行时与镜像的生命周期是彼此隔离的，因此需要定义两个服务。该接口使用[Protocol Buffer](https://developers.google.com/protocol-buffers/)定义，基于[gRPC](https://grpc.io/)。

当前iSulad使用默认CRI版本为v1alpha2版本，官方API描述文件如下：

<https://github.com/kubernetes/kubernetes/blob/release-1.14/pkg/kubelet/apis/cri/runtime/v1alpha2/api.proto>，

iSulad使用的为pass使用的1.14版本API描述文件，与官方API略有出入，以本文档描述的接口为准。

> \[!NOTE]说明
>
> CRI接口websocket流式服务，服务端侦听地址为127.0.0.1，端口为10350，端口可通过命令行--websocket-server-listening-port参数接口或者daemon.json配置文件进行配置。

## 接口

各接口中可能用到的参数清单如下，部分参数暂不支持配置，已在配置中标出。

### 接口参数列表

* **DNSConfig**

  配置sandbox的DNS服务器和搜索域

  | 参数成员                 | 描述                                                       |
  |--------------------------|------------------------------------------------------------|
  | repeated string servers  | 集群的DNS服务器列表                                        |
  | repeated string searches | 集群的DNS搜索域列表                                        |
  | repeated string options  | DNS可选项列表，参考<https://linux.die.net/man/5/resolv.conf> |

* **Protocol**

  协议的enum值列表

* **PortMapping**

  指定sandbox的端口映射配置

  | **参数成员**         | **描述**           |
  |----------------------|--------------------|
  | Protocol protocol    | 端口映射使用的协议 |
  | int32 container\_port | 容器内的端口号     |
  | int32 host\_port      | 主机上的端口号     |
  | string host\_ip       | 主机IP地址         |

* **MountPropagation**

  挂载传播属性的enum列表

* **Mount**

  Mount指定host上的一个挂载卷挂载到容器中（只支持文件和文件夹)

  | **参数成员**                 | **描述**                                                                        |
  |------------------------------|---------------------------------------------------------------------------------|
  | string container\_path        | 容器中的路径                                                                    |
  | string host\_path             | 主机上的路径                                                                    |
  | bool readonly                | 是否配置在容器中是只读的， 缺省值： false                                       |
  | bool selinux\_relabel         | 是否设置SELinux标签（不支持配置）                                               |
  | MountPropagation propagation | 挂载传播属性配置（取值**0/1/2**，分别对应**private/rslave/rshared**传播属性） **缺省值：0** |

* **NamespaceOption**

* **Capability**

  包含待添加与待删除的权能信息

* **Int64Value**

  int64类型的封装

* **UInt64Value**

  uint64类型的封装

* **LinuxSandboxSecurityContext**

  配置sandbox的linux安全选项。

  注意，这些安全选项不会应用到sandbox中的容器中，也可能不适用于没有任何running进程的sandbox。

  | **参数成员**                       | **描述**                                                                                                                                                                                       |
  |------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | NamespaceOption namespace\_options  | 配置sandbox的命名空间选项                                                                                                                                                                      |
  | SELinuxOption selinux\_options      | 配置SELinux选项（不支持）                                                                                                                                                                      |
  | Int64Value run\_as\_user             | 配置sandbox中进程的uid                                                                                                                                                                         |
  | bool readonly\_rootfs               | 配置sandbox的根文件系统是否只读                                                                                                                                                                |
  | repeated int64 supplemental\_groups | 配置除主GID之外的sandbox的1号进程用户组信息                                                                                                                                                    |
  | bool privileged                    | 配置sandbox是否为特权容器                                                                                                                                                                      |
  | string seccomp\_profile\_path        | seccomp配置文件路径，有效值为： // unconfined: 不配置seccomp  // localhost/<配置文件的全路径>： 安装在系统上的配置文件路径  // <配置文件的全路径>： 配置文件全路径  // 默认不配置，即unconfined。 |

* **LinuxPodSandboxConfig**

  设定和Linux主机及容器相关的一些配置

  | **参数成员**                                 | **描述**                                                                                |
  |----------------------------------------------|-----------------------------------------------------------------------------------------|
  | string cgroup\_parent                         | sandbox的cgroup父路径，runtime可根据实际情况使用cgroupfs或systemd的语法。（不支持配置） |
  | LinuxSandboxSecurityContext security\_context | sandbox的安全属性                                                                       |
  | map\<string, string> sysctls                  | sandbox的linux sysctls配置                                                              |

* **PodSandboxMetadata**

  Sandbox元数据包含构建sandbox名称的所有信息，鼓励容器运行时在用户界面中公开这些元数据以获得更好的用户体验，例如，运行时可以根据元数据生成sandbox的唯一命名。

* **PodSandboxConfig**

  包含创建sandbox的所有必选和可选配置信息

  | **参数成员**                       | **描述**                                                                                                                                            |
  |------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
  | PodSandboxMetadata metadata        | sandbox的元数据，这项信息唯一标识一个sandbox，runtime必须利用此信息确保操作正确，runtime也可以根据此信息来改善用户体验，例如构建可读的sandbox名称。 |
  | string hostname                    | sandbox的hostname                                                                                                                                   |
  | string log\_directory               | 配置sandbox内的容器的日志文件所存储的文件夹                                                                                                         |
  | DNSConfig dns\_config               | sandbox的DNS配置                                                                                                                                    |
  | repeated PortMapping port\_mappings | sandbox的端口映射                                                                                                                                   |
  | map\<string, string> labels         | 可用于标识单个或一系列sandbox的键值对                                                                                                               |
  | map\<string, string> annotations    | 存储任意信息的键值对，这些值是不可更改的，且能够利用PodSandboxStatus接口查询                                                                        |
  | LinuxPodSandboxConfig linux        | 与linux主机相关的可选项                                                                                                                             |

* **PodSandboxNetworkStatus**

  描述sandbox的网络状态

* **Namespace**

  命名空间选项

  | **参数成员**            | **描述**           |
  |-------------------------|--------------------|
  | NamespaceOption options | Linux 命名空间选项 |

* **LinuxPodSandboxStatus**

  描述Linux sandbox的状态

  | **参数成员**         | **描述**        |
  |----------------------|-----------------|
  | Namespace **namespaces** | sandbox命名空间 |

* **PodSandboxState**

  sandbox状态值的enum数据

* **PodSandboxStatus**

  描述Podsandbox的状态信息

  | **参数成员**                              | **描述**                                          |
  |-------------------------------------------|---------------------------------------------------|
  | string id                                 | sandbox的ID                                       |
  | PodSandboxMetadata metadata               | sandbox的元数据                                   |
  | PodSandboxState state                     | sandbox的状态值                                   |
  | int64 created\_at                          | sandbox的创建时间戳，单位纳秒                     |
  | repeated PodSandboxNetworkStatus networks | sandbox的多平面网络状态                           |
  | LinuxPodSandboxStatus linux               | Linux规范的sandbox状态                            |
  | map\<string, string> labels                | 可用于标识单个或一系列sandbox的键值对             |
  | map\<string, string> annotations           | 存储任意信息的键值对，这些值是不可被runtime更改的 |

* **PodSandboxStateValue**

  对PodSandboxState的封装

  | **参数成员**          | **描述**        |
  |-----------------------|-----------------|
  | PodSandboxState state | sandbox的状态值 |

* **PodSandboxFilter**

  用于列出sandbox时添加过滤条件，多个条件取交集显示

  | **参数成员**                       | **描述**                                             |
  |------------------------------------|------------------------------------------------------|
  | string id                          | sandbox的ID                                          |
  | PodSandboxStateValue state         | sandbox的状态                                        |
  | map\<string, string> label\_selector | sandbox的labels，label只支持完全匹配，不支持正则匹配 |

* **PodSandbox**

  包含最小化描述一个sandbox的数据

  | **参数成员**                    | **描述**                                          |
  |---------------------------------|---------------------------------------------------|
  | string id                       | sandbox的ID                                       |
  | PodSandboxMetadata metadata     | sandbox的元数据                                   |
  | PodSandboxState state           | sandbox的状态值                                   |
  | int64 created\_at                | sandbox的创建时间戳，单位纳秒                     |
  | map\<string, string> labels      | 可用于标识单个或一系列sandbox的键值对             |
  | map\<string, string> annotations | 存储任意信息的键值对，这些值是不可被runtime更改的 |

* **KeyValue**

  键值对的封装

* **SELinuxOption**

  应用于容器的SELinux标签

* **ContainerMetadata**

  Container元数据包含构建container名称的所有信息，鼓励容器运行时在用户界面中公开这些元数据以获得更好的用户体验，例如，运行时可以根据元数据生成container的唯一命名。

* **ContainerState**

  容器状态值的enum列表

* **ContainerStateValue**

  封装ContainerState的数据结构

  | **参数成员**         | **描述**   |
  |----------------------|------------|
  | ContainerState **state** | 容器状态值 |

* **ContainerFilter**

  用于列出container时添加过滤条件，多个条件取交集显示

  | **参数成员**                       | **描述**                                               |
  |------------------------------------|--------------------------------------------------------|
  | string id                          | container的ID                                          |
  | PodSandboxStateValue state         | container的状态                                        |
  | string pod\_sandbox\_id              | sandbox的ID                                            |
  | map\<string, string> label\_selector | container的labels，label只支持完全匹配，不支持正则匹配 |

* **LinuxContainerSecurityContext**

  指定应用于容器的安全配置

  | **参数成员**                       | **描述**                                                                                                                           |
  |------------------------------------|------------------------------------------------------------------------------------------------------------------------------------|
  | Capability capabilities            | 新增或去除的权能                                                                                                                   |
  | bool privileged                    | 指定容器是否未特权模式， **缺省值：false**                                                                                             |
  | NamespaceOption namespace\_options  | 指定容器的namespace选项                                                                                                            |
  | SELinuxOption selinux\_options      | SELinux context(可选配置项) **暂不支持**                                                                                               |
  | Int64Value run\_as\_user             | 运行容器进程的UID。 一次只能指定run\_as\_user与run\_as\_username其中之一，run\_as\_username优先生效                                      |
  | string run\_as\_username             | 运行容器进程的用户名。 如果指定，用户必须存在于容器映像中（即在映像内的/etc/passwd中），并由运行时在那里解析; 否则，运行时必须出错 |
  | bool readonly\_rootfs               | 设置容器中根文件系统是否为只读 **缺省值由config.json配置**                                                                             |
  | repeated int64 supplemental\_groups | 容器运行的除主GID外首进程组的列表                                                                                                  |
  | string apparmor\_profile            | 容器的AppArmor配置文件 **暂不支持**                                                                                                    |
  | string seccomp\_profile\_path        | 容器的seccomp配置文件路径                                                                                                          |
  | bool no\_new\_privs                  | 是否在容器上设置no\_new\_privs的标志                                                                                                 |

* **LinuxContainerResources**

  指定Linux容器资源的特定配置

* **Image**

  Image信息描述一个镜像的基本数据。

  | **参数成员**                 | **描述**               |
  |------------------------------|------------------------|
  | string id                    | 镜像ID                 |
  | repeated string repo\_tags    | 镜像tag 名称 repo\_tags |
  | repeated string repo\_digests | 镜像digest信息         |
  | uint64 size                  | 镜像大小               |
  | Int64Value uid               | 镜像默认用户UID        |
  | string username              | 镜像默认用户名称       |

* **ImageSpec**

  表示镜像的内部数据结构，当前，ImageSpec只封装容器镜像名称

* **StorageIdentifier**

  唯一定义storage的标识

* **FilesystemUsage**

  | **参数成员**                 | **描述**                   |
  |------------------------------|----------------------------|
  | int64 timestamp              | 收集文件系统信息时的时间戳 |
  | StorageIdentifier storage\_id | 存储镜像的文件系统UUID     |
  | UInt64Value used\_bytes       | 存储镜像元数据的大小       |
  | UInt64Value inodes\_used      | 存储镜像元数据的inodes个数 |

* **AuthConfig**

* **Container**

  用于描述容器信息，例如ID, 状态等。

  | **参数成员**                    | **描述**                                                    |
  |---------------------------------|-------------------------------------------------------------|
  | string id                       | container的ID                                               |
  | string pod\_sandbox\_id           | 该容器所属的sandbox的ID                                     |
  | ContainerMetadata metadata      | container的元数据                                           |
  | ImageSpec image                 | 镜像规格                                                    |
  | string image\_ref                | 代表容器使用的镜像，对大多数runtime来产，这是一个image ID值 |
  | ContainerState state            | container的状态                                             |
  | int64 created\_at                | container的创建时间戳，单位为纳秒                           |
  | map\<string, string> labels      | 可用于标识单个或一系列container的键值对                     |
  | map\<string, string> annotations | 存储任意信息的键值对，这些值是不可被runtime更改的           |

* **ContainerStatus**

  用于描述容器状态信息

  | **参数成员**                    | **描述**                                                                  |
  |---------------------------------|---------------------------------------------------------------------------|
  | string id                       | container的ID                                                             |
  | ContainerMetadata metadata      | container的元数据                                                         |
  | ContainerState state            | container的状态                                                           |
  | int64 created\_at                | container的创建时间戳，单位为纳秒                                         |
  | int64 started\_at                | container启动时的时间戳，单位为纳秒                                       |
  | int64 finished\_at               | container退出时的时间戳，单位为纳秒                                       |
  | int32 exit\_code                 | 容器退出码                                                                |
  | ImageSpec image                 | 镜像规格                                                                  |
  | string image\_ref                | 代表容器使用的镜像，对大多数runtime来产，这是一个image ID值               |
  | string reason                   | 简要描述为什么容器处于当前状态                                            |
  | string message                  | 易于人工阅读的信息，用于表述容器处于当前状态的原因                        |
  | map\<string, string> labels      | 可用于标识单个或一系列container的键值对                                   |
  | map\<string, string> annotations | 存储任意信息的键值对，这些值是不可被runtime更改的                         |
  | repeated Mount mounts           | 容器的挂载点信息                                                          |
  | string log\_path                 | 容器日志文件路径，该文件位于PodSandboxConfig中配置的log\_directory文件夹下 |

* **ContainerStatsFilter**

  用于列出container stats时添加过滤条件，多个条件取交集显示

* **ContainerStats**

  用于列出container stats时添加过滤条件，多个条件取交集显示

  | **参数成员**                   | **描述**       |
  |--------------------------------|----------------|
  | ContainerAttributes attributes | 容器的信息     |
  | CpuUsage cpu                   | CPU使用情况    |
  | MemoryUsage memory             | 内存使用情况   |
  | FilesystemUsage writable\_layer | 可写层使用情况 |

* **ContainerAttributes**

  列出container的基本信息

  | **参数成员**                   | **描述**                                          |
  |--------------------------------|---------------------------------------------------|
  | string id                      | 容器的ID                                          |
  | ContainerMetadata metadata     | 容器的metadata                                    |
  | map\<string,string> labels      | 可用于标识单个或一系列container的键值对           |
  | map\<string,string> annotations | 存储任意信息的键值对，这些值是不可被runtime更改的 |

* **CpuUsage**

  列出container的CPU使用信息

* **MemoryUsage**

  列出container的内存使用信息

* **FilesystemUsage**

  列出container的读写层信息

* **Device**

  指定待挂载至容器的主机卷

* **LinuxContainerConfig**

  包含特定于Linux平台的配置

  | **参数成员**                                   | **描述**                |
  |------------------------------------------------|-------------------------|
  | LinuxContainerResources resources              | 容器的资源规范          |
  | LinuxContainerSecurityContext security\_context | 容器的Linux容器安全配置 |

* **ContainerConfig**

  包含用于创建容器的所有必需和可选字段

  | **参数成员**                    | **描述**                                                                                                                                             |
  |---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
  | ContainerMetadata metadata      | 容器的元数据。 此信息将唯一标识容器，运行时应利用此信息来确保正确操作。 运行时也可以使用此信息来提升UX（用户体检设计），例如通过构造可读名称。(必选) |
  | ImageSpec image                 | 容器使用的镜像 (**必选**)                                                                                                                                |
  | repeated string command         | 待执行的命令 **缺省值： "/bin/sh"**                                                                                                                      |
  | repeated string args            | 待执行命令的参数                                                                                                                                     |
  | string working\_dir              | 命令执行的当前工作路径                                                                                                                               |
  | repeated KeyValue envs          | 在容器中配置的环境变量                                                                                                                               |
  | repeated Mount mounts           | 待在容器中挂载的挂载点信息                                                                                                                           |
  | repeated Device devices         | 待在容器中映射的设备信息                                                                                                                             |
  | map\<string, string> labels      | 可用于索引和选择单个资源的键值对。                                                                                                                   |
  | map\<string, string> annotations | 可用于存储和检索任意元数据的非结构化键值映射。                                                                                                       |
  | string log\_path                 | 相对于PodSandboxConfig.LogDirectory的路径，用于存储容器主机上的日志（STDOUT和STDERR）。                                                              |
  | bool stdin                      | 是否打开容器的stdin                                                                                                                                  |
  | bool stdin\_once                 | 当某次连接stdin的数据流断开时，是否立即断开其他与stdin连接的数据流 **（暂不支持）**                                                                       |
  | bool tty                        | 是否使用伪终端连接容器的stdio                                                                                                                        |
  | LinuxContainerConfig linux      | linux系统上容器的特定配置信息                                                                                                                        |

* **RuntimeConfig**

  Runtime的网络配置

  | **参数成员**                 | **描述**          |
  |------------------------------|-------------------|
  | NetworkConfig network\_config | Runtime的网络配置 |

* **RuntimeCondition**

  描述runtime的状态信息

* **RuntimeStatus**

  Runtime的状态

### Runtime服务

Runtime服务中包含操作pod和容器的接口，以及查询runtime自身配置和状态信息的接口。

#### RunPodSandbox

#### 接口原型

```text
rpc RunPodSandbox(RunPodSandboxRequest) returns (RunPodSandboxResponse) {}
```

#### 接口描述

创建和启动一个pod sandbox，若运行成功，sandbox处于ready状态。

#### 注意事项

1. 启动sandbox的默认镜像为rnd-dockerhub.huawei.com/library/pause-${machine}:3.0， 其中${machine}为架构，在x86\_64上，machine的值为amd64，在arm64上，machine的值为aarch64，当前rnd-dockerhub仓库上只有amd64和aarch64镜像可供下载，若机器上无此镜像，请确保机器能从rnd-dockerhub下载，若要使用其他镜像，请参考“iSulad部署配置”中的pod-sandbox-image指定镜像。
2. 由于容器命名以PodSandboxMetadata中的字段为来源，且以下划线"\_"为分割字符，因此限制metadata中的数据不能包含下划线，否则会出现sandbox运行成功，但无法使用ListPodSandbox接口查询的现象。

#### 参数

| **参数成员**            | **描述**                                                              |
|-------------------------|-----------------------------------------------------------------------|
| PodSandboxConfig config | sandbox的配置                                                         |
| string runtime\_handler  | 指定创建sandbox的runtime运行时，当前支持lcr、kata-runtime运行时类型。 |

#### 返回值

#### StopPodSandbox

#### 接口原型

```text
rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse) {}
```

#### 接口描述

停止pod sandbox，停止sandbox容器，回收分配给sandbox的网络资源（比如IP地址）。如果有任何running的容器属于该sandbox，则必须被强制停止。

#### 参数

#### 返回值

#### RemovePodSandbox

#### 接口原型

```text
rpc RemovePodSandbox(RemovePodSandboxRequest) returns (RemovePodSandboxResponse) {}
```

#### 接口描述

删除sandbox，如果有任何running的容器属于该sandbox，则必须被强制停止和删除，如果sandbox已经被删除，不能返回错误。

#### 注意事项

1. 删除sandbox时，不会删除sandbox的网络资源，在删除pod前必须先调用StopPodSandbox才能清理网络资源，调用者应当保证在删除sandbox之前至少调用一次StopPodSandbox。
2. 删除sandbox时，如果sandbox中的容器删除失败，则会出现sanbox被删除但容器还残留的情况，此时需要手动删除残留的容器进行清理。

#### 参数

#### 返回值

#### PodSandboxStatus

#### 接口原型

```text
rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse) {}
```

#### 接口描述

查询sandbox的状态，如果sandbox不存在，返回错误。

#### 参数

#### 返回值

| **返回值**               | **描述**                                                                                                                                 |
|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------|
| PodSandboxStatus status  | sandbox的状态信息                                                                                                                        |
| map\<string, string> info | sandbox的额外信息，key是任意string，value是json格式的字符串，这些信息可以是任意调试内容。当verbose为true时info不能为空。（暂不支持配置） |

#### ListPodSandbox

#### 接口原型

```text
rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse) {}
```

#### 接口描述

返回sandbox信息的列表，支持条件过滤。

#### 参数

| **参数成员**            | **描述**     |
|-------------------------|--------------|
| PodSandboxFilter filter | 条件过滤参数 |

#### 返回值

| **返回值**                | **描述**          |
|---------------------------|-------------------|
| repeated PodSandbox items | sandbox信息的列表 |

#### CreateContainer

#### 接口原型

```text
rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse) {}
```

#### 接口描述

在PodSandbox内创建一个容器。

#### 注意事项

* 请求CreateContainerRequest 中的sandbox\_config与传递给RunPodSandboxRequest以创建PodSandbox的配置相同。 它再次传递，只是为了方便参考。 PodSandboxConfig是不可变的，在pod的整个生命周期内保持不变。
* 由于容器命名以ContainerMetadata中的字段为来源，且以下划线"\_"为分割字符，因此限制metadata中的数据不能包含下划线，否则会出现sandbox运行成功，但无法使用ListContainers接口查询的现象。
* CreateContainerRequest中无runtime\_handler字段，创建container时的runtime类型和其对应的sandbox的runtime相同。

#### 参数

| **参数成员**                    | **描述**                           |
|---------------------------------|------------------------------------|
| string pod\_sandbox\_id           | 待在其中创建容器的PodSandbox的ID。 |
| ContainerConfig config          | 容器的配置信息                     |
| PodSandboxConfig sandbox\_config | PodSandbox的配置信息               |

#### 补充

可用于存储和检索任意元数据的非结构化键值映射。有一些字段由于cri接口没有提供特定的参数，可通过该字段将参数传入

* 自定义

#### 返回值

#### StartContainer

#### 接口原型

```text
rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {}
```

#### 接口描述

启动一个容器。

#### 参数

#### 返回值

#### StopContainer

#### 接口原型

```text
rpc StopContainer(StopContainerRequest) returns (StopContainerResponse) {}
```

#### 接口描述

停止一个running的容器，支持配置优雅停止时间timeout，如果容器已经停止，不能返回错误。

#### 参数

#### 返回值

无

#### RemoveContainer

#### 接口原型

```text
rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse) {}
```

#### 接口描述

删除一个容器，如果容器正在运行，必须强制停止，如果容器已经被删除，不能返回错误。

#### 参数

#### 返回值

无

#### ListContainers

#### 接口原型

```text
rpc ListContainers(ListContainersRequest) returns (ListContainersResponse) {}
```

#### 接口描述

返回container信息的列表，支持条件过滤。

#### 参数

| **参数成员**           | **描述**     |
|------------------------|--------------|
| ContainerFilter filter | 条件过滤参数 |

#### 返回值

| **返回值**                    | **描述**       |
|-------------------------------|----------------|
| repeated Container containers | 容器信息的列表 |

#### ContainerStatus

#### 接口原型

```text
rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse) {}
```

#### 接口描述

返回容器状态信息，如果容器不存在，则返回错误。

#### 参数

#### 返回值

| **返回值**               | **描述**                                                                                                                                 |
|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------|
| ContainerStatus status   | 容器的状态信息                                                                                                                           |
| map\<string, string> info | sandbox的额外信息，key是任意string，value是json格式的字符串，这些信息可以是任意调试内容。当verbose为true时info不能为空。（暂不支持配置） |

#### UpdateContainerResources

#### 接口原型

```text
rpc UpdateContainerResources(UpdateContainerResourcesRequest) returns (UpdateContainerResourcesResponse) {}
```

#### 接口描述

该接口用于更新容器资源配置。

#### 注意事项

* 该接口仅用于更新容器的资源配置，不能用于更新Pod的资源配置。
* 当前不支持更新容器oom\_score\_adj配置。

#### 参数

| **参数成员**                  | **描述**          |
|-------------------------------|-------------------|
| string container\_id           | 容器id            |
| LinuxContainerResources linux | linux资源配置信息 |

#### 返回值

无

#### ExecSync

#### 接口原型

```text
rpc ExecSync(ExecSyncRequest) returns (ExecSyncResponse) {}
```

#### 接口描述

以同步的方式在容器中执行命令，采用的gRPC通讯方式。

#### 注意事项

执行一条单独的命令，不能打开终端与容器交互。

#### 参数

#### 返回值

#### Exec

#### 接口原型

```text
rpc Exec(ExecRequest) returns (ExecResponse) {}
```

#### 接口描述

在容器中执行命令，采用的gRPC通讯方式从CRI服务端获取url，再通过获得的url与websocket服务端建立长连接，实现与容器的交互。

#### 注意事项

执行一条单独的命令，也能打开终端与容器交互。stdin/stdout/stderr之一必须是真的。如果tty为真，stderr必须是假的。 不支持多路复用,  在这种情况下, stdout和stderr的输出将合并为单流。

#### 参数

#### 返回值

#### Attach

#### 接口原型

```text
rpc Attach(AttachRequest) returns (AttachResponse) {}
```

#### 接口描述

接管容器的1号进程，采用gRPC通讯方式从CRI服务端获取url，再通过获取的url与websocket服务端建立长连接，实现与容器的交互。

#### 参数

#### 返回值

#### ContainerStats

#### 接口原型

```text
rpc ContainerStats(ContainerStatsRequest) returns (ContainerStatsResponse) {}
```

#### 接口描述

返回单个容器占用资源信息，仅支持runtime类型为lcr的容器。

#### 参数

#### 返回值

| **返回值**           | **描述**                                                |
|----------------------|---------------------------------------------------------|
| ContainerStats stats | 容器信息。注：disk和inodes只支持oci格式镜像起的容器查询 |

#### ListContainerStats

#### 接口原型

```text
rpc ListContainerStats(ListContainerStatsRequest) returns (ListContainerStatsResponse) {}
```

#### 接口描述

返回多个容器占用资源信息，支持条件过滤

#### 参数

| **参数成员**                | **描述**     |
|-----------------------------|--------------|
| ContainerStatsFilter filter | 条件过滤参数 |

#### 返回值

| **返回值**                    | **描述**                                                        |
|-------------------------------|-----------------------------------------------------------------|
| repeated ContainerStats stats | 容器信息的列表。注：disk和inodes只支持oci格式镜像启动的容器查询 |

#### UpdateRuntimeConfig

#### 接口原型

```text
rpc UpdateRuntimeConfig(UpdateRuntimeConfigRequest) returns (UpdateRuntimeConfigResponse);
```

#### 接口描述

提供标准的CRI接口，目的为了更新网络插件的Pod CIDR，当前CNI网络插件无需更新Pod CIDR，因此该接口只会记录访问日志。

#### 注意事项

接口操作不会对系统管理信息修改，只是记录一条日志。

#### 参数

| **参数成员**                 | **描述**                |
|------------------------------|-------------------------|
| RuntimeConfig runtime\_config | 包含Runtime要配置的信息 |

#### 返回值

无

#### Status

#### 接口原型

```text
rpc Status(StatusRequest) returns (StatusResponse) {};
```

#### 接口描述

获取runtime和pod的网络状态，在获取网络状态时，会触发网络配置的刷新。

#### 注意事项

如果网络配置刷新失败，不会影响原有配置；只有刷新成功时，才会覆盖原有配置。

#### 参数

#### 返回值

| **返回值**               | **描述**                                                                                                    |
|--------------------------|-------------------------------------------------------------------------------------------------------------|
| RuntimeStatus status     | Runtime的状态                                                                                               |
| map\<string, string> info | Runtime额外的信息，info的key为任意值，value为json格式，可包含任何debug信息；只有Verbose为true是才应该被赋值 |

### Image服务

提供了从镜像仓库拉取、查看、和移除镜像的gRPC API。

#### ListImages

#### 接口原型

```text
rpc ListImages(ListImagesRequest) returns (ListImagesResponse) {}
```

#### 接口描述

列出当前已存在的镜像信息。

#### 注意事项

为统一接口，对于embedded格式镜像，可以通过cri images查询到。但是因embedded镜像不是标准OCI镜像，因此查询得到的结果有以下限制：

* 因embedded镜像无镜像ID，显示的镜像ID为镜像的config digest。
* 因embedded镜像本身无digest仅有config的digest，且格式不符合OCI镜像规范，因此无法显示digest。

#### 参数

| **参数成员**     | **描述**       |
|------------------|----------------|
| ImageSpec filter | 筛选的镜像名称 |

#### 返回值

| **返回值**            | **描述**     |
|-----------------------|--------------|
| repeated Image images | 镜像信息列表 |

#### ImageStatus

#### 接口原型

```text
rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse) {}
```

#### 接口描述

查询指定镜像信息。

#### 注意事项

1. 查询指定镜像信息，若镜像不存在，则返回ImageStatusResponse，其中Image设置为nil。
2. 为统一接口，对于embedded格式镜像，因不符合OCI格式镜像，缺少字段，无法通过本接口进行查询。

#### 参数

| **参数成员**    | **描述**                               |
|-----------------|----------------------------------------|
| ImageSpec image | 镜像名称                               |
| bool verbose    | 查询额外信息，暂不支持，无额外信息返回 |

#### 返回值

| **返回值**               | **描述**                               |
|--------------------------|----------------------------------------|
| Image image              | 镜像信息                               |
| map\<string, string> info | 镜像额外信息，暂不支持，无额外信息返回 |

#### PullImage

#### 接口原型

```text
 rpc PullImage(PullImageRequest) returns (PullImageResponse) {}
```

#### 接口描述

下载镜像。

#### 注意事项

当前支持下载public镜像，使用用户名、密码、auth信息下载私有镜像，不支持authconfig中的server\_address、identity\_token、registry\_token字段。

#### 参数

| **参数成员**                    | **描述**                          |
|---------------------------------|-----------------------------------|
| ImageSpec image                 | 要下载的镜像名称                  |
| AuthConfig auth                 | 下载私有镜像时的验证信息          |
| PodSandboxConfig sandbox\_config | 在Pod上下文中下载镜像（暂不支持） |

#### 返回值

#### RemoveImage

#### 接口原型

```text
rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse) {}
```

#### 接口描述

删除指定镜像。

#### 注意事项

为统一接口，对于embedded格式镜像，因不符合OCI格式镜像，缺少字段，无法通过本接口使用image id进行删除。

#### 参数

| **参数成员**    | **描述**               |
|-----------------|------------------------|
| ImageSpec image | 要删除的镜像名称或者ID |

#### 返回值

无

#### ImageFsInfo

#### 接口原型

```text
rpc ImageFsInfo(ImageFsInfoRequest) returns (ImageFsInfoResponse) {}
```

#### 接口描述

查询存储镜像的文件系统信息。

#### 注意事项

查询到的为镜像元数据下的文件系统信息。

#### 参数

无

#### 返回值

| **返回值**                                 | **描述**             |
|--------------------------------------------|----------------------|
| repeated FilesystemUsage image\_filesystems | 镜像存储文件系统信息 |

### 约束

1. 如果创建sandbox时，PodSandboxConfig参数中配置了log\_directory，则所有属于该sandbox的container在创建时必须在ContainerConfig中指定log\_path，否则可能导致容器无法使用CRI接口启动，甚至无法使用CRI接口删除。

   容器的真实LOGPATH=log\_directory/log\_path，如果log\_path不配置，那么最终的LOGPATH会变为LOGPATH=log\_directory。

   * 如果该路径不存在，isulad在启动容器时会创建一个软链接，指向最终的容器日志真实路径，此时log\_directory变成一个软链接，此时有两种情况：
     1. 第一种情况，如果该sandbox里其他容器也没配置log\_path，在启动其他容器时，log\_directory会被删除，然后重新指向新启动容器的log\_path，导致之前启动的容器日志指向后面启动容器的日志。
     2. 第二种情况，如果该sandbox里其他容器配置了log\_path，则该容器的LOGPATH=log\_directory/log\_path，由于log\_directory实际是个软链接，使用log\_directory/log\_path为软链接指向容器真实日志路径时，创建会失败。

   * 如果该路径存在，isulad在启动容器时首先会尝试删除该路径（非递归），如果该路径是个文件夹，且里面有内容，删除会失败，从而导致创建软链接失败，容器启动失败，删除该容器时，也会出现同样的现象，导致删除失败。

2. 如果创建sandbox时，PodSandboxConfig参数中配置了log\_directory，且container创建时在ContainerConfig中指定log\_path，那么最终的LOGPATH=log\_directory/log\_path，isulad不会递归的创建LOGPATH，因而用户必须保证dirname(LOGPATH)存在，即最终的日志文件的上一级路径存在。

3. 如果创建sandbox时，PodSandboxConfig参数中配置了log\_directory，如果有两个或多个container创建时在ContainerConfig中指定了同一个log\_path，或者不同的sandbox内的容器最终指向的LOGPATH是同一路径，若容器启动成功，则后启动的容器日志路径会覆盖掉之前启动的容器日志路径。

4. 如果远程镜像仓库中镜像内容发生变化，调用CRI Pull image接口重新下载该镜像时，若本地原来存储有原镜像，则原镜像的镜像名称、TAG会变更为“none”

   举例如下：

   本地已存储镜像：

   ```text
   IMAGE                                        TAG                 IMAGE ID            SIZE
   rnd-dockerhub.huawei.com/pproxyisulad/test   latest              99e59f495ffaa       753kB
   ```

   远程仓库中rnd-dockerhub.huawei.com/pproxyisulad/test:latest 镜像更新后，重新下载后：

   ```text
   IMAGE                                        TAG                 IMAGE ID            SIZE
   <none>                                       <none>              99e59f495ffaa       753kB
   rnd-dockerhub.huawei.com/pproxyisulad/test   latest              d8233ab899d41       1.42MB
   ```

   使用isula images 命令行查询，REF显示为"-"：

   ```text
   REF                                               IMAGE ID               CREATED              SIZE       
   rnd-dockerhub.huawei.com/pproxyisulad/test:latest d8233ab899d41          2019-02-14 19:19:37  1.42MB     
   -                                                 99e59f495ffaa          2016-05-04 02:26:41  753kB
   ```

5. iSulad CRI exec/attach接口采用websocket协议实现，需要采用同样协议的客户端与iSulad进行交互；使用exec/attach接口时，请避免进行串口大量数据及文件的传输，仅用于基本命令交互，若用户侧处理不及时将存在数据丢失的风险；同时请勿使用cri exec/attach接口进行二进制数据及文件传输。

6. iSulad CRI exec/attach流式接口依赖libwebsockets实现，流式接口建议仅用于长连接交互使用，不建议在大并发场景下使用，可能会因为宿主机资源不足导致连接失败，建议并发量不超过100。

---

---
url: /zh/docs/22.03_LTS_SP4/server/releasenotes/cve.md
---
# CVE漏洞

版本涉及的CVE可通过[CVE列表](https://www.openeuler.org/zh/security/cve)查询。

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/dde/dde_userguide.md
---
# DDE Desktop Environment

## Overview

DDE desktop environment is an elegant, secure, reliable and easy to use GUI comprised of the desktop, dock, launcher and control center. Acting as the key basis for our operating system, its main interface is shown as below.

![1|desk](./figures/43.jpg)

### Getting Started

When you enter DDE for the very first time, a welcome program will automatically start. You can watch the introduction video, select your desktop style and icon theme, and learn more about the system functions.

![0|welcome](./figures/46.png)

## Desktop

Desktop is the main screen you see after logging in. On the desktop, you can create a new file/folder, sort files, open in terminal, set wallpaper and screensaver and etc. You can also add shortcuts for applications on desktop by using [Send to desktop](#set-app-shortcut).

![0|contextmenu](./figures/41.png)

### Create New Folder/Document

Just as in File Manager, you can create a new folder/document on the desktop, or do some operations for the files on it.

* Right-click the desktop, select **New folder** and enter the name for it.
* Right-click the desktop, select **New document**, select the type and enter its name.

Right-click a file or folder on the desktop, and use the features of File Manager as below:

| Function         | Description                                                  |
| ---------------- | ------------------------------------------------------------ |
| Open with        | Select an app to open it.                                    |
| Cut              | Move it to another location.                                 |
| Copy             | Copy it to another location.                                 |
| Rename           | Change its name.                                             |
| Delete           | Delete and move it to the trash.                             |
| Create link      | Create a shortcut of the file or folder.                     |
| Tag information  | Add a tag.                                                   |
| Compress/Extract | Compress the file or folder, or extract the compressed file. |
| Properties       | View the basic info, share it or change the permission.      |

### Sort Files

Sort the files on your desktop to make it organized and fit your needs.

1. Right-click the desktop.
2. Click **Sort by**, you can:

* Click **Name** to display files in the name sequence.
* Click **Size** to display files in the size sequence.
* Click **Type** to display files in type.
* Click **Time modified** to display files in the order of last modified date.

> ![tips](./figures/icon125-o.svg)Tips: *Check **Auto arrange**, icons on the desktop will be listed in order automatically, and if an icon is removed, another one will fill in the blank.*

### Adjust Icon Size

1. Right-click the desktop.
2. Click **Icon size**, and choose a proper size.

> ![tips](./figures/icon125-o.svg)Tips: *Press **Ctrl** + ![=](./figures/icon134-o.svg)/![-](./figures/icon132-o.svg) scrolling mouse wheel to adjust icon size on the desktop and in Launcher.*

### Set Display

You can set display scaling, screen resolution, brightness and so on from the desktop.

1. Right-click the desktop.
2. Click **Display Settings** to open the settings in Control Center.

> ![notes](./figures/icon99-o.svg)Notes: *For specific operations, please refer to [Display](#display).*

### Change Wallpaper

Select some elegant and fashionable wallpapers to beautify your desktop and make it distinctive.

1. Right-click the desktop.
2. Click **Wallpaper and Screensaver** to preview all the wallpapers.
3. Click your favorite one and it will apply in your desktop and screen lock.
4. You can also choose **Only desktop** or **Only lock screen**.

![1|wallpaper](./figures/63.jpg)

> ![tips](./figures/icon125-o.svg)Tips: *You can also set your favorite picture as wallpaper in an image viewer.*

### Clipboard

All the texts, pictures and documents cut and copied by the current user after login are displayed in the clipboard, which can be copied quickly by double-clicking the clipboard. The clipboard is cleared automatically after logout and shutdown.

1. Use the shortcuts **Ctrl**+**Alt**+ **V** to wake up the clipboard.

2. Double-click in the clipboard to copy the current content quickly and the corresponding block will be moved to the top of the clipboard.

3. Select the target destination to paste it.

4. Click![close](./figures/icon57-o.svg)to delete the current content and click **Clear All** to clear the clipboard.

   ![1|clipboard](./figures/40.png)

## Dock

Dock is at the bottom of the desktop by default to help you quickly open frequently-used applications, which includes Launcher, applications, system tray, and plugins. In the dock, you can open launcher, show the desktop, enter the workspaces, open and exit apps, set input methods, adjust the volume, connect to the network, view the calendar and enter the shutdown interface, and so on.

### Icons on Dock

In the Dock, there are icons of Launcher, applications, system tray, and plugins.

![1|fashion](./figures/45.png)

| Icon | Description |
| ---- | ---- |
| ![launcher](./figures/icon66-o.svg) | Launcher - click to view all the installed applications. |
| ![deepin-toggle-desktop](./figures/icon69-o.svg) | Click to show the desktop. |
| ![dde-file-manager](./figures/icon63-o.svg) | File Manager - click to view files and folders on the disk. |
| ![dde-calendar](./figures/icon62-o.svg) | Calendar -  view dates and create new schedules. |
| ![controlcenter](./figures/icon58-o.svg) |   Control Center - click to check or change system settings.   |
| ![notification](./figures/icon101-o.svg) | Notification Center - show all notifications from the system and applications. |
| ![onboard](./figures/icon103-o.svg) | Onboard virtual keyboard. |
| ![shutdown](./figures/icon122-o.svg) | Click to enter the shutdown interface. |
| ![trash](./figures/icon126-o.svg) | Trash. |

> ![tips](./figures/icon125-o.svg)Tips: *In Efficient Mode, you can click the right side of Dock to show the desktop. Move the cursor to the running app in the Dock and you will see its preview window.*

### Switch Display Mode

There are two display modes of Dock: fashion mode and efficient mode, icon sizes are different in them.

![1|fashion](./figures/46.png)

![1|efficient](./figures/63.png)

You can switch the display modes by the following operations:

1. Right-click the Dock and select **Mode**.
2. Select the display mode.

### Change Dock Location

You can place Dock on any direction of your desktop.

1. Right-click the Dock and select **Location**.
2. Select a location.

### Change Dock Height

Drag the top edge to increase or decrease the height.

### Show/Hide Plugins

1. Right-click the Dock and select **Plugins**.
2. On the submenu, you can check or uncheck **Trash, Power, Show Desktop, Onboard**, and **Datetime** to show or hide the corresponding icon in the Dock.

### View Notifications

When there are system or application notifications, they will be shown in the middle of the screen. If there are buttons in the message, click buttons to do the actions; if there are not, click the message to close it.

![notification](./figures/51.png)

Click  in Dock to view all the notifications.

### View Date and Time

* Hover the cursor over the Time icon in Dock to view the current time, date and day of the week.
* Click the Time icon to open Calendar.

### Enter Shutdown Interface

There are two ways to enter the shutdown interface:

* Click ![shutdown](./figures/icon122-o.svg) in Dock.
* Click ![poweroff\_normal](./figures/icon136-o.svg) at the bottom right corner of Launcher mini mode.

| Function                                                     | Description                                                  |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| Shut down ![poweroff\_normal](./figures/icon136-o.svg)        | Shut down the computer.                                      |
| Reboot ![reboot\_normal](./figures/icon110-o.svg)             | Restart the computer.                                        |
| Lock ![lock\_normal](./figures/icon90-o.svg)                  | Lock the computer with the password. Or press **Super** + **L** to lock it. |
| Switch user ![userswitch\_normal](./figures/icon128-o.svg)    | Log in with another user account.                            |
| Log out ![logout\_normal](./figures/icon92-o.svg)             | End all the processes and initialize the system.             |
| Start system monitor![deepin-system-monitor](./figures/icon68-o.svg) | View the running processes and end the one you want.         |

> ![notes](./figures/icon99-o.svg)Notes: ![userswitch\_normal](./figures/icon128-o.svg) *will be shown if there are multiple accounts in the system.*

### Trash

You can find all deleted files in the trash, which can be restored or emptied.

#### Restore Files

You can restore deleted files in Trash or press **Ctrl** + **Z** to restore the lately deleted files.

1. Select the file in the trash.
2. Right-click the file and select **Restore**.
3. The file will be in its original path.

> ![attention](./figures/icon52-o.svg)Attention: *If the original folder of the file has been deleted, the deleted file will be restored to a new folder automatically created.*

#### Empty Trash

In the trash, click **Empty** to permanently delete all the files in the trash.

## Launcher

Launcher ![launcher](./figures/icon66-o.svg)  helps you manage all the installed applications, where you can quickly find an application by category navigation or by a search.

> ![tips](./figures/icon125-o.svg)Tips: *You can view newly installed applications in Launcher. The newly-installed ones are followed with a blue dot.*

### Switch Launcher Modes

There are two display modes of Launcher: fullscreen mode and mini mode. Click the icon at the upper right corner to switch modes.

Both modes support searching applications and sending them to the desktop or Dock.

The mini mode also supports opening File Manager, Control Center and shutdown interface directly.

![1|fullscreen](./figures/47.jpg)
![1|mini](./figures/52.png)

### Sort Applications

In fullscreen mode, all applications in Launcher are listed by the installation time by default. You can sort the application icons as the ways below:

* Hover the cursor over an application icon, hold down the left key of mouse, drag and drop the application icon to arrange it freely.
* Click the category icon ![category](./figures/icon56-o.svg) on the upper left in Launcher to arrange the icons by category.

![1|sortapp](./figures/60.jpg)

In mini mode,  applications are displayed according to using frequency by default.

### Find Applications

In Launcher, you can scroll up and down to find an application, or locate it with the category navigation.

If you already know the application name, just search for it.

### Set App Shortcut

The shortcut offers a method to run applications easily and quickly.

#### Create App Shortcut

Send the application icon to the desktop or Dock to facilitate the follow-up operations.

In Launcher, right-click an app icon and you can:

* Select **Send to desktop** to create a shortcut on the desktop.
* Select **Send to dock** to fix the application icon in Dock.

![0|sendto](./figures/58.png)

> ![notes](./figures/icon99-o.svg)Notes: *You can drag the application icon from Launcher to Dock. But you cannot drag and drop the application while it is running. Then you can right-click the application icon in Dock and select **Dock** to fix it in order to open it quickly for the next time.*

#### Delete Shortcut

Delete a shortcut from the desktop directly, or remove it from Dock or Launcher.

**Remove the shortcut from Dock:**

* Hold down the left key of mouse, drag and drop the icon away from Dock.
* You cannot drag and drop the application icon while it is running. Then you can right-click the application icon in Dock and select **Undock** to remove it from Dock.

**Remove the shortcut from Launcher:**

In Launcher, right-click the icon and you can:

* Select **Remove from desktop** to delete the shortcut from the desktop.
* Select **Remove from dock** to remove the application icon from Dock.

> ![notes](./figures/icon99-o.svg)Notes: *The above operations only delete the shortcut rather than uninstall the applications.*

### Run Applications

For the applications whose shortcuts have been created on the desktop or Dock, you can open them in the following ways:

* Double-click the desktop icon or right-click it and select **Open**.
* Click the application icon in Dock or right-click it and select **Open**.

To open the application only shown in Launcher, click the icon or right-click it and select **Open**.

> ![tips](./figures/icon125-o.svg)Tips: *For the frequently-used applications, right-click the app icon and select **Add to startup** to run it when the computer boots.*

## Control Center

You can manage the system settings in Control Center, including account management, network settings, date and time, personalization, display settings, etc. After entering the desktop environment, click ![controlcenter](./figures/icon58-o.svg) to open Control Center.

### Homepage Introduction

The homepage of Control Center provides several setting modules and click one to enter the detailed settings.

![0|dcchomepage](./figures/42.png)

Once you open a setting module in Control Center, the navigation appears on the left. Click the left navigation to quickly switch to other settings.

![0|cc-navigation](./figures/39.png)

#### Title Bar

The title bar contains the back button, search box, main menu and the window buttons.

* Back button: Click ![back](./figures/icon53-o.svg) to go back to the homepage.
* Search box: Input a keyword and search the related settings.
* Main menu: Click ![menu](./figures/icon83-o.svg) to enter the main menu where you can set the window theme, view the manual and exit.

### Accounts

You have already created an account when installing the system. Here you can modify account settings or create a new one.

![0|account](./figures/38.png)

#### Create New Account

1. On the homepage of Control Center, click ![account\_normal](./figures/icon49-o.svg).
2. Click ![add](./figures/icon50-o.svg).
3. Input a username and a password twice.
4. Click **Create**.
5. Input the password of  the current user account in the authentication dialog box, and the new account will be added to the account list.

#### Change Account Avatar

1. On the homepage of Control Center, click ![account\_normal](./figures/icon49-o.svg).
2. Click an existing account in the list.
3. Click the user avatar.
4. Select a avatar or upload a local avatar.

#### Set Full Name

The account full name is shown in account list and system login interface and you can set it as needed.

1. On the homepage of Control Center, click ![account\_normal](./figures/icon49-o.svg).
2. Click an existing account in the list.
3. Click ![edit](./figures/icon75-o.svg) after **Full Name**, and input a name.

#### Change Password

1. On the homepage of Control Center, click ![account\_normal](./figures/icon49-o.svg).

2. Click the current account.

3. Click **Change Password**.

4. Input a new password twice and confirm.

#### Delete Account

1. On the homepage of Control Center, click ![account\_normal](./figures/icon49-o.svg).
2. Click an account that's not logged in.
3. Click **Delete Account**.
4. Click **Delete** in the pop-up window.

> ![attention](./figures/icon52-o.svg)Attention: *The logged in account cannot be deleted.*

#### Privilege

The first account has administrator privilege when you install the system. All other accounts you add after that are common users. One account can be grouped in many user groups.

##### Group setting

When you add or modify accounts, you can:

* Select a group existing in the system.
* Select the group with the same name as the current user.
* Select the group with the same name as another user when the account was previously added.

### Display

Set screen resolution, brightness, direction and display scaling properly to have the best visual effect.

![0|display](./figures/44.png)

#### Single Screen Settings

##### Change Resolution

1. On the homepage of Control Center, click ![display\_normal](./figures/icon72-o.svg).
2. Click **Resolution**.
3. Select a proper resolution in the list.
4. Click **Save**.

##### Adjust Brightness

1. On the homepage of Control Center, click ![display\_normal](./figures/icon72-o.svg).
2. Click **Brightness**.
   * Drag the slider to set screen brightness.
   * Switch on **Night Shift**, the screen hue will be auto-adjusted according to your location.
   * Switch on **Auto Brightness**, the monitor will change the brightness automatically according to ambient light (shown only if PC has a light sensor).

##### Change Refresh Rate

1. On the homepage of Control Center, click ![display\_normal](./figures/icon72-o.svg).
2. Click **Refresh Rate**.
3. Select a proper one, and click **Save**.

##### Change Display Direction

1. On the homepage of Control Center, click ![display\_normal](./figures/icon72-o.svg).
2. Click ![rotate](./figures/icon112-o.svg).
3. Every time you click, the screen will rotate 90 degrees counterclockwise.
4. To restore to the original direction, click the right button to exit; to use the current direction,  press **Ctrl**+ **S** to save it.

#### Multiple Screen Settings

Expand your desktop by multiple screens! Use VGA/HDMI/DP cable to connect your computer to other display devices.

1. On the homepage of Control Center, click ![display\_normal](./figures/icon72-o.svg).
2. Click **Multiple Displays**.
3. Select a display mode:
   * **Duplicate**: display the same image on other screens.
   * **Extend**: expand the desktop across the screens.
   * **Customize**: customize the display settings for multiple screens.

In multiple displays, press **Super** + **P** to show its OSD.

Operations are as follows:

1. Hold **Super** and press **P** or click to select the options.
2. Release the keys, the selected mode will take into effect.

> ![notes](./figures/icon99-o.svg)Notes: *When the multiple displays are in the extend mode, only the main screen supports desktop icon display, right-click menu operation and other functions, while the sub-screens do not.*

##### Custom Settings

1. On the homepage of Control Center, click ![display\_normal](./figures/icon72-o.svg).
2. Click **Multiple Displays** > **Customize**.
3. Click **Recognize**.
4. Choose **Merge** or **Split** the screens, specify the main screen, set the resolution and refresh rate, and rotate screen if you want.
5. Click **Save**.

> ![notes](./figures/icon99-o.svg)Notes: *"Merge" means duplicate mode, "Split" means extend mode.*

### Default Application Settings

If you have installed several applications with similar functions, such as text editor, choose one of them to be the default application to open that type of file.

![0|default](./figures/39.png)

#### Set Default Application

1. Right-click the file, choose **Open with** > **Set default program**.
2. Select one application, **Set as default** is checked by default, and click **Confirm**.
3. The application will automatically be added to the default application list in Control Center.

#### Change Default Application

1. On the homepage of Control Center, click ![default\_applications\_normal](./figures/icon70-o.svg).
2. Select a file type.
3. Select another one in the list as the default application.

#### Add Default Application

1. On the homepage of Control Center, click ![default\_applications\_normal](./figures/icon70-o.svg).
2. Select a file type.
3. Click ![add](./figures/icon50-o.svg) below to add a desktop file (usually at /usr/share/applications) or a specified binary file as the default application.
4. The application will be added to the list and set as default application automatically.

#### Delete Default Application

In the default application list, you can only delete the applications you added. To remove other applications from the list,  the only way is to uninstall them. Once uninstalled, they will automatically be deleted from the list.

To delete the default applications you have added, do as below:

1. On the homepage of Control Center, click ![default\_applications\_normal](./figures/icon70-o.svg).
2. Select a file type.
3. Click ![close](./figures/icon57-o.svg) after the application name to delete it.

### Personalization Settings

You can set theme, accent color, font, change the appearance of the desktop and windows to your favorite style.

![0|personalise](./figures/56.png)

#### Set Window Theme

1. On the homepage of Control Center, click ![personalization\_normal](./figures/icon105-o.svg).
2. Click **General**.
3. Select one window theme, which will be used as system theme.

> ![notes](./figures/icon99-o.svg)Notes: *"Auto" means changing window theme automatically according to the sunset and sunrise time. After sunrise, it is light theme; after sunset, it is dark theme.*

#### Change Accent Color

Accent color refers to the color used when you select one option or file in the system.

1. On the homepage of Control Center, click ![personalization\_normal](./figures/icon105-o.svg).
2. Click **General**.
3. Pick a color under **Accent Color** and view its effects.

#### Set Icon Theme

1. On the homepage of Control Center, click ![personalization\_normal](./figures/icon105-o.svg).
2. Click **Icon Theme** and select an icon style.

#### Set Cursor Theme

1. On the homepage of Control Center, click ![personalization\_normal](./figures/icon105-o.svg).
2. Click **Cursor Theme** and select a set of cursors.

#### Change Font

1. On the homepage of Control Center, click ![personalization\_normal](./figures/icon105-o.svg).
2. Click **Font**.
3. Set the font and font size for the system.

### Network Settings

After login, you need to connect to a network first and then surf the Internet!

> ![tips](./figures/icon125-o.svg)Tips: *Check your network status by hovering over or clicking the network icon in Dock.*

![0|network](./figures/54.png)

#### Wired Network

Wired network is secure and stable, which makes it the most common way to connect to the Internet. After your router is set, connect both ends of the network cable to the computer and router to connect to a wired network.

1. Plug the cable into the network slot of a computer.
2. Plug another end of the cable into the router or network port.
3. On the homepage of Control Center, click ![network\_normal](./figures/icon97-o.svg).
4. Click **Wired Network** to enter the setting page of wired network.
5. Switch on **Wired Network Adapter** to enable wired network.
6. If it is successfully connected to the network, there will be a prompt "Wired Connection connected".

You can also edit and add a new wired network in the setting page.

#### Mobile Network

If you are at a place without network, mobile network adapter is a useful tool to help you connect to the Internet as long as the place is covered by telephone signals.

1. Plug the mobile network adapter into your computer USB port.
2. Your computer will auto connect to the network.
3. On the homepage of Control Center, click ![network\_normal](./figures/icon97-o.svg).
4. Click **Mobile Network** to view the detailed network info.

#### DSL/PPPoE Connections

DSL is a dial-up connection using a standard phone line and analog modem to access the Internet. Configure the modem, plug the telephone line into the network interface of the computer, create a broadband dial-up connection, and enter the user name and password provided by the operator to dial up the Internet.

##### Create a PPPoE Connection

1. On the homepage of Control Center, click ![network\_normal](./figures/icon97-o.svg).
2. Click **DSL**.
3. Click ![add](./figures/icon50-o.svg).
4. Enter the name, your account and password the operator provides.
5. Click **Save**. The connection will automatically start.

#### VPN

VPN is a virtual private network. Its main function is to establish a private network on the public network for encrypted communication. Whether you are on a business trip or working at home, you can use VPN to access intranet resources as long as you can access the Internet. You can also use VPN to speed up access to websites in other countries.

1. On the homepage of Control Center, click ![network\_normal](./figures/icon97-o.svg).
2. Click **VPN**, and click ![add](./figures/icon50-o.svg) or ![import](./figures/icon84-o.svg).
3. Select the VPN protocol type, and enter the name, gateway, account, password and other information. (Importing VPN will automatically fill in information)
4. Click **Save**,  the system will try to connect VPN network automatically.
5. You can export the VPN settings to backup or share with other users.

> ![notes](./figures/icon99-o.svg)Notes: *If you don't want to use the VPN as the default routing, but only want it to take effect on specific network resources, switch on **Only applied in corresponding resources**.*

#### System Proxy

1. On the homepage of Control Center, click ![network\_normal](./figures/icon97-o.svg).
2. Click **System Proxy**.

* Click **None** and **Save** to disable the proxy.
* Click **Manual** and input the address and port of proxy servers.
* Click **Auto** and input a URL to configure the proxy info.

#### Application Proxy

1. On the homepage of Control Center, click ![network\_normal](./figures/icon97-o.svg).
2. Click **Application Proxy**.
3. Select a proxy type, and fill in the IP address, port, etc.
4. Click **Save** to save the proxy settings.

> ![notes](./figures/icon99-o.svg)Notes: *After being configured, run Launcher, right-click any application's icon and check **Use a proxy**, and then the application will be opened by proxy.*

#### Network Info

You can view MAC, IP address, gateway and other network info in network details.

1. On the homepage of Control Center, click ![network\_normal](./figures/icon97-o.svg).
2. Click **Network Details**.
3. View the network info of the current network.

### Sound Settings

Set your speaker and microphone properly to make you hear more comfortable and make clearer recordings.

![0|sound](./figures/61.png)

#### Output

1. On the homepage of Control Center, click ![sound\_normal](./figures/icon116-o.svg).

2. Click **Output** to:

   * Select output device type from the dropdown list after **Output Device**.

   * Drag the slider to adjust output volume and left/right balance.

   * Switch on **Volume Boost**, the volume could be adjustable from 0~150% (the former range is 0~100%).

#### Input

1. On the homepage of Control Center, click ![sound\_normal](./figures/icon116-o.svg).
2. Click **Input** to:
   * Select input device type from the dropdown list after **Input Device**.
   * Adjust input volume by dragging the slider.
   * You can enable **Automatic Noise Suppression** by clicking the button after "Automatic Noise Suppression".

> ![tips](./figures/icon125-o.svg)Tips: *Usually, you need to turn up the input volume to make sure that you can hear the sound of the sound source, but the volume should not be too high, because it will cause distortion of the sound. Here is how to set input volume: Speak to your microphone at a normal volume and view "Input Level". If the indicator changes obviously according to the volume, then the input volume is at a proper level.*

#### System Sound Effects

1. On the homepage of Control Center, click ![sound\_normal](./figures/icon116-o.svg).
2. Click **Sound Effects**, check the options you want to switch on the sound when the corresponding event occurs.

> ![tips](./figures/icon125-o.svg)Tips: *Click to listen to the sound effect.*

### Date and Time

Set your timezone properly to have correct date and time. You can also change them manually.

![0|time](./figures/62.png)

#### Change Timezone

You have selected the timezone during system installation and do as follows to change it.

1. On the homepage of Control Center, click ![time](./figures/icon124-o.svg).
2. Click **Timezone List**.
3. Click **Change System Timezone** and select a timezone by searching or clicking on the map.
4. Click **Confirm**.

#### Add Timezone

Add another timezone to see the date and time there.

1. On the homepage of Control Center, click ![time](./figures/icon124-o.svg).
2. Click **Timezone List**.
3. Click ![add](./figures/icon50-o.svg), select a timezone by searching or clicking on the map.
4. Click **Add**.

#### Delete Timezone

1. On the homepage of Control Center, click ![time](./figures/icon124-o.svg).
2. Click **Timezone List**.
3. Click **Edit** after "Timezone List".
4. Click ![delete](./figures/icon71-o.svg) to remove the timezone.

#### Change Date and Time

Note that the auto-sync function will be disabled after changing date and time manually.

1. On the homepage of Control Center, click ![time](./figures/icon124-o.svg).
2. Click **Time Settings**.
   * Switch on/off **Auto Sync**.
   * Enter the correct date and time.
3. Click **Confirm**.

#### Set Time Format

Setting the format of time and date is supported.

1. On the homepage of Control Center, click ![time](./figures/icon124-o.svg).
2. Click **Time Format** to set the first day of week, long date, short date, long time, and short time.

### Power Management

Power management helps you to improve system safety.

![0|power](./figures/57.png)

#### Time to Suspend

1. On the homepage of Control Center, click ![power\_normal](./figures/icon107-o.svg).
2. Click **Plugged In**.
3. Set the time to suspend.

#### Time to Lock Screen

1. On the homepage of Control Center, click ![power\_normal](./figures/icon107-o.svg).
2. Click **Plugged In**.
3. Set the time to lock screen.

#### Power button settings

1. On the homepage of Control Center, click ![power\_normal](./figures/icon107-o.svg).
2. Click **Plugged In**.
3. You can select **Shut down, Suspend, Hibernate, Turn off the monitor, Do nothing** from the drop-down list after **When pressing the power button**.

Any operation done here will take effect immediately. At the same time, the system will notify the user that the power button setting is changed.

### Mouse

Mouse is common computer input device. Using the mouse, you can make the operation easier and faster.

![0|mouse](./figures/53.png)

#### General Settings

1. On the homepage of Control Center, click ![mouse\_touchpad\_normal](./figures/icon94-o.svg).
2. Click **General**.
3. Switch on **Left Hand**, and adjust **Scrolling Speed**, **Double-click Speed**.

> ![notes](./figures/icon99-o.svg)Notes: *If "Left Hand" is enabled, left-click and right-click of the mouse exchange.*

#### Mouse

After inserting or connecting the mouse, make relevant settings in the Control Center to make it more in line with your usage habits.

1. On the homepage of Control Center, click ![mouse\_touchpad\_normal](./figures/icon94-o.svg).
2. Click **Mouse**.
3. Adjust **Pointer Speed**, which helps you to control the speed at which the pointer moves as the mouse moves.
4. Switch on **Natural Scrolling**/**Mouse Acceleration** if you want.

> ![notes](./figures/icon99-o.svg)Notes:
>
> * *Turn on the mouse acceleration to improve the accuracy of the pointer. The moving distance of the mouse pointer on the screen will increase according to the acceleration of the moving speed. It can be turned on or off according to the usage.*
> * *If Natural Scrolling is enabled, when you scroll down, the page will scroll down, when you scroll up, the page will scroll up as well.*

### Keyboard and Language

Set keyboard properties and select your keyboard layout to keep your typing habit. You can also adjust the keyboard layout according to the country and language, change system language, and customize shortcuts here.

![0|keyboard](./figures/59.png)

#### Keyboard Properties

1. On the homepage of Control Center, click ![keyboard\_normal](./figures/icon86-o.svg).
2. Click **General**.
3. Adjust **Repeat Delay**/**Repeat Rate**.
4. Click "Test here" and hold down a key to test the repeat rate.
5. Switch on **Numeric Keypad** and **Caps Lock Prompt** if you want.

#### Keyboard Layout

Set the keyboard layout to customize the keyboard for the current language. When you press a key on the keyboard, the keyboard layout controls which characters are displayed on the screen. After changing the keyboard layout, the characters on the screen may not match the characters on the keyboard keys.

You have set a keyboard layout during system installation, but you can add more for other purposes.

![layout](./figures/50.png)

##### Add Keyboard Layout

1. On the homepage of Control Center, click ![keyboard\_normal](./figures/icon86-o.svg).
2. Click **Keyboard Layout**.
3. Click ![add](./figures/icon50-o.svg). Click a keyboard layout to add it.

##### Delete Keyboard Layout

1. On the homepage of Control Center, click ![keyboard\_normal](./figures/icon86-o.svg).
2. Click **Keyboard Layout**.
3. Click **Edit**.
4. Click ![delete](./figures/icon71-o.svg) to delete keyboard layout.

##### Switch Keyboard Layout

1. On the homepage of Control Center, click ![keyboard\_normal](./figures/icon86-o.svg).
2. Click **Keyboard Layout**.
3. Click the layout you want to switch to.
4. After successful switching, the layout will be marked with a check.

> ![tips](./figures/icon125-o.svg)Tips: *You can also select one or more shortcuts to switch the keyboard layouts in order. Select **Applies to** to make the keyboard layout after switching be applied to the whole system or current application.*

#### System Language

The system language is the language you selected when you installed the system by default, which can be changed at any time.

##### Add System Language

Add multiple languages into the list to change language conveniently.

1. On the homepage of Control Center, click ![keyboard\_normal](./figures/icon86-o.svg).
2. Click **System Language**.
3. Click ![add](./figures/icon50-o.svg) to enter the language list.
4. Select the language you want, and it will be added into system language list automatically.

##### Change System Language

1. On the homepage of Control Center, click ![keyboard\_normal](./figures/icon86-o.svg).
2. Click **System Language**.
3. Select the language you want to switch to, and the language package will be installed automatically.
4. After being successfully installed, log out and log in again to view the changes.

> ![attention](./figures/icon52-o.svg)Attention: *The keyboard layout may also be changed in the process of switching the system language. Please make sure that you select a correct keyboard layout to enter the login password.*

#### Shortcuts

The shortcut list includes all shortcuts in the system. View, modify and customize the shortcuts here as you want.

![0|shortcut](./figures/59.png)

##### View Shortcuts

1. On the homepage of Control Center, click ![keyboard\_normal](./figures/icon86-o.svg).
2. Click **Shortcuts**.
3. You can search or view the default shortcuts for system, window and workspace.

##### Modify Shortcuts

1. On the homepage of Control Center, click ![keyboard\_normal](./figures/icon86-o.svg).
2. Click **Shortcuts**.
3. Click the shortcut you want to modify.
4. Press new shortcut to change it.

> ![tips](./figures/icon125-o.svg)Tips: *To disable a shortcut, please press ![Backspace](./figures/icon54-o.svg) on the keyboard. To cancel modifying, press **Esc** or click Restore Defaults at the bottom.*

##### Customize Shortcuts

1. On the homepage of Control Center, click ![keyboard\_normal](./figures/icon86-o.svg).
2. Click **Shortcuts**.
3. Click ![add](./figures/icon50-o.svg).
4. Enter the name, command and shortcut.
5. Click **Add**.
6. After being successfully added, click **Edit**.
7. Click ![delete](./figures/icon71-o.svg) to delete the custom shortcut.

> ![tips](./figures/icon125-o.svg)Tips: *To change the shortcut, click it and press a new shortcut to change it directly. To edit the name and command of the custom shortcut, click\*\*Edit \*\* > ![edit](./figures/icon75-o.svg) near the shortcut name to enter the shortcut settings.*

### System Info

You can view system version, authorization info, hardware info, and the agreements here.

![0|info](./figures/48.png)

#### About This PC

1. On the homepage of Control Center, click ![system\_info\_normal](./figures/icon120-o.svg).
2. Under **About This PC**, you can view system version, authorization and hardware information.
3. If the system has not been activated, click **Activate** to activate the system.

#### Edition License

1. On the homepage of Control Center, click ![system\_info\_normal](./figures/icon120-o.svg).
2. View the system edition license under **Edition License**.

#### End User License Agreement

1. On the homepage of Control Center, click ![system\_info\_normal](./figures/icon120-o.svg).
2. View the End User License Agreement under **End User License Agreement**.

## Keyboard Interaction

You can use the keyboard to switch between various interface areas, select objects and perform operations.

| Key                                                          | Function                                                     |
| :----------------------------------------------------------- | :----------------------------------------------------------- |
| **Tab**                                                      | Switch between different areas or dialog buttons.            |
| ![Up](./figures/icon127-o.svg)   ![Down](./figures/icon73-o.svg)     ![Left](./figures/icon88-o.svg) ![Right](./figures/icon111-o.svg) | Used to select different objects in the same area. Press ![Right](./figures/icon111-o.svg) to enter the lower menu and  ![Left](./figures/icon88-o.svg) to return to the upper menu. Press![Up](./figures/icon127-o.svg)and ![Down](./figures/icon73-o.svg) to switch between up and down. |
| **Enter**                                                    | Execute the selected operation.                              |
| **Space**                                                    | Preview the selected object in File Manager; start and pause the playback in Music and Movie; expand the drop-down options in the drop-down list (The enter key is also available.). |
| **Ctrl** + **M**                                             | Open the right-click menu.                                   |

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/dde/dde_installation.md
---
# DDE Installation

## Introduction

DDE is a powerful desktop environment developed by UnionTech. It contains dozens of self-developed desktop applications.

## Procedure

1. [Download](https://openeuler.org/en/download/) the openEuler ISO file and install the OS.

2. Update the software source.

   ```bash
   sudo dnf update
   ```

3. Install DDE.

   ```bash
   sudo dnf install dde
   ```

4. Set the system to start with the graphical interface.

   ```bash
   sudo systemctl set-default graphical.target
   ```

5. Reboot the system.

   ```bash
   sudo reboot
   ```

6. After the reboot is complete, use the user created during the installation process or the **openeuler** user to log in to the desktop.

   > ![notes](./figures/icon99-o.svg)Notes:
   >
   > DDE does not allow login as the root user.
   > DDE has a built-in openeuler user whose password is openeuler.

Now you can use DDE.

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/dde/dde_userguide.md
---
# DDE桌面环境用户手册

## 概述

DDE桌面环境是一款美观易用、安全可靠的图形化操作界面。桌面环境主要由桌面、任务栏、启动器、控制中心等组成，是您使用该操作系统的基础，主界面如下图所示。

![1|desk](./figures/43.jpg)

### 欢迎

初次进入DDE桌面环境，会自动打开欢迎程序。您可以观看视频了解系统功能，选择桌面样式和图标主题，进一步了解该系统。

![welcome](./figures/64.png)

## 桌面

桌面是您登录后看到的主屏幕区域。在桌面上，您可以新建文件/文件夹、排列文件、打开终端、设置壁纸和屏保等，还可以通过启动器 [发送到桌面](#设置快捷方式) 向桌面添加应用的快捷方式。

![0|rightbuttonmenu](./figures/41.png)

### 新建文件夹/文档

在桌面新建文件夹或文档，也可以对文件进行常规操作，和在文件管理器中一样。

* 在桌面上，单击鼠标右键，单击 **新建文件夹**，输入新建文件夹的名称。
* 在桌面上，单击鼠标右键，单击 **新建文档**，选择新建文档的类型，输入新建文档的名称。

在桌面文件或文件夹上，单击鼠标右键，您可以使用文件管理器的相关功能：

| 功能        | 说明                                                     |
| ----------- | -------------------------------------------------------- |
| 打开方式    | 选定系统默认打开方式，也可以选择其他关联应用程序来打开。 |
| 剪切        | 移动文件或文件夹。                                       |
| 复制        | 复制文件或文件夹。                                       |
| 重命名      | 重命名文件或文件夹。                                     |
| 删除        | 删除文件或文件夹。                                       |
| 创建链接    | 创建一个快捷方式。                                       |
| 标记信息    | 添加标记信息，以对文件或文件夹进行标签化管理。           |
| 压缩/解压缩 | 压缩文件或文件夹，或对压缩文件进行解压。                 |
| 属性        | 查看文件或文件夹的基本信息，共享方式，及其权限。         |

### 设置排列方式

您可以对桌面上的图标按照需要进行排序。

1. 在桌面上，单击鼠标右键。
2. 单击 **排序方式**，您可以：
   * 单击 **名称**，将按文件的名称顺序显示。
   * 单击 **大小**，将按文件的大小顺序显示。
   * 单击 **类型**，将按文件的类型顺序显示。
   * 单击 **修改时间**，文件将按最近一次的修改日期顺序显示。

> ![tips](./figures/icon125-o.svg)窍门：*您也可以勾选 **自动排列**，桌面图标将从上往下，从左往右按照当前排序规则排列，有图标被删除时后面的图标会自动向前填充。*

### 调整图标大小

1. 在桌面上，单击鼠标右键。
2. 单击 **图标大小**。
3. 选择一个合适的图标大小。

> ![tips](./figures/icon125-o.svg)窍门：*您也可以用 **Ctrl** + ![=](./figures/icon134-o.svg)/![-](./figures/icon132-o.svg) 鼠标滚动来调整桌面和启动器中的图标大小。*

### 设置显示器

从这里快速进入控制中心设置显示器的缩放比例、分辨率和亮度等。

1. 在桌面上，单击鼠标右键。
2. 单击 **显示设置**，快速进入控制中心的显示设置界面。

> ![notes](./figures/icon99-o.svg)说明：*关于显示的设置，具体操作请参阅 [显示设置](#显示设置) 。*

### 更改壁纸

您可以选择一些精美、时尚的壁纸来美化桌面，让您的电脑显示与众不同。

1. 在桌面上，单击鼠标右键。
2. 单击 **壁纸与屏保**，在桌面底部预览所有壁纸。
3. 选择某一壁纸后，壁纸就会在桌面和锁屏中生效。
4. 您可以单击 **仅设置桌面** 和 **仅设置锁屏** 来控制壁纸的生效范围。

![1|wallpaper](./figures/63.jpg)

> ![tips](./figures/icon125-o.svg)窍门： *您还可以在图片查看器中设置您喜欢的图片为桌面壁纸。*

### 剪贴板

剪贴板展示当前用户登录系统后复制和剪切的所有文本、图片和文件。使用剪贴板可以快速复制其中的某项内容。注销或关机后，剪贴板会自动清空。

1. 使用快捷键 **Ctrl** + **Alt** + **V** 唤出剪贴板。
2. 双击剪贴板内的某一区块，会快速复制当前内容， 且当前区块会被移动到剪贴板顶部。
3. 选择目标位置粘贴。
4. 鼠标移入剪贴板的某一区块，单击上方的![close](./figures/icon57-o.svg)，删除当前内容；单击顶部的 **全部清除**，清空剪贴板。

![1|clipboard](./figures/40.png)

## 任务栏

任务栏是指位于桌面底部的长条，主要由启动器、应用程序图标、托盘区、系统插件等组成。在任务栏，您可以打开启动器、显示桌面、进入工作区，对其上的应用程序进行打开、新建、关闭、强制退出等操作，还可以设置输入法，调节音量，连接网络，查看日历，进入关机界面等。

### 认识任务栏图标

任务栏图标包括启动器图标、应用程序图标、托盘区图标、系统插件图标等。

![1|fashion](./figures/45.png)

| 图标                                        | 说明                                        | 图标                                             | 说明                                  |
| ------------------------------------------- | :------------------------------------------ | ------------------------------------------------ | ------------------------------------- |
| ![launcher](./figures/icon66-o.svg)         | 启动器 - 点击查看所有已安装的应用。         | ![deepin-toggle-desktop](./figures/icon69-o.svg) | 显示桌面。                            |
| ![dde-file-manager](./figures/icon63-o.svg) | 文件管理器 - 点击查看磁盘中的文件、文件夹。 | ![dde-calendar](./figures/icon62-o.svg)          | 日历 - 查看日期、新建日程。           |
| ![controlcenter](./figures/icon58-o.svg)    | 控制中心 - 点击进入系统设置。               | ![notification](./figures/icon101-o.svg)         | 通知中心 - 显示所有系统和应用的通知。 |
| ![onboard](./figures/icon103-o.svg)         | 屏幕键盘 - 点击使用虚拟键盘。               | ![shutdown](./figures/icon122-o.svg)             | 电源 - 点击进入关机界面。             |
| ![trash](./figures/icon126-o.svg)           | 回收站。                                    |                                                  |                                       |

> ![tips](./figures/icon125-o.svg)窍门：*在高效模式下，单击任务栏最右侧可显示桌面。将鼠标指针移到任务栏上已打开窗口的图标时，会显示相应的预览窗口。*

### 切换显示模式

任务栏提供两种显示模式：时尚模式和高效模式，显示不同的图标大小和应用窗口激活效果。

![1|fashion](./figures/46.png)

![1|efficient](./figures/63.png)

您可以通过以下操作来切换显示模式：

1. 右键单击任务栏。
2. 在 **模式** 子菜单中选择一种显示模式。

### 设置任务栏位置

您可以将任务栏放置在桌面的任意方向。

1. 右键单击任务栏。
2. 在 **位置** 子菜单中选择一个方向。

### 调整任务栏高度

鼠标拖动任务栏边缘，改变任务栏高度。

### 显示/隐藏插件

1. 右键单击任务栏。
2. 在 **插件** 子菜单中勾选或取消勾选 **回收站、电源、显示桌面、屏幕键盘、通知中心、时间**，可以设置这些插件在任务栏上的显示和隐藏效果。

### 查看通知

当有系统或应用通知时，会在桌面上方弹出通知消息。若有按钮，单击按钮执行对应操作；若无按钮，单击关闭此消息。

![message](./figures/51.png)

您还可以单击任务栏上的 ![notification](./figures/icon101-o.svg)， 打开通知中心，查看所有通知。

### 查看日期时间

* 鼠标指针悬停在任务栏的时间上，查看当前日期、星期和时间。
* 单击时间，打开日历。

### 进入关机界面

您可以单击任务栏上的 ![shutdown](./figures/icon136-o.svg) 进入关机界面，也可以在启动器的小窗口模式中单击 ![poweroff\_normal](./figures/icon136-o.svg)。

| 功能                                                       | 说明                                                    |
| ---------------------------------------------------------- | ------------------------------------------------------- |
| 关机![poweroff\_normal](./figures/icon136-o.svg)            | 关闭电脑。                                              |
| 重启![reboot\_normal](./figures/icon110-o.svg)              | 关机后再次重新运行您的电脑。                            |
| 锁定![lock\_normal](./figures/icon90-o.svg)                 | 锁定电脑，或按下键盘上的 **Super** + **L** 组合键锁定。 |
| 切换用户![userswitch\_normal](./figures/icon128-o.svg)      | 选择另一个用户帐户登录。                                |
| 注销![logout\_normal](./figures/icon92-o.svg)               | 清除当前登录用户的信息。                                |
| 系统监视器![deepin-system-monitor](./figures/icon68-o.svg) | 快速启动系统监视器。                                    |

> ![notes](./figures/icon99-o.svg)说明：*当系统存在多个帐户时才显示 ![userswitch\_normal](./figures/icon128-o.svg)。*

### 回收站

电脑中临时被删除的所有文件您都可以在回收站中找到，回收站中的文件可以被恢复或清空。

#### 还原文件

对于已删除的文件，您可以进入回收站进行还原，或使用 **Ctrl** + **Z** 还原刚删除的文件。

1. 在回收站中，选择要恢复的文件。
2. 单击鼠标右键，选择 **还原**。
3. 还原文件到原来的存储路径下。

> ![attention](./figures/icon52-o.svg)注意：*如果原来所在的文件夹已经删除，还原文件时会自动新建文件夹*。

#### 清空回收站

在回收站中，单击 **清空**，将彻底删除回收站的所有内容。

## 启动器

启动器 ![launcher](./figures/icon66-o.svg) 帮助您管理系统中已安装的所有应用，在启动器中使用分类导航或搜索功能可以快速找到您需要的应用程序。

> ![tips](./figures/icon125-o.svg)窍门：*您可以进入启动器查看新安装的应用。新安装应用的旁边会出现一个小蓝点提示*。

### 切换模式

启动器有全屏和小窗口两种模式。单击启动器界面右上角的图标来切换模式。

两种模式均支持搜索应用、设置快捷方式等操作。

小窗口模式还支持快速打开文件管理器，控制中心和进入关机界面等功能。

![1|fullscreen](./figures/47.jpg)![1|ini](./figures/52.png)

### 排列应用

在全屏模式下，系统默认按照安装时间排列所有应用。

* 将鼠标悬停在应用图标上，按住鼠标左键不放，将应用图标拖拽到指定的位置自由排列。
* 单击启动器界面左上角分类图标![category](./figures/icon56-o.svg)进行排列。

![1|sortapp](./figures/60.jpg)

在小窗口模式下，默认按照使用频率排列应用。

### 查找应用

在启动器中，您可以滚动鼠标滚轮或切换分类导航查找应用。

如果知道应用名称，直接在搜索框中输入关键字，快速定位到需要的应用。

### 设置快捷方式

快捷方式提供了一种简单快捷地启动应用的方法。

#### 创建快捷方式

将应用发送到桌面或任务栏上，方便您的后续操作。

在启动器中，右键单击应用图标，您可以：

* 单击 **发送到桌面**，在桌面创建快捷方式。

* 单击 **发送到任务栏**，将应用固定到任务栏。

![0|sendto](./figures/58.png)

> ![notes](./figures/icon99-o.svg)说明：*您还可以从启动器拖拽应用图标到任务栏上放置。但是当应用处于运行状态时您将无法拖拽固定，此时您可以右键单击任务栏上的应用图标，选择 **驻留** 将应用固定到任务栏，以便下次使用时从任务栏上快速打开。*

#### 删除快捷方式

您既可以在桌面直接删除应用的快捷方式，也可以在任务栏和启动器中删除。

**从任务栏上删除**

* 在任务栏上，按住鼠标左键不放，将应用图标拖拽到任务栏以外的区域移除快捷方式。
* 当应用处于运行状态时您将无法拖拽移除，此时可以右键单击任务栏上的应用图标，选择 **移除驻留** 将应用从任务栏上移除。

**从启动器中删除**

在启动器中，右键单击应用图标，您可以：

* 单击 **从桌面上移除**，删除桌面快捷方式。
* 单击 **从任务栏上移除**，将固定到任务栏上的应用移除。

> ![notes](./figures/icon99-o.svg)说明：*以上操作，只会删除应用的快捷方式，而不会卸载应用。*

### 运行应用

对于已经创建了桌面快捷方式或固定到任务栏上的应用，您可以通过以下途径来打开应用。

* 双击桌面图标，或右键单击桌面图标选择 **打开**。
* 直接单击任务栏上的应用图标，或右键单击任务栏上的应用图标选择 **打开**。

在启动器中，直接单击应用图标打开，或右键单击应用图标选择 **打开**。

> ![tips](./figures/icon125-o.svg)窍门：*对于经常使用的应用，您可以在启动器中，右键单击应用图标选择 **开机自动启动**。*

## 控制中心

DDE桌面操作系统通过控制中心来管理系统的基本设置，包括帐户管理、网络设置、日期和时间、个性化设置、显示设置、系统信息查看等。当您进入桌面环境后，单击任务栏上的 ![controlcenter](./figures/icon58-o.svg) 即可打开控制中心窗口。

### 首页介绍

控制中心首页主要展示各个设置模块，方便日常查看和快速设置。

![2|dcchomepage](./figures/42.png)

打开控制中心的某一设置模块后，可以通过左侧导航栏快速切换到另一设置模块。

![2|cc-navigation](./figures/39.png)

#### 标题栏

标题栏包含返回按钮，搜索框，主菜单及窗口按钮。

* 返回按钮：若要返回首页，单击 ![back](./figures/icon53-o.svg)。
* 搜索框：输入关键字后，回车，搜索相应设置。
* 主菜单：单击![menu](./figures/icon83-o.svg) 进入主菜单。在主菜单中，您可以设置窗口主题，查看版本，或退出控制中心。

### 帐户设置

在安装系统时您已经创建了一个帐户。在这里，您可以修改帐户设置或创建一个新帐户。

![0|account](./figures/38.png)

#### 创建新帐户

1. 在控制中心首页，单击 ![account\_normal](./figures/icon49-o.svg)。
2. 单击![add](./figures/icon50-o.svg)。
3. 输入用户名、密码和重复密码。
4. 单击 **创建**。
5. 在授权对话框输入当前帐户的密码，新帐户就会添加到帐户列表中。

#### 更改头像

1. 在控制中心首页，单击 ![account\_normal](./figures/icon49-o.svg)。
2. 单击列表中的帐户。
3. 单击帐户头像，选择一个头像或添加本地头像，头像就替换完成了。

#### 设置全名

帐户全名会显示在帐户列表和系统登录界面，可根据需要设置。

1. 在控制中心首页，单击 ![account\_normal](./figures/icon49-o.svg)。
2. 单击列表中的帐户。
3. 单击 **设置全名** 后的 ![edit](./figures/icon75-o.svg)，输入帐户全名。

#### 修改密码

1. 在控制中心首页，单击 ![account\_normal](./figures/icon49-o.svg)。
2. 单击当前帐户。
3. 单击 **修改密码**，进入修改密码页面。
4. 输入当前密码、新密码和重复密码。

#### 删除帐户

1. 在控制中心首页，单击 ![account\_normal](./figures/icon49-o.svg)。
2. 单击其他未登录的帐户。
3. 单击 **删除帐户** 。
4. 在弹出的确认界面中单击 **删除**。

> ![attention](./figures/icon52-o.svg)注意: *已登录的帐户无法被删除。*

#### 权限设置

除安装时的第一个帐户是管理员权限外，后面所添加的所有帐户都是普通用户。一个帐户可以在多个用户组内。

##### 设置组

添加或修改帐户时，可以：

* 选择系统内已有的组。
* 选择当前用户同名的组。
* 选择之前添加帐户时和其他用户同名的组。

### 显示设置

设置显示器的分辨率、亮度、屏幕方向等，让您的电脑显示到达最佳状态。

![0|video](./figures/44.png)

#### 单屏设置

##### 更改分辨率

1. 在控制中心首页，单击 ![display\_normal](./figures/icon72-o.svg)。
2. 单击 **分辨率**，进入分辨率设置界面。
3. 在列表中选择合适的分辨率参数。
4. 单击 **保存**。

##### 调节亮度

1. 在控制中心首页，单击 ![display\_normal](./figures/icon72-o.svg)。
2. 单击 **亮度**，进入亮度设置界面。

   * 拖动亮度条滑块，调节屏幕亮度。
   * 打开 **自动调节色温** 开关，开启进入护眼模式，自动调节色温。
   * 打开 **手动调节** 亮度开关，可以调节屏幕亮度 。

##### 设置屏幕刷新率

1. 在控制中心首页，单击 ![display\_normal](./figures/icon72-o.svg)。
2. 单击 **刷新率**。
3. 选择一个合适的刷新率，单击 **保存**。

##### 改变屏幕方向

1. 在控制中心首页，单击 ![display\_normal](./figures/icon72-o.svg)。
2. 单击 ![rotate](./figures/icon112-o.svg) 。
3. 每单击一下鼠标左键，屏幕逆时针旋转90°。
4. 要还原为之前的屏幕方向，单击鼠标右键退出；要使用当前屏幕方向，请按下组合键 **Ctrl** + **S** 保存。

#### 多屏设置

多屏显示，让您的视野无限延伸！使用VGA、HDMI、EDP等线缆将您的电脑和另一台显示器、投影仪等连接起来，同时在多个屏幕显示您电脑上的内容。

1. 在控制中心首页，单击 ![display\_normal](./figures/icon72-o.svg)。
2. 单击 **多屏显示模式**。
3. 选择一种显示模式。
   * **复制** 将主屏的显示内容复制到其他屏幕。
   * **扩展** 将主屏的显示内容扩展到其他屏幕，扩大桌面区域。
   * **自定义** 设置显示模式，主屏、分辨率、刷新率和屏幕旋转方向。

在多屏环境下，按下 **Super** + **P** 调出多屏显示模式的OSD。

详细操作方法如下。

1. 按住 **Super** 不放，再按下 **P** 或鼠标单击来进行模式选择。
2. 松开按键，确认选择，模式生效。

> ![notes](./figures/icon99-o.svg)说明：*当多屏显示模式为扩展模式时，仅主屏支持桌面图标显示、操作右键菜单等功能，而副屏不支持。*

##### 自定义设置

1. 在控制中心首页，单击 ![display\_normal](./figures/icon72-o.svg)。
2. 单击 **多屏显示模式** > **自定义**。
3. 单击 “识别”，查看屏幕名称。
4. 选择“合并”或“拆分”，然后对多个屏幕进行设置，如主屏、分辨率、刷新率，旋转屏幕等。
5. 单击 **保存**。

> ![notes](./figures/icon99-o.svg)说明：*合并即复制模式，拆分即扩展模式。*

### 默认程序设置

当安装有多个功能相似的应用程序时，可以选择其中的一个应用作为对应文件类型的默认启动程序。

![0|default](./figures/39.png)

#### 设置默认程序

1. 右键单击文件，选择 **打开方式** > **选择默认程序**。
2. 选择一个应用，自动勾选"设为默认"，单击 **确定**。
3. 该应用将自动添加到控制中心的默认程序列表。

#### 更改默认程序

1. 在控制中心首页，单击 ![default\_applications\_normal](./figures/icon70-o.svg)。
2. 选择一个文件类型进入默认程序列表。
3. 在列表中选择另一个应用程序。

#### 添加默认程序

1. 在控制中心首页，单击 ![default\_applications\_normal](./figures/icon70-o.svg)。
2. 选择文件类型进入默认程序列表。
3. 单击列表下的![add](./figures/icon50-o.svg)，选择desktop文件（一般在/usr/share/applications)，或特定的二进制文件。
4. 该程序将添加到列表，并自动设置为默认程序。

#### 删除默认程序

在默认程序列表中，您只能删除自己添加的应用程序，不能删除系统已经安装的应用。要删除系统已经安装的应用，只能卸载应用。卸载后该应用将自动从默认程序列表中删除。

可用以下方法删除自己添加的默认程序。

1. 在控制中心首页，单击 ![default\_applications\_normal](./figures/icon70-o.svg)。
2. 选择文件类型进入默认程序列表。
3. 单击程序后面的![close](./figures/icon57-o.svg)，删除默认程序。

### 个性化设置

在这里，您可以设置系统主题、活动用色、字体等，改变桌面和窗口的外观，设置成您喜欢的显示风格。

![0|personalise](./figures/56.png)

#### 设置窗口主题

1. 在控制中心首页，单击 ![personalization\_normal](./figures/icon105-o.svg)。
2. 单击 **通用**，选择一种窗口主题。
3. 该主题即为系统窗口主题。

> ![tips](./figures/icon125-o.svg)窍门：*自动主题表示根据当前时区的时间，根据日出日落的时间自动更换窗口主题。日出后是浅色，日落后是深色。*

#### 更改活动用色

活动用色是指选中某一选项时的强调色。

1. 在控制中心首页，单击 ![personalization\_normal](./figures/icon105-o.svg)。
2. 单击 **通用**。
3. 单击 **活动用色** 下的一种颜色，可实时查看该颜色效果。

#### 设置图标主题

1. 在控制中心首页，单击 ![personalization\_normal](./figures/icon105-o.svg)。
2. 单击 **图标主题**，选择一款图标样式。

#### 设置光标主题

1. 在控制中心首页，单击 ![personalization\_normal](./figures/icon105-o.svg)。
2. 单击 **光标主题**，选择一款光标样式。

#### 更改系统字体

1. 在控制中心首页，单击 ![personalization\_normal](./figures/icon105-o.svg)。
2. 单击 **字体**，进入设置字体界面。
3. 设置系统字号和字体。

### 网络设置

登录系统后，您需要连接网络，才能接收邮件、浏览新闻、下载文件、聊天、网上购物等。

> ![tips](./figures/icon125-o.svg)窍门：*您可以单击任务栏托盘区的网络图标，查看当前网络状态。*

![0|network](./figures/54.png)

#### 有线网络

有线网络安全快速稳定，是最常见的网络连接方式。当您设置好路由器后，把网线两端分别插入电脑和路由器，即可连接有线网络。

1. 将网线插入电脑上的网络插孔。
2. 将网线的另一端插入路由器或网络端口。
3. 在控制中心首页，单击  ![network\_normal](./figures/icon97-o.svg)。
4. 单击 **有线网络**，进入有线网络设置界面。
5. 打开 **有线网卡**，开启有线网络连接功能。
6. 当网络连接成功后，桌面右上角将弹出“已连接有线连接”的提示信息。

您还可以在有线网络的设置界面，编辑或新建有线网络设置。

#### 移动网络

当您处于一个没有网络信号的地方时，可以使用无线上网卡来上网。在有电话信号覆盖的任何地方，无线上网卡通过运营商的移动数据网络接入宽带服务。

1. 将移动网卡插入电脑上的USB接口中。
2. 电脑将根据移动网卡和运营商信息，自动适配并自动连接网络。
3. 在控制中心首页，单击  ![network\_normal](./figures/icon97-o.svg)。
4. 单击 **移动网络**，查看详细设置信息。

#### 拨号网络

拨号上网（DSL）是指通过本地电话拨号连接到网络的连接方式。配置好调制解调器，把电话线插入电脑的网络接口，创建宽带拨号连接，输入运营商提供的用户名和密码，即可拨号连接到Internet上。

##### 新建拨号连接

1. 在控制中心首页，单击  ![network\_normal](./figures/icon97-o.svg)。
2. 单击 **DSL**，单击 ![add](./figures/icon50-o.svg)。
3. 输入宽带名称、帐户、密码。
4. 单击 **保存**，系统自动创建宽带连接并尝试连接。

#### VPN

VPN即虚拟专用网络，其主要功能是在公用网络上建立专用网络，进行加密通讯。无论您是在外地出差还是在家中办公，只要能上网就能利用VPN访问企业的内网资源。您还可以使用VPN加速访问其他国家的网站。

1. 在控制中心首页，单击  ![network\_normal](./figures/icon97-o.svg)。
2. 单击 **VPN**，选择 ![add](./figures/icon50-o.svg) 或 ![import](./figures/icon84-o.svg)。
3. 选择VPN协议类型，并输入名称、网关、帐号、密码等信息。（导入VPN会自动填充信息）
4. 单击 **保存**，系统自动尝试连接VPN网络。
5. 您可以将VPN设置导出，备用或共享给其他用户。

> ![notes](./figures/icon99-o.svg)说明：*打开 **仅用于相对应的网络上的资源** 开关，可以不将VPN设置为默认路由，只在特定的网络资源上生效。*

#### 系统代理

1. 在控制中心首页，单击  ![network\_normal](./figures/icon97-o.svg)。
2. 单击 **系统代理**，进入系统代理界面。
   * 单击 **无**，关闭代理服务器功能。
   * 单击 **手动**，输入代理服务器的地址和端口信息。
   * 单击 **自动**，输入URL，系统将自动配置代理服务器的信息。

#### 应用代理

1. 在控制中心首页，单击  ![network\_normal](./figures/icon97-o.svg)。
2. 单击 **应用代理**。
3. 设置应用代理参数。
4. 单击 **保存**。

> ![notes](./figures/icon99-o.svg)说明：*应用代理设置成功后，打开启动器，右键单击应用图标，可以选择 **使用代理**。*

#### 网络详情

在网络详情界面，您可以查看MAC、IP地址、网关和其他网络信息。

1. 在控制中心首页，单击 ![network\_normal](./figures/icon97-o.svg)。
2. 单击 **网络详情**，进入网络信息界面。
3. 查看当前有线网络或无线网络的信息。

### 声音设置

输入输出设备声音的设置（如设置扬声器和麦克风），让您听得更舒适，录音更清晰。

![0|sound](./figures/61.png)

#### 输出设备

1. 在控制中心首页，单击 ![sound\_normal](./figures/icon116-o.svg)。
2. 单击 **输出**，进入输出设备配置界面，您可以：
   * 在输出设备后面的下拉框中选择输出设备类型。
   * 通过拖曳滑块调节输出音量和左/右声道平衡。
   * 打开 **音量增强**，音量的可调节区间由0~100% 转变为0~150%。

#### 输入设备

1. 在控制中心首页，单击 ![sound\_normal](./figures/icon116-o.svg)。
2. 单击 **输入**，进入输入设备配置界面，您可以：
   * 在输入设备后面的下拉框中选择输入设备类型。
   * 通过拖曳滑块调节输入音量。
   * 打开 **开启** 按钮，还可以设置 **噪音抑制** 功能。

> ![tips](./figures/icon125-o.svg)窍门：*通常，需要调大输入音量，确保能够听到声源的声音，但是音量不宜过大，因为这会导致声音失真。可以对着麦克风以正常说话的音量讲话，并观察反馈音量的变化，变化较明显，则说明输入音量合适。*

#### 系统音效

1. 在控制中心首页，单击 ![sound\_normal](./figures/icon116-o.svg)。
2. 单击 **系统音效**，勾选选项，开启某一事件发生时的声音效果。

> ![tips](./figures/icon125-o.svg)窍门：*您可以单击试听音效。*

### 时间日期

正确选择您所在的时区，一般即可显示正确的日期和时间。您也可以手动修改时间和日期。

![0|time](./figures/62.png)

#### 修改时区

在您安装系统时，已选择了系统时区。若要修改系统时区，请按如下步骤设置。

1. 在控制中心首页，单击 ![time](./figures/icon124-o.svg)。
2. 单击 **时区列表**。
3. 单击 **修改系统时区**， 通过搜索或单击地图选择时区。
4. 单击 **确定**。

#### 添加时区

您可以同时使用多个时区，以便查看另一时区的时间。

1. 在控制中心首页，单击 ![time](./figures/icon124-o.svg)。
2. 单击 **时区列表**。
3. 单击![add](./figures/icon50-o.svg)，通过搜索或单击地图选择时区。
4. 单击 **添加**。

#### 删除时区

1. 在控制中心首页，单击 ![time](./figures/icon124-o.svg)。
2. 单击 **时区列表**。
3. 单击时区列表后面的 **编辑**。
4. 单击 ![delete](./figures/icon71-o.svg)，删除已添加的时区。

#### 修改时间和日期

默认情况下，系统通过网络自动同步该时区的本地时间和日期。您也可以手动修改时间和日期。手动设置后，自动同步功能会被关闭。

1. 在控制中心首页，单击 ![time](./figures/icon124-o.svg)。
2. 单击 **时间设置** 。
   * 开启或关闭自动同步配置。
   * 设置正确的时间和日期。
3. 单击 **确定**。

#### 设置时间日期格式

支持即时设置时间日期的格式。

1. 在控制中心首页，单击 ![time](./figures/icon124-o.svg)。
2. 单击 **格式设置**，可以设置星期、长短日期、长短时间等格式。

### 电源管理

对系统电源进行一些设置，让系统更安全。

![0|power](./figures/57.png)

#### 设置显示器关闭时间

1. 在控制中心首页，单击 ![power\_normal](./figures/icon107-o.svg)。
2. 单击 **使用电源**。
3. 选择关闭显示器的时间。

#### 设置自动锁屏时间

1. 在控制中心首页，单击 ![power\_normal](./figures/icon107-o.svg)。
2. 单击 **使用电源**。
3. 选择自动锁屏的时间。

#### 设置电源按钮

1. 在控制中心首页，单击 ![power\_normal](./figures/icon107-o.svg)。
2. 单击 **使用电源**。
3. 选择电源按钮 **关机**、**关闭显示器** 或 **无任何操作**，更改电源设置。

更改设置后会即时生效，同时系统通知用户已修改电源设置。

### 鼠标

鼠标是计算机的常用输入设备。使用鼠标，可以使操作更加简便快捷。

![0|mouse](./figures/53.png)

#### 通用设置

1. 在控制中心首页，单击 ![mouse\_touchpad\_normal](./figures/icon94-o.svg)。
2. 单击 **通用**。
3. 开启 **左手模式**，调节鼠标和触控板的**滚动速度**，**双击速度**。

> ![notes](./figures/icon99-o.svg)说明：*开启左手模式后，鼠标的左右键功能互换。*

#### 鼠标设置

插入或连接鼠标后，在控制中心进行相关设置，让其更符合您的使用习惯。

1. 在控制中心首页，单击 ![mouse\_touchpad\_normal](./figures/icon94-o.svg)。
2. 单击 **鼠标**。
3. 调节 **指针速度**， 控制鼠标移动时指针移动的速度。
4. 单击 **自然滚动** / **鼠标加速** 开关，开启相应功能。

> ![notes](./figures/icon99-o.svg)说明：
>
> * *开启鼠标加速，提高了指针精确度，鼠标指针在屏幕上的移动距离会根据移动速度的加快而增加。可以根据使用情况开启或关闭。*
> * *自然滚动开启后，鼠标滚轮向下滚动，内容会向下滚动；鼠标滚轮向上滚动，内容会向上滚动。*

### 键盘和语言

在此模块，您可以设置键盘属性，以便符合您的输入习惯，还可以根据国家和语言调整键盘布局，设置系统语言，以及自定义快捷键。

![0|keyboard](./figures/59.png)

#### 键盘属性

1. 在控制中心首页，单击 ![keyboard\_normal](./figures/icon86-o.svg)。
2. 单击 **通用**。
3. 调节 **重复延迟**/**重复速度**。
4. 单击“请在此测试”，按下键盘上的任意字符不松开，查看调节效果。
5. 单击 **启用数字键盘**/**大写锁定提示** 开关，开启相应功能。

#### 键盘布局

设置键盘布局，可以为当前语言自定义键盘。按下键盘上的按键时，键盘布局会控制哪些字符显示在屏幕上。更改键盘布局后，屏幕上的字符可能与键盘按键上的字符不相符。

一般在安装系统时，就已经设置了键盘布局，您也可以添加其他的键盘布局。

![layout](./figures/50.png)

##### 添加键盘布局

1. 在控制中心首页，单击 ![keyboard\_normal](./figures/icon86-o.svg)。
2. 单击 **键盘布局**，进入键盘布局界面。
3. 单击![add](./figures/icon50-o.svg)，单击某一键盘布局即可添加到列表。

##### 删除键盘布局

1. 在控制中心首页，单击 ![keyboard\_normal](./figures/icon86-o.svg)。
2. 单击 **键盘布局**，进入键盘布局界面。
3. 单击”键盘布局“后的 **编辑**。
4. 单击 ![delete](./figures/icon71-o.svg)，删除该键盘布局。

##### 切换键盘布局

1. 在控制中心首页，单击 ![keyboard\_normal](./figures/icon86-o.svg)。
2. 单击 **键盘布局**，进入键盘布局界面。
3. 选择一个键盘布局进行切换。
4. 切换成功后，该键盘布局将标记为已选择。

> ![tips](./figures/icon125-o.svg)窍门：*您也可以选择一组或多组快捷键，按顺序切换已添加的键盘布局。选择 **切换方式**， 让切换后的键盘布局应用于整个系统或当前应用。*

#### 系统语言

系统语言默认为您安装系统时所选择的语言，可以随时更改。

##### 添加系统语言

您可以添加多个语言到系统语言列表，以便切换系统语言。

1. 在控制中心首页，单击 ![keyboard\_normal](./figures/icon86-o.svg)。
2. 单击 **系统语言**，进入系统语言界面。
3. 单击 ![add](./figures/icon50-o.svg) 进入语言列表。
4. 选择语言，该语言将自动添加到系统语言列表。

##### 切换系统语言

1. 在控制中心首页，单击 ![keyboard\_normal](./figures/icon86-o.svg)。
2. 单击 **系统语言**，进入系统语言界面。
3. 选择要切换的语言，系统将自动开始安装语言包。
4. 语言包安装完成后，需要注销后重新登录，以便设置生效。

> ![attention](./figures/icon52-o.svg)注意：*更改系统语言后，键盘布局可能也会发生改变。重新登录时，请确保使用正确的键盘布局来输入密码。*

#### 快捷键

快捷键列表显示了系统所有的快捷键。您可以在这里查看、修改和自定义快捷键。

![0|shortcut](./figures/59.png)

##### 查看快捷键

1. 在控制中心首页，单击 ![keyboard\_normal](./figures/icon86-o.svg)。
2. 单击 **快捷键**，进入快捷键设置界面。
3. 搜索或查看默认的系统快捷键、窗口快捷键和工作区快捷键。

##### 修改快捷键

1. 在控制中心首页，单击 ![keyboard\_normal](./figures/icon86-o.svg)。
2. 单击 **快捷键**，进入快捷键设置界面。
3. 单击需要修改的快捷键。
4. 使用键盘输入新的快捷键。

> ![tips](./figures/icon125-o.svg)窍门：*若要禁用快捷键，请按下键盘上的 ![Backspace](./figures/icon54-o.svg)。若要取消修改快捷键，按下键盘上 **Esc** 键， 或单击下方的”恢复默认”按钮。*

##### 自定义快捷键

您可以为常用的应用自定义一个快捷键。

1. 在控制中心首页，单击 ![keyboard\_normal](./figures/icon86-o.svg)。
2. 单击 **快捷键**。
3. 单击![add](./figures/icon50-o.svg)，进入添加快捷键界面。
4. 输入快捷键名称、命令和快捷键。
5. 单击 **添加**。
6. 添加成功后，单击”自定义快捷键“后的 **编辑**。
7. 单击某个快捷键后 ![delete](./figures/icon71-o.svg)， 删除自定义的快捷键。

> ![tips](./figures/icon125-o.svg)窍门：*若要修改快捷键，单击输入新的快捷键即可。若要修改自定义快捷键的名称和命令，单击“自定义快捷键”后的 **编辑** ，单击快捷键名称后的 ![edit](./figures/icon75-o.svg)，进入修改页面。*

### 系统信息

您可以查看系统版本、版本授权和电脑硬件等信息，以及该系统的一些协议。

![0|info](./figures/48.png)

#### 关于本机

1. 在控制中心首页，单击 ![system\_info\_normal](./figures/icon120-o.svg)。
2. 在 **关于本机** 下，您可以查看当前系统版本、版本授权及电脑硬件信息。
3. 若系统未激活，可在此页面单击 **激活**，进行系统激活。

#### 版本协议

1. 在控制中心首页，单击 ![system\_info\_normal](./figures/icon120-o.svg)。
2. 在 **版本协议** 下，查看系统版本协议。

#### 最终用户许可协议

1. 在控制中心首页，单击 ![system\_info\_normal](./figures/icon120-o.svg)。
2. 在 **最终用户许可协议** 下，查看最终用户许可协议。

## 键盘交互

您可以使用键盘在各个界面区域内切换，并选择对象，执行操作。

| 按键                                                         | 功能                                                         |
| :----------------------------------------------------------- | :----------------------------------------------------------- |
| **Tab**                                                      | 在不同区域或对话框按钮之间切换。                             |
| ![Up](./figures/icon127-o.svg) ![Down](./figures/icon73-o.svg)     ![Left](./figures/icon88-o.svg) ![Right](./figures/icon111-o.svg) | 在同区域内对不同的对象进行选择。使用 ![Right](./figures/icon111-o.svg) 进入下级菜单，使用 ![Left](./figures/icon88-o.svg) 返回上级菜单。使用![Up](./figures/icon127-o.svg)![Down](./figures/icon73-o.svg) 键进行上下切换 。 |
| **Enter**                                                    | 执行选定对象。                                               |
| **Space**                                                    | 在文件管理器中，预览选定对象；在影院和音乐中，开始/暂停播放；在下拉列表中，展开下拉选项（也可使用回车键）。 |
| **Ctrl**+**M**                                               | 打开右键菜单。                                               |

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/eggo_deploying_a_cluster.md
---
# Deploying a Cluster

This section describes how to deploy a Kubernetes cluster.

## Preparing the Environment

The Kubernetes cluster automatic deployment tool provided by openEuler:

* Supports Kubernetes clusters deployment in various common Linux distributions, such as openEuler, CentOS, and Ubuntu.
* Supports hybrid deployment of different CPU architectures (such as AMD64 and ARM64).

### Prerequisites

The following requirements must be met to use the Kubernetes cluster automatic deployment tool:

* You have root permission, for cluster deployment.
* The hostname has been configured for the hosts where Kubernetes is to be deployed. Ensure that the tar command is installed and can be used to decompress the tar.gz packages.
* SSH has been configured on the hosts where Kubernetes is to be deployed for remote access. Ensure that the password-free sudo permission is provided when a common user logs in using SSH.

## Preparing the Installation Packages

For offline installation, prepare dependency packages (such as etcd software packages, container engine software packages, Kubernetes cluster component software packages, network software packages, CoreDNS software packages, and required container images) based on the cluster architecture.

Assume that the network plugin is Calico and the architecture of all hosts in the cluster is ARM64. Prepare the installation packages as follows:

1. Download the required software packages and calico.yaml.

2. Export the container image.

   ```shell
   docker save -o images.tar calico/node:v3.19.1 calico/cni:v3.19.1 calico/kube-controllers:v3.19.1 calico/pod2daemon-flexvol:v3.19.1 k8s.gcr.io/pause:3.2
   ```

3. Store the downloaded installation packages, files, and images in the specified directory accordingly. For details about the storage format, see "Preparing the Environment." For example:

   ```shell
   $ tree package
   package
   ├── bin
   │   ├── bandwidth
   │   ├── bridge
   │   ├── conntrack
   │   ├── containerd
   │   ├── containerd-shim
   │   ├── coredns
   │   ├── ctr
   │   ├── dhcp
   │   ├── docker
   │   ├── dockerd
   │   ├── docker-init
   │   ├── docker-proxy
   │   ├── etcd
   │   ├── etcdctl
   │   ├── firewall
   │   ├── flannel
   │   ├── host-device
   │   ├── host-local
   │   ├── ipvlan
   │   ├── kube-apiserver
   │   ├── kube-controller-manager
   │   ├── kubectl
   │   ├── kubelet
   │   ├── kube-proxy
   │   ├── kube-scheduler
   │   ├── loopback
   │   ├── macvlan
   │   ├── portmap
   │   ├── ptp
   │   ├── runc
   │   ├── sbr
   │   ├── socat
   │   ├── static
   │   ├── tuning
   │   ├── vlan
   │   └── vrf
   ├── file
   │   ├── calico.yaml
   │   └── docker.service
   ├── image
   │   └── images.tar
   └── packages_notes.md
   ```

4. Compile packages\_notes.md and declare the software package sources for users to view.

   ```shell
   1.  etcd
       - etcd,etcdctl
       - Architecture: ARM64
       - Version: 3.5.0
       - Address: https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-arm64.tar.gz

   2. Docker Engine
       - containerd,containerd-shim,ctr,docker,dockerd,docker-init,docker-proxy,runc
       - Architecture: ARM64
       - Version: 19.03.0
       - Address: https://download.docker.com/linux/static/stable/aarch64/docker-19.03.0.tgz

   3. Kubernetes
       - kube-apiserver,kube-controller-manager,kube-scheduler,kubectl,kubelet,kube-proxy
       - Architecture: ARM64
       - Version: 1.21.3
       - Address: https://www.downloadkubernetes.com/

   4. network
       - bandwidth,dhcp,flannel,host-local,loopback,portmap,sbr,tuning,vrf,bridge,firewall,host-device,ipvlan,macvlan,ptp,static,vlan
       - Architecture: ARM64
       - Version: 0.9.1
       - Address: https://github.com/containernetworking/plugins/releases/download/v0.9.1/cni-plugins-linux-arm64-v0.9.1.tgz

   5. CoreDNS
       - coredns
       - Architecture: ARM64
       - Version: 1.8.4
       - Address: https://github.com/coredns/coredns/releases/download/v1.8.4/coredns_1.8.4_linux_arm64.tgz

   6. images.tar
       - calico/node:v3.19.1 calico/cni:v3.19.1 calico/kube-controllers:v3.19.1 calico/pod2daemon-flexvol:v3.19.1 k8s.gcr.io/pause:3.2
       - Architecture: ARM64
       - Version: N/A
       - Address: N/A
    7. calico.yaml
       - Architecture: NA
       - Version: v3.19.1
       - Address: https://docs.projectcalico.org/manifests/calico.yaml
   ```

5. Go to the package directory and pack the downloaded software packages into packages-arm64.tar.gz.

   ```shell
   tar -zcf package-arm64.tar.gz *
   ```

6. Check the compressed package to ensure that the packaging is successful.

   ```shell
   $ tar -tvf package/packages-arm64.tar.gz
   drwxr-xr-x root/root   0 2021-07-29 10:37 bin/
   -rwxr-xr-x root/root   3636214 2021-02-05 23:43 bin/sbr
   -rwxr-xr-x root/root   40108032 2021-07-28 16:40 bin/kube-proxy
   -rwxr-xr-x root/root   4186218 2021-02-05 23:43 bin/vlan
   -rwxr-xr-x root/root   3076118 2021-02-05 23:43 bin/static
   -rwxr-xr-x root/root   3496425 2021-02-05 23:43 bin/host-local
   -rwxr-xr-x root/root   3847814 2021-02-05 23:43 bin/portmap
   -rwxr-xr-x root/root   9681959 2021-02-05 23:43 bin/dhcp
   -rwxr-xr-x root/root   4054640 2021-02-05 23:43 bin/host-device
   -rwxr-xr-x root/root   43909120 2021-07-28 16:41 bin/kube-scheduler
   -rwxr-xr-x root/root   32831616 2019-07-18 02:27 bin/containerd
   -rwxr-xr-x root/root   3284795 2021-02-05 23:43 bin/flannel
   -rwxr-xr-x root/root   21757952 2021-06-16 05:52 bin/etcd
   -rwxr-xr-x root/root   546520 2019-07-18 02:27 bin/docker-init
   -rwxr-xr-x root/root   5878304 2019-07-18 02:27 bin/containerd-shim
   -rwxr-xr-x root/root   4191734 2021-02-05 23:43 bin/macvlan
   -rwxr-xr-x root/root   55248437 2019-07-18 02:27 bin/docker
   -rwxr-xr-x root/root   376208 2019-10-27 01:42 bin/socat
   -rwxr-xr-x root/root   4053707 2021-02-05 23:43 bin/bandwidth
   -rwxr-xr-x root/root   4328311 2021-02-05 23:43 bin/ptp
   -rwxr-xr-x root/root   3633613 2021-02-05 23:43 bin/vrf
   -rwxr-xr-x root/root   3432839 2021-02-05 23:43 bin/loopback
   -rwxr-xr-x root/root   109617672 2021-07-28 16:42 bin/kubelet
   -rwxr-xr-x root/root   113442816 2021-07-28 16:42 bin/kube-apiserver
   -rwxr-xr-x root/root   44171264 2021-05-28 18:33 bin/coredns
   -rwxr-xr-x root/root   43122688 2021-07-28 16:41 bin/kubectl
   -rwxr-xr-x root/root   16711680 2021-06-16 05:52 bin/etcdctl
   -rwxr-xr-x root/root   3570597 2021-02-05 23:43 bin/tuning
   -rwxr-xr-x root/root   4397098 2021-02-05 23:43 bin/bridge
   -rwxr-xr-x root/root   4612178 2021-02-05 23:43 bin/firewall
   -rwxr-xr-x root/root   68921120 2019-07-18 02:27 bin/dockerd
   -rwxr-xr-x root/root   2898746 2019-07-18 02:27 bin/docker-proxy
   -rwxr-xr-x root/root   4186585 2021-02-05 23:43 bin/ipvlan
   -rwxr-xr-x root/root   18446016 2019-07-18 02:27 bin/ctr
   -rwxr-xr-x root/root   80752 2019-01-27 19:40 bin/conntrack
   -rwxr-xr-x root/root   8037728 2019-07-18 02:27 bin/runc
   drwxr-xr-x root/root   0 2021-07-29 10:39 file/
   -rw-r--r-- root/root   20713 2021-07-29 10:39 file/calico.yaml
   -rw-r--r-- root/root   1004 2021-07-29 10:39 file/docker.service
   drwxr-xr-x root/root   0 2021-07-29 11:02 image/
   -rw-r--r-- root/root   264783872 2021-07-29 11:02 image/images.tar
   -rw-r--r-- root/root   1298 2021-07-29 11:05 packages_notes.md
   ```

## Preparing the Configuration File

Prepare the YAML configuration file used for deployment. You can run the following command to generate a configuration template and modify the generated template.yaml based on deployment requirements:

```shell
eggo template -f template.yaml
```

You can also directly modify the default configurations using command lines. For example:

```shell
eggo template -f template.yaml -n k8s-cluster -u username -p password --masters 192.168.0.1 --masters 192.168.0.2 --workers 192.168.0.3 --etcds 192.168.0.4 --loadbalancer 192.168.0.5
```

## Installing the Kubernetes Cluster

Install the Kubernetes cluster. In this example, template.yaml is the specified configuration file for deployment.

```shell
eggo -d deploy -f template.yaml
```

After the installation is complete, verify whether each node in the cluster is successfully installed based on the command output.

```shell
\-------------------------------
message: create cluster success
summary:
192.168.0.1  success
192.168.0.2  success
192.168.0.3  success
\-------------------------------
To start using cluster: cluster-example, you need following as a regular user:

​ export KUBECONFIG=/etc/eggo/cluster-example/admin.conf
```

## Adding Nodes

If the nodes in the cluster cannot meet service requirements, you can add nodes to the cluster to expand the capacity.

* Using the command line to add a single node. The following is an example:

  ```shell
  eggo -d join --id k8s-cluster --type master,worker --arch arm64 --port 22 192.168.0.5
  ```

* Using the configuration file to add multiple nodes:

  ```shell
  eggo -d join --id k8s-cluster --file join.yaml
  ```

  Configure the nodes to be added in join.yaml. The following is an example:

  ```yaml
  masters:                          # Configure the master node list. It is recommended that each master node is also set as a worker node. Otherwise, the master nodes may fail to directly access the pods.
  - name: test0                     # Name of the node, which is the node name displayed to the Kubernetes cluster.
    ip: 192.168.0.2                 #IP address of the node.
    port: 22                        # Port number for SSH login.
    arch: arm64                     # Architecture. Set this parameter to amd64 for x86_64.
  - name: test1
    ip: 192.168.0.3
    port: 22
    arch: arm64
  workers:                          # Configure the worker node list.
  - name: test0                     # Name of the node, which is the node name displayed to the Kubernetes cluster.
    ip: 192.168.0.4                 #IP address of the node.
    port: 22                        # Port number for SSH login.
    arch: arm64                     # Architecture. Set this parameter to amd64 for x86_64.
  - name: test2
    ip: 192.168.0.5
    port: 22
    arch: arm64
  ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/deploying_a_kubernetes_cluster_manually.md
---
# Deploying a Kubernetes Cluster Manually

**Note: Manual deployment applies only to experimental and learning environments and is not intended for commercial environments.**

This chapter describes how to deploy a Kubernetes cluster.

## Environment

Deploy VMs based on the VM installation section and obtain the following VM list:

| HostName   | MAC               | IPv4               |
| ---------- | ----------------- | -------------------|
| k8smaster0 | 52:54:00:00:00:80 | 192.168.122.154/24 |
| k8smaster1 | 52:54:00:00:00:81 | 192.168.122.155/24 |
| k8smaster2 | 52:54:00:00:00:82 | 192.168.122.156/24 |
| k8snode1   | 52:54:00:00:00:83 | 192.168.122.157/24 |
| k8snode2   | 52:54:00:00:00:84 | 192.168.122.158/24 |
| k8snode3   | 52:54:00:00:00:85 | 192.168.122.159/24 |

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/deploying_a_node_component.md
---
# Deploying a Node Component

This section uses the `k8snode1` node as an example.

## Environment Preparation

```bash
# A proxy needs to be configured for the intranet.
$ dnf install -y docker iSulad conntrack-tools socat containernetworking-plugins
$ swapoff -a
$ mkdir -p /etc/kubernetes/pki/
$ mkdir -p /etc/cni/net.d
$ mkdir -p /opt/cni
# Delete the default kubeconfig file.
$ rm /etc/kubernetes/kubelet.kubeconfig

## Use iSulad as the runtime ########.
# Configure the iSulad.
cat  /etc/isulad/daemon.json
{
        "registry-mirrors": [
                "docker.io"
        ],
        "insecure-registries": [
                "k8s.gcr.io",
                "quay.io"
        ],
        "pod-sandbox-image": "k8s.gcr.io/pause:3.2",# pause type
        "network-plugin": "cni", # If this parameter is left blank, the CNI network plug-in is disabled. In this case, the following two paths become invalid. After the plug-in is installed, restart iSulad.
        "cni-bin-dir": "/usr/libexec/cni/",
        "cni-conf-dir": "/etc/cni/net.d",
}

# Add the proxy to the iSulad environment variable and download the image.
cat /usr/lib/systemd/system/isulad.service
[Service]
Type=notify
Environment="HTTP_PROXY=http://name:password@proxy:8080"
Environment="HTTPS_PROXY=http://name:password@proxy:8080"

# Restart the iSulad and set it to start automatically upon power-on.
systemctl daemon-reload
systemctl restart isulad

## If Docker is used as the runtime, run the following command: ########
$ dnf install -y docker
# If a proxy environment is required, configure a proxy for Docker, add the configuration file http-proxy.conf, and edit the following content. Replace name, password, and proxy-addr with the actual values.
$ cat /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://name:password@proxy-addr:8080"
$ systemctl daemon-reload
$ systemctl restart docker
```

## Creating kubeconfig Configuration Files

Perform the following operations on each node to create a configuration file:

```bash
$ kubectl config set-cluster openeuler-k8s \
    --certificate-authority=/etc/kubernetes/pki/ca.pem \
    --embed-certs=true \
    --server=https://192.168.122.154:6443 \
    --kubeconfig=k8snode1.kubeconfig

$ kubectl config set-credentials system:node:k8snode1 \
    --client-certificate=/etc/kubernetes/pki/k8snode1.pem \
    --client-key=/etc/kubernetes/pki/k8snode1-key.pem \
    --embed-certs=true \
    --kubeconfig=k8snode1.kubeconfig

$ kubectl config set-context default \
    --cluster=openeuler-k8s \
    --user=system:node:k8snode1 \
    --kubeconfig=k8snode1.kubeconfig

$ kubectl config use-context default --kubeconfig=k8snode1.kubeconfig
```

**Note: Change k8snode1 to the corresponding node name.**

## Copying the Certificate

Similar to the control plane, all certificates, keys, and related configurations are stored in the `/etc/kubernetes/pki/` directory.

```bash
$ ls /etc/kubernetes/pki/
ca.pem                 k8snode1.kubeconfig  kubelet_config.yaml     kube-proxy-key.pem     kube-proxy.pem
k8snode1-key.pem  k8snode1.pem         kube_proxy_config.yaml  kube-proxy.kubeconfig
```

## CNI Network Configuration

containernetworking-plugins is used as the CNI plug-in used by kubelet. In the future, plug-ins such as calico and flannel can be introduced to enhance the network capability of the cluster.

```bash
# Bridge Network Configuration
$ cat /etc/cni/net.d/10-bridge.conf
{
  "cniVersion": "0.3.1",
  "name": "bridge",
  "type": "bridge",
  "bridge": "cnio0",
  "isGateway": true,
  "ipMasq": true,
  "ipam": {
    "type": "host-local",
    "subnet": "10.244.0.0/16",
    "gateway": "10.244.0.1"
  },
  "dns": {
    "nameservers": [
      "10.244.0.1"
    ]
  }
}

# Loopback Network Configuration
$ cat /etc/cni/net.d/99-loopback.conf
{
    "cniVersion": "0.3.1",
    "name": "lo",
    "type": "loopback"
}
```

## Deploying the kubelet Service

### Configuration File on Which Kubelet Depends

```bash
$ cat /etc/kubernetes/pki/kubelet_config.yaml
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.pem
authorization:
  mode: Webhook
clusterDNS:
- 10.32.0.10
clusterDomain: cluster.local
runtimeRequestTimeout: "15m"
tlsCertFile: "/etc/kubernetes/pki/k8snode1.pem"
tlsPrivateKeyFile: "/etc/kubernetes/pki/k8snode1-key.pem"
```

**Note: The IP address of the cluster DNS is 10.32.0.10, which must be the same as the value of service-cluster-ip-range.**

### Compiling the systemd Configuration File

```bash
$ cat /usr/lib/systemd/system/kubelet.service
[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=https://kubernetes.io/docs/
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/bin/kubelet \
 --config=/etc/kubernetes/pki/kubelet_config.yaml \
 --network-plugin=cni \
 --pod-infra-container-image=k8s.gcr.io/pause:3.2 \
 --kubeconfig=/etc/kubernetes/pki/k8snode1.kubeconfig \
 --register-node=true \
 --hostname-override=k8snode1 \
 --cni-bin-dir="/usr/libexec/cni/" \
 --v=2

Restart=always
StartLimitInterval=0
RestartSec=10

[Install]
WantedBy=multi-user.target
```

**Note: If iSulad is used as the runtime, add the following configuration:**

```bash
--container-runtime=remote \
--container-runtime-endpoint=unix:///var/run/isulad.sock \
```

## Deploying kube-proxy

### Configuration File on Which kube-proxy Depends

```bash
cat /etc/kubernetes/pki/kube_proxy_config.yaml
kind: KubeProxyConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
clientConnection:
  kubeconfig: /etc/kubernetes/pki/kube-proxy.kubeconfig
clusterCIDR: 10.244.0.0/16
mode: "iptables"
```

### Compiling the systemd Configuration File

```bash
$ cat /usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://kubernetes.io/docs/reference/generated/kube-proxy/
After=network.target

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/proxy
ExecStart=/usr/bin/kube-proxy \
     $KUBE_LOGTOSTDERR \
     $KUBE_LOG_LEVEL \
     --config=/etc/kubernetes/pki/kube_proxy_config.yaml \
     --hostname-override=k8snode1 \
     $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
```

## Starting a Component Service

```bash
systemctl enable kubelet kube-proxy
systemctl start kubelet kube-proxy
```

Deploy other nodes in sequence.

## Verifying the Cluster Status

Wait for several minutes and run the following command to check the node status:

```bash
$ kubectl get nodes --kubeconfig /etc/kubernetes/pki/admin.kubeconfig
NAME            STATUS   ROLES    AGE   VERSION
k8snode1   Ready    <none>   17h   v1.20.2
k8snode2   Ready    <none>   19m   v1.20.2
k8snode3   Ready    <none>   12m   v1.20.2
```

## Deploying coredns

coredns can be deployed on a node or master node. In this document, coredns is deployed on the `k8snode1` node.

### Compiling the coredns Configuration File

```bash
$ cat /etc/kubernetes/pki/dns/Corefile
.:53 {
    errors
    health {
      lameduck 5s
    }
    ready
    kubernetes cluster.local in-addr.arpa ip6.arpa {
      pods insecure
      endpoint https://192.168.122.154:6443
      tls /etc/kubernetes/pki/ca.pem /etc/kubernetes/pki/admin-key.pem /etc/kubernetes/pki/admin.pem
      kubeconfig /etc/kubernetes/pki/admin.kubeconfig default
      fallthrough in-addr.arpa ip6.arpa
    }
    prometheus :9153
    forward . /etc/resolv.conf {
      max_concurrent 1000
    }
    cache 30
    loop
    reload
    loadbalance
}
```

Note:

* Listen to port 53.
* Configure the Kubernetes plug-in, including the certificate and the URL of kube api.

### Preparing the service File of systemd

```bash
cat /usr/lib/systemd/system/coredns.service
[Unit]
Description=Kubernetes Core DNS server
Documentation=https://github.com/coredns/coredns
After=network.target

[Service]
ExecStart=bash -c "KUBE_DNS_SERVICE_HOST=10.32.0.10 coredns -conf /etc/kubernetes/pki/dns/Corefile"

Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
```

### Starting the Service

```bash
systemctl enable coredns
systemctl start coredns
```

### Creating the Service Object of coredns

```bash
$ cat coredns_server.yaml
apiVersion: v1
kind: Service
metadata:
  name: kube-dns
  namespace: kube-system
  annotations:
    prometheus.io/port: "9153"
    prometheus.io/scrape: "true"
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: "CoreDNS"
spec:
  clusterIP: 10.32.0.10
  ports:
  - name: dns
    port: 53
    protocol: UDP
  - name: dns-tcp
    port: 53
    protocol: TCP
  - name: metrics
    port: 9153
    protocol: TCP
```

### Creating the Endpoint Object of coredns

```bash
$ cat coredns_ep.yaml
apiVersion: v1
kind: Endpoints
metadata:
  name: kube-dns
  namespace: kube-system
subsets:
  - addresses:
      - ip: 192.168.122.157
    ports:
      - name: dns-tcp
        port: 53
        protocol: TCP
      - name: dns
        port: 53
        protocol: UDP
      - name: metrics
        port: 9153
        protocol: TCP
```

### Confirming the coredns Service

```bash
# View the service object.
$ kubectl get service -n kube-system kube-dns
NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
kube-dns   ClusterIP   10.32.0.10   <none>        53/UDP,53/TCP,9153/TCP   51m
# View the endpoint object.
$ kubectl get endpoints -n kube-system kube-dns
NAME       ENDPOINTS                                                    AGE
kube-dns   192.168.122.157:53,192.168.122.157:53,192.168.122.157:9153   52m
```

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/aops/deploying_aops.md
---
# Deploying A-Ops

## 1 Introduction to A-Ops

A-Ops is a service used to improve the overall security of hosts. It provides functions such as asset management, vulnerability management, and configuration source tracing to identify and manage information assets, monitor software vulnerabilities, and rectify system faults on hosts, ensuring stable and secure running of hosts.

The following table describes the modules related to the A-Ops service.

| Module              | Description                                                                                                                                                                                                                                                                                                                                                       |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| aops-ceres          | Client of the A-Ops service.Collects host data and manages other data collectors (such as gala-gopher).Responds to the commands delivered by the management center and processes the requirements and operations of the management center.                                                                                                                |
| aops-zeus           | A-Ops basic application management center, which interacts with other modules. The default port number is 11111.Provides the basic host management service externally, such as adding and deleting hosts and host groups.                                                                                                                                                                    |
| aops-hermes         | Provides a visualized operation interface for A-Ops to display data information to users, improving service usability.                                                                                                                                                                                                                                                                         |
| aops-apollo         | Vulnerability management module of A-Ops. The default port number is 11116.Identifies clients, and periodically obtains security notices released by the openEuler community and updates them to the vulnerability database.Detects vulnerabilities in the system and software by comparing the vulnerabilities with those in the vulnerability database. |
| aops-vulcanus       | Basic tool library of A-Ops. **Except aops-ceres and aops-hermes, modules must be installed and used together with this module.**                                                                                                                                                                                                                   |
| aops-tools          | Provides the basic environment deployment script and database table initialization. The script is available in the **/opt/aops/scripts** directory after A-Ops is installed.                                                                                                                                                                                                                                     |
| gala-ragdoll        | Configuration source tracing module of A-Ops.Uses Git to monitor and record configuration file changes. The default port number is 11114.                                                                                                                                                                                                                     |
| dnf-hotpatch-plugin | DNF plug-in, which allows DNF to recognize hot patch information and provides hot patch scanning and application.                                                                                                                                                                                                                                                 |

## 2 Environment Requirements

You are advised to use four hosts running openEuler 24.03 LTS for deployment. Use three as the server and one as the managed host managed by A-Ops. **Configure the update repository** ([Q6: update Repository Configuration](#q6-update-repository-configuration)). The deployment scheme is as follows:

* Host A: For MySQL, Redis, and Elasticsearch deployment. It provides data service support. The recommended memory is more than 8 GB.
* Host B: For the A-Ops asset management service (zeus),  frontend display, and complete service function support. The recommended memory is more than 6 GB.
* Host C: For the A-Ops configuration source tracing service (gala-ragdoll) and vulnerability management. The recommended memory is 4 GB or more.
* Host D: As an A-Ops client and is used as a host managed and monitored by A-Ops. (aops-ceres can be deployed on hosts that need to be managed.)

| Host   | IP Address  | Module                                |
| ------ | ----------- | ------------------------------------- |
| Host A | 192.168.1.1 | MySQL, Elasticsearch, Redis           |
| Host B | 192.168.1.2 | aops-zeus, aops-hermes, aops-diana    |
| Host C | 192.168.1.3 | aops-apollo, gala-ragdoll, aops-diana |
| Host D | 192.168.1.4 | aops-ceres, dnf-hotpatch-plugin       |

> Before deployment, disable the **firewall and SELinux** on each host.

* Disable the firewall.

```shell
systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld
setenforce 0

```

* Disable SELinux.

```shell
# Change the status of SELinux to disabled in /etc/selinux/config.

vi /etc/selinux/config
SELINUX=disabled

# After changing the value, press ESC and enter :wq to save the modification.
```

Note: SELinux will be disabled after a reboot.

## 3. Server Deployment

### 3.1 Asset Management

To use the asset management function, you need to deploy the aops-zeus, aops-hermes, MySQL, and Redis services.

#### 3.1.1 Node Information

| Host   | IP Address  | Module                                |
| ------ | ----------- | ------------------------------------- |
| Host A | 192.168.1.1 | MySQL, Redis  |
| Host B | 192.168.1.2 | aops-zeus, aops-hermes              |

#### 3.1.2 Deployment Procedure

##### 3.1.2.1 Deploying MySQL

* Install MySQL.

```shell
yum install mysql-server
```

* Modify the MySQL configuration file.

```bash
vim /etc/my.cnf
```

* Add **bind-address** and set it to the IP address of the local host in the **mysqld** section.

```ini
[mysqld]
bind-address=192.168.1.1
```

* Restart the MySQL service.

```bash
systemctl restart mysqld
```

* Set the MySQL database access permission for the **root** user.

```mysql
$ mysql

mysql> show databases;
mysql> use mysql;
mysql> select user,host from user; -- If the value of host is localhost, only the local host can connect to the MySQL database. The external network and local software client cannot connect to the MySQL database.

+---------------+-----------+
| user          | host      |
+---------------+-----------+
| root          | localhost |
| mysql.session | localhost |
| mysql.sys     | localhost |
+---------------+-----------+
3 rows in set (0.00 sec)
```

```mysql
mysql> update user set host = '%' where user='root'; -- Allow the access of the root user using any IP address.
mysql> flush privileges; -- Refresh the permissions.
mysql> exit
```

##### 3.1.2.2 Deploying Redis

* Install Redis.

```shell
yum install redis -y
```

* Modify the Redis configuration file.

```shell
vim /etc/redis.conf
```

Bind IP addresses.

```ini
# It is possible to listen to just one or multiple selected interfaces using
# the "bind" configuration directive, followed by one or more IP addresses.
#
# Examples:
#
# bind 192.168.1.100 10.0.0.1
# bind 127.0.0.1 ::1
#
# ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the
# internet, binding to all the interfaces is dangerous and will expose the
# instance to everybody on the internet. So by default we uncomment the
# following bind directive, that will force Redis to listen only into
# the IPv4 lookback interface address (this means Redis will be able to
# accept connections only from clients running into the same computer it
# is running).
#
# IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES
# JUST COMMENT THE FOLLOWING LINE.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bind 127.0.0.1 192.168.1.1# Add the actual IP address of host A.
```

* Start the Redis service.

```shell
systemctl start redis
```

##### 3.1.2.3 Deploying Prometheus

* Install Prometheus.

```shell
yum install prometheus2 -y
```

* Modify the Prometheus configuration file.

```shell
vim /etc/prometheus/prometheus.yml
```

* Add the gala-gopher IP addresses of the managed client to the monitored targets of Prometheus.

> In this document, host D is the client. Add the gala-gopher address of host D.

* Modify the **targets** configuration item.

```yaml
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090', '192.168.1.4:8888']
```

Start the Prometheus service.

```shell
systemctl start prometheus
```

##### 3.1.2.4 Deploying aops-zeus

* Install aops-zeus.

```shell
yum install aops-zeus -y
```

* Modify the configuration file.

```shell
vim /etc/aops/zeus.ini
```

* Change the IP address of each service in the configuration file to the actual IP address. In this document, aops-zeus is deployed on host B. Therefore, you need to set the IP address to the IP address of host B.

```ini
[zeus]
ip=192.168.1.2  // Change the IP address to the actual IP address of host B.
port=11111

[uwsgi]
wsgi-file=manage.py
daemonize=/var/log/aops/uwsgi/zeus.log
http-timeout=600
harakiri=600
processes=2     // Generate a specified number of workers or processes.
gevent=100      // Number of gevent asynchronous cores

[mysql]
ip=192.168.1.1  // Change the IP address to the actual IP address of host A.
port=3306
database_name=aops
engine_format=mysql+pymysql://@%s:%s/%s
pool_size=100
pool_recycle=7200

[agent]
default_instance_port=8888

[redis]
ip=192.168.1.1  // Change the IP address to the actual IP address of host A.
port=6379

[apollo]
ip=192.168.1.3  // Change the IP address to the actual IP address of the apollo service deployment. It is recommended that apollo and zeus be deployed separately. This section is not required if apollo is not used.
port=11116
```

> **Set the MySQL database mode to password mode**. For details, see [Q5: MySQL Password Mode](#q5-mysql-password-mode)

* Start the aops-zeus service.

```shell
systemctl start aops-zeus
```

**Note: [Initialize the aops-zeus database](#3125-initializing-the-aops-zeus-database) before starting the service.**

> If the zeus service fails to be started and the error message indicates that the MySQL database cannot be connected, check if a MySQL password is set. If yes, see [Q5: MySQL Password Mode](#q5-mysql-password-mode).

#### 3.1.2.5 Initializing the aops-zeus Database

* Initialize the database.

```shell
cd /opt/aops/scripts/deploy
bash aops-basedatabase.sh init zeus
```

**Note: If aops-tools is not installed, run the SQL script to initialize. The script path is /opt/aops/database/zeus.sql**

[Q5: MySQL Password Mode](#q5-mysql-password-mode)

[Q7: Nonexisting /opt/aops/scripts/deploy](#q7-nonexisting-optaopsscriptsdeploy)

##### 3.1.2.6 Deploying aops-hermes

* Install aops-hermes.

```shell
yum install aops-hermes -y
```

* Modify the configuration file.

```shell
vim /etc/nginx/aops-nginx.conf
```

* Some service configurations:

  > As the services are deployed on host B, configure the Nginx proxy to set the services addresses to the actual IP address of host B.

```ini
 # Ensure that Nginx still uses index.html as the entry when the front-end route changes.
  location / {
      try_files $uri $uri/ /index.html;
      if (!-e $request_filename){
          rewrite ^(.*)$ /index.html last;
      }
  }
    # Change it to the actual IP address of the host where aops-zeus is deployed.
  location /api/ {
      proxy_pass http://192.168.1.2:11111/;
  }
  # Enter the IP address of gala-ragdoll. IP addresses that involve port 11114 need to be configured.
  location /api/domain {
      proxy_pass http://192.168.1.3:11114/;
      rewrite ^/api/(.*) /$1 break;
  }
    # Enter the IP address of gala-apollo.
  location /api/vulnerability {
      proxy_pass http://192.168.1.3:11116/;
      rewrite ^/api/(.*) /$1 break;
  }
```

* Enable the aops-hermes service.

```shell
systemctl start aops-hermes
```

### 3.2 Vulnerability Management

The CVE management module is implemented based on the [asset management](#31-asset-management) module. Therefore, you need to [deploy the module](#312-deployment-procedure) before deploying aops-apollo.

The running of the aops-apollo service requires the support of the **MySQL, Elasticsearch, and Redis** databases.

#### 3.2.1 Node Information

| Host   | IP Address  | Module        |
| ------ | ----------- | ------------- |
| Host A | 192.168.1.1 | Elasticsearch |
| Host C | 192.168.1.3 | aops-apollo   |

#### 3.2.2 Deployment Procedure

See [Asset Management](#312-deployment-procedure).

##### 3.2.2.1 Deploying Elasticsearch

* Configure the repository for Elasticsearch.

```shell
echo "[aops_elasticsearch]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md" > "/etc/yum.repos.d/aops_elasticsearch.repo"
```

* Install Elasticsearch.

```shell
yum install elasticsearch-7.14.0-1 -y
```

* Modify the Elasticsearch configuration file.

```shell
vim /etc/elasticsearch/elasticsearch.yml
```

```yml
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
```

```yml
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
# Change the value to the actual IP address of host A.
network.host: 192.168.1.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
```

```yml
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1"]
# Cross-domain configurations
http.cors.enabled: true
http.cors.allow-origin: "*"
#
```

* Restart the Elasticsearch service.

```shell
systemctl restart elasticsearch
```

##### 3.2.2.2 Deploying aops-apollo

* Install aops-apollo.

```shell
yum install aops-apollo
```

* Modify the configuration file.

```shell
vim /etc/aops/apollo.ini
```

* Change the IP address of each service in the **apollo.ini** to the actual IP address.

```ini
[apollo]
ip=192.168.1.3// Change it to the actual IP address of host C.
port=11116
host_vault_dir=/opt/aops
host_vars=/opt/aops/host_vars

[zeus]
ip=192.168.1.2 // Change it to the actual IP address of host B.
port=11111

# hermes info is used to send mail.
[hermes]
ip=192.168.1.2  // Change it to the actual IP address of aops-hermes, for example, the IP address of host B.
port=80         // Change it to the actual port of the hermes service.

[cve]
cve_fix_function=yum
# value between 0-23, for example, 2 means 2:00 in a day.
cve_scan_time=2

[mysql]
ip=192.168.1.1 // Change it to the actual IP address of host A.
port=3306
database_name=aops
engine_format=mysql+pymysql://@%s:%s/%s
pool_size=100
pool_recycle=7200

[elasticsearch]
ip=192.168.1.1 // Change it to the actual IP address of host A.
port=9200
max_es_query_num=10000000

[redis]
ip=192.168.1.1 // Change it to the actual IP address of host A.
port=6379

[uwsgi]
wsgi-file=manage.py
daemonize=/var/log/aops/uwsgi/apollo.log
http-timeout=600
harakiri=600
processes=2
gevent=100
```

> **Set the MySQL database to the password mode**. For details, see [Q5: MySQL Password Mode](#q5-mysql-password-mode).

* Start the aops-apollo service.

```shell
systemctl start aops-apollo
```

**Note: [Initialize the aops-apollo database](#3223-initializing-the-aops-apollo-database) before starting the service.**

> If the apollo service fails to be started and the error message indicates that the MySQL database cannot be connected, check if a MySQL password is set. If yes, see [Q5: MySQL Password Mode](#q5-mysql-password-mode).

#### 3.2.2.3 Initializing the aops-apollo Database

* Initialize the apollo database.

```shell
cd /opt/aops/scripts/deploy
bash aops-basedatabase.sh init apollo
```

**Note: If aops-tools is not installed, run the SQL script to initialize. The script path is /opt/aops/database/apollo.sql**

[Q5: MySQL Password Mode](#q5-mysql-password-mode)

[FAQs: Nonexisting /opt/aops/scripts/deploy](#q7-nonexisting-optaopsscriptsdeploy)

### 3.3 Configuring Source Tracing

A-Ops configuration source tracing depends on gala-ragdoll. Therefore, you need to complete the deployment of [Asset Management](#31-asset-management) and then deploy gala-ragdoll.

#### 3.3.1 Node Information

| Host   | IP Address  | Module       |
| ------ | ----------- | ------------ |
| Host C | 192.168.1.3 | aops-ragdoll |

#### 3.3.2 Deployment Procedure

See [Asset Management](#31-asset-management).

##### 3.3.2.1 Deploying gala-ragdoll

* Install gala-ragdoll.

```shell
yum install gala-ragdoll python3-gala-ragdoll -y
```

* Modify the configuration file.

```shell
vim /etc/ragdoll/gala-ragdoll.conf
```

> **Change the IP address in collect\_address of the collect section to the IP address of host B, and change the values of collect\_api and collect\_port to the actual API and port number.**

```ini
[git]
git_dir = "/home/confTraceTest"
user_name = "user_name"
user_email = "user_email"

[collect]
collect_address = "http://192.168.1.2"    // Change it to the actual IP address of host B.
collect_api = "/manage/config/collect"    // The value is an example. Change it to the actual value.
collect_port = 11111                      // Change it to the actual port number of the aops-zeus service.

[sync]
sync_address = "http://192.168.1.2"
sync_api = "/manage/config/sync"          // The value is an example. Change it to the actual value.
sync_port = 11111

[objectFile]
object_file_address = "http://192.168.1.2"
object_file_api = "/manage/config/objectfile"   // The value is an example. Change it to the actual value.
object_file_port = 11111

[ragdoll]
port = 11114
```

* Start the gala-ragdoll service.

```shell
systemctl start gala-ragdoll
```

## 3.4 Exception Detection

The exception detection function is implemented based on the aops-zeus service. Therefore, you need to deploy aops-zeus and then aops-diana.

Considering distributed deployment, the aops-diana service must be deployed on both host B and host C to act as the producer and consumer in the message queue, respectively.

The running of the aops-diana service requires the support of MySQL, Elasticsearch, Kafka, and Prometheus.

### 3.4.1 Node Information

| Host   | IP Address  | Module     |
| ------ | ----------- | ---------- |
| Host A | 192.168.1.1 | Kafka      |
| Host B | 192.168.1.2 | aops-diana |
| Host C | 192.168.1.3 | aops-diana |

### 3.4.2 Deployment Procedure

[Asset Management](#312-deployment-procedure)

[Deploying Elasticsearch](#3221-deploying-elasticsearch)

#### 3.4.2.1 Deploying Kafka

Kafka uses ZooKeeper to manage and coordinate agents. Therefore, you need to deploy ZooKeeper when deploying Kafka.

* Install ZooKeeper.

```shell
yum install zookeeper -y
```

* Start the ZooKeeper service.

```shell
systemctl start zookeeper
```

* Install Kafka.

```shell
yum install kafka -y
```

* Modify the Kafka configuration file.

```shell
vim /opt/kafka/config/server.properties
```

Change the value of **listeners** to the IP address of the local host.

```yaml
############################# Socket Server Settings #############################

# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://192.168.1.1:9092
```

* Start the Kafka service.

```shell
cd /opt/kafka/bin
nohup ./kafka-server-start.sh ../config/server.properties &

# Check all the outputs of nohup. If the IP address of host A and the Kafka startup success INFO are displayed, Kafka is started successfully.
tail -f ./nohup.out
```

#### 3.4.2.2 Deploying diana

* Install aops-diana.

```shell
yum install aops-diana
```

Modify the configuration file.

> The aops-dianas on host B and host C play different roles, which are **distinguished based on the differences in the configuration file**.

```shell
vim /etc/aops/diana.ini
```

(1) Start aops-diana on host C in executor mode. It functions as the consumer in the Kafka message queue. The configuration file to be modified is as follows:

```ini
[diana]
ip=192.168.1.3  // Change the IP address to the actual IP address of host C.
port=11112
mode=executor  // This mode is the executor mode. It is used as the executor in common diagnosis mode and functions as the consumer in Kafka.
timing_check=on

[default_mode]
period=60
step=60

[elasticsearch]
ip=192.168.1.1  // Change the IP address to the actual IP address of host A.
port=9200
max_es_query_num=10000000

[mysql]
ip=192.168.1.1  // Change the IP address to the actual IP address of host A.
port=3306
database_name=aops
engine_format=mysql+pymysql://@%s:%s/%s
pool_size=10000
pool_recycle=7200

[redis]
ip=192.168.1.1  // Change the IP address to the actual IP address of host A.
port=6379

[prometheus]
ip=192.168.1.1  // Change the IP address to the actual IP address of host A.
port=9090
query_range_step=15s

[agent]
default_instance_port=8888

[zeus]
ip=192.168.1.2  // Change the IP address to the actual IP address of host B.
port=11111

[consumer]
kafka_server_list=192.168.1.1:9092  // Change the IP address to the actual IP address of host C.
enable_auto_commit=False
auto_offset_reset=earliest
timeout_ms=5
max_records=3
task_name=CHECK_TASK
task_group_id=CHECK_TASK_GROUP_ID
result_name=CHECK_RESULT

[producer]
kafka_server_list = 192.168.1.1:9092  // Change the IP address to the actual IP address of host C.
api_version = 0.11.5
acks = 1
retries = 3
retry_backoff_ms = 100
task_name=CHECK_TASK
task_group_id=CHECK_TASK_GROUP_ID

[uwsgi]
wsgi-file=manage.py
daemonize=/var/log/aops/uwsgi/diana.log
http-timeout=600
harakiri=600
processes=2
threads=2
```

> **Set the MySQL database to the password mode**. For details, see [Q5: MySQL Password Mode](#q5-mysql-password-mode).

(2) Start aops-diana on host B in configurable mode. It functions as the producer in the Kafka message queue. The aops-diana port configuration in the aops-hermes file is subject to the IP address and port number of this host. The configuration file to be modified is as follows:

```ini
[diana]
ip=192.168.1.2  // Change the IP address to the actual IP address of host B.
port=11112
mode=configurable  // This mode is the configurable mode. It is used as a scheduler in common diagnosis mode and functions as the producer.
timing_check=on

[default_mode]
period=60
step=60

[elasticsearch]
ip=192.168.1.1  // Change the IP address to the actual IP address of host A.
port=9200
max_es_query_num=10000000

[mysql]
ip=192.168.1.1  // Change the IP address to the actual IP address of host A.
port=3306
database_name=aops
engine_format=mysql+pymysql://@%s:%s/%s
pool_size=100
pool_recycle=7200

[redis]
ip=192.168.1.1  // Change the IP address to the actual IP address of host A.
port=6379

[prometheus]
ip=192.168.1.1  // Change the IP address to the actual IP address of host A.
port=9090
query_range_step=15s

[agent]
default_instance_port=8888

[zeus]
ip=192.168.1.2  // Change the IP address to the actual IP address of host B.
port=11111

[consumer]
kafka_server_list=192.168.1.1:9092  // Change the IP address to the actual IP address of host A.
enable_auto_commit=False
auto_offset_reset=earliest
timeout_ms=5
max_records=3
task_name=CHECK_TASK
task_group_id=CHECK_TASK_GROUP_ID
result_name=CHECK_RESULT

[producer]
kafka_server_list = 192.168.1.1:9092  // Change the IP address to the actual IP address of host A.
api_version = 0.11.5
acks = 1
retries = 3
retry_backoff_ms = 100
task_name=CHECK_TASK
task_group_id=CHECK_TASK_GROUP_ID

[uwsgi]
wsgi-file=manage.py
daemonize=/var/log/aops/uwsgi/diana.log
http-timeout=600
harakiri=600
processes=2
threads=2
```

> **Set the MySQL database to the password mode**. For details, see [Q5: MySQL Password Mode](#q5-mysql-password-mode).

Start the aops-diana service.

```shell
systemctl start aops-diana
```

**Note: [Initialize the aops-diana database](#3423-initializing-the-aops-diana-database) before starting the service.**

> If the diana service fails to be started and the error message indicates that the MySQL database cannot be connected, check if a MySQL password is set. If yes, see [Q5: MySQL Password Mode](#q5-mysql-password-mode).

#### 3.4.2.3 Initializing the aops-diana Database

* Initialize the diana database.

```shell
cd /opt/aops/scripts/deploy
bash aops-basedatabase.sh init diana
```

**Note:If aops-tools is not installed, run the SQL script to initialize. The script path is /opt/aops/database/diana.sql**

[Q5: MySQL Password Mode](#q5-mysql-password-mode)

[FAQs: Nonexisting /opt/aops/scripts/deploy](#q7-nonexisting-optaopsscriptsdeploy)

## 3.5 Client Installation

aops-ceres functions as the client of A-Ops. It communicates with the A-Ops management center through SSH and provides functions such as host information collection and command response.

### 3.5.1 Node Information

| Host   | IP Address  | Module     |
| ------ | ----------- | ---------- |
| Host D | 192.168.1.4 | aops-ceres |

### 3.5.2  Client Deployment

```shell
yum install aops-ceres dnf-hotpatch-plugin -y
```

## FAQs

### Q1: Max Number of Connections

When host interfaces are added in batches, due to the max number of SSH connections (**MaxStartups**) of the host where aops-zeus is installed, some hosts may fail to be connected. You can temporarily increase **MaxStartups** as required. For details, see the [SSH documentation](https://www.man7.org/linux/man-pages/man5/sshd_config.5.html).

### Q2: 504 Gateway Timeout

Some HTTP interfaces may take a long time to execute, resulting in error 504 on the web client. You can reduce the probability of error 504 by adding **proxy\_read\_timeout** to the Nginx configuration or increase its value.

### Q3: Firewall

If firewall cannot be disabled, open the ports involved in service deployment on the firewall. Otherwise, services may be inaccessible and A-Ops cannot be used properly.

### Q4: Elasticsearch Access Denied

If Elasticsearch is deployed on multiple nodes in a distributed manner, set the cross-domain access configurations properly to enable the access of the nodes.

### Q5: MySQL Password Mode

* **Configure the mysql section in the service configuration.**

To set the password mode for the MySQL database connection (for example, the user is **root**, and the password is **123456**), change the value of **engine\_format** in the **\[mysql]** section in apollo and zeus configurations.

```ini
[mysql]
engine_format=mysql+pymysql://root:123456@%s:%s/%s
```

* **Modify the aops-basedatabase.sh initialization script.**

Modify the 145th line of **aops-basedatabase.sh**.

> Before modification:

```shell
database = pymysql.connect(host='$mysql_ip', port=$port, database='mysql', autocommit=True,client_flag=CLIENT.MULTI_STAT    EMENTS)
```

> After modification:

```shell
database = pymysql.connect(host='$mysql_ip', port=$port, database='mysql', password='password', user='user', autocommit=True, client_flag=CLIENT.MULTI_STATEMENTS)
```

* **Database connection error upon service startup**

Modify the 178th line in **/usr/bin/aops-vulcanus**.

> Before modification:

```shell
connect = pymysql.connect(host='$mysql_ip', port=$port, database='$aops_database')
```

> After modification:

```shell
connect = pymysql.connect(host='$mysql_ip', port=$port, database='$aops_database', password='password', user='user')
```

**Note: If a non-root user is used for logging in to the server, add user ="root" or a user allowed by MySQL.**

### Q6: update Repository Configuration

```shell
echo "[update]
name=update
baseurl=http://repo.openeuler.org/openEuler-24.03-LTS/update/$basearch/
enabled=1
gpgcheck=0
[update-epol]
name=update-epol
baseurl=http://repo.openeuler.org/openEuler-24.03-LTS/EPOL/update/main/$basearch/
enabled=1
gpgcheck=0" > /etc/yum.repos.d/openEuler-update.repo
```

> Note: Change **openEuler-24.03-LTS** to the actual OS version. You can also refer to the repository description in the openEuler official documentation.

### Q7: Nonexisting /opt/aops/scripts/deploy

During database initialization, if **/opt/aops/scripts/deploy** does not exits, install the aops-tools package.

```shell
yum install aops-tools -y
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/deploying_control_plane_components.md
---
# Deploying Components on the Control Plane

## Preparing the kubeconfig File for All Components

### kube-proxy

```bash
kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.122.154:6443 --kubeconfig=kube-proxy.kubeconfig
kubectl config set-credentials system:kube-proxy --client-certificate=/etc/kubernetes/pki/kube-proxy.pem --client-key=/etc/kubernetes/pki/kube-proxy-key.pem --embed-certs=true --kubeconfig=kube-proxy.kubeconfig
kubectl config set-context default --cluster=openeuler-k8s --user=system:kube-proxy --kubeconfig=kube-proxy.kubeconfig
kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
```

### kube-controller-manager

```bash
kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://127.0.0.1:6443 --kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-credentials system:kube-controller-manager --client-certificate=/etc/kubernetes/pki/kube-controller-manager.pem --client-key=/etc/kubernetes/pki/kube-controller-manager-key.pem --embed-certs=true --kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-context default --cluster=openeuler-k8s --user=system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig
kubectl config use-context default --kubeconfig=kube-controller-manager.kubeconfig
```

### kube-scheduler

```bash
kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://127.0.0.1:6443 --kubeconfig=kube-scheduler.kubeconfig
kubectl config set-credentials system:kube-scheduler --client-certificate=/etc/kubernetes/pki/kube-scheduler.pem --client-key=/etc/kubernetes/pki/kube-scheduler-key.pem --embed-certs=true --kubeconfig=kube-scheduler.kubeconfig
kubectl config set-context default  --cluster=openeuler-k8s --user=system:kube-scheduler --kubeconfig=kube-scheduler.kubeconfig
kubectl config use-context default --kubeconfig=kube-scheduler.kubeconfig
```

### admin

```bash
kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://127.0.0.1:6443 --kubeconfig=admin.kubeconfig
kubectl config set-credentials admin --client-certificate=/etc/kubernetes/pki/admin.pem --client-key=/etc/kubernetes/pki/admin-key.pem --embed-certs=true --kubeconfig=admin.kubeconfig
kubectl config set-context default --cluster=openeuler-k8s --user=admin --kubeconfig=admin.kubeconfig
kubectl config use-context default --kubeconfig=admin.kubeconfig
```

### Obtaining the kubeconfig Configuration File

```bash
admin.kubeconfig kube-proxy.kubeconfig  kube-controller-manager.kubeconfig  kube-scheduler.kubeconfig
```

## Configuration for Generating the Key Provider

When api-server is started, a key pair `--encryption-provider-config=/etc/kubernetes/pki/encryption-config.yaml` needs to be provided. In this document, a key pair `--encryption-provider-config=/etc/kubernetes/pki/encryption-config.yaml` is generated by using urandom:

```bash
$ cat generate.bash
#!/bin/bash

ENCRYPTION_KEY=$(head -c 32 /dev/urandom | base64)

cat > encryption-config.yaml <<EOF
kind: EncryptionConfig
apiVersion: v1
resources:
  - resources:
      - secrets
    providers:
      - aescbc:
          keys:
            - name: key1
              secret: ${ENCRYPTION_KEY}
      - identity: {}
EOF
# api-server startup configuration --encryption-provider-config=/etc/kubernetes/pki/encryption-config.yaml
```

## Copying Certificates

In this document, the certificates, keys, and configurations used by all components are stored in the `/etc/kubernetes/pki/` directory.

```bash
# Prepare the certificate directory.
$ mkdir -p /etc/kubernetes/pki/
$ ls /etc/kubernetes/pki/
admin-key.pem  encryption-config.yaml              kube-proxy-key.pem     kubernetes.pem             service-account-key.pem
admin.pem      kube-controller-manager-key.pem     kube-proxy.kubeconfig  kube-scheduler-key.pem     service-account.pem
ca-key.pem     kube-controller-manager.kubeconfig  kube-proxy.pem         kube-scheduler.kubeconfig
ca.pem         kube-controller-manager.pem         kubernetes-key.pem     kube-scheduler.pem
```

## Deploying RBAC of the Admin Role

Enable the admin role.

```bash
$ cat admin_cluster_role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: system:kube-apiserver-to-kubelet
rules:
  - apiGroups:
      - ""
    resources:
      - nodes/proxy
      - nodes/stats
      - nodes/log
      - nodes/spec
      - nodes/metrics
    verbs:
      - "*"

# Enable the admin role.
$ kubectl apply --kubeconfig admin.kubeconfig -f admin_cluster_role.yaml
```

Bind the admin role.

```bash
$ cat admin_cluster_rolebind.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: system:kube-apiserver
  namespace: ""
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:kube-apiserver-to-kubelet
subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: kubernetes

# Bind the admin role.
$ kubectl apply --kubeconfig admin.kubeconfig -f admin_cluster_rolebind.yaml
```

## Deploying the API Server Service

Modify the etc configuration file of apiserver.

```bash
$ cat /etc/kubernetes/apiserver
KUBE_ADVERTIS_ADDRESS="--advertise-address=192.168.122.154"
KUBE_ALLOW_PRIVILEGED="--allow-privileged=true"
KUBE_AUTHORIZATION_MODE="--authorization-mode=Node,RBAC"
KUBE_ENABLE_ADMISSION_PLUGINS="--enable-admission-plugins=NamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota"
KUBE_SECURE_PORT="--secure-port=6443"
KUBE_ENABLE_BOOTSTRAP_TOKEN_AUTH="--enable-bootstrap-token-auth=true"
KUBE_ETCD_CAFILE="--etcd-cafile=/etc/kubernetes/pki/ca.pem"
KUBE_ETCD_CERTFILE="--etcd-certfile=/etc/kubernetes/pki/kubernetes.pem"
KUBE_ETCD_KEYFILE="--etcd-keyfile=/etc/kubernetes/pki/kubernetes-key.pem"
KUBE_ETCD_SERVERS="--etcd-servers=https://192.168.122.154:2379,https://192.168.122.155:2379,https://192.168.122.156:2379"
KUBE_CLIENT_CA_FILE="--client-ca-file=/etc/kubernetes/pki/ca.pem"
KUBE_KUBELET_CERT_AUTH="--kubelet-certificate-authority=/etc/kubernetes/pki/ca.pem"
KUBE_KUBELET_CLIENT_CERT="--kubelet-client-certificate=/etc/kubernetes/pki/kubernetes.pem"
KUBE_KUBELET_CLIENT_KEY="--kubelet-client-key=/etc/kubernetes/pki/kubernetes-key.pem"
KUBE_KUBELET_HTTPS="--kubelet-https=true"
KUBE_PROXY_CLIENT_CERT_FILE="--proxy-client-cert-file=/etc/kubernetes/pki/kube-proxy.pem"
KUBE_PROXY_CLIENT_KEY_FILE="--proxy-client-key-file=/etc/kubernetes/pki/kube-proxy-key.pem"
KUBE_TLS_CERT_FILE="--tls-cert-file=/etc/kubernetes/pki/kubernetes.pem"
KUBE_TLS_PRIVATE_KEY_FILE="--tls-private-key-file=/etc/kubernetes/pki/kubernetes-key.pem"
KUBE_SERVICE_CLUSTER_IP_RANGE="--service-cluster-ip-range=10.32.0.0/16"
KUBE_SERVICE_ACCOUNT_ISSUER="--service-account-issuer=https://kubernetes.default.svc.cluster.local"
KUBE_SERVICE_ACCOUNT_KEY_FILE="--service-account-key-file=/etc/kubernetes/pki/service-account.pem"
KUBE_SERVICE_ACCOUNT_SIGN_KEY_FILE="--service-account-signing-key-file=/etc/kubernetes/pki/service-account-key.pem"
KUBE_SERVICE_NODE_PORT_RANGE="--service-node-port-range=30000-32767"
KUB_ENCRYPTION_PROVIDER_CONF="--encryption-provider-config=/etc/kubernetes/pki/encryption-config.yaml"
KUBE_REQUEST_HEADER_ALLOWED_NAME="--requestheader-allowed-names=front-proxy-client"
KUBE_REQUEST_HEADER_EXTRA_HEADER_PREF="--requestheader-extra-headers-prefix=X-Remote-Extra-"
KUBE_REQUEST_HEADER_GROUP_HEADER="--requestheader-group-headers=X-Remote-Group"
KUBE_REQUEST_HEADER_USERNAME_HEADER="--requestheader-username-headers=X-Remote-User"
KUBE_API_ARGS=""
```

All apiserver configurations are defined in the `/etc/kubernetes/config` file and can be directly used in the service file.

Most configurations are fixed. Note the following:

* The value of `--service-cluster-ip-range` must be the same as that of `clusterDNS`.

### Compiling the systemd Configuration of apiserver

```bash
cat /usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://kubernetes.io/docs/reference/generated/kube-apiserver/
After=network.target
After=etcd.service

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver \
     $KUBE_ADVERTIS_ADDRESS \
     $KUBE_ALLOW_PRIVILEGED \
     $KUBE_AUTHORIZATION_MODE \
     $KUBE_ENABLE_ADMISSION_PLUGINS \
      $KUBE_SECURE_PORT \
     $KUBE_ENABLE_BOOTSTRAP_TOKEN_AUTH \
     $KUBE_ETCD_CAFILE \
     $KUBE_ETCD_CERTFILE \
     $KUBE_ETCD_KEYFILE \
     $KUBE_ETCD_SERVERS \
     $KUBE_CLIENT_CA_FILE \
     $KUBE_KUBELET_CERT_AUTH \
     $KUBE_KUBELET_CLIENT_CERT \
     $KUBE_KUBELET_CLIENT_KEY \
     $KUBE_PROXY_CLIENT_CERT_FILE \
     $KUBE_PROXY_CLIENT_KEY_FILE \
     $KUBE_TLS_CERT_FILE \
     $KUBE_TLS_PRIVATE_KEY_FILE \
     $KUBE_SERVICE_CLUSTER_IP_RANGE \
     $KUBE_SERVICE_ACCOUNT_ISSUER \
     $KUBE_SERVICE_ACCOUNT_KEY_FILE \
     $KUBE_SERVICE_ACCOUNT_SIGN_KEY_FILE \
     $KUBE_SERVICE_NODE_PORT_RANGE \
     $KUBE_LOGTOSTDERR \
     $KUBE_LOG_LEVEL \
     $KUBE_API_PORT \
     $KUBELET_PORT \
     $KUBE_ALLOW_PRIV \
     $KUBE_SERVICE_ADDRESSES \
     $KUBE_ADMISSION_CONTROL \
     $KUB_ENCRYPTION_PROVIDER_CONF \
     $KUBE_REQUEST_HEADER_ALLOWED_NAME \
     $KUBE_REQUEST_HEADER_EXTRA_HEADER_PREF \
     $KUBE_REQUEST_HEADER_GROUP_HEADER \
     $KUBE_REQUEST_HEADER_USERNAME_HEADER \
     $KUBE_API_ARGS
Restart=on-failure
Type=notify
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
```

## Deploying the controller-manager Service

Modify the controller-manager configuration file.

```bash
$ cat /etc/kubernetes/controller-manager
KUBE_BIND_ADDRESS="--bind-address=127.0.0.1"
KUBE_CLUSTER_CIDR="--cluster-cidr=10.200.0.0/16"
KUBE_CLUSTER_NAME="--cluster-name=kubernetes"
KUBE_CLUSTER_SIGNING_CERT_FILE="--cluster-signing-cert-file=/etc/kubernetes/pki/ca.pem"
KUBE_CLUSTER_SIGNING_KEY_FILE="--cluster-signing-key-file=/etc/kubernetes/pki/ca-key.pem"
KUBE_KUBECONFIG="--kubeconfig=/etc/kubernetes/pki/kube-controller-manager.kubeconfig"
KUBE_LEADER_ELECT="--leader-elect=true"
KUBE_ROOT_CA_FILE="--root-ca-file=/etc/kubernetes/pki/ca.pem"
KUBE_SERVICE_ACCOUNT_PRIVATE_KEY_FILE="--service-account-private-key-file=/etc/kubernetes/pki/service-account-key.pem"
KUBE_SERVICE_CLUSTER_IP_RANGE="--service-cluster-ip-range=10.32.0.0/24"
KUBE_USE_SERVICE_ACCOUNT_CRED="--use-service-account-credentials=true"
KUBE_CONTROLLER_MANAGER_ARGS="--v=2"
```

### Compiling the systemd Configuration File of controller-manager

```bash
$ cat /usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://kubernetes.io/docs/reference/generated/kube-controller-manager/

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager \
     $KUBE_BIND_ADDRESS \
     $KUBE_LOGTOSTDERR \
     $KUBE_LOG_LEVEL \
     $KUBE_CLUSTER_CIDR \
     $KUBE_CLUSTER_NAME \
     $KUBE_CLUSTER_SIGNING_CERT_FILE \
     $KUBE_CLUSTER_SIGNING_KEY_FILE \
     $KUBE_KUBECONFIG \
     $KUBE_LEADER_ELECT \
     $KUBE_ROOT_CA_FILE \
     $KUBE_SERVICE_ACCOUNT_PRIVATE_KEY_FILE \
     $KUBE_SERVICE_CLUSTER_IP_RANGE \
     $KUBE_USE_SERVICE_ACCOUNT_CRED \
     $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
```

## Deploying the scheduler Service

Modify the scheduler configuration file.

```bash
$ cat /etc/kubernetes/scheduler
KUBE_CONFIG="--kubeconfig=/etc/kubernetes/pki/kube-scheduler.kubeconfig"
KUBE_AUTHENTICATION_KUBE_CONF="--authentication-kubeconfig=/etc/kubernetes/pki/kube-scheduler.kubeconfig"
KUBE_AUTHORIZATION_KUBE_CONF="--authorization-kubeconfig=/etc/kubernetes/pki/kube-scheduler.kubeconfig"
KUBE_BIND_ADDR="--bind-address=127.0.0.1"
KUBE_LEADER_ELECT="--leader-elect=true"
KUBE_SCHEDULER_ARGS=""
```

### Compiling the systemd Configuration File of scheduler

```bash
$ cat /usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler Plugin
Documentation=https://kubernetes.io/docs/reference/generated/kube-scheduler/

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler \
     $KUBE_LOGTOSTDERR \
     $KUBE_LOG_LEVEL \
     $KUBE_CONFIG \
     $KUBE_AUTHENTICATION_KUBE_CONF \
     $KUBE_AUTHORIZATION_KUBE_CONF \
     $KUBE_BIND_ADDR \
     $KUBE_LEADER_ELECT \
     $KUBE_SCHEDULER_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
```

## Enabling Each Component

```bash
systemctl enable kube-controller-manager kube-scheduler kube-proxy
systemctl restart kube-controller-manager kube-scheduler kube-proxy
```

## Verifying Basic Functions

```bash
$ curl --cacert /etc/kubernetes/pki/ca.pem https://192.168.122.154:6443/version
{
  "major": "1",
  "minor": "20",
  "gitVersion": "v1.20.2",
  "gitCommit": "faecb196815e248d3ecfb03c680a4507229c2a56",
  "gitTreeState": "archive",
  "buildDate": "2021-03-02T07:26:14Z",
  "goVersion": "go1.15.7",
  "compiler": "gc",
  "platform": "linux/arm64"
}
```

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop.md
---


---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/performance/powerapi/development_using_powerapi.md
---
# Development Using powerapi

## (Optional) Source Code Download

Source code download: <https://atomgit.com/openeuler/powerapi>

Some interfaces in the source code are experimental and have not been released. **RELEASE\_MODE** in the code indicates whether the interfaces are officially released.

For details about released APIs, see the API document in the source code repository.

## Installation

Run the following command to install the powerapi-devel software package and use the provided interfaces for development:

```sh
yum install powerapi-devel
```

## Basic Procedure

To use query interfaces, you only need to register with powerapi. To use setting interfaces, you need to register with powerapi and request control. Based on service scenarios, the service processes are as follows:
Discrete perception service scenario: log callback setting -> registration -> query interface calling -> deregistration
Configuration service scenario: log callback setting -> registration -> Control request -> Configuration interface calling -> Control release -> deregistration

## powerapi APIs

### General APIs

#### Setting the Log Callback Function

Definition:

```c
PWR_API int PWR_SetLogCallback(void(LogCallback)(int level, const char *fmt, va_list vl))
```

Description:\
Sets the callback logs. After the logs are set, the powerapi library prints the **LogCallBack** function to the logs. If they are not set, the **LogCallBack** function is printed to the terminal by default. This API can be called before registration.

Parameters:

Parameter|Type|Description
\--------|---------|--------
LogCallBack|void(\*)|Log callback function pointer

Returns:

Type|Description
\--------|---------
int|0: Succeeded.4: Failed. The callback function pointer is empty.

#### Setting Server Information

Definition:

```c
PWR_API int PWR_SetServerInfo(const char* socketPath)
```

Description:\
Sets the address of the Unix domain socket communication server.

Parameters:

Parameter|Type|Description
\--------|---------|--------
socketPath|const char\*|Path of the server.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.4: Failed. The callback function pointer is empty.

Note:\
The default path of the server socket file is **/etc/sysconfig/pwrapis/pwrserver.sock**. If you change the default path in the pwrapis configuration file, ensure that the directory permission is 755 and the file permission is 722.
After the path is changed, use this interface to specify the new path of the socket before registration. Otherwise, the connection fails.

#### Registration

Definition:

```c
PWR_API int PWR_Register(void)
```

Description:\
Registers with the powerapi service.

Parameters: None\
Returns:

Type|Description
\--------|---------
int|0: Succeeded.1: Failed to initialize the socket client.

#### Deregistration

Definition:

```c
PWR_API int PWR_UnRegister(void)
```

Description:\
Deregisters with the powerapi service.

Parameters: None\
Returns:

Type|Description
\--------|---------
int|0: Succeeded.

#### Requesting Energy Efficiency Control

Definition:

```c
PWR_API int PWR_RequestControlAuth(void)
```

Description:\
Requests control of the system energy efficiency. After an upper-layer application takes over energy efficiency control, the system does not automatically adjust energy efficiency anymore.

Parameters: None\
Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Releasing Energy Efficiency Control

Definition:

```c
PWR_API int PWR_ReleaseControlAuth(void)
```

Description:\
Releases control of the system energy efficiency.

Parameters: None\
Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

### CPU

#### Obtaining CPU Information

Definition:

```c
PWR_API int PWR_CPU_GetInfo(PWR_CPU_Info *cpuInfo)
```

Description:\
Obtains CPU information, including basic CPU information and NUMA information.

Parameters:

Parameter|Type|Description
\--------|---------|--------
cpuinfo|PWR\_CPU\_Info\*|CPU information.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Obtaining the CPU Frequency Ability

Definition:

```c
PWR_API int PWR_CPU_GetFreqAbility(PWR_CPU_FreqAbility *freqAbi, uint32_t bufferSize)
```

Description:\
Queries the information about the available CPU frequency domain, governor, and currently used CPU frequency driver.

Parameters:

Parameter|Type|Description
\--------|---------|--------
freqAbi|PWR\_CPU\_FreqAbility\*|CPU frequency ability information
bufferSize|uint32\_t|Size of the freqAbi memory block.Recommended size:**sizeof(PWR\_CPU\_FreqAbility) + *CPU core count* x (sizeof(int) + 5)**&#x49;f the size is too small, only the frequency domain data that can be contained is returned.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Obtaining the CPU Frequency Governor

Definition:

```c
PWR_API int PWR_CPU_GetFreqGovernor(char gov[], uint32_t size)
```

Description:\
Obtains the CPU frequency governor in use. By default, the governor of the first frequency domain is obtained.

Parameters:

Parameter|Type|Description
\--------|---------|--------
gov|char\[]|Governor name. The value can contain a maximum of 31 characters.
size|uint32\_t|Size of the **gov** array. The value must be greater than or equal to **PWR\_MAX\_ELEMENT\_NAME\_LEN(32)**.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Setting the CPU Frequency Governor

Definition:

```c
PWR_API int PWR_CPU_SetFreqGovernor(const char gov[])
```

Description:\
Sets the CPU frequency governor in use. (The governor will be set for all frequency domains).

Parameters:

Parameter|Type|Description
\--------|---------|--------
gov|char\[]|Governor name. The value can contain a maximum of 31 characters.Examples:**conservative****ondemand****userspace****powersave****performance****schedutil****seep**

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Obtaining All Attributes of the CPU Frequency Governor

Definition:

```c
PWR_API int PWR_CPU_GetFreqGovAttrs(PWR_CPU_FreqGovAttrs *govAttrs)
```

Description:\
Obtains all attributes of the CPU frequency governor in use.

Parameters:

Parameter|Type|Description
\--------|---------|--------
govAttrs|PWR\_CPU\_FreqGovAttrs\*|Attributes of the frequency governor.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Obtaining an Attribute of the CPU Frequency Governor

Definition:

```c
PWR_API int PWR_CPU_GetFreqGovAttr(PWR_CPU_FreqGovAttr *govAttr)
```

Description:\
Obtains the attribute of the current CPU frequency in use. The attribute corresponding to the governor used by the first frequency domain (**policy0**) is obtained.

Parameters:

Parameter|Type|Description
\--------|---------|--------
govAttrs|PWR\_CPU\_FreqGovAttrs\*|Attributes of the frequency governor.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Setting an Attribute of the CPU Frequency Governor

Definition:

```c
PWR_API int PWR_CPU_SetFreqGovAttr(const PWR_CPU_FreqGovAttr *govAttr)
```

Description:\
Sets the attribute of the current CPU frequency in use. The attribute corresponding to the governor used by the first frequency domain (**policy0**) is set.

Parameters:

Parameter|Type|Description
\--------|---------|--------
govAttrs|PWR\_CPU\_FreqGovAttrs\*|Attribute of the frequency governor.You need to specify the name and value of the attribute to be set.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

Note:\
Different governor support different attributes. The attributes supported by the governor are stored in **/sys/devices/system/cpu/cpufreq/{gov}/**, where **{gov}** indicates the name of the current governor.

#### Obtaining the CPU Frequency Range

Definition:

```c
PWR_API int PWR_CPU_GetFreqRange(PWR_CPU_FreqRange *freqRange)
```

Description:\
Obtains the CPU frequency range. By default, the frequency range of the first frequency domain is obtained.

Parameters:

Parameter|Type|Description
\--------|---------|--------
freqRange|PWR\_CPU\_FreqRange\*|Frequency range to obtain.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Setting the CPU Frequency Range

Definition:

```c
PWR_API int PWR_CPU_SetFreqRange(const PWR_CPU_FreqRange *freqRange)
```

Description:\
Sets the CPU frequency range. The frequency range will be set for all frequency domains.

Parameters:

Parameter|Type|Description
\--------|---------|--------
freqRange|PWR\_CPU\_FreqRange\*|Frequency range to set.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Obtaining the Current CPU Frequency

Definition:

```c
PWR_API int PWR_CPU_GetFreq(PWR_CPU_CurFreq curFreq[], uint32_t *num, int spec)
```

Description:\
Obtains the current frequency of the frequency domain.

Parameters:

Parameter|Type|Description
\--------|---------|--------
curFreq|PWR\_CPU\_CurFreq\[]|Frequency information of the current frequency domain of the policy to be queried.When **spec** is set to 1, **policyId** of the corresponding member needs to be set.The current frequency of the frequency domain will be output.
num|uint32\_t \*|Length of the **curFreq** array, indicating the number of policies to be queried.The output is the length of the valid data returned by the system (the smaller value between the actual number of policies and the input **num**).
spec|int|Whether to obtain the information about one or more specific frequency domains.0: No.1: Yes. In this case, you need to set the **policyId** corresponding to the specific frequency domain in the **curFreq** member, for example, 32 or 64.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Setting the Current CPU Frequency

Definition:

```c
PWR_API int PWR_CPU_SetFreq(const PWR_CPU_CurFreq curFreq[], uint32_t num)
```

Description:\
Sets the frequency of the frequency domain, which can be set only when the CPU frequency governor is set to **userspace**.

Parameters:

Parameter|Type|Description
\--------|---------|--------
curFreq|PWR\_CPU\_CurFreq\[]|Frequency domain to be set and its frequency list.
num|uint32\_t|Length of the **curFreq** array, indicating the number of policies to be set.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Obtaining the CPU Idle Ability and Status Information

Definition:

```c
PWR_API int PWR_CPU_GetIdleInfo(PWR_CPU_IdleInfo *idleInfo)
```

Description:\
Obtains the CPU idle ability and status information.

Parameters:

Parameter|Type|Description
\--------|---------|--------
idleInfo|PWR\_CPU\_IdleInfo\*|CPU idle ability and status information.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Obtaining the CPU Idle Governor

Definition:

```c
PWR_API int PWR_CPU_GetIdleGovernor(char idleGov[], uint32_t size)
```

Description:\
Obtains the CPU idle mode.

Parameters:

Parameter|Type|Description
\--------|---------|--------
idleGov|char\[]|Governor name.
size|uint32\_t|Size of the **idleGov** buffer. The minimum value is **PWR\_MAX\_ELEMENT\_NAME\_LEN(32)**.

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Setting the CPU Idle Governor

Definition:

```c
PWR_API int PWR_CPU_SetIdleGovernor(const char idleGov[])
```

Description:\
Sets the CPU idle mode.

Parameters:

Parameter|Type|Description
\--------|---------|--------
idleGov|char\[]|Governor name, for example:**laddermenuteohaltpoll**

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Obtaining the CPU and DMA Latency

Definition:

```c
PWR_API int PWR_CPU_DmaGetLatency(int *latency)
```

Description:\
Obtains the acceptable latency of the CPU and DMA.

Parameters:

Parameter|Type|Description
\--------|---------|--------
latency|int\*|Latency (us).

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

#### Setting the CPU and DMA Latency

Definition:

```c
PWR_API int PWR_CPU_DmaSetLatency(int latency)
```

Description:\
Sets the acceptable latency of the CPU and DMA.

Parameters:

Parameter|Type|Description
\--------|---------|--------
latency|int\*|Latency (us). Value range: \[0, 2000000000]

Returns:

Type|Description
\--------|---------
int|0: Succeeded.Other value: Failed. See the error codes for details.

Note:\
The CPU requires different wake-up time in different C-states. The wake-up latency increases as the C-states get deeper. Therefore, the system checks the CPU and DMA latency before entering a C-state. If the latency in the C-state is longer than the CPU and DMA latency, the CPU does not enter the C-state.
Reference Wake-up Latency of Each C-state (us)

C-state|Latency
\--------|---------
C0 POLL|0
C1|2
C1E|10
C3|40
C6|133
C7S|166
C8|300
C9|600
C10|2600

## Usage

Save the following code as **powerapi\_test.c**.

```c
#include <stdio.h>
#include <unistd.h>
#include <signal.h>
#include <string.h>
#include <stdlib.h>
#include <pwrapic/powerapi.h>

#define MAIN_LOOP_INTERVAL 5
#define TEST_FREQ 2400
#define TEST_CORE_NUM 128
#define AVG_LEN_PER_CORE 5
#define TEST_CPU_DMA_LATENCY 2000
#define TASK_INTERVAL 1000
#define TASK_RUN_TIME 10
#define TEST_FREQ_RANGE_MIN 500
#define TEST_FREQ_RANGE_MAX 2500

static int g_run = 1;

static void PrintResult(char *function, int ret)
{
    int length = 24;
    printf("[TEST ]    ");
    printf("%-*s", length, function);
    printf(":");
    if (ret == PWR_SUCCESS) {
        printf("SUCCESS ret: %d\n", ret);
    } else {
        printf("ERROR   ret: %d\n", ret);
    }
}

enum {
    DEBUG = 0,
    INFO,
    WARNING,
    ERROR
};

static const char *GetLevelName(int level)
{
    static char debug[] = "DEBUG";
    static char info[] = "INFO";
    static char warning[] = "WARNING";
    static char error[] = "ERROR";
    switch (level) {
        case DEBUG:
            return debug;
        case INFO:
            return info;
        case WARNING:
            return warning;
        case ERROR:
            return error;
        default:
            return info;
    }
}

void LogCallback(int level, const char *fmt, va_list vl)
{
    char logLine[4096] = {0};
    char message[4000] = {0};
    int length = 5;

    if (vsnprintf(message, sizeof(message) - 1, fmt, vl) < 0) {
        return;
    }

    printf("[");
    printf("%-*s", length, GetLevelName(level));
    printf("]    %s\n", message);
}

static void SignalHandler(int none)
{
    g_run = 0;
}

static void SetupSignal(void)
{
    // regist signal handler
    (void)signal(SIGINT, SignalHandler);
    (void)signal(SIGUSR1, SignalHandler);
    (void)signal(SIGUSR2, SignalHandler);
    (void)signal(SIGTERM, SignalHandler);
    (void)signal(SIGKILL, SignalHandler);
}
/************************** COMMON ************************/
static void TEST_PWR_SetLogCallback(void)
{
    int ret = -1;
    ret = PWR_SetLogCallback(LogCallback);
    PrintResult("PWR_SetLogCallback", ret);
}

static void TEST_PWR_SetServerInfo(void)
{
    int ret = -1;
    char str[] = "/etc/sysconfig/pwrapis/pwrserver.sock";
    ret = PWR_SetServerInfo(str);
    PrintResult("PWR_SetServerInfo", ret);
}

static void TEST_PWR_Register(void)
{
    while (PWR_Register() != PWR_SUCCESS) {
        sleep(MAIN_LOOP_INTERVAL);
        PrintResult("PWR_Register", PWR_ERR_COMMON);
        continue;
    }
    PrintResult("PWR_Register", PWR_SUCCESS);
}

static void TEST_PWR_RequestControlAuth(void)
{
    int ret = -1;
    ret = PWR_RequestControlAuth();
    PrintResult("PWR_RequestControlAuth", ret);
}
/************************** COMMON END************************/

/***************************** CPU ***************************/
static void TEST_PWR_CPU_GetInfo(void)
{
    int ret = -1;
    PWR_CPU_Info *info = (PWR_CPU_Info *)malloc(sizeof(PWR_CPU_Info));
    if (!info) {
        return;
    }
    bzero(info, sizeof(PWR_CPU_Info));
    ret = PWR_CPU_GetInfo(info);
    PrintResult("PWR_CPU_GetInfo", ret);
    printf("    arch: %s\n    coreNum: %d\n    maxFreq: %f\n    minFreq: %f\n    "
        "modelName: %s\n    numaNum: %d\n    threadsPerCore: %d\n", info->arch,
        info->coreNum, info->maxFreq, info->minFreq, info->modelName, info->numaNum,
        info->threadsPerCore);
    for (int i = 0; i < info->numaNum; i++) {
        printf("    numa node[%d]  cpuList: %s\n", info->numa[i].nodeNo, info->numa[i].cpuList);
    }
    free(info);
}

static void TEST_PWR_CPU_GetFreq(void)
{
    int ret = -1;
    int num = 0;
    int spec = 0;
    int i = 0;

    /**
     * Test 1: spec = 0, get all policy freq.
     * Set the num to the number of CPU cores
     * (it is possible that one kernel corresponds to one policy)
     */
    num = TEST_CORE_NUM;
    spec = 0;
    PWR_CPU_CurFreq cpuCurFreq1[num];
    bzero(cpuCurFreq1, num * sizeof(PWR_CPU_CurFreq));
    ret = PWR_CPU_GetFreq(cpuCurFreq1, &num, spec);
    PrintResult("1  PWR_CPU_GetFreq", ret);
    for (i = 0; i < num; i++) {
        printf("    policy[%d]: %lf\n", cpuCurFreq1[i].policyId, cpuCurFreq1[i].curFreq);
    }

    /**
     * Test 2: spec = 0 num = 2. get the previous 2 policies freq
     */
    ret = -1;
    // 2: previous 2 policies
    num = 2;
    spec = 0;
    PWR_CPU_CurFreq cpuCurFreq2[num];
    bzero(cpuCurFreq2, num * sizeof(PWR_CPU_CurFreq));
    ret = PWR_CPU_GetFreq(cpuCurFreq2, &num, spec);
    PrintResult("2  PWR_CPU_GetFreq", ret);
    for (i = 0; i < num; i++) {
        printf("    policy[%d]: %lf\n", cpuCurFreq2[i].policyId, cpuCurFreq2[i].curFreq);
    }

    /**
     * Test 3: spec = 1, get the two target policy freq
     */
    ret = -1;
    // 2: previous 2 policies
    num = 2;
    spec = 1;
    PWR_CPU_CurFreq cpuCurFreq3[num];
    bzero(cpuCurFreq3, num * sizeof(PWR_CPU_CurFreq));
    cpuCurFreq3[0].policyId = 0;
    // 32 : the Id of the second policy.
    cpuCurFreq3[1].policyId = 32;
    ret = PWR_CPU_GetFreq(cpuCurFreq3, &num, spec);
    PrintResult("3  PWR_CPU_GetFreq", ret);
    for (i = 0; i < num; i++) {
        printf("    policy[%d]: %lf\n", cpuCurFreq3[i].policyId, cpuCurFreq3[i].curFreq);
    }
}

static void TEST_PWR_CPU_SetFreq(void)
{
    int ret = -1;
    int num = 1;
    PWR_CPU_CurFreq cpuCurFreq[num];
    bzero(cpuCurFreq, num * sizeof(PWR_CPU_CurFreq));
    cpuCurFreq[0].policyId = 0;
    cpuCurFreq[0].curFreq = TEST_FREQ;
    ret = PWR_CPU_SetFreq(cpuCurFreq, num);
    PrintResult("PWR_CPU_SetFreq", ret);

    int spec = 1;
    bzero(cpuCurFreq, num * sizeof(PWR_CPU_CurFreq));
    cpuCurFreq[0].policyId = 0;
    ret = PWR_CPU_GetFreq(cpuCurFreq, &num, spec);
    printf("    current policy[%d]: %lf\n", cpuCurFreq[0].policyId, cpuCurFreq[0].curFreq);
}
/*************************** CPU END *************************/

int main(int argc, const char *args[])
{
    /********** Common **********/
    TEST_PWR_SetServerInfo();
    TEST_PWR_SetLogCallback();
    TEST_PWR_Register();
    TEST_PWR_RequestControlAuth();

    /************ CPU ***********/
    TEST_PWR_CPU_GetInfo();
    TEST_PWR_CPU_GetFreq();
    TEST_PWR_CPU_SetFreq();

    PWR_ReleaseControlAuth();
    PWR_UnRegister();
    return 0;
}
```

Run `gcc` to compile the program.

```sh
gcc powerapi_test.c -o powerapi_test -lpwrapi
```

Run the program to view the result.

```sh
./powerapi_test
```

---

---
url: /en/docs/22.03_LTS_SP4/server/administration/sysmaster/device_management.md
---
# Device Management

The device manager is a bridge between user-mode software and underlying physical devices, supporting the operation of key base software such as lvm2 and NetworkManager. As the device management component of sysMaster, devmaster supports quick startup of sysMaster and ecosystem compatibility of user-mode software. In addition, devmaster provides layered, decoupled, and scalable device management capabilities for common OSs based on the summary and contemplation of mainstream Linux device management solutions.

devmaster consists of a daemon, a client tool, and a dynamic library. The devmaster daemon utilizes kernel mechanisms such as netlink, inotify, and sysfs to monitor device events and trigger rule processing tasks. The `devctl` client tool and **libs** dynamic library provide a set of CLI commands and public interfaces for debugging rules, controlling daemons, and querying device status. The following figure shows the overall architecture of devmaster.

**Figure 1 devmaster overall architecture**
![devmaster\_architecture](./figures/devmaster_architecture.png)

devmaster is written in the Rust language to ensure memory safety. The core functions of devmaster are as follows:

1. Event-driven operations: The queue cache and worker pool mechanisms are used to meet the requirements of highly concurrent device events. In addition, user-mode processes can be dynamically notified of the readiness of devices.
2. Separation of mechanisms and policies: Device processing logic is defined as rules rather than hard-coded in service code, allowing for on-demand customization and flexible combination.
3. Ecosystem compatibility: devmaster is compatible with the udev syntax and udev user-mode broadcast protocol. Existing services can be migrated to the devmaster environment with low costs.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/eggo_dismantling_a_cluster.md
---
# Dismantling a Cluster

When service requirements decrease and the existing number of nodes is not required, you can delete nodes from the cluster to save system resources and reduce costs. Or, when the service does not require a cluster, you can delete the entire cluster.

## Deleting Nodes

You can use the command line to delete nodes from the cluster. For example, to delete all node types whose IP addresses are *192.168.0.5* and *192.168.0.6* from the k8s-cluster, run the following command:

```shell
eggo -d delete --id k8s-cluster 192.168.0.5 192.168.0.6
```

## Deleting the Entire Cluster

> \[!NOTE]**NOTE:**
>
> * When a cluster is deleted, all data in the cluster is deleted and cannot be restored. Exercise caution when performing this operation.
> * Currently, dismantling a cluster does not delete the containers and the container images. However, if the Kubernetes cluster is configured to install a container engine during the deployment, the container engine will be deleted. As a result, the containers may run abnormally.
> * Some error information may be displayed when dismantling the cluster. Generally, this is caused by the error results returned during the delete operations. The cluster can still be properly dismantled.

You can use the command line to delete the entire cluster. For example, run the following command to delete the k8s-cluster:

```shell
eggo -d cleanup --id k8s-cluster
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/distributed/distributed_data_management.md
---
# Distributed Data Management

Distributed data management leverages distributed virtual bus to manage application data and user data distributed on different devices. Under such management, user data is no longer bound to a single physical device, service logic is decoupled from storage, and applications are running across devices.

Distributed data management is ported from the upstream OpenHarmony 3.2 Release. It consists of the following components.

| Component                      | openEuler Software Package          | Description                                                                                    |
| ------------------------------ | ----------------------------------- | ---------------------------------------------------------------------------------------------- |
| Distributed Data Service (DDS) | distributeddatamgr\_datamgr\_service  | Provides the capability to store data in the databases of different devices.                   |
| KV store                       | distributeddatamgr\_kv\_store         | Manages key-value pairs for device applications.                                               |
| Relational database            | distributeddatamgr\_relational\_store | Manages data using a relational model.                                                         |
| Distributed Data Object        | distributeddatamgr\_data\_object      | An object-oriented in-memory data management framework featuring multi-device synchronization. |

For more information about distributed data management, see section [Data Management](https://gitcode.com/openharmony/docs/blob/master/en/application-dev/database/data-mgmt-overview.md) in the OpenHarmony document .

## Installation

Distributed data management has been integrated in openEuler 22.03 LTS SP4 by default. You can directly install it.

```shell
dnf install distributeddatamgr_kv_store distributeddatamgr_relational_store distributeddatamgr_datamgr_service distributeddatamgr_data_object
```

## Service Startup

1. You can run the **start\_services.sh** script to start the DDS.

   ```shell
   /system/bin/start_services.sh datamgr
   ```

2. You can run the `ps` command to check whether the distributed data management service is started.

   ```shell
   ps -ef | grep distributeddata
   ```

## Usage

1. Create the **/data** directory required for running the demo. OpenHarmony applications require specific directories for running. Therefore, you need to create the directories in openEuler.

   ```shell
   mkdir -p /data/app/el0/0/database/com.example.distributed.rdb/rdb
   mkdir -p /data/app/el1/0/database/distributeddata/kvdb
   mkdir -p /data/service/el1/public/database/distributeddata/meta
   mkdir -p /data/service/el1/public/database/distributeddata/kvdb
   mkdir -p /data/service/el1/public/database/distributeddata/meta/backup
   ```

2. Write the client programs for the three databases (**kv\_store**, **data\_object**, and **relational\_store**) to use the distributed data function.

   For details about how to write the client programs, see the demo source code of each database in [Repository](https://gitee.com/heppen/distributed-data-files). The source code is stored in the **demo** directory in the directory corresponding to each database, for example, **kv\_store/demo**. You can use the **build.sh** script in the **demo** directory for compilation.

   > **Notice**
   >
   > It is recommended that the database path **db** specified by the demo be the same as the path of the sample demo. Otherwise, the path may not exist or the permission may be insufficient.

3. Stop the distributed management service **datamgr\_service**.

   ```shell
   ./stop_services.sh all #Stop all services.
   ./stop_services.sh datamgr #Stop only the datamgr service.
   ```

## FAQs

* When a service is started, the error message "Binder Driver died " is displayed.

  Cause: Binder is not enabled in the system. You can check whether the **/dev/binder** file exists. If the file does not exist, Binder is not enabled.

  Solution: Enable the binder function. For details, see [communication\_ipc Repository Description](https://atomgit.com/src-openeuler/communication_ipc/blob/46d83ed1462e521ce356aec48ef980dbf84cff80/README.md).

* The softbus\_server service fails to be started, and the error message "GetNetworkIfIp ifName:eth0 fail" is displayed.

  Cause: Run the `ip a` command to view the name of the NIC in the current system and check whether the wired NIC **eth0** exists. The softbus\_server service obtains information such as the IP address through the wired NIC **eth0**. If **eth0** does not exist, softbus\_server cannot be started.

  Solution 1: Change the NIC name to **eth0**.

  Solution 2: Modify the softbus\_server source code and change the name of the dependent wired NIC to that of the NIC in the current system.

## References

[hmdfs Distributed File System Overview](hmdfs_distributed_file_system_overview.md)

---

---
url: /en/docs/22.03_LTS_SP4/server/development/distributed/overview.md
---
# Distributed Middleware User Guide

This document describes how to use DSoftBus on openEuler for for multi-device communication and introduces the DSoftBus-based distributed file system.

You can learn about the native discovery and connection mode between openEuler edge servers, embedded devices, and OpenHarmony devices, as well as related extension applications.

Users must:

* Know basic Linux operations.

* Understand the application development and test processes of OpenHarmony and openEuler.

* Be familiar with the background knowledge of IPC and PRC, as well as data synchronization in distributed architectures.

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/aops/dnf_command_usage.md
---
# DNF Command Usage

Af ter installing dnf-hotpatch-plugin, you can run `dnf` commands to use Ceres functions related to hot/cold patches, such as hot patch scanning (`dnf hot-updateinfo`), setting and querying (`dnf hotpatch`), applying (`dnf hotupgrade`), and kabi check before kernel upgrade (`dnf upgrade-en`). This document describes the usage of the commands.

> Hot patches include ACC (accumulate) and SGL (single) types.
>
> * ACC: A hot patch of the higher version fixes all problems that can be fixed by lower-version hot patches.
> * SGL\_xxx: A hot patch fixes the problems related to issue *xxx*. Multiple issue IDs are concatenated by underscores (\_).

## Hot Patch Scanning

`dnf hot-updateinfo` can scan hot patches and query hot patches for specified CVEs.

```shell
$ dnf hot-updateinfo list cves [--available(default) | --installed] [--cve [cve_id]] 
General DNF options:
  -h, --help, --help-cmd
                        show command help
  --cve CVES, --cves CVES
                        Include packages needed to fix the given CVE, in updates
Hot-updateinfo command-specific options:                     
  --available          
                        cves about newer versions of installed packages
                        (default)
  --installed
                        cves about equal and older versions of installed packages
```

* `list cves`

  1. Query the CVEs on the host that can be fixed and their related cold and hot patches.

     ```shell
     $ dnf hot-updateinfo list cves
     # cve-id   level    cold-patch   hot-patch
     Last metadata expiration check: 2:39:04 ago on Fri 29 Dec 2023 07:45:02.
     CVE-2022-30594  Important/Sec. kernel-4.19.90-2206.1.0.0153.oe1.x86_64                        patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64
     CVE-2023-1111   Important/Sec. redis-6.2.5-2.x86_64                                           patch-redis-6.2.5-1-ACC-1-1.x86_64
     CVE-2023-1112   Important/Sec. redis-6.2.5-2.x86_64                                           patch-redis-6.2.5-1-ACC-1-1.x86_64
     CVE-2023-1111   Important/Sec. redis-6.2.5-2.x86_64                                           patch-redis-6.2.5-1-SGL_CVE_2023_1111_CVE_2023_1112-1-1.x86_64
     ```

  2. Query hot and cold patches corresponding to fixed CVEs.

     ```shell
     $ dnf hot-updateinfo list cves --installed
     # cve-id   level    cold-patch   hot-patch
     Last metadata expiration check: 2:39:04 ago on Fri 29 Dec 2023 07:45:02.
     CVE-2022-36298  Important/Sec. -      patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_36298-1-1.x86_64
     ```

  3. Query hot and cold patches for specified CVEs.

     ```shell
     $ dnf hot-updateinfo list cves --cve CVE-2022-30594
     # cve-id   level    cold-patch   hot-patch
     Last metadata expiration check: 2:39:04 ago on Fri 29 Dec 2023 07:45:02.
     CVE-2022-30594 Important/Sec. kernel-4.19.90-2206.1.0.0153.oe1.x86_64       patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64
     ```

  4. An empty list will be displayed if the CVE does not exist.

     ```shell
     $ dnf hot-updateinfo list cves --cve CVE-2022-3089
     # cve-id   level    cold-patch   hot-patch
     Last metadata expiration check: 2:39:04 ago on Fri 29 Dec 2023 07:45:02.
     ```

## Hot Patch Statuses

* A hot patch can be in the following statuses:

  * NOT-APPLIED: The hot patch is not applied.

  * DEACTIVED: The hot patch is not activated.

  * ACTIVED: The hot patch is activated.

  * ACCEPT: The hot patch has been activated and will be applied after a reboot.

  ![Hot patch statuses](./figures/syscare_hot_patch_statuses.png)

## Querying and Changing Hot Patch Statuses

`dnf hotpatch` can be used to query and convert hot patch statuses.

```shell
$ dnf hotpatch 
General DNF options:
  -h, --help, --help-cmd
                        show command help
  --cve CVES, --cves CVES
                        Include packages needed to fix the given CVE, in updates

Hotpatch command-specific options:
  --list [{cve, cves}]  show list of hotpatch
  --apply APPLY_NAME apply hotpatch
  --remove REMOVE_NAME remove hotpatch
  --active ACTIVE_NAME active hotpatch
  --deactive DEACTIVE_NAME
                       deactive hotpatch
  --accept ACCEPT_NAME accept hotpatch
```

* Using `dnf hotpatch` to query hot patch statuses.

  * `dnf hotpatch --list` lists available hot patches in the system.

    ```shell
    $ dnf hotpatch --list
    Last metadata expiration check: 0:09:25 ago on Fri 29 Dec 2023 10:26:45.
    base-pkg/hotpatch                                               status
    kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux NOT-APPLIED
    ```

  * `dnf hotpatch --list cves` queries hot patches related to CVEs.

    ```shell
    $ dnf hotpatch --list cves
    Last metadata expiration check: 0:09:25 ago on Fri 29 Dec 2023 10:26:45.
    CVE-id         base-pkg/hotpatch                                               status
    CVE-2022-30594 kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux NOT-APPLIED
    ```

  * `dnf hotpatch --list cves --cve <CVE-id>` queries hot patches for specified CVEs.

    ```shell
    $ dnf hotpatch --list cves --cve CVE-2022-30594
    Last metadata expiration check: 0:09:25 ago on Fri 29 Dec 2023 10:26:45.
    CVE-id         base-pkg/hotpatch                                               status
    CVE-2022-30594 kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux NOT-APPLIED
    ```

  * An empty list will be displayed if the specified CVE does not exist when running `dnf hotpatch --list cves --cve <CVE-id>`.

    ```shell
    $ dnf hotpatch --list cves --cve CVE-2023-1
    Last metadata expiration check: 0:09:25 ago on Fri 29 Dec 2023 10:26:45.
    ```

* `dnf hotpatch --apply <patch name>` applies a hot patch. You can run `dnf hotpatch --list` to query the hot patch status after applying the hot patch. For details about hot patch statuses, see the previous section.

  ```shell
  $ dnf hotpatch --list
  Last metadata expiration check: 0:13:55 ago on Fri 29 Dec 2023 10:26:45.
  base-pkg/hotpatch                                               status
  kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux NOT-APPLIED
  $ dnf hotpatch --apply kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
  Last metadata expiration check: 0:15:37 ago on Fri 29 Dec 2023 10:26:45.
  Gonna apply this hot patch: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
  apply hot patch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' succeed
  $ dnf hotpatch --list
  Last metadata expiration check: 0:16:20 ago on Fri 29 Dec 2023 10:26:45.
  base-pkg/hotpatch                                               status
  kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux ACTIVED
  ```

* `dnf hotpatch --deactive <patch name>` deactivates a hot patch. You can run `dnf hotpatch --` to query the hot patch status after deactivating the hot patch. For details about hot patch statuses, see the previous section.

  ```shell
  $ dnf hotpatch --deactive kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
  Last metadata expiration check: 0:19:00 ago on Fri 29 Dec 2023 10:26:45.
  Gonna deactive this hot patch: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
  deactive hot patch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' succeed
  $ dnf hotpatch --list
  Last metadata expiration check: 0:19:12 ago on Fri 29 Dec 2023 10:26:45.
  base-pkg/hotpatch                                               status
  kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux DEACTIVED
  ```

* `dnf hotpatch --remove <patch name>` removes a hot patch. You can run `dnf hotpatch --list` to query the hot patch status after removing the hot patch. For details about hot patch statuses, see the previous section.

  ```shell
  $ dnf hotpatch --remove kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
  Last metadata expiration check: 0:20:12 ago on Fri 29 Dec 2023 10:26:45.
  Gonna remove this hot patch: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
  remove hot patch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' succeed
  $ dnf hotpatch --list
  Last metadata expiration check: 0:20:23 ago on Fri 29 Dec 2023 10:26:45.
  base-pkg/hotpatch                                               status
  kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux NOT-APPLIED
  ```

* `dnf hotpatch --active <patch name>` activating a hot patch.You can run `dnf hotpatch --list` to query the hot patch status after activating the hot patch. For details about hot patch statuses, see the previous section.

  ```shell
  $ dnf hotpatch --active kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
  Last metadata expiration check: 0:15:37 ago on Fri 29 Dec 2023 10:26:45.
  Gonna active this hot patch: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
  active hot patch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' succeed
  $ dnf hotpatch --list
  Last metadata expiration check: 0:16:20 ago on Fri 29 Dec 2023 10:26:45.
  base-pkg/hotpatch                                               status
  kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux ACTIVED
  ```

* `dnf hotpatch --accept <patch name>` accepts a hot patch. You can run `dnf hotpatch --list` to query the hot patch status after accepting the hot patch. For details about hot patch statuses, see the previous section.

  ```shell
  $ dnf hotpatch --accept kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
  Last metadata expiration check: 0:14:19 ago on Fri 29 Dec 2023 10:47:38.
  Gonna accept this hot patch: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
  accept hot patch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' succeed
  $ dnf hotpatch --list
  Last metadata expiration check: 0:14:34 ago on Fri 29 Dec 2023 10:47:38.
  base-pkg/hotpatch                                               status
  kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux ACCEPTED
  ```

## Applying Hot Patches

The `hotupgrade` command is used to apply hot patches to fix specified or all CVEs.

```shell
$ dnf hotupgrade [--cve [cve_id]] [PACKAGE ...] [--takeover] [-f]

General DNF options:
  -h, --help, --help-cmd
                        show command help
  --cve CVES, --cves CVES
                        Include packages needed to fix the given CVE, in updates

command-specific options:
  --takeover
                                            kernel cold patch takeover operation
  -f
                                            force retain kernel rpm package if kernel kabi check fails
  PACKAGE
                                            Package to upgrade
```

* Using `dnf hotupgrade PACKAGE` to install target hot patches.

  * Using `dnf hotupgrade PACKAGE` to install target hot patches.

  ```shell
  $ dnf hotupgrade patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64
  Last metadata expiration check: 0:26:25 ago on Fri 29 Dec 2023 10:47:38.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx(Install process)
  Complete!
  Apply hot patch succeed: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1.
  ```

  * Using `dnf hotupgrade PACKAGE` to install target hot patches when target hot patches have been activated.

  ```shell
  $ dnf hotupgrade patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64
  Last metadata expiration check: 0:28:35 ago on Fri 29 Dec 2023 10:47:38.
  The hotpatch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' already has a 'ACTIVED' sub hotpatch of binary file 'vmlinux'
  Package patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64 is already installed.
  Dependencies resolved.
  Nothing to do.
  Complete!
  ```

  * Using `dnf hotupgrade PACKAGE` to install target hot patches and automatically uninstall hot patches that fail to be activated.

  ```shell
  $ dnf hotupgrade patch-redis-6.2.5-1-ACC-1-1.x86_64
  Last metadata expiration check: 0:30:30 ago on Fri 29 Dec 2023 10:47:38.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx(Install process)
  Complete!
  Apply hot patch failed: redis-6.2.5-1/ACC-1-1.
  Error: Operation failed

  Caused by:
      1. Transaction "Apply patch 'redis-6.2.5-1/ACC-1-1'" failed

      Caused by:
          Cannot match any patch named "redis-6.2.5-1/ACC-1-1"

  Gonna remove unsuccessfully activated hotpatch rpm.
  Remove package succeed: patch-redis-6.2.5-1-ACC-1-1.x86_64.
  ```

* Using `--cve <cve_id>` to install hot patches for a CVE.

  * Using `--cve <cve_id>` to install hot patches for a CVE.

  ```shell
  $ dnf hotupgrade --cve CVE-2022-30594
  Last metadata expiration check: 0:26:25 ago on Fri 29 Dec 2023 10:47:38.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx(Install process)
  Complete!
  Apply hot patch succeed: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1.
  ```

  * Using `dnf hotupgrade --cve CVE-2022-2021` to install hot patches for the CVE, which does not exist.

  ```shell
  $ dnf hotupgrade --cve CVE-2022-2021
  Last metadata expiration check: 1:37:44 ago on Fri 29 Dec 2023 13:49:39.
  The cve doesn't exist or cannot be fixed by hotpatch: CVE-2022-2021
  No hot patches marked for install.
  Dependencies resolved.
  Nothing to do.
  Complete!
  ```

  * Using `dnf hotupgrade --cve <cve_id>`  to install and apply a hot patch of a higher version for a CVE that has an ACC hot patch of a lower version. The hot patch of the lower version is uninstalled.

  ```shell
  $ dnf hotupgrade --cve CVE-2023-1070
  Last metadata expiration check: 0:00:48 ago on Tue 02 Jan 2024 11:21:55.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx (Install messages and process upgrade)
  Complete!
  Apply hot patch succeed: kernel-5.10.0-153.12.0.92.oe2203sp2/ACC-1-3.
  $ 
  ```

  * Installing and applying a hot patch for a CVE that already has the latest hot patch.

  ```shell
  $ dnf hotupgrade --cve CVE-2023-1070
  Last metadata expiration check: 1:37:44 ago on Fri 29 Dec 2023 13:49:39.
  The cve doesn't exist or cannot be fixed by hotpatch: CVE-2023-1070
  No hot patches marked for install.
  Dependencies resolved.
  Nothing to do.
  Complete!
  ```

* Using `dnf hotupgrade` to install all hot patches.
  * When no hot patch is installed, running `dnf hotupgrade` will install all available hot patches.

  * When some of the hot patches are installed, running `dnf hotupgrade` will install the remaining hot patches.

* Using `--takeover` to take over kernel hot patches.

  * Using `dnf hotupgrade PACKAGE --takeover` to install hot patches and take over the related kernel hot patches. If a target kernel cold patch fails to pass the kabi check, it will be automatically uninstalled. The hot patches will be accepted and remain in effect after a reboot. The default kernel boot options will be restored.

  ```shell
  $ dnf hotupgrade patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64 --takeover
  Last metadata expiration check: 2:23:22 ago on Fri 29 Dec 2023 13:49:39.
  Gonna takeover kernel cold patch: ['kernel-4.19.90-2206.1.0.0153.oe1.x86_64']
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  xxxx(Install process)
  Complete!
  Apply hot patch succeed: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1.
  Kabi check for kernel-4.19.90-2206.1.0.0153.oe1.x86_64:
  [Fail] Here are 81 loaded kernel modules in this system, 78 pass, 3 fail.
  Failed modules are as follows:
  No. Module      Difference
  1   nf_nat_ipv6 secure_ipv6_port_ephemeral                : 0xe1a4f16a != 0x0209f3a7
  2   nf_nat_ipv4 secure_ipv4_port_ephemeral                : 0x57f70547 != 0xe3840e18
  3   kvm_intel   kvm_lapic_hv_timer_in_use                 : 0x54981db4 != 0xf58e6f1f
  Gonna remove kernel-4.19.90-2206.1.0.0153.oe1.x86_64 due to Kabi check failed.
  Rebuild rpm database succeed.
  Remove package succeed: kernel-4.19.90-2206.1.0.0153.oe1.x86_64.
  Restore the default boot kernel succeed: kernel-4.19.90-2112.8.0.0131.oe1.x86_64.
  No available kernel cold patch for takeover, gonna accept available kernel hot patch.
  Accept hot patch succeed: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1.
  ```

  * Using `dnf hotupgrade PACKAGE --takeover -f` to install hot patches. If a kernel cold patch fails to pass the kabi check, the `-f` option forcibly keeps the cold patch.

  ```shell
  $ dnf hotupgrade patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64 --takeover
  Last metadata expiration check: 2:23:22 ago on Fri 29 Dec 2023 13:49:39.
  Gonna takeover kernel cold patch: ['kernel-4.19.90-2206.1.0.0153.oe1.x86_64']
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  xxxx(Install process)
  Complete!
  Apply hot patch succeed: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1.
  Kabi check for kernel-4.19.90-2206.1.0.0153.oe1.x86_64:
  [Fail] Here are 81 loaded kernel modules in this system, 78 pass, 3 fail.
  Failed modules are as follows:
  No. Module      Difference
  1   nf_nat_ipv6 secure_ipv6_port_ephemeral                : 0xe1a4f16a != 0x0209f3a7
  2   nf_nat_ipv4 secure_ipv4_port_ephemeral                : 0x57f70547 != 0xe3840e18
  3   kvm_intel   kvm_lapic_hv_timer_in_use                 : 0x54981db4 != 0xf58e6f1f
  ```

## kabi Check before Kernel Upgrade

`dnf upgrade-en` supports the kabi check before kernel cold patch upgrade.

```shell
dnf upgrade-en [PACKAGE] [--cve [cve_id]] 

upgrade with KABI(Kernel Application Binary Interface) check. If the loaded
kernel modules have KABI compatibility with the new version kernel rpm, the
kernel modules can be installed and used in the new version kernel without
recompling.

General DNF options:
  -h, --help, --help-cmd
                        show command help
  --cve CVES, --cves CVES
                        Include packages needed to fix the given CVE, in updates
Upgrade-en command-specific options:                     
  PACKAGE
                                            Package to upgrade
```

* Using `dnf upgrade-en PACKAGE` to install target cold patches.

  * Using `dnf upgrade-en` to install target cold patches. If the kabi check is not passed, the kabi difference report will be generated, and the target kernel upgrade package will be uninstalled.

  ```shell
  $ dnf upgrade-en kernel-4.19.90-2206.1.0.0153.oe1.x86_64
  Last metadata expiration check: 1:51:54 ago on Fri 29 Dec 2023 13:49:39.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx(Install process)                                                                                       
  Complete!
  Kabi check for kernel-4.19.90-2206.1.0.0153.oe1.x86_64:
  [Fail] Here are 81 loaded kernel modules in this system, 78 pass, 3 fail.
  Failed modules are as follows:
  No. Module      Difference
  1   nf_nat_ipv6 secure_ipv6_port_ephemeral                : 0xe1a4f16a != 0x0209f3a7
  2   nf_nat_ipv4 secure_ipv4_port_ephemeral                : 0x57f70547 != 0xe3840e18
  3   kvm_intel   kvm_lapic_hv_timer_in_use                 : 0x54981db4 != 0xf58e6f1f
                  kvm_apic_write_nodecode                   : 0x56c989a1 != 0x24c9db31
                  kvm_complete_insn_gp                      : 0x99c2d256 != 0xcd8014bd
  Gonna remove kernel-4.19.90-2206.1.0.0153.oe1.x86_64 due to kabi check failed.
  Rebuild rpm database succeed.
  Remove package succeed: kernel-4.19.90-2206.1.0.0153.oe1.x86_64.
  Restore the default boot kernel succeed: kernel-4.19.90-2112.8.0.0131.oe1.x86_64.
  ```

  * Using `dnf upgrade-en` to install target cold patches and the kabi check is passed.

  ```shell
  $ dnf upgrade-en kernel-4.19.90-2201.1.0.0132.oe1.x86_64
  Last metadata expiration check: 2:02:10 ago on Fri 29 Dec 2023 13:49:39.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx(Install process)  
  Complete!
  Kabi check for kernel-4.19.90-2201.1.0.0132.oe1.x86_64:
  [Success] Here are 81 loaded kernel modules in this system, 81 pass, 0 fail.
  ```

* Using `dnf upgrade-en` to install all cold patches.

  If the target kernel upgrade is included in the cold patches, the output is the same as `dnf upgrade-en PACKAGE` according to the kabi check result.

## Usage Example

Assume that the repositories of hot and cold patches on this host have been enabled.

* Hot patches

Scan CVEs that can be fixed on the host.

```shell
$ dnf hot-updateinfo list cves
Last metadata expiration check: 0:00:38 ago on Sat 25 Mar 2023 11:53:46.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-1     Important/Sec. redis-6.2.5-2.x86_64                                   patch-redis-6.2.5-1-ACC-1-1.x86_64
CVE-2021-11    Important/Sec. redis-6.2.5-2.x86_64                                   patch-redis-6.2.5-1-ACC-1-1.x86_64
CVE-2021-2     Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-22    Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

CVE-2021-1, CVE-2021-11, CVE-2021-2, and CVE-2021-22 can be fixed by hot patches.

Start the Redis service based on the **redis.conf** configuration file.

````shell
$ sudo redis-server ./redis.conf &
[1] 285075
$ 285076:C 25 Mar 2023 12:09:51.503 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
285076:C 25 Mar 2023 12:09:51.503 # Redis version=255.255.255, bits=64, commit=00000000, modified=0, pid=285076, just started
285076:C 25 Mar 2023 12:09:51.503 # Configuration loaded
285076:M 25 Mar 2023 12:09:51.504 * Increased maximum number of open files to 10032 (it was originally set to 1024).
285076:M 25 Mar 2023 12:09:51.504 * monotonic clock: POSIX clock_gettime
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 255.255.255 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                  
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6380
 |    `-._   `._    /     _.-'    |     PID: 285076
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           https://redis.io       
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

285076:M 25 Mar 2023 12:09:51.505 # Server initialized
285076:M 25 Mar 2023 12:09:51.505 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
285076:M 25 Mar 2023 12:09:51.506 * Ready to accept connections

````

Test the function before applying the hot patch.

```shell
$ telnet 127.0.0.1 6380
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

*100

-ERR Protocol error: expected '$', got ' '
Connection closed by foreign host.
```

Specify CVE-2021-1 and ensure that the related hot patch is associated and applied.

```shell
$ dnf hotupgrade patch-redis-6.2.5-1-ACC-1-1.x86_64
Last metadata expiration check: 0:01:39 ago on Tue 02 Jan 2024 20:16:45.
The hotpatch 'redis-6.2.5-1/ACC-1-1' already has a 'ACTIVED' sub hotpatch of binary file 'redis-benchmark'
The hotpatch 'redis-6.2.5-1/ACC-1-1' already has a 'ACTIVED' sub hotpatch of binary file 'redis-cli'
The hotpatch 'redis-6.2.5-1/ACC-1-1' already has a 'ACTIVED' sub hotpatch of binary file 'redis-server'
Package patch-redis-6.2.5-1-ACC-1-1.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
```

Run `dnf hotpatch --list` to check whether the hot patch has been applied (the status is **ACTIVED**).

```shell
$ dnf hotpatch --list
Last metadata expiration check: 0:04:43 ago on Tue 02 Jan 2024 20:16:45.
base-pkg/hotpatch                                   status
redis-6.2.5-1/ACC-1-1/redis-benchmark               ACTIVED
redis-6.2.5-1/ACC-1-1/redis-cli                     ACTIVED
redis-6.2.5-1/ACC-1-1/redis-server                  ACTIVED
```

Check whether the CVE has been fixed. Because the **patch-redis-6.2.5-1-ACC-1-1.x86\_64** hot patch also fixes CVE-2021-11, CVE-2021-1 and CVE-2021-11 no longer exists.

```shell
$ dnf hot-updateinfo list cves
Last metadata expiration check: 0:08:48 ago on Sat 25 Mar 2023 11:53:46.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-1076  Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26607 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-22    Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-2     Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

Test the function after applying the hot patch.

```shell
$ telnet 127.0.0.1 6380
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

*100

-ERR Protocol error: unauthenticated multibulk length
Connection closed by foreign host.
```

Run `dnf hotpatch --remove` and specify the patch name to manually remove the hot patch.

```shell
$ dnf hotpatch --remove redis-6.2.5-1
Last metadata expiration check: 0:11:52 ago on Tue 02 Jan 2024 20:16:45.
Gonna remove this hot patch: redis-6.2.5-1
remove hot patch 'redis-6.2.5-1' succeed
$ dnf hotpatch --list
Last metadata expiration check: 0:12:00 ago on Tue 02 Jan 2024 20:16:45.
base-pkg/hotpatch                                   status
redis-6.2.5-1/ACC-1-1/redis-benchmark               NOT-APPLIED
redis-6.2.5-1/ACC-1-1/redis-cli                     NOT-APPLIED
redis-6.2.5-1/ACC-1-1/redis-server                  NOT-APPLIED
```

Scan the CVEs to be fixed on the host. CVE-2021-1 and CVE-2021-11 are displayed.

```shell
$ dnf hot-updateinfo list cves
Last metadata expiration check: 0:00:38 ago on Sat 25 Mar 2023 11:53:46.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-1     Important/Sec. redis-6.2.5-2.x86_64                                   patch-redis-6.2.5-1-ACC-1-1.x86_64
CVE-2021-11    Important/Sec. redis-6.2.5-2.x86_64                                   patch-redis-6.2.5-1-ACC-1-1.x86_64
CVE-2021-2     Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-22    Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

* installing an ACC patch of a higher version.

Apply hot patch **patch-redis-6.2.5-1-HP002-1-1.x86\_64**.

```shell
$ dnf hotupgrade patch-redis-6.2.5-1-ACC-1-2.x86_64
Last metadata expiration check: 0:36:12 ago on Tue 02 Jan 2024 20:16:45.
The hotpatch 'redis-6.2.5-1/ACC-1-2' already has a 'ACTIVED' sub hotpatch of binary file 'redis-benchmark'
The hotpatch 'redis-6.2.5-1/ACC-1-2' already has a 'ACTIVED' sub hotpatch of binary file 'redis-cli'
The hotpatch 'redis-6.2.5-1/ACC-1-2' already has a 'ACTIVED' sub hotpatch of binary file 'redis-server'
Package patch-redis-6.2.5-1-ACC-1-2.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
```

Scan the CVEs to be fixed on the host. Because **patch-redis-6.2.5-1-ACC-1-2.x86\_64** is of a higher version than **patch-redis-6.2.5-1-ACC-1-1.x86\_64**, **patch-redis-6.2.5-1-ACC-1-2.x86\_64** also fixes CVE-2021-1, CVE-2021-11, CVE-2021-2, and CVE-2021-22.

```shell
$ dnf hot-updateinfo list cves
Last metadata expiration check: 0:00:38 ago on Sat Mar 25 11:53:46 2023.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

* Version of the software package fixed by the hot patch higher than that of the installed one.

Open the **xxx-updateinfo.xml.gz** file in the **repodata** directory of the hot patch repository. Check the information related to CVE-2021-33 and CVE-2021-3.

```xml
<update from="openeuler.org" type="security" status="stable">
          <id>openEuler-HotPatchSA-2023-3</id>
          <title>An update for mariadb is now available for openEuler-{version}</title>
          <severity>Important</severity>
          <release>openEuler</release>
          <issued date="2022-04-16"></issued>
          <references>
                  <reference href="https://nvd.nist.gov/vuln/detail/CVE-2021-3" id="CVE-2021-3" title="CVE-2021-3" type="cve"></reference>
                  <reference href="https://nvd.nist.gov/vuln/detail/CVE-2021-33" id="CVE-2021-33" title="CVE-2021-33" type="cve"></reference>
          </references>
          <description>patch-redis-6.2.5-2-ACC.(CVE-2021-3, CVE-2021-33)</description>
          <pkglist>
               <hot_patch_collection>
                    <name>openEuler</name>
                    <package arch="aarch64" name="patch-redis-6.2.5-2-ACC" release="1" version="1">
                         <filename>patch-redis-6.2.5-2-ACC-1-1.aarch64.rpm</filename>
                    </package>
                    <package arch="x86_64" name="patch-redis-6.2.5-2-ACC" release="1" version="1">
                         <filename>patch-redis-6.2.5-2-ACC-1-1.x86_64.rpm</filename>
                    </package>
               </hot_patch_collection>
          </pkglist>
  </update>
```

The format of the **name** field of **package** (**patch-redis-6.2.5-2-ACC**) is **patch-\<source package name>-\<source package version>-\<source package release>-\<hot patch name>**. In the example, **patch-redis-6.2.5-2-ACC** requires the source code version of redis-6.2.5-2 to be installed. Check the version of Redis on the host.

```shell
$ rpm -qa | grep redis
redis-6.2.5-1.x86_64
```

The installed Redis version is lower than 6.2.5-2. Therefore, the hot patch will not be displayed.

```shell
$ dnf hot-updateinfo list cves
Last metadata expiration check: 0:00:38 ago on Sat 25 Mar 2023 11:53:46.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

* Version of the software package fixed by the hot patch lower than that of the installed one.

Open the **xxx-updateinfo.xml.gz** file in the **repodata** directory of the hot patch repository. Check the information related to CVE-2021-44 and CVE-2021-4.

```xml
<update from="openeuler.org" type="security" status="stable">
          <id>openEuler-HotPatchSA-2023-4</id>
          <title>An update for mariadb is now available for openEuler-{version}</title>
          <severity>Important</severity>
          <release>openEuler</release>
          <issued date="2022-04-16"></issued>
          <references>
                  <reference href="https://nvd.nist.gov/vuln/detail/CVE-2021-4" id="CVE-2021-4" title="CVE-2021-4" type="cve"></reference>
                  <reference href="https://nvd.nist.gov/vuln/detail/CVE-2021-44" id="CVE-2021-44" title="CVE-2021-44" type="cve"></reference>
          </references>
          <description>patch-redis-6.2.4-1-ACC.(CVE-2021-44, CVE-2021-4)</description>
          <pkglist>
               <hot_patch_collection>
                    <name>openEuler</name>
                    <package arch="aarch64" name="patch-redis-6.2.4-1-ACC" release="1" version="1">
                         <filename>patch-redis-6.2.4-1-ACC-1-1.aarch64.rpm</filename>
                    </package>
                    <package arch="x86_64" name="patch-redis-6.2.4-1-ACC" release="1" version="1">
                         <filename>patch-redis-6.2.4-1-ACC-1-1.x86_64.rpm</filename>
                    </package>
               </hot_patch_collection>
          </pkglist>
  </update>
```

The format of the **name** field of **package** (**patch-redis-6.2.4-1-ACC**) is **patch-\<source package name>-\<source package version>-\<source package release>-\<hot patch name>**. In the example, **patch-redis-6.2.4-1-ACC** requires the source code version of redis-6.2.4-1 to be installed. Check the version of Redis on the host.

```shell
$ rpm -qa | grep redis
redis-6.2.5-1.x86_64
```

The installed Redis version is higher than 6.2.4-1. Therefore, the CVE will not be displayed.

```shell
$ dnf hot-updateinfo list cves
Last metadata expiration check: 0:00:38 ago on Sat 25 Mar 2023 11:53:46.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/aops/dnf_command_usage.md
---
# dnf插件命令使用手册

将dnf-hotpatch-plugin安装部署完成后，可使用dnf命令调用A-ops ceres中的冷/热补丁操作，命令包含热补丁扫描（dnf hot-updateinfo），热补丁状态设置及查询（dnf hotpatch ），热补丁应用（dnf hotupgrade），内核升级前kabi检查（dnf upgrade-en）。本文将介绍上述命令的具体使用方法。

> 热补丁包括ACC/SGL（accumulate/single）类型
>
> * ACC：增量补丁。目标高版本热补丁包含低版本热补丁所修复问题。
> * SGL\_xxx：单独补丁，xxx为issue id，如果有多个issue id，用多个'\_'拼接。目标修复issue id相关问题。

## 热补丁扫描

`dnf hot-updateinfo`命令支持扫描热补丁并指定cve查询相关热补丁，命令使用方式如下：

```shell
dnf hot-updateinfo list cves [--available(default) | --installed] [--cve [cve_id]] 

General DNF options:
  -h, --help, --help-cmd
                        show command help
  --cve CVES, --cves CVES
                        Include packages needed to fix the given CVE, in updates
Hot-updateinfo command-specific options:                     
  --available          
                        cves about newer versions of installed packages
                        (default)
  --installed
                        cves about equal and older versions of installed packages
```

* `list cves`

1、查询主机所有可修复的cve和对应的冷/热补丁。

```shell
[root@localhost ~]# dnf hot-updateinfo list cves
# cve-id   level    cold-patch   hot-patch
Last metadata expiration check: 2:39:04 ago on 2023年12月29日 星期五 07时45分02秒.
CVE-2022-30594  Important/Sec. kernel-4.19.90-2206.1.0.0153.oe1.x86_64                        patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64
CVE-2023-1111   Important/Sec. redis-6.2.5-2.x86_64                                           patch-redis-6.2.5-1-ACC-1-1.x86_64
CVE-2023-1112   Important/Sec. redis-6.2.5-2.x86_64                                           patch-redis-6.2.5-1-ACC-1-1.x86_64
CVE-2023-1111   Important/Sec. redis-6.2.5-2.x86_64                                           patch-redis-6.2.5-1-SGL_CVE_2023_1111_CVE_2023_1112-1-1.x86_64
```

2、查询主机所有已修复的cve和对应的冷/热补丁

```shell
[root@localhost ~]# dnf hot-updateinfo list cves --installed
# cve-id   level    cold-patch   hot-patch
Last metadata expiration check: 2:39:04 ago on 2023年12月29日 星期五 07时45分02秒.
CVE-2022-36298  Important/Sec. -      patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_36298-1-1.x86_64
```

2、指定cve查询对应的可修复冷/热补丁。

```shell
[root@localhost ~]# dnf hot-updateinfo list cves --cve CVE-2022-30594
# cve-id   level    cold-patch   hot-patch
Last metadata expiration check: 2:39:04 ago on 2023年12月29日 星期五 07时45分02秒.
CVE-2022-30594 Important/Sec. kernel-4.19.90-2206.1.0.0153.oe1.x86_64       patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64
```

3、cve不存在时列表为空。

```shell
[root@localhost ~]# dnf hot-updateinfo list cves --cve CVE-2022-3089
# cve-id   level    cold-patch   hot-patch
Last metadata expiration check: 2:39:04 ago on 2023年12月29日 星期五 07时45分02秒.
```

## 热补丁状态及转换图

* 热补丁状态图

  NOT-APPLIED: 热补丁尚未应用。

  DEACTIVED: 热补丁未被激活。

  ACTIVED: 热补丁已被激活。

  ACCEPTED: 热补丁已被激活，后续重启后会被自动应用激活。

  ![热补丁状态转换图](./figures/syscare热补丁状态图.png)

## 热补丁状态查询和切换

`dnf hotpatch`命令支持查询、切换热补丁的状态，命令使用方式如下：

```shell
dnf hotpatch 

General DNF options:
  -h, --help, --help-cmd
                        show command help
  --cve CVES, --cves CVES
                        Include packages needed to fix the given CVE, in updates

Hotpatch command-specific options:
  --list [{cve, cves}]  show list of hotpatch
  --apply APPLY_NAME apply hotpatch
  --remove REMOVE_NAME remove hotpatch
  --active ACTIVE_NAME active hotpatch
  --deactive DEACTIVE_NAME
                       deactive hotpatch
  --accept ACCEPT_NAME accept hotpatch
```

* 使用`dnf hotpatch`命令查询热补丁状态

  * 使用`dnf hotpatch --list`命令查询当前系统中可使用的热补丁状态并展示。

  ```shell
  [root@localhost ~]# dnf hotpatch --list
  Last metadata expiration check: 0:09:25 ago on 2023年12月29日 星期五 10时26分45秒.
  base-pkg/hotpatch                                               status
  kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux NOT-APPLIED
  ```

  * 使用`dnf hotpatch --list cves`查询漏洞（CVE-id）对应热补丁及其状态并展示。

  ```shell
  [root@openEuler ~]# dnf hotpatch --list cves
  Last metadata expiration check: 0:11:05 ago on 2023年12月29日 星期五 10时26分45秒.
  CVE-id         base-pkg/hotpatch                                               status
  CVE-2022-30594 kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux NOT-APPLIED
  ```

  * `dnf hotpatch --list cves --cve <CVE-id>`筛选指定CVE对应的热补丁及其状态并展示。

  ```shell
  [root@openEuler ~]# dnf hotpatch --list cves --cve CVE-2022-30594
  Last metadata expiration check: 0:12:25 ago on 2023年12月29日 星期五 10时26分45秒.
  CVE-id         base-pkg/hotpatch                                               status
  CVE-2022-30594 kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux NOT-APPLIED
  ```

  * 使用`dnf hotpatch --list cves --cve <CVE-id>`查询无结果时展示为空。

  ```shell
  [root@openEuler ~]# dnf hotpatch --list cves --cve CVE-2023-1
  Last metadata expiration check: 0:13:11 ago on 2023年12月29日 星期五 10时26分45秒.
  ```

* 使用`dnf hotpatch --apply <patch name>`命令应用热补丁，可使用 `dnf hotpatch --list`查询应用后的状态变化，变化逻辑见上文的热补丁状态转换图。

```shell
[root@openEuler ~]# dnf hotpatch --list
Last metadata expiration check: 0:13:55 ago on 2023年12月29日 星期五 10时26分45秒.
base-pkg/hotpatch                                               status
kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux NOT-APPLIED
[root@openEuler ~]# dnf hotpatch --apply kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
Last metadata expiration check: 0:15:37 ago on 2023年12月29日 星期五 10时26分45秒.
Gonna apply this hot patch: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
apply hot patch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' succeed
[root@openEuler ~]# dnf hotpatch --list
Last metadata expiration check: 0:16:20 ago on 2023年12月29日 星期五 10时26分45秒.
base-pkg/hotpatch                                               status
kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux ACTIVED
```

* 使用`dnf hotpatch --deactive <patch name>`停用热补丁，可使用`dnf hotpatch --list`查询停用后的状态变化，变化逻辑见上文的热补丁状态转换图。

```shell
[root@openEuler ~]# dnf hotpatch --deactive kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
Last metadata expiration check: 0:19:00 ago on 2023年12月29日 星期五 10时26分45秒.
Gonna deactive this hot patch: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
deactive hot patch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' succeed
[root@openEuler ~]# dnf hotpatch --list
Last metadata expiration check: 0:19:12 ago on 2023年12月29日 星期五 10时26分45秒.
base-pkg/hotpatch                                               status
kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux DEACTIVED
```

* 使用`dnf hotpatch --remove <patch name>`删除热补丁，可使用`dnf hotpatch --list`查询删除后的状态变化，变化逻辑见上文的热补丁状态转换图。

```shell
[root@openEuler ~]# dnf hotpatch --remove kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
Last metadata expiration check: 0:20:12 ago on 2023年12月29日 星期五 10时26分45秒.
Gonna remove this hot patch: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
remove hot patch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' succeed
[root@openEuler ~]# dnf hotpatch --list
Last metadata expiration check: 0:20:23 ago on 2023年12月29日 星期五 10时26分45秒.
base-pkg/hotpatch                                               status
kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux NOT-APPLIED
```

* 使用`dnf hotpatch --active <patch name>`激活热补丁，可使用`dnf hotpatch --list`查询激活后的状态变化，变化逻辑见上文的热补丁状态转换图。

```shell
[root@openEuler ~]# dnf hotpatch --active kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
Last metadata expiration check: 0:15:37 ago on 2023年12月29日 星期五 10时26分45秒.
Gonna active this hot patch: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
active hot patch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' succeed
[root@openEuler ~]# dnf hotpatch --list
Last metadata expiration check: 0:16:20 ago on 2023年12月29日 星期五 10时26分45秒.
base-pkg/hotpatch                                               status
kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux ACTIVED
```

* 使用`dnf hotpatch --accept <patch name>`接收热补丁，可使用`dnf hotpatch --list`查询接收后的状态变化，变化逻辑见上文的热补丁状态转换图。

```shell
[root@openEuler ~]# dnf hotpatch --accept kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
Last metadata expiration check: 0:14:19 ago on 2023年12月29日 星期五 10时47分38秒.
Gonna accept this hot patch: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1
accept hot patch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' succeed
[root@openEuler ~]# dnf hotpatch --list
Last metadata expiration check: 0:14:34 ago on 2023年12月29日 星期五 10时47分38秒.
base-pkg/hotpatch                                               status
kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1/vmlinux ACCEPTED
```

## 热补丁应用

`hotupgrade`命令根据cve id和热补丁名称进行热补丁修复，同时也支持全量修复。命令使用方式如下：

```shell
dnf hotupgrade [--cve [cve_id]] [PACKAGE ...] [--takeover] [-f]

General DNF options:
  -h, --help, --help-cmd
                        show command help
  --cve CVES, --cves CVES
                        Include packages needed to fix the given CVE, in updates
 
command-specific options:
  --takeover            
                        kernel cold patch takeover operation
  -f
                        force retain kernel rpm package if kernel kabi check fails
  PACKAGE               
                        Package to upgrade
```

* 使用`dnf hotupgrade PACKAGE`安装目标热补丁。

  * 使用`dnf hotupgrade PACKAGE`安装目标热补丁

  ```shell
  [root@openEuler ~]# dnf hotupgrade patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64
  Last metadata expiration check: 0:26:25 ago on 2023年12月29日 星期五 10时47分38秒.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx(Install process)
  Complete!
  Apply hot patch succeed: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1.
  ```

  * 当目标热补丁已经应用激活，使用`dnf hotupgrade PACKAGE`安装目标热补丁

  ```shell
  [root@openEuler ~]# dnf hotupgrade patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64
  Last metadata expiration check: 0:28:35 ago on 2023年12月29日 星期五 10时47分38秒.
  The hotpatch 'kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1' already has a 'ACTIVED' sub hotpatch of binary file 'vmlinux'
  Package patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64 is already installed.
  Dependencies resolved.
  Nothing to do.
  Complete!
  ```

  * 使用`dnf hotupgrade PACKAGE`安装目标热补丁，自动卸载激活失败的热补丁。

  ```shell
  [root@openEuler ~]# dnf hotupgrade patch-redis-6.2.5-1-ACC-1-1.x86_64
  Last metadata expiration check: 0:30:30 ago on 2023年12月29日 星期五 10时47分38秒.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx(Install process)
  Complete!
  Apply hot patch failed: redis-6.2.5-1/ACC-1-1.
  Error: Operation failed

  Caused by:
      0. Transaction "Apply patch 'redis-6.2.5-1/ACC-1-1'" failed
      
      Caused by:
          Cannot match any patch named "redis-6.2.5-1/ACC-1-1"

  Gonna remove unsuccessfully activated hotpatch rpm.
  Remove package succeed: patch-redis-6.2.5-1-ACC-1-1.x86_64.
  ```

* 使用`--cve <cve_id>`指定cve\_id安装CVE对应的热补丁

  * 使用`dnf hotupgrade --cve CVE-2022-30594`安装CVE对应的热补丁

  ```shell
  [root@openEuler ~]# dnf hotupgrade --cve CVE-2022-30594
  Last metadata expiration check: 0:26:25 ago on 2023年12月29日 星期五 10时47分38秒.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx(Install process)
  Complete!
  Apply hot patch succeed: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1.
  ```

  * 使用`dnf hotupgrade --cve CVE-2022-2021`安装CVE对应的热补丁，对应的CVE不存在。

  ```shell
  [root@openEuler ~]# dnf hotupgrade --cve CVE-2022-2021
  Last metadata expiration check: 1:37:44 ago on 2023年12月29日 星期五 13时49分39秒.
  The cve doesn't exist or cannot be fixed by hotpatch: CVE-2022-2021
  No hot patches marked for install.
  Dependencies resolved.
  Nothing to do.
  Complete!
  ```

  * 使用`dnf hotupgrade --cve <cve_id>`指定cve\_id安装时，该CVE对应的ACC低版本热补丁已安装时，删除低版本热补丁，安装高版本ACC热补丁包。

  ```shell
  [root@openEuler ~]# dnf hotupgrade --cve CVE-2023-1070
  Last metadata expiration check: 0:00:48 ago on 2024年01月02日 星期二 11时21分55秒.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx (Install messages and process upgrade)
  Complete!
  Apply hot patch succeed: kernel-5.10.0-153.12.0.92.oe2203sp2/ACC-1-3.
  [root@openEuler tmp]# 
  ```

  * 指定cve\_id安装时，该CVE对应的最高版本热补丁包已存在

  ```shell
  [root@openEuler ~]# dnf hotupgrade --cve CVE-2023-1070
  Last metadata expiration check: 1:37:44 ago on 2023年12月29日 星期五 13时49分39秒.
  The cve doesn't exist or cannot be fixed by hotpatch: CVE-2023-1070
  No hot patches marked for install.
  Dependencies resolved.
  Nothing to do.
  Complete!
  ```

* 使用`dnf hotupgrade`进行热补丁全量修复

  * 热补丁未安装时，使用`dnf hotupgrade`命令安装所有可安装热补丁。
  * 当部分热补丁已经安装时，使用`dnf hotupgrade`命令进行全量修复，将保留已安装的热补丁，然后安装其他热补丁

* 使用`--takeover`进行内核热补丁收编

  * 使用`dnf hotupgrade PACKAGE --takeover`安装热补丁，收编相应内核冷补丁；由于目标内核冷补丁kabi检查失败，进行自动卸载；accept热补丁，使热补丁重启后仍旧生效；恢复内核默认引导启动项。

  ```shell
  [root@openEuler ~]# dnf hotupgrade patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64 --takeover
  Last metadata expiration check: 2:23:22 ago on 2023年12月29日 星期五 13时49分39秒.
  Gonna takeover kernel cold patch: ['kernel-4.19.90-2206.1.0.0153.oe1.x86_64']
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  xxxx(Install process)
  Complete!
  Apply hot patch succeed: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1.
  Kabi check for kernel-4.19.90-2206.1.0.0153.oe1.x86_64:
  [Fail] Here are 81 loaded kernel modules in this system, 78 pass, 3 fail.
  Failed modules are as follows:
  No. Module      Difference
  1   nf_nat_ipv6 secure_ipv6_port_ephemeral                : 0xe1a4f16a != 0x0209f3a7
  2   nf_nat_ipv4 secure_ipv4_port_ephemeral                : 0x57f70547 != 0xe3840e18
  3   kvm_intel   kvm_lapic_hv_timer_in_use                 : 0x54981db4 != 0xf58e6f1f
  Gonna remove kernel-4.19.90-2206.1.0.0153.oe1.x86_64 due to Kabi check failed.
  Rebuild rpm database succeed.
  Remove package succeed: kernel-4.19.90-2206.1.0.0153.oe1.x86_64.
  Restore the default boot kernel succeed: kernel-4.19.90-2112.8.0.0131.oe1.x86_64.
  No available kernel cold patch for takeover, gonna accept available kernel hot patch.
  Accept hot patch succeed: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1.
  ```

  * 使用`dnf hotupgrade PACKAGE --takeover -f`安装热补丁，如果内核冷补丁kabi检查未通过，使用`-f`强制保留内核冷补丁

  ```shell
  [root@openEuler ~]# dnf hotupgrade patch-kernel-4.19.90-2112.8.0.0131.oe1-SGL_CVE_2022_30594-1-1.x86_64 --takeover
  Last metadata expiration check: 2:23:22 ago on 2023年12月29日 星期五 13时49分39秒.
  Gonna takeover kernel cold patch: ['kernel-4.19.90-2206.1.0.0153.oe1.x86_64']
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  xxxx(Install process)
  Complete!
  Apply hot patch succeed: kernel-4.19.90-2112.8.0.0131.oe1/SGL_CVE_2022_30594-1-1.
  Kabi check for kernel-4.19.90-2206.1.0.0153.oe1.x86_64:
  [Fail] Here are 81 loaded kernel modules in this system, 78 pass, 3 fail.
  Failed modules are as follows:
  No. Module      Difference
  1   nf_nat_ipv6 secure_ipv6_port_ephemeral                : 0xe1a4f16a != 0x0209f3a7
  2   nf_nat_ipv4 secure_ipv4_port_ephemeral                : 0x57f70547 != 0xe3840e18
  3   kvm_intel   kvm_lapic_hv_timer_in_use                 : 0x54981db4 != 0xf58e6f1f
  ```

## 内核升级前kabi检查

`dnf upgrade-en` 命令支持内核冷补丁升级前kabi检查，命令使用方式如下：

```shell
dnf upgrade-en [PACKAGE] [--cve [cve_id]] 

upgrade with KABI(Kernel Application Binary Interface) check. If the loaded
kernel modules have KABI compatibility with the new version kernel rpm, the
kernel modules can be installed and used in the new version kernel without
recompling.

General DNF options:
  -h, --help, --help-cmd
                        show command help
  --cve CVES, --cves CVES
                        Include packages needed to fix the given CVE, in updates
Upgrade-en command-specific options:                     
  PACKAGE
                        Package to upgrade
```

* 使用`dnf upgrade-en PACKAGE`安装目标冷补丁

  * 使用`dnf upgrade-en`安装目标冷补丁，kabi检查未通过，输出kabi差异性报告，自动卸载目标升级kernel包。

  ```shell
  [root@openEuler ~]# dnf upgrade-en kernel-4.19.90-2206.1.0.0153.oe1.x86_64
  Last metadata expiration check: 1:51:54 ago on 2023年12月29日 星期五 13时49分39秒.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx(Install process)                                                                                       
  Complete!
  Kabi check for kernel-4.19.90-2206.1.0.0153.oe1.x86_64:
  [Fail] Here are 81 loaded kernel modules in this system, 78 pass, 3 fail.
  Failed modules are as follows:
  No. Module      Difference
  1   nf_nat_ipv6 secure_ipv6_port_ephemeral                : 0xe1a4f16a != 0x0209f3a7
  2   nf_nat_ipv4 secure_ipv4_port_ephemeral                : 0x57f70547 != 0xe3840e18
  3   kvm_intel   kvm_lapic_hv_timer_in_use                 : 0x54981db4 != 0xf58e6f1f
                  kvm_apic_write_nodecode                   : 0x56c989a1 != 0x24c9db31
                  kvm_complete_insn_gp                      : 0x99c2d256 != 0xcd8014bd
  Gonna remove kernel-4.19.90-2206.1.0.0153.oe1.x86_64 due to kabi check failed.
  Rebuild rpm database succeed.
  Remove package succeed: kernel-4.19.90-2206.1.0.0153.oe1.x86_64.
  Restore the default boot kernel succeed: kernel-4.19.90-2112.8.0.0131.oe1.x86_64.
  ```

  * 使用`dnf upgrade-en`安装目标冷补丁，kabi检查通过

  ```shell
  [root@openEuler ~]# dnf upgrade-en kernel-4.19.90-2201.1.0.0132.oe1.x86_64
  Last metadata expiration check: 2:02:10 ago on 2023年12月29日 星期五 13时49分39秒.
  Dependencies resolved.
  xxxx(Install messgaes)
  Is this ok [y/N]: y
  Downloading Packages:
  xxxx(Install process)  
  Complete!
  Kabi check for kernel-4.19.90-2201.1.0.0132.oe1.x86_64:
  [Success] Here are 81 loaded kernel modules in this system, 81 pass, 0 fail.
  ```

* 使用`dnf upgrade-en` 进行全量修复

​全量修复如果包含目标kernel的升级，输出根据不同的kabi检查情况与`dnf upgrade-en PACKAGE`命令相同。

## 使用场景说明

本段落介绍上述命令的使用场景及顺序介绍，需要提前确认本机的热补丁repo源和相应冷补丁repo源已开启。

* 热补丁修复。

使用热补丁扫描命令查看本机待修复cve。

```shell
[root@openEuler ~]# dnf hot-updateinfo list cves
Last metadata expiration check: 0:00:38 ago on 2023年03月25日 星期六 11时53分46秒.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-1     Important/Sec. redis-6.2.5-2.x86_64                                   patch-redis-6.2.5-1-ACC-1-1.x86_64
CVE-2021-11    Important/Sec. redis-6.2.5-2.x86_64                                   patch-redis-6.2.5-1-ACC-1-1.x86_64
CVE-2021-2     Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-22    Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

找到提供热补丁的相应cve，发现CVE-2021-1、CVE-2021-11、CVE-2021-2和CVE-2021-22可用热补丁修复。

在安装补丁前测试功能，基于redis.conf配置文件启动redis服务。

````shell
[root@openEuler ~]# sudo redis-server ./redis.conf &
[1] 285075
[root@openEuler ~]# 285076:C 25 Mar 2023 12:09:51.503 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
285076:C 25 Mar 2023 12:09:51.503 # Redis version=255.255.255, bits=64, commit=00000000, modified=0, pid=285076, just started
285076:C 25 Mar 2023 12:09:51.503 # Configuration loaded
285076:M 25 Mar 2023 12:09:51.504 * Increased maximum number of open files to 10032 (it was originally set to 1024).
285076:M 25 Mar 2023 12:09:51.504 * monotonic clock: POSIX clock_gettime
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 255.255.255 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                  
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6380
 |    `-._   `._    /     _.-'    |     PID: 285076
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           https://redis.io       
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

285076:M 25 Mar 2023 12:09:51.505 # Server initialized
285076:M 25 Mar 2023 12:09:51.505 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
285076:M 25 Mar 2023 12:09:51.506 * Ready to accept connections

````

安装前测试功能。

```shell
[root@openEuler ~]# telnet 127.0.0.1 6380
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

*100

-ERR Protocol error: expected '$', got ' '
Connection closed by foreign host.
```

指定修复CVE-2021-1，确认关联到对应的热补丁包，显示安装成功。

```shell
[root@openEuler ~]# dnf hotupgrade patch-redis-6.2.5-1-ACC-1-1.x86_64
Last metadata expiration check: 0:01:39 ago on 2024年01月02日 星期二 20时16分45秒.
The hotpatch 'redis-6.2.5-1/ACC-1-1' already has a 'ACTIVED' sub hotpatch of binary file 'redis-benchmark'
The hotpatch 'redis-6.2.5-1/ACC-1-1' already has a 'ACTIVED' sub hotpatch of binary file 'redis-cli'
The hotpatch 'redis-6.2.5-1/ACC-1-1' already has a 'ACTIVED' sub hotpatch of binary file 'redis-server'
Package patch-redis-6.2.5-1-ACC-1-1.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
```

使用dnf hotpatch --list确认该热补丁是否安装成功，确认Status为ACTIVED。

```shell
[root@openEuler ~]# dnf hotpatch --list
Last metadata expiration check: 0:04:43 ago on 2024年01月02日 星期二 20时16分45秒.
base-pkg/hotpatch                                   status
redis-6.2.5-1/ACC-1-1/redis-benchmark               ACTIVED
redis-6.2.5-1/ACC-1-1/redis-cli                     ACTIVED
redis-6.2.5-1/ACC-1-1/redis-server                  ACTIVED
```

确认该cve是否已被修复，由于CVE-2021-1所使用的热补丁包patch-redis-6.2.5-1-ACC-1-1.x86\_64同样修复CVE-2021-11，CVE-2021-1和CVE-2021-11都不予显示。

```shell
[root@openEuler ~]# dnf hot-updateinfo list cves
Last metadata expiration check: 0:08:48 ago on 2023年03月25日 星期六 11时53分46秒.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-1076  Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26607 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-22    Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-2     Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

激活后测试功能，对比激活前回显内容。

```shell
[root@openEuler ~]# telnet 127.0.0.1 6380
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

*100

-ERR Protocol error: unauthenticated multibulk length
Connection closed by foreign host.
```

使用dnf hotpatch --remove指定热补丁手动卸载。

```shell
[root@openEuler ~]# dnf hotpatch --remove redis-6.2.5-1
Last metadata expiration check: 0:11:52 ago on 2024年01月02日 星期二 20时16分45秒.
Gonna remove this hot patch: redis-6.2.5-1
remove hot patch 'redis-6.2.5-1' succeed
[root@openEuler ~]# dnf hotpatch --list
Last metadata expiration check: 0:12:00 ago on 2024年01月02日 星期二 20时16分45秒.
base-pkg/hotpatch                                   status
redis-6.2.5-1/ACC-1-1/redis-benchmark               NOT-APPLIED
redis-6.2.5-1/ACC-1-1/redis-cli                     NOT-APPLIED
redis-6.2.5-1/ACC-1-1/redis-server                  NOT-APPLIED
```

使用热补丁扫描命令查看本机待修复cve，确认CVE-2021-1和CVE-2021-11正常显示。

```shell
[root@openEuler ~]# dnf hot-updateinfo list cves
Last metadata expiration check: 0:00:38 ago on 2023年03月25日 星期六 11时53分46秒.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-1     Important/Sec. redis-6.2.5-2.x86_64                                   patch-redis-6.2.5-1-ACC-1-1.x86_64
CVE-2021-11    Important/Sec. redis-6.2.5-2.x86_64                                   patch-redis-6.2.5-1-ACC-1-1.x86_64
CVE-2021-2     Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-22    Important/Sec. redis-6.2.5-3.x86_64                                   patch-redis-6.2.5-1-ACC-1-2.x86_64
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

* 安装高版本ACC热补丁

指定安装热补丁包patch-redis-6.2.5-1-ACC-1-2.x86\_64。

```shell
[root@openEuler ~]# dnf hotupgrade patch-redis-6.2.5-1-ACC-1-2.x86_64
Last metadata expiration check: 0:36:12 ago on 2024年01月02日 星期二 20时16分45秒.
The hotpatch 'redis-6.2.5-1/ACC-1-2' already has a 'ACTIVED' sub hotpatch of binary file 'redis-benchmark'
The hotpatch 'redis-6.2.5-1/ACC-1-2' already has a 'ACTIVED' sub hotpatch of binary file 'redis-cli'
The hotpatch 'redis-6.2.5-1/ACC-1-2' already has a 'ACTIVED' sub hotpatch of binary file 'redis-server'
Package patch-redis-6.2.5-1-ACC-1-2.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
```

使用热补丁扫描命令查看本机待修复cve，由于patch-redis-6.2.5-1-ACC-1-2.x86\_64比patch-redis-6.2.5-1-ACC-1-1.x86\_64的热补丁版本高，低版本热补丁对应的CVE-2021-1和CVE-2021-11，以及高版本热补丁对应的CVE-2021-2和CVE-2021-22都被修复。

```shell
[root@openEuler ~]# dnf hot-updateinfo list cves
Last metadata expiration check: 0:00:38 ago on 2023年03月25日 星期六 11时53分46秒.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

* 热补丁目标软件包版本大于本机安装版本

查看热补丁repo源中repodata目录下的xxx-updateinfo.xml.gz，确认文件中的CVE-2021-33、CVE-2021-3相关信息。

```xml
<update from="openeuler.org" type="security" status="stable">
          <id>openEuler-HotPatchSA-2023-3</id>
          <title>An update for mariadb is now available for openEuler-22.03-LTS</title>
          <severity>Important</severity>
          <release>openEuler</release>
          <issued date="2022-04-16"></issued>
          <references>
                  <reference href="https://nvd.nist.gov/vuln/detail/CVE-2021-3" id="CVE-2021-3" title="CVE-2021-3" type="cve"></reference>
                  <reference href="https://nvd.nist.gov/vuln/detail/CVE-2021-33" id="CVE-2021-33" title="CVE-2021-33" type="cve"></reference>
          </references>
          <description>patch-redis-6.2.5-2-ACC.(CVE-2021-3, CVE-2021-33)</description>
          <pkglist>
               <hot_patch_collection>
                    <name>openEuler</name>
                    <package arch="aarch64" name="patch-redis-6.2.5-2-ACC" release="1" version="1">
                         <filename>patch-redis-6.2.5-2-ACC-1-1.aarch64.rpm</filename>
                    </package>
                    <package arch="x86_64" name="patch-redis-6.2.5-2-ACC" release="1" version="1">
                         <filename>patch-redis-6.2.5-2-ACC-1-1.x86_64.rpm</filename>
                    </package>
               </hot_patch_collection>
          </pkglist>
  </update>
```

package中的name字段"patch-redis-6.2.5-2-ACC"的组成部分为：patch-源码包名-源码包version-源码包release-热补丁patch名，该热补丁包需要本机安装redis-6.2.5-2源码版本，检查本机redis安装版本。

```shell
[root@openEuler ~]# rpm -qa | grep redis
redis-6.2.5-1.x86_64
```

由于本机安装版本不匹配，大于本机安装版本，该热补丁包名不显示，以'-'显示。

```shell
[root@openEuler ~]# dnf hot-updateinfo list cves
Last metadata expiration check: 0:00:38 ago on 2023年03月25日 星期六 11时53分46秒.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

* 热补丁目标软件包版本小于本机安装版本。

查看热补丁repo源中repodata目录下的xxx-updateinfo.xml.gz，确认文件中的CVE-2021-44、CVE-2021-4相关信息。

```xml
<update from="openeuler.org" type="security" status="stable">
          <id>openEuler-HotPatchSA-2023-4</id>
          <title>An update for mariadb is now available for openEuler-22.03-LTS</title>
          <severity>Important</severity>
          <release>openEuler</release>
          <issued date="2022-04-16"></issued>
          <references>
                  <reference href="https://nvd.nist.gov/vuln/detail/CVE-2021-4" id="CVE-2021-4" title="CVE-2021-4" type="cve"></reference>
                  <reference href="https://nvd.nist.gov/vuln/detail/CVE-2021-44" id="CVE-2021-44" title="CVE-2021-44" type="cve"></reference>
          </references>
          <description>patch-redis-6.2.4-1-ACC.(CVE-2021-44, CVE-2021-4)</description>
          <pkglist>
               <hot_patch_collection>
                    <name>openEuler</name>
                    <package arch="aarch64" name="patch-redis-6.2.4-1-ACC" release="1" version="1">
                         <filename>patch-redis-6.2.4-1-ACC-1-1.aarch64.rpm</filename>
                    </package>
                    <package arch="x86_64" name="patch-redis-6.2.4-1-ACC" release="1" version="1">
                         <filename>patch-redis-6.2.4-1-ACC-1-1.x86_64.rpm</filename>
                    </package>
               </hot_patch_collection>
          </pkglist>
  </update>
```

package中的name字段"patch-redis-6.2.4-1-ACC"的组成部分为：patch-源码包名-源码包version-源码包release-热补丁patch名，该热补丁包需要本机安装redis-6.2.4-1源码版本，检查本机redis安装版本。

```shell
[root@openEuler ~]# rpm -qa | grep redis
redis-6.2.5-1.x86_64
```

由于本机安装版本不匹配，小于本机安装版本，该CVE不予显示。

```shell
[root@openEuler ~]# dnf hot-updateinfo list cves
Last metadata expiration check: 0:00:38 ago on 2023年03月25日 星期六 11时53分46秒.
CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64       -
CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch           -
CVE-2021-33    Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2021-3     Important/Sec. redis-6.2.5-4.x86_64                                   -
CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64                 -
```

---

---
url: /en/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/overview.md
---
# Docker Container

Docker is an open-source Linux container engine that enables quick application packaging, deployment, and delivery. The original meaning of Docker is dork worker, whose job is to pack the goods to the containers, and move containers, and load containers. Similarly, the job of Docker in Linux is to pack applications to containers, and deploy and run applications on various platforms using containers. Docker uses Linux Container technology to turn applications into standardized, portable, and self-managed components, enabling the "build once" and "run everywhere" features of applications. Features of Docker technology include: quick application release, easy application deployment and management, and high application density.

> \[!NOTE]**Note:**
>
> Root privileges are necessary for installing and operating Docker containers.

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/overview.md
---
# Docker容器

Docker是一个开源的Linux容器引擎项目， 用以实现应用的快速打包、部署和交付。Docker的英文本意是码头工人，码头工人的工作就是将商品打包到container(集装箱)并且搬运container、装载container。 对应到Linux中，Docker就是将app打包到container，通过container实现app在各种平台上的部署、运行。Docker通过Linux Container技术将app变成一个标准化的、可移植的、自管理的组件，从而实现应用的“一次构建，到处运行”。Docker技术特点就是：应用快速发布、部署简单、管理方便，应用密度更高。

> \[!NOTE]说明
> Docker容器的安装和使用需要root权限。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/distributed/dsoftbus_application_based_on_containers.md
---
# DSoftBus Application Based on Containers

## Background

Migrating user software to containers is an inevitable trend. This document describes how to deploy DSoftBus based on containers, simplifying the installation and deployment of DSoftBus clients and facilitating compatibility with service software.

## Environment Setup

**Hardware devices**

| Device            | OS               | Description                           | Quantity|
| -------------------- | ----------------------- | ------------------------------- | --- |
| Raspberry Pi 4B           | openEuler 22.03-LTS-SP4  | Raspberry Pi with openEuler installed  | 2 |

## Code Repository

<https://atomgit.com/openeuler/distributed-codelabs.git>

## Description

### Installing the Services

Perform the following steps on both devices.

1. To install openEuler 22.03 LTS SP2 or later on a physical machine, the [Binder driver](https://atomgit.com/src-openeuler/communication_ipc/blob/openEuler-22.03-LTS-SP2/README.md "binder") must be installed in the environment.

2. Build softbus\_client.

   ```sh
   bash distributed-codelabs/build-repo/demo/dsoftbus/build.sh
   ```

3. Install softbus\_server.

   ```sh
   dnf install dsoftbus -y
   ```

4. Start softbus\_server.

   ```sh
   /system/bin/start_services.sh all
   ```

### Testing the Multi-Client Container Scenario

1. Run the script to build and load the container image on device A.

   ```sh
   bash distributed-codelabs/build-repo/demo/dsoftbus/docker_img_build.sh
   ```

2. Start the **softbus\_client** container image on device A and map the SDKs and binder driver to the container.

   ```sh
   docker run  -it  --privileged --net=host --name=softbus  -v /dev/binderfs/binder:/dev/binder  -v  /system:/system  -v /usr/lib64:/usr/lib64  -p 5684:5684/udp softbus_client_image  bash
   ```

3. Write to the **/etc/SI** file in the image as the flag of the DSoftBus client in the container. Note that the flag must be different from those of other clients on the network to avoid conflicts.

   ```sh
   echo 123 > /etc/SI
   ```

4. Start **softbus\_client** in the container.

   ```sh
   ./home/softbus_client
   ```

5. You can repeat steps 2 and 3 to start different client containers on this node.

6. Start DSoftBus on device B.

   ```sh
   ./build-repo/demo/dsoftbus/softbus_client
   ```

7. Run the following command on each client of device A to enable all connections:

   ```sh
   openA
   ```

8. View all opened sessions on the client of device B.

   ```sh
   conDevices
   ```

   The command output indicates that two sessions whose IDs are 4 and 3 have been opened. The session IDs are allocated by softbus\_server of the local host and will be used for subsequent message sending.

   ```sh
   conDevices
   12-11 20:18:58.350 2780335 2780335 I A0fffe/SOFTBUS_DEMO: [SOFTBUS_DEMO]::PrintConnectedDevicesInfo: sessionId:4, networkId: 3c95f61941b81c48ecd73fef881262b82fcbc58e9b1f545e2097b0dc6fecea37
   12-11 20:18:58.350 2780335 2780335 I A0fffe/SOFTBUS_DEMO: [SOFTBUS_DEMO]::PrintConnectedDevicesInfo: sessionId:3, networkId: 3c95f61941b81c48ecd73fef881262b82fcbc58e9b1f545e2097b0dc6fecea37
   ```

9. Use device B to send a message to the client of device A through session 4.

   ```sh
   send 4 "hello4"
   12-11 20:19:14.975 2780335 2780335 I C015c0/dsoftbus: [TRAN]SendBytes: sessionId=4
   ```

   If output is displayed in a client of device A, the client communicates with device B through session 4. Similarly, you can send messages to another client through session 3.

10. Use the client in the container of device A to send a message to the client of device B and check the opened session ID.

    ```sh
    conDevices
    12-11 20:25:48.995 344047 344047 I A0fffe/SOFTBUS_DEMO: [SOFTBUS_DEMO]::PrintConnectedDevicesInfo: sessionId:1, networkId: e69eab4e2d657264dfbb2006fdfa15524f4a27edeff0baa26d5d2a2b9502f300
    ```

    The output indicates that session 1 is used for communication.

11. Send a message. If device B receives the string, the message is received successfully.

    ```sh
    send 1 "hello1"
    ```

12. Use the client of device A to send a message to the client of device B and check the opened session ID.

    ```sh
    conDevices
    12-11 20:37:24.823 3512580 3512580 I A0fffe/SOFTBUS_DEMO: [SOFTBUS_DEMO]::PrintConnectedDevicesInfo: sessionId:1, networkId: e69eab4e2d657264dfbb2006fdfa15524f4a27edeff0baa26d5d2a2b9502f300
    ```

    The output indicates that session 1 is used for communication.

13. Send a message. If device B receives the string, the message is received successfully.

    ```sh
    send 1 "hello1"
    ```

### Session ID Description

In the test in the previous section, it is found that the session IDs used for communication between the two clients are different. The reason is that the session IDs are allocated by the server on the local host and are unique only on the local host. Similarly, the session names and group names of the container are isolated from those of the VM. Therefore, the session IDs of the local container are the same as those of the client on the VM.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/dynamically_loading_the_kernel_module.md
---
# Dynamically Loading the Kernel Module

## Function Description

Services in a container may depend on some kernel modules. You can set environment variables to dynamically load the kernel modules required by services in the container to the host before the system container starts. This feature must be used together with isulad-hooks. For details, see [Dynamically Managing Container Resources (syscontainer-tools)](./dynamically_managing_container_resources_syscontainer_tools.md).

## Parameter Description

## Constraints

* If loaded kernel modules are not verified or conflict with existing modules on the host, an unpredictable error may occur on the host. Therefore, exercise caution when loading kernel modules.
* Dynamic kernel module loading transfers kernel modules to be loaded to containers. This function is implemented by capturing environment variables for container startup using isulad-tools. Therefore, this function relies on the proper installation and deployment of isulad-tools.
* Loaded kernel modules need to be manually deleted.

## Example

When starting a system container, specify the  **-e KERNEL\_MODULES**  parameter. After the system container is started, the ip\_vs module is successfully loaded to the kernel.

```shell
[root@localhost ~]# lsmod | grep ip_vs
[root@localhost ~]# isula run -tid -e KERNEL_MODULES=ip_vs,ip_vs_wrr --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/myrootfs none init
ae18c4281d5755a1e153a7bff6b3b4881f36c8e528b9baba8a3278416a5d0980
[root@localhost ~]# lsmod | grep ip_vs
ip_vs_wrr              16384  0
ip_vs                 176128  2 ip_vs_wrr
nf_conntrack          172032  7 xt_conntrack,nf_nat,nf_nat_ipv6,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink,ip_vs
nf_defrag_ipv6         20480  2 nf_conntrack,ip_vs
libcrc32c              16384  3 nf_conntrack,nf_nat,ip_vs
```

> \[!NOTE] **NOTE:**
>
> * isulad-tools must be installed on the host.
> * **--hooks-spec**  must be set to  **isulad hooks**.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/dynamically_managing_container_resources_syscontainer_tools.md
---
# Dynamically Managing Container Resources (syscontainer-tools)

Resources in common containers cannot be managed. For example, a block device cannot be added to a common container, and a physical or virtual NIC cannot be inserted to a common container. In the system container scenario, the syscontainer-tools can be used to dynamically mount or unmount block devices, network devices, routes, and volumes for containers.

To use this function, you need to install the syscontainer-tools first.

```sh
[root@localhost ~]# yum install syscontainer-tools
```

## Device Management

### Function Description

isulad-tools allows you to add block devices (such as disks and logical volume managers) or character devices (such as GPUs, binners, and FUSEs) on the host to a container. The devices can be used in the container. For example, you can run the  **fdisk**  command to format the disk and write data to the file system. If the devices are not required, isulad-tools allows you to delete them from the container and return them to the host.

### Command Format

```sh
isulad-tools [COMMAND][OPTIONS] <container_id> [ARG...]
```

In the preceding format:

**COMMAND**: command related to device management.

**OPTIONS**: option supported by the device management command.

**container\_id**: container ID.

**ARG**: parameter corresponding to the command.

### Parameter Description

### Constraints

* You can add or delete devices when container instances are not running. After the operation is complete, you can start the container to view the device status. You can also dynamically add a device when the container is running.
* Do not concurrently run the  **fdisk**  command to format disks in a container and on the host. Otherwise, the container disk usage will be affected.
* When you run the  **add-device**  command to add a disk to a specific directory of a container, if the parent directory in the container is a multi-level directory (for example,  **/dev/a/b/c/d/e**) and the directory level does not exist, isulad-tools will automatically create the corresponding directory in the container. When the disk is deleted, the created parent directory is not deleted. If you run the  **add-device**  command to add a device to this parent directory again, a message is displayed, indicating that a device already exists and cannot be added.
* When you run the**add-device**  command to add a disk or update disk parameters, you need to configure the disk QoS. Do not set the write or read rate limit for the block device (I/O/s or byte/s) to a small value. If the value is too small, the disk may be unreadable (the actual reason is the speed is too slow), affecting service functions.
* When you run the  **--blkio-weight-device**  command to limit the weight of a specified block device, if the block device supports only the BFQ mode, an error may be reported, prompting you to check whether the current OS environment supports setting the weight of the BFQ block device.

### Example

* Start a system container, and set  **hook spec**  to the isulad hook execution script.

  ```sh
  [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init
  eed1096c8c7a0eca6d92b1b3bc3dd59a2a2adf4ce44f18f5372408ced88f8350
  ```

* Add a block device to a container.

  ```sh
  [root@localhost ~]# isulad-tools add-device ee /dev/sdb:/dev/sdb123
  Add device (/dev/sdb) to container(ee,/dev/sdb123) done.
  [root@localhost ~]# isula exec ee fdisk -l /dev/sdb123
  Disk /dev/sdb123: 50 GiB, 53687091200 bytes, 104857600 sectors
  Units: sectors of 1 * 512 = 512 bytes
  Sector size (logical/physical): 512 bytes / 512 bytes
  I/O size (minimum/optimal): 512 bytes / 512 bytes
  Disklabel type: dos
  Disk identifier: 0xda58a448

  Device        Boot Start       End   Sectors Size Id Type
  /dev/sdb123p1       2048 104857599 104855552  50G  5 Extended
  /dev/sdb123p5       4096 104857599 104853504  50G 83 Linux
  ```

* Update the device information.

  ```sh
  [root@localhost ~]# isulad-tools update-device --device-read-bps /dev/sdb:10m ee
  Update read bps for device (/dev/sdb,10485760) done.
  ```

* Delete a device.

  ```sh
  [root@localhost ~]# isulad-tools remove-device ee /dev/sdb:/dev/sdb123
  Remove device (/dev/sdb) from container(ee,/dev/sdb123) done.
  Remove read bps for device (/dev/sdb) done.
  ```

## NIC Management

### Function Description

isulad-tools allows you to insert physical or virtual NICs on the host to a container. If the NICs are not required, isulad-tools allows you to delete them from the container and return them to the host. In addition, the NIC configurations can be dynamically modified. To insert a physical NIC, add the NIC on the host to the container. To insert a virtual NIC, create a veth pair and insert its one end to the container.

### Command Format

```sh
isulad-tools [COMMAND][OPTIONS] <container_id>
```

In the preceding format:

**COMMAND**: command related to NIC management.

**OPTIONS**: option supported by the NIC management command.

**container\_id**: container ID.

### Parameter Description

### Constraints

* Physical NICs (eth) and virtual NICs (veth) can be added.
* When adding a NIC, you can also configure the NIC. The configuration parameters include  **--ip**,  **--mac**,  **--bridge**,  **--mtu**,  **--qlen**.
* A maximum of eight physical NICs can be added to a container.
* If you run the  **isulad-tools add-nic**  command to add an eth NIC to a container and do not add a hook, you must manually delete the NIC before the container exits. Otherwise, the name of the eth NIC on the host will be changed to the name of that in the container.
* For a physical NIC (except 1822 VF NIC), use the original MAC address when running the  **add-nic**  command. Do not change the MAC address in the container, or when running the  **update-nic**  command.
* When using the  **isulad-tools add-nic**  command, set the MTU value. The value range depends on the NIC model.
* When using isulad-tools to add NICs and routes to containers, you are advised to run the  **add-nic**  command to add NICs and then run the  **add-route**  command to add routes. When using isulad-tools to delete NICs and routes from a container, you are advised to run the  **remove-route**  command to delete routes and then run the  **remove-nic**  command to delete NICs.
* When using isulad-tools to add NICs, add a NIC to only one container.

### Example

* Start a system container, and set  **hook spec**  to the isulad hook execution script.

  ```sh
  [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init
  2aaca5c1af7c872798dac1a468528a2ccbaf20b39b73fc0201636936a3c32aa8
  ```

* Add a virtual NIC to a container.

  ```sh
  [root@localhost ~]# isulad-tools add-nic --type "veth" --name abc2:bcd2 --ip 172.17.28.5/24 --mac 00:ff:48:13:xx:xx --bridge docker0 2aaca5c1af7c
  Add network interface to container 2aaca5c1af7c (bcd2,abc2) done  
  ```

* Add a physical NIC to a container.

  ```sh
  [root@localhost ~]# isulad-tools add-nic --type "eth" --name eth3:eth1 --ip 172.17.28.6/24  --mtu 1300  --qlen 2100 2aaca5c1af7c
  Add network interface to container 2aaca5c1af7c (eth3,eth1) done
  ```

  > \[!NOTE] **NOTE:**\
  > When adding a virtual or physical NIC, ensure that the NIC is in the idle state. Adding a NIC in use will disconnect the system network.

## Route Management

### Function Description

isulad-tools can be used to dynamically add or delete routing tables for system containers.

### Command Format

```sh
isulad-tools [COMMAND][OPTIONS] <container_id> [ARG...]
```

In the preceding format:

**COMMAND**: command related to route management.

**OPTIONS**: option supported by the route management command.

**container\_id**: container ID.

**ARG**: parameter corresponding to the command.

### API Description

### Constraints

* When using isulad-tools to add NICs and routes to containers, you are advised to run the  **add-nic**  command to add NICs and then run the  **add-route**  command to add routes. When using isulad-tools to delete NICs and routes from a container, you are advised to run the  **remove-route**  command to delete routes and then run the  **remove-nic**  command to delete NICs.
* When adding a routing rule to a container, ensure that the added routing rule does not conflict with existing routing rules in the container.

### Example

* Start a system container, and set  **hook spec**  to the isulad hook execution script.

  ```sh
  [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init
  0d2d68b45aa0c1b8eaf890c06ab2d008eb8c5d91e78b1f8fe4d37b86fd2c190b
  ```

* Use isulad-tools to add a physical NIC to the system container.

  ```sh
  [root@localhost ~]# isulad-tools add-nic --type "eth" --name enp4s0:eth123 --ip 172.17.28.6/24  --mtu 1300  --qlen 2100 0d2d68b45aa0
  Add network interface (enp4s0) to container (0d2d68b45aa0,eth123) done
  ```

* isulad-tools adds a routing rule to the system container. Format example:  **\[{"dest":"default", "gw":"192.168.10.1"},{"dest":"192.168.0.0/16","dev":"eth0","src":"192.168.1.2"}]**. If  **dest**  is left blank, its value will be  **default**.

  ```sh
  [root@localhost ~]# isulad-tools add-route 0d2d68b45aa0 '[{"dest":"172.17.28.0/32", "gw":"172.17.28.5","dev":"eth123"}]'
  Add route to container 0d2d68b45aa0, route: {dest:172.17.28.0/32,src:,gw:172.17.28.5,dev:eth123} done
  ```

* Check whether a routing rule is added in the container.

  ```sh
  [root@localhost ~]# isula exec -it 0d2d68b45aa0 route
  Kernel IP routing table
  Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
  172.17.28.0     172.17.28.5     255.255.255.255 UGH   0      0        0 eth123
  172.17.28.0     0.0.0.0         255.255.255.0   U     0      0        0 eth123
  ```

## Volume Mounting Management

### Function Description

In a common container, you can set the  **--volume**  parameter during container creation to mount directories or volumes of the host to the container for resource sharing. However, during container running, you cannot unmount directories or volumes that are mounted to the container, or mount directories or volumes of the host to the container. Only the system container can use the isulad-tools tool to dynamically mount directories or volumes of the host to the container and unmount directories or volumes from the container.

### Command Format

```sh
isulad-tools [COMMAND][OPTIONS] <container_id> [ARG...]
```

In the preceding format:

**COMMAND**: command related to route management.

**OPTIONS**: option supported by the route management command.

**container\_id**: container ID.

**ARG**: parameter corresponding to the command.

### API Description

**Table  1**    

### Constraints

* When running the  **add-path**  command, specify an absolute path as the mount path.
* The mount point /.sharedpath is generated on the host after the mount path is specified by running the  **add-path**  command.
* A maximum of 128 volumes can be added to a container.
* Do not overwrite the root directory (/) in a container with the host directory by running the  **add-path**  command. Otherwise, the function is affected.

### Example

* Start a system container, and set  **hook spec**  to the isulad hook execution script.

  ```sh
  [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init
  e45970a522d1ea0e9cfe382c2b868d92e7b6a55be1dd239947dda1ee55f3c7f7
  ```

* Use isulad-tools to mount a directory on the host to a container, implementing resource sharing.

  ```sh
  [root@localhost ~]# isulad-tools add-path e45970a522d1 /home/test123:/home/test123
  Add path (/home/test123) to container(e45970a522d1,/home/test123) done.
  ```

* Create a file in the  **/home/test123**  directory on the host and check whether the file can be accessed in the container.

  ```sh
  [root@localhost ~]# echo "hello world" > /home/test123/helloworld
  [root@localhost ~]# isula exec e45970a522d1 bash
  [root@localhost /]# cat /home/test123/helloworld
  hello world
  ```

* Use isulad-tools to delete the mount directory from the container.

  ```sh
  [root@localhost ~]# isulad-tools remove-path e45970a522d1 /home/test123:/home/test123
  Remove path (/home/test123) from container(e45970a522d1,/home/test123) done
  [root@localhost ~]# isula exec e45970a522d1 bash
  [root@localhost /]# ls /home/test123/helloworld
  ls: cannot access '/home/test123/helloworld': No such file or directory
  ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/performance/eagle/eagle_installation_and_usage.md
---
# eagle

## 介绍

EAGLE（Energy Aware intelliGent scheduLEr）是一种基于能效动态调整操作系统的服务。

## 说明

* 发布版本：22.03-LTS-SP4
* eagle版本：1.1.0
* rpm包：eagle，纳管所有功耗策略的实现，依赖powerapi。
* rpm子包：python3-eagle-mpctool，mpctool子包，是控制风扇转速的一个服务
* mpctool目前只支持以下arm服务器机型，x86下无法使用
  1. Taishan200 2280(VD)
  2. Taishan200 Pro 2280
  3. Taishan200 2280v2

## 安装

使用dnf安装软件(确认已经配置22.03-LTS-SP4的yum源)

```shell
dnf install -y eagle python3-eagle-mpctool
```

安装完软件之后，会自动拉起eagle和mpctool服务。可以使用systemctl查看服务状态

```shell
systemctl status pwrapis # eagle依赖powerapi,启动前确认pwrapis服务已经启动
systemctl status eagle
systemctl status mpctool
```

## 关键文件说明

所有eagle相关的配置等文件，均可以通过查看rpm包提供的文件获取，使用命令

```shell
rpm -ql eagle
```

具体每一项说明：

* `/etc/eagle/eagle_policy.ini`: 策略配置文件，用户可以操作这个配置文件配置需要的功耗策略。
* `/etc/eagle/eagle_config.ini`: 日志相关配置、更新策略文件的周期等配置，用户可配置。
* `/etc/eagle/plugin/lib_*.so`: 功耗插件实现动态库，用户不可配置。目前支持sched\_service、freq\_service、idle\_service、mpc\_service这四类功耗策略，所以会有四个 lib\_\*.so 插件库。
* `/usr/sbin/eagle`: eagle可执行文件。
* `/etc/systemd/system/eagle.service`: eagle服务启动配置文件。

## 主要功能说明

* 动态更新功耗策略：eagle目前可以根据policy配置文件（/etc/eagle/eagle\_policy.ini）中的配置，动态调整系统的功耗相关配置。具体每一项配置作用，在配置文件中均有注释说明。
* 功耗配置还原：当eagle服务退出时，会恢复eagle起来之前的系统功耗配置。

---

---
url: /en/docs/22.03_LTS_SP4/edge_computing.md
---


---

---
url: /en/docs/22.03_LTS_SP4/embedded.md
---


---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/environment_preparation.md
---
# Environment Preparation

## Preparing a VM Image

### Overview

A VM image is a file that contains a virtual disk that has been installed and can be used to start the OS. VM images are in different formats, such as raw and qcow2. Compared with the raw format, the qcow2 format occupies less space and supports features such as snapshot, copy-on-write, AES encryption, and zlib compression. However, the performance of the qcow2 format is slightly lower than that of the raw format. The qemu-img tool is used to create image files. This section uses the qcow2 image file as an example to describe how to create a VM image.

### Creating an Image

To create a qcow2 image file, perform the following steps:

1. Install the  **qemu-img**  software package.

   ```shell
   yum install -y qemu-img
   ```

2. Run the  **create**  command of the qemu-img tool to create an image file. The command format is as follows:

   ```shell
   qemu-img create -f <imgFormat> -o <fileOption> <fileName> <diskSize>
   ```

   The parameters are described as follows:

   * *imgFormat*: Image format. The value can be  **raw**  or  **qcow2**.
   * *fileOption*: File option, which is used to set features of an image file, such as specifying a backend image file, compression, and encryption.
   * *fileName*: File name.
   * *diskSize*: Disk size, which specifies the size of a block disk. The unit can be K, M, G, or T, indicating KiB, MiB, GiB, or TiB.

   For example, to create an image file **openEuler-image.qcow2** whose disk size is 4 GB and format is qcow2, the command and output are as follows:

   ```shell
   $ qemu-img create -f qcow2 openEuler-image.qcow2 4G
   Formatting 'openEuler-image.qcow2', fmt=qcow2 size=4294967296 cluster_size=65536 lazy_refcounts=off refcount_bits=16
   ```

### Changing the Image Disk Space

If a VM requires larger disk space, you can use the qemu-img tool to change the disk space of the VM image. The method is as follows:

1. Run the following command to query the disk space of the VM image:

   ```shell
   qemu-img info <imgFileName>
   ```

   For example, if the command and output for querying the disk space of the openEuler-image.qcow2 image are as follows, the disk space of the image is 4 GiB.

   ```shell
   $ qemu-img info openEuler-image.qcow2 
   image: openEuler-image.qcow2
   file format: qcow2
   virtual size: 4.0G (4294967296 bytes)
   disk size: 196K
   cluster_size: 65536
   Format specific information:
       compat: 1.1
       lazy refcounts: false
       refcount bits: 16
       corrupt: false
   ```

2. Run the following command to change the image disk space. In the command,  *imgFileName*  indicates the image name, and  **+**  and  **-**  indicate the image disk space to be increased and decreased, respectively. The unit is KB, MB, GB, and T, indicating KiB, MiB, GiB, and TiB, respectively.

   ```shell
   qemu-img resize <imgFileName> [+|-]<size>
   ```

   For example, to expand the disk space of the openEuler-image.qcow2 image to 24 GiB, that is, to add 20 GiB to the original 4 GiB, the command and output are as follows:

   ```shell
   $ qemu-img resize openEuler-image.qcow2 +20G
   Image resized.
   ```

3. Run the following command to check whether the image disk space is changed successfully:

   ```shell
   qemu-img info <imgFileName>
   ```

   For example, if the openEuler-image.qcow2 image disk space has been expanded to 24 GiB, the command and output are as follows:

   ```shell
   $ qemu-img info openEuler-image.qcow2 
   image: openEuler-image.qcow2
   file format: qcow2
   virtual size: 24G (25769803776 bytes)
   disk size: 200K
   cluster_size: 65536
   Format specific information:
       compat: 1.1
       lazy refcounts: false
       refcount bits: 16
       corrupt: false
   ```

## Preparing the VM Network

### Overview

To enable the VM to communicate with external networks, you need to configure the network environment for the VM. KVM virtualization supports multiple types of bridges, such as Linux bridge and Open vSwitch bridge. As shown in  [Figure 1](#fig1785384714917), the data transmission path is  **VM > virtual NIC device > Linux bridge or Open vSwitch bridge > physical NIC**. In addition to configuring virtual NICs (vNICs) for VMs, creating a bridge for a host is the key to connecting to a virtualized network.

This section describes how to set up a Linux bridge and an Open vSwitch bridge to connect a VM to the network. You can select a bridge type based on the site requirements.

**Figure  1**  Virtual network structure\
![](./figures/virtual-network-structure.png)

### Setting Up a Linux Bridge

The following describes how to bind the physical NIC eth0 to the Linux bridge br0.

1. Install the  **bridge-utils**  software package.

   The Linux bridge is managed by the brctl tool. The corresponding installation package is bridge-utils. The installation command is as follows:

   ```shell
   yum install -y bridge-utils
   ```

2. Create bridge br0.

   ```shell
   brctl addbr br0
   ```

3. Bind the physical NIC eth0 to the Linux bridge.

   ```shell
   brctl addif br0 eth0
   ```

   > \[!NOTE] **Note:**
   > If you run the `brctl addif br0 eth0` command through an SSH connection, the connection will be closed. You need to perform the following operations on iBMC to complete the VM network configuration.

4. After eth0 is connected to the bridge, the IP address is no longer required. Install net-tools and set the IP address of eth0 to 0.0.0.0.

   ```shell
   yum install -y net-tools
   ifconfig eth0 0.0.0.0
   ```

5. Set the IP address of br0.
   * If a DHCP server is available, set a dynamic IP address through the dhclient.

     ```shell
     dhclient br0
     ```

   * If no DHCP server is available, configure a static IP address for br0. For example, set the static IP address to 192.168.1.2 and subnet mask to 255.255.255.0.

     ```shell
     ifconfig br0 192.168.1.2 netmask 255.255.255.0
     ```

### Setting Up an Open vSwitch Bridge

The Open vSwitch bridge provides more convenient automatic orchestration capabilities. This section describes how to install network virtualization components to set up an Open vSwitch bridge.

**1. Install the Open vSwitch component.**

If the Open vSwitch is used to provide virtual network, you need to install the Open vSwitch network virtualization component.

1. Install the Open vSwitch component.

   ```shell
   yum install -y openvswitch
   ```

2. Start the Open vSwitch service.

   ```shell
   systemctl start openvswitch
   ```

**2. Check whether the installation is successful.**

1. Check whether the openvswitch component is successfully installed. If the installation is successful, the software package information is displayed. The command and output are as follows:

   ```shell
   $ rpm -qi openvswitch
   Name        : openvswitch
   Version     : 2.12.4
   Release     : 3.oe2203SP3
   Architecture: x86_64
   Install Date: Tue 09 May 2023 10:58:53 AM CST
   Group       : Unspecified
   Size        : 7920016
   License     : ASL 2.0 and ISC
   Signature   : RSA/SHA256, Wed 19 Apr 2023 09:40:31 AM CST, Key ID 007fb747fb37bc6f
   Source RPM  : openvswitch-2.12.4-3.oe2203SP3.src.rpm
   Build Date  : Wed 19 Apr 2023 09:39:49 AM CST
   Build Host  : dc-64g.compass-ci
   Packager    : http://openeuler.org
   URL         : http://www.openvswitch.org/
   Summary     : Production Quality, Multilayer Open Virtual Switch
   Description :
   Open vSwitch is a production quality, multilayer virtual switch licensed under
   the open source Apache 2.0 license.
   ```

2. Check whether the Open vSwitch service is started successfully. If the service is in the  **Active**  state, the service is started successfully. You can use the command line tool provided by the Open vSwitch. The command and output are as follows:

   ```shell
   $ systemctl status openvswitch
   ● openvswitch.service - LSB: Open vSwitch switch
      Loaded: loaded (/etc/rc.d/init.d/openvswitch; generated)
      Active: active (running) since Sat 2019-08-17 09:47:14 CST; 4min 39s ago
        Docs: man:systemd-sysv-generator(8)
     Process: 54554 ExecStart=/etc/rc.d/init.d/openvswitch start (code=exited, status=0/SUCCESS)
       Tasks: 4 (limit: 9830)
      Memory: 22.0M
      CGroup: /system.slice/openvswitch.service
              ├─54580 ovsdb-server: monitoring pid 54581 (healthy)
              ├─54581 ovsdb-server /etc/openvswitch/conf.db -vconsole:emer -vsyslog:err -vfile:info --remote=punix:/var/run/openvswitch/db.sock --private-key=db:Open_vSwitch,SSL,private_key --certificate>
              ├─54602 ovs-vswitchd: monitoring pid 54603 (healthy)
              └─54603 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/open>
   ```

**3. Set up an Open vSwitch bridge**

The following describes how to set up an Open vSwitch layer-1 bridge br0.

1. Create the Open vSwitch bridge br0.

   ```shell
   ovs-vsctl add-br br0
   ```

2. Add the physical NIC eth0 to br0.

   ```shell
   ovs-vsctl add-port br0 eth0
   ```

3. After eth0 is connected to the bridge, the IP address of eth0 is set to 0.0.0.0.

   ```shell
   ifconfig eth0 0.0.0.0
   ```

4. Assign an IP address to OVS bridge br0.
   * If a DHCP server is available, set a dynamic IP address through the dhclient.

     ```shell
     dhclient br0
     ```

   * If no DHCP server is available, configure a static IP address for br0, for example, 192.168.1.2.

     ```shell
     ifconfig br0 192.168.1.2
     ```

## Preparing Boot Firmware

### Overview

The boot mode varies depending on the architecture. x86 servers support the Unified Extensible Firmware Interface (UEFI) and legacy boot modes, and AArch64 servers support only the UEFI boot mode. By default, boot files corresponding to the BIOS mode have been installed on openEuler. No additional operations are required. This section describes how to install boot files corresponding to the UEFI mode.

The Unified Extensible Firmware Interface (UEFI) is a new interface standard used for power-on auto check and OS boot. It is an alternative to the traditional BIOS. EDK II is a set of open source code that implements the UEFI standard. In virtualization scenarios, the EDK II tool set is used to start a VM in UEFI mode. Before using the EDK II tool, you need to install the corresponding software package before starting a VM. This section describes how to install the EDK II tool.

### Installation Methods

If the UEFI mode is used, the tool set EDK II needs to be installed. The installation package for the AArch64 architecture is  **edk2-aarch64**, and that for the x86 architecture is  **edk2-ovmf**. This section uses the AArch64 architecture as an example to describe the installation method. For the x86 architecture, you only need to replace  **edk2-aarch64**  with  **edk2-ovmf**.

1. Run the following command to install the  **edk**  software package:

   In the AArch64 architecture, the  **edk2**  package name is  **edk2-aarch64**.

   ```shell
   yum install -y edk2-aarch64
   ```

   In the x86\_64 architecture, the  **edk2**  package name is  **edk2-ovmf**.

   ```shell
   yum install -y edk2-ovmf
   ```

2. Run the following command to check whether the  **edk**  software package is successfully installed:

   In the AArch64 architecture, the command is as follows:

   ```shell
   rpm -qi edk2-aarch64
   ```

   If information similar to the following is displayed, the  **edk**  software package is successfully installed:

   ```console
   Name        : edk2-aarch64
   Version     : 202011
   Release     : 11.oe2203SP3
   Architecture: noarch
   Install Date: Tue 09 May 2023 11:28:22 AM CST
   Group       : Unspecified
   ```

   In the x86\_64 architecture, the command is as follows:

   ```shell
   rpm -qi edk2-ovmf
   ```

   If information similar to the following is displayed, the  **edk**  software package is successfully installed:

   ```console
   Name        : edk2-ovmf
   Version     : 202011
   Release     : 11.oe2203SP3
   Architecture: noarch
   Install Date: Tue 09 May 2023 11:06:06 AM CST
   ```

## Configuring as a Non-Root User

### Overview

openEuler uses the `virsh` command to manage VMs. If you want to use `virsh` as a non-root user, you need to perform some configurations.

### Non-Root User Configurations

In the following commands, replace *userName* with the actual user name.

1. Log in to the host as the **root** user.

2. Add the non-root user t the **libvirt** user group.

   ```shell
   usermod -a -G libvirt userName
   ```

3. Switch to the non-root user.

   ```shell
   su userName
   ```

4. Configure the environment variables. Open the **~/.bashrc** file.

   ```shell
   vim ~/.bashrc
   ```

   Add the following content to the end of the file.

   ```text
   export LIBVIRT_DEFAULT_URI="qemu:///system"
   ```

   Run the following command for the configuration to take effect.

   ```shell
   source ~/.bashrc
   ```

5. Add the following content to the domain root element in the VM XML configuration file to allow qemu-kvm to access the drive image.

   ```xml
   <seclabel type='dynamic' model='dac' relabel='yes'>
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/environment_variable_persisting.md
---
# Environment Variable Persisting

## Function Description

In a system container, you can make the  **env**  variable persistent to the configuration file in the rootfs directory of the container by specifying the  **--env-target-file**  interface parameter.

## Parameter Description

## Constraints

* If the target file specified by  **--env-target-file**  exists, the size cannot exceed 10 MB.
* The parameter specified by  **--env-target-file**  must be an absolute path in the rootfs directory.
* If the value of  **--env**  conflicts with that of  **env**  in the target file, the value of  **--env**  prevails.

## Example

Start a system container and specify the  **env**  environment variable and  **--env-target-file**  parameter.

```sh
[root@localhost ~]# isula run -tid -e abc=123 --env-target-file /etc/environment --system-container --external-rootfs /root/myrootfs none init
b75df997a64da74518deb9a01d345e8df13eca6bcc36d6fe40c3e90ea1ee088e
[root@localhost ~]# isula exec b7 cat /etc/environment
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TERM=xterm
abc=123
```

The preceding information indicates that the  **env**  variable (**abc=123**) of the container has been made persistent to the  **/etc/environment**  configuration file.

---

---
url: /zh/docs/22.03_LTS_SP4/server/memory_storage/etmem/etmem_user_guide.md
---
# etmem

## 介绍

随着CPU算力的发展，尤其是ARM核成本的降低，内存成本和内存容量成为约束业务成本和性能的核心痛点，因此如何节省内存成本，如何扩大内存容量成为存储迫切要解决的问题。

etmem内存分级扩展技术，通过DRAM+内存压缩/高性能存储新介质形成多级内存存储，对内存数据进行分级，将分级后的内存冷数据从内存介质迁移到高性能存储介质中，达到内存容量扩展的目的，从而实现内存成本下降。

etmem软件包运行的工具主要分为etmem客户端和etmemd服务端。etmemd服务端工具，运行后常驻，其中实现了目的进程的内存冷热识别及淘汰等功能。etmem客户端工具，调用时运行一次，根据命令参数的不同，控制etmemd服务端响应不同的操作。

## 编译教程

1. 下载etmem源码。

   ```bash
   $ git clone https://atomgit.com/openeuler/etmem.git
   ```

2. 编译和运行依赖。

   etmem的编译和运行依赖于libboundscheck组件。

   安装命令：

   ```bash
   $ yum -y install libboundscheck
   ```

   通过rpm包进行确认是否安装：

   ```bash
   rpm -qa |grep libboundscheck
   ```

3. 编译。

   ```bash
   $ cd etmem

   $ mkdir build

   $ cd build

   $ cmake ..

   $ make
   ```

## 注意事项

### 运行依赖

etmem作为内存扩展工具，需要依赖于内核态的特性支持，为了可以识别内存访问情况和支持主动将内存写入swap分区来达到内存垂直扩展的需求，etmem在运行时需要插入`etmem_scan`和`etmem_swap`模块：

```bash
modprobe etmem_scan
modprobe etmem_swap
```

### 权限限制

运行etmem进程需要root权限，root用户具有系统最高权限，在使用root用户进行操作时，请严格按照操作指导进行操作，避免其他操作造成系统管理及安全风险。

### 使用约束

* etmem的客户端和服务端需要在同一个服务器上部署，不支持跨服务器通信的场景。
* etmem仅支持扫描进程名小于或等于15个字符长度的目标进程。在使用进程名时，支持的进程名有效字符为：“字母”， “数字”，特殊字符“./%-\_”以及上述三种的组合，其余组合认为是非法字符。
* 在使用AEP介质进行内存扩展的时候，依赖于系统可以正确识别AEP设备并将AEP设备初始化为`numa node`。并且配置文件中的`vm_flags`字段只能配置为`ht`。
* 引擎私有命令仅针对对应引擎和引擎下的任务有效，比如cslide所支持的`showhostpages`和`showtaskpages`。
* 第三方策略实现代码中，`eng_mgt_func`接口中的`fd`不能写入`0xff`和`0xfe`字。
* 支持在一个工程内添加多个不同的第三方策略动态库，以配置文件中的`eng_name`来区分。
* 禁止并发扫描同一个进程。
* 未加载`etmem_scan`和`etmem_swap` ko时，禁止使用`/proc/xxx/idle_pages`和`/proc/xxx/swap_pages`文件。
* etmem对应配置文件，其权限要求为属主为root用户，且权限为600或400，配置文件大小不超过10M。
* etmem在进行第三方策略注入时，第三方策略的`so`权限要求为属主为root用户，且权限为500或700。

## 使用说明

### etmem配置文件

在运行etmem进程之前，需要管理员预先规划哪些进程需要做内存扩展，将进程信息配置到etmem配置文件中，并配置内存扫描的周期、扫描次数、内存冷热阈值等信息。

配置文件的示例文件在源码包中，放置在`/etc/etmem`文件路径下，按照功能划分为3个示例文件：

```text
/etc/etmem/cslide_conf.yaml
/etc/etmem/slide_conf.yaml
/etc/etmem/thirdparty_conf.yaml
```

示例内容分别为：

```sh
#slide引擎示例
#slide_conf.yaml
[project]
name=test
loop=1
interval=1
sleep=1
sysmem_threshold=50
swapcache_high_vmark=10
swapcache_low_vmark=6

[engine]
name=slide
project=test

[task]
project=test
engine=slide
name=background_slide
type=name
value=mysql
T=1
max_threads=1
swap_threshold=10g
swap_flag=yes

#cslide引擎示例
#cslide_conf.yaml
[engine]
name=cslide
project=test
node_pair=2,0;3,1
hot_threshold=1
node_mig_quota=1024
node_hot_reserve=1024

[task]
project=test
engine=cslide
name=background_cslide
type=pid
name=23456
vm_flags=ht
anon_only=no
ign_host=no

#thirdparty引擎示例
#thirdparty_conf.yaml
[engine]
name=thirdparty
project=test
eng_name=my_engine
libname=/usr/lib/etmem_fetch/my_engine.so
ops_name=my_engine_ops
engine_private_key=engine_private_value

[task]
project=test
engine=my_engine
name=background_third
type=pid
value=12345
task_private_key=task_private_value
```

配置文件各字段说明：

| 配置项       | 配置项含义               | 是否必需 | 是否有参数 | 参数范围       | 示例说明                                                            |
|-----------|---------------------|------|-------|------------|-----------------------------------------------------------------|
| \[project] | project公用配置段起始标识    | 否    | 否     | NA         | project参数的开头标识，表示下面的参数直到另外的\[xxx]或文件结尾为止的范围内均为project section的参数 |
| name      | project的名字          | 是    | 是     | 64个字以内的字符串 | 用来标识project，engine和task在配置时需要指定要挂载到的project                     |
| loop      | 内存扫描的循环次数           | 是    | 是     | 1~120      | loop=3 //扫描3次                                                   |
| interval  | 每次内存扫描的时间间隔         | 是    | 是     | 1~1200     | interval=5 //每次扫描之间间隔5s                                         |
| sleep     | 每个内存扫描+操作的大周期之间时间间隔 | 是    | 是     | 1~1200     | sleep=10 //每次大周期之间间隔10s                                         |
| sysmem\_threshold| slide engine的配置项，系统内存换出阈值 | 否    | 是     | 0~100     | sysmem\_threshold=50 //系统内存剩余量小于50%时，etmem才会触发内存换出|
| swapcache\_high\_wmark| slide engine的配置项，swacache可以占用系统内存的比例，高水线 | 否    | 是     | 1~100     | swapcache\_high\_wmark=5 //swapcache内存占用量可以为系统内存的5%，超过该比例，etmem会触发swapcache回收 注： swapcache\_high\_wmark需要大于swapcache\_low\_wmark|
| swapcache\_low\_wmark| slide engine的配置项，swacache可以占用系统内存的比例，低水线 | 否    | 是     | \[1~swapcache\_high\_wmark)     | swapcache\_low\_wmark=3 //触发swapcache回收后，系统会将swapcache内存占用量回收到低于3%|
| \[engine]      | engine公用配置段起始标识                           | 否                  | 否     | NA                                               | engine参数的开头标识，表示下面的参数直到另外的\[xxx]或文件结尾为止的范围内均为engine section的参数 |
| project       | 声明所在的project                              | 是                  | 是     | 64个字以内的字符串                                       | 已经存在名字为test的project，则可以写为project=test                        |
| engine        | 声明所在的engine                               | 是                  | 是     | slide/cslide/thirdparty                          | 声明使用的是slide或cslide或thirdparty策略                              |
| node\_pair     | cslide engine的配置项，声明系统中AEP和DRAM的node pair | engine为cslide时必须配置 | 是     | 成对配置AEP和DRAM的node号，AEP和DRAM之间用逗号隔开，每对pair之间用分号隔开 | node\_pair=2,0;3,1                                            |
| hot\_threshold | cslide engine的配置项，声明内存冷热水线的阈值             | engine为cslide时必须配置 | 是     | 大于等于0，小于等于INT\_MAX的整数                                          | hot\_threshold=3 //访问次数小于3的内存会被识别为冷内存                         |
|node\_mig\_quota|cslide engine的配置项，流控，声明每次DRAM和AEP互相迁移时单向最大流量|engine为cslide时必须配置|是|大于等于0，小于等于INT\_MAX的整数|node\_mig\_quota=1024 //单位为MB，AEP到DRAM或DRAM到AEP搬迁一次最大1024M|
|node\_hot\_reserve|cslide engine的配置项，声明DRAM中热内存的预留空间大小|engine为cslide时必须配置|是|大于等于0，小于等于INT\_MAX的整数|node\_hot\_reserve=1024 //单位为MB，当所有虚拟机热内存大于此配置值时，热内存也会迁移到AEP中|
|eng\_name|thirdparty engine的配置项，声明engine自己的名字，供task挂载|engine为thirdparty时必须配置|是|64个字以内的字符串|eng\_name=my\_engine //对此第三方策略engine挂载task时，task中写明engine=my\_engine|
|libname|thirdparty engine的配置项，声明第三方策略的动态库的地址，绝对地址|engine为thirdparty时必须配置|是|256个字以内的字符串|libname=/user/lib/etmem\_fetch/code\_test/my\_engine.so|
|ops\_name|thirdparty engine的配置项，声明第三方策略的动态库中操作符号的名字|engine为thirdparty时必须配置|是|256个字以内的字符串|ops\_name=my\_engine\_ops //第三方策略实现接口的结构体的名字|
|engine\_private\_key|thirdparty engine的配置项，预留给第三方策略自己解析私有参数的配置项，选配|否|否|根据第三方策略私有参数自行限制|根据第三方策略私有engine参数自行配置|
| \[task]  | task公用配置段起始标识 | 否 | 否 | NA          | task参数的开头标识，表示下面的参数直到另外的\[xxx]或文件结尾为止的范围内均为task section的参数 |
| project | 声明所挂的project  | 是 | 是 | 64个字以内的字符串  | 已经存在名字为test的project，则可以写为project=test                     |
| engine  | 声明所挂的engine   | 是 | 是 | 64个字以内的字符串  | 所要挂载的engine的名字                                            |
| name    | task的名字       | 是 | 是 | 64个字以内的字符串  | name=background1 //声明task的名字是backgound1                   |
| type    | 目标进程识别的方式     | 是 | 是 | pid/name    | pid代表通过进程号识别，name代表通过进程名称识别                               |
| value   | 目标进程识别的具体字段   | 是 | 是 | 实际的进程号/进程名称 | 与type字段配合使用，指定目标进程的进程号或进程名称，由使用者保证配置的正确及唯一性               |
| T                | engine为slide的task配置项，声明内存冷热水线的阈值                               | engine为slide时必须配置 | 是 | 0~loop \* 3           | T=3 //访问次数小于3的内存会被识别为冷内存                                        |
| max\_threads      | engine为slide的task配置项，etmemd内部线程池最大线程数，每个线程处理一个进程/子进程的内存扫描+操作任务 | 否                 | 是 | 1~2 \* core数 + 1，默认为1 | 对外部无表象，控制etmemd服务端内部处理线程个数，当目标进程有多个子进程时，配置越大，并发执行的个数也多，但占用资源也越多 |
| vm\_flags         | engine为cslide的task配置项，通过指定flag扫描的vma，不配置此项时扫描则不会区分             | 否                 | 是 | 256长度以内的字符串，不同flag以空格隔开           | vm\_flags=ht //扫描flags为ht（大页）的vma内存                              |
| anon\_only        | engine为cslide的task配置项，标识是否只扫描匿名页                               | 否                 | 是 | yes/no               | anon\_only=no //配置为yes时只扫描匿名页，配置为no时非匿名页也会扫描                     |
| ign\_host         | engine为cslide的task配置项，标识是否忽略host上的页表扫描信息                       | 否                 | 是 | yes/no               | ign\_host=no //yes为忽略，no为不忽略                                     |
| task\_private\_key | engine为thirdparty的task配置项，预留给第三方策略的task解析私有参数的配置项，选配           | 否                 | 否 | 根据第三方策略私有参数自行限制      | 根据第三方策略私有task参数自行配置                                             |
| swap\_threshold |slide engine的配置项，进程内存换出阈值           | 否                 | 是 | 进程可用内存绝对值      | swap\_threshold=10g //进程占用内存在低于10g时不会触发换出。当前版本下，仅支持g/G作为内存绝对值单位。与sysmem\_threshold配合使用，仅系统内存低于阈值时，进行白名单中进程阈值判断 |
| swap\_flag|slide engine的配置项，进程指定内存换出           | 否                 | 是 | yes/no      | swap\_flag=yes//使能进程指定内存换出 |

### etmemd服务端启动

在使用etmem提供的服务时，首先根据需要修改相应的配置文件，然后运行etmemd服务端，常驻在系统中来操作目标进程的内存。除了支持在命令行中通过二进制来启动etmemd的进程外，还可以通过配置`service`文件来使etmemd服务端通过`systemctl`方式拉起，此场景需要通过`mode-systemctl`参数来指定支持。

#### 使用方法

可以通过下列示例命令启动etmemd的服务端：

```bash
etmemd -l 0 -s etmemd_socket
```

或者：

```bash
etmemd --log-level 0 --socket etmemd_socket
```

其中`-l`的`0`和`-s`的`etmemd_socket`是用户自己输入的参数，参数具体含义参考以下列表：

#### 命令行参数说明

| 参数            | 参数含义                           | 是否必须 | 是否有参数 | 参数范围              | 示例说明                                                     |
| --------------- | ---------------------------------- | -------- | ---------- | --------------------- | ------------------------------------------------------------ |
| -l或--log-level | etmemd日志级别                     | 否       | 是         | 0~3                   | 0：debug级别   1：info级别   2：warning级别   3：error级别   只有大于等于配置的级别才会打印到/var/log/message文件中 |
| -s或--socket    | etmemd监听的名称，用于与客户端交互 | 是       | 是         | 107个字符之内的字符串 | 指定服务端监听的名称                                         |
| -m或--mode-systemctl| 指定通过systemctl方式来拉起stmemd服务| 否| 否| NA| service文件中需要指定-m参数|
| -h或--help      | 帮助信息                           | 否       | 否         | NA                    | 执行时带有此参数会打印后退出                                 |

### 通过etmem客户端添加或者删除工程/引擎/任务

#### 场景描述

1）管理员创建etmem的project/engine/task（一个工程可包含多个etmem engine，一个engine可以包含多个任务）。

2）管理员删除已有的etmem project/engine/task（删除工程前，会自动先停止该工程中的所有任务）。

#### 使用方法

在etmemd服务端正常运行后，通过etmem客户端，通过第二个参数指定为obj，来进行创建或删除动作，对project/engine/task则是通过配置文件中配置的内容来进行识别和区分。

* 添加对象：

  ```bash
  etmem obj add -f /etc/etmem/slide_conf.yaml -s etmemd_socket
  ```

  或

  ```bash
  etmem obj add --file /etc/etmem/slide_conf.yaml --socket etmemd_socket
  ```

* 删除对象：

  ```bash
  etmem obj del -f /etc/etmem/slide_conf.yaml -s etmemd_socket
  ```

  或

  ```bash
  etmem obj del --file /etc/etmem/slide_conf.yaml --socket etmemd_socket
  ```

#### 命令行参数说明

| 参数         | 参数含义                                                     | 是否必须 | 是否有参数 | 示例说明                                                 |
| ------------ | ------------------------------------------------------------ | -------- | ---------- | -------------------------------------------------------- |
| -f或--file   | 指定对象的配置文件                                        | 是      | 是         | 需要指定路径名称                                         |
| -s或--socket | 与etmemd服务端通信的socket名称，需要与etmemd启动时指定的保持一致 | 是      | 是         | 必须配置，在有多个etmemd时，由管理员选择与哪个etmemd通信 |

### 通过etmem客户端查询/启动/停止工程

#### 场景描述

在已经通过`etmem obj add`添加工程之后，在还未调用`etmem obj del`删除工程之前，可以对etmem的工程进行启动和停止。

1）管理员启动已添加的工程。

2）管理员停止已启动的工程。

在管理员调用`obj del`删除工程时，如果工程已经启动，则会自动停止。

#### 使用方法

对于已经添加成功的工程，可以通过`etmem project`的命令来控制工程的启动和停止，命令示例如下：

* 查询工程：

  ```bash
  etmem project show -n test -s etmemd_socket
  ```

  或

  ```bash
  etmem project show --name test --socket etmemd_socket
  ```

* 启动工程：

  ```bash
  etmem project start -n test -s etmemd_socket
  ```

  或

  ```bash
  etmem project start --name test --socket etmemd_socket
  ```

* 停止工程：

  ```bash
  etmem project stop -n test -s etmemd_socket
  ```

  或

  ```bash
  etmem project stop --name test --socket etmemd_socket
  ```

* 打印帮助：

  ```bash
  etmem project help
  ```

#### 命令行参数说明

| 参数         | 参数含义                                                     | 是否必须 | 是否有参数 | 示例说明                                                 |
| ------------ | ------------------------------------------------------------ | -------- | ---------- | -------------------------------------------------------- |
| -n或--name   | 指定project名称                                              | 是      | 是         | project名称，与配置文件一一对应                          |
| -s或--socket | 与etmemd服务端通信的socket名称，需要与etmemd启动时指定的保持一致 | 是     | 是         | 必须配置，在有多个etmemd时，由管理员选择与哪个etmemd通信 |

### 通过etmem客户端，支持内存阈值换出以及指定内存换出

当前支持的策略中，只有slide策略支持私有的功能特性。

* 进程或系统内存阈值换出。

为了获得业务的极致性能，需要考虑etmem内存扩展进行内存换出的时机；当系统可用内存足够，系统内存压力不大时，不进行内存交换；当进程占用内存不高时，不进行内存交换；提供系统内存换出阈值控制以及进程内存换出阈值控制。

* 进程指定内存换出。

在存储环境下，具有IO时延敏感型业务进程，上述进程内存不希望进行换出，因此提供一种机制，由业务指定可换出内存。

针对进程或系统内存阈值换出，进程指定内存换出功能，可以在配置文件中添加`sysmem_threshold`，`swap_threshold`，`swap_flag`参数，示例如下，具体含义请参考etmem配置文件说明章节。

```sh
#slide_conf.yaml
[project]
name=test
loop=1
interval=1
sleep=1
sysmem_threshold=50

[engine]
name=slide
project=test

[task]
project=test
engine=slide
name=background_slide
type=name
value=mysql
T=1
max_threads=1
swap_threshold=10g
swap_flag=yes
```

#### 系统内存阈值换出

配置文件中`sysmem_threshold`用于指示系统内存阈值换出功能，`sysmem_threshold`取值范围为0-100，如果配置文件中设定了`sysmem_threshold`，那么只有系统内存剩余量低于该比例时，etmem才会触发内存换出流程。

示例使用方法如下：

1. 参考示例编写配置文件，配置文件中填写`sysmem_threshold`参数，例如`sysmem_threshold=20`。

2. 启动服务端，并通过服务端添加，启动工程。

   ```bash
   etmemd -l 0 -s monitor_app &
   etmem obj add -f etmem_config -s monitor_app
   etmem project start -n test -s monitor_app
   etmem project show -s monitor_app
   ```

3. 观察内存换出结果，只有系统可用内存低于20%时，etmem才会触发内存换出。

#### 进程内存阈值换出

配置文件中`swap_threshold`用于指示进程内存阈值换出功能，`swap_threshold`为进程内存占用量绝对值(格式为"数字+单位g/G")，如果配置文件中设定了`swap_threshold`，那么该进程内存占用量在小于该设定的可用内存量时，etmem不会针对该进程触发换出流程。

示例使用方法如下：

1. 参考示例编写配置文件，配置文件中填写`swap_threshold`参数，例如`swap_threshold=5g`。

2. 启动服务端，并通过服务端添加，启动工程。

   ```bash
   etmemd -l 0 -s monitor_app &
   etmem obj add -f etmem_config -s monitor_app
   etmem project start -n test -s monitor_app
   etmem project show -s monitor_app
   ```

3. 观察内存换出结果，只有进程占用内存绝对值高于5G时，etmem才会触发内存换出。

#### 进程指定内存换出

配置文件中`swap_flag`用于指示进程指定内存换出功能，`swap_flag`取值仅有两个：`yes/no`，如果配置文件中设定了`swap_flag`为no或者未配置，那么etmem换出功能无变化，如果`swap_flag`设定为yes，那么etmem仅仅换出进程指定的内存。

示例使用方法如下：

1. 参考示例编写配置文件，配置文件中填写`swap_flag`参数，例如`swap_flag=yes`。

2. 业务进程对需要进行换出的内存打标记。

   ```bash
   madvise(addr_start, addr_len, MADV_SWAPFLAG)
   ```

3. 启动服务端，并通过服务端添加，启动工程。

   ```bash
   etmemd -l 0 -s monitor_app &
   etmem obj add -f etmem_config -s monitor_app
   etmem project start -n test -s monitor_app
   etmem project show -s monitor_app
   ```

4. 观察内存换出结果，只有进程打标记的部分内存会被换出，其余内存保留在DRAM中，不会被换出。

针对进程指定页面换出的场景中，在原扫描接口`idle_pages`中添加`ioctl`命令字的形式，来确认不带有特定标记的vma不进行扫描与换出操作。

扫描管理接口：

* 函数原型。

  ```c
  ioctl(fd, cmd, void *arg);
  ```

* 输入参数。
  1. fd:文件描述符，通过open调用在/proc/pid/idle\_pages下打开文件获得。

  2. cmd:控制扫描行为，当前支持如下cmd：
     VMA\_SCAN\_ADD\_FLAGS:新增vma指定内存换出标记，仅扫描带有特定标记的VMA。
     VMA\_SCAN\_REMOVE\_FLAGS:删除新增的VMA指定内存换出标记。

  3. args:int指针参数，传递具体标记掩码，当前仅支持如下参数：
     VMA\_SCAN\_FLAG：在etmem\_scan.ko扫描模块开始扫描前，会调用接口walk\_page\_test接口判断vma地址是否符合扫描要求，此标记置位时，会仅扫描带有特定换出标记的vma地址段，而忽略其他vma地址。

* 返回值。
  1. 成功，返回0。
  2. 失败，返回非0。

* 注意事项。\
  所有不支持的标记都会被忽略，但是不会返回错误。

### 通过etmem客户端，支持swapcache内存回收指令

用户态etmem发起内存淘汰回收操作，通过`write procfs`接口与内核态的内存回收模块交互，内存回收模块解析用户态下发的虚拟地址，获取地址对应的page页面，并调用内核原生接口将该page对应内存进行换出回收，在内存换出的过程中，swapcache会占用部分系统内存，为进一步节约内存，添加swapcache内存回收功能。

针对swapcache内存回收功能，可以在配置文件中添加`swapcache_high_wmark`，`swapcache_low_wmark`参数。

* `swapcache_high_wmark`: swapcache可以占用系统内存的高水位线。
* `swapcache_low_wmark`：swapcache可以占用系统内存的低水位线。

在etmem进行一轮内存换出后，会进行swapcache占用系统内存比例的检查，当占用比例超过高水位线后，会通过`swap_pages`下发`ioctl`命令，触发swapcache内存回收，并回收到低水位线停止。

配置参数示例如下，具体请参考etmem配置文件相关章节：

```sh
#slide_conf.yaml
[project]
name=test
loop=1
interval=1
sleep=1
swapcache_high_vmark=5
swapcache_low_vmark=3

[engine]
name=slide
project=test

[task]
project=test
engine=slide
name=background_slide
type=name
value=mysql
T=1
max_threads=1
```

针对swap换出场景中，需要通过swapcache内存回收进一步节约内存，在原内存换出接口`swap_pages`中通过添加`ioctl`接口的方式，来提供swapcache水线的设定以及swapcache内存占用量回收的启动与关闭。

* 函数原型。

  ```c
  ioctl(fd, cmd, void *arg);
  ```

* 输入参数。

  ```text
  1. fd:文件描述符，通过open调用在/proc/pid/idle_pages下打开文件获得

  2. cmd:控制扫描行为，当前支持如下cmd：
  RECLAIM_SWAPCACHE_ON:启动swapcache内存换出
  RECLAIM_SWAPCACHE_OFF:关闭swapcache内存换出
  SET_SWAPCACHE_WMARK:设定swapcache内存水线

  3. args:int指针参数，传递具体标记掩码，当前仅支持如下参数：
  参数用来传递swapcache水线具体值
  ```

* 返回值。

  ```text
  1. 成功，返回0。
  2. 失败，返回非0。
  ```

* 注意事项

  ```text
  所有不支持的标记都会被忽略，但是不会返回错误
  ```

### 通过etmem客户端，执行引擎私有命令或功能

当前支持的策略中，只有cslide策略支持私有的命令。

* `showtaskpages`。
* `showhostpages`。

针对使用此策略引擎的engine和engine所有的task，可以通过这两个命令分别查看task相关的页面访问情况和虚拟机的host上系统大页的使用情况。

示例命令如下：

```bash
etmem engine showtaskpages <-t task_name> -n proj_name -e cslide -s etmemd_socket

etmem engine showhostpages -n proj_name -e cslide -s etmemd_socket
```

**注意** ：`showtaskpages`和`showhostpages`仅支持引擎使用cslide的场景。

#### 命令行参数说明

| 参数 | 参数含义 | 是否必须 | 是否有参数 | 示例说明 |
|----|------|------|-------|------|
|-n或--proj\_name| 指定project的名字| 是| 是| 指定已经存在，所需要执行的project的名字|
|-s或--socket| 与etmemd服务端通信的socket名称，需要与etmemd启动时指定的保持一致| 是| 是| 必须配置，在有多个etmemd时，由管理员选择与哪个etmemd通信|
|-e或--engine| 指定执行的引擎的名字| 是| 是| 指定已经存在的，所需要执行的引擎的名字|
|-t或--task\_name| 指定执行的任务的名字| 否| 是| 指定已经存在的，所需要执行的任务的名字|

### 支持kernel swap功能开启与关闭

针对swap换出到磁盘场景，当etmem用于内存扩展时，用户可以选择是否同时开启内核swap功能。用户可以关闭内核原生swap机制，以免原生swap机制换出不应被换出的内存，导致用户态进程出现问题。

通过提供sys接口实现上述控制，在`/sys/kernel/mm/swap`目录下创建`kobj`对象，对象名为`kernel_swap_enable`，默认为`true`，用于控制kernel swap的启动与关闭。

具体示例如下：

```sh
#开启kernel swap
echo true > /sys/kernel/mm/swap/kernel_swap_enbale
或者
echo 1 > /sys/kernel/mm/swap/kernel_swap_enbale

#关闭kernel swap
echo false > /sys/kernel/mm/swap/kernel_swap_enbale
或者
echo 0 > /sys/kernel/mm/swap/kernel_swap_enbale

```

### etmem支持随系统自启动

#### 场景描述

etmemd支持由用户配置`systemd`配置文件后，以`fork`模式作为`systemd`服务被拉起运行。

#### 使用方法

编写`service`配置文件，来启动etmemd，必须使用-m参数来指定此模式，例如：

```bash
etmemd -l 0 -s etmemd_socket -m
```

#### 命令行参数说明

| 参数             | 参数含义       | 是否必须 | 是否有参数 | 参数范围 | 示例说明      |
|----------------|------------|------|-------|------|-----------|
| -l或--log-level | etmemd日志级别 | 否    | 是     | 0~3  | 0：debug级别；1：info级别；2：warning级别；3：error级别；只有大于等于配置的级别才会打印到/var/log/message文件中|
| -s或--socket    | etmemd监听的名称，用于与客户端交互 | 是       | 是         | 107个字符之内的字符串 | 指定服务端监听的名称                                         |
| -m或--mode-systemctl| 指定通过systemctl方式来拉起stmemd服务| 否| 否| NA| service文件中需要指定-m参数|
| -h或--help      | 帮助信息                           | 否       | 否         | NA                    | 执行时带有此参数会打印后退出                                 |

### etmem支持第三方内存扩展策略

#### 场景描述

etmem支持用户注册第三方内存扩展策略，同时提供扫描模块动态库，运行时通过第三方策略淘汰算法淘汰内存。

用户使用etmem所提供的扫描模块动态库并实现对接etmem所需要的结构体中的接口。

#### 使用方法

用户使用自己实现的第三方扩展淘汰策略，主要需要按下面步骤进行实现和操作：

1. 按需调用扫描模块提供的扫描接口。

2. 按照etmem头文件中提供的函数模板来实现各个接口，最终封装成结构体。

3. 编译出第三方扩展淘汰策略的动态库。

4. 在配置文件中按要求声明类型为thirdparty的engine。

5. 将动态库的名称和接口结构体的名称按要求填入配置文件中task对应的字段。

其他操作步骤与使用etmem的其他engine类似。

接口结构体模板：

```c
struct engine_ops {

/* 针对引擎私有参数的解析，如果有，需要实现，否则置NULL */

int (*fill_eng_params)(GKeyFile *config, struct engine *eng);

/* 针对引擎私有参数的清理，如果有，需要实现，否则置NULL */

void (*clear_eng_params)(struct engine *eng);

/* 针对任务私有参数的解析，如果有，需要实现，否则置NULL */

int (*fill_task_params)(GKeyFile *config, struct task *task);

/* 针对任务私有参数的清理，如果有，需要实现，否则置NULL */

void (*clear_task_params)(struct task *tk);

/* 启动任务的接口 */

int (*start_task)(struct engine *eng, struct task *tk);

/* 停止任务的接口 */

void (*stop_task)(struct engine *eng, struct task *tk);

/* 填充pid相关私有参数 */

int (*alloc_pid_params)(struct engine *eng, struct task_pid **tk_pid);

/* 销毁pid相关私有参数 */

void (*free_pid_params)(struct engine *eng, struct task_pid **tk_pid);

/* 第三方策略自身所需要的私有命令支持，如果没有，置为NULL */

int (*eng_mgt_func)(struct engine *eng, struct task *tk, char *cmd, int fd);

};
```

扫描模块对外接口说明：

| 接口名称         |接口描述|
| ------------ | --------------------- |
| etmemd\_scan\_init | scan模块初始化|
| etmemd\_scan\_exit | scan模块析构|
| etmemd\_get\_vmas | 获取需要扫描的vma|
| etmemd\_free\_vmas | 释放etmemd\_get\_vmas扫描到的vma|
| etmemd\_get\_page\_refs | 扫描vmas中的页面|
| etmemd\_free\_page\_refs | 释放etmemd\_get\_page\_refs获取到的页访问信息链表|

针对扫描虚拟机的场景中，在原扫描接口`idle_pages`中添加`ioctl`接口的方式，来提供区分扫描`ept`的粒度和是否忽略host上页访问标记的机制。

针对进程指定页面换出的场景中，在原扫描接口`idle_pages`中添加`ioctl`命令字的形式，来确认不带有特定标记的vma不进行扫描和换出操作。

扫描管理接口:

* 函数原型。

  ```c
  ioctl(fd, cmd, void *arg);
  ```

* 输入参数。

  ```text
  1. fd:文件描述符，通过open调用在/proc/pid/idle_pages下打开文件获得

  2. cmd:控制扫描行为，当前支持如下cmd：
  IDLE_SCAN_ADD_FLAG：新增一个扫描标记
  IDLE_SCAM_REMOVE_FLAGS：删除一个扫描标记
  VMA_SCAN_ADD_FLAGS:新增vma指定内存换出标记，仅扫描带有特定标记的VMA
  VMA_SCAN_REMOVE_FLAGS:删除新增的VMA指定内存换出标记

  3. args:int指针参数，传递具体标记掩码，当前仅支持如下参数：
  SCAN_AS_HUGE:扫描ept页表时，按照2M大页粒度扫描页是否被访问过。此标记未置位时，按照ept页表自身粒度扫描
  SCAN_IGN_HUGE：扫描虚拟机时，忽略host侧页表上的访问标记。此标记未置位时，不会忽略host侧页表上的访问标记。
  VMA_SCAN_FLAG：在etmem_scan.ko扫描模块开始扫描前，会调用接口walk_page_test接口判断vma地址是否符合扫描要求，此标记置位时，会仅扫描带有特定换出标记的vma地址段，而忽略其他vma地址
  ```

* 返回值。

  ```text
  1. 成功，返回0
  2. 失败，返回非0
  ```

* 注意事项。

  ```text
  所有不支持的标记都会被忽略，但是不会返回错误
  ```

配置文件示例如下所示，具体含义请参考配置文件说明章节：

```sh
#thirdparty
[engine]

name=thirdparty

project=test

eng_name=my_engine

libname=/user/lib/etmem_fetch/code_test/my_engine.so

ops_name=my_engine_ops

engine_private_key=engine_private_value

[task]

project=test

engine=my_engine

name=background1

type=pid

value=1798245

task_private_key=task_private_value
```

**注意** ：

1、用户需使用etmem所提供的扫描模块动态库并实现对接etmem所需要的结构体中的接口。

2、`eng_mgt_func`接口中的`fd`不能写入`0xff`和`0xfe`字。

3、支持在一个工程内添加多个不同的第三方策略动态库，以配置文件中的`eng_name`来区分。

### etmem客户端和服务端帮助说明

通过下列命令可以打印etmem服务端帮助说明：

```bash
etmemd -h
```

或：

```bash
etmemd --help
```

通过下列命令可以打印etmem客户端帮助说明：

```bash
etmem help
```

通过下列命令可以打印etmem客户端操作工程/引擎/任务相关帮助说明：

```bash
etmem obj help
```

通过下列命令可以打印etmem客户端对项目相关帮助说明：

```bash
etmem project help
```

---

---
url: /en/docs/22.03_LTS_SP4/server/memory_storage/etmem/etmem_user_guide.md
---
# etmem User Guide

## Introduction

The development of CPU computing power, particularly lower costs of ARM cores, makes memory cost and capacity become the core frustration that restricts business costs and performance. Therefore, the most pressing issue is how to save memory cost and how to expand memory capacity.

etmem is a tiered memory expansion technology that uses DRAM+memory compression/high-performance storage media to form tiered memory storage. Memory data is tiered, and cold data is migrated from memory media to high-performance storage media to release memory space and reduce memory costs.

The tools provided by the etmem software package include the etmem client and the etmemd server. etmemd runs continuously after being launched and implements functions such as recognition and elimination of cold and hot memory of target processes. etmem runs once when called and controls etmemd to respond with different operations based on different command parameters.

## Compilation

1. Download the etmem source code.

   ```shell
   git clone https://atomgit.com/openeuler/etmem.git
   ```

2. Install the compilation and running dependency. The compilation and running of etmem depend on the libboundscheck component.

   Install the dependency:

   ```bash
   yum install libboundscheck
   ```

   Use the `rpm` command to check if the package is installed:

   ```bash
   rpm -qi libboundscheck
   ```

3. Build source code.

   ```bash
   cd etmem
   mkdir build
   cd build
   cmake ..
   make
   ```

## Precautions

### Dependencies

As a memory expansion tool, etmem needs to rely on kernel features. To identify memory access conditions and support the active writing of memory into the swap partition to achieve the requirement of vertical memory expansion, etmem needs to insert the **etmem\_scan** and **etmem\_swap** modules at runtime:

```bash
modprobe etmem_scan
modprobe etmem_swap
```

### Restrictions

The etmem process requires root privileges. The root user has the highest system privileges. When using the root user to perform operations, strictly follow the operation instructions to avoid system management and security risks.

### Constraints

* The client and server of etmem must be deployed on the same server. Cross-server communication is not supported.
* etmem can scan target processes whose process name is less than or equal to 15 characters. Supported characters in process names are letters, numbers, periods (.), slashes (/), hyphens (-), and underscores (\_).
* When AEP media is used for memory expansion, it relies on the system being able to correctly recognize the AEP device and initialize the device as a NUMA node. Additionally, the **vm\_flags** field in the configuration file can only be configured as **ht**.
* The private commands of the engine are only valid for the corresponding engine and tasks under the engine, such as `showhostpages` and `showtaskpages` supported by cslide.
* In a third-party policy implementations, **fd** in the `eng_mgt_func` interface cannot be written with the **0xff** and **0xfe** characters.
* Multiple different third-party policy dynamic libraries, distinguished by **eng\_name** in the configuration file, can be added within a project.
* Concurrent scanning of the same process is prohibited.
* Using the **/proc/xxx/idle\_pages** and **/proc/xxx/swap\_pages** files is prohibited when **etmem\_scan** and **etmem\_swap** modules are not loaded.
* The etmem configuration file requires the owner to be the root user, with permissions of 600 or 400. The size of the configuration file cannot exceed 10 MB.
* When etmem injects a third-party policy, the **so** of the third-party policy requires the owner to be the root user, with permissions of 500 or 700.

## Instructions

### etmem Configuration Files

Before running the etmem process, the administrator needs to decide the memory of which processes needs to be expanded, configure the process information in the etmem configuration files, and configure information such as the memory scanning cycle, scanning times, and  memory hot and cold thresholds.

The configuration file examples are included in the source package and stored in the **/etc/etmem** directory. There are three example files:

```text
/etc/etmem/cslide_conf.yaml
/etc/etmem/slide_conf.yaml
/etc/etmem/thirdparty_conf.yaml
```

Contents of the files are as follows:

```sh
#slide engine example
#slide_conf.yaml
[project]
name=test
loop=1
interval=1
sleep=1
sysmem_threshold=50
swapcache_high_vmark=10
swapcache_low_vmark=6

[engine]
name=slide
project=test

[task]
project=test
engine=slide
name=background_slide
type=name
value=mysql
T=1
max_threads=1
swap_threshold=10g
swap_flag=yes

#cslide engine example
#cslide_conf.yaml
[engine]
name=cslide
project=test
node_pair=2,0;3,1
hot_threshold=1
node_mig_quota=1024
node_hot_reserve=1024

[task]
project=test
engine=cslide
name=background_cslide
type=pid
name=23456
vm_flags=ht
anon_only=no
ign_host=no

#Third-party engine example
#thirdparty_conf.yaml
[engine]
name=thirdparty
project=test
eng_name=my_engine
libname=/usr/lib/etmem_fetch/my_engine.so
ops_name=my_engine_ops
engine_private_key=engine_private_value

[task]
project=test
engine=my_engine
name=background_third
type=pid
value=12345
task_private_key=task_private_value
```

Fields in the configuration files are described as follows:

| Item                 | Description                                                                                                                                                                        | Mandatory                             | Contains Parameters | Parameter Range                                                                                                      | Example                                                                                                                                                                                                                                                                                                           |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| \[project]          | Beginning identifier of the project public configuration section                                                                                                                   | No                                    | No                  | N/A                                                                                                                  | Beginning identifier of the project parameters, indicating that the parameters below are within the range of the project section until another \[xxx] or the end of the file                                                                                                                                     |
| name                 | Name of the project                                                                                                                                                                | Yes                                   | Yes                 | String of up to 64 characters                                                                                        | Specifies that the project, engine and task need to be mounted to the specified project during configuration.                                                                                                                                                                                                     |
| loop                 | Number of loops for memory scan                                                                                                                                                    | Yes                                   | Yes                 | 1~120                                                                                                                | loop=3 // Memory is scanned 3 times.                                                                                                                                                                                                                                                                              |
| interval             | Time interval for each memory scan                                                                                                                                                 | Yes                                   | Yes                 | 1~1200                                                                                                               | interval=5 // The interval is 5s.                                                                                                                                                                                                                                                                                 |
| sleep                | Time interval for each memory scan+operation                                                                                                                                       | Yes                                   | Yes                 | 1~1200                                                                                                               | sleep=10 //The interval is 10s                                                                                                                                                                                                                                                                                    |
| sysmem\_threshold     | Memory swapping threshold. This is a slide engine configuration item.                                                                                                              | No                                    | Yes                 | 0~100                                                                                                                | sysmem\_threshold=50 // When available memory is less than 50%, etmem swaps out memory.                                                                                                                                                                                                                            |
| swapcache\_high\_wmark | High watermark of swapcache. This is a slide engine configuration item.                                                                                                            | No                                    | Yes                 | 1~100                                                                                                                | swapcache\_high\_wmark=5 // swapcache can be up to 5% of the system memory. If this ratio is reached, etmem triggers swapcache recycling. Note: swapcache\_high\_wmark must be greater than swapcache\_low\_wmark.                                                                                                 |
| swapcache\_low\_wmark  | Low watermark of swapcache. This is a slide engine configuration item.                                                                                                             | No                                    | Yes                 | \[1~swapcache\_high\_wmark)                                                                                           | swapcache\_low\_wmark=3 //When swapcache recycling is triggered, the system recycles the swapcache memory occupancy to less than 3%.                                                                                                                                                                                |
| \[engine]           | Beginning identifier of the engine public configuration section                                                                                                                    | No                                    | No                  | N/A                                                                                                                  | Beginning identifier of the engine parameters, indicating that the parameters below are within the range of the engine section until another \[xxx] or the end of the file                                                                                                                                       |
| project              | project to which the engine belongs                                                                                                                                                | Yes                                   | Yes                 | String of up to 64 characters                                                                                        | If a project named test exists, the item can be **project=test**.                                                                                                                                                                                                                                                 |
| engine               | engine to which the engine belongs                                                                                                                                                 | Yes                                   | Yes                 | slide/cslide/thirdparty                                                                                              | Specifies the policy to use (**slide**, **cslide**, or **thirdparty**)                                                                                                                                                                                                                                            |
| node\_pair            | Node pair of AEP and DRAM. This is a cslide engine configuration item.                                                                                                             | Yes when **engine** is **cslide**     | Yes                 | Pair the node numbers of AEP and DRAM. Separate AEP and DRAM using a comma, and separate each pair using semicolons. | node\_pair=2,0;3,1                                                                                                                                                                                                                                                                                                 |
| hot\_threshold        | Threshold of hot memory watermark. This is a cslide engine configuration item.                                                                                                     | Yes when **engine** is **cslide**     | Yes                 | An integer greater than or equal to 0 and less than or equal to INT\_MAX                                              | hot\_threshold=3 // Memory with less than 3 accesses will be recognized as cold memory.                                                                                                                                                                                                                            |
| node\_mig\_quota       | Maximum one-way flow when DRAM and AEP migrate mutually. This is a cslide engine configuration item.                                                                               | Yes when **engine** is **cslide**     | Yes                 | An integer greater than or equal to 0 and less than or equal to INT\_MAX                                              | node\_mig\_quota=1024 // The unit is MB. A maximum of 1024 MB can be migrated from AEP to DRAM or from DRAM to AEP each time.                                                                                                                                                                                       |
| node\_hot\_reserve     | Size of the reserved space for hot memory in DRAM. This is a cslide engine configuration item.                                                                                     | Yes when **engine** is **cslide**     | Yes                 | An integer greater than or equal to 0 and less than or equal to INT\_MAX                                              | node\_hot\_reserve=1024 //The unit is MB. When the hot memory of all VMs is greater than this configuration value, the hot memory will also be migrated to AEP.                                                                                                                                                     |
| eng\_name             | Name of the engine for mounting by task. This is a third-party engine configuration item.                                                                                          | Yes when **engine** is **thirdparty** | Yes                 | String of up to 64 characters                                                                                        | eng\_name=my\_engine // When mounting a task to the third-party policy engine, specify **engine=my\_engine** in the task.                                                                                                                                                                                            |
| libname              | Absolute path to the dynamic library of the third-party policy. This is a third-party engine configuration item.                                                                   | Yes when **engine** is **thirdparty** | Yes                 | String of up to 256 characters                                                                                       | libname=/user/lib/etmem\_fetch/code\_test/my\_engine.so                                                                                                                                                                                                                                                              |
| ops\_name             | Name of the operator in the dynamic library of the third-party policy. This is a third-party engine configuration item.                                                            | Yes when **engine** is **thirdparty** | Yes                 | String of up to 256 characters                                                                                       | ops\_name=my\_engine\_ops // Name of the struct for the third-party policy implementation interface                                                                                                                                                                                                                  |
| engine\_private\_key   | Reserved item for third-party policies to parse private parameters by themselves. This is a third-party engine configuration item.                                                 | No                                    | No                  | Restrict according to the third-party policy's private parameters.                                                   | Configure the private engine parameters according to the third-party policy.                                                                                                                                                                                                                                      |
| \[task]             | Beginning identifier of the task public configuration section                                                                                                                      | No                                    | No                  | N/A                                                                                                                  | Beginning identifier of the task parameters, indicating that the parameters below are within the range of the project section until another \[xxx] or the end of the file                                                                                                                                        |
| project              | project to which the task belongs                                                                                                                                                  | Yes                                   | Yes                 | String of up to 64 characters                                                                                        | If a project named test exists, the item can be **project=test**.                                                                                                                                                                                                                                                 |
| engine               | engine to which the task belongs                                                                                                                                                   | Yes                                   | Yes                 | String of up to 64 characters                                                                                        | Name of the engine to which the task belongs                                                                                                                                                                                                                                                                      |
| name                 | Name of the task                                                                                                                                                                   | Yes                                   | Yes                 | String of up to 64 characters                                                                                        | name=background1 // The name of the task is background1.                                                                                                                                                                                                                                                          |
| type                 | How the target process is identified                                                                                                                                               | Yes                                   | Yes                 | pid/name                                                                                                             | **pid** specifies to identify by PID. **name** specifies to identify by name.                                                                                                                                                                                                                                     |
| value                | Value to be identified for the target process                                                                                                                                      | Yes                                   | Yes                 | Actual PID/name                                                                                                      | Used with **type** to specify the PID or name of the target process. Ensure the configuration is correct and unique.                                                                                                                                                                                              |
| T                    | Threshold of hot memory watermark. This is a slide engine configuration item.                                                                                                      | Yes when **engine** is **slide**      | Yes                 | 0~loop \* 3                                                                                                           | T=3 // Memory with less than 3 accesses will be recognized as cold memory.                                                                                                                                                                                                                                        |
| max\_threads          | Maximum number of threads in the etmem internal thread pool, with each thread handling a process/subprocess memory scan+operation task. This is a slide engine configuration item. | No                                    | Yes                 | 1~2 \* number of cores + 1, the default value is 1.                                                                   | Controls the number of internal processing threads for the etmemd server without external representation. When the target process has multiple child processes, the larger the item value, the more concurrent executions, but the more resources consumed.                                                       |
| vm\_flags             | Flag of the VMA to be scanned. This is a cslide engine configuration item.                                                                                                         | No                                    | Yes                 | String of up to 256 characters, with different flags separated by spaces.                                            | vm\_flags=ht // Scans memory of the VMA whose flag is ht.                                                                                                                                                                                                                                                          |
| anon\_only            | Scans anonymous pages only. This is a cslide engine configuration item.                                                                                                            | No                                    | Yes                 | yes/no                                                                                                               | anon\_only=no                                                                                                                                                                                                                                                                                                      |
| ign\_host             | Ignores page table scan information on the host. This is a cslide engine configuration item.                                                                                       | No                                    | Yes                 | yes/no                                                                                                               | ign\_host=no                                                                                                                                                                                                                                                                                                       |
| task\_private\_key     | Reserved for a task of a third-party policy to parse private parameters. This is a third-party engine configuration item.                                                          | No                                    | No                  | Restrict according to the third-party policy's private parameters.                                                   | Configure the private task parameters according to the third-party policy.                                                                                                                                                                                                                                        |
| swap\_threshold       | Process memory swapping threshold. This is a slide engine configuration item.                                                                                                      | No                                    | Yes                 | Absolute value of memory available to the process                                                                    | swap\_threshold=10g // Memory swapping will not be triggered when the process memory is less than 10 GB. Currently, the unit can only be **g** or **G**. This item is used with **sysmem\_threshold**. When system memory is lower than **sysmem\_threshold**, memory of processes in the allowlist is checked. |
| swap\_flag            | Enables process memory swapping. This is a slide engine configuration item.                                                                                                        | No                                    | Yes                 | yes/no                                                                                                               | swap\_flag=yes                                                                                                                                                                                                                                                                                                     |

### Starting etmemd

Modify related configuration files before using etmem services. After being started, etmemd stays in the system to operate the memory of the target processes.To start etmemd, you can either run the `etmemd` command or configure a service file for `systemctl` to start etmemd. The latter requires the `mode-systemctl` option.

#### How to Use

Run the following command to start etmemd:

```bash
etmemd -l 0 -s etmemd_socket
```

or

```bash
etmemd --log-level 0 --socket etmemd_socket
```

The `0` parameter of option `-l` and the `etmemd_socket` parameter of option `-s` are user-defined parameters and are described as follows.

#### Command Parameters

| Option            | Description                           | Mandatory |  Contains Parameters | Parameter Range       | Example                                                            |
| --------------- | ---------------------------------- | -------- | ---------- | --------------------- | ------------------------------------------------------------ |
| -l or --log-level | etmemd log level                      | No       | Yes         | 0~3                   | 0: debug level   1: info level   2: warning level   3: error level   Logs whose levels are higher than the specified value are printed to **/var/log/message**. |
| -s or --socket    | Socket listened by etmemd to interact with the client | Yes       | Yes         | String of up to 107 characters | Socket listened by etmemd                                         |
| -m or --mode-systemctl| Starts the etmemd service through systemctl |   No| No| N/A|    The `-m` option needs to be specified in the service file.|
| -h or --help      | Prints help information                           | No       | No         | N/A                    | This option prints help information and exit.                                 |

### Adding and Deleting Projects, Engines, and Tasks Using the etmem Client

#### Scenario

1. The administrator adds a project, engine, or task to etmem (a project can contain multiple etmem engines, an engine can contain multiple tasks).

2. The administrator deletes an existing etmem project, engine, or task (all tasks in a project is stopped before the project is deleted).

#### Usage

When etmemd is running normally, run `etmem` with the `obj` option to perform addition and deletion. etmem automatically identifies projects, engines, or tasks according to the content of the configuration file.

* Add an object.

  ```bash
  etmem obj add -f /etc/etmem/slide_conf.yaml -s etmemd_socket
  ```

  or

  ```bash
  etmem obj add --file /etc/etmem/slide_conf.yaml --socket etmemd_socket
  ```

* Delete an object.

  ```bash
  etmem obj del -f /etc/etmem/slide_conf.yaml -s etmemd_socket
  ```

  or

  ```bash
  etmem obj del --file /etc/etmem/slide_conf.yaml --socket etmemd_socket
  ```

#### Command Parameters

| Option           | Description                                                                                                    | Mandatory | Contains Parameters | Parameter Range                                                                                       | Example |
| ---------------- | -------------------------------------------------------------------------------------------------------------- | --------- | ------------------- | ----------------------------------------------------------------------------------------------------- | ------- |
| -f or --file   | Specifies the configuration file of the object.                                                                | Yes       | Yes                 | Specify the path.                                                                                     |         |
| -s or --socket | Socket used for communication with etmemd, which must be the same as the one specified when etmemd is started. | Yes       | Yes                 | The administrator can use this option to specify an etmemd server when multiple etmemd servers exist. |         |

### Querying, Starting, and Stopping Projects Using the etmem Client

#### Scenario

A project is added by using `etmem obj add` and is not deleted by using `etmem obj del`. In this case,  the project can be started and stopped.

1. The administrator starts an added project.

2. The administrator stops a started project.

A started project will be stopped if the administrator run `obj del` to delete the project.

#### Usage

Added projects can be started and stopped by using `etmem project` commands.

* Query a project.

  ```bash
  etmem project show -n test -s etmemd_socket
  ```

  or

  ```bash
  etmem project show --name test --socket etmemd_socket
  ```

* Start a project.

  ```bash
  etmem project start -n test -s etmemd_socket
  ```

  or

  ```bash
  etmem project start --name test --socket etmemd_socket
  ```

* Stop a project.

  ```bash
  etmem project stop -n test -s etmemd_socket
  ```

  or

  ```bash
  etmem project stop --name test --socket etmemd_socket
  ```

* Print help information.

  ```bash
  etmem project help
  ```

#### Command Parameters

| Option           | Description                                                                                                    | Mandatory | Contains Parameters | Parameter Range                                                                                       | Example |
| ---------------- | -------------------------------------------------------------------------------------------------------------- | --------- | ------------------- | ----------------------------------------------------------------------------------------------------- | ------- |
| -n or --name   | Name of the project                                                                                            | Yes       | Yes                 | The project name corresponds to the configuration file.                                               |         |
| -s or --socket | Socket used for communication with etmemd, which must be the same as the one specified when etmemd is started. | Yes       | Yes                 | The administrator can use this option to specify an etmemd server when multiple etmemd servers exist. |         |

### Specifying System Memory Swapping Threshold and Process Memory Swapping Using the etmem Client

Only slide policies support private features.

* Process or system memory swapping threshold

It is necessary to consider the timing of etmem memory swapping for optimal performance. Memory swapping is not performed when the system has enough available memory or a process occupies a low amount of memory. Memory swapping threshold can be specified for the system and processes.

* Process memory swapping

The memory of I/O latency-sensitive service processes should not be swapped in the storage scenario. In this case, you can disable memory swapping for certain services.

Process and system memory swapping thresholds and process memory swapping are controlled by the **sysmem\_threshold**, **swap\_threshold**, and **swap\_flag** parameters in the configuration file. For details, see [etmem Configuration Files](#etmem-configuration-files).

```sh
#slide_conf.yaml
[project]
name=test
loop=1
interval=1
sleep=1
sysmem_threshold=50

[engine]
name=slide
project=test

[task]
project=test
engine=slide
name=background_slide
type=name
value=mysql
T=1
max_threads=1
swap_threshold=10g
swap_flag=yes
```

#### System Memory Swapping Threshold

The **sysmem\_threshold** parameter is used to set system memory swapping threshold. The value range for **sysmem\_threshold** is 0 to 100. If **sysmem\_threshold** is set in the configuration file, etmem will swap memory when system memory is lower than **sysmem\_threshold**.

For example:

1. Compose the configuration according to the example. Set **sysmem\_threshold** to **20**.

2. Start the server, add a project to the server, and start the project.

   ```bash
   etmemd -l 0 -s monitor_app &
   etmem obj add -f etmem_config -s monitor_app
   etmem project start -n test -s monitor_app
   etmem project show -s monitor_app
   ```

3. Observe the memory swapping results. etmem swaps memory only when the system available memory is less than 20%.

#### Process Memory Swapping Threshold

The **swap\_threshold** parameter is used to set process memory swapping threshold. **swap\_threshold** is the absolute memory usage of a process in the format of \<number>**g/G**. If **swap\_threshold** is set in the configuration file, etmem will not swap memory of the process when the process memory usage is lower then **swap\_threshold**.

For example:

1. Compose the configuration according to the example. Set **swap\_threshold** to **5g**.

2. Start the server, add a project to the server, and start the project.

   ```bash
   etmemd -l 0 -s monitor_app &
   etmem obj add -f etmem_config -s monitor_app
   etmem project start -n test -s monitor_app
   etmem project show -s monitor_app
   ```

3. Observe the memory swapping results. etmem swaps memory only when the process memory usage reaches 5 GB.

#### Process Memory Swapping

The **swap\_flag** parameter is used to enable the process memory swapping feature. The value of **swap\_flag** can be **yes** or **no**. If **swap\_flag** is **no** or not configured, etmem swaps memory normally. If **swap\_flag** is **yes**, etmem swaps memory of the specified processes only.

For example:

1. Compose the configuration according to the example. Set **swap\_flag** to **yes**.

2. Flag the memory to be swapped for the service process.

   ```bash
   madvise(addr_start, addr_len, MADV_SWAPFLAG)
   ```

3. Start the server, add a project to the server, and start the project.

   ```bash
   etmemd -l 0 -s monitor_app &
   etmem obj add -f etmem_config -s monitor_app
   etmem project start -n test -s monitor_app
   etmem project show -s monitor_app
   ```

4. Observe the memory swapping results. Only the flagged memory is swapped. Other memory is retained in the DRAM.

In the process memory page swapping scenario, `ioctl` is added to the original scan interface file **idle\_pages** to ensure that VMAs that are not flagged do not participate in memory scanning and swapping.

Scan management interface:

* Function prototype

  ```c
  ioctl(fd, cmd, void *arg);
  ```

* Input parameters
  1. fd: file descriptor, which is obtained by opening a file under /proc/pid/idle\_pages using the open system call
  2. cmd: controls the scan actions. The following values are supported:
     VMA\_SCAN\_ADD\_FLAGS: adds VMA memory swapping flags to scan only flagged VMAs
     VMA\_SCAN\_REMOVE\_FLAGS: removes added VMA memory swapping flags
  3. args: integer pointer parameter used to pass a specific mask. The following value is supported:
     VMA\_SCAN\_FLAG: Before the etmem\_scan.ko module starts scanning, the walk\_page\_test interface is called to determine whether the VMA address meets the scanning requirements. If this flag is set, only the VMA addresses that contain specific swap flags are scanned.

* Return values
  1. 0 if the command succeeds
  2. Other values if the command fails

* Precautions
  Unsupported flags are ignored and do not return errors.

### Specifying swapcache Memory Recycling Instructions Using the etmem Client

The user-mode etmem initiates a memory elimination and recycling operation and interacts with the kernel-mode memory recycling module through the **write procfs** interface. The memory recycling module parses the virtual address sent from the user space, obtains the page corresponding to the address, and calls the native kernel interface to swap and recycle the memory corresponding to the page. During memory swapping, swapcache will use some system memory. To further save memory, the swapcache memory recycling feature is added.

Add **swapcache\_high\_wmark** and **swapcache\_low\_wmark** parameters to use the swapcache memory recycling feature.

* **swapcache\_high\_wmark**: High system memory water of swapcache
* **swapcache\_low\_wmark**: Low system memory water of swapcache

After etmem swaps memory, it checks the swapcache memory occupancy. When the occupancy exceeds the high watermark, an `ioctl` instruction will be issued through **swap\_pages** to trigger the swapcache memory recycling and stop when swapcache memory occupancy reaches the low watermark.

An example configuration file is as follows. For details, see [etmem Configuration Files](#etmem-configuration-files).

```sh
#slide_conf.yaml
[project]
name=test
loop=1
interval=1
sleep=1
swapcache_high_vmark=5
swapcache_low_vmark=3

[engine]
name=slide
project=test

[task]
project=test
engine=slide
name=background_slide
type=name
value=mysql
T=1
max_threads=1
```

During memory swapping, swapcache memory needs to be recycled to further save memory. An `ioctl` interface is added to the original memory swap interface to configure swapcache watermarks and swapcache memory recycling.

* Function prototype

  ```c
  ioctl(fd, cmd, void *arg);
  ```

* Input parameters
  1. fd: file descriptor, which is obtained by opening a file under /proc/pid/idle\_pages using the open system call
  2. cmd: controls the scan actions. The following values are supported:
     RECLAIM\_SWAPCACHE\_ON: enables swapcache memory swapping
     RECLAIM\_SWAPCACHE\_OFF: disables swapcache memory swapping
     SET\_SWAPCACHE\_WMARK: configures swapcache memory watermarks
  3. args: integer pointer parameter used to pass a specific mask. The following value is supported:
     Parameters that pass the values of swapcache watermarks

* Return values
  1. 0 if the command succeeds
  2. Other values if the command fails

* Precautions
  Unsupported flags are ignored and do not return errors.

### Executing Private Commands and Functions Using the etmem Client

Only the cslide policy support private commands.

* `showtaskpages`
* `showhostpages`

For engines and tasks of engines that use the cslide policy, you can run the commands above to query the page access of tasks and the usage of system huge pages on the host of VMs.

For example:

```bash
etmem engine showtaskpages <-t task_name> -n proj_name -e cslide -s etmemd_socket

etmem engine showhostpages -n proj_name -e cslide -s etmemd_socket
```

**Note**: `showtaskpages` and `showhostpages` are supported by the cslide policy only.

#### Command Parameters

| Option              | Description                                                                                                    | Mandatory | Contains Parameters | Parameter Range                                                                                       | Example |
| ------------------- | -------------------------------------------------------------------------------------------------------------- | --------- | ------------------- | ----------------------------------------------------------------------------------------------------- | ------- |
| -n or --proj\_name | Name of the project                                                                                            | Yes       | Yes                 | Name of an existing project to run                                                                    |         |
| -s or --socket    | Socket used for communication with etmemd, which must be the same as the one specified when etmemd is started. | Yes       | Yes                 | The administrator can use this option to specify an etmemd server when multiple etmemd servers exist. |         |
| -e or --engine    | Name of the engine to run                                                                                      | Yes       | Yes                 | Name of an existing engine to run                                                                     |         |
| -t or --task\_name | Name of the task to run                                                                                        | No        | Yes                 | Name of an existing task to run                                                                       |         |

### Enabling and Disabling Kernel Swap

When etmem swaps memory to the drive to expand memory, you can choose to enable the kernel swap feature. You can disable the native kernel swap mechanism to void swapping memory undesirably, resulting in problems with user-mode processes.

A sys interface is provided to implement such control. A **kobj** named **kernel\_swap\_enable** is created in **/sys/kernel/mm/swap** to enable and disable kerne swap. The default value of **kernel\_swap\_enable** is **true**.

For example:

```sh
# Enable kernel swap
echo true > /sys/kernel/mm/swap/kernel_swap_enable
or
echo 1 > /sys/kernel/mm/swap/kernel_swap_enable

# Disable kernel swap
echo false > /sys/kernel/mm/swap/kernel_swap_enable
or
echo 0 > /sys/kernel/mm/swap/kernel_swap_enable

```

### Starting etmem Upon System Startup

#### Scenario

You can configure the systemd configuration file to run etmemd as a forking service of systemd.

#### Usage

Compose a service configuration file to start etmemd with the `-m` option. For example:

```bash
etmemd -l 0 -s etmemd_socket -m
```

#### Command Parameters

| Option            | Description                           | Mandatory |  Contains Parameters | Parameter Range       | Example                                                            |
| --------------- | ---------------------------------- | -------- | ---------- | --------------------- | ------------------------------------------------------------ |
| -l or --log-level | etmemd log level                      | No       | Yes         | 0~3                   | 0: debug level   1: info level   2: warning level   3: error level   Logs whose levels are higher than the specified value are printed to **/var/log/message**. |
| -s or --socket    | Socket listened by etmemd to interact with the client | Yes       | Yes         | String of up to 107 characters | Socket listened by etmemd                                         |
| -m or --mode-systemctl| Starts the etmemd service through systemctl |   No| No| N/A|    The `-m` option needs to be specified in the service file.|
| -h or --help      | Prints help information                           | No       | No         | N/A                    | This option prints help information and exit.                                 |

### Supporting Third-party Memory Expansion Policies With etmem

#### Scenario

etmem provides third-party memory expansion policy registration and module scanning dynamic library and can eliminate memory according to third-party policies.

You can use the module scanning dynamic library to implement the interface of the struct required for connecting to etmem.

#### Usage

To use a third-party memory expansion elimination policy, perform the following steps:

1. Invoke the scanning interface of the module as required.

2. Implement the interfaces using the function template provided by the etmem header file and encapsulate them into a struct.

3. Build a dynamic library of the third-party memory expansion elimination policy.

4. Specify the **thirdparty** engine in the configuration file.

5. Enter the names of the library and the interface struct to the corresponding **task** fields in the configuration file.

Other steps are similar to those of using other engines.

Interface struct template:

```c
struct engine_ops {

/* Parsing private parameters of the engine. Implement the interface if required, otherwise, set it to NULL. */

int (*fill_eng_params)(GKeyFile *config, struct engine *eng);

/* Clearing private parameters of the engine. Implement the interface if required, otherwise, set it to NULL. */

void (*clear_eng_params)(struct engine *eng);

/* Parsing private parameters of the task. Implement the interface if required, otherwise, set it to NULL. */

int (*fill_task_params)(GKeyFile *config, struct task *task);

/* Clearing private parameters of the task. Implement the interface if required, otherwise, set it to NULL. */

void (*clear_task_params)(struct task *tk);

/* Task starting interface */

int (*start_task)(struct engine *eng, struct task *tk);

/* Task stopping interface */

void (*stop_task)(struct engine *eng, struct task *tk);

/* Allocate PID-related private parameters */

int (*alloc_pid_params)(struct engine *eng, struct task_pid **tk_pid);

/* Destroy PID-related private parameters */

void (*free_pid_params)(struct engine *eng, struct task_pid **tk_pid);

/* Support for private commands required by the third-party policy. If this interface is not required, set it to NULL */

int (*eng_mgt_func)(struct engine *eng, struct task *tk, char *cmd, int fd);

};
```

External interfaces of the scanning module:

| Interface         |Description|
| ------------ | --------------------- |
| etmemd\_scan\_init | Initializes the scanning module|
| etmemd\_scan\_exit | Exits the scanning module|
| etmemd\_get\_vmas | Gets the VMAs to be scanned|
| etmemd\_free\_vmas | Releases VMAs scanned by `etmemd_get_vmas`|
| etmemd\_get\_page\_refs | Scans pages in VMAs|
| etmemd\_free\_page\_refs | Release the page access information list obtained by `etmemd_get_page_refs` |

In the VM scanning scenario, `ioctl` is added to the original scan interface file **idle\_pages** to distinguish the EPT scanning granularity and specify whether to ignore page access flags on the hosts.

In the process memory page swapping scenario, `ioctl` is added to the original scan interface file **idle\_pages** to ensure that VMAs that are not flagged do not participate in memory scanning and swapping.

Scan management interface:

* Function prototype

  ```c
  ioctl(fd, cmd, void *arg);
  ```

* Input parameters
  1. fd: file descriptor, which is obtained by opening a file under /proc/pid/idle\_pages using the open system call
  2. cmd: controls the scan actions. The following values are supported:
     IDLE\_SCAN\_ADD\_FLAG: adds a scanning flag
     IDLE\_SCAM\_REMOVE\_FLAGS: removes a scanning flag
     VMA\_SCAN\_ADD\_FLAGS: adds VMA memory swapping flags to scan only flagged VMAs
     VMA\_SCAN\_REMOVE\_FLAGS: removes added VMA memory swapping flags
  3. args: integer pointer parameter used to pass a specific mask. The following value is supported:
     SCAN\_AS\_HUGE: scans the pages according to the 2 MB granularity to see if the pages have been accessed when scanning the EPT page table. If this parameter is not set, the granularity will be the granularity of the EPT page table itself.
     SCAN\_IGN\_HUGE: ignores page access flags on the hosts when scanning VMs.
     VMA\_SCAN\_FLAG: Before the etmem\_scan.ko module starts scanning, the walk\_page\_test interface is called to determine whether the VMA address meets the scanning requirements. If this flag is set, only the VMA addresses that contain specific swap flags are scanned.

* Return values
  1. 0 if the command succeeds
  2. Other values if the command fails

* Precautions
  Unsupported flags are ignored and do not return errors.

An example configuration file is as follows. For details, see [etmem Configuration Files](#etmem-configuration-files).

```text
#thirdparty
[engine]

name=thirdparty

project=test

eng_name=my_engine

libname=/user/lib/etmem_fetch/code_test/my_engine.so

ops_name=my_engine_ops

engine_private_key=engine_private_value

[task]

project=test

engine=my_engine

name=background1

type=pid

value=1798245

task_private_key=task_private_value
```

**Note**:

You need to use the module scanning dynamic library to implement the interface of the struct required for connecting to etmem.

**fd** in the `eng_mgt_func` interface cannot be written with the **0xff** and **0xfe** characters.

Multiple different third-party policy dynamic libraries, distinguished by **eng\_name** in the configuration file, can be added within a project.

### Help Information of the etmem Client and Server

Run the following command to print help information of the etmem server:

```bash
etmemd -h
```

or:

```bash
etmemd --help
```

Run the following command to print help information of the etmem client:

```bash
etmem help
```

Run the following command to print help information of project, engine, and task operations:

```bash
etmem obj help
```

Run the following command to print help information of projects:

```bash
etmem project help
```

## How to Contribute

1. Fork this repository.
2. Create a branch.
3. Commit your code.
4. Create a pull request (PR).

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/hybrid_deployment/rubik/example_of_isolation_for_hybrid_deployed_services.md
---
# Example of Isolation for Hybrid Deployed Services

## Environment Preparation

Check whether the kernel supports isolation of hybrid deployed services.

```bash
# Check whether isolation of hybrid deployed services is enabled in the /boot/config-<kernel version> system configuration.
# If CONFIG_QOS_SCHED=y, the function is enabled. Example:
cat /boot/config-5.10.0-60.18.0.50.oe2203.x86_64 | grep CONFIG_QOS
CONFIG_QOS_SCHED=y
```

Install the Docker engine.

```bash
yum install -y docker-engine
docker version
# The following shows the output of docker version.
Client:
 Version:           18.09.0
 EulerVersion:      18.09.0.300
 API version:       1.39
 Go version:        go1.17.3
 Git commit:        aa1eee8
 Built:             Wed Mar 30 05:07:38 2022
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.09.0
  EulerVersion:     18.09.0.300
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.17.3
  Git commit:       aa1eee8
  Built:            Tue Mar 22 00:00:00 2022
  OS/Arch:          linux/amd64
  Experimental:     false
```

## Hybrid Deployed Services

### Online Service ClickHouse

Use the clickhouse-benchmark tool to test the performance and collect statistics on performance metrics such as QPS, P50, P90, and P99. For details, see <https://clickhouse.com/docs/en/operations/utilities/clickhouse-benchmark/>.

### Offline Service Stress

Stress is a CPU-intensive test tool. You can specify the **--cpu** option to start multiple concurrent CPU-intensive tasks to increase the stress on the system.

## Usage Instructions

1. Start a ClickHouse container (online service).

2. Access the container and run the **clickhouse-benchmark** command. Set the number of concurrent queries to **10**, the number of queries to **10000**, and time limit to **30**.

3. Start a Stress container (offline service) at the same time and concurrently execute 10 CPU-intensive tasks to increase the stress on the environment.

4. After the **clickhouse-benchmark** command is executed, a performance test report is generated.

The **test\_demo.sh** script for the isolation test for hybrid deployed services is as follows:

```bash
#!/bin/bash

with_offline=${1:-no_offline}
enable_isolation=${2:-no_isolation}
stress_num=${3:-10}
concurrency=10
timeout=30
output=/tmp/result.json
online_container=
offline_container=

exec_sql="echo \"SELECT * FROM system.numbers LIMIT 10000000 OFFSET 10000000\" | clickhouse-benchmark -i 10000 -c $concurrency -t $timeout"

function prepare() {
    echo "Launch clickhouse container."
    online_container=$(docker run -itd \
            -v /tmp:/tmp:rw \
            --ulimit nofile=262144:262144 \
            -p 34424:34424 \
            yandex/clickhouse-server)

    sleep 3
    echo "Clickhouse container launched."
}

function clickhouse() {
    echo "Start clickhouse benchmark test."
    docker exec $online_container bash -c "$exec_sql --json $output"
    echo "Clickhouse benchmark test done."
}

function stress() {
    echo "Launch stress container."
    offline_container=$(docker run -itd joedval/stress --cpu $stress_num)
    echo "Stress container launched."

    if [ $enable_isolation == "enable_isolation" ]; then
        echo "Set stress container qos level to -1."
        echo -1 > /sys/fs/cgroup/cpu/docker/$offline_container/cpu.qos_level
    fi
}

function benchmark() {
    if [ $with_offline == "with_offline" ]; then
        stress
        sleep 3
    fi
    clickhouse
    echo "Remove test containers."
    docker rm -f $online_container
    docker rm -f $offline_container
    echo "Finish benchmark test for clickhouse(online) and stress(offline) colocation."
    echo "===============================clickhouse benchmark=================================================="
    cat $output
    echo "===============================clickhouse benchmark=================================================="
}

prepare
benchmark
```

## Test Results

Independently execute the online service ClickHouse.

```bash
sh test_demo.sh no_offline no_isolation
```

The baseline QoS data (QPS/P50/P90/P99) of the online service is as follows:

```json
{
    "localhost:9000": {
        "statistics": {
            "QPS": 1.8853412284364512,
            ......
        }
    },
    "query_time_percentiles": {
        ......
        "50": 0.484905256,
        "60": 0.519641313,
        "70": 0.570876148,
        "80": 0.632544937,
        "90": 0.728295525,
        "95": 0.808700418,
        "99": 0.873945121,
        ......
    }
}
```

Execute the **test\_demo.sh** script to start the offline service Stress and run the test with the isolation function disabled.

```bash
# **with_offline** indicates that the offline service Stress is enabled.
# **no_isolation** indicates that isolation of hybrid deployed services is disabled.
sh test_demo.sh with_offline no_isolation
```

**When isolation of hybrid deployed services is disabled**, the QoS data (QPS/P80/P90/P99) of the ClickHouse service is as follows:

```json
{
    "localhost:9000": {
        "statistics": {
            "QPS": 0.9424028693636205,
            ......
        }
    },
    "query_time_percentiles": {
        ......
        "50": 0.840476774,
        "60": 1.304607373,
        "70": 1.393591017,
        "80": 1.41277543,
        "90": 1.430316688,
        "95": 1.457534764,
        "99": 1.555646855,
        ......
    }
}
```

Execute the **test\_demo.sh** script to start the offline service Stress and run the test with the isolation function enabled.

```bash
# **with_offline** indicates that the offline service Stress is enabled.
# **enable_isolation** indicates that isolation of hybrid deployed services is enabled.
sh test_demo.sh with_offline enable_isolation
```

**When isolation of hybrid deployed services is enabled**, the QoS data (QPS/P80/P90/P99) of the ClickHouse service is as follows:

```json
{
    "localhost:9000": {
        "statistics": {
            "QPS": 1.8825798759270718,
            ......
        }
    },
    "query_time_percentiles": {
        ......
        "50": 0.485725185,
        "60": 0.512629901,
        "70": 0.55656488,
        "80": 0.636395956,
        "90": 0.734695906,
        "95": 0.804118275,
        "99": 0.887807409,
        ......
    }
}
```

The following table lists the test results.

| Service Deployment Mode                          | QPS           | P50           | P90           | P99           |
| -------------------------------------- | ------------- | ------------- | ------------- | ------------- |
| ClickHouse (baseline)    | 1.885         | 0.485         | 0.728         | 0.874         |
| ClickHouse + Stress (isolation disabled)| 0.942 (-50%)   | 0.840 (-42%)   | 1.430 (-49%)   | 1.556 (-44%)   |
| ClickHouse + Stress (isolation enabled)  | 1.883 (-0.11%) | 0.486 (-0.21%) | 0.735 (-0.96%) | 0.888 (-1.58%) |

When isolation of hybrid deployed services is disabled, the QPS of ClickHouse decreases from approximately 1.9 to 0.9, the service response delay (P90) increases from approximately 0.7s to 1.4s, and the QoS decreases by about 50%. When isolation of hybrid deployed services is enabled, the QPS and response delay (P50/P90/P99) of ClickHouse decrease by less than 2% compared with the baseline, and the QoS remains unchanged.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/distributed/expanding_the_ecosystem_through_distributed_soft_bus.md
---
# Expanding the Ecosystem Through Distributed Soft Bus

## Background

openEuler aims to build an operating system for digital infrastructure. To promote cooperation with the OpenHarmony ecosystem and implement interoperability in device-edge scenarios, openEuler introduces the distributed soft bus (DSoftBus) technology to the embedded field.

DSoftBus is an open source communication base for distributed devices developed by the OpenHarmony community. It enables unified distributed communication between devices, achieving imperceptible device discovery and efficient data transmission. OpenHarmony is designed for smart devices, IoT devices, and industrial devices that require strong interaction, while openEuler is oriented to servers, edge computing, cloud, and embedded devices that require high reliability and performance. DSoftBus is an example of the collaborative technologies that allow users from both communities to explore more industry applications.

For details about the working principles and processes of DSoftBus, see [openEuler Distributed Soft Bus](https://pages.openeuler.openatom.cn/embedded/docs/build/html/master/features/distributed_softbus.html).

## Environment

**Hardware**

| Device Name            | OS               | Description                           |
| -------------------- | ----------------------- | ------------------------------- |
| Raspberry Pi 4B           | openEuler 22.03-LTS-SP4  | Raspberry Pi with openEuler installed  |
| DAYU200 rk3568 development board| openHarmony 3.1 Release | Development board with OpenHarmony installed|

**Software**

| Item                 | Download URL                                                    | Description                                                    |
| ----------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| openEuler 22.03-LTS-SP4  | <https://www.openeuler.org/en/download/>                       | Download the openEuler image in the raspi\_img directory. For details, see [Installing openEuler on Raspberry Pi](https://www.cnblogs.com/rocedu/p/14615565.html).|
| openHarmony 3.1 Release | <https://gitee.com/openharmony/manifest.git>                   | Download the OpenHarmony OS. For details, see [Standard System Overview](https://docs.openharmony.cn/pages/v3.1/en/device-dev/quick-start/quickstart-ide-standard-overview.md/).|
| HUAWEI DevEco Studio    | <https://developer.harmonyos.com/cn/develop/deveco-studio#download> | Integrated development environment for OpenHarmony. For details, see [DevEco Studio User Guide (OpenHarmony)](https://developer.harmonyos.com/en/docs/documentation/doc-guides-V3/ohos-deveco-studio-overview-0000001263280421-V3). The code in this document is based on OpenHarmony SDK 9.|

## Obtaining Code

This document uses a Raspberry Pi (openEuler) and an RK3568 development board (openHarmony) for ecosystem interconnection demonstration.

**Obtain the code for openEuler (Raspberry Pi)**:

```shell
# Operations on openEuler are performed on Raspberry Pi.

# The server software package has been integrated into openEuler 22.03 LTS SP4. Run the following command to install the server software package:

dnf install dsoftbus -y

# For the client code, use softbus_client_main.c provided by the openEuler Embedded SIG (https://openeuler.gitee.io/yocto-meta-openeuler/master/features/distributed_softbus.html).
```

**Obtain the code for OpenHarmony (RK3568 development board)**:

```shell
# Operations on OpenHarmony (RK3568 development board) are performed on Windows.

# DSoftBus has been integrated into the openHarmony 3.1 Release image. You do not need to manually obtain the server code.

# The client code is based on Native C++. You can run the following git command to download the reference code.

git clone https://gitee.com/liheavy/softbus_client_app.git
```

## Compiling Code

**Compile the code for openEuler (Raspberry Pi)**:

```shell
# After installing DSoftBus, save softbus_client_main.c to any path and run the following gcc command to compile the file. After the compilation is complete, the executable file softbus_client_main is generated in the path. Add the execute permission to the file.

gcc softbus_client_main.c -I/usr/include/dsoftbus/ -L /usr/lib64 -lsoftbus_client.z -lboundscheck -o softbus_client_main

chmod u+x softbus_client_main
```

**Compile the code for OpenHarmony (RK3568 development board)**:

* Configure the IDE by referring to the [HUAWEI DevEco Studio User Guide](https://developer.harmonyos.com/en/docs/documentation/doc-guides-V3/ohos-deveco-studio-overview-0000001263280421-V3).

* Open the downloaded **softbus\_client\_app/softbus\_client\_sample** reference code using DevEco. DevEco automatically initializes the project based on the configuration file.

  After the initialization, generate a signature file: `File --> Project Structure --> Project --> Signing Configs --> Automatically generate signature`.

  ![image-20221201101959764](figures/image-20221201101959764.png)

* OpenHarmony SDK 9 does not provide the API of DSoftBus. Therefore, this project directly uses the API of DSoftBus service in the application. The dependent .so files (**libsoftbus\_client.z.so** and **libsec\_shared.z.so**) are required. Copy the file to the dynamic library path of the local openHarmony SDK. The .so files are downloaded to the local PC (**required\_so** folder) together with the source code. You can also use the hdc tool to copy the two .so files in RK3568. The .so file path is **/lib/xxx.so**.

  To view the local OpenHarmony SDK installation path: `File --> Settings --> OpenHarmony SDK`.

  ![image-20221201103709581](figures/image-20221201103709581.png)

  Copy **libsoftbus\_client.z.so** and **libsec\_shared.z.so** to **OpenHarmony\_SDK\_installation\_path/native/x.x.x.x/sysroot/usr/lib/arm-linux-ohos/**.

  After the compilation is complete, compile the code in DevEco: `Build --> Build Hap(s)/App(s)  --> Build Hap(s)`.

  ![image-20221201111227158](figures/image-20221201111227158.png)

## Running Code

### Preparations

**Configuring Device ID**

**Configure device ID on openEuler (Raspberry Pi)**:

The dependency on the device management module is temporarily removed in the current version of DSoftBus. To simplify device ID obtaining, the device ID is read from the **/etc/SN** file. Therefore, you need to write the device ID to **/etc/SN** before starting DSoftBus. Each device must have a unique ID to avoid authentication and communication errors.

```shell
# Assume the device ID is 1.
echo "1" >>/etc/SN
```

**Connecting to the Network**

The two devices must be in the same LAN and reachable to each other. You can run the `ifconfig` command on the OSs of each device to view the IP addresses.

**On openEuler (Raspberry Pi)**:

* Start the DSoftBus service.

  ```shell
  # DSoftBus provides the softbus_server_main command. Execute the command directly.

  softbus_server_main
  ```

**On OpenHarmony (RK3568 development board)**:

* Connect to the RK3568 development board from the local PC.

  Connect the USB port of the local PC and the OTG USB port of the RK3568 development board. If the development board is displayed in the device list in the upper right corner of DevEco, The device is connected.

  ![image-20221201115003525](figures/image-20221201115003525.png)

* Modify the permission configuration file of DSoftBus.

  OpenHarmony DSoftBus restricts access to its functions based on application permissions. Therefore, you need to modify the DSoftBus configuration file to run the demo.

  ```shell
  # The permission configuration file can be replaced by the hdc_std tool, which is installed with OpenHarmony SDK. The installation path is:

  OpenHarmony_SDK_installation_path/toolchains/x.x.x.x/hdc_std.exe

  # The DSoftBus permission configuration file on RK3568 is in read-only mode. Run the following command to change the file system to read-write mode:

  hdc_std.exe shell "mount -o remount,rw /"

  # Replace the softbus_trans_permission.json file on RK3568 with the one from the softbus_client_app repository: 

  hdc_std.exe file send softbus_trans_permission.json /system/etc/communication/softbus/
  ```

* Restart the DSoftBus service.

  ```shell
  # Run the following command to restart the DSoftBus service for the modified permission configuration file to take effect:

  ps -ef | grep softbus_server | grep -v grep

  kill -9 PID_queried_in_the_last_step
  ```

### Device Authentication

The OpenHarmony and openEuler devices need to be added as trusted devices for each other during networking. Therefore, before the two devices communicate with each other, they need to be authenticated through the Hichain module. For details, see [Adding Trusted Devices](https://pages.openeuler.openatom.cn/embedded/docs/build/html/master/features/distributed_softbus.html#id7). The following operations are performed on Raspberry Pi.

* Create a soft link of the dynamic library.

  ```shell
  # This method requires libsec_shared.z.so, which is replaced with libboundscheck.so in openEuler DSoftBus. Therefore, you need to create a soft link to use libsec_shared.z.so.
  ln -s /usr/lib64/libboundscheck.so /usr/lib64/libsec_shared.z.so

  # In the Arm environment, /lib64/1d-linux-aarch64.so.1 is also required.
  ln -s /lib/ld-linux-aarch64.so.1 /lib64/ld-linux-aarch64.so.1
  ```

* Run the authentication client.

  ```shell
  # The demo executable file (devicemanager) of the authentication client is also stored in the hichain_sample directory of the softbus_client_app repository.

  chmod u+x devicemanager
  ./devicemanager
  ```

  Enter **l** as prompted to list the devices in the same LAN.

  ![image-20221201145904897](figures/image-20221201145904897.png)

  Enter the OpenHarmony device number as prompted for authentication. After the number is entered, a confirmation dialog box is displayed on the RK3568 development board, asking whether to allow the peer device to connect. Click **Allow** and enter **l** on the Raspberry Pi. The device status changes from discovery to online, indicating a successful authentication.

  ![image-20221201150148988](figures/image-20221201150148988.png)

### Device Communication

* Burn and run the client on OpenHarmony (RK3568 development board).

  Connect the local PC to the RK3568 development board and click **Run** in the upper part of DevEco.

  ![image-20221201151951393](figures/image-20221201151951393.png)

  After the burning is complete, RK3568 directly runs the app.

  ![image-20221130193253259](figures/image-20221130193253259.png)

* Send data from openEuler (Raspberry Pi).

  ```shell
  # Run the compiled softbus_client_main executable file.

  ./softbus_client_main
  ```

  The networking devices are displayed.

  ![image-20221201162316137](figures/image-20221201162316137.png)

  Enter **c** as prompted.

  ![image-20221201163115659](figures/image-20221201163115659.png)

  Enter the ID of the openHarmony device as prompted.

  ![image-20221201163310745](figures/image-20221201163310745.png)

  The received character string is displayed on the top of the RK3568 development board screen.

![](figures/receiving_data.png)

* View terminal information OpenHarmony (RK3568 development board).

  Click the refresh icon on the app page. The devices connected to the device are displayed. Click the device icon to display the basic information about the device.

  ![](figures/terminal_info.png)

* Send data from OpenHarmony (RK3568 development board):

  Click **Send Data** on the app page to send data to openEuler.

![](figures/sending_data.png)

The received information is displayed on openEuler (Raspberry Pi).

![image-20221201165948378](figures/image-20221201165948378.png)

The demonstration of interconnection between openEuler and OpenHarmony through DSoftBus is complete.

## 6. Summary

This document demonstrates how OpenHarmony devices and openEuler devices communicate with each other through the DSoftBus. The server code is being continuously optimized to support more distributed scenarios. The client code is only a demo and needs to be optimized. For example, the standard method for invoking DSoftBus in an OpenHarmony app is to develop a system ability to call the DSoftBus API and integrate the system ability to the SDK. However, in this document, the app directly invokes the DSoftBus API. We welcome enthusiasts to participate in the development to enrich the embedded capabilities of openEuler.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/fangtian/fangtian_for_linux_waylan_and_openharmony_applications.md
---
# FangTian for Linux Wayland and OpenHarmony Applications

The FangTian window engine integrates multiple application ecosystems, allowing Linux and OpenHarmony applications to run on openEuler simultaneously.

## Wayland Application Support

### Wayland Protocols

To support native Linux applications, FangTian is compatible with Wayland applications. Due to the complexity of Wayland protocols, currently, FangTian supports core, stable, and unstable protocols.

### Application Running

1. After [starting the FangTian engine](./fangtian_environment_configuration.md#starting-fangtian), start the SA of the Wayland adapter.

   ```shell
   mkdir -p ~/tmp
   sa_main /system/profile/ft/ft_wl.xml > ~/tmp/ftwlsa.log 2>&1 &
   ```

2. Configure the Wayland environment.

   ```shell
   export XDG_SESSION_TYPE=wayland
   export WAYLAND_DISPLAY="wayland-0"
   export QT_QPA_PLATFORMTHEME=ukui
   ```

3. Download and install Linux Wayland applications.

   ```shell
   sudo dnf install kylin-calculator deepin-terminal
   ```

4. The following applications are installed:

![](./figures/wayland_apps.png)

## OpenHarmony Application Support

### ArkUI Framework

Currently, FangTian supports some ArkUI controls, such as texts, buttons, and images. Developers can develop Harmony applications using [DevEco Studio](https://developer.harmonyos.com/en/develop/deveco-studio/).

### Application Source Code

* [Electronic Album](https://gitee.com/openharmony/codelabs/tree/master/ETSUI/ElectronicAlbum)
* [Simple Calculator](https://gitee.com/openharmony/codelabs/tree/master/ETSUI/SimpleCalculator)

### Installation and Running

1. Copy the **.hap** file of the application from DevEco Studio to an openEuler directory, for example, **~/apps/tmp**.

2. Decompress the **.hap** file, for example, **eletronicAlbum.hap**.

   ```shell
   unzip eletronicAlbum.hap
   ```

   After the decompression, the application is in **~/apps/tmp/eletronicAlbum**.

3. After [starting the FangTian engine](./fangtian_environment_configuration.md#starting-fangtian), run the application.

   ```shell
   hap_executor ~/apps/tmp/eletronicAlbum
   ```

4. The following window is displayed:

![](./figures/arkui_ele.png)

### Constraints

* Currently, ArkUI controls are not fully supported. Web and video controls are unavailable. You need to develop and port the NAPI interfaces.

* ArkUI supports only the x86 architecture in this version.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/fangtian/fangtian_environment_configuration.md
---
# FangTian Installation and Deployment

This chapter describes how to install FangTian in openEuler.

## Software and Hardware Requirements

### Hardware Requirements

Currently, only the x86 and AArch64 architectures are supported.

### Software Requirements

OS: openEuler 22.03 LTS SP4

### Environment Setup

Install the openEuler OS. For details, see the *[openEuler Installation Guide](../../installation_upgrade/installation/installation_guide.md)*.

### Installing the FangTian Software Package

On the x86 platform:

```shell
sudo dnf install ft_multimedia ft_mmi ft_flutter ft_engine arkui-linux ft_utils
sudo dnf install ft_multimedia-devel ft_mmi-devel ft_flutter-devel ft_engine-devel
```

On the AArch64 platform:

```shell
sudo dnf install ft_multimedia ft_mmi ft_flutter ft_engine ft_utils
sudo dnf install ft_multimedia-devel ft_mmi-devel ft_flutter-devel ft_engine-devel
```

## Starting FangTian

* Start the SAMGR system service.

  Assume that binder and ashmem have been installed.

  ```shell
  sudo /usr/share/sa/pre_oneshot_samgr 
  ```

  Directly start SAMGR.

  ```shell
  mkdir -p ~/tmp
  sudo samgr > ~/tmp/samgr.log 2>&1 &
  ```

  Alternatively, set SAMGR as a service and start the service.

  ```shell
  sudo systemctl restart samgr
  ```

* Start the SA engine.

  ```shell
  sa_main /system/profile/ft/ft.xml > ~/tmp/ftsa.log 2>&1 &
  ```

  > Description
  >
  > * SA stands for system ability. A process can have multiple SAs. The **ft.xml** file specifies multiple SAs for the ft process. For details about SAMGR and SAs, see the OpenHarmony documentation.
  > * The SA configuration XML file, **sa\_main**, and SAMGR are automatically deployed during software package installation.

## Developing and Running a Simple GUI Application Using FangTian

[Example](https://atomgit.com/openeuler/ft_engine/blob/master/samples/) of a simple C++ GUI application.

Run the application:

```shell
desktop &
```

The following window is displayed:

![](./figures/desktop_simple_apps.png)

> **Description**
>
> For details about FangTian application development, see [FT interfaces](https://atomgit.com/openeuler/ft_engine/wikis/1.0-alpha%E6%8E%A5%E5%8F%A3/1.0-alpha%20Interface%20Overview).

---

---
url: /en/docs/22.03_LTS_SP4/server/development/fangtian/overview.md
---
# FangTian Window Engine User Guide

This document describes how to install and develop the FangTian window engine in openEuler.

This article is intended for community developers, open source enthusiasts, and partners who use the openEuler OS and want to learn and use FangTian. Users must:

* Know basic Linux operations.
* Understand Linux GUI development and ArkUI development.

---

---
url: /zh/docs/22.03_LTS_SP4/server/development/fangtian/overview.md
---
# FangTian 视窗引擎指南

本文档介绍基于 openEuler 系统的 FangTian 视窗引擎的安装及开发使用指南。

本文档适用于使用 openEuler 系统并希望了解和使用 FangTian 视窗引擎的社区开发者、开源爱好者以及相关合作伙伴。使用人员需要具备以下经验和技能：

* 熟悉 Linux 基本操作。
* 了解 Linux GUI 开发、ArkUI 开发。

---

---
url: /zh/docs/22.03_LTS_SP4/server/installation_upgrade/installation/faq.md
---
# FAQ

## 安装openEuler时选择第二盘位为安装目标，操作系统无法启动

### 问题现象

安装操作系统时，直接将系统安装到第二块磁盘sdb，重启系统后启动失败。

### 原因分析

当安装系统到第二块磁盘时，MBR和GRUB会默认安装到第二块磁盘sdb。这样会有下面两种情况：

1. 如果第一块磁盘中有完整系统，则加载第一块磁盘中的系统启动。
2. 如果第一块磁盘中没有完好的操作系统，则会导致硬盘启动失败。

以上两种情况都是因为BIOS默认从第一块磁盘sda中加载引导程序启动系统，如果sda没有系统，则会导致启动失败。

### 解决方法

有以下两种解决方案：

* 当系统处于安装过程中，在选择磁盘（选择第一块或者两块都选择）后，指定引导程序安装到第一块盘sda中。
* 当系统已经安装完成，若BIOS支持选择从哪个磁盘启动，则可以通过修改BIOS中磁盘启动顺序，尝试重新启动系统。

## openEuler开机后进入emergency模式

### 问题现象

openEuler系统开机后进入emergency模式，如下图所示：

![](./figures/zh-cn_image_0229291264.jpg)

### 原因分析

操作系统文件系统损坏导致磁盘挂载失败，或者io压力过大导致磁盘挂载超时（超时时间为90秒）。

系统异常掉电、物理磁盘io性能低等情况都可能导致该问题。

### 解决方法

1. 用户直接输入root帐号的密码，登录系统。
2. 使用fsck工具，检测并修复文件系统，然后重启。

   > \[!NOTE]说明
   > fsck（file system check）用来检查和维护不一致的文件系统。若系统掉电或磁盘发生问题，可利用fsck命令对文件系统进行检查。 用户可以通过“fsck.ext3 -h”、“fsck.ext4 -h”命令查看fsck的使用方法。

另外，如果用户需要取消磁盘挂载超时时间，可以直接在“/etc/fstab”文件中添加“x-systemd.device-timeout=0”。如下：

```sh
# /etc/fstab
# Created by anaconda on Mon Sep 14 17:25:48 2015
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/openEuler-root / ext4 defaults,x-systemd.device-timeout=0 0 0
UUID=afcc811f-4b20-42fc-9d31-7307a8cfe0df /boot ext4 defaults,x-systemd.device-timeout=0 0 0
/dev/mapper/openEuler-home /home ext4 defaults 0 0
/dev/mapper/openEuler-swap swap swap defaults 0 0
```

## 系统中存在无法激活的逻辑卷组时，重装系统失败

### 问题现象

由于磁盘故障，系统中存在无法激活的逻辑卷组，重装系统出现异常。

### 原因分析

安装时有激活逻辑卷组的操作，无法激活时会提示异常。

### 解决方法

重装系统前如果系统中存在无法激活的逻辑卷组，为了避免重装系统过程出现异常，需在重装前将逻辑卷组恢复到正常状态或者清除这些逻辑卷组。举例如下：

* 恢复逻辑卷组状态
  1. 使用以下命令清除vg激活状态， 防止出现“Can't open /dev/sdc exclusively mounted filesystem”。

     ```sh
      vgchange -a n testvg32947
     ```

  2. 根据备份文件重新创建pv。

     ```sh
     pvcreate --uuid JT7zlL-K5G4-izjB-3i5L-e94f-7yuX-rhkLjL --restorefile /etc/lvm/backup/testvg32947 /dev/sdc
     ```

  3. 恢复vg信息。

     ```sh
     vgcfgrestore testvg32947
     ```

  4. 重新激活vg。

     ```sh
     vgchange -ay testvg32947
     ```

* 清除逻辑卷组

  ```sh
  vgchange -a n testvg32947
  vgremove -y testvg32947
  ```

## 选择安装源出现异常

### 问题现象

选择安装源后出现："Error checking software selection"。

### 原因分析

这种现象是由于安装源中的软件包依赖存在问题。

### 解决方法

检查安装源是否存在异常。如果异常，使用新的安装源。

## 如何手动开启kdump服务

### 问题现象

执行systemctl status kdump命令，显示状态信息如下，提示无预留内存。

![](./figures/zh-cn_image_0229291280.png)

### 原因分析

kdump服务需要系统预留一段内存用于运行kdump内核，而当前系统没有为kdump服务预留内存，所以无法运行kdump服务。

### 解决方法

已安装操作系统的场景

1. 修改/boot/efi/EFI/openEuler/grub.cfg，添加crashkernel=1024M,high。
2. 重启系统使配置生效。
3. 执行如下命令，检查kdump状态：

   ```sh
   systemctl status kdump
   ```

   若回显如下，即kdump的状态为active，说明kdump已使能，操作结束。

   ![](./figures/zh-cn_image_0229291272.png)

### 参数说明

kdump内核预留内存参数说明如下：

**表 1**  crashkernel参数说明

## 多块磁盘组成逻辑卷安装系统后，再次安装不能只选其中一块磁盘

### 问题现象

在安装系统时，如果之前的系统选择多块磁盘组成逻辑卷进行安装，再次安装时，如果只选择了其中的一块或几块磁盘，没有全部选择，在保存配置时提示配置错误，如[图1](#fig115949762617)所示。

**图 1**  配置错误提示\
![](./figures/Configuration_error_prompt.png)

### 原因分析

之前的逻辑卷包含了多块磁盘，只在一块磁盘上安装会破坏逻辑卷。

### 解决方法

因为多块磁盘组成逻辑卷相当于一个整体，所以只需要删除对应的卷组即可。

1. 按“Ctrl+Alt+F2”可以切换到命令行，执行如下命令找到卷组。

   ```sh
   vgs
   ```

   ![](./figures/zh-cn_image_0231657950.png)

2. 执行如下命令，删除卷组。

   ```sh
   vgremove euleros
   ```

3. 执行如下命令，重启安装程序即可生效。

   ```sh
   systemctl restart anaconda
   ```

   > \[!NOTE]说明
   > 图形模式下也可以按“Ctrl+Alt+F6”回到图形界面，点击[图1](#fig115949762617)右下角的“Refresh”刷新存储配置生效。

## x86物理机UEFI模式由于Secure Boot安全选项问题无法安装

### 问题现象

x86物理机安装系统时，由于设置了BIOS选项Secure Boot 为enable（默认是disable），导致系统一直停留在“No bootable device”提示界面，无法继续安装，如[图2](#fig115949762618)所示。

**图 2**  “No bootable device”提示界面\
![](./figures/No-bootable-device.png)

### 原因分析

开启Secure Boot后，主板会验证引导程序及操作系统 ，若没有用对应的私钥进行签名，则无法通过主板上内置公钥的认证。

### 解决方法

进入BIOS，设置Secure Boot为disable，重新安装即可。

1. 系统启动时，按“F11”，输入密码“Admin@9000”进入BIOS。

   > \[!NOTE]说明
   > 这里的服务器特指华为的泰山服务器，如果是其他服务器，应当自行确认自己的密码。

   ![](./figures/BIOS.png)

2. 选择进入Administer Secure Boot。

   ![](./figures/security.png)

3. 设置Enforce Secure Boot为Disabled。

   ![](./figures/select.png)

   > \[!NOTE]说明
   > 设置Secure Boot Status为Disabled之后，保存退出，重新安装即可。

## 安装openEuler时，软件选择页面选择“服务器-性能工具”，安装后messages日志有pmie\_check报错信息

### 问题现象

安装系统时软件选择勾选服务器-性能工具，会安装pcp相关软件包，正常安装并重启后，/var/log/messages日志文件中会产生报错：pmie\_check failed in /usr/share/pcp/lib/pmie。

### 原因分析

anaconda不支持在chroot环境中安装selinux策略模块，当安装pcp-selinux时，postin脚本安装pcp相关selinux策略模块执行失败，从而导致重启后产生报错。

### 解决办法

完成安装并重启后，以下方法选择其一。

1. 执行如下命令，安装selinux策略模块pcpupstream。

   ```sh
   /usr/libexec/pcp/bin/selinux-setup /var/lib/pcp/selinux install "pcpupstream"
   ```

2. 重新安装pcp-selinux。

   ```sh
   sudo dnf reinstall pcp-selinux
   ```

## 在两块已经安装了系统的磁盘上进行重复选择，并自定义分区时，安装失败

### 问题现象

用户在安装操作系统过程中，存在两块都已经安装过的磁盘，此时如果先选择一块盘，进行自定义分区，然后点击取消按钮，再选择第二块盘，并进行自定义分区时，会出现安装失败。

![](./figures/cancle_disk.png)
![](./figures/custom_paratition.png)

### 原因分析

用户存在两次选择磁盘的操作，当前点击取消后，再选择第二块磁盘，磁盘信息不正确，导致安装失败。

### 解决方法

直接选择目标磁盘进行自定义分区，请勿频繁取消操作，如果一定要进行取消重选建议重新安装。

### issue访问链接

<https://gitee.com/src-openeuler/anaconda/issues/I29P84?from=project-issue>

## 安装LSI MegaRAID卡的物理机kdump无法生成vmcore

### 问题现象

部署好kdump服务后，手动执行`echo c > /proc/sysrq-trigger`命令或由于kernel故障导致kernel宕机，触发kdump启动second kernel过程中，MegaRAID驱动报错“BRCM Debug mfi stat 0x2d，data len requested/completed 0x200/0x0”，报错信息如下图，最终导致无法生成vmcore。

![](./figures/Megaraid_IO_Request_uncompleted.png)

### 原因分析

由于默认配置了reset\_devices启动参数，second kernel启动过程中会触发设备复位(reset\_devices)操作，设备复位操作导致MegaRAID控制器或磁盘状态故障，转储vmcore文件时访问MegaRAID卡的磁盘报错，进而无法生成vmcore。

### 解决方法

在物理机`/etc/sysconfig/kdump`文件中将second kernel默认启动参数`reset_devices`删除，可以规避second kernel启动过程中由于MegaRAID卡驱动复位设备所致IO请求未完成问题，以成功生成vmcore。
![](./figures/reset_devices.png)

---

---
url: /en/docs/22.03_LTS_SP4/edge_computing/ros/faqs.md
---
# FAQs

## Question 1

![](./figures/problem.png)

Cause: The reason for this warning is that there are both ROS1 and ROS2 in the environment variable.\
Solution: Modify the environment variable to avoid the conflict between the two versions.

```shell
vim /opt/ros/humble/share/ros_environment/environment/1.ros_distro.sh
```

```shell
# generated from ros_environment/env-hooks/1.ros_distro.sh.in

#export ROS_DISTRO=humble
```

Comment out everything inside.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/nestos/nestos/feature_description.md
---
# Feature Description

## Container Technology

NestOS provides computing resources for applications using a containerized computing environment. Applications share a system kernel and resources, but are invisible to each other. This means that applications are no longer directly installed in the OS. Instead, they run in containers through Docker. This greatly reduces the coupling among the OS, applications, and running environment. Compared with the traditional application deployment mode, the NestOS cluster provides more flexible and convenient application deployment, less interference between application running environments , and the easier maintenance of OSs.

## rpm-ostree

### System Upgrade

rpm-ostree is a hybrid image/package system that combines RPM and OSTree. It provides RPM-based software package installation and management, and OSTree-based OS update and upgrade. rpm-ostree sees the two operations as updates to the OS. Each update to the system is similar to a transaction submitted by rpm-ostree. This ensures that the update completely succeeds or fails completely and allows the system to be rolled back to the status before the update.

When updating the OS, rpm-ostree keeps two bootable deployments: one before the update and one after the update. The update takes effect only after the OS is restarted. If an error occurs during software installation or upgrade, the rpm-ostree rollback allows NestOS to revert to the previous deployment. The **/ostree/** and **/boot/** directories of NestOS are the OSTree repository environment and show which OSTree deployment is booted into.

### File System

In the rpm-ostree file system layout, only the **/etc** and **/var** directories are writable. Any data in the **/var** directory is not touched and is shared across upgrades. During the system upgrade, rpm-ostree takes the new default **/etc** and adds the changes on the top. This means that the upgrades will receive new default files in **/etc**, which is a critical feature.

OSTree is designed to parallel install multiple versions of multiple independent operating systems. OSTree relies on a new top-level **ostree** directory; it can in fact parallel install inside an existing OS or distribution occupying the physical **/root**. On each client machine, there is an OSTree repository stored in **/ostree/repo**, and a set of deployments stored in **/ostree/deploy/$STATEROOT/$CHECKSUM**. Each deployment is primarily composed of a set of hard links into the repository. This means each version is deduplicated; an upgrade process only costs disk space proportional to the new files, plus some constant overhead.

The model OSTree emphasizes is that the OS read-only content is kept in **/usr**; it comes with code to create a Linux read-only bind mount to prevent inadvertent corruption. There is exactly one **/var** writable directory shared between each deployment for a given OS. The OSTree core code does not touch content in this directory; it is up to the code in each operating system for how to manage and upgrade state.

### OS Extensions

NestOS keeps the base image as simple and small as possible for security and maintainability reasons. However, in some cases it is necessary to add software to the base OS itself. For example, drivers or VPN software are potential candidates because they are harder to containerize. These software packages extend the functionality of the base OS rather than providing runtimes for user applications. For this reason, rpm-ostree treats these packages as extensions. That said, there are no restrictions on which packages you can actually install. By default, packages are downloaded from the openEuler repositories.

To layer a software package, you need to write a systemd unit that executes the `rpm-ostree` command to install the wanted package. The changes are added to a new deployment, which takes effect after restart.

## nestos-installer

nestos-installer helps with NestOS installation. It provides the following functions:

(1) Installing the OS to a target disk, optionally customizing it with an Ignition configuration or first-boot kernel parameters (`nestos-installer install`)

(2) Downloading and verify an OS image for various cloud, virtualization, or bare metal platforms (`nestos-installer download`)

(3) Listing NestOS images available for download (`nestos-installer list-stream`)

(4) Embed an Ignition configuration in a live ISO image to customize the running system that boots from it (`nestos-installer iso ignition`)

(5) Wrap an Ignition configuration in an initrd image that can be appended to the live PXE initramfs to customize the running system that boots from it (`nestos-installer pxe ignition`)

## Zincati

Zincati is an auto-update agent for NestOS hosts. It works as a client for Cincinnati and rpm-ostree, taking care of automatically updating/rebooting machines. Zincati has the following features:

(1) Agent for continuous automatic updates, with support for phased rollouts

(2) Runtime customization via TOML dropins, allowing users to overwrite the default configuration.

(3) Multiple update strategies

(4) Local maintenance windows on a weekly schedule for planned upgrades

(5) Tracks and exposes Zincati internal metrics to Prometheus to ease monitoring tasks across a large fleet of nodes

(6) Logging with configurable priority levels

(7) Support for complex update-graphs via Cincinnati protocol

(8) Support for cluster-wide reboot orchestration, via an external lock-manager

## System Initialization (Ignition)

Ignition is a distribution-agnostic provisioning utility that not only installs, but also reads configuration files (in JSON format) to initialize NestOS. Configurable components include storage and file systems, systemd units, and users.

Ignition runs only once during the first boot of the system (while in the initramfs). Because Ignition runs so early in the boot process, it can re-partition disks, format file systems, create users, and write files before the userspace begins to boot. As a result, systemd services are already written to disk when systemd starts, speeding the time to boot.

(1) Ignition runs only on the first boot\
Ignition is designed to be used as a provisioning tool, not as a configuration management tool. Ignition encourages immutable infrastructure, in which machine modification requires that users discard the old node and re-provision the machine.

(2) Ignition produces the machine specified or no machine at all\
Ignition does what it needs to make the system match the state described in the Ignition configuration. If for any reason Ignition cannot deliver the exact machine that the configuration asked for, Ignition prevents the machine from booting successfully. For example, if the user wanted to fetch the document hosted at **<https://example.com/foo.conf>** and write it to disk, Ignition would prevent the machine from booting if it were unable to resolve the given URL.

(3) Ignition configurations are declarative\
Ignition configurations describe the state of a system. Ignition configurations do not list a series of steps that Ignition should take.\
Ignition configurations do not allow users to provide arbitrary logic (including scripts for Ignition to run). Users describe which file systems must exist, which files must be created, which users must exist, and more. Any further customization must use systemd services, created by Ignition.

(4) Ignition configurations should not be written by hand\
Ignition configurations were designed to be human readable, but difficult to write, to discourage users from attempting to write configs by hand. Use Butane, or a similar tool, to generate Ignition configurations.

## Afterburn

Afterburn is a one-shot agent for cloud-like platforms which interacts with provider-specific metadata endpoints. It is typically used in conjunction with Ignition.

Afterburn comprises several modules which may run at different times during the lifecycle of an instance. Depending on the specific platform, the following services may run in the initramfs on first boot:

* setting local hostname

* injecting network command-line arguments

The following features are conditionally available on some platforms as systemd service units:

* installing public SSH keys for local system users

* retrieving attributes from instance metadata

* checking in to the provider in order to report a successful boot or instance provisioning

---

---
url: /en/docs/22.03_LTS_SP4/cloud/hybrid_deployment/rubik/feature_introduction.md
---
# Feature Introduction

## Absolute Preemption

Rubik allows you to configure priorities of services. In the hybrid deployment of online and offline services, Rubik ensures that online services preempt resources. CPU and memory resources can be preempted.

You can enable preemption using the following configuration:

```yaml
...
  "agent": {
    "enabledFeatures": [
      "preemption"
    ]
  },
  "preemption": {
    "resource": [
      "cpu",
      "memory"
    ]
  }
...
```

For details, see [Configuration Description](./configuration.md#preemption).

In addition, you need to add **volcano.sh/preemptable** to the YAML annotation of the pod to specify service priorities. For example:

```yaml
annotations:
    volcano.sh/preemptable: true
```

> This annotation is used by all Rubik features to identify whether the service is online or offline.
> **true** indicates an online service.
> **false** indicates an offline service.

### CPU Absolute Preemption

**Prerequisites**

* The kernel supports CPU priority configuration based on control groups (cgroups). The CPU subsystem provides the **cpu.qos\_level** interface. The kernel of openEuler 22.03 or later is recommended.

**Kernel interface**

* The interface exists in the cgroup of the container in the `/sys/fs/cgroup/cpu*` directory, for example, `/sys/fs/cgroup/cpu/kubepods/burstable/<PodUID>/<container-longid>`.
  * **cpu.qos\_level**: enables the CPU priority configuration. The value can be **0** or **-1**, with **0** being the default.
    * **0** indicates an online service.
    * **1** indicates an offline service.

### Memory Absolute Preemption

In the hybrid deployment of online and offline services, Rubik ensures that offline services are first terminated in the case of out-of-memory (OOM).

**Prerequisites**

* The kernel supports memory priority configuration based on cgroups. The memory subsystem provides the **memory.qos\_level** interface. The kernel of openEuler 22.03 or later is recommended.
* To enable the memory priority feature, run `echo 1 > /proc/sys/vm/memcg_qos_enable`.

**Kernel interface**

* **/proc/sys/vm/memcg\_qos\_enable**: enables the memory priority feature. The value can be **0** or **1**, with **0** being the default. You can run `echo 1 > /proc/sys/vm/memcg_qos_enable` to enable the feature.
  * **0**: The feature is disabled.
  * **1**: The feature is enabled.

* The interface exists in the cgroup of the container in the `/sys/fs/cgroup/memory` directory, for example, `/sys/fs/cgroup/memory/kubepods/burstable/<PodUID>/<container-longid>`.
  * **memory.qos\_level**: enables the memory priority configuration. The value can be **0** or **-1**, with **0** being the default.
    * **0** indicates an online service.
    * **1** indicates an offline service.

## dynCache Memory Bandwidth and L3 Cache Access Limit

Rubik can limit pod memory bandwidth and L3 cache access for offline services to reduce the impact on online services.

**Prerequisites**

* The cache access and memory bandwidth limit feature supports only physical machines.
  * For x86 physical machines, the CAT and MBA functions of Intel RDT must be enabled in the OS by adding **rdt=l3cat,mba** to the kernel command line parameters (**cmdline**).
  * For ARM physical machines, the MPAM function must be enabled in the OS by adding **mpam=acpi** to the kernel command line parameters (**cmdline**).

* Due to kernel restrictions, RDT does not support the pseudo-locksetup mode.

**New Permissions and Directories of Rubik**

* Mount point: **/sys/fs/resctrl**. Rubik reads and sets files in the **/sys/fs/resctrl** directory. This directory must be mounted before Rubik is started and cannot be unmounted during Rubik running.
* Permission: SYS\_ADMIN. To set files in the **/sys/fs/resctrl** directory on the host, the SYS\_ADMIN permission must be assigned to the Rubik container.
* namespace: pid namespace. Rubik obtains the PID of the service container process on the host. Therefore, the Rubik container needs to share the PID namespace with the host.

**Rubik RDT Cgroups**

Rubik creates five cgroups (**rubik\_max**, **rubik\_high**, **rubik\_middle**, **rubik\_low** and **rubik\_dynamic**) in the RDT resctrl directory (**/sys/fs/resctrl** by default). Rubik writes the watermarks to the **schemata** file of each corresponding cgroup upon startup. The low, middle, and high watermarks can be configured in **dynCache**. The max cgroup uses the default maximum value. The initial watermark of the dynamic cgroup is the same as that of the low cgroup.

**Rubik dynamic Cgroup**

When offline pods whose cache level is dynamic exist, Rubik collects the cache miss and LLC miss metrics of online service pods on the current node and adjusts the watermark of the rubik\_dynamic cgroup. In this way, Rubik dynamically controls offline service pods in the dynamic cgroup.

### Memory Bandwidth and LLC Limit of the Pod

Rubik allows you to configure the memory bandwidth and LLC cgroup for a service pod in either of the following ways:

* Global annotation
  You can set **defaultLimitMode** in the global parameters of Rubik. Rubik automatically configures cgroups for offline service pods (marked by the **volcano.sh/preemptable** annotation in the absolute preemption configuration).
  * If the value is **static**, the pod is added to the **rubik\_max** cgroup.
  * If the value is **dynamic**, the pod is added to the **rubik\_dynamic** cgroup.
* Manual annotation

  * You can set the cache level for a service pod using the **volcano.sh/cache-limit** annotation and the pod to the specified cgroup. For example, the pod with the following configuration is added to the **rubik\_low** cgroup:

  ```yaml
  annotations:
      volcano.sh/cache-limit: "low"
  ```

> Note 1: Cache limits apply to offline services only.
> Note 2: The manual annotation overrides the global one. If you set **defaultLimitMode** in the global Rubik configuration and specify the cache level in the YAML configuration of a pod, the actual dynCache limit is the one specified in the pod YAML configuration.

### dynCache Kernel Interface

* Rubik creates five cgroup directories in **/sys/fs/resctrl** and modifies the **schemata** and **tasks** files of each cgroup.

### dynCache Configuration

The dynCache function is configured as follows:

```json
"agent": {
  "enabledFeatures": [
    "dynCache"
   ]
},
"dynCache": {
  "defaultLimitMode": "static",
  "adjustInterval": 1000,
  "perfDuration": 1000,
  "l3Percent": {
  "low": 20,
  "mid": 30,
  "high": 50
  },
  "memBandPercent": {
    "low": 10,
    "mid": 30,
    "high": 50
  }
}
```

For details, see [Configuration Description](./configuration.md#dyncache)

* **l3Percent** and **memBandPercent**:
  **l3Percent** and **memBandPercent** are used to configure the watermarks of the low, mid, and high cgroups.

  Assume that in the current environment **rdt bitmask=fffff** and **numa=2**. Based on the **low** value of **l3Percent** (20) and the **low** value of **memBandPercent** (10), Rubik configures **/sys/fs/resctrl/rubik\_low** as follows:

  ```text
  L3:0=f;1=f
  MB:0=10;1=10
  ```

* defaultLimitMode:
  * If the **volcano.sh/cache-limit** annotation is not specified for an offline pod, the **defaultLimitMode** of **cacheConfig** determines the cgroup to which the pod is added.

* **adjustInterval**:
  * Interval for dynCache to dynamically adjust the **rubik\_dynamic** cgroup, in milliseconds. The default value is **1000**.

* **perfDuration**:
  * perf execution duration for dynCache, in milliseconds. The default value is **1000**.

### Precautions for dynCache

* dynCache takes affect only for offline pods.
* If a service container is manually restarted during running (the container ID remains unchanged but the container process ID changes), dynCache does not take effect for the container.
* After a service container is started and the dynCache level is set, the limit level cannot be changed.
* The sensitivity of adjusting the dynamic cgroup is affected **adjustInterval** and **perfDuration** values in the Rubik configuration file and the number of online service pods on the node. If the impact detection result indicates that adjustment is required, the adjustment interval fluctuates within the range **\[adjustInterval + perfDuration, adjustInterval + perfDuration x Number of pods]**. You can set the configuration items based on your required sensitivity.

## dynMemory Tiered Memory Reclamation

Rubik supports multiple memory strategies. You can apply different memory allocation methods to different scenarios.

### fssr

fssr is kernel cgroup-based dynamic watermark control. **memory.high** is a memcg-level watermark interface provided by the kernel. Rubik continuously detects memory usage and dynamically adjusts the **memory.high** limit of offline services to suppress the memory usage of offline services, ensuring the quality of online services.

The core logic of fssr is as follows:

* Rubik calculates the memory to reserve upon startup. The default value is the smaller of 10% of total memory or 10 GB.
* Rubik sets the cgroup-level watermark of the offline container. The kernel provides the **memory.high** and **memory.high\_async\_ratio** interfaces for configuring the soft upper limit and alarm watermark of the cgroup. By default, **memory.high** is 80% of the total memory (**total\_memory**).
* Rubik obtains the free memory (**free\_memory**).
* When **free\_memory** is less than **reserved\_memory**, Rubik decreases **memory.high** for the offline container. The amount decreased each time is 10% of **total\_memory**.
* If **free\_memory** is more than double the amount of **reserved\_memory**, Rubik increases **memory.high**. The amount increased each time is 1% of **total\_memory**.

**Kernel interface**

* The interface exists in the cgroup of the container in the `/sys/fs/cgroup/memory` directory, for example, `/sys/fs/cgroup/memory/kubepods/burstable/<PodUID>/<container-longid>`. When the fssr strategy is used, Rubik adjusts the following value of offline service containers based on the memory usage of the current node:
* memory.high

### dynMemory Configuration

The strategy and check interval of the dynMemory module can be specified in **dynMemory**:

```json
"dynMemory": {
    "policy": "fssr"
}
```

* **policy** indicates the dynMemory policy, which supports **fssr**.

## Flexible Bandwidth

To effectively solve the problem of QoS deterioration caused by the CPU bandwidth limit of a service, the Rubik provides flexible bandwidth to allow the container to use extra CPU resources, ensuring stable service running. The flexible bandwidth solution is implemented in both kernel mode and user mode. They cannot be used at the same time.

The user-mode solution is implemented through the CFS bandwidth control capability provided by the Linux kernel. On the premise that the load watermark of the entire system is secure and stable and does not affect the running of other services, the dual-watermark mechanism allows service containers to adaptively adjust the CPU bandwidth limit, relieving CPU resource bottlenecks and improving service performance.

The kernel-mode solution is implemented through the CPU burst capability provided by the Linux kernel, which allows containers to temporarily exceed its CPU usage limit. You need to manually configure the kernel-mode configuration by setting the burst value for each pod. Rubik does not automatically sets the values.

### quotaTurbo User-Mode Solution

You need manually set the **volcano.sh/quota-turbo="true"** annotation for the service pod that requires flexible CPU bandwidth. This annotation takes effect only for the pod whose CPU quota is limited, that is, **CPULimit** is specified in the YAML file.
The user-mode flexible bandwidth policy periodically adjusts the CPU quota of an allowlist container based on the CPU load of the entire system and container running status, and automatically checks and restores the quota values of all containers when Rubik is started or stopped. (The CPU quota described in this section refers to the **cpu.cfs\_quota\_us** parameter of the container.) The adjustment policies are as follows:

1. When the CPU load of the entire system is lower than the alarm threshold, if the allowlist container is suppressed by the CPU in the current period, Rubik slowly increases the CPU quota of the container based on the suppression status. The total container quota increase in a single period cannot exceed 1% of the total CPU quota of the current node.
2. When the CPU load of the entire system is higher than the high watermark, if the allowlist container is not suppressed by the CPU in the current period, Rubik slowly increases the container quota based on the watermark.
3. When the CPU load of the entire system is higher than the alarm threshold, if the current quota value of the allowlist container exceeds the configured value, Rubik quickly decreases the CPU quotas of all containers to ensure that the load is lower than the alarm watermark.
4. The maximum CPU quota that a container can have cannot exceed twice the configured value (for example, the **CPULimit** parameter specified in the pod YAML file), and cannot be less than the configured value.
5. The overall CPU usage of the container within 60 synchronization periods cannot exceed the configured value.
6. If the overall CPU usage of a node exceeds 10% within 1 minute, the container quota will not be increased in this period.

**Kernel interface**

The interface exists in the cgroup of the container in the `/sys/fs/cgroup/cpu` directory, for example, `/sys/fs/cgroup/cpu,cpuacct/kubepods/burstable/<PodUID>/<container-longid>`. The following files are involved:

* **cpu.cfs\_quota\_us**
* **cpu.cfs\_period\_us**
* **cpu.stat**

#### quotaTurbo Configuration

The quotaTurbo function is configured as follows:

```json
"agent": {
    "enabledFeatures": [
      "quotaTurbo"
    ]
  },
"quotaTurbo": {
    "highWaterMark": 60,
    "alarmWaterMark": 80,
    "syncInterval": 100
}
```

For details, see [Configuration Description](./configuration.md#quotaturbo).

* **highWaterMark** is the high watermark of CPU load.
* **alarmWaterMark** is the alarm watermark of CPU load.
* **syncInterval** is the interval for triggering container quota updates, in milliseconds.

You need to manually specify the **volcano.sh/quota-turbo="true"** annotation for the service pod.

```yaml
metadata:
  annotations:
    # true means to add the pod to the allowlist of quotaTurbo
    volcano.sh/quota-turbo : "true"
```

### quotaBurst Kernel-Mode Solution

quotaBurst can be enabled through the **cpu.cfs\_burst\_us** kernel interface. Rubik allows a container to accumulate CPU resources when the CPU usage of the container is lower than the quota and uses the accumulated CPU resources when the CPU usage exceeds the quota.

**Kernel interface**

The interface exists in the cgroup of the container in the `/sys/fs/cgroup/cpu` directory, for example, `/sys/fs/cgroup/cpu/kubepods/burstable/<PodUID>/<container-longid>`. The annotation value is written into the following file:

* **cpu.cfs\_burst\_us**

> The kernel-mode solution is implemented through the **cpu.cfs\_burst\_us** interface. The **cpu.cfs\_burst\_us** file must exist in the CPU subsystem directory of the cgroup. The value of **cpu.cfs\_burst\_us** can be as follows:
>
> 1. When **cpu.cfs\_quota\_us** is not -1, the sum of **cfs\_burst\_us** and **cfs\_quota\_us** must not be greater than $2^{44}$-1, and **cfs\_burst\_us** is less than or equal to **cfs\_quota\_us**.
> 2. When **cpu.cfs\_quota\_us** is -1, the CPU burst function is not enabled, and **cfs\_burst\_us** is 0.

#### quotaBurst Configuration

The quotaBurst function is configured as follows:

```json
"agent": {
  "enabledFeatures": [
    "quotaBurst"
  ]
}
```

You need to manually specify the **volcano.sh/quota-burst-time** annotation for the service pod or run `kubectl annotate` to dynamically add the annotation.

* In the YAML file upon pod creation:

  ```yaml
  metadata:
    annotations:
      # The default unit is microsecond.
      volcano.sh/quota-burst-time : "2000"
  ```

* Modify annotation: You can run the kubectl annotate command to dynamically modify annotation. For example:

  ```bash
  kubectl annotate --overwrite pods <podname> volcano.sh/quota-burst-time='3000'
  ```

### Constraints

* The user-mode CPU bandwidth control is implemented through the **cpu.cfs\_period\_us** (CFS bandwidth control) and **cpu.cfs\_quota\_us** parameters. The following restrictions apply:
  * To avoid unknown errors, other users are not allowed to modify CFS bandwidth control parameters (including but not limited to **cpu.cfs\_quota\_us** and **cpu.cfs\_period\_us**).
  * Do not use this function together with similar programs that limit CPU resources. Otherwise, the user-mode function cannot be used properly. Similar programs include but are not limited to Kubernetes VPA and HPA, Tencent EVPA, Alibaba CPU Burst, and CPU-share and core binding provided by cgroup.
  * If you monitor the metrics related to CFS bandwidth control, using this feature may affect the consistency of the monitored metrics.
* The following restrictions apply to the kernel-mode solution:
  * Use the Kubernetes interface to set the burst value of the pod. Do not manually modify the **cpu.cfs\_burst\_us** file in the CPU cgroup directory of the container.
* Do not enable both kernel-mode and user-mode flexible bandwidth solutions at the same time.

## I/O Weight Control Based on ioCost

To solve the problem that the QoS of online services deteriorates due to high I/O usage of offline services, Rubik provides the I/O weight control function based on ioCost of cgroup v1.
For more, see the [ioCost description](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html#io:~:text=correct%20memory%20ownership.-,IO,-%C2%B6).

**Prerequisites**

Rubik can control the I/O weight distribution of different pods through iocost of cgroup v1. Therefore, the kernel must support the following features:

* cgroup v1 blkcg iocost
* cgroup v1 writeback

The **blkio.cost.qos** and **blkio.cost.model** file interfaces exist in the **blkcg** root system file. For details about the implementation and interface description, see the openEuler kernel document.

### ioCost Implementation Description

![](./figures/iocost.PNG)

The procedure of the Rubik implementation is as follows:

* When Rubik is deployed, Rubik parses the configuration and sets iocost parameters.
* Rubik registers the detection event to the Kubernetes API server.
* When a pod is deployed, the pod configuration information is write back to Rubik.
* Rubik parses the pod configuration information and configures the pod iocost weight based on the QoS level.

### ioCost Configuration

```json
"agent": {
  "enabledFeatures": [
    "ioCost"
  ]
}
"ioCost": [{
  "nodeName": "k8s-single",
  "config": [
    {
      "dev": "sdb",
      "enable": true,
      "model": "linear",
      "param": {
        "rbps": 10000000,
        "rseqiops": 10000000,
        "rrandiops": 10000000,
        "wbps": 10000000,
        "wseqiops": 10000000,
        "wrandiops": 10000000
      }
    }
  ]
}]
```

For details, see [Configuration Description](./configuration.md#iocost).

> Note: Parameters related to the ioCost linear model can be obtained through [**iocost\_coef\_gen.py**](https://github.com/torvalds/linux/blob/master/tools/cgroup/iocost_coef_gen.py).

## Interference Detection Based on Pressure Stall Information Metrics

Rubik can observe the pressure stall information (PSI) metrics of online pods to determine the pressure, evicts offline pods, and generates log alarms. Rubik uses **some avg10** as the indicator, which indicates the average blocking time proportion of any task within 10s. You can choose to monitor the CPU, memory, and I/O resources as required and set thresholds. If the blocking time proportion exceeds the threshold, Rubik evicts offline pods based on certain policies to release corresponding resources. If the CPU and memory usage of an online pod is high, Rubik evicts the offline service that occupies the most CPU or memory resources. If the I/O of offline services is high, Rubik evicts the offline service that occupies the most CPU resources.

The offline service is identified by the annotation **volcano.sh/preemptable="true"/"false"**.

```yaml
annotations:
    volcano.sh/preemptable: true
```

**Prerequisites**

Rubik depends on the PSI feature of cgroup v1. openEuler 22.03 LTS and later versions support the PSI interface of cgroup v1.
You can run the following command to check whether the PSI interface is enabled in the kernel:

```bash
cat /proc/cmdline | grep "psi=1 psi_v1=1"
```

If no results are returned, add the boot parameter to the kernel cmdline:

```bash
# View the kernel version.
uname -a
# View the boot file of the kernel.
ls /boot/linux openEuler 5.10.0-153.12.0.92.oe2203SP3.x86_64
grubby --update-kernel="/boot/linux openEuler 5.10.0-153.12.0.92.oe2203SP3.x86_64" --args="psi=1 psi_v1=1"
# Reboot.
reboot
```

**Kernel interface**

The interface exists in the cgroup of the container in the `/sys/fs/cgroup/cpuacct` directory, for example, `/sys/fs/cgroup/cpu,cpuacct/kubepods/burstable/<PodUID>/<container-longid>`. The following items are involved:

* **cpu.pressure**
* **memory.pressure**
* **io.pressure**

### psi Configuration

```json
"agent": {
  "enabledFeatures": [
    "psi"
  ]
}
"psi": {
    "interval": 10,
    "resource": [
      "cpu",
      "memory",
      "io"
    ],
    "avg10Threshold": 5.0
}
```

For details, see [Configuration Description](./configuration.md#psi).

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secharden/file_permissions.md
---
# File Permissions

## Setting the Permissions on and Ownership of Files

### Description

In Linux, all objects are processed as files. Even a directory will be processed as a large file containing many files. Therefore, the most important thing in Linux is the security of files and directories. Their security is ensured by permissions and owners.

By default, the permissions and ownership of common directories, executable files, and configuration files in the system are set in openEuler.

### Implementation

The following uses the  **/bin**  directory as an example to describe how to change the permission and ownership of a file:

* Modify the file permission. For example, set the permission on the  **/bin**  directory to  **755**.

  ```bash
  chmod 755 /bin
  ```

* Change the ownership of the file. For example, set the ownership and group of the  **/bin**  directory to  **root:root**.

  ```bash
  chown root:root /bin
  ```

## Deleting Unowned Files

### Description

When deleting a user or group, the system administrator may forget to delete the files of the user or group. If the name of a new user or group is the same as that of the deleted user or group, the new user or group will own files on which it has no permission. You are advised to delete these files.

### Implementation

Delete the file whose user ID does not exist.

1. Search for the file whose user ID does not exist.

   ```bash
   find / -nouser
   ```

2. Delete the found file. In the preceding command,  *filename*  indicates the name of the file whose user ID does not exist.

   ```bash
   rm -f filename
   ```

Delete the file whose group ID does not exist.

1. Search for the file whose group ID does not exist.

   ```bash
   find / -nogroup
   ```

2. Delete the found file. In the preceding command,  *filename*  indicates the name of the file whose group ID does not exist.

   ```bash
   rm -f filename
   ```

## Removing a Symbolic Link to /dev/null

### Description

A symbolic link to  **/dev/null**  may be used by malicious users. This affects system security. You are advised to delete these symbolic links to improve system security.

### Special Scenario

After openEuler is installed, symbolic links to  **/dev/null**  may exist. These links may have corresponding functions. (Some of them are preconfigured and may be depended by other components.) Rectify the fault based on the site requirements. For details, see  [Implementation](#en-us_topic_0152100319_l4dc74664c4fb400aaf91fb314c4f9da6).

For example, openEuler supports UEFI and legacy BIOS installation modes. The GRUB packages supported in the two boot scenarios are installed by default. If you select the legacy BIOS installation mode, a symbolic link  **/etc/grub2-efi.cfg**  is generated. If you select the UEFI installation mode, a symbolic link  **/etc/grub2.cfg**  is generated. You need to process these symbolic links based on the site requirements.

### Implementation

1. Run the following command to search for symbolic links to  **/dev/null**:

   ```bash
   find dirname -type l -follow 2>/dev/null
   ```

   > \[!NOTE] **NOTE:**
   > *dir\_\_name*  indicates the directory to be searched. Normally, key system directories, such as  **/bin**,  **/boot**,  **/usr**,  **/lib64**,  **/lib**, and  **/var**, need to be searched.

2. If these symbolic links are useless, run the following command to delete them:

   ```bash
   rm -f filename
   ```

   > \[!NOTE] **NOTE:**
   > *filename*  indicates the file name obtained in  [Step 1](#en-us_topic_0152100319_l4dc74664c4fb400aaf91fb314c4f9da6).

## Setting the umask Value for a Daemon

### Description

The  **umask**  value is used to set default permission on files and directories. If the  **umask**  value is not specified, the file has the globally writable permission. This brings risks. A daemon provides a service for the system to receive user requests or network customer requests. To improve the security of files and directories created by the daemon, you are advised to set  **umask**  to  **0027**. The  **umask**  value indicates the complement of a permission. For details about how to convert the  **umask**  value to a permission, see  [umask Values](./appendix.md#umask-values).

> \[!NOTE] **NOTE:**
> By default, the  **umask**  value of the daemon is set to  **0022**  in openEuler.

### Implementation

In configuration file  **/etc/sysconfig/init**, add  **umask 0022**  as a new row.

## Adding a Sticky Bit Attribute to Globally Writable Directories

### Description

Any user can delete or modify a file or directory in a globally writable directory, which leads to unauthorized file or directory deletion. Therefore, the sticky bit attribute is required for globally writable directories.

### Implementation

1. Search for globally writable directories.

   ```bash
   find / -type d -perm -0002 ! -perm -1000 -ls | grep -v proc
   ```

2. Add the sticky bit attribute to globally writable directories.  *dirname*  indicates the name of the directory that is found.

   ```bash
   chmod +t dirname
   ```

## Disabling the Globally Writable Permission on Unauthorized Files

### Description

Any user can modify globally writable files, which affects system integrity.

### Implementation

1. Search for all globally writable files.

   ```bash
   find / -type d -perm -o+w | grep -v proc
   find / -type f -perm -o+w | grep -v proc
   ```

2. View the settings of files (excluding files and directories with sticky bits) listed in step 1, and delete the files or disable the globally writable permission on them. Run the following command to remove the permission. In the command,  *filename*  indicates the file name.

   ```bash
   chmod o-w  filename
   ```

   > \[!NOTE] **NOTE:**
   > You can run the following command to check whether the sticky bit is set for the file or directory. If the command output contains the  **T**  flag, the file or directory is with a sticky bit. In the command,  *filename*  indicates the name of the file or directory to be queried.
   >
   > ```bash
   > ls -l filename  
   > ```

## Restricting Permissions on the at Command

### Description

The  **at**  command is used to create a scheduled task. Users who can run the  **at**  command must be specified to protect the system from being attacked.

### Implementation

1. Delete the  **/etc/at.deny**  file.

   ```bash
   rm -f /etc/at.deny
   ```

2. Create the  **/etc/at.allow**  file.

   ```bash
   touch /etc/at.allow
   ```

3. Run the following command to change the ownership of the  **/etc/at.allow**  file to  **root:root**.

   ```bash
   chown root:root /etc/at.allow
   ```

4. Set that only user  **root**  can operate file  **/etc/at.allow**.

   ```bash
   chmod og-rwx /etc/at.allow
   ```

## Restricting Permissions on the cron Command

### Description

The  **cron**  command is used to create a routine task. Users who can run the  **cron**  command must be specified to protect the system from being attacked.

### Implementation

1. Delete the  **/etc/cron.deny**  file.

   ```bash
   rm -f /etc/at.deny
   ```

2. Create the  **/etc/cron.allow**  file.

   ```bash
   touch /etc/cron.allow
   ```

3. Run the following command to change the ownership of the  **/etc/cron.allow**  file to  **root:root**:

   ```bash
   chown root:root /etc/cron.allow
   ```

4. Set that only user  **root**  can operate file  **/etc/cron.allow**.

   ```bash
   chmod og-rwx /etc/cron.allow
   ```

## Restricting Permissions on the sudo Command

### Description

A common user can use the  **sudo**  command to run commands as the user  **root**. To harden system security, it is necessary to restrict permissions on the  **sudo**  command. Only user  **root**  can use the  **sudo**  command. By default, openEuler does not restrict the permission of non-root users to run the sudo command.

### Implementation

Modify the  **/etc/sudoers**  file to restrict permissions on the  **sudo**  command. Comment out the following configuration line:

```text
#%wheel ALL=(ALL)       ALL
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/performance/fuse/fuse_acceleration_feature_guide.md
---
# FUSE fastpath特性说明和使用指南

## 介绍

FUSE（Filesystem in Userspace）允许非 root 用户在用户空间创建自己的文件系统，开发者不必修改内核代码，也无需内核模块开发经验，就能够快速实现新的、自定义的文件系统。FUSE 具有简化文件系统开发、扩展文件系统能力、提升安全性和稳定性的优点，但由于对 IO 性能的影响较大，其使用有所受限。为了提高 FUSE 的性能，openEuler 为 FUSE 提供了 fastpath 特性，该特性通过预创建线程绑核、共享内存、快速进程切换等功能实现了单线程和多线程下 FUSE 的加速。具体而言，有如下 3 项技术：

1. 当挂载用户态文件系统时，就在每个 cpu 上都创建 1 个对应的 FUSE daemon，并将其绑定到对应的 CPU 上。当用户进程下发 IO 后，直接唤起同一 CPU 上的 FUSE daemon，这减少了创建和销毁线程的开销，同时可以尽可能利用 CPU 的 cache （用户进程和 FUSE daemon 通常会访问同样的内存）。每个线程绑核后相当于有了一个 percpu 的变量，不同 CPU 上的用户进程下发 IO 后不会互相竞争锁，提高了并行性。
2. Fuse内核模块与FUSE daemon间创建一块共享内存，将FUSE的头信息直接放入到共享内存中，避免了寻址和复制部分的开销。
3. 通过快速线程切换技术，可以在内核中显式地切换 CPU 上运行的线程，无需常规的线程切换流程，实现更高的 CPU 利用率。

## 使用方式

### 环境要求

* 硬件要求：ARM64架构处理器。
* 软件要求：目前 FUSE fastpath 特性仅在 openEuler 22.03 LTS SP4 上提供，且需要内核态和用户态的配合。使能 fastpath 特性要求内核最低版本为 5.10.0-264.0.0；用户态部分目前仅对 libfuse 进行了适配，要求 fuse3 软件包的最低版本为 3.10.5-11。开发者可以在 openEuler 的 update 源中获取到更新的软件包。

### 使用方法

#### 动态库链接

fastpath 特性用户态部分以 fuse3 软件包中的 libfuse 动态库形式提供。对于使用 libfuse 动态库的用户态文件系统，链接时可以使用如下方式：

```bash
gcc program.o -L/path/to/lib -lfuse -o program
```

对于已经编译为二进制的用户态文件系统，开发者可以通过如下命令查看当前链接的动态库：

```bash
ldd /path/to/binary
```

若当前链接的动态库并非目标版本，可以使用如下方法：

* 方法一：更新 `/etc/ld.so.conf`

  打开或创建一个新的配置文件，例如 `/etc/d.so.conf.d/newlib.conf`，并添加库所在路径：

  ```bash
  /path/to/lib
  ```

  运行以下命令更新缓存：

  ```bash
  sudo ldconfig
  ```

* 方法二：设置 LD\_LIBRARY\_PATH

  可以通过设置环境变量 LD\_LIBRARY\_PATH 来指定库的路径。这种方法不需要管理员权限，但只对当前会话有效。

  ```bash
  export LD_LIBRARY_PATH=/path/to/lib:$LD_LIBRARY_PATH
  ```

#### 适配和使能

确定链接到正确版本的动态库后，开发者可以通过在创建 fuse session 时增加参数的方式使能 fastpath，具体而言，包括以下3个配置：

* use\_fastpath：使能 fastpath，即上述的预创建线程绑核、共享内存、快速进程切换功能。
* no\_interrupt：不处理 interrupt 请求。
* no\_forget：不处理 forget 请求。

no\_forget 和 no\_interrupt 可以提升性能，但会造成部分功能缺失，需确定用户态文件系统不会使用这两类请求时再启用，否则可能造成问题。

需要注意的是，增加参数是指在使用 libfuse 中创建 fuse session 的函数 fuse\_session\_new 时传入的参数，而非使用挂载所需用户态文件系统的二进制时增加的参数，具体的适配方式因具体的用户态文件系统而异。以libfuse自带的演示demo ( [passthrough\_hp](https://github.com/libfuse/libfuse/blob/fuse-3.10.5/example/passthrough_hp.cc) ) 为例，其代码需做如下适配：

```diff
diff --git a/example/passthrough_hp.cc b/example/passthrough_hp.cc
index 872fc73..1f96820 100644
--- a/example/passthrough_hp.cc
+++ b/example/passthrough_hp.cc
@@ -1146,7 +1146,10 @@ static cxxopts::ParseResult parse_options(int argc, char **argv) {
         ("help", "Print help")
         ("nocache", "Disable all caching")
         ("nosplice", "Do not use splice(2) to transfer data")
-        ("single", "Run single-threaded");
+        ("single", "Run single-threaded")
+        ("nointerrupt", "Do not process interrupt request")
+        ("noforget", "Do not process forget request")
+        ("usefastpath", "use fastpath");
 
     // FIXME: Find a better way to limit the try clause to just
     // opt_parser.parse() (cf. https://github.com/jarro2783/cxxopts/issues/146)
@@ -1225,7 +1228,10 @@ int main(int argc, char *argv[]) {
     if (fuse_opt_add_arg(&args, argv[0]) ||
         fuse_opt_add_arg(&args, "-o") ||
         fuse_opt_add_arg(&args, "default_permissions,fsname=hpps") ||
-        (options.count("debug-fuse") && fuse_opt_add_arg(&args, "-odebug")))
+        (options.count("debug-fuse") && fuse_opt_add_arg(&args, "-odebug")) ||
+        (options.count("nointerrupt") && fuse_opt_add_arg(&args, "-ono_interrupt")) ||
+        (options.count("noforget") && fuse_opt_add_arg(&args, "-ono_forget")) ||
+        (options.count("usefastpath") && fuse_opt_add_arg(&args, "-ouse_fastpath")))
         errx(3, "ERROR: Out of memory");
 
     fuse_lowlevel_ops sfs_oper {};
```

挂载方式：

```bash
passthrough_hp --usefastpath /path/to/src /path/to/mnt
```

参数 `--usefastpath` 进入 passthrough\_hp 后会被解析为 `-ouse_fastpath` 参数，用于创建新的 fuse session，带有该参数的 session 后续在挂载和创建 FUSE daemon 时会使能fastpath。

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/gala/using_gala_anteater.md
---
# gala-anteater使用手册

gala-anteater是一款基于AI的操作系统异常检测平台。主要提供时序数据预处理、异常点发现、异常上报等功能。基于线下预训练、线上模型的增量学习与模型更新，能够很好地适用于多维多模态数据故障诊断。

本文主要介绍如何部署和使用gala-anteater服务，检测训练集群中的慢节点/慢卡。

## 安装

挂载repo源：

```basic
[everything]
name=everything
baseurl=http://121.36.84.172/dailybuild/EBS-openEuler-22.03-LTS-SP4/EBS-openEuler-22.03-LTS-SP4/everything/$basearch/
enabled=1
gpgcheck=0
priority=1

[EPOL]
name=EPOL
baseurl=http://repo.openeuler.org/EBS-openEuler-22.03-LTS-SP4/EPOL/main/$basearch/
enabled=1
gpgcheck=0
priority=1

```

安装gala-anteater：

```bash
yum install gala-anteater
```

## 配置

> ![](./figures/icon-note.gif)**说明：**
>
> gala-anteater采用配置的config文件设置参数启动，配置文件位置: /etc/gala-anteater/config/gala-anteater.yaml。

### 配置文件默认参数

```yaml
Global:
  data_source: "prometheus"

Arangodb:
  url: "http://localhost:8529"
  db_name: "spider"

Kafka:
  server: "192.168.122.100"
  port: "9092"
  model_topic: "gala_anteater_hybrid_model"
  rca_topic: "gala_cause_inference"
  meta_topic: "gala_gopher_metadata"
  group_id: "gala_anteater_kafka"
  # auth_type: plaintext/sasl_plaintext, please set "" for no auth
  auth_type: ""
  username: ""
  password: ""

Prometheus:
  server: "localhost"
  port: "9090"
  steps: "5"

Aom:
  base_url: ""
  project_id: ""
  auth_type: "token"
  auth_info:
    iam_server: ""
    iam_domain: ""
    iam_user_name: ""
    iam_password: ""
    ssl_verify: 0

Schedule:
  duration: 1
  
Suppression:
  interval: 10
```

| 参数        | 含义                                                         | 默认值                       |
| ----------- | ------------------------------------------------------------ | ---------------------------- |
| Global      | 全局配置                                                     | 字典类型                     |
| data\_source | 设置数据来源                                                 | "prometheus"                 |
| Arangodb    | Arangodb图数据库配置信息                                     | 字典类型                     |
| url         | 图数据库Arangodb的ip地址                                     | "<http://localhost:8529>"      |
| db\_name     | 图数据库名                                                   | "spider"                     |
| Kafka       | kafka配置信息                                                | 字典类型                     |
| server      | Kafka Server的ip地址，根据安装节点ip配置                     | "192.168.122.100"            |
| port        | Kafka Server的port，如：9092                                 | "9092"                       |
| model\_topic | 故障检测结果上报topic                                        | "gala\_anteater\_hybrid\_model" |
| rca\_topic   | 根因定位结果上报topic                                        | "gala\_cause\_inference"       |
| meta\_topic  | gopher采集指标数据topic                                      | "gala\_gopher\_metadata"       |
| group\_id    | kafka设置组名                                                | "gala\_anteater\_kafka"        |
| Prometheus  | 数据源prometheus配置信息                                     | 字典类型                     |
| server      | Prometheus Server的ip地址，根据安装节点ip配置                | "localhost"                  |
| port        | Prometheus Server的port，如：9090                            | "9090"                       |
| steps       | 指标采样间隔                                                 | "5"                          |
| Schedule    | 循环调度配置信息                                             | 字典类型                     |
| duration    | 异常检测模型执行频率（单位：分），每x分钟，检测一次          | 1                            |
| Suppression | 告警抑制配置信息                                             | 字典类型                     |
| interval    | 告警抑制间隔(单位: 分)，表示距离上一次告警x分钟内相同告警过滤 | 10                           |

## 启动

执行如下命令启动gala-anteater

```shell
systemctl start gala-anteater
```

> ![](./figures/icon-note.gif)**说明：**
>
> gala-anteater支持启动一个进程实例，启动多个会导致内存占用过大，日志混乱。

### 查询gala-anteater服务慢节点检测执行状态

若日志显示如下内容，说明慢节点正常运行，启动日志也会保存到当前运行目录下`/var/log/gala-anteater/gala-anteater.log`文件中。

```log
2024-12-02 16:25:20,727 - INFO - anteater - Groups-0, metric: npu_chip_info_hbm_used_memory, start detection.
2024-12-02 16:25:20,735 - INFO - anteater - Metric-npu_chip_info_hbm_used_memory single group has data 8. ranks: [0, 1, 2, 3, 4, 5, 6, 7]
2024-12-02 16:25:20,739 - INFO - anteater - work on npu_chip_info_hbm_used_memory, slow_node_detection start.
2024-12-02 16:25:21,128 - INFO - anteater - time_node_compare result: [].
2024-12-02 16:25:21,137 - INFO - anteater - dnscan labels: [-1  0  0  0 -1  0 -1 -1]
2024-12-02 16:25:21,139 - INFO - anteater - dnscan labels: [-1  0  0  0 -1  0 -1 -1]
2024-12-02 16:25:21,141 - INFO - anteater - dnscan labels: [-1  0  0  0 -1  0 -1 -1]
2024-12-02 16:25:21,142 - INFO - anteater - space_nodes_compare result: [].
2024-12-02 16:25:21,142 - INFO - anteater - Time and space aggregated result: [].
2024-12-02 16:25:21,144 - INFO - anteater - work on npu_chip_info_hbm_used_memory, slow_node_detection end.

2024-12-02 16:25:21,144 - INFO - anteater - Groups-0, metric: npu_chip_info_aicore_current_freq, start detection.
2024-12-02 16:25:21,153 - INFO - anteater - Metric-npu_chip_info_aicore_current_freq single group has data 8. ranks: [0, 1, 2, 3, 4, 5, 6, 7]
2024-12-02 16:25:21,157 - INFO - anteater - work on npu_chip_info_aicore_current_freq, slow_node_detection start.
2024-12-02 16:25:21,584 - INFO - anteater - time_node_compare result: [].
2024-12-02 16:25:21,592 - INFO - anteater - dnscan labels: [0 0 0 0 0 0 0 0]
2024-12-02 16:25:21,594 - INFO - anteater - dnscan labels: [0 0 0 0 0 0 0 0]
2024-12-02 16:25:21,597 - INFO - anteater - dnscan labels: [0 0 0 0 0 0 0 0]
2024-12-02 16:25:21,598 - INFO - anteater - space_nodes_compare result: [].
2024-12-02 16:25:21,598 - INFO - anteater - Time and space aggregated result: [].
2024-12-02 16:25:21,598 - INFO - anteater - work on npu_chip_info_aicore_current_freq, slow_node_detection end.

2024-12-02 16:25:21,598 - INFO - anteater - Groups-0, metric: npu_chip_roce_tx_err_pkt_num, start detection.
2024-12-02 16:25:21,607 - INFO - anteater - Metric-npu_chip_roce_tx_err_pkt_num single group has data 8. ranks: [0, 1, 2, 3, 4, 5, 6, 7]
2024-12-02 16:25:21,611 - INFO - anteater - work on npu_chip_roce_tx_err_pkt_num, slow_node_detection start.
2024-12-02 16:25:22,040 - INFO - anteater - time_node_compare result: [].
2024-12-02 16:25:22,040 - INFO - anteater - Skip space nodes compare.
2024-12-02 16:25:22,040 - INFO - anteater - Time and space aggregated result: [].
2024-12-02 16:25:22,040 - INFO - anteater - work on npu_chip_roce_tx_err_pkt_num, slow_node_detection end.

2024-12-02 16:25:22,041 - INFO - anteater - accomplishment: 1/9
2024-12-02 16:25:22,041 - INFO - anteater - accomplishment: 2/9
2024-12-02 16:25:22,041 - INFO - anteater - accomplishment: 3/9
2024-12-02 16:25:22,041 - INFO - anteater - accomplishment: 4/9
2024-12-02 16:25:22,042 - INFO - anteater - accomplishment: 5/9
2024-12-02 16:25:22,042 - INFO - anteater - accomplishment: 6/9
2024-12-02 16:25:22,042 - INFO - anteater - accomplishment: 7/9
2024-12-02 16:25:22,042 - INFO - anteater - accomplishment: 8/9
2024-12-02 16:25:22,042 - INFO - anteater - accomplishment: 9/9
2024-12-02 16:25:22,043 - INFO - anteater - SlowNodeDetector._execute costs 1.83 seconds!
2024-12-02 16:25:22,043 - INFO - anteater - END!
```

## 异常检测输出数据

gala-anteater如果检测到异常点，会将结果输出至kafka的model\_topic，输出数据格式如下：

```json
{
    "Timestamp": 1730732076935, 
    "Attributes": {
        "resultCode": 201, 
        "compute": false, 
        "network": false, 
        "storage": true, 
        "abnormalDetail": [{
            "objectId": "-1", 
            "serverIp": "96.13.19.31", 
            "deviceInfo": "96.13.19.31:8888*-1", 
            "kpiId": "gala_gopher_disk_wspeed_kB", 
            "methodType": "TIME", 
            "kpiData": [], 
            "relaIds": [], 
            "omittedDevices": []
        }], 
        "normalDetail": [], 
        "errorMsg": ""
    }, 
    "SeverityText": "WARN", 
    "SeverityNumber": 13, 
    "is_anomaly": true
}
```

## 输出字段说明

| 输出字段       | 单位   | 含义                                                  |
| -------------- | ------ | ----------------------------------------------------- |
| Timestamp      | ms     | 检测到故障上报的时刻                                  |
| resultCode     | int    | 故障码，201表示故障，200表示无故障                    |
| compute        | bool   | 故障类型是否为计算类型                                |
| network        | bool   | 故障类型是否为网络类型                                |
| storage        | bool   | 故障类型是否为存储类型                                |
| abnormalDetail | list   | 表示故障的细节                                        |
| objectId       | int    | 故障对象id，-1表示节点故障，0-7表示具体的故障卡号     |
| serverIp       | string | 故障对象ip                                            |
| deviceInfo     | string | 详细的故障信息                                        |
| kpiId          | string | 检测到故障的算法类型，"TIME", "SPACE"                 |
| kpiData        | list   | 故障时序数据，需开关打开，默认关闭                    |
| relaIds        | list   | 故障卡关联的正常卡，表示在”SPACE“算法下对比的正常卡号 |
| omittedDevices | list   | 忽略显示的卡号                                        |
| normalDetail   | list   | 正常卡的时序数据                                      |
| errorMsg       | string | 错误信息                                              |
| SeverityText   | string | 错误类型，表示"WARN", "ERROR"                         |
| SeverityNumber | int    | 错误等级                                              |
| is\_anomaly     | bool   | 表示是否故障                                          |

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/maintenance/aops/configuration_tracing_service_user_manual.md
---
# gala-ragdoll User Guide

This document is currently not available in English.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/maintenance/aops/configuration_tracing_service_user_manual.md
---
# gala-ragdoll的使用指导

\============================

## 安装

### 手动安装

* 通过yum挂载repo源实现

  配置yum源：openEuler23.09 和 openEuler23.09:Epol，repo源路径：/etc/yum.repos.d/openEuler.repo。

  ```ini
  [everything] # openEuler 23.09 官方发布源
  name=openEuler23.09
  baseurl=https://repo.openeuler.org/openEuler-23.09/everything/$basearch/ 
  enabled=1
  gpgcheck=1
  gpgkey=https://repo.openeuler.org/openEuler-23.09/everything/$basearch/RPM-GPG-KEY-openEuler
   
  [Epol] # openEuler 23.09:Epol 官方发布源
  name=Epol
  baseurl=https://repo.openeuler.org/openEuler-23.09/EPOL/main/$basearch/ 
  enabled=1
  gpgcheck=1
  gpgkey=https://repo.openeuler.org/openEuler-23.09/OS/$basearch/RPM-GPG-KEY-openEuler
  ```

  然后执行如下指令下载以及安装gala-ragdoll及其依赖。

  ```shell
  yum install gala-ragdoll # A-Ops 配置溯源
  yum install python3-gala-ragdoll

  yum install gala-spider # A-Ops 架构感知
  yum install python3-gala-spider
  ```

* 通过安装rpm包实现。先下载gala-ragdoll-vx.x.x-x.oe1.aarch64.rpm，然后执行如下命令进行安装（其中x.x-x表示版本号，请用实际情况替代）

  ```shell
  rpm -ivh gala-ragdoll-vx.x.x-x.oe1.aarch64.rpm
  ```

### 使用Aops部署服务安装

#### 编辑任务列表

修改部署任务列表，打开gala\_ragdoll步骤开关：

```yaml
---
step_list:
 ...
 gala_ragdoll:
   enable: false
   continue: false
 ...
```

#### 编辑主机清单

具体步骤参见gala-ragdoll模块主机配置

#### 编辑变量列表

具体步骤参见gala-ragdoll模块变量配置

#### 执行部署任务

具体步骤参见执行部署任务

### 配置文件介绍

`/etc/yum.repos.d/openEuler.repo`是用来规定yum源地址的配置文件，该配置文件内容为：

```shell
[OS]
name=OS
baseurl=http://repo.openeuler.org/openEuler-23.09/OS/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-23.09/OS/$basearch/RPM-GPG-KEY-openEuler
```

### yang模型介绍

`/etc/yum.repos.d/openEuler.repo`采用yang语言进行表示，参见`gala-ragdoll/yang_modules/openEuler-logos-openEuler.repo.yang`;
其中增加了三个拓展字段：

| 拓展字段名称 | 拓展字段格式           | 样例                                      |
| ------------ | ---------------------- | ----------------------------------------- |
| path         | OS类型：配置文件的路径 | openEuler:/etc/yum.repos.d/openEuler.repo |
| type         | 配置文件类型           | ini、key-value、json、text等              |
| spacer       | 配置项和配置值的中间键 | “ ”、“=”、“：”等                          |

附：yang语言的学习地址：<https://tonydeng.github.io/rfc7950-zh/>

### 通过配置溯源创建域

#### 查看配置文件

gala-ragdoll中存在配置溯源的配置文件

```shell
[root@openeuler-development-1-1drnd ~]# cat /etc/ragdoll/gala-ragdoll.conf
[git]                                       // 定义当前的git信息：包括git仓的目录和用户信息
git_dir = "/home/confTraceTestConf" 
user_name = "user"
user_email = "email"

[collect]                                  // A-OPS 对外提供的collect接口
collect_address = "http://192.168.0.0:11111"
collect_api = "/manage/config/collect"

[ragdoll]
port = 11114
```

#### 创建配置域

![](./figures/config_trace/chuangjianyewuyu.png)

#### 添加配置域纳管node

![](./figures/config_trace/tianjianode.png)

#### 添加配置域配置

![](./figures/config_trace/xinzengpeizhi.png)

#### 查询预期配置

![](./figures/config_trace/chakanyuqi.png)

#### 删除配置

![](./figures/config_trace/shanchupeizhi.png)

#### 查询实际配置

![](./figures/config_trace/chaxunshijipeizhi.png)

#### 配置校验

![](./figures/config_trace/zhuangtaichaxun.png)

#### 配置同步

![](./figures/config_trace/peizhitongbu.png)

#### 配置文件追溯

##### 打开监控开关

![](./figures/config_trace/chuangjianyewuyu.png)

##### 配置文件修改记录追溯

![](./figures/config_trace/conf_file_trace.png)

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/gala/using_gala_spider.md
---
# gala-spider使用手册

本章主要介绍如何部署和使用gala-spider和gala-inference。

## gala-spider

gala-spider 提供 OS 级别的拓扑图绘制功能，它将定期获取 gala-gopher （一个 OS 层面的数据采集软件）在某个时间点采集的所有观测对象的数据，并计算它们之间的拓扑关系，最终将生成的拓扑图保存到图数据库 arangodb 中。

### 安装

挂载 yum 源：

```basic
[oe-22.03-lts-SP4-everything] # openEuler 22.03-LTS-SP4 官方发布源
name=oe-2203-lts-SP4-everything
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-update] # openEuler 22.03-LTS-SP4 Update 官方发布源
name=oe-22.03-lts-SP4-epol-update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/update/main/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-main] # openEuler 22.03-LTS-SP4 EPOL 官方发布源
name=oe-22.03-lts-SP4-epol-main
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/x86_64/
enabled=1
gpgcheck=0
priority=1
```

安装 gala-spider：

```sh
# yum install gala-spider
```

### 配置

#### 配置文件说明

gala-spider 配置文件为 `/etc/gala-spider/gala-spider.yaml` ，该文件配置项说明如下。

* global：全局配置信息。
  * data\_source：指定观测指标采集的数据库，当前只支持 prometheus。
  * data\_agent：指定观测指标采集代理，当前只支持 gala\_gopher。
* spider：spider配置信息。
  * log\_conf：日志配置信息。
    * log\_path：日志文件路径。
    * log\_level：日志打印级别，值包括 DEBUG/INFO/WARNING/ERROR/CRITICAL 。
    * max\_size：日志文件大小，单位为兆字节（MB）。
    * backup\_count：日志备份文件数量。
* storage：拓扑图存储服务的配置信息。
  * period：存储周期，单位为秒，表示每隔多少秒存储一次拓扑图。
  * database：存储的图数据库，当前只支持 arangodb。
  * db\_conf：图数据库的配置信息。
    * url：图数据库的服务器地址。
    * db\_name：拓扑图存储的数据库名称。
* kafka：kafka配置信息。
  * server：kafka服务器地址。
  * metadata\_topic：观测对象元数据消息的topic名称。
  * metadata\_group\_id：观测对象元数据消息的消费者组ID。
* prometheus：prometheus数据库配置信息。
  * base\_url：prometheus服务器地址。
  * instant\_api：单个时间点采集API。
  * range\_api：区间采集API。
  * step：采集时间步长，用于区间采集API。

#### 配置文件示例

```yaml
global:
    data_source: "prometheus"
    data_agent: "gala_gopher"

prometheus:
    base_url: "http://localhost:9090/"
    instant_api: "/api/v1/query"
    range_api: "/api/v1/query_range"
    step: 1

spider:
    log_conf:
        log_path: "/var/log/gala-spider/spider.log"
        # log level: DEBUG/INFO/WARNING/ERROR/CRITICAL
        log_level: INFO
        # unit: MB
        max_size: 10
        backup_count: 10

storage:
    # unit: second
    period: 60
    database: arangodb
    db_conf:
        url: "http://localhost:8529"
        db_name: "spider"

kafka:
    server: "localhost:9092"
    metadata_topic: "gala_gopher_metadata"
    metadata_group_id: "metadata-spider"
```

### 启动

1. 通过命令启动。

   ```sh
   # spider-storage
   ```

2. 通过 systemd 服务启动。

   ```sh
   # systemctl start gala-spider
   ```

### 使用方法

#### 外部依赖软件部署

gala-spider 运行时需要依赖多个外部软件进行交互。因此，在启动 gala-spider 之前，用户需要将gala-spider依赖的软件部署完成。下图为 gala-spider 项目的软件依赖图。

![gala-spider软件架构图](./figures/gala-spider软件架构图.png)

其中，右侧虚线框内为 gala-spider 项目的 2 个功能组件，绿色部分为 gala-spider 项目直接依赖的外部组件，灰色部分为 gala-spider 项目间接依赖的外部组件。

* **spider-storage**：gala-spider 核心组件，提供拓扑图存储功能。
  1. 从 kafka 中获取观测对象的元数据信息。
  2. 从 Prometheus 中获取所有的观测实例信息。
  3. 将生成的拓扑图存储到图数据库 arangodb 中。
* **gala-inference**：gala-spider 核心组件，提供根因定位功能。它通过订阅 kafka 的异常 KPI 事件触发异常 KPI 的根因定位流程，并基于 arangodb 获取的拓扑图来构建故障传播图，最终将根因定位的结果输出到 kafka 中。
* **prometheus**：时序数据库，gala-gopher 组件采集的观测指标数据会上报到 prometheus，再由 gala-spider 做进一步处理。
* **kafka**：消息中间件，用于存储 gala-gopher 上报的观测对象元数据信息，异常检测组件上报的异常事件，以及 cause-inference 组件上报的根因定位结果。
* **arangodb**：图数据库，用于存储 spider-storage 生成的拓扑图。
* **gala-gopher**：数据采集组件，请提前部署gala-gopher。
* **arangodb-ui**：arangodb 提供的 UI 界面，可用于查询拓扑图。

gala-spider 项目中的 2 个功能组件会作为独立的软件包分别发布。

​ **spider-storage** 组件对应本节中的 gala-spider 软件包。

​ **gala-inference** 组件对应 gala-inference 软件包。

gala-gopher软件的部署参见[gala-gopher使用手册](./using_gala_gopher.md)，此处只介绍 arangodb 的部署。

当前使用的 arangodb 版本是 3.8.7 ，该版本对运行环境有如下要求：

* 只支持 x86 系统
* gcc10 以上

arangodb 官方部署文档参见：[arangodb部署](https://www.arangodb.com/docs/3.9/deployment.html) 。

arangodb 基于 rpm  的部署流程如下：

1. 配置 yum 源。

   ```basic
   [oe-22.03-lts-SP4-everything] # openEuler 22.03-LTS-SP4 官方发布源
   name=oe-2203-lts-SP4-everything
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
   enabled=1
   gpgcheck=0
   priority=1

   [oe-22.03-lts-SP4-epol-main] # openEuler 22.03-LTS-SP4 EPOL 官方发布源
   name=oe-22.03-lts-SP4-epol-main
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/x86_64/
   enabled=1
   gpgcheck=0
   priority=1
   ```

2. 安装 arangodb3。

   ```sh
   # yum install arangodb3
   ```

3. 配置修改。

   arangodb3 服务器的配置文件路径为 `/etc/arangodb3/arangod.conf` ，需要修改如下的配置信息：

   * endpoint：配置 arangodb3 的服务器地址。
   * authentication：访问 arangodb3 服务器是否需要进行身份认证，当前 gala-spider 还不支持身份认证，故此处将authentication设置为 false。

   示例配置如下：

   ```yaml
   [server]
   endpoint = tcp://0.0.0.0:8529
   authentication = false
   ```

4. 启动 arangodb3。

   ```sh
   # systemctl start arangodb3
   ```

#### gala-spider配置项修改

依赖软件启动后，需要修改 gala-spider 配置文件的部分配置项内容。示例如下：

配置 kafka 服务器地址：

```yaml
kafka:
    server: "localhost:9092"
```

配置 prometheus 服务器地址：

```yaml
prometheus:
    base_url: "http://localhost:9090/"
```

配置 arangodb 服务器地址：

```yaml
storage:
    db_conf:
        url: "http://localhost:8529"
```

#### 启动服务

运行 `systemctl start gala-spider` 。查看启动状态可执行 `systemctl status gala-spider` ，输出如下信息说明启动成功。

```sh
[root@openEuler ~]# systemctl status gala-spider
● gala-spider.service - a-ops gala spider service
     Loaded: loaded (/usr/lib/systemd/system/gala-spider.service; enabled; vendor preset: disabled)
     Active: active (running) since Tue 2022-08-30 17:28:38 CST; 1 day 22h ago
   Main PID: 2263793 (spider-storage)
      Tasks: 3 (limit: 98900)
     Memory: 44.2M
     CGroup: /system.slice/gala-spider.service
             └─2263793 /usr/bin/python3 /usr/bin/spider-storage
```

#### 输出示例

用户可以通过 arangodb 提供的 UI 界面来查询 gala-spider 输出的拓扑图。使用流程如下：

1. 在浏览器输入 arangodb 服务器地址，如：<http://localhost:8529> ，进入 arangodb 的 UI 界面。

2. 界面右上角切换至 `spider` 数据库。

3. 在 `Collections` 面板可以看到在不同时间段存储的观测对象实例的集合、拓扑关系的集合，如下图所示：

   ![spider拓扑关系图](./figures/spider拓扑关系图.png)

4. 可进一步根据 arangodb 提供的 AQL 查询语句查询存储的拓扑关系图，详细教程参见官方文档： [aql文档](https://www.arangodb.com/docs/3.8/aql/)。

## gala-inference

gala-inference 提供异常 KPI 根因定位能力，它将基于异常检测的结果和拓扑图作为输入，根因定位的结果作为输出，输出到 kafka 中。gala-inference 组件在 gala-spider 项目下进行归档。

### 安装

挂载 yum 源：

```basic
[oe-22.03-lts-SP4-everything] # openEuler 22.03-LTS-SP4 官方发布源
name=oe-2203-lts-SP4-everything
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-update] # openEuler 22.03-LTS-SP4 Update 官方发布源
name=oe-22.03-lts-SP4-epol-update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/update/main/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-main] # openEuler 22.03-LTS-SP4 EPOL 官方发布源
name=oe-22.03-lts-SP4-epol-main
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/x86_64/
enabled=1
gpgcheck=0
priority=1
```

安装 gala-inference：

```sh
# yum install gala-inference
```

### 配置

#### 配置文件说明

gala-inference 配置文件 `/etc/gala-inference/gala-inference.yaml` 配置项说明如下。

* inference：根因定位算法的配置信息。
  * tolerated\_bias：异常时间点的拓扑图查询所容忍的时间偏移，单位为秒。
  * topo\_depth：拓扑图查询的最大深度。
  * root\_topk：根因定位结果输出前 K 个根因指标。
  * infer\_policy：根因推导策略，包括 dfs 和 rw 。
  * sample\_duration：指标的历史数据的采样周期，单位为秒。
  * evt\_valid\_duration：根因定位时，有效的系统异常指标事件周期，单位为秒。
  * evt\_aging\_duration：根因定位时，系统异常指标事件的老化周期，单位为秒。
* kafka：kafka配置信息。
  * server：kafka服务器地址。
  * metadata\_topic：观测对象元数据消息的配置信息。
    * topic\_id：观测对象元数据消息的topic名称。
    * group\_id：观测对象元数据消息的消费者组ID。
  * abnormal\_kpi\_topic：异常 KPI 事件消息的配置信息。
    * topic\_id：异常 KPI 事件消息的topic名称。
    * group\_id：异常 KPI 事件消息的消费者组ID。
  * abnormal\_metric\_topic：系统异常指标事件消息的配置信息。
    * topic\_id：系统异常指标事件消息的topic名称。
    * group\_id：系统异常指标事件消息的消费者组ID。
    * consumer\_to：消费系统异常指标事件消息的超时时间，单位为秒。
  * inference\_topic：根因定位结果输出事件消息的配置信息。
    * topic\_id：根因定位结果输出事件消息的topic名称。
* arangodb：arangodb图数据库的配置信息，用于查询根因定位所需要的拓扑子图。
  * url：图数据库的服务器地址。
  * db\_name：拓扑图存储的数据库名称。
* log\_conf：日志配置信息。
  * log\_path：日志文件路径。
  * log\_level：日志打印级别，值包括 DEBUG/INFO/WARNING/ERROR/CRITICAL。
  * max\_size：日志文件大小，单位为兆字节（MB）。
  * backup\_count：日志备份文件数量。
* prometheus：prometheus数据库配置信息，用于获取指标的历史时序数据。
  * base\_url：prometheus服务器地址。
  * range\_api：区间采集API。
  * step：采集时间步长，用于区间采集API。

#### 配置文件示例

```yaml
inference:
  # 异常时间点的拓扑图查询所容忍的时间偏移，单位：秒
  tolerated_bias: 120
  topo_depth: 10
  root_topk: 3
  infer_policy: "dfs"
  # 单位： 秒
  sample_duration: 600
  # 根因定位时，有效的异常指标事件周期，单位：秒
  evt_valid_duration: 120
  # 异常指标事件的老化周期，单位：秒
  evt_aging_duration: 600

kafka:
  server: "localhost:9092"
  metadata_topic:
    topic_id: "gala_gopher_metadata"
    group_id: "metadata-inference"
  abnormal_kpi_topic:
    topic_id: "gala_anteater_hybrid_model"
    group_id: "abn-kpi-inference"
  abnormal_metric_topic:
    topic_id: "gala_anteater_metric"
    group_id: "abn-metric-inference"
    consumer_to: 1
  inference_topic:
    topic_id: "gala_cause_inference"

arangodb:
  url: "http://localhost:8529"
  db_name: "spider"

log:
  log_path: "/var/log/gala-inference/inference.log"
  # log level: DEBUG/INFO/WARNING/ERROR/CRITICAL
  log_level: INFO
  # unit: MB
  max_size: 10
  backup_count: 10

prometheus:
  base_url: "http://localhost:9090/"
  range_api: "/api/v1/query_range"
  step: 5
```

### 启动

1. 通过命令启动。

   ```sh
   # gala-inference
   ```

2. 通过 systemd 服务启动。

   ```sh
   # systemctl start gala-inference
   ```

### 使用方法

#### 依赖软件部署

gala-inference 的运行依赖和 gala-spider一样，请参见[外部依赖软件部署](#外部依赖软件部署)。此外，gala-inference 还间接依赖 [gala-spider](#gala-spider) 和 [gala-anteater](./using_gala_anteater.md) 软件的运行，请提前部署gala-spider和gala-anteater软件。

#### 配置项修改

修改 gala-inference 的配置文件中部分配置项。示例如下：

配置 kafka 服务器地址：

```yaml
kafka:
  server: "localhost:9092"
```

配置 prometheus 服务器地址：

```yaml
prometheus:
  base_url: "http://localhost:9090/"
```

配置 arangodb 服务器地址：

```yaml
arangodb:
  url: "http://localhost:8529"
```

#### 启动服务

直接运行 `systemctl start gala-inference` 即可。可通过执行 `systemctl status gala-inference` 查看启动状态，如下打印表示启动成功。

```sh
[root@openEuler ~]# systemctl status gala-inference
● gala-inference.service - a-ops gala inference service
     Loaded: loaded (/usr/lib/systemd/system/gala-inference.service; enabled; vendor preset: disabled)
     Active: active (running) since Tue 2022-08-30 17:55:33 CST; 1 day 22h ago
   Main PID: 2445875 (gala-inference)
      Tasks: 10 (limit: 98900)
     Memory: 48.7M
     CGroup: /system.slice/gala-inference.service
             └─2445875 /usr/bin/python3 /usr/bin/gala-inference
```

#### 输出示例

当异常检测模块 gala-anteater 检测到 KPI 异常后，会将对应的异常 KPI 事件输出到 kafka 中，gala-inference 会一直监测该异常 KPI 事件的消息，如果收到异常 KPI 事件的消息，就会触发根因定位。根因定位会将定位结果输出到 kafka 中，用户可以在 kafka 服务器中查看根因定位的输出结果，基本步骤如下：

1. 若通过源码安装 kafka ，需要进入 kafka 的安装目录下。

   ```sh
   cd /root/kafka_2.13-2.8.0
   ```

2. 执行消费 topic 的命令获取根因定位的输出结果。

   ```sh
   ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic gala_cause_inference
   ```

   输出示例如下：

   ```json
   {
     "Timestamp": 1661853360000,
     "event_id": "1661853360000_1fd37742xxxx_sli_12154_19",
     "Attributes": {
       "event_id": "1661853360000_1fd37742xxxx_sli_12154_19"
     },
     "Resource": {
       "abnormal_kpi": {
         "metric_id": "gala_gopher_sli_rtt_nsec",
         "entity_id": "1fd37742xxxx_sli_12154_19",
         "timestamp": 1661853360000,
         "metric_labels": {
           "machine_id": "1fd37742xxxx",
           "tgid": "12154",
           "conn_fd": "19"
         }
       },
       "cause_metrics": [
         {
           "metric_id": "gala_gopher_proc_write_bytes",
           "entity_id": "1fd37742xxxx_proc_12154",
           "metric_labels": {
             "__name__": "gala_gopher_proc_write_bytes",
             "cmdline": "/opt/redis/redis-server x.x.x.172:3742",
             "comm": "redis-server",
             "container_id": "5a10635e2c43",
             "hostname": "openEuler",
             "instance": "x.x.x.172:8888",
             "job": "prometheus",
             "machine_id": "1fd37742xxxx",
             "pgid": "12154",
             "ppid": "12126",
             "tgid": "12154"
           },
           "timestamp": 1661853360000,
           "path": [
             {
               "metric_id": "gala_gopher_proc_write_bytes",
               "entity_id": "1fd37742xxxx_proc_12154",
               "metric_labels": {
                 "__name__": "gala_gopher_proc_write_bytes",
                 "cmdline": "/opt/redis/redis-server x.x.x.172:3742",
                 "comm": "redis-server",
                 "container_id": "5a10635e2c43",
                 "hostname": "openEuler",
                 "instance": "x.x.x.172:8888",
                 "job": "prometheus",
                 "machine_id": "1fd37742xxxx",
                 "pgid": "12154",
                 "ppid": "12126",
                 "tgid": "12154"
               },
               "timestamp": 1661853360000
             },
             {
               "metric_id": "gala_gopher_sli_rtt_nsec",
               "entity_id": "1fd37742xxxx_sli_12154_19",
               "metric_labels": {
                 "machine_id": "1fd37742xxxx",
                 "tgid": "12154",
                 "conn_fd": "19"
               },
               "timestamp": 1661853360000
             }
           ]
         }
       ]
     },
     "SeverityText": "WARN",
     "SeverityNumber": 13,
     "Body": "A cause inferring event for an abnormal event"
   }
   ```

---

---
url: /en/docs/22.03_LTS_SP4/server/network/gazelle/gazelle_user_guide.md
---
# Gazelle User Guide

## Overview

Gazelle is a high-performance user-mode protocol stack. It directly reads and writes NIC packets in user mode based on DPDK and transmit the packets through shared hugepage memory, and uses the LwIP protocol stack. Gazelle greatly improves the network I/O throughput of applications and accelerates the network for the databases, such as MySQL and Redis.

* High Performance

Zero-copy and lock-free packets that can be flexibly scaled out and scheduled adaptively.

* Universality

Compatible with POSIX without modification, and applicable to different types of applications.

In the single-process scenario where the NIC supports multiple queues, use **liblstack.so** only to shorten the packet path. In other scenarios, use the ltran process to distribute packets to each thread.

## Installation

Configure the Yum source of openEuler and run the`yum` command to install Gazelle.

```sh
yum install dpdk
yum install libconfig
yum install numactl
yum install libboundscheck
yum install libpcap
yum install gazelle
```

> NOTE:
> The version of dpdk must be 21.11-2 or later.

## How to Use

To configure the operating environment and use Gazelle to accelerate applications, perform the following steps:

### 1. Installing the .ko File as the root User

Install the .ko files based on the site requirements to enable the virtual network ports and bind NICs to the user-mode driver.
To enable the virtual network port function, use **rte\_kni.ko**.

```sh
modprobe rte_kni carrier="on"
```

Configure NetworkManager not to manage the KNI NIC.

```sh
[root@localhost ~]# cat /etc/NetworkManager/conf.d/99-unmanaged-devices.conf
[keyfile]
unmanaged-devices=interface-name:kni
[root@localhost ~]# systemctl reload NetworkManager
```

Bind the NIC from the kernel driver to the user-mode driver. Choose one of the following .ko files based on the site requirements.

```sh
#If the IOMMU is available
modprobe vfio-pci

#If the IOMMU is not available and the VFIO supports the no-IOMMU mode
modprobe vfio enable_unsafe_noiommu_mode=1
modprobe vfio-pci

#Other cases
modprobe igb_uio
```

> NOTE:
> You can check whether the IOMMU is enabled based on the BIOS configuration.

### 2. Binding the NIC Using DPDK

Bind the NIC to the driver selected in Step 1 to provide an interface for the user-mode NIC driver to access the NIC resources.

```sh
#Using vfio-pci
dpdk-devbind -b vfio-pci enp3s0 

#Using igb_uio
dpdk-devbind -b igb_uio enp3s0
```

### 3. Configuring Memory Huge Pages

Gazelle uses hugepage memory to improve efficiency. You can configure any size for the memory huge pages reserved by the system using the **root** permissions. Each memory huge page requires a file descriptor. If the memory is large, you are advised to use 1 GB huge pages to avoid occupying too many file descriptors.
Select a page size based on the site requirements and configure sufficient memory huge pages. Run the following commands to configure huge pages:

```sh
#Configuring 1024 2 MB huge pages on node0. The total memory is 2 GB.
echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

#Configuring 5 1 GB huge pages on node0. The total memory is 5 GB.
echo 5 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
```

> NOTE:
> Run the **cat** command to query the actual number of reserved pages. If the continuous memory is insufficient, the number may be less than expected.

### 4. Mounting Memory Huge Pages

Create two directories for the lstack and ltran processes to access the memory huge pages. Run the following commands:

```sh
mkdir -p /mnt/hugepages-ltran
mkdir -p /mnt/hugepages-lstack
chmod -R 700 /mnt/hugepages-ltran
chmod -R 700 /mnt/hugepages-lstack

mount -t hugetlbfs nodev /mnt/hugepages-ltran -o pagesize=2M
mount -t hugetlbfs nodev /mnt/hugepages-lstack -o pagesize=2M
```

> NOTE:
> The huge pages mounted to **/mnt/hugepages-ltran** and **/mnt/hugepages-lstack** must be in the same page size.

### 5. Enabling Gazelle for an Application

Enable Gazelle for an application using either of the following methods as required.

* Recompile the application and replace the sockets interface.

```sh
#Add the Makefile of Gazelle to the application makefile.
-include /etc/gazelle/lstack.Makefile

#Add the LSTACK_LIBS variable when compiling the source code.
gcc test.c -o test ${LSTACK_LIBS}
```

* Use the **LD\_PRELOAD** environment variable to load the Gazelle library.

Use the **GAZELLE\_BIND\_PROCNAME** environment variable to specify the process name, and **LD\_PRELOAD** to specify the Gazelle library path.

```sh
GAZELLE_BIND_PROCNAME=test LD_PRELOAD=/usr/lib64/liblstack.so ./test
```

### 6. Configuring Gazelle

* The **lstack.conf** file is used to specify the startup parameters of lstack. The default path is **/etc/gazelle/lstack.conf**. The parameters in the configuration file are as follows:

|Options|Value|Remarks|
|:---|:---|:---|
|dpdk\_args|--socket-mem (mandatory)--huge-dir (mandatory)--proc-type (mandatory)--legacy-mem--map-perfect-d|DPDK initialization parameter. For details, see the DPDK description.**--map-perfect** is an extended feature. It is used to prevent the DPDK from occupying excessive address space and ensure that extra address space is available for lstack.The **-d** option is used to load the specified .so library file.|
|listen\_shadow| 0/1 | Whether to use the shadow file descriptor for listening. This function is enabled when there is a single listen thread and multiple protocol stack threads.|
|use\_ltran| 0/1 | Whether to use ltran.|
|num\_cpus|"0,2,4 ..."|IDs of the CPUs bound to the lstack threads. The number of IDs is the number of lstack threads (less than or equal to the number of NIC queues). You can select CPUs by NUMA nodes.|
|low\_power\_mode|0/1|Whether to enable the low-power mode. This parameter is not supported currently.|
|kni\_switch|0/1|Whether to enable the rte\_kni module. The default value is **0**. This module can be enabled only when ltran is not used.|
|flow\_bifurcation|0/1 |flow bifurcation switch, transfer ports that Gazelle is not listening to the kernel for processing, default to 0 |
|unix\_prefix|"string"|Prefix string of the Unix socket file used for communication between Gazelle processes. By default, this parameter is left blank. The value must be the same as the value of **unix\_prefix** in **ltran.conf** of the ltran process that participates in communication, or the value of the **-u** option for `gazellectl`. The value cannot contain special characters and can contain a maximum of 128 characters.|
|host\_addr|"192.168.xx.xx"|IP address of the protocol stack, which is also the IP address of the application.|
|mask\_addr|"255.255.xx.xx"|Subnet mask.|
|gateway\_addr|"192.168.xx.1"|Gateway address.|
|devices|"aa:bb:cc:dd:ee:ff"|MAC address for NIC communication. The value must be the same as that of **bond\_macs** in the **ltran.conf** file.|
|app\_bind\_numa|0/1|Whether to bind the epoll and poll threads of an application to the NUMA node where the protocol stack is located. The default value is 1, indicating that the threads are bound.|
|send\_connect\_number|4|Number of connections for sending packets in each protocol stack loop. The value is a positive integer.|
|read\_connect\_number|4|Number of connections for receiving packets in each protocol stack loop. The value is a positive integer.|
|rpc\_number|4|Number of RPC messages processed in each protocol stack loop. The value is a positive integer.|
|nic\_read\_num|128|Number of data packets read from the NIC in each protocol stack cycle. The value is a positive integer.|
|mbuf\_pool\_size|1024000|Size of the mbuf address pool applied for during initialization. Set this parameter based on the NIC configuration. The value must be a positive integer less than 5120000 and not too small, otherwise the startup fails.|

lstack.conf example:

```sh
dpdk_args=["--socket-mem", "2048,0,0,0", "--huge-dir", "/mnt/hugepages-lstack", "--proc-type", "primary", "--legacy-mem", "--map-perfect"]

use_ltran=1
kni_switch=0
flow_bifurcation=0
low_power_mode=0

num_cpus="2,22"

host_addr="192.168.1.10"
mask_addr="255.255.255.0"
gateway_addr="192.168.1.1"
devices="aa:bb:cc:dd:ee:ff"

send_connect_number=4
read_connect_number=4
rpc_number=4
nic_read_num=128
mbuf_pool_size=1024000
```

* The **ltran.conf** file is used to specify ltran startup parameters. The default path is **/etc/gazelle/ltran.conf**. To enable ltran, set **use\_ltran=1** in the **lstack.conf** file. The configuration parameters are as follows:

|Options|Value|Remarks|
|:---|:---|:---|
|forward\_kit|"dpdk"|Specified transceiver module of an NIC.This field is reserved and is not used currently.|
|forward\_kit\_args|-l--socket-mem (mandatory)--huge-dir (mandatory)--proc-TYPE (mandatory)--legacy-mem (mandatory)--map-perfect (mandatory)-d|DPDK initialization parameter. For details, see the DPDK description.**--map-perfect** is an extended feature. It is used to prevent the DPDK from occupying excessive address space and ensure that extra address space is available for lstack.The **-d** option is used to load the specified .so library file.|
|kni\_switch|0/1|Whether to enable the rte\_kni module. The default value is **0**.|
|unix\_prefix|"string"|Prefix string of the Unix socket file used for communication between Gazelle processes. By default, this parameter is left blank. The value must be the same as the value of **unix\_prefix** in **lstack.conf** of the lstack process that participates in communication, or the value of the **-u** option for `gazellectl`.|
|dispatch\_max\_clients|n|Maximum number of clients supported by ltran.The total number of lstack protocol stack threads cannot exceed 32.|
|dispatch\_subnet|192.168.xx.xx|Subnet mask, which is the subnet segment of the IP addresses that can be identified by ltran. The value is an example. Set the subnet based on the site requirements.|
|dispatch\_subnet\_length|n|Length of the Subnet that can be identified by ltran. For example, if the value of length is 4, the value ranges from 192.168.1.1 to 192.168.1.16.|
|bond\_mode|n|Bond mode. Currently, only Active Backup(Mode1) is supported. The value is 1.|
|bond\_miimon|n|Bond link monitoring time. The unit is millisecond. The value ranges from 1 to 2^64 - 1 - (1000 x 1000).|
|bond\_ports|"0x01"|DPDK NIC to be used. The value **0x01** indicates the first NIC.|
|bond\_macs|"aa:bb:cc:dd:ee:ff"|MAC address of the bound NIC, which must be the same as the MAC address of the KNI.|
|bond\_mtu|n|Maximum transmission unit. The default and maximum value is 1500. The minimum value is 68.|

ltran.conf example:

```sh
forward_kit_args="-l 0,1 --socket-mem 1024,0,0,0 --huge-dir /mnt/hugepages-ltran --proc-type primary --legacy-mem --map-perfect --syslog daemon"
forward_kit="dpdk"

kni_switch=0

dispatch_max_clients=30
dispatch_subnet="192.168.1.0"
dispatch_subnet_length=8

bond_mode=1
bond_mtu=1500
bond_miimon=100
bond_macs="aa:bb:cc:dd:ee:ff"
bond_ports="0x1"

tcp_conn_scan_interval=10
```

### 7. Starting an Application

* Start the ltran process.

If there is only one process and the NIC supports multiple queues, the NIC multi-queue is used to distribute packets to each thread. You do not need to start the ltran process. Set the value of **use\_ltran** in the **lstack.conf** file to **0**.
If you do not use `--config-file` to specify a configuration file when starting ltran, the default configuration file path **/etc/gazelle/ltran.conf** is used.

```sh
ltran --config-file ./ltran.conf
```

* Start the application.

If the environment variable **LSTACK\_CONF\_PATH** is not used to specify the configuration file before the application is started, the default configuration file path **/etc/gazelle/lstack.conf** is used.

```sh
export LSTACK_CONF_PATH=./lstack.conf
LD_PRELOAD=/usr/lib64/liblstack.so  GAZELLE_BIND_PROCNAME=redis-server redis-server redis.conf
```

### 8. APIs

Gazelle wraps the POSIX interfaces of the application. The code of the application does not need to be modified.

### 9. Commissioning Commands

* If the ltran mode is not used, the **gazellectl ltran xxx** and **gazellectl lstack show {ip | pid} -r** commands are not supported.

```sh
Usage: gazellectl [-h | help]
  or:  gazellectl ltran  {quit | show | set} [LTRAN_OPTIONS] [time] [-u UNIX_PREFIX]
  or:  gazellectl lstack {show | set} {ip | pid} [LSTACK_OPTIONS] [time] [-u UNIX_PREFIX]

  quit            ltran process exit

  where  LTRAN_OPTIONS :=
                  show ltran all statistics
  -r, rate        show ltran statistics per second
  -i, instance    show ltran instance register info
  -b, burst       show ltran NIC packet len per second
  -l, latency     show ltran latency
  set:
  loglevel        {error | info | debug} set ltran loglevel

  where  LSTACK_OPTIONS :=
                  show lstack all statistics
  -r, rate        show lstack statistics per second
  -s, snmp        show lstack snmp
  -c, connetct    show lstack connect
  -l, latency     show lstack latency
  set:
  loglevel        {error | info | debug} set lstack loglevel
  lowpower        {0 | 1} set lowpower enable
  [time]          measure latency time default 1S
```

The `-u` option specifies the prefix of the Unix socket for communication between Gazelle processes. The value of this parameter must be the same as that of **unix\_prefix** in the **ltran.conf** or **lstack.conf** file.

**Packet Capturing Tool**
The NIC used by Gazelle is managed by DPDK. Therefore, tcpdump cannot capture Gazelle packets. As a substitute, Gazelle uses gazelle-pdump provided in the dpdk-tools software package as the packet capturing tool. gazelle-pdump uses the multi-process mode of DPDK to share memory with the lstack or ltran process. In ltran mode, gazelle-pdump can capture only ltran packets that directly communicate with the NIC. By filtering tcpdump data packets, gazelle-pdump can filter packets of a specific lstack process. ([Usage](https://atomgit.com/openeuler/gazelle/blob/master/doc/en/pdump_en.md))

### 10. Precautions

#### Location of the DPDK Configuration File

For the **root** user, the configuration file is stored in the **/var/run/dpdk** directory after the DPDK is started.
For a non-root user, the path of the DPDK configuration file is determined by the environment variable **XDG\_RUNTIME\_DIR**.

* If **XDG\_RUNTIME\_DIR** is not set, the DPDK configuration file is stored in **/tmp/dpdk**.
* If **XDG\_RUNTIME\_DIR** is set, the DPDK configuration file is stored in the path specified by **XDG\_RUNTIME\_DIR**.
* Note that **XDG\_RUNTIME\_DIR** is set by default on some servers.

#### Impact on Gazelle Performance by the Retbleed Vulnerability Patch

* The patch to fix the Retbleed vulnerability is merged in kernel 5.10.0-60.57.0.85. This patch impacts the performance of Gazelle in x86 environments. You can add **retbleed=off mitigations=off** to the boot parameters to disable the patch and prevent the performance impact based on your security requirements. By default, the patch is enabled for security.
* In the test scenario, 1024 KB of data is sent from kernel space to user space through ltran. The performance decreases from 17,000 Mb/s to 5,000 Mb/s.
* openEuler 22.03 LTS and its SP versions (kernel version 5.10.0-60.57.0.85 or later) are affected.
* For details, see <https://atomgit.com/openeuler/kernel/pulls/110>.

## Restrictions

Restrictions of Gazelle are as follows:

### Function Restrictions

* Blocking **accept()** or **connect()** is not supported.
* A maximum of 1500 TCP connections are supported.
* Currently, only TCP, UDP, IGMPv2, ICMP, ARP, and IPv4 are supported.
* When a peer end pings Gazelle, the specified packet length must be less than or equal to 14,000 bytes.
* Transparent huge pages are not supported.
* ltran does not support the hybrid bonding of multiple types of NICs.
* The active/standby mode (bond1 mode) of ltran supports active/standby switchover only when a fault occurs at the link layer (for example, the network cable is disconnected), but does not support active/standby switchover when a fault occurs at the physical layer (for example, the NIC is powered off or removed).
* VM NICs do not support multiple queues.
* KNI must be enabled with UDP unless the NIC driver (such as mlx5) supports user mode and kernel mode at the same time.

### Operation Restrictions

* By default, the command lines and configuration files provided by Gazelle requires **root** permissions. Privilege escalation and changing of file owner are required for non-root users.
* To bind the NIC from user-mode driver back to the kernel driver, you must exit Gazelle first.
* Memory huge pages cannot be remounted to subdirectories created in the mount point.
* The minimum huge page memory required by ltran is 1 GB.
* The minimum hugepage memory of each application instance protocol stack thread is 800 MB.
* Gazelle supports only 64-bit OSs.
* The `-march=native` option is used when building the x86 version of Gazelle to optimize Gazelle based on the CPU instruction set of the build environment (Intel® Xeon® Gold 5118 CPU @ 2.30GHz). Therefore, the CPU of the operating environment must support the SSE4.2, AVX, AVX2, and AVX-512 instruction set extensions.
* The maximum number of IP fragments is 10 (the maximum ping packet length is 14,790 bytes). TCP does not use IP fragments.
* You are advised to set the **rp\_filter** parameter of the NIC to 1 using the `sysctl` command. Otherwise, the Gazelle protocol stack may not be used as expected. Instead, the kernel protocol stack is used.
* If ltran is not used, the KNI cannot be configured to be used only for local communication. In addition, you need to configure the NetworkManager not to manage the KNI network adapter before starting Gazelle.
* The IP address and MAC address of the virtual KNI must be the same as those in the **lstack.conf** file.

## Precautions

You need to evaluate the use of Gazelle based on application scenarios.

### Shared Memory

* Current situation:
  The memory huge pages are mounted to the **/mnt/hugepages-lstack** directory. During process initialization, files are created in the **/mnt/hugepages-lstack** directory. Each file corresponds to a huge page, and the mmap function is performed on the files. After receiving the registration information of lstask, ltran configures the files in the **mmap** directory of the information page based on the huge page memory configurations, implementing shared huge page memory.
  The procedure also applies to the files in the **/mnt/hugepages-ltran** directory.
* Current mitigation measures
  The huge page file permission is **600**. Only the owner can access the files. The default owner is the **root** user. Other users can be configured.
  Huge page files are locked by DPDK and cannot be directly written or mapped.
* Caution
  Malicious processes belonging to the same user imitate the DPDK implementation logic to share huge page memory using huge page files and perform write operations to damage the huge page memory. As a result, the Gazelle program crashes. It is recommended that the processes of a user belong to the same trust domain.

### Traffic Limit

Gazelle does not limit the traffic. Users can send packets at the maximum NIC line rate to the network, which may congest the network.

### Process Spoofing

If two lstack processes A and B are legitimately registered with ltran, A can impersonate B to send spoofing messages to ltran and modify the ltran forwarding control information. As a result, the communication of B becomes abnormal, and information leakage occurs when packets for B are sent to A. Ensure that all lstack processes are trusted.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/network/gazelle/gazelle_for_opengauss_acceleration.md
---
# Gazelle 加速 openGauss

## 背景介绍

openGauss是一款高性能数据库，Gazelle作为一款高性能用户态协议栈，能够大幅提高应用的网络I/O吞吐能力。通过使用Gazelle加速openGauss，可有效提高openGauss性能。
gazelle在openGauss单机场景中，能够提升15%的性能，一主一备场景中，能够提升5%的性能。

## 功能约束

* 当前验证使用的openGauss版本是openGauss 6.0.0(LTS)版本，其他版本未做过验证，可能不支持。
* 当前gazelle验证通过openGauss单机、一主一备场景的加速使能，其他场景未做过验证，可能不支持。
* Gazelle不支持端口复用：如果在用户态协议栈的IP上使用了某个端口，其他IP不能再使用这个端口。

## 用户态环境搭建

### 前置条件

1. 数据库安装完成以及完成内核态数据库正常启动。内核态具体可参考：[openGauss官网](https://opengauss.org/zh/download/)

2. 安装gazelle及依赖

   ````
    ```sh
    yum -y install gazelle dpdk libconfig numactl libboundscheck libcap 
    ``` 
   ````

3. benchmark 安装使用可参考 [benchmarkSql使用](https://opengauss.org/zh/blogs/optimize/opengauss-tpcc.html)

### Gazelle环境配置

通过以下命令查看支持的大页大小。

```shell
ll /sys/kernel/mm/hugepages/
```

一般来说，每个numa节点需要设置2-3G的大页内存。通常为4个numa节点，即需要8-12G的大页内存。根据支持的大页大小设置大页数量。

例如加入`hugepages-1048576kB`（1G）的大页,这里选择的是20个大页。对应内存为20GB，足够分配使用。需要保证数量为4的倍数。这个数值不宜过大，过大时会占据过多操作系统内存。

```shell
echo 20 > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages

#设置完成之后，在本地创建磁盘挂载：
mkdir /mnt/hugepages-1G
mount -t hugetlbfs -o pagesize=1G nodev /mnt/hugepages-1G

#加载ko
modprobe vfio enable_unsafe_noiommu_mode=1
modprobe vfio-pci
```

> \[!NOTE]说明\
> Gazelle部署详见[Gazelle用户指南](https://atomgit.com/openeuler/gazelle/blob/master/doc/zh/user-guide.md)
> 不同网卡绑定用户态方法详见[Gazelle网卡支持及使用](https://atomgit.com/openeuler/gazelle/blob/master/doc/zh/nic-support.md)

### 配置lstack.conf文件

使用对应数据库用户编写lstack.conf，即保证用户具备该文件的访问权限。

几个关键配置项说明及修改点：

* dpdk\_args

`3096,3096,3096,3096`为每个numa设置的内存大小。这里设置3GB。

`/mnt/hugepages-1G`修改为大页环境配置中设置的路径。

* num\_cpus

`30,31,62,63,94,95,126,127`：lstack设置的网络中断核。这里设置的是每个numa的最后两个CPU号。一共使用8个中断。

* app\_bind\_numa

该参数决定是否将epoll/poll线程绑定到对应的numa节点上。需设置为0，即不绑定。gazelle绑核策略与gauss绑核策略冲突，此处使用gauss绑核策略。

* host\_addr、mask\_addr、gateway\_addr、devices

获取需要的网卡ip信息。一般通过使用`ip addr show`命令查看，即可获取所需信息。gateway\_addr一般是ip将最后一位改为1。可以通过`route -n`查看gateway。
需要配置的网卡信息：

host\_addr：20.20.20.119

mask\_addr：255.255.255.0

gateway\_addr：20.20.20.1

devices：78:b4:6a:40:16:30

执行`ip addr show`命令后截取的网卡信息：

```sh
 8: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
     link/ether 78:b4:6a:40:16:30 brd ff:ff:ff:ff:ff:ff
     inet 20.20.20.149/24 brd 20.20.20.255 scope global enp7s0
        valid_lft forever preferred_lft forever
```

* app\_exclude\_cpus

排除的CPU编号。openGauss采用CPU0作为xlog写入线程。采用`app_exclude_cpus="0"`这个方式排除对CPU0的绑核处理。

* listen\_shadow

是否使用影子fd监听，0为关闭，1为开启。单listen线程，多协议栈线程时使能。此处需要开启。

配置文件参考实例：

```shell
dpdk_args=["--socket-mem", "3096,3096,3096,3096", "--huge-dir", "/mnt/hugepages-1G", "--proc-type", "primary", "--legacy-mem"]

use_ltran=0
kni_switch=0

low_power_mode=0

#needed mbuf count = tcp_conn_count * mbuf_count_per_conn
tcp_conn_count = 1500
mbuf_count_per_conn = 350

# send ring size, default is 32, max is 2048
send_ring_size = 32

# 0: when send ring full, send return
# 1: when send ring full, alloc mbuf from mempool to send data
expand_send_ring = 0

#protocol stack thread per loop params
#read data form protocol stack into recv_ring
read_connect_number = 4
#process rpc msg number
rpc_number = 4
#read nic pkts number
nic_read_number = 128

#each cpu core start a protocol stack thread.
num_cpus="30,31,62,63,94,95,126,127"

#app worker thread bind to numa in epoll/poll.
app_bind_numa=0
#app main thread affinity set by dpdk.
main_thread_affinity=0

host_addr="20.20.20.119"
mask_addr="255.255.255.0"
gateway_addr="20.20.20.1"
devices="78:b4:6a:40:16:31"

udp_enable=0
#0: use rss rule
#1: use tcp tuple rule to specify packet to nic queue
tuple_filter=0

#tuple_filter=1, below cfg valid
num_process=1
process_numa="0,1"
process_idx=0

#tuple_filer=0, below cfg valid
listen_shadow=1
#exclude cpu
app_exclude_cpus="0"
```

## 提权

对gaussdb二进制、liblstack.so提权，对ls命令提权。

```shell
#对gaussdb二进制提权
sudo setcap 'cap_chown,cap_dac_override,cap_dac_read_search,cap_sys_rawio,cap_net_admin,cap_net_raw,cap_sys_admin+eip' `which gaussdb`
#对liblstack.so提权
sudo chmod u+s /usr/lib64/liblstack.so
#对ls命令提权
setcap 'cap_chown,cap_dac_override,cap_dac_read_search,cap_sys_rawio,cap_net_admin,cap_net_raw,cap_sys_admin+eip' $(which ls)
```

提权之后的gaussdb不会再从LD\_LIBRARY\_PATH中查找动态库路径，因此还需要配置全局动态库路径：`vim /etc/ld.so.conf.d/opengauss.conf`。将实际的动态库对应路径内容写入。

```shell
/home/code/openGauss-server/dest/lib
```

## 必要路径权限

需要修改大页内存对应的文件路径权限和liblstack.so权限，允许用户访问。

```shell
chown -R 777 /mnt/
chmod 777 /usr/lib64/liblstack.so
```

还需要确保编写的lstack.conf文件也具备用户访问权限。

## 数据库测试

### 必要信息准备

首先数据库测试需要至少一块NVME硬盘用于数据存储。

本地测试为HINIC网卡。

一些信息可以下面这些命令查询，假设网卡名为：enp8s0。

```shell
# PCI查询：
 ethtool -i enp8s0 | grep bus-info
```

### 切换用户态网卡

该操作会导致该网卡无法使用。请使用BMC机器操作，或者具备双网口的环境操作。否则环境会不可用。
使用root用户切换网卡为用户态网卡。该操作会停止该网卡并用dpdk接管。如果使用该网卡建立ssh连接，会直接断开连接。

```shell
#PCI=0000:08:00.0
#NET_DEVICE=enp8s0
ifconfig enp8s0 down
sudo modprobe vfio enable_unsafe_noiommu_mode=1
sudo echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
sudo dpdk-devbind -b vfio-pci 0000:08:00.0
```

### 启动数据库

openGauss 配置文件修改：

在openGauss数据库的配置文件‘postgresql.conf’ 追加参数：[追加参数文件](#gauss_config)

数据库配置 postgresql.conf文件修改，需要需根据实际情况修改线程池相关参数，以及增加数据配置文件中性能参数。

```shell
# 此参数为 openGauss 绑定cpu 个数， 如 openGauss 绑定 1-29 32-61 64-93 96-125 共计 4 numa 分区，119 个 cpu, 线程数为 cpu 的整数倍，以4倍为例，为 476
thread_pool_attr = '476,4,(cpubind:1-29,32-61,64-93,96-125)'

# 此参数在使用 gazelle 加速时打开此选项
enable_gazelle_performance_mode=on
```

环境清理：
为避免后台缓存和后台信号量对测试有影响，需要清理下对应的数据。

```shell
ipcs -m | awk '$2 ~/[0-9]+/ {print $2}' | while read s; do ipcrm -m $s; done
ipcs -s | awk '$2 ~/[0-9]+/ {print $2}' | while read s; do ipcrm -s $s; done
echo 3 > /proc/sys/vm/drop_caches
```

启动数据库：

```shell
# 设置Gazelle 配置文件路径
LSTACK_CONF_PATH=/usr1/gazelle/gazelle_conf/lstack.conf
# 启动 Gazelle 加速 gaussdb, 已启动 openGauss 下dn1 数据库节点为例
LD_PRELOAD=liblstack.so GAZELLE_BIND_PROCNAME=gaussdb LSTACK_CONF_PATH=$LSTACK_CONF_PATH /home/opengauss/software/openGauss/bin/gaussdb -D /home/opengauss/software/openGauss/data/dn1
```

### 启动TPCC测试

客户端需要根据实际情况配置网络中断，此处以配置20个中断为例，对应的bind\_irq.sh文件：

```shell
intf=enp4s0

ethtool -G ${intf} rx 1024 tx 1024
ethtool -K ${intf} lro on
ethtool -L ${intf} combined ${combined}

irq_list=`cat /proc/interrupts | grep $intf | awk {'print $1'} | tr -d ":"`
irq_array_net=($irq_list)

cpu_array_irq=(27 28 29 30 31 59 60 61 62 63 91 92 93 94 95 123 124 125 126 127)

for (( i=0;i<20;i++))
do
        echo "${cpu_array_irq[$i]}" > /proc/irq/${irq_array_net[$i]}/smp_affinity_list
done

for j in ${irq_array_net[@]}
do
        cat /proc/irq/$j/smp_affinity_list
done
```

benchmark客户端需要根据实际情况修改并发数。
通常修改为postgresql.conf配置文件中cpubind参数，核数量的倍数如1-29,32-61,64-93,96-125核数为119，取4倍得714，则将并发数修改为714：

```shell
terminals=714
```

### 恢复

1. 停止数据库。
2. 使用root切换回内核态网卡。

将用户态网卡切换为内核态网卡:

```shell
#PCI=0000:08:00.0
#NET_DEVICE=enp8s0
#IP=20.20.20.119
sudo dpdk-devbind -u 0000:08:00.0
sudo dpdk-devbind -b hinic 0000:08:00.0
sudo ifconfig enp8s0 20.20.20.119 netmask 255.255.255.0
```

### 测试结论

在单机测试中，实测TPCC性能大致提升15%左右。

## openGauss文件配置参考

postgresql.conf配置文件：打开postgresql.conf文件后，在文件末尾追加如下内容(IP需根据实际ip更改)：

```shell
max_connections = 4096
allow_concurrent_tuple_update = true
audit_enabled = off
checkpoint_segments = 1024
checkpoint_timeout = 15min
cstore_buffers = 16MB
enable_alarm = off
enable_codegen = false
enable_data_replicate = off
full_page_writes = off
max_files_per_process = 100000
max_prepared_transactions = 2048
shared_buffers = 350GB
#max_process_memory = 100GB
use_workload_manager = off
wal_buffers = 1GB
work_mem = 1MB
log_min_messages = FATAL
transaction_isolation = 'read committed'
default_transaction_isolation = 'read committed'
synchronous_commit = on
fsync = on
maintenance_work_mem = 2GB
vacuum_cost_limit = 10000
autovacuum = on
autovacuum_mode = vacuum
autovacuum_max_workers = 20
autovacuum_naptime = 5s
autovacuum_vacuum_cost_delay = 10
#xloginsert_locks = 48
update_lockwait_timeout = 20min

enable_mergejoin = off
enable_nestloop = off
#-----
enable_hashjoin = off
#enable_cbm_tracking = off
enable_bitmapscan = on
enable_material = off

wal_log_hints = off
log_duration = off
checkpoint_timeout = 15min
autovacuum_vacuum_scale_factor = 0.1
autovacuum_analyze_scale_factor = 0.02
enable_save_datachanged_timestamp = false

# enable_stat_send = off
# enable_commandid_send = off
log_timezone = 'PRC'
timezone = 'PRC'
lc_messages = 'C'
lc_monetary = 'C'
lc_numeric = 'C'
lc_time = 'C'

enable_thread_pool = on

enable_double_write = on

enable_incremental_checkpoint = on
enable_opfusion = on
instr_unique_sql_count=0
advance_xlog_file_num = 100
numa_distribute_mode = 'all'

track_activities = off
enable_instr_track_wait = off
enable_instr_rt_percentile = off
track_counts = off
track_sql_count = off
enable_instr_cpu_timer = off

plog_merge_age = 0

# numa_excluded_cpus = '29,30,31,61,62,63,93,94,95,125,126,127'
# numa_excluded_cpus = '28,29,30,31,60,61,62,63,92,93,94,95,124,125,126,127'

#enable_crc_check = off
session_timeout = 0

enable_instance_metric_persistent = off
enable_logical_io_statistics = off
enable_page_lsn_check = off
enable_user_metric_persistent = off
enable_xlog_prune = off

enable_resource_track = off
instr_unique_sql_count=0

remote_read_mode = non_authentication
client_min_messages = ERROR
log_min_messages = FATAL
enable_asp = off
enable_bbox_dump = off
bgwriter_flush_after = 32
#gs_clean_timeout = '300s'
#minimum_pool_size = 200
wal_keep_segments = 1025


#scan_fusion
enable_bitmapscan = off
enable_seqscan = off

track_activities='off'
enable_resource_track='off'
enable_instr_rt_percentile='off'
enable_instr_cpu_timer='off'
# bypass_workload_manager='off'
enable_asp='off'

enable_thread_pool = on
xloginsert_locks=16
checkpoint_segments=8000

enable_stmt_track=false

data_replicate_buffer_size=128MB

#bgwriter_thread_num = 1

bgwriter_delay = 5s
incremental_checkpoint_timeout = 5min
pagewriter_thread_num = 2
candidate_buf_percent_target = 0.3
pagewriter_sleep = 100ms

standby_shared_buffers_fraction = 0.9

#replconninfo1 = 'localhost=20.20.20.58 localport=21131 localheartbeatport=21135 localservice=21134 remotehost=20.20.20.64 remoteport=21131 remoteheartbeatport=21135 remoteservice=21134'

# 2020-12-20
enable_incremental_checkpoint = on
checkpoint_segments = '4096'
checkpoint_timeout = 50min
#enable_thread_pool = off

max_process_memory = 240GB
shared_buffers = 180GB

wal_level = archive
hot_standby = off

wal_receiver_buffer_size = 256MB
walsender_max_send_size = 32MB


# 2021-03-16
xloginsert_locks = 8
wal_file_init_num = 30
#xlog_idle_flushes_before_sleep = 500000000
walwriter_sleep_threshold = 50000
wal_writer_delay = 150

undo_zone_count = 0

walwriter_cpu_bind = 0
checkpoint_segments = 32768

local_syscache_threshold = 16MB
enable_cachedplan_mgr = off
enable_global_syscache = off

time_record_level = 1
enable_beta_opfusion = on

#wal_file_init_num = 10000
#advance_xlog_file_num = 50000
light_comm = on
enable_indexscan_optimization = on

hot_standby = off

checkpoint_segments = 3000
advance_xlog_file_num = 100000
thread_pool_attr = '476,4,(cpubind:1-29,32-61,64-93,96-125)'
enable_gazelle_performance_mode=on

#------------------------------------------------------------------------------
wal_file_init_num = 40000
advance_xlog_file_num = 100000
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/network/gazelle/gazelle_for_redis_acceleration.md
---
# Gazelle 加速 redis

## 背景介绍

Gazelle是一款高性能用户态协议栈。它基于DPDK在用户态直接读写网卡报文，共享大页内存传递报文，使用轻量级LwIP协议栈。能够大幅提高应用的网络I/O吞吐能力。专注于数据库网络性能加速，如MySQL、redis等。

Gazelle相比于内核协议栈在redis测试中有明显的提升，以arm架构、8u32g规格、ovs+dpdk虚拟机的set和get测试为例，测是结果如下，set提升约1.7倍，get提升约1.5倍，其中测试连接数为1k，包长为默认包长3。

```sh
#kernel
redis-benchmark -h 192.168.1.127 -p 6379 -c 1000 -n 10000000 -r 10000000 -t set,get --threads 12
#set
Summary:
  throughput summary: 132402.98 requests per second
  latency summary (msec):
          avg       min       p50       p95       p99       max
        7.474     1.376     7.207     9.399    14.255    30.879
#get
Summary:
  throughput summary: 142834.69 requests per second
  latency summary (msec):
          avg       min       p50       p95       p99       max
        6.919     1.384     6.663     8.751    13.311    24.207
```

```sh
#gazelle
redis-benchmark -h 192.168.1.127 -p 6379 -c 1000 -n 10000000 -r 10000000 -t set,get --threads 12
#set
Summary:
  throughput summary: 359389.03 requests per second
  latency summary (msec):
          avg       min       p50       p95       p99       max
        2.736     0.240     2.735     2.895     3.127     9.471
#get
Summary:
  throughput summary: 359401.97 requests per second
  latency summary (msec):
          avg       min       p50       p95       p99       max
        2.752     0.488     2.751     2.903     3.135    16.671
```

> \[!NOTE]说明
> 不同的硬件配置及组网环境会影响性能提升的效果，上述测试结果仅供参考。

## 功能约束

* 当前仅支持IPV4，IPV6暂不支持。
* 并发数限制最大为2w。
* 当前不支持gazelle多进程，即一个节点上不能用gazelle启动多个redis server。
* redis主从集群中，内核态日志反复打印主从重连日志，用户态只打印一次，此为正常现象，无功能影响。原因：一个内核态主节点下线后，表现为从节点尝试连接主节点，发送SYN报文，主节点处于TIME\_WAIT状态回复RST报文关闭连接，以此反复。一个用户态主节点下线后，由于gazelle+redis退出后连接完全关闭，未处于TIME\_WAIT状态，从节点尝试连接主节点，发送SYN报文，主节点不会回复RST报文，所以gazellectl lstack show ip -c查询时，会存在一个处于SYN\_SENT状态的连接。

## Gazelle加速redis测试步骤

### 环境要求

#### 硬件

单机测试需要服务端（Server）、客户端（Client）各一台；
主从模式测试至少两台服务端（一主一从）、一台客户端；
哨兵模式测试至少两台redis服务端（一主一从）、两台哨兵服务端、一台客户端；
集群模式测试至少六台redis服务端（三主三从）、一台客户端。

#### 软件

[redis软件包下载](https://download.redis.io/releases/)，本次使用版本为redis-6.2.9。

### Server端部署

#### 关闭测试影响项

```sh
#关闭防火墙
systemctl stop iptables
systemctl stop firewalld
```

#### 编译安装redis

```sh
tar zxvf redis-6.2.9.tar.gz 
cd redis-6.2.9/
make clean
make -j 32
make install
```

#### gazelle运行环境部署

* 安装gazelle及依赖

```sh
yum -y install gazelle dpdk libconfig numactl libboundscheck libcap 
```

* 修改/etc/gazelle/lstack.conf配置文件中参数如下，其他配置参数可保持默认值。

| 配置项        | 值                                                           | 描述                                                         |
| ------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| dpdk\_args     | \["--socket-mem", "2400,0,0,0", "--huge-dir", "/mnt/hugepages-lstack", "--proc-type", "primary"] | 配置cpu和网卡所在的numa使用2400M内存（可根据并发数减少）；如果cpu和网卡不在一个numa上，则对应numa都需要配置内存；如果是mlx网卡，需要追加配置"-d", "librte\_net\_mlx5.so" |
| num\_cpus      | "2"                                                          | 选择一个cpu绑定lstack                                         |
| mbuf\_count\_per\_conn| 34                                                       | 每个连接需要的mbuf数量                                        |
| tcp\_conn\_count     | 20000                                                         | redis测试最大并发数                                         |

```sh
#服务端分配大页
mkdir -p /mnt/hugepages-lstack
chmod -R 700 /mnt/hugepages-lstack
mount -t hugetlbfs nodev /mnt/hugepages-lstack -o pagesize=2M #不能重复操作，否则大页被占用不能释放
echo 2048 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages #根据实际选择pagesize
cat /sys/devices/system/node/node0/hugepages/hugepages-2048kB/free_hugepages #查询对应node上实际可用的大页内存

#服务端加载ko（mlx网卡可跳过此步骤）
modprobe vfio enable_unsafe_noiommu_mode=1
modprobe vfio-pci

#服务端绑定网卡到用户态（mlx网卡可跳过此步骤）
ip link set enp4s0 down
dpdk-devbind -b vfio-pci enp4s0

#gazelle部署完成，待app部署
```

> \[!NOTE]说明\
> Gazelle部署详见[Gazelle用户指南](https://atomgit.com/openeuler/gazelle/blob/master/doc/zh/user-guide.md)
> 不同网卡绑定用户态方法详见[Gazelle网卡支持及使用](https://atomgit.com/openeuler/gazelle/blob/master/doc/zh/nic-support.md)

#### redis服务端部署

redis环境部署包括单机部署、主从模式部署、哨兵模式部署和集群模式部署，所有场景的redis server的redis.conf文件，均需要做如下配置：

```sh
#关闭保护模式
protected-mode no
#gazelle暂不支持此参数进行后台运行
daemonize no
#开启AOF持久化，redis单机测试可不配置
appendonly yes
```

##### redis单机部署

redis单机测试包含一台server，部署好gazelle和redis后，可以直接启动gazelle+redis服务

```sh
LD_PRELOAD=/usr/lib64/liblstack.so GAZELLE_BIND_PROCNAME=redis-server redis-server /root/redis-6.2.9/redis.conf
```

##### redis主从模式部署

主从复制，是指将一台Redis服务器的数据，复制到其他的Redis服务器。前者称为主节点(Master)，后者称为从节点(Slave)；数据的复制是单向的，只能由主节点到从节点。redis主从模式包括至少两台server，配置方法有两种：

主从配置方式1
从节点redis.conf配置文件中添加如下配置，然后分别启动redis主从节点

```sh
#192.168.1.127 6379为主节点服务的ip和port
slaveof 192.168.1.127 6379
```

```sh
LD_PRELOAD=/usr/lib64/liblstack.so GAZELLE_BIND_PROCNAME=redis-server redis-server /root/redis-6.2.9/redis.conf
```

主从配置方式2
完成主从节点通用配置修改后，启动redis主从节点（此时还没有建立主从关系），在客户端执行以下命令

```sh
redis-cli -h 192.168.1.127 slaveof NO ONE #主节点
redis-cli -h 192.168.1.128 slaveof 192.168.1.127 6379 #从节点
```

主从信息查询

```sh
[root@openEuler redis-6.2.9]# redis-cli -h 192.168.1.127 info Replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.1.128,port=6379,state=online,offset=780,lag=0
......
```

```sh
[root@openEuler redis-6.2.9]# redis-cli -h 192.168.1.128 info Replication
# Replication
role:slave
master_host:192.168.1.127
master_port:6379
master_link_status:up
......
```

##### redis哨兵模式部署

哨兵模式基于主从复制模式，只是引入了哨兵来监控与自动处理故障。主从切换技术的方法是：当服务器宕机后，需要手动一台从机切换为主机，这需要人工干预，不仅费时费力而且还会造成一段时间内服务不可用。为了解决主从复制的缺点，就有了哨兵机制。redis哨兵模式测试至少需要两台redis服务端和两台哨兵服务端。

* 按照主从模式部署方法将两台redis服务端启动
* 在两台哨兵服务端安装部署redis，分别修改sentinel.conf配置文件

```sh
protected-mode no #关闭保护模式
daemonize yes #后台运行，日志记录在logfile
logfile "/var/log/sentinel.log" #指定日志存放路径
sentinel monitor mymaster 192.168.1.127 6379 1 #该主节点的名称是mymaster，监控master的ip、端口，1是至少需要1个哨兵节点同意，才能判定主节点故障并进行故障转移
sentinel down-after-milliseconds mymaster 30000 #判断服务器down掉的时间周期，默认30000毫秒（30秒）
sentinel failover-timeout mymaster 50000 #故障节点的最大超时时间为50000
```

* 启动哨兵（内核态启动），查询哨兵信息

```sh
[root@openEuler redis-6.2.9]#redis-sentinel sentinel.conf
[root@openEuler redis-6.2.9]#ps -ef|grep redis-sentinel 
root       5961      1  0 13:36 ?        00:00:00 redis-sentinel *:26379 [sentinel]
[root@openEuler redis-6.2.9]#redis-cli -p 26379 info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.1.127:6379,slaves=2,sentinels=3
```

> \[!NOTE]说明\
> redis server和redis 哨兵不可以在同一个节点上，否则无法正常主备切换；
> redis 哨兵暂不支持用户态启动。

##### redis集群模式部署

单节点Redis的并发能力是有上限的，要进一步提高Redis的并发能力，就需要搭建主从集群，其作用是提供在多个Redis节点间共享数据的程序集。redis集群测试至少需要六台redis服务端。

* 在六台redis服务端安装部署redis，分别修改redis.conf配置文件

```sh
protected-mode no #关闭保护模式
daemonize no #前台运行
bind 0.0.0.0
port 6379 #redis部署在不同的虚机上，ip不一样，端口可以保持默认
appendonly yes #开启aof持久化
cluster-enabled yes #开启集群模式
cluster-config-file nodes.conf #集群模式的配置文件名称，无需手动创建，由集群自动维护
cluster-node-timeout 5000 #集群中节点之间心跳超时时间
```

* 分别启动六台redis服务端

```sh
LD_PRELOAD=/usr/lib64/liblstack.so GAZELLE_BIND_PROCNAME=redis-server redis-server /root/redis-6.2.9/redis.conf
```

* 在客户端执行命令创建集群，同意集群中master与slave节点的分配情况

```sh
[root@openEuler redis-6.2.9]#redis-cli --cluster create --cluster-replicas 1 192.168.1.127:6379 192.168.1.128:6379 192.168.1.129:6379 192.168.1.130:6379 192.168.1.131:6379 192.168.1.132:6379
>>> Performing hash slots allocation on 6 nodes...
......
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 192.168.1.127:6379)
......
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered. #hash slots分配OK则集群创建成功

#redis-cli --cluster：代表集群操作命令；create：代表是创建集群；--cluster-replicas 1 ：指定集群中每个master的副本个数为1
#此时节点总数 ÷ (replicas + 1) 得到的就是master的数量n。因此节点列表中的前n个节点就是master，其它节点都是slave节点，随机分配到不同master
```

* 查询集群信息

集群中任意一个正常运行的server都可以作为切入点

```sh
#查看集群状态信息
[root@openEuler redis-6.2.9]# redis-cli -h 192.168.1.127 cluster info
cluster_state:ok #如果这里是fail，可以看下hash slots分配失败
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
......
```

```sh
#查看集群的主从关系
[root@openEuler redis-6.2.9]# redis-cli -h 192.168.1.128 cluster nodes
514b35aaa5035d489b60a0e8f8fb01d1c20734ce 192.168.1.129:6379@16379 slave 50aa44a1e4a6a0c75cf2f9b20055bfaa77d1b163 0 1724919619916 1 connected
50aa44a1e4a6a0c75cf2f9b20055bfaa77d1b163 192.168.1.127:6379@16379 master - 0 1724919617960 1 connected 0-5460
a94402ca747ead08e4b93ff975dfbe995068ecbf 192.168.1.130:6379@16379 slave 0a569e1ac4e373a22abcbf6ce6b8118fba3d4d6e 0 1724919618000 3 connected
8c4040a4fa8456044acad2518dc45b8236ba44c4 192.168.1.128:6379@16379 myself,slave 44a96161651c8383fb4966c6dde45d400fe2a203 0 1724919617000 2 connected
0a569e1ac4e373a22abcbf6ce6b8118fba3d4d6e 192.168.1.132:6379@16379 master - 0 1724919618975 3 connected 10923-16383
44a96161651c8383fb4966c6dde45d400fe2a203 192.168.1.131:6379@16379 master - 0 1724919619617 2 connected 5461-10922
#可以看出127、131、132是master节点，128、129、130为slave节点，且可以通过slave后的node id找到其对应的master节点
```

### Client部署redis-benchmark工具

* 编译安装

redis-benchmark为redis自带的测试工具，与服务端一样编译安装redis即可。

* 测试命令

```sh
#单机、主从模式、哨兵模式
#set,get
redis-benchmark -h 192.168.1.127 -p 6379 -c 1000 -n 10000000 -d 3 -r 10000000 -t set,get --threads 12
#其中，-h：指定redis服务端地址；-p：指定redis服务端端口；-c：指定客户端并发连接数；-n：指定请求总数；-t：指定测试命令；-d：指定数据包大小
#主从模式和哨兵模式下，由于默认配置了slave-read-only yes，从节点redis只能执行get命令。
```

```sh
#集群模式
#set,get
redis-benchmark -h 192.168.1.127 -p 6379 -c 1000 -n 10000000 -d 3 -r 10000000 -t set,get --threads 12 --cluster
#-h只需指定集群中任意一个可用节点即可
```

### Gazelle加速redis成功日志样例

```sh
[root@openEuler redis-6.2.9]# LD_PRELOAD=/usr/lib64/liblstack.so GAZELLE_BIND_PROCNAME=redis-server redis-server redis.conf
LSTACK[2555459]: LD_PRELOAD ok
dpdk argv: --socket-mem 2400,0,0,0 --huge-dir /mnt/hugepages-lstack --proc-type primary
LSTACK[2555459]: cfg_init success
#dpdk启动log省略，环境不同存在差异
......
#gazelle打印init success则启动成功
LSTACK: gazelle_stack_thread:533 stack_00 init success
LSTACK: gazelle_network_init:328 gazelle_network_init success
#redis启动log省略，与内核态redis启动log相同
```

启动成功后，可以在客户端ping lstack.conf中配置的用户态ip，ping通则gazelle加速redis部署成功。

---

---
url: /zh/docs/22.03_LTS_SP4/server/network/gazelle/gazelle_single_nic_user_guide.md
---
# Gazelle 单网卡功能使用说明

## 背景介绍

目前使用 Gazelle 进行应用程序进行加速时，只能处理应用程序使用端口的报文，对网卡接收到的其他报文(如运维命令ssh 等)，无法进行处理。

### 功能介绍

对于网卡接收到的所有报文，在 Gazelle 内部进行分流处理。分流原则如下：对于Gazelle 记录端口的报文在 Gazelle 处理后送到应用程序，对未记录端口的报文转送到内核。

举例：当Gazelle 加速redis时，Gazelle 会在启动时记录redis监听的端口6379。当Gazelle 收到网卡的报文后，进行每个报文进行判断，对于目的端口是6379 的报文送到Gazelle，其余的报文送到内核进行处理。

### 使用说明

#### 虚机场景

1. 安装dpdk Gazelle 配置大页

   ```shell
   yum install -y dpdk gazelle
   echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
   mkdir -p /mnt/hugepages-lstack
   chmod -R 700 /mnt/hugepages-lstack
   mount -t hugetlbfs nodev /mnt/hugepages-lstack
   ```

   详细步骤可参考：[挂载大页内存](https://atomgit.com/openeuler/gazelle/blob/master/doc/zh/user-guide.md#3-%E5%A4%A7%E9%A1%B5%E5%86%85%E5%AD%98%E9%85%8D%E7%BD%AE)

2. dpdk 绑定网卡

   以网卡绑定 igb\_uio 为例

   ```shell
   cd /lib/modules
   my_var=$(find /lib/modules/ -name igb_uio.ko)
   modprobe uio
   # 加载ko
   insmod ${my_var}
   #使用igb_uio
   dpdk-devbind -b igb_uio enp3s0
   ```

   详细步骤可参考： [dpdk绑定网卡](https://atomgit.com/openeuler/gazelle/blob/master/doc/zh/user-guide.md#2-dpdk%E7%BB%91%E5%AE%9A%E7%BD%91%E5%8D%A1)

3. 修改Gazelle配置文件

   ```shell
   flow_bifurcation=1 # 打开分流开关
   ```

4. 加速应用程序，启动Gazelle 以加速 redis 为例

   ```shell
   LD_PRELOAD=/usr/lib64/liblstack.so Gazelle_BIND_PROCNAME=redis-server /root/redis-server /root/redis.conf
   ```

#### 容器场景

1. 安装dpdk、配置大页、dpdk 绑定网卡 同上述章节虚拟场景配置

2. 安装 docker

   ```shell
   yum install -y docker
   ```

3. 导入镜像

   ```shell
   docker load -i openEuler-docker.x86_64.tar.xz
   ```

##### host模式

1. 启动容器

   ```shell
   docker run -d -it --privileged -v /lib/modules:/lib/modules -v /mnt:/mnt -v /dev:/dev -v /sys/bus/pci/drivers:/sys/bus/pci/drivers -v /sys/kernel/mm/hugepages:/sys/kernel/mm/hugepages -v /sys/devices/system/node:/sys/devices/system/node -v /dev:/dev openeuler-22.03-lts-sp4 bash
   ```

   ```shell
   docker 启动映射文件解释
   -v /lib/modules:/lib/modules 映射内核模块
   -v /mnt:/mnt 映射外部存储设备 文件系统
   -v /dev:/dev 映射内核设备
   -v /sys/bus/pci/drivers:/sys/bus/pci/drivers 映射驱动文件
   -v /sys/kernel/mm/hugepages:/sys/kernel/mm/hugepages 映射大页信息
   -v /sys/devices/system/node:/sys/devices/system/node 映射节点信息
   ```

2. 进入容器

   ```shell
   docker exec -it xxxxx bash
   ```

3. 安装dpdk Gazelle

   ```shell
   yum install -y dpdk gazele
   ```

4. 修改配置文件

   ```shell
   flow_bifurcation=1 # 打开分流开关
   devices="52:54:00:de:2a:57" # 修改mac地址为 dpdk绑定的网卡地址
   ```

5. 启动Gazelle
   以加速 redis 为例

   ```shell
   LD_PRELOAD=/usr/lib64/liblstack.so Gazelle_BIND_PROCNAME=redis-server /root/redis-server /root/redis.conf
   ```

##### VF 直通模式

1. 启动容器

   ```shell
   docker run -d -it --network host --privileged -v /lib/modules:/lib/modules -v \
   /mnt:/mnt -v /dev:/dev -v /sys/bus/pci/drivers:/sys/bus/pci/drivers -v \
   /sys/kernel/mm/hugepages:/sys/kernel/mm/hugepages -v \
   /sys/devices/system/node:/sys/devices/system/node -v /dev:/dev  \
   openeuler-22.03-lts-sp4 bash
   ```

2. 配置VF 直通网卡

   ```shell
   echo 2 > /sys/class/net/enp130s0f1/device/sriov_numvfs 
   docker ps
   PID=$(docker inspect -f '{{.State.Pid}}' 容器名称)
   mkdir -p /var/run/netns
   ln -s /proc/PID/ns/net /var/run/netns/PID
   ip link set enp129s0f1v0 netns PID
   ```

3. 进入容器

   ```shell
   docker exec -it xxx bash
   ```

4. 安装dpdk Gazelle

   ```shell
   yum install -y dpdk gazele
   ```

5. 修改配置文件

   ```shell
   flow_bifurcation=1 # 打开分流开关
   devices="52:54:00:de:2a:57" # 修改mac地址为VF直通的网卡
   ```

6. 启动Gazelle
   以加速 redis 为例

   ```shell
   LD_PRELOAD=/usr/lib64/liblstack.so Gazelle_BIND_PROCNAME=redis-server /root/redis-server /root/redis.conf
   ```

### 功能限制

1. 同节点通信（用户态服务端+内核态客户端 或者 用户态客户端+ 内核态服务端）只支持TCP协议，UDP 协议暂不支持。
2. 不支持和kni 功能同时开启。
3. 对不携带端口的报文无法做到分流。
4. 开启此功能后，性能会下降 2% 左右。
5. 虚拟环境及容器环境需要支持`ip a`命令查询网卡信息。若不支持此命令，可能影响虚拟网卡IPV6地址状态，进而影响IPV6通信。

### 已支持运维命令

* ifconfig

* tcpdump

* ifconfig

* ftp/sftp

* sar

* netstat

* ssh -- 需开启ssh 登录，若未开启可按照下面方式开启

```shell
[root@eb2936ebeaaf ~]# yum install openssh-server
[root@eb2936ebeaaf ~]# vim /etc/ssh/sshd_config
Port 22 # 开启端口
PubkeyAuthentication yes # 修改登录验证方式
[root@eb2936ebeaaf ~]# /usr/sbin/sshd  # 启动ssh服务
[root@eb2936ebeaaf ~]# netstat -pant | grep sshd # 查询 ssh 服务是否开启
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/gcc/gcc_basic_performance_optimization_user_guide.md
---
# GCC Base Performance Optimization Guide

## Overview

The optimization of compiler base performance is crucial to improving the development efficiency, running performance, and maintainability of applications. It is an important research direction in computer science and one of the key steps in the process of software development. Based on the general compilation optimization capability, GCC for openEuler enhances mid- and back-end performance optimization technologies, including instruction optimization, vectorization enhancement, prefetch enhancement, and data flow analysis enhancement.

## Installation and Deployment

### Software Requirements

OS: openEuler 22.03 LTS SP4

### Hardware Requirements

AArch64 architecture

### Software Installation

Install GCC and related components as required. For example, install GCC:

```shell
yum install gcc
```

## Usage

### CRC Optimization

#### Description

Cyclic redundancy check (CRC) code is identified to generate efficient hardware instructions.

#### Usage

Add the `-floop-crc` option during compilation.

Note: `-floop-crc` must be used together with `-O3 -march=armv8.1-a`.

### IF-conversion Enhancement

#### Description

IF-conversion is enhanced to use more registers to reduce conflicts.

#### Usage

This enhancement is part of the IF-conversion optimization of the Register Transfer Language (RTL). Enable the enhancement by using the following options.

`-fifcvt-allow-complicated-cmps`

`-param=ifcvt-allow-register-renaming=[0,1,2]` The default value is 0. The number is used to control the optimization scope.

Note: This enhancement requires the `-O2` optimization level and must be used together with `--param=max-rtl-if-conversion-unpredictable-cost=48` and `--param=max-rtl-if-conversion-predictable-cost=48`.

### Multiplication Optimization

#### Description

Arm instructions are combined to convert low-order multiplications into high-order multiplication instructions.

#### Usage

Use the `-fuaddsub-overflow-match-all` and `-fif-conversion-gimple` options.

Note: This optimization requires the `-O3` or higher optimization level and must be used together with `-ftree-fold-phiopt option`.

### CMLT Instruction Generation

#### Description

CMLT instructions are generated for some elementary arithmetic operations to reduce the number of instructions.

#### Usage

Use the `-mcmlt-arith` option.

Note: This optimization requires the `-O3` or higher optimization level.

### Vectorization Enhancement

#### Description

Redundant instructions generated during vectorization are identified and simplified, and shorter arrays can be vectorized.

#### Usage

Use `--param=tree-forwprop-perm=1` and `--param=vect-alias-flexible-segment-len=1`. The default values are 0.

Note: This optimization requires the `-O3` or higher optimization level.

### maxmin and UZP1/UZP2 Instruction Optimization

#### Description

The maxmin and UZP1/UZP2 instructions are optimized to reduce the total instructions and improve performance.

#### Usage

Use the `-fconvert-minmax` option. UZP1/UZP2 instruction optimization is enabled by default at a level higher than `-O3`.

Note: This optimization requires the `-O3` or higher optimization level.

### LDP and STP Optimization

#### Description

Each LDP and STP instruction with poor performance is split into two LDR and STR instructions.

#### Usage

Use the `-fsplit-ldp-stp` option. Use `--param=param-ldp-dependency-search-range= [1,32]` to control the search range. The default value is 16.

Note: This optimization requires the `-O1` or higher optimization level.

### AES Instruction Optimization

#### Description

The AES algorithm code is identified to accelerate instructions using hardware.

#### Usage

Use the `-fcrypto-accel-aes` option.

Note: This optimization requires the `-O3` or higher optimization level.

### Indirect Call Optimization

#### Description

Indirect calls in programs are identified and analyzed to convert them into direct calls.

#### Usage

Use the `-ficp -ficp-speculatively` option.

Note: This optimization must be used together with `-O2 -flto -flto-partition=one`.

### IPA-prefetch

#### Description

Indirect memory accesses in a loop are identified to insert a prefetch instruction, thereby reducing the delay of indirect memory accesses.

#### Usage

Use the `-fipa-prefetch -fipa-ic` option.

Note: This optimization must be used together with `-O3 -flto`.

### LLC-prefetch

#### Description

GCC for openEuler analyzes main execution paths in programs, performs memory multiplexing analysis on loops on the primary path, calculates and sorts top hot data, and inserts prefetch instructions to pre-allocate data to LLCs, reducing LLC misses.

#### Usage

Use the `-fllc-allocate` option. The `-O2` or higher optimization level is required.

Other related interfaces:

|  Option  | Default Value | Description |
|  ----  | ----  | ----  |
| -param=mem-access-ratio=\[0,100]  | 20 | Ratio of the number of memory accesses in a loop to the number of instructions.|
| -param=mem-access-num=unsigned  | 3 | Number of memory accesses in a loop. |
|  -param=outer-loop-nums=\[1,10]  |  1 | Maximum number of outer loop layers that can be unrolled. |
|  -param=filter-kernels=\[0,1]  |  1 | Whether to perform path series filtering on loops. |
|  -param=branch-prob-threshold=\[50,100]  |  80 | Probability threshold for a branch to be considered highly probable. |
|  -param=prefetch-offset=\[1,999999]  | 1024  |  Prefetch offset distance. Generally, the value is a power of 2.|
|  -param=issue-topn=unsigned  |  1 |  Number of prefetch instructions.|
|  -param=force-issue=\[0,1]  |  0 |  Whether to perform forcible prefetch, that is, the static mode.|
|  -param=llc-capacity-per-core=\[0,999999]  |  114 | Average LLC capacity allocated to each core in multi-branch prefetch mode. |

---

---
url: /en/docs/22.03_LTS_SP4/server/development/gcc/gcc_toolset_user_guide.md
---
# GCC Toolset User Guide

## Overview

To ensure the stability of the OS, the latest version of base software is not selected generally. Instead, a relatively stable version is used. openEuler 22.03 LTS uses GCC 10.3.1 as the baseline for development. For GCC 10.3.1, Fortran supports only some OpenMP 4.5 specifications, while C/C++ supports a few OpenMP 5.0 specifications. To support all OpenMP 4.5 specifications with Fortran, the GCC toolset is designed. For applications that require the OpenMP 4.5 specifications not supported by GCC 10.3.1, GCC Toolset 12 can be used to compile and build.

## Installation and Deployment

### Software Requirements

OS: openEuler 22.03 LTS SP4

### Hardware Requirements

AArch64 architecture

### Software Installation

To prevent conflicts between installation dependencies of GCC Toolset 12 and the default GCC, the software package of GCC Toolset 12 is prefixed with **gcc-toolset-12-**, followed by the name of the original GCC software package.

Install the default compiler GCC 10.3.1 in `/usr/`.

```shell
yum install -y gcc gcc-c++ gcc-gfortran
```

Install GCC Toolset 12 in `/opt/openEuler/gcc-toolset-12/root/usr/`.

```shell
yum install -y gcc-toolset-12-gcc*
```

## How to Use

Because GCC Toolset 12 is installed in `/opt/openEuler/gcc-toolset-12/root/usr/`, run the following commands to use the software:

```shell
export PATH=/opt/openEuler/gcc-toolset-12/root/usr/bin/:$PATH
export LD_LIBRARY_PATH=/opt/openEuler/gcc-toolset-12/root/usr/lib64/:$LD_LIBRARY_PATH
```

**Note: GCC Toolset 12 is used only to support the OpenMP 4.5 specifications not supported by GCC 10.3.1. For other features, the default compiler GCC 10.3.1 is recommended to prevent unknown compilation errors.**

## Compatibility

This section describes the compatibility issues in some special scenarios. This project is in continuous iteration and issues will be fixed as soon as possible. Developers are welcome to join this project.

* Currently, the GCC toolset solution applies only to the scenario requiring OpenMP 4.5 specifications.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/gcc/gcc_basic_performance_optimization_user_guide.md
---
# GCC 基础性能优化用户指南

## 简介

编译器基础性能优化对于提高应用程序的开发效率、运行性能和可维护性都非常重要。它是计算机科学领域的一个重要研究方向，也是软件开发过程中的重要环节之一。GCC for openEuler 在通用编译优化能力的基础上，对中后端性能优化技术进行了增强，包括指令优化、向量化增强、预取增强、数据流分析增强等优化。

## 安装与部署

### 软件要求

操作系统：openEuler 22.03 LTS SP4

### 硬件要求

aarch64 架构

### 安装软件

按需安装 GCC 和相关组件即可，以 GCC 为例。

```shell
yum install gcc
```

## 使用方法

### CRC优化

#### 说明

识别CRC软件循环代码，生成高效硬件指令。

#### 使用方法

在编译时增加 -floop-crc 选项。

注：`-floop-crc`选项需要和`-O3 -march=armv8.1-a`一起使用。

### If-conversion 增强

#### 说明

增强 If conversion 优化，使用更多的寄存器以减少冲突。

#### 使用方法

本优化是 RTL 优化 if-conversion 的一部分，使用如下编译选项控制优化启用。

`-fifcvt-allow-complicated-cmps`

`-param=ifcvt-allow-register-renaming=[0,1,2]`默认为0，数字用于控制优化范围。

注：此优化依赖`-O2`优化等级，以及与`--param=max-rtl-if-conversion-unpredictable-cost=48`、`--param=max-rtl-if-conversion-predictable-cost=48`共同使用。

### 乘法计算优化

#### 说明

Arm 相关指令合并优化，实现32位复杂组合的64位整形乘法逻辑的识别，并以高效的64位指令数输出。

#### 使用方法

使用`-fuaddsub-overflow-match-all`和`-fif-conversion-gimple`选项使能优化。

注：此优化需要`-O3`及以上优化等级以及`-ftree-fold-phiopt`选项共同使用。

### cmlt 指令生成优化

#### 说明

对一些四则运算生成`cmlt`指令，减少指令数。

#### 使用方法

使用选项`-mcmlt-arith`使能优化。

注：此优化需要`-O3`以上优化等级使用。

### 向量化优化增强

#### 说明

识别并简化向量化过程中生成的冗余指令，允许更短的循环进入向量化。

#### 使用方法

使用参数`--param=tree-forwprop-perm=1`和`--param=vect-alias-flexible-segment-len=1`使能，默认均为0。

注：此优化需要`-O3`及以上优化等级。

### min max 和 uzp1/uzp2 指令联合优化

#### 说明

识别 min max 和 uzp1/uzp2 指令联合优化机会，减少指令数从而提升性能。

#### 使用方法

使用`-fconvert-minmax`选项使能`min max`优化，`uzp1/uzp2`指令优化在`-O3`以上等级默认使能。

注：依赖`-O3`及以上优化等级。

### ldp/stp 优化

#### 说明

识别某些性能表现差的 ldp/stp，将其拆分成2个 ldr 和 str。

#### 使用方法

使用`-fsplit-ldp-stp`选项使能优化，使用参数`--param=param-ldp-dependency-search-range=[1,32]`控制搜索范围，默认16。

注：依赖`-O1`及以上优化等级。

### AES指令优化

#### 说明

识别 AES 软件算法指令序列，使用硬件指令加速。

#### 使用方法

使用`-fcrypto-accel-aes`选项使能优化。

注：依赖`-O3`及以上优化等级。

### 间接调用提升

#### 说明

识别和分析程序中的间接调用，尝试将其优化为直接调用。

#### 使用方法

使用选项`-ficp -ficp-speculatively`使能优化。

注：此优化需要和-O2 -flto -flto-partition=one共同使用。

### IPA-prefetch

#### 说明

识别循环中的间接访存，插入预取指令，从而减少间接访存的延迟。

#### 使用方法

通过选项-fipa-prefetch -fipa-ic使能优化。

注：此优化需要和-O3 -flto共同使用。

### LLC-prefetch

#### 说明

通过分析程序中主要的执行路径，对主路径上的循环进行访存的复用分析，计算排序出 TOP 的热数据，并插入预取指令将数据先分配至 LLC 中，减少 LLC miss。

#### 使用方法

使能 LLC 特性，需开启 -O2 及以上优化等级，同时使用编译选项 -fllc-allocate。

其他相关接口：

|  选项   | 默认值  | 说明  |
|  ----  | ----  | ----  |
| -param=mem-access-ratio=\[0,100]  | 20 | 循环内访存数对指令数的占比。|
| -param=mem-access-num=unsigned  | 3 | 循环内访存数量。  |
|  -param=outer-loop-nums=\[1,10]  |  1 | 允许扩展的外层循环的最大层数。  |
|  -param=filter-kernels=\[0,1]  |  1 | 是否针对循环做路径串联筛选。  |
|  -param=branch-prob-threshold=\[50,100]  |  80 | 高概率执行分支的概率阈值。  |
|  -param=prefetch-offset=\[1,999999]  | 1024  |  预取偏移距离，一般为2的次幂。 |
|  -param=issue-topn=unsigned  |  1 |  预取指令个数。 |
|  -param=force-issue=\[0,1]  |  0 |  是否执行强制预取，即静态模式。 |
|  -param=llc-capacity-per-core=\[0,999999]  |  114 | 多分支预取下每个核平均分配的 LLC 容量。  |

---

---
url: /zh/docs/22.03_LTS_SP4/server/development/gcc/gcc_toolset_user_guide.md
---
# GCC 多版本支持用户指南

## 简介

为了保障操作系统的稳定性，一般在进行基础软件选型时不会选用最新的版本，而是倾向于使用相对稳定的版本，因此当前 openEuler 22.03 LTS 使用 GCC 10.3.1 作为基线进行开发。然而GCC 10.3.1 只能支持部分 Fortran OpenMP 4.5 规范和较少的 C/C++ OpenMP 5.0 语言规范。为了支持全部 Fortran OpenMP 4.5 规范，设计双版本 GCC，对于需要使用 GCC 10.3.1 不支持的 OpenMP 4.5 规范的应用，可以使用多版本编译器 gcc-12 来支持编译构建。

## 安装与部署

### 软件要求

操作系统：openEuler 22.03 LTS SP4

### 硬件要求

aarch64 架构

### 安装软件

为了和默认 GCC 安装做出区分，防止多版本 GCC 的安装和默认 GCC 的安装依赖库冲突，gcc-12 的多版本软件包名以 “gcc-toolset-12-” 为前缀，后面接上原有 GCC 软件包名。

默认编译器 gcc-10.3.1，安装路径为  `/usr/`：

```shell
yum install -y gcc gcc-c++ gcc-gfortran
```

多版本编译器 gcc-12，安装路径为`/opt/openEuler/gcc-toolset-12/root/usr/`：

```shell
yum install -y gcc-toolset-12-gcc*
```

## 使用方法

gcc-12 多版本编译器软件包安装在`/opt/openEuler/gcc-toolset-12/root/usr/`下，因此可使用如下命令使用软件包：

```shell
export PATH=/opt/openEuler/gcc-toolset-12/root/usr/bin/:$PATH
export LD_LIBRARY_PATH=/opt/openEuler/gcc-toolset-12/root/usr/lib64/:$LD_LIBRARY_PATH
```

**注意：多版本编译器 gcc-12 仅用于支持 GCC 10.3.1 不支持的 OpenMP 4.5 规范，其他特性建议使用默认 gcc-10.3.1 防止发生未知编译错误。**

## 兼容性说明

此节主要列出当前一些特殊场景下的兼容性问题。本项目持续迭代中，会尽快进行修复，也欢迎广大开发者加入。

* 当前 GCC 多版本方案仅适配 OpenMP 4.5 语言规范场景。

---

---
url: /en/docs/22.03_LTS_SP4/server/performance/atune/getting_to_know_a_tune.md
---
# Getting to Know A-Tune

## Introduction

An operating system (OS) is basic software that connects applications and hardware. It is critical for users to adjust OS and application configurations and make full use of software and hardware capabilities to achieve optimal service performance. However, numerous workload types and varied applications run on the OS, and the requirements on resources are different. Currently, the application environment composed of hardware and software involves more than 7000 configuration objects. As the service complexity and optimization objects increase, the time cost for optimization increases exponentially. As a result, optimization efficiency decreases sharply. Optimization becomes complex and brings great challenges to users.

Second, as infrastructure software, the OS provides a large number of software and hardware management capabilities. The capability required varies in different scenarios. Therefore, capabilities need to be enabled or disabled depending on scenarios, and a combination of capabilities will maximize the optimal performance of applications.

In addition, the actual business embraces hundreds and thousands of scenarios, and each scenario involves a wide variety of hardware configurations for computing, network, and storage. The lab cannot list all applications, business scenarios, and hardware combinations.

To address the preceding challenges, openEuler launches A-Tune.

A-Tune is an AI-based engine that optimizes system performance. It uses AI technologies to precisely profile business scenarios, discover and infer business characteristics, so as to make intelligent decisions, match with the optimal system parameter configuration combination, and give recommendations, ensuring the optimal business running status.

![](figures/en-us_image_0227497000.png)

## Architecture

The following figure shows the A-Tune core technical architecture, which consists of intelligent decision-making, system profile, and interaction system.

* Intelligent decision-making layer: consists of the awareness and decision-making subsystems, which implements intelligent awareness of applications and system optimization decision-making, respectively.
* System profile layer: consists of the feature engineering and two-layer classification model. The feature engineering is used to automatically select service features, and the two-layer classification model is used to learn and classify service models.
* Interaction system layer: monitors and configures various system resources and executes optimization policies.

![](figures/en-us_image_0227497343.png)

## Supported Features and Service Models

### Supported Features

[Table 1](#table1919220557576)  describes the main features supported by A-Tune, feature maturity, and usage suggestions.

**Table  1**  Feature maturity

### Supported Service Models

Based on the workload characteristics of applications, A-Tune classifies services into 11 types. For details about the bottleneck of each type and the applications supported by A-Tune, see  [Table 2](#table2819164611311).

**Table  2**  Supported workload types and applications

---

---
url: /en/docs/22.03_LTS_SP4/server/performance/powerapi/getting_to_know_powerapi.md
---
# Getting to Know powerapi

## Background

The power consumption of computing centers is increasing, and the power cost of enterprise data centers accounts for an increasing proportion of enterprise operation costs. The industry has shifted from simply pursuing server performance metrics to pursuing energy efficiency.
With the continuous improvement of the scale of processor integration, both servers and ultra-large-scale integrated computing systems need to consider energy efficiency. In terms of energy saving, software plays an important role. With the development of hardware, more power consumption metrics and control functions are generated. These functions require software participation to utilize the advantages.
openEuler provides powerapi, a lightweight API collection for unified power consumption management of user-mode programs. powerapi shields the complex OS power consumption system calls, especially the differences between interfaces on different hardware platforms, and provides a set of unified measurement and control services.

## Overview

powerapi of openEuler is a collection of APIs for managing system power consumption. it provides a standardized method to manage system power usage, including monitoring, adjusting, and optimizing system power consumption. powerapi helps system administrators better manage system energy consumption, thereby improving system efficiency and reliability and reducing energy costs.\
powerapi provides unified energy efficiency control of the applications, especially the cluster scheduler. It can control the energy efficiency of the entire system to save energy. For example, powerapi detects the CPU load and the IPC/memory access miss rate, and adjusts the CPU frequency.
powerapi interfaces are provided as **.so** library functions. Currently, the x86 and Arm platforms are supported.

## Features

* Detects the power consumption of each device and the system running status.
* Detects whether the system automatically adjusts the frequency.
* Automatically adjusts the system working status based on the configured policy.

## Components

powerapi is provided as an RPM package, which contains the following two components:

* pwrapis: a service process that provides power consumption management services for eagle and third-party systems.
* **libpwrapi.so**: powerapi SDK, which contains header files and is used for secondary development of applications.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/isulad_k8s/gitlab_deploy.md
---
# GitLab Deployment

## Description

GitLab deployment is required in Scenario 1 (openEuler native deployment CI/CD based on GitLab CI/CD). In Scenario 2 (openEuler native development cluster managed by GitLab CI/CD), skip this step.

## Preparing the Server

Prepare a machine running openEuler 20.03 LTS or later versions.

## Starting GitLab

Copy the required YAML files to the **/home** directory and start the related pod.

> **Note**: The YAML files related to GitLab can be obtained from the GitLab official site.

Example YAML files are as follows. Modify them as required.

gitlab-redis.yaml

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: default
  labels:
    name: redis
spec:
  selector:
    matchLabels:
      name: redis
  template:
    metadata:
      name: redis
      labels:
        name: redis
    spec:
      containers:
      - name: redis
        image: 10.35.111.11:5000/redis:latest
        imagePullPolicy: IfNotPresent
        ports:
        - name: redis
          containerPort: 6379
        volumeMounts:
        - mountPath: /var/lib/redis
          name: data
        livenessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 30
          timeoutSeconds: 5
        readinessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        emptyDir: {}

---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: default
  labels:
    name: redis
spec:
  ports:
    - name: redis
      port: 6379
      targetPort: redis
  selector:
    name: redis
```

gitlab-postgresql.yaml

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgresql
  namespace: default
  labels:
    name: postgresql
spec:
  selector:
    matchLabels:
      name: postgresql
  template:
    metadata:
      name: postgresql
      labels:
        name: postgresql
    spec:
      containers:
      - name: postgresql
        image: 10.35.111.11:5000/postgres:13.6
        imagePullPolicy: IfNotPresent
        env:
        - name: POSTGRES_HOST_AUTH_METHOD
          value: trust
        - name: DB_USER
          value: gitlab
        - name: DB_PASS
          value: passw0rd
        - name: DB_NAME
          value: gitlab_production
        - name: DB_EXTENSION
          value: pg_trgm
        ports:
        - name: postgres
          containerPort: 5432
        volumeMounts:
        - mountPath: /var/lib/postgresql
          name: data
        livenessProbe:
          exec:
            command:
            - pg_isready
            - -h
            - localhost
            - -U
            - postgres
          initialDelaySeconds: 30
          timeoutSeconds: 5
        readinessProbe:
          exec:
            command:
            - pg_isready
            - -h
            - localhost
            - -U
            - postgres
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        emptyDir: {}

---
apiVersion: v1
kind: Service
metadata:
  name: postgresql
  namespace: default
  labels:
    name: postgresql
spec:
  ports:
    - name: postgres
      port: 5432
      targetPort: postgres
  selector:
    name: postgresql
```

gitlab.yaml

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitlab
  namespace: default
  labels:
    name: gitlab
spec:
  selector:
    matchLabels:
      name: gitlab
  template:
    metadata:
      name: gitlab
      labels:
        name: gitlab
    spec:
      containers:
      - name: gitlab
        image: 10.35.111.11:5000/yrzr/gitlab-ce-arm64v8:14.3.2-ce.0
        imagePullPolicy: IfNotPresent
        env:
        - name: TZ
          value: Asia/Shanghai
        - name: GITLAB_TIMEZONE
          value: Beijing
        - name: GITLAB_SECRETS_DB_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_SECRETS_SECRET_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_SECRETS_OTP_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_ROOT_PASSWORD
          value: admin321
        - name: GITLAB_ROOT_EMAIL
          value: 517554016@qq.com
        - name: GITLAB_HOST
          value: git.qikqiak.com
        - name: GITLAB_PORT
          value: "80"
        - name: GITLAB_SSH_PORT
          value: "22"
        - name: GITLAB_NOTIFY_ON_BROKEN_BUILDS
          value: "true"
        - name: GITLAB_NOTIFY_PUSHER
          value: "false"
        - name: GITLAB_BACKUP_SCHEDULE
          value: daily
        - name: GITLAB_BACKUP_TIME
          value: 01:00
        - name: DB_TYPE
          value: postgres
        - name: DB_HOST
          value: postgresql
        - name: DB_PORT
          value: "5432"
        - name: DB_USER
          value: gitlab
        - name: DB_PASS
          value: passw0rd
        - name: DB_NAME
          value: gitlab_production
        - name: REDIS_HOST
          value: redis
        - name: REDIS_PORT
          value: "6379"
        ports:
        - name: http
          containerPort: 80
        - name: ssh
          containerPort: 22
        volumeMounts:
        - mountPath: /home/git/data
          name: data
        livenessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 180
          timeoutSeconds: 5
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        emptyDir: {}

---
apiVersion: v1
kind: Service
metadata:
  name: gitlab
  namespace: default
  labels:
    name: gitlab
spec:
  ports:
    - name: http
      port: 80
      targetPort: http
      nodePort: 30852
    - name: ssh
      port: 22
      nodePort: 32353
      targetPort: ssh
  selector:
    name: gitlab
  type: NodePort
```

Start the containers.

```shell
kubectl apply -f gitlab-redis.yaml
kubectl apply -f gitlab-postgresql.yaml
kubectl apply -f gitlab.yaml
```

Check whether the GitLab pod is set up successfully.

```shell
kubectl get pod -A -owide
```

## Logging in to GitLab

Log in to the GitLab Web UI. The address is the IP address and the configured port.

![](figures/4.gitlab-entrance.jpg)
The user name is **root**. The default password can be viewed in the password file in the container.

```shell
kubectl exec -it gitlab-lab -n default /bin/sh
cat /etc/gitlab/initial_root_password
```

![](figures/5.view-password.jpg)

* After you log in, this page is displayed:

![](figures/6.logged-in.png)

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/isulad_k8s/gitlab_runner_deploy.md
---
# GitLab Runner Deployment and Testing

## Images and Software

The following table lists the images required during installation. The version numbers are for reference only.

| Image                                | Version     |
|------------------------------------|----------|
| gitlab/gitlab-runner               | alpine-v14.4.0  |
| gitlab/gitlab-runner-helper        | x86\_64-54944146  |

If the Internet is unavailable in the environment, download the required images in advance. Download the images from the Docker Hub official website.

## Using gitlab-runner.yaml to Start the Runner Container

In the **gitlab-runner.yaml** file, change the image name. The following is an example of the **.yaml** file. Modify the file as required.

```bash
vim gitlab-runner.yaml
```

```conf
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitlab-runner
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      name: gitlab-runner
  template:
    metadata:
      labels:
        name: gitlab-runner
    spec:
      containers:
      - args:
        - run
        image: gitlab/gitlab-runner:alpine-v14.4.0
        imagePullPolicy: IfNotPresent
        name: gitlab-runner
        volumeMounts:
        - mountPath: /etc/gitlab-runner
          name: config
          readOnly: false
        - mountPath: /etc/ssl/certs
          name: cacerts
          readOnly: true
      restartPolicy: Always
      volumes:
      - hostPath:
          path: /etc/gitlab-runner
        name: config
      - hostPath:
          path: /etc/ssl/key
        name: cacerts

```

Start the container.

```bash
# kubectl apply -f gitlab-runner.yaml
# kubectl get pod -A -o wide
```

![image](figures/7.image.png)

## Creating a Container Project That Uses User Certificates for Authentication in GitLab

1. Click **New project**.

2. Select **Create blank project**.

3. Enter a name for the project.

4. Choose **Settings** > **CI/CD** > **Runners** > **Expand**.

5. Record the address and token for registering the Runner.

6. Import certificate files.

   Check and generate certificate files **admin.crt**, **admin.key**, and **ca.crt** on the master node.

   * View certificate information.

     ```bash
     # cat /etc/kubernetes/admin.conf
     ```

     ![view-cert-config](figures/13.view-cert-config.png)

   * Generate the encrypted **admin.crt**.

     ```bash
     # echo “${client-certificate-data}” | base64 -d > admin.crt
     ```

   * Generate the encrypted **admin.key**.

     ```bash
     # echo “${client-key-data}” | base64 -d > admin.key
     ```

   * Obtain the CA certificate on the manager node.

     ```bash
     # cp /etc/kubernetes/pki/ca.crt ./
     ```

7. Import the three certificate files to the GitLab Runner container on the node where the Runner is running.

   > \[!NOTE]**Note**
   >
   > To import the certificate files, check the node where the GitLab Runner is running, copy the certificate files to the node, and run the **isula cp** command to import the certificate files.

   ```bash
   # isula cp admin.crt [Container ID]:Storage path
   # isula cp admin.key [Container ID]:Storage path
   # isula cp ca.crt [Container ID]:Storage path
   ```

   Note: The **isula cp** command can copy only one file at a time.

   ![import-cert](figures/14.import-cert.png)

## Registering the GitLab Runner

Perform registration in the GitLab Runner container. Currently, interactive registration is used. Obtain the registration information from GitLab. Choose **GitLab** > **Group runners** > **Settings** > **CI/CD** > **Runners**.

![register-gitlab-runner](figures/15.register-gitlab-runner.jpg)

Upload the prepared **gitlab-runner-helper** image to the private image repository in advance, go to the GitLab Runner container, and modify the configuration file.

```bash
# cd /etc/gitlab-runner
# mkdir kubessl
# cp /home/admin.crt /etc/gitlab-runner/kubessl
# cp /home/ca.crt /etc/gitlab-runner/kubessl
# cp /home/admin.key /etc/gitlab-runner/kubessl
# vim /etc/gitlab-runner/config.toml
```

![](figures/17.png)

## Adding the DNS Record of the GitLab Container to the Manager Node

1. View the IP address of the GitLab container.

   ```bash
   # kubectl get pods –Aowide
   ```

2. Add the IP address of the GitLab container to the Kubernetes DNS configuration file.

   ```bash
   # kubectl edit configmaps coredns -n kube-system
   ```

   ![dns](figures/18.dns-config.png)

3. Restart the CoreDNS service.

   ```bash
   # kubectl scale deployment coredns -n kube-system --replicas=0
   # kubectl scale deployment coredns -n kube-system --replicas=2
   ```

## GitLab Running Testing

Return to the GitLab web IDE and choose **CI/CD** > **Editor** > **Create new CI/CD pipeline**.

* Compile the YAML file as follows:

![yaml](figures/20.yaml.png)

* Choose **Pipelines** and view the status.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/cluster_deployment/isulad_k8s/gitlab_runner_deploy.md
---
# gitlab runner部署及测试

## 镜像/软件信息

安装过程中需要用到的镜像名称如下表，版本号为示例安装时用到的版本，仅供参考。

| 镜像                                | 版本     |
|------------------------------------|----------|
| gitlab/gitlab-runner               | alpine-v14.4.0  |
| gitlab/gitlab-runner-helper        | x86\_64-54944146  |

如果在无外网环境中搭建，可以从下方链接提前下载对应的镜像。可在dockerhub官网下载镜像。

## 使用gitlab-runner.yaml启动runner容器

配置gitlab-runner.yaml文件，修改文件中的镜像名，以下为yaml文件的示例参考，请根据实际搭建进行修改。

```bash
vim gitlab-runner.yaml
```

```conf
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitlab-runner
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      name: gitlab-runner
  template:
    metadata:
      labels:
        name: gitlab-runner
    spec:
      containers:
      - args:
        - run
        image: gitlab/gitlab-runner:alpine-v14.4.0
        imagePullPolicy: IfNotPresent
        name: gitlab-runner
        volumeMounts:
        - mountPath: /etc/gitlab-runner
          name: config
          readOnly: false
        - mountPath: /etc/ssl/certs
          name: cacerts
          readOnly: true
      restartPolicy: Always
      volumes:
      - hostPath:
          path: /etc/gitlab-runner
        name: config
      - hostPath:
          path: /etc/ssl/key
        name: cacerts

```

启动容器：

```bash
# kubectl apply -f gitlab-runner.yaml
# kubectl get pod -A -o wide
```

![镜像](figures/7.镜像.png)

## 登录gitlab容器网页-用户证书认证

1. 新建项目。

   ![新建项目](figures/8.新建项目.png)

2. 创建空白项目。

   ![创建空白项目](figures/9.创建空白项目.png)

3. 自定义项目名称。

   ![自定义项目名称](figures/10.自定义项目名称.jpg)

4. 设置--CI/CD--Runner--展开。

   ![设置-cicd-runner](figures/11.设置-cicd-runner.png)

5. 记录注册Runner的地址和令牌。

   ![记下runner地址与令牌](figures/12.记下runner地址与令牌.jpg)

6. 导入证书文件。

   在master节点上查看并生成证书文件，共三个文件admin.crt、admin.key、ca.crt。

   * 查看证书信息

     ```bash
     # cat /etc/kubernetes/admin.conf
     ```

     ![查看证书配置文件](figures/13.查看证书配置文件.png)

   * 加密生成admin.crt

     ```bash
     # echo “${client-certificate-data}” | base64 -d > admin.crt
     ```

   * 加密生成admin.key

     ```bash
     # echo “${client-key-data}” | base64 -d > admin.key
     ```

   * 在manager节点上获取ca的证书

     ```bash
     # cp /etc/kubernetes/pki/ca.crt ./
     ```

7. 在runner运行的节点处将三个证书文件导入gitlab-runner容器。

   > \[!NOTE]说明
   >
   > 导入容器需查看gitlab-runner运行在哪个节点上，将三个证书文件拷贝至该节点，然后使用isula cp命令导入。

   ```bash
   # isula cp admin.crt [容器id]:存放位置
   # isula cp admin.key [容器id]:存放位置
   # isula cp ca.crt [容器id]:存放位置
   ```

   注：isula cp 命令只能一次拷贝一个文件

   ![证书导入文件](figures/14.证书导入文件.png)

## 注册gitlab-runner

进入到runner的容器内进行注册；目前采用交互式注册，注册信息在gitlab上获得，当前配置的 runner服务于项目组，此信息的界面在gitlab->项目组（group）->设置->CI/CD->runner中查看。

![注册gitlab-runner](figures/15.注册gitlab-runner.jpg)

![web端已加入](figures/16.web端已加入_LI.jpg)

将准备好的gitlab-runner-helper镜像提前上传至私有镜像仓，进入gitlab-runner容器中，修改配置文件。

```bash
# cd /etc/gitlab-runner
# mkdir kubessl
# cp /home/admin.crt /etc/gitlab-runner/kubessl
# cp /home/ca.crt /etc/gitlab-runner/kubessl
# cp /home/admin.key /etc/gitlab-runner/kubessl
# vim /etc/gitlab-runner/config.toml
```

![](figures/17.png)

## 在manager节点进行如下操作添加gitlab容器的dns记录

1. 查看gitlab容器的ip地址。

   ```bash
   # kubectl get pods –Aowide
   ```

2. 添加gitlabip地址到k8s dns配置文件。

   ```bash
   # kubectl edit configmaps coredns -n kube-system
   ```

   ![dns](figures/18.dns配置.png)

3. 重启coredns服务。

   ```bash
   # kubectl scale deployment coredns -n kube-system --replicas=0
   # kubectl scale deployment coredns -n kube-system --replicas=2
   ```

## gitlab运行测试

返回gitlab的web界面，选择CI/CD--编辑器--创建CI/CD流水线。

![CICD界面](figures/19.CICD界面.png)

* 编译yaml文件如下：

![yaml文件](figures/20.yaml文件.png)

* 流水线-查看状态。

![流水线状态](figures/21.流水线状态.png)

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/cluster_deployment/isulad_k8s/gitlab_deploy.md
---
# gitlab部署

## 文档说明

gitlab部署 是场景一（基于gitlab-ci从“0”开始构建欧拉原生开发CICD部署）所需步骤，场景二（欧拉原生开发执行机集群被gitlab-ci纳管）可跳过此步骤进入gitlab-runner部署。

## 准备服务器

需准备1台openEuler机器，建议在openEuler-22.03及以上版本运行。

## 启动gitlab

将需要的yaml文件拷贝至/home目录，并启动对应的pod。

> \[!NOTE]说明
> gitlab相关的yaml文件可从官网获得。

以下为yaml文件的示例参考，请根据实际情况进行修改。

gitlab-redis.yaml：

```bash
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: default
  labels:
    name: redis
spec:
  selector:
    matchLabels:
      name: redis
  template:
    metadata:
      name: redis
      labels:
        name: redis
    spec:
      containers:
      - name: redis
        image: 10.35.111.11:5000/redis:latest
        imagePullPolicy: IfNotPresent
        ports:
        - name: redis
          containerPort: 6379
        volumeMounts:
        - mountPath: /var/lib/redis
          name: data
        livenessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 30
          timeoutSeconds: 5
        readinessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        emptyDir: {}

---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: default
  labels:
    name: redis
spec:
  ports:
    - name: redis
      port: 6379
      targetPort: redis
  selector:
    name: redis

```

gitlab-postgresql.yaml：

```bash
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgresql
  namespace: default
  labels:
    name: postgresql
spec:
  selector:
    matchLabels:
      name: postgresql
  template:
    metadata:
      name: postgresql
      labels:
        name: postgresql
    spec:
      containers:
      - name: postgresql
        image: 10.35.111.11:5000/postgres:13.6
        imagePullPolicy: IfNotPresent
        env:
        - name: POSTGRES_HOST_AUTH_METHOD
          value: trust
        - name: DB_USER
          value: gitlab
        - name: DB_PASS
          value: passw0rd
        - name: DB_NAME
          value: gitlab_production
        - name: DB_EXTENSION
          value: pg_trgm
        ports:
        - name: postgres
          containerPort: 5432
        volumeMounts:
        - mountPath: /var/lib/postgresql
          name: data
        livenessProbe:
          exec:
            command:
            - pg_isready
            - -h
            - localhost
            - -U
            - postgres
          initialDelaySeconds: 30
          timeoutSeconds: 5
        readinessProbe:
          exec:
            command:
            - pg_isready
            - -h
            - localhost
            - -U
            - postgres
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        emptyDir: {}

---
apiVersion: v1
kind: Service
metadata:
  name: postgresql
  namespace: default
  labels:
    name: postgresql
spec:
  ports:
    - name: postgres
      port: 5432
      targetPort: postgres
  selector:
    name: postgresql

```

gitlab.yaml：

```bash
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitlab
  namespace: default
  labels:
    name: gitlab
spec:
  selector:
    matchLabels:
      name: gitlab
  template:
    metadata:
      name: gitlab
      labels:
        name: gitlab
    spec:
      containers:
      - name: gitlab
        image: 10.35.111.11:5000/yrzr/gitlab-ce-arm64v8:14.3.2-ce.0
        imagePullPolicy: IfNotPresent
        env:
        - name: TZ
          value: Asia/Shanghai
        - name: GITLAB_TIMEZONE
          value: Beijing
        - name: GITLAB_SECRETS_DB_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_SECRETS_SECRET_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_SECRETS_OTP_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_ROOT_PASSWORD
          value: admin321
        - name: GITLAB_ROOT_EMAIL
          value: 517554016@qq.com
        - name: GITLAB_HOST
          value: git.qikqiak.com
        - name: GITLAB_PORT
          value: "80"
        - name: GITLAB_SSH_PORT
          value: "22"
        - name: GITLAB_NOTIFY_ON_BROKEN_BUILDS
          value: "true"
        - name: GITLAB_NOTIFY_PUSHER
          value: "false"
        - name: GITLAB_BACKUP_SCHEDULE
          value: daily
        - name: GITLAB_BACKUP_TIME
          value: 01:00
        - name: DB_TYPE
          value: postgres
        - name: DB_HOST
          value: postgresql
        - name: DB_PORT
          value: "5432"
        - name: DB_USER
          value: gitlab
        - name: DB_PASS
          value: passw0rd
        - name: DB_NAME
          value: gitlab_production
        - name: REDIS_HOST
          value: redis
        - name: REDIS_PORT
          value: "6379"
        ports:
        - name: http
          containerPort: 80
        - name: ssh
          containerPort: 22
        volumeMounts:
        - mountPath: /home/git/data
          name: data
        livenessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 180
          timeoutSeconds: 5
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        emptyDir: {}

---
apiVersion: v1
kind: Service
metadata:
  name: gitlab
  namespace: default
  labels:
    name: gitlab
spec:
  ports:
    - name: http
      port: 80
      targetPort: http
      nodePort: 30852
    - name: ssh
      port: 22
      nodePort: 32353
      targetPort: ssh
  selector:
    name: gitlab
  type: NodePort
```

启动相应的容器：

```bash
# kubectl apply -f gitlab-redis.yaml
# kubectl apply -f gitlab-postgresql.yaml
# kubectl apply -f gitlab.yaml
```

可通过命令查看gitlab pod是否搭建完成：

```bash
# kubectl get pod -A -owide
```

## 登录gitlab

查看是否可以登录gitlab网页，网址为ip地址加设定的端口。

![网页入口](figures/4.gitlab网页入口.jpg)
用户名为root，默认密码需进入容器后查看密码文件。

```bash
# kubectl exec -it gitlab-lab -n default /bin/sh
# cat /etc/gitlab/initial_root_password
```

![查询密码](figures/5.查询密码.jpg)

* 登录后界面如图：

![登录后页面](figures/6.登录后页面.png)

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/gnome/gnome_user_guide.md
---
# GNOME User Guide

## 1. Overview

GNOME is a desktop environment for Unix-like operating systems. As the officially desktop of GNU Project, GNOME aims to build a comprehensive, easy-to-use, and user-friendly desktop environment for Unix or Unix-like operating systems based on free software.

GNOME provides the following functional components:

ATK: accessibility toolkit.

Bonobo: component framework to compound documents.

GObject: object-oriented framework in C language.

GConf: system for storing configuration settings of apps.

GNOME VFS: virtual file system.

GNOME Keyring: security system.

GNOME Print: software for printing documents.

GStreamer: multimedia framework of GNOME.

GTK+: building toolkit.

Cairo: complex 2D graphics library.

Human Interface Guidelines: software development documents provided by Sun Microsystems to facilitate GNOME usage.

LibXML: XML library designed for GNOME.

ORBit: CORBA Object Request Broker (ORB) that makes software componentized.

Pango: library for i18n text arrangement and transformation.

Metacity: window manager.

This document describes how to use GNOME.

The following figure shows the GUI.

![Figure 1](./figures/gnome-1.PNG)

## 2. Desktop

### 2.1 Desktop

The GNOME desktop is clean because it does not display any files or directories. Only the left, middle, and right parts of the top bar on the desktop have entry options. They are the activity entry, message notification entry, and system status entry.

![Figure 2](./figures/gnome-2.PNG)

### 2.2 Shortcut Menu

After you right-click in the blank area on the desktop, a shortcut menu shown in the following figure is displayed, providing users with some shortcut functions.

![Figure 3](./figures/gnome-3.PNG)

The following table describes the shortcuts.

| Shortcut| Description|
| :------------ | :------------ |
| Change Background| Changes the image displayed on the desktop.|
| Display Settings| Sets the resolution, screen rotation, and night light.|
| Settings| Navigates to system settings.|

## 3. Top Bar on the Desktop

### 3.1 Activities

The **Activities** entry is located in the upper left corner of the desktop. It contains app favorites, lists of all apps and active apps, a multi-view switchover function, and an indicator to the current active app.

#### 3.1.1 App Favorites

![Figure 4](./figures/gnome-4.PNG)

You can right-click an app icon in **Favorites** and choose **Remove from Favorites** from the shortcut menu to remove the app from **Favorites**.

#### 3.1.2 List of All Apps

To display the list of all apps, click the ![Figure 5](./figures/gnome-5.PNG) icon under the app favorites folder.

![Figure 6](./figures/gnome-6.PNG)

Similarly, you can right-click an app icon in the app list and choose **Add to Favorites** from the shortcut menu to add the app to **Favorites**.

If there are so many apps and you know their names, you can enter an app name in the search box to search for it.

![Figure 7](./figures/gnome-7.PNG)

#### 3.1.3 List of Active Apps

Active apps, that is, running apps are displayed one by one after the last app in **Favorites**. There is a white dot under the icon of each active app.

![Figure 8](./figures/gnome-8.PNG)

If you right-click an active app, operations that can be performed on the app are displayed. The operations vary with apps. Take **Screenshot** as an example. See the following figure.

![Figure 9](./figures/gnome-9.PNG)

#### 3.1.4 Multi-View Switchover

As you view the active app list, the active apps are displayed on the right of the list in multi-view mode.

![Figure 10](./figures/gnome-10.PNG)

When you move the cursor to the right of the multi-view page, the vertical bar on the right becomes wider to display the window and desktop of the current active app. You can click the desktop image to switch back to the desktop.

![Figure 11](./figures/gnome-11.PNG)

If you click another app, it will be displayed on the top of the vertical bar.

#### 3.1.5 Indicator to the Current Active App

The indicator to the current active app is displayed on the right of **Activities**. You can click the indicator to display the operations that can be performed on the app. The operations vary with the apps. Take **Terminal** as an example. See the following figure.

![Figure 12](./figures/gnome-12.PNG)

You can click **Preferences** to set the terminal preferences.

### 3.2 Message Notification

The message notification entry is located in the middle of the top bar on the desktop, including message notification, calendar, clock, and weather.

![Figure 13](./figures/gnome-13.PNG)

#### 3.2.1 Message Notification

If you set an alarm or countdown timer in **Clocks**, messages will be displayed on the left of the notification pane when the timer expires. The detailed information about the to-do items set in **Calendar** are also displayed on the left of the notification pane, and the summary information is displayed below the calendar on the right.

![Figure 14](./figures/gnome-14.PNG)

You can click **Do Not Disturb** to close pop-up notifications on the desktop.

#### 3.2.2 Calendar

As shown in the preceding figure, the calendar is displayed on the right, and there is a dot under the date of a to-do item. You can click the date to view the summary about a to-do item at the bottom of the calendar.

#### 3.2.3 Clock and Weather

You can also add the clock and weather to areas under the calendar. Clicking the **World Clocks** area will invoke the **Clocks** app, and clicking the **Weather** area will invoke the **Weather** app.

![Figure 15](./figures/gnome-15.PNG)

### 3.3 System Status

The system status entry is located in the upper right corner of the desktop. It contains multiple options, as described in the following table.

| Option| Description|
| :------------ | :------------ |
| Sound| Volume slider|
| Ethernet| Ethernet cards and their connections|
| Location In Use| Location of the system|
| Settings| System settings|
| Lock| Immediate screen lock. A password is required to unlock the screen.|
| Power Off/Log Out| Suspension, shutdown, restart, and logout|

![Figure 16](./figures/gnome-16.PNG)

The system status displayed here varies according to different settings and system configurations, such as Wi-Fi, Bluetooth, and battery. System statuses can also be appended to the left of the upper right corner by other apps, such as the input source display in the preceding figure.

#### 3.3.1 Sound

Quickly adjust the volume. To further set the sound, open the system settings.

#### 3.3.2 Network

Quickly enable or disable the network. To further configure the network, open the system settings.

![Figure 17](./figures/gnome-17.PNG)

#### 3.3.3 Location Service

Quick enable or disable the location service. To further set the location, open the system settings.

![Figure 18](./figures/gnome-18.PNG)

#### 3.3.4 Settings

It is one of the convenient entries to system settings.

![Figure 19](./figures/gnome-19.PNG)

You can set a large number of system-related options in the **Settings** window, which are shown in the left pane of the preceding and following figures.

![Figure 20](./figures/gnome-20.PNG)

The settings are also dynamically extended. For example, if the hardware where the system is located has Wi-Fi, the Wi-Fi item is displayed. Some important settings are described in the following sections.

#### 3.3.5 Lock

If you click **Lock**, the screen is locked and turns black. When you move the cursor, the screen turns on immediately. You can press any key to access the login page and enter the password to log in to the system again. The following figure shows the lock screen.

![Figure 21](./figures/gnome-21.PNG)

#### 3.3.6 Power-off/Logout

The actions include suspension, power-off, restart, and logout. The difference between suspension and locking is that a black screen is directly displayed after suspension. You need to use the keyboard to wake up the login page, which takes a longer time than screen locking. Logout is to log out the current user and return to the login page without a black screen. You can use the same or another user account to log in again.

![Figure 22](./figures/gnome-22.PNG)

The following figure shows the user login page.

![Figure 23](./figures/gnome-23.PNG)

After the locking and suspension is waked up, the lock screen is displayed first. You need to press a key or click the screen to enter the user login page. The login page is directly displayed after the logout and restart.

## 4. Common System Settings and App Examples

### 4.1 Examples of System Settings

There are four entries to system settings:

Right-click on the desktop and choose **Settings**.

Click the system status entry in the upper right corner and choose **Settings**.

Click the **Activities** entry in the upper left corner and choose **Settings**.

On the **Terminal**, run the **gnome-control-center** command.

#### 4.1.1 Network

![Figure 24](./figures/gnome-19.PNG)

Wired networks are displayed here. You can click the button to enable or disable a network. You can also set the VPN and network proxy.

Click the gear icon on the right of an Ethernet connection to view details, and modify or remove the connection.

![Figure 25](./figures/gnome-24.PNG)

Change the connection name.

![Figure 26](./figures/gnome-25.PNG)

Change the IP address obtaining mode (**Automatic** or **Manual**), and add the DNS and a route.

![Figure 27](./figures/gnome-26.PNG)

You can also click the plus sign (+) above the gear icon to create a connection. The settings of the new connection are similar to those shown in preceding figures. The prerequisite is that the Ethernet port exists.

#### 4.1.2 Displays

You can set the fixed resolution on the **Displays** tab page. If the resolution of your hardware system is not included, set it on the command line. Then, the newly set resolution will be displayed here.

![Figure 28](./figures/gnome-27.PNG)

Select a resolution and click **Keep Changes** to make the settings take effect.

![Figure 29](./figures/gnome-28.PNG)

Some displays allow you to rotate the screen vertically, for example, to view the text at the bottom of the screen at a time. The **Orientation** here also provides such support.

![Figure 30](./figures/gnome-29.PNG)

#### 4.1.3 Keyboard Shortcuts

You can set keyboard shortcuts to perform shortcut operations, such as quickly opening the home folder, camera, or browser. GNOME does not provide a shortcut for starting the **Terminal**. You can set a default one.

View existing shortcut settings in scrolling mode or search for shortcuts.

![Figure 31](./figures/gnome-30.PNG)

Clicking a disabled item, such as the home folder and web browser, triggers shortcut settings.

![Figure 32](./figures/gnome-31.PNG)

![Figure 33](./figures/gnome-32.PNG)

Effect after the setting is successful.

![Figure 34](./figures/gnome-33.PNG)

Scroll the keyboard shortcuts page to the bottom and click + to add a shortcut for opening the **Terminal**.

![Figure 35](./figures/gnome-34.PNG)

![Figure 36](./figures/gnome-35.PNG)

![Figure 37](./figures/gnome-36.PNG)

![Figure 38](./figures/gnome-37.PNG)

Now, you can press **Ctrl+Alt+T** to open the **Terminal**. Settings of the home folder and web browser are similar.

![Figure 39](./figures/gnome-38.PNG)

#### 4.1.4 Region and Language

The system can be switched between multiple languages, even if a language is not selected during system installation.

![Figure 40](./figures/gnome-39.PNG)

You can click **Language** and **Formats** to change the language from Chinese to English, and click **Restart**. You need to log in to the system again and restart the session for the language settings to take effect.

![Figure 41](./figures/gnome-40.PNG)

![Figure 42](./figures/gnome-41.PNG)

![Figure 43](./figures/gnome-42.PNG)

Click the gear icon on the right of **Input Sources** to view the keyboard shortcuts and input source options. You can click the plus sign (+) to add an input source.

![Figure 44](./figures/gnome-43.PNG)

When you use the shortcut to switch the input method, you can view the change in the system status area in the upper right corner.

![Figure 45](./figures/gnome-44.PNG)

#### 4.1.5 Users

You can add and delete users on the **Users** GUI. For a non-root user, you need to click **Unlock** and enter the password of the super user to display the complete information.

![Figure 46](./figures/gnome-45.PNG)

Click **Password** to change the password of the current user.

![Figure 47](./figures/gnome-46.PNG)

Click **Account Activity** to view the login status of the user in this week.

![Figure 48](./figures/gnome-47.PNG)

Click **Add User** in the upper right corner to add a user and set the password when adding the user or when logging in to the system as the new user. To log in to the system as a new user, log out of the system and then log in as the new user. The new user can be removed by clicking **Remove User**. The current login user cannot be removed.

![Figure 49](./figures/gnome-48.PNG)

### 4.2 Application Examples

#### 4.2.1 Files

The binary file name of the **Files** app is **nautilus**. You can create, modify, move, save, and delete files in the file system displayed in **Files**.

![Figure 50](./figures/gnome-49.PNG)

#### 4.2.2 Terminal

The running **Terminal** is a special process under the GNOME login session. It functions as a console and is a new session in essence. It can perform almost all the tasks that the console can do, and it is what Linux would be without a graphical interface.

![Figure 51](./figures/gnome-50.PNG)

In the **Preferences** dialog box, you can set the font, character spacing, and theme background.

#### 4.2.3 Software

In **Software**, you can search for and install many free open source apps, and view and uninstall installed apps.

![Figure 52](./figures/gnome-51.PNG)

![Figure 53](./figures/gnome-52.PNG)

#### 4.2.4 Browser

GNOME has a built-in browser named **Web**. Its interface and functions are simpler than those of Chrome or Firefox, but supports common functions, such as bookmarks, search engine settings, history, and file download.

![Figure 54](./figures/gnome-53.PNG)

#### 4.2.5 System Monitor

It is similar to the Task Manager in Windows operating systems, on which you can view the process name, user, and usage of CPU and memory resources. This monitor is dynamic, but its change effect is much worse than that of running the top command.

![Figure 55](./figures/gnome-54.PNG)

You can also view the usage trend of important components such as the CPU, memory, and network.

![Figure 56](./figures/gnome-55.PNG)

#### 4.2.6 Text Editor

A text editor is required for creating, modifying, and saving files. In its **Preferences** dialog box, you can set the font, tab width, theme, and plug-ins.

![Figure 57](./figures/gnome-56.PNG)

#### 4.2.7 Sysprof

Sysprof samples and presents a system, including the software and hardware, and is used to locate system performance problems, for example, app startup freezing and system response delay. You can select the project to be traced and click **Record** to start sampling.

![Figure 58](./figures/gnome-57.PNG)

![Figure 59](./figures/gnome-58.PNG)

After the sampling is stopped, the result provides abundant information for diagnosis and analysis.

![Figure 60](./figures/gnome-59.PNG)

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/gnome/gnome_user_guide.md
---
# Gnome 用户指南

## 一 概述

Gnome是运行在类Unix操作系统中最常用桌面环境。其目标是基于自由软件，为Unix或者类Unix操作系统构造一个功能完善、操作简单以及界面友好的桌面环境，是GNU计划的正式桌面。

Gnome提供了多个功能部件：

ATK：可达性工具包。

Bonobo：复合文档技术。

GObject：用于C语言的面向对象框架。

GConf：保存应用软件设置。

GNOME VFS：虚拟文件系统。

GNOME Keyring：安全系统。

GNOME Print：GNOME软件打印文档。

GStreamer：GNOME软件的多媒体框架。

GTK+：构件工具包。

Cairo：复杂的2D图形库。

Human Interface Guidelines：Sun微系统公司提供的使得GNOME应用软件易于使用的研究和文档。

LibXML：为GNOME设计的XML库。

ORBit：使软件组件化的CORBAORB。

Pango：i18n文本排列和变换库。

Metacity：窗口管理器。

本文主要描述 Gnome 的使用。

界面如下图所示：

![图 1 桌面主界面-big](./figures/gnome-1.png)

## 二 桌面

### 2.1 桌面

Gnome桌面比较干净，不摆放任何文件或者目录。桌面仅在顶部左、中、右三部分有入口选项。它们分别是活动程序入口、消息通知入口、系统状态入口。

![图 2 桌面顶部图标-big](./figures/gnome-2.png)

### 2.2 右键菜单

在桌面空白处单击鼠标右键，出现的菜单如下图所示，为用户提供了一些快捷功能。

![图 3 右键菜单](./figures/gnome-3.png)

选项说明如下表：

| 选项 | 说明|
| :------------ | :------------ |
| 更换壁纸 | 更换桌面显示图像 |
| 显示设置 | 屏幕分辨率、屏幕旋转及夜间模式设置 |
| 设置 | 系统设置 |

## 三 桌面顶部

### 3.1 活动程序

活动程序入口位于桌面左上角，其包括应用程序收藏夹、所有应用程序列表、活动程序列表、多视图切换、当前活动程序指示。

#### 3.1.1 应用程序收藏夹

![图 4 应用程序收藏夹-big](./figures/gnome-4.png)

右键点击收藏夹内的应用程序图标，可以选择"从收藏夹中移除"，从而把应用移出收藏夹。

#### 3.1.2 所有应用程序列表

点击应用程序收藏夹下面的九个小点"![](./figures/gnome-5.png)"——"显示应用程序"，可打开所有应程序列表。

![图 5 所有应用程序列表1-big](./figures/gnome-6.png)

右键点击列表内的应用程序图标，可以选择"添加到收藏夹"，从而把应用加入收藏夹。

当应用程序很多并且知道其名字时，可以在搜索一栏输入应用名字进行搜索打开。

![图 6 所有应用程序列表2-big](./figures/gnome-7.png)

#### 3.1.3 活动程序列表

当多个程序打开，即存在多个活动程序时，活动程序会在收藏夹内最后一个应用程序后逐个显示。打开的程序图标下有个点表示已打开。

![图 7 活动程序列表-big](./figures/gnome-8.png)

右键单击活动程序，可以弹出活动状态下此程序可以进行的一些操作，不同的程序可执行的操作不同。以"截图"程序为例，如图所示：

![图 8 活动程序右键菜单-big](./figures/gnome-9.png)

#### 3.1.4 多视图切换

查看活动程序列表的同时，活动程序也以多视图方式显示在活动程序列表右侧。

![图 9 多视图切换1-big](./figures/gnome-10.png)

鼠标移到多视图右侧，右侧竖条将会变宽，显示当前桌面处于顶层的活动程序窗口和桌面。点击桌面图像将会切换回桌面。

![图 10 多视图切换2-big](./figures/gnome-11.png)

点击不属于当前活动程序的其他程序，则顶层切换到相应应用程序。

#### 3.1.5 当前活动程序指示

当前活动程序的指示会显示在活动程序入口右侧，并且点击可弹出活动状态下此程序可以进行的一些操作，不同的程序可执行的操作不同。以"终端"程序为例，如图所示：

![图 11 当前活动程序指示-big](./figures/gnome-12.png)

点击"首选项"可对终端进行设置。

### 3.2 消息通知

消息通知入口位于桌面顶部中央，内容包括消息通知、日历、时钟和天气。

![图 12 消息通知-big](./figures/gnome-13.png)

#### 3.2.1 消息通知

当在"时钟"程序中设置闹钟和倒计时，定时到了后都会将消息通知到消息通知入口的左侧。在"日历"程序中设置的待办事项，详细信息也会显示到左侧消息通知，其概要信息将会显示到右侧日历下方。

![图 13 消息-big](./figures/gnome-14.png)

点击"请勿打扰"可以关闭这些消息在外部(没有点击此入口之前)的弹窗通知。

#### 3.2.2 日历

如上图，右侧显示日历，对于有待办事项的日期，其下会有一个点。点击这些日期，可在日历下方看到待办事项概要信息。

#### 3.2.3 时钟和天气

可以把时钟和天气添加到右下角连同日历一并作信息显示。点击时钟位置将调用"时钟"程序并使用其"世界时钟"的功能，点击"天气"将调用"天气"程序。

![图 14 时钟和天气-big](./figures/gnome-15.png)

### 3.3 系统状态

系统状态入口位于桌面右上角，其包括多个选项，部分说明如下表：

| 选项 | 说明|
| :------------ | :------------ |
| 声音 | 音量调节 |
| 以太网 | 以太网卡及其连接状况 |
| 定位服务 | 系统所在位置 |
| 设置 | 系统设置 |
| 锁定 | 立即锁屏，再次打开需要密码 |
| 关机/注销 | 包括挂起、关机、重启、注销四种动作 |

![图 15 系统状态-big](./figures/gnome-16.png)

不同的设置和系统配置此处显示的系统状态也有所不同，例如wifi、蓝牙、电池。系统状态还可以由其他程序追加到右上角左侧，例如上图中输入源相关的显示。

#### 3.3.1 声音

快捷的音量调节设置。如要进一步设置声音需要打开系统设置。

#### 3.3.2 网络

快捷的网络禁用与否设置。如要进一步设置网络需要打开系统设置。

![图 16 网络状态-big](./figures/gnome-17.png)

#### 3.3.3 定位服务

快捷的定位服务禁用与否设置。如要进一步设置定位需要打开系统设置。

![图 17 定位状态-big](./figures/gnome-18.png)

#### 3.3.4 设置

便捷的系统设置入口之一。

![图 18 设置1-big](./figures/gnome-19.png)

系统设置可以设置数量众多系统相关的选项，除了上图左侧已经展示的项，余下的项如下图左侧。

![图 19 设置2-big](./figures/gnome-20.png)

设置项也是动态拓展的，例如当系统所在的硬件有wifi时，wifi的设置项将会出现在设置中。一些重要的设置项将在下文中举例。

#### 3.3.4 锁定

点击"锁定"将立即回到锁屏界面并黑屏，鼠标移动立马亮屏，点击任意按键进入登录界面，输入用户名密码以实现再次登录。以下是锁屏界面。

![图 20 锁屏界面-big](./figures/gnome-21.png)

#### 3.3.4 关机/注销

包括挂起、关机、重启、注销四种动作。其中挂起和锁定的区别在于挂起后直接黑屏，需要使用键盘唤醒到登录界面，时间较锁屏久。注销是登出当前用户，退回到登录界面并且不黑屏，以选择另外的用户登录或者相同用户再次登录。
风险提示:
ARM架构（鲲鹏920）不支持OS挂起，如果用户点击“挂起”或者开启“自动挂起”功能，会导致OS挂死，无法正常使用。

![图 21 关机\_注销-big](./figures/gnome-22.png)

以下是用户登录界面。

![图 22 登录界面-big](./figures/gnome-23.png)

锁定和挂起唤醒后首先进入锁屏界面，再次点按钮或者按键才进入用户登录界面。注销和重启后直接进入登录界面。

## 四 常用系统设置和应用举例

### 4.1 系统设置举例

系统设置有四个入口，分别是：

* 桌面右键->设置

* 右上角系统状态入口->设置

* 左上角活动程序入口->设置

* 在终端中->执行gnome-control-center

#### 4.1.1 网络

![图 23 网络设置1-big](./figures/gnome-19.png)

这里显示有线网络，点击按钮可以打开和关闭网络。还可以设置vpn和网络代理。

点击以太网的某个连接右侧的小齿轮，可查看此连接的详细信息，也可对此连接进行修改，包括移除此连接。

![图 24 网络设置2-big](./figures/gnome-24.png)

修改此连接名字。

![图 25 网络设置3-big](./figures/gnome-25.png)

修改ip地址获取方式(自动，手动)，添加DNS，添加路由等。

![图 26 网络设置4-big](./figures/gnome-26.png)

点击小齿轮上的"+"号，可以从头创建一个连接，新连接的设置项和上图类似，前提是这个以太网口要存在。

#### 4.1.2 显示器

固定分辨率设置可在"显示器"一项进行配置，如果这里不包含你的硬件系统的分辨率，那么需要在命令设置好分辨率，再打开此处设置，新加入的分辨率将会显示在这里。

![图 27 分辨率设置-big](./figures/gnome-27.png)

选择分辨率后点击弹出的"保留更改"以使设置生效。

![图 28 分辨率设置确认-big](./figures/gnome-28.png)

有些显示器允许旋转以便竖向观察屏幕，例如文本可以一次查看到更底部的内容。这里的"方向"也提供支持。

![图 29 屏幕方向-big](./figures/gnome-29.png)

#### 4.1.3 键盘快捷键

设置键盘快捷键可以执行快捷的操作，例如快速打开主目录，摄像头或者浏览器等。gnome没有为打开终端设置快捷键，可以考虑设置打开终端的默认快捷键。

已有的快捷键设置可以滚动查看也可以进行搜索。

![图 30 键盘快捷键-big](./figures/gnome-30.png)

单击已禁用的项，例如主目录和网页浏览器，可以触发快捷键设置。

![图 31 快捷键设置-big](./figures/gnome-31.png)

![图 32 快捷键设置反馈-big](./figures/gnome-32.png)

设置成功后的效果。

![图 33 快捷键设置结果-big](./figures/gnome-33.png)

拖动"键盘快捷键"设置到底部，点击"+"新加一个打开终端的快捷键。

![图 34 快捷键添加设置1-big](./figures/gnome-34.png)

![图 35 快捷键添加设置2-big](./figures/gnome-35.png)

![图 36 快捷键添加设置3-big](./figures/gnome-36.png)

![图 37 快捷键添加设置结果-big](./figures/gnome-37.png)

按住“ctrl+alt+t”可打开终端。主目录和网页浏览器与此相似。

![图 38 快捷键测试-big](./figures/gnome-38.png)

#### 4.1.4 区域和语言

系统语言可以在多种语言间切换，即使安装系统时没有选择这种语言。

![图 39 区域和语言-big](./figures/gnome-39.png)

点击语言和格式选择，可以把语言从中文换为英文，并点击重启按钮，需要重新登录重启会话设置才能生效。

![图 40 语言设置-big](./figures/gnome-40.png)

![图 41 格式设置-big](./figures/gnome-41.png)

![图 42 语言和格式设置结果-big](./figures/gnome-42.png)

点击"输入源"右侧的小齿轮可以查看到输入法的快捷键以及输入源选项设置。点击其下方的"+"可以添加输入源。

![图 43 输入设置-big](./figures/gnome-43.png)

当使用快捷键切换输入法时，可以在右上角系统状态处查看到变化。

![图 44 输入切换结果查看-big](./figures/gnome-44.png)

#### 4.1.5 用户

可以在图形界面添加和删除用户。使用非root用户登录时，此项功能需要点击"解锁"，输入超级权限用户的密码才能显示完整。

![图 45 用户-big](./figures/gnome-45.png)

点击密码可以修改当前用户的密码。

![图 46 修改用户密码-big](./figures/gnome-46.png)

点击帐号活动可以看到本周此用户的登录状况。

![图 47 用户活动记录-big](./figures/gnome-47.png)

点击右上角添加用户可以添加一个新用户并在添加用户时设置密码或者在登录时设置密码。登录新用户需要先登出当前用户再选择新用户登录。新加的用户可以点"移除用户"移除，当然，不能移除当前登录的用户。

![图 48 添加新用户-big](./figures/gnome-48.png)

### 4.2 应用举例

#### 4.2.1 文件

"文件"应用的二进制文件名为nautilus。可在"文件"所图形化显示的文件系统内创建、修改、移动、保存和删除文件等操作。

![图 49 文件系统主目录-big](./figures/gnome-49.png)

#### 4.2.2 终端

运行的"终端"是gnome登录会话下的特殊进程，它相当于一个控制台，本质是一个新的会话。它几乎能完成控制台能完成的所有任务，这是没有图形界面的linux原本的样子。

![图 50 终端及其设置项-big](./figures/gnome-50.png)

在其"配置文件首选项"中能对字体，字符间隔，主题背景等选项进行设置。

#### 4.2.3 软件

"软件"内能搜索安装许多开源免费的应用，也能查看和卸载当前已经安装的程序。

![图 51 软件-big](./figures/gnome-51.png)

![图 52 已安装-big](./figures/gnome-52.png)

#### 4.2.4 浏览器

gnome自带了名为"Web"的浏览器，其界面和功能较google或者firefox浏览器要简单，但是常见的书签、搜索引擎设置、历史记录、文件下载等均支持。

![图 53 浏览器-big](./figures/gnome-53.png)

#### 4.2.5 系统监视器

相当于Windows下的任务管理器，可以看到进程名字、用户、cpu和内存等资源的使用状况。这个是动态的，不过其变化效果远逊于top命令。

![图 54 进程-big](./figures/gnome-54.png)

还可以看到cpu、内存、网络等重要部件的利用率走势。

![图 55 资源-big](./figures/gnome-55.png)

#### 4.2.6 文本编辑器

创建、修改、保存文件等操作需要文本编辑器。在其菜单栏的"首选项"中可进行字体、制表符宽度、主题、插件等选项进行设置。

![图 56 文本编辑器-big](./figures/gnome-56.png)

#### 4.2.7 Sysprof

这是对系统包括软硬件的采样和呈现，这可以用来定位系统的性能问题，例如应用启动卡顿，系统响应延迟等。点击选择要跟踪的项目并点击"Record"，便开始采样。

![图 57 采样对象选择-big](./figures/gnome-57.png)

![图 58 开始记录-big](./figures/gnome-58.png)

停止采样后，采样结果提供了非常丰富的信息，可用于诊断和分析。

![图 59 采样结果-big](./figures/gnome-59.png)

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/high_availability/ha/ha_installation_and_deployment.md
---
# HA Installation and Deployment

This document describes how to install and deploy an HA cluster.

## Installation and Deployment

* Prepare the environment: At least two physical machines or VMs with openEuler installed are required. (This section uses two physical machines or VMs as an example.) For details about how to install openEuler openEuler 22.03 LTS SP4, see the [*openEuler Installation Guide*](../../installation_upgrade/installation/installation_on_servers.md).

### Modifying the Host Name and the /etc/hosts File

* **Note: You need to perform the following operations on both hosts. The following takes one host as an example. IP addresses in this document are for reference only.**

Before using the HA software, ensure that all host names have been changed and written into the **/etc/hosts** file.

* Run the following command to change the host name:

```shell
hostnamectl set-hostname ha1
```

* Edit the **/etc/hosts** file and write the following fields:

```conf
172.30.30.65 ha1
172.30.30.66 ha2
```

### Configuring the Yum Repository

After the system is successfully installed, the Yum source is configured by default. The file location is stored in the **/etc/yum.repos.d/openEuler.repo** file. The HA software package uses the following sources:

```conf
[OS]
name=OS
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler

[everything]
name=everything
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/RPM-GPG-KEY-openEuler

[EPOL]
name=EPOL
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler
```

### Installing the HA Software Package Components

```shell
yum install -y corosync pacemaker pcs fence-agents fence-virt corosync-qdevice sbd drbd drbd-utils
```

### Setting the hacluster User Password

```shell
passwd hacluster
```

### Modifying the /etc/corosync/corosync.conf File

```conf
totem {
        version: 2
        cluster_name: hacluster
         crypto_cipher: none
        crypto_hash: none
}
logging {         
        fileline: off
        to_stderr: yes
        to_logfile: yes
        logfile: /var/log/cluster/corosync.log
        to_syslog: yes
        debug: on
       logger_subsys {
               subsys: QUORUM
               debug: on
        }
}
quorum {
           provider: corosync_votequorum
           expected_votes: 2
           two_node: 1
       }
nodelist {
       node {
               name: ha1
               nodeid: 1
               ring0_addr: 172.30.30.65
               }
        node {
               name: ha2
               nodeid: 2
               ring0_addr: 172.30.30.66
               }
        }
```

### Managing the Services

#### Disabling the firewall

1. Stop the firewall.

   ```shell
   systemctl stop firewalld
   ```

2. Change the status of SELINUX in the **/etc/selinux/config** file to disabled.

   ```conf
   # SELINUX=disabled
   ```

#### Managing the pcs service

1. Start the pcs service.

   ```shell
   systemctl start pcsd
   ```

2. Query the pcs service status.

   ```shell
   systemctl status pcsd
   ```

   The service is started successfully if the following information is displayed:

   ![](./figures/HA-pcs.png)

#### Managing the Pacemaker service

1. Start the Pacemaker service.

   ```shell
   systemctl start pacemaker
   ```

2. Query the Pacemaker service status.

   ```shell
   systemctl status pacemaker
   ```

   The service is started successfully if the following information is displayed:

   ![](./figures/HA-pacemaker.png)

#### Managing the Corosync service

1. Start the Corosync service.

   ```shell
   systemctl start corosync
   ```

2. Query the Corosync service status.

   ```shell
   systemctl status corosync
   ```

   The service is started successfully if the following information is displayed:

   ![](./figures/HA-corosync.png)

### Performing Node Authentication

* **Note: Run this command on any node.**

```shell
pcs host auth ha1 ha2
```

### Accessing the Front-End Management Platform

After the preceding services are started, open the browser (Chrome or Firefox is recommended) and enter `https://localhost:2224` in the navigation bar.

* This page is the native management platform.

![](./figures/HA-login.png)

For details about how to install the management platform newly developed by the community, see <https://atomgit.com/openeuler/ha-api/blob/master/docs/build.md>.

* The following is the management platform newly developed by the community.

![](./figures/HA-api.png)

* For how to quickly use an HA cluster and add an instance, see the [HA Usage Example](ha_usecase_examples.md).

---

---
url: /en/docs/22.03_LTS_SP4/server/high_availability/ha/ha_usecase_examples.md
---
# HA Usage Examples

This section describes how to get started with the HA cluster and add an instance. If you are not familiar with HA cluster installation, see [HA Installation and Deployment](ha_installation_and_deployment.md).

## Quick Start Guide

The following operations use the management platform newly developed by the community as an example.

### Login Page

The user name is `hacluster`, and the password is the one set on the host by the user.

![](./figures/HA-api.png)

### Home page

After logging in to the system, the main page is displayed. The main page consists of the side navigation bar, the top operation area, the resource node list area, and the node operation floating area.

The following describes the features and usage of the four areas in detail.

![](./figures/HA-home-page.png)

#### Navigation bar

The side navigation bar consists of two parts: the name and logo of the HA cluster software, and the system navigation. The system navigation consists of three parts: **System**, **Cluster Configurations**, and **Tools**. **System** is the default option and the corresponding item to the home page. It displays the information and operation entries of all resources in the system. **Preference Settings** and **Heartbeat Configurations** are set under **Cluster Configurations**. **Log Download** and **Quick Cluster Operation** are set under **Tools**. These two items are displayed in a pop-up box after you click them.

#### Top Operation Area

The current login user is displayed statically. When you hover the mouse cursor on the user icon, the operation menu items are displayed, including **Refresh Settings** and **Log Out**. After you click **Refresh Settings**, the **Refresh Settings** dialog box is displayed with the **Refresh Settings** option. You can set the automatic refresh modes for the system, the options are **Do not refresh automatically**, **Refresh every 5 seconds**, and **Refresh every 10 seconds**. By default, **Do not refresh automatically** is selected. Click **Log Out** to log out and jump to the login page. After that, a re-login is required if you want to continue to access the system.

![](./figures/HA-refresh.png)

#### Resource Node List Area

The resource node list displays the resource information such as **Resource Name**, **Status**, **Resource Type**, **Service**, and **Running Node** of all resources in the system, and the node information such as all nodes in the system and the running status of the nodes. In addition, you can **Add**, **Edit**, **Start**, **Stop**, **Clear**, **Migrate**, **Migrate Back**, **Delete**, and **Associate** the resources.

#### Node Operation Floating Area

By default, the node operation floating area is collapsed. When you click a node in the heading of the resource node list, the node operation area is displayed on the right, as shown in the preceding figure. This area consists of the collapse button, the node name, the stop button, and the standby button, and provides the stop and standby operations. Click the arrow in the upper left corner of the area to collapse the area.

### Preference Settings

The following operations can be performed using command lines. The following is a simple example. For more command details, run the `pcs --help` command.

* Through the CLI

  ```shell
  # pcs property set stonith-enabled=false
  # pcs property set no-quorum-policy=ignore
  ```

  Run the following command to view all configurations:

  ```shell
  pcs property
  ```

  ![](./figures/HA-firstchoice-cmd.png)

* Through the GUI
  Clicking **Preference Settings** in the navigation bar, the **Preference Settings** dialog box is displayed. Change the values of **No Quorum Policy** and **Stonith Enabled** from the default values to the values shown in the following figure. Then, click OK.

  ![](./figures/HA-firstchoice.png)

### Add Resource

#### Adding Common Resources

1. Click **Add Common Resource**. The **Create Resource** dialog box is displayed.
   All mandatory configuration items of a resource are displayed on the **Basic** page. After you select a resource type on the **Basic** page, other mandatory and optional configuration items of the resource are displayed.

2. Enter the resource configuration information.
   A gray text area is displayed on the right of the dialog box to describe the current configuration item. After all mandatory parameters are set, click **OK** to create a common resource or click **Cancel** to cancel the add operation.
   The optional configuration items on the **Instance Attribute**, **Meta Attribute**, or **Operation Attribute** page are optional. If they are not configured, the resource creation process is not affected. You can modify them as required. Otherwise, the default values are used.

The following uses Apache as an example to describe how to add resources through the CLI and GUI.

* Through the CLI

  ```shell
  # pcs resource create httpd ocf:heartbeat:apache
  ```

  Check the resource running status:

  ```shell
  # pcs status
  ```

  ![](./figures/HA-pcs-status.png)

* Through the GUI

1. Enter the resource name and resource type, as shown in the following figure.

   ![](./figures/HA-add-resource.png)

2. If the following information is displayed, the resource is successfully added and started, and runs on a node, for example, ha1.

   ![](./figures/HA-apache-suc.png)

3. Access the Apache page.

   ![](./figures/HA-apache-show.png)

#### Adding Group Resources

> **Note:**
> Adding group resources requires at least one common resource in the cluster.

1. Click **Add Group Resource**. The **Create Resource** dialog box is displayed.
   All the parameters on the **Basic** tab page are mandatory. After setting the parameters, click **OK** to add the resource or click **Cancel** to cancel the add operation.

   ![](./figures/HA-group.png)

   > **Notes:**
   > Group resources are started in the sequence of child resources. Therefore, you need to select child resources in sequence.

2. If the following information is displayed, the resource is added successfully.

   ![](./figures/HA-group-suc.png)

#### Adding Clone Resources

1. Click **Add Clone Resource**. The **Create Resource** dialog box is displayed.
   On the **Basic** page, enter the object to be cloned. The resource name is automatically generated. After entering the object name, click **OK** to add the resource, or click **Cancel** to cancel the add operation.

   ![](./figures/HA-clone.png)

2. If the following information is displayed, the resource is added successfully.

   ![](./figures/HA-clone-suc.png)

### Editing Resources

* Starting a resource: Select a target resource from the resource node list. The target resource must not be running. Start the resource.
* Stopping a resource: Select a target resource from the resource node list. The target resource must be running. Stop the resource.
* Clearing a resource: Select a target resource from the resource node list. Clear the resource.
* Migrating a resource: Select a target resource from the resource node list. The resource must be a common resource or group resource in the running status. Migrate the resource to migrate it to a specified node.
* Migrating back a resource: Select a target resource from the resource node list. The resource must be a migrated resource. Migrate back the resource to clear the migration settings of the resource and migrate the resource back to the original node. After you click **Migrate Back**, the status change of the resource item in the list is the same as that when the resource is started.
* Deleting a resource: Select a target resource from the resource node list. Delete the resource.

### Setting Resource Relationships

Resource relationships are used to set restrictions for the target resources. There are three types resource restrictions: resource location, resource collaboration, and resource order.

* Resource location: sets the running level of the resource on the nodes in the cluster to determine the node where the resource runs during startup or switchover. The running levels are Master Node and Slave 1 in descending order.
* Resource collaboration: indicates whether the target resource and other resources in the cluster run on the same node. **Same Node** indicates that this node must run on the same node as the target resource. **Mutually Exclusive** indicates that this node cannot run on the same node as the target resource.
* Resource order: Set the order in which the target resource and other resources in the cluster are started. **Front Resource** indicates that this resource must be started before the target resource. **Follow-up Resource** indicates that this resource can be started only after the target resource is started.

## HA MySQL Configuration Example

### Configuring the Virtual IP Address

1. On the home page, choose **Add** > **Add Common Resource**, and set the parameters as follows:

   ![](./figures/HA-vip.png)

2. The resource is successfully created and started, and runs on a node, for example, ha1.

3. The IP address can be pinged and connected. After login, you can perform various operations normally. Resources can be switched to ha2 and can be accessed normally. See the following figure.
   ![](./figures/HA-vip-suc.png)

### Configuring NFS Storage

Perform the following steps to configure another host as the NFS server:

1. Install the software package.

   ```shell
   # yum install -y nfs-utils rpcbind
   ```

2. Disable the firewall.

   ```shell
   # systemctl stop firewalld && systemctl disable firewalld
   ```

3. Modify the /etc/selinux/config file to change the status of SELinux to disabled.

   ```shell
   # SELINUX=disabled
   ```

4. Start services.

   ```shell
   # systemctl start rpcbind && systemctl enable rpcbind
   # systemctl start nfs-server && systemctl enable nfs-server
   ```

5. Create a shared directory on the server.

   ```shell
   # mkdir -p /test
   ```

6. Modify the NFS configuration file.

   ```shell
   # vim /etc/exports
   # /test *(rw,no_root_squash)
   ```

7. Reload the service.

   ```shell
   # systemctl reload nfs
   ```

8. Install the software package on the client. Install MySQL first and then mount NFS to the MySQL data path.

   ```shell
   # yum install -y nfs-utils mariadb-server
   ```

9. On the home page, choose **Add** > **Add Common Resource** and configure the NFS resource as follows:

   ![](./figures/HA-nfs.png)

10. The resource is successfully created and started, and runs on a node, for example, ha1. The NFS is mounted to the `/var/lib/mysql` directory. The resource is switched to ha2. The NFS is unmounted from ha1 and automatically mounted to ha2. See the following figure.

    ![](./figures/HA-nfs-suc.png)

### Configuring MySQL

1. On the home page, choose **Add** > **Add Common Resource** and configure the MySQL resource as follows:

   ![](./figures/HA-mariadb.png)

2. If the following information is displayed, the resource is successfully added:

   ![](./figures/HA-mariadb-suc.png)

### Adding the Preceding Resources as a Group Resource

1. Add the three resources in the resource startup sequence.

   On the home page, choose **Add** > **Add Group Resource** and configure the group resource as follows:

   ![](./figures/HA-group-new.png)

2. The group resource is successfully created and started. If the command output is the same as that of the preceding common resources, the group resource is successfully added.

   ![](./figures/HA-group-new-suc.png)

3. Use ha1 as the standby node and migrate the group resource to the ha2 node. The system is running properly.

   ![](./figures/HA-group-new-suc2.png)

---

---
url: /zh/docs/22.03_LTS_SP4/server/high_availability/ha/ha_usecase_examples.md
---
# HA使用实例

本章介绍如何快速使用HA高可用集群，以及添加一个实例。若不了解怎么安装，请参考《[HA的安装与部署文档](./ha_installation_and_deployment.md)》。

## 快速使用指南

* 以下操作均以社区新开发的管理平台为例。

### 登录页面

用户名为`hacluster`，密码为该用户在主机上设置的密码。

![](./figures/HA-api.png)

### 主页面

登录系统后显示主页面，主页面由四部分组成：侧边导航栏、顶部操作区、资源节点列表区以及节点操作浮动区。

以下将详细介绍这四部分的特点与使用方法。

![](./figures/HA-home-page.png)

#### 导航栏

侧边导航栏由两部分组成：高可用集群软件名称和 logo 以及系统导航。系统导航由三项组成：【系统】、【集群配置】和【工具】。【系统】是默认选项，也是主页面的对应项，主要展示系统中所有资源的相关信息以及操作入口；【集群配置】下设【首选项配置】和【心跳配置】两项；【工具】下设【日志下载】和【集群快捷操作】两项，点击后以弹出框的形式出现。

#### 顶部操作区

登录用户是静态显示，鼠标滑过用户图标，出现操作菜单项，包括【刷新设置】和【退出登录】两项，点击【刷新设置】，弹出【刷新设置】对话框，包含【刷新设置】选项，可以设置系统的自动刷新模式，包括【不自动刷新】、【每 5 秒刷新】和【每 10 秒刷新】三种选择，默认选择【不自动刷新】、【退出登录】即可注销本次登录，系统将自动跳到登录页面，此时，如果希望继续访问系统，则需要重新进行登录。

![](./figures/HA-refresh.png)

#### 资源节点列表区

资源节点列表集中展现系统中所有资源的【资源名】、【状态】、【资源类型】、【服务】、【运行节点】等资源信息，以及系统中所有的节点和节点的运行情况等节点信息。同时提供资源的【添加】、【编辑】、【启动】、【停止】、【清理】、【迁移】、【回迁】、【删除】和【关系】操作。

#### 节点操作浮动区

节点操作浮动区域默认是收起的状态，每当点击资源节点列表表头中的节点时，右侧会弹出节点操作扩展区域，如图所示，该区域由收起按钮、节点名称、停止和备用四个部分组成，提供节点的【停止】和【备用】操作。点击区域左上角的箭头，该区域收起。

### 首选项配置

以下操作均可用命令行配置，现只做简单示例，若想使用更多命令可以使用`pcs --help`进行查询。

```sh
# pcs property set stonith-enabled=false
# pcs property set no-quorum-policy=ignore
```

`pcs property`查看全部设置

![](./figures/HA-firstchoice-cmd.png)

* 点击侧边导航栏中的【首选项配置】按钮，弹出【首选项配置】对话框。将No Quorum Policy和Stonith Enabled由默认状态改为如下对应状态；修改完成后，点击【确定】按钮完成配置。

![](./figures/HA-firstchoice.png)

#### 添加资源

##### 添加普通资源

鼠标点击【添加普通资源】，弹出【创建资源】对话框，其中资源的所有必填配置项均在【基本】页面内，选择【基本】页面内的【资源类型】后会进一步给出该类资源的其他必填配置项以及选填配置项。填写资源配置信息时，对话框右侧会出现灰色文字区域，对当前的配置项进行解释说明。全部必填项配置完毕后，点击【确定】按钮即可创建普通资源，点击【取消】按钮，取消本次添加动作。【实例属性】、【元属性】或者【操作属性】页面中的选填配置项为选填项，不配置不会影响资源的创建过程，可以根据场景需要可选择修改，否则将按照系统默认值处理。

下面以apache为例，添加apache资源

```sh
# pcs resource create httpd ocf:heartbeat:apache
```

查看资源运行状态

```sh
# pcs status
```

![](./figures/HA-pcs-status.png)

* 添加apache资源

![](./figures/HA-add-resource.png)

* 若回显为如下，则资源添加成功

![](./figures/HA-apache-suc.png)

* 资源创建成功并启动，运行于其中一个节点上，例如ha1；成功访问apache界面。

![](./figures/HA-apache-show.png)

##### 添加组资源

添加组资源时，集群中需要至少存在一个普通资源。鼠标点击【添加组资源】，弹出【创建资源】对话框。【基本】页面内均为必填项，填写完毕后，点击【确定】按钮，即可完成资源的添加，点击【取消】按钮，取消本次添加动作。

* **注：组资源的启动是按照子资源的顺序启动的，所以选择子资源时需要注意按照顺序选择。**

![](./figures/HA-group.png)

若回显为如下，则资源添加成功

![](./figures/HA-group-suc.png)

##### 添加克隆资源

鼠标点击【添加克隆资源】，弹出【创建资源】对话框。【基本】页面内填写克隆对象，资源名称会自动生成，填写完毕后，点击【确定】按钮，即可完成资源的添加，点击【取消】按钮，取消本次添加动作。

![](./figures/HA-clone.png)

若回显为如下，则资源添加成功

![](./figures/HA-clone-suc.png)

#### 编辑资源

* 启动资源：资源节点列表中选中一个目标资源，要求：该资源处于非运行状态。对该资源执行启动动作。
* 停止资源：资源节点列表中选中一个目标资源，要求：该资源处于运行状态。对该资源执行停止操作。
* 清理资源：资源节点列表中选中一个目标资源，对该资源执行清理操作。
* 迁移资源：资源节点列表中选中一个目标资源，要求：该资源为处于运行状态的普通资源或者组资源，执行迁移操作可以将资源迁移到指定节点上运行。
* 回迁资源：资源节点列表中选中一个目标资源，要求：该资源已经完成迁移动作，执行回迁操作，可以清除该资源的迁移设置，资源重新迁回到原来的节点上运行。点击按钮后，列表中该资源项的变化状态与启动资源时一致。
* 删除资源：资源节点列表中选中一个目标资源，对该资源执行删除操作。

#### 设置资源关系

资源关系即为目标资源设定限制条件，资源的限制条件分为三种：资源位置、资源协同和资源顺序。

* 资源位置：设置集群中的节点对于该资源的运行级别，由此确定启动或者切换时资源在哪个节点上运行，运行级别按照从高到低的顺序依次为：Master Node、Slave 1。
* 资源协同：设置目标资源与集群中的其他资源是否运行在同一节点上，同节点资源表示该资源与目标资源必须运行在相同节点上，互斥节点资源表示该资源与目标资源不能运行在相同的节点上。
* 资源顺序：设置目标资源与集群中的其他资源启动时的先后顺序，前置资源是指目标资源运行之前，该资源必须已经运行；后置资源是指目标资源运行之后，该资源才能运行。

## 高可用mysql实例配置

* 先单独配置三个普通资源，待成功后添加为组资源。

### 配置虚拟IP

在首页中点击添加-->添加普通资源，并按如下进行配置。

![](./figures/HA-vip.png)

* 资源创建成功并启动，运行于其中一个节点上，例如ha1；可以ping通并连接，登录后可正常执行各种操作；资源切换到ha2运行；能够正常访问。
* 若回显为如下，则资源添加成功。

![](./figures/HA-vip-suc.png)

### 配置NFS存储

* 另外找一台机器作为nfs服务端进行配置。

安装软件包

```sh
# yum install -y nfs-utils rpcbind
```

关闭防火墙

```sh
# systemctl stop firewalld && systemctl disable firewalld
```

修改/etc/selinux/config文件中SELINUX状态为disabled

```sh
# SELINUX=disabled
```

启动服务

```sh
# systemctl start rpcbind && systemctl enable rpcbind
# systemctl start nfs-server && systemctl enable nfs-server
```

服务端创建一个共享目录

```sh
# mkdir -p /test
```

修改NFS配置文件

```sh
# vim /etc/exports
# /test *(rw,no_root_squash)
```

重新加载服务

```sh
# systemctl reload nfs
```

客户端安装软件包，先把mysql安装上，为了把下面nfs挂载到mysql数据路径

```sh
# yum install -y nfs-utils mariadb-server
```

在首页中点击添加-->添加普通资源，并按如下进行配置NFS资源。

![](./figures/HA-nfs.png)

* 资源创建成功并启动，运行于其中一个节点上，例如ha1；nfs成功挂载到/var/lib/mysql路径下。资源切换到ha2运行；nfs从ha1节点取消挂载，并自动在ha2节点上挂载成功。
* 若回显为如下，则资源添加成功。

![](./figures/HA-nfs-suc.png)

### 配置mysql

在首页中点击添加-->添加普通资源，并按如下进行配置mysql资源。

![](./figures/HA-mariadb.png)

* 若回显为如下，则资源添加成功

![](./figures/HA-mariadb-suc.png)

### 添加上述资源为组资源

* 按资源启动顺序添加三个资源

在首页中点击添加-->添加组资源，并按如下进行配置组资源。

![](./figures/HA-group-new.png)

* 组资源创建成功并启动，若回显与上述三个普通资源成功现象一致，则资源添加成功

![](./figures/HA-group-new-suc.png)

* 将ha1节点备用，成功迁移到ha2节点，运行正常

![](./figures/HA-group-new-suc2.png)

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/high_availability/ha/ha_installation_and_deployment.md
---
# HA的安装与部署

本章介绍如何安装和部署HA高可用集群。

## 安装与部署

* 环境准备：需要至少两台安装了openEuler 22.03 LTS SP4的物理机/虚拟机（现以两台为例），安装方法参考《openEuler 22.03 LTS SP4 安装指南》。

### 修改主机名称及/etc/hosts文件

* **注：两台主机均需要进行以下操作，现以其中一台为例。**

在使用HA软件之前，需要确认修改主机名并将所有主机名写入/etc/hosts文件中。

* 修改主机名

```shell
# hostnamectl set-hostname ha1
```

* 编辑`/etc/hosts`文件并写入以下字段

```conf
172.30.30.65 ha1
172.30.30.66 ha2
```

### 配置yum源

成功安装系统后，会默认配置好yum源，文件位置存放在`/etc/yum.repos.d/openEuler.repo`文件中，HA软件包会用到以下源：

```conf
[OS]
name=OS
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler

[everything]
name=everything
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/RPM-GPG-KEY-openEuler

[EPOL]
name=EPOL
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler
```

### 安装HA软件包组件

```shell
# yum install -y corosync pacemaker pcs fence-agents fence-virt corosync-qdevice sbd drbd drbd-utils
```

### 设置hacluster用户密码

```shell
# passwd hacluster
```

### 修改`/etc/corosync/corosync.conf`文件

```conf
totem {
        version: 2
        cluster_name: hacluster
         crypto_cipher: none
        crypto_hash: none
}
logging {         
        fileline: off
        to_stderr: yes
        to_logfile: yes
        logfile: /var/log/cluster/corosync.log
        to_syslog: yes
        debug: on
       logger_subsys {
               subsys: QUORUM
               debug: on
        }
}
quorum {
           provider: corosync_votequorum
           expected_votes: 2
           two_node: 1
       }
nodelist {
       node {
               name: ha1
               nodeid: 1
               ring0_addr: 172.30.30.65
               }
        node {
               name: ha2
               nodeid: 2
               ring0_addr: 172.30.30.66
               }
        }
```

### 管理服务

#### 关闭防火墙

```shell
# systemctl stop firewalld
```

修改/etc/selinux/config文件中SELINUX状态为disabled

```conf
# SELINUX=disabled
```

#### 管理pcs服务

* 启动pcs服务：

```shell
# systemctl start pcsd
```

* 查询pcs服务状态：

```shell
# systemctl status pcsd
```

若回显为如下，则服务启动成功。

![](./figures/HA-pcs.png)

#### 管理pacemaker服务

* 启动pacemaker服务：

```shell
# systemctl start pacemaker
```

* 查询pacemaker服务状态：

```shell
# systemctl status pacemaker
```

若回显为如下，则服务启动成功。

![](./figures/HA-pacemaker.png)

#### 管理corosync服务

* 启动corosync服务：

```shell
# systemctl start corosync
```

* 查询corosync服务状态：

```shell
# systemctl status corosync
```

若回显为如下，则服务启动成功。

![](./figures/HA-corosync.png)

### 节点鉴权

* **注：一个节点上执行即可**

```shell
# pcs host auth ha1 ha2
```

### 访问前端管理平台

上述服务启动成功后，打开浏览器（建议使用：Chrome，Firefox），在浏览器导航栏中输入`https://localhost:2224`即可。

* 此界面为原生管理平台

![](./figures/HA-login.png)

若安装社区新开发的管理平台请参考此文档<https://atomgit.com/openeuler/ha-api/blob/master/docs/build.md>。

* 下面为社区新开发的管理平台

![](./figures/HA-api.png)

* 下一章将介绍如何快速使用HA高可用集群，以及添加一个实例。请参考[HA的使用实例文档](./ha_usecase_examples.md)。

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/syssentry/online_repair_plugin.md
---
# HBM ACLS/SPPR 在线修复插件

## 简介

该插件用于保障业务可靠性，维护业务的连续运行。具体为针对业务运行时的CE故障诊断或预测结果，采用ACLS/SPPR进行故障地址/故障行进行在线修复，避免UCE导致业务中断。整个流程涉及到FW、IMU、BIOS、OS、BMC等多方面的整体能力，该插件为OS端使能插件，主要功能为监听接受ACLS/SPPR消息、持久化保存故障信息与在线修复故障。

## 硬件规格要求

* 仅支持aarch64架构
* 仅支持华为服务器，且服务器中装载HaiyanHBM设备
* FW需要具有将故障行列地址解析为物理地址并通知OS的能力，需要相应固件支持
* IMU/BIOS需要有透传FW下发给OS在线修复请求的能力，需要相应固件支持
* BMC需要有接收在线修复请求并返回接受结果的能力，需要相应固件支持

## 安装插件

### 前置条件

已通过《[安装和使用](./installation_and_usage.md)》安装sysSentry巡检框架。

### 安装软件包

```shell
yum install hbm_online_repair -y
```

## 在线修复插件参数配置

在线修复插件的配置文件默认保存在/etc/sysconfig/hbm\_online\_repair.env中。

* 配置项说明

| 配置项                      | 默认值  | 取值范围           | 必选项 | 含义                                                         |
| --------------------------- | ------- | ------------------ | ------ | ------------------------------------------------------------ |
| HBM\_ONLINE\_REPAIR\_LOG\_LEVEL | 1       | 0~3的整数          | Y      | 在线修复插件日志级别，可设置等级有0（DEBUG）、1（INFO）、2（WARNING）、3（ERROR），当设置为低级别时，高级别等级日志也会打印 |
| PAGE\_ISOLATION\_THRESHOLD    | 3355443 | 0~2147483647的整数 | Y      | 系统隔离资源的阈值，单位为KB。插件的在线修复功能需要先对页面进行隔离，会占用系统的隔离资源，当系统整体的隔离资源超过该阈值时，插件将不会进行隔离以及后续的修复、上线操作 |

* 配置示例

```ini
HBM_ONLINE_REPAIR_LOG_LEVEL=1
PAGE_ISOLATION_THRESHOLD=3355443
```

## 管理在线修复插件

管理插件方式与《[安装和使用](./installation_and_usage.md)》方式基本相同，不在此赘述。

注意，插件在使用`sentryctl get_result <module_name>`方法时，该方法只会返回插件的运行起止时间`"start_time"`与`"end_time"`，其余字段为空，需要通过读取日志的方式获取插件的运行状态及相关信息。

## 查看在线修复插件日志

在线修复插件日志记录在 /var/log/sysSentry/hbm\_online\_repair.log 文件中，日志中主要记录插件的初始配置以及收到消息后的各种处理结果。

### 启动日志

插件启动后，一般会有如下日志：

![输入图片说明](figures/hbm-start-log.png)

### 运行日志

插件在接收到正确的ACLS/SPPR类型的故障上报后，会出现`Received ACLS/SPPR repair request`字样，然后插件按照两个阶段处理信息并执行操作，日志中会打印对应操作的信息，如下图示例：

![输入图片说明](figures/acls-example.png)

#### 阶段一：持久化保存信息

该阶段的日志会出现两种情况：

1. 出现`write hbm fault info to flash xxx success`表示持久化信息保存成功；
2. 出现其余信息，表示持久化信息保存失败，可对照下表查询失败原因：

| 报错信息                                                     | 含义                                      |
| ------------------------------------------------------------ | ----------------------------------------- |
| fault info storage reach threshold, cannot save new record into flash | 持久化信息保存大小超过了阈值（默认128KB） |
| invalid fault info                                           | 解析出的持久化信息格式错误                |
| read variable xx-xx attribute failed, stop writing           | 对应guid位置的efivar属性读取错误          |
| write to xx-xx failed                                        | 持久化信息写入失败                        |

#### 阶段二：Flat模式下尝试进行修复

由BIOS透传至OS的ACLS/SPPR故障上报信息会有两种模式：Flat模式与Cache模式。只有接收到Flat模式的消息，插件才会尝试进行修复操作，因此该阶段会有三种情况的日志出现：

1. 无任何打印，说明消息模式为Cache模式，此时插件只会进行持久化信息保存（阶段一）而不会尝试修复操作。
2. 出现`HBM ACLS/SPPR: Page xxx repair and online success`字样，表示接收到了Flat模式信息且修复成功；
3. 出现其他信息，表示接收到了Flat模式消息并执行操作，在隔离/修复/上线中任一操作失败，可对照下表查询失败原因：

| 报错信息                                                  | 含义                                         |
| --------------------------------------------------------- | -------------------------------------------- |
| Page isolate failed: Get hardware\_corrupted\_size failed   | 无法读取当前系统的隔离资源使用情况，隔离失败 |
| Page isolate failed: the isolation resource is not enough | 系统隔离资源超阈值，隔离失败                 |
| HBM: ACLS/SPPR offline failed, address is xxx             | ACLS/SPPR调用隔离接口，返回失败              |
| Repair driver is not loaded, skip error                   | 未找到修复相关的驱动，修复失败               |
| No HBM device memory type found, skip error               | 没有找到HBM设备，修复失败                    |
| Err addr is not in device, skip error                     | 地址不在任何一个HBM设备中，修复失败          |
| HBM: Address xxx is not supported to ACLS/SPPR repair     | 该地址不支持ACLS/SPPR修复，修复失败          |
| HBM ACLS/SPPR: Page xxx online failed                     | 页面上线失败                                 |

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/distributed/hmdfs_distributed_file_system_overview.md
---
# hmdfs Distributed File System Overview

OpenHarmony distributed file system (hmdfs) distributed file system provides cross-device file access capabilities in the following scenarios:

* When two devices are deployed on a network, device A can transparently read and modify files on device B.

* The edge server can automatically synchronize file data from multiple embedded devices on the network.

The hmdfs provides a globally consistent access view across devices dynamically connected to a network via DSoftBus and allows you to implement high-performance read and write operations on files with low latency by using basic file system APIs.

It consists of the following core modules:

* distributed\_file\_daemon: user-mode daemon for distributed file management, which is responsible for access device networking, data transmission, and hmdfs mounting.

* hmdfs: core module of the distributed filesystem. It is a high-performance, kernel-mode, and layered file system for mobile distributed scenarios.

## Constraints

### Supported Interfaces

Distributed file management does not support or partially supports the following system calls of the Virtual File System (VFS):

* `symlink` is not supported.
* `mmap` supports read only.
* `rename` supports only operations within the same directory.

### Specifications

* Maximum number of directory levels

  The value is the same as the overlaid file system, that is, the file system used by the **data** partition, such as ext4 and F2FS.
* Maximum file name length

  The smaller of 680 bytes and the length supported by the overlaid file system. For F2FS and ext4, the value is 255 bytes.
* Maximum size of a single file

  The smaller of $2^{64}$B and the size supported by the overlaid file system. The value is 16 TB for ext4 and 3.94 TB for F2FS.

### Environment Restrictions

* The name of the wired NIC in the running environment must be **eth0**, and the name of the wireless NIC must be **wlan0**. You can run the `ip a` command to check the NIC name in the current environment. If **eth0** or **wlan0** does not exist, softbus\_server fails to be started and the function is invalid. For details, see [FAQs](#faqs).

* The kernel version of openEuler must be 5.10.x. You can run the `uname -r` command to view the kernel version.

* All openEuler devices are in the same subnet, and the connections between the devices are normal. The firewall does not intercept data packets from DSoftBus.

## Description

### Installation

Note: If a step fails to be performed, rectify the fault by referring to [FAQs](#faqs).

1. Install the **hmdfs** and **filemanagement\_dfs\_service** software packages. Run the following command:

   ```shell
   sudo dnf install hmdfs filemanagement_dfs_service
   ```

2. Install the hmdfs file system. After the **hmdfs** software package is installed, the **hmdfs.ko** file is provided and stored in the **/lib/modules/$(uname -r)/hmdfs** directory. Insert the **.ko** file to install hmdfs.

   ```shell
   cd /lib/modules/$(uname -r)/hmdfs
   insmod hmdfs.ko
   ```

   > **Notice**
   >
   > * If the **hmdfs.ko** file does not exist in the **/lib/modules/$(uname -r)/hmdfs/** directory, the kernel version used for building the hmdfs project is not the same as the kernel version in the current environment. You can search for the **hmdfs.ko** file of other kernel versions in the **/lib/modules** directory.
   > * In the Raspberry Pi environment, use [**hmdfs.ko**](https://gitee.com/heppen/hmdfs_test/blob/dev/out/hmdfs_sp3_rasp.ko) compiled on openEuler 22.03 LTS SP4 raspberry-pi kernel.

### Configuration

1. The startup of some services depends on the dynamic library **libsec\_shared.z.so**, which is named **libboundscheck.so** in openEuler (provided by the **libboundscheck** software package). Therefore, you need to create a soft link to **libsec\_shared.z.so** in **/usr/lib64**.

   ```shell
   ln -s /usr/lib64/libboundscheck.so /usr/lib64/libsec_shared.z.so
   ```

2. Configure the SN of each device. Currently, services such as softbus\_sever use the SN set in the **/etc/SN** file to obtain the UDID of the device. Therefore, you need to set **a unique SN** for each openEuler device.

   ```shell
   echo "111" > /etc/SN # Set different values for different devices.
   ```

### Usage

To use hmdfs, you need to mount the hmdfs directory and start the distributed\_file\_daemon service.

> **Notice**
>
> Perform the following steps on each openEuler device.

#### Mounting the hmdfs Directory

1. Run the `mount` command to mount the hmdfs directory. Ensure that the directory structure is the same as that of OpenHarmony. Mount **/data/service/el2/100/non\_account** to **/mnt/hmdfs/100/non\_account**.

   ```shell
   mkdir -p /data/service/el2/100/non_account
   mkdir -p /mnt/hmdfs/100/non_account
   sudo mount -t hmdfs -o merge,local_dst="/mnt/hmdfs/100/non_account" "/data/service/el2/100//non_account" "/mnt/hmdfs/100/non_account"
   ```

   After the directory is mounted, you can run the `df -h` command to view the mounted directory, which contains the **device\_view** and **merge\_view** directories.

   ```txt
   ├── device_view
   │   └── local
   └── merge_view
   ```

#### Starting the dfs\_service Service

After **filemanagement\_dfs\_service** and its dependent software packages are installed, related executable binary files are stored in **/system/bin/**, and library files are stored in **/system/lib64**.

1. Start the dfs\_service distributed file service:

   ```shell
   cd /system/bin
   ./start_services.sh dfs
   ```

2. Stop the dfs\_service distributed file service:

   ```shell
   cd /system/bin
   ./stop_service.sh dfs
   ```

### Function Usage

After distributed\_file\_daemon is started on each openEuler device, you can view the directories of the remote devices in **/mnt/hmdfs/100/non\_account**. In this example, only two openEuler devices are connected.

```txt
├── device_view
│   ├── fceda1e26c36d1dd0ba65c00d71c1ab619fcf088ad2adf33cd1e2f396dc70ee2
│   └── local
└── merge_view
```

There are two file views in the directory: **device\_view**, which contains **local** file view and remote file view; and **merge\_view**, which is the merged file view, containing files of multiple devices.

If you need to perform cross-device file operations, simply perform operations on the files in the remote device directory in **device\_view**.

## FAQs

* When a service is started, the error message "Binder Driver died" is displayed.

  Cause: Binder is not enabled in the system. You can check whether the **/dev/binder** file exists. If the file does not exist, Binder is not enabled.

  Solution: Start Binder by referring to the [communication\_ipc repository README](https://atomgit.com/src-openeuler/communication_ipc/blob/openEuler-22.03-LTS-SP4/README.md).

* The **hmdfs.ko** file cannot be inserted, and the error "insmod: ERROR: could not insert module hmdfs.ko: Invalid parameters" is reported.

  Cause: The kernel used for compiling hmdfs is different from that in the current environment.

  Solution 1: Compile a **hmdfs.ko** file that matches the kernel of the current environment, and then insert the **.ko** file.

  Solution 2: Use an openEuler version that has the same kernel as openEuler 22.03 LTS SP4.

* In the openEuler 22.03 LTS SP4 Raspberry Pi version, the **hmdfs.ko** file cannot be inserted and the error "insmod: ERROR: could not insert module hmdfs.ko: Invalid module format" is reported.

  Cause: The Raspberry Pi version uses a special kernel. However, the hmdfs software in the repository is built using the kernel of the openEuler 22.03 LTS SP4 server version. Therefore, the **.ko** file cannot be inserted.

  Solution: Use [**hmdfs.ko**](https://gitee.com/heppen/hmdfs_test/blob/dev/out/hmdfs_sp3_rasp.ko) compiled on openEuler 22.03 LTS SP4 raspberry-pi kernel.

* The softbus\_server service fails to be started, and the error message "GetNetworkIfIp ifName:eth0 fail" is displayed.

  Cause: Run the ip a command to view the name of the NIC in the current system and check whether the wired NIC **eth0** exists. The softbus\_server service obtains information such as the IP address through the wired NIC **eth0**. If **eth0** does not exist, softbus\_server cannot be started.

  Solution 1: Change the NIC name to **eth0**.

  Solution 2: Modify the softbus\_server source code and change the name of the dependent wired NIC to that of the NIC in the current system.

* After the softbus\_server service is started on multiple openEuler devices, the distributed\_file\_daemon service logs show that no online device is found.

  Cause: The network between devices is disconnected, or the firewall blocks DSoftBus data.

  Solution: Check whether the network is normal. (You can run `systemctl stop firewalld.service` to temporarily disable the firewall and test the network if services will not be affected.)

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/distributed/hmdfs_distributed_file_system_overview.md
---
# hmdfs 分布式文件系统概述

分布式文件系统提供跨设备的文件访问能力，适用于如下场景：

* 两台设备组网，A 设备可以无感读取和修改 B 设备的文件。

* 边缘服务器可以自动同步组网中多个嵌入式设备中的文件数据。

hmdfs 在分布式软总线动态组网的基础上，为网络上各个设备结点提供一个全局一致的访问视图，支持开发者通过基础文件系统接口进行读写访问，具有高性能、低延时等优点。

其包括如下几个核心模块:

* distributed\_file\_daemon：分布式文件管理常驻用户态服务，负责接入设备组网、数据传输能力，并负责挂载 hmdfs。

* hmdfs(Harmony Distributed File System)：分布式文件系统核心模块，是一种面向移动分布式场景的、高性能的、基于内核实现的、堆叠式文件系统。

## 约束

### 接口支持情况

分布式文件管理当前不支持或有限支持如下 VFS 系统调用：

* symlink：不支持。
* mmap：仅支持读。
* rename：仅支持同目录操作。

### 规格

* 最大目录层级

  与被堆叠文件系统，即 data 分区所用文件系统，如 ext4、f2fs 等保持一致。
* 最大文件名长度

  取决于 680B 与被堆叠文件支持长度的最小值。f2fs 和 ext4 均为 255B。
* 最大单文件大小

  取决于 $2^{64}$B 与被堆叠文件系统支持最大单文件大小的最小值。ext4 单文件最大为 16TB，f2fs 单文件最大为 3.94TB。

### 环境约束

* 运行环境的有线网卡名称必须是 `eth0` ，无线网卡的名称必须是 `wlan0` 。可使用 `ip a` 命令查看当前环境的网卡名称，如果没有 `eth0` 或者 `wlan0` 的网卡，那么 softbus\_server 会启动失败，功能失效。解决方案参考 [常见问题](#常见问题)。

* openEuler 的内核版本需要是 5.10.x，可以通过使用 `uname -r` 查看内核版本。

* openEuler 各个设备在同一个网段中，并且设备之间网络通畅，防火墙未拦截 softbus 的数据包。

## 说明

### 安装

说明：如果碰到步骤未成功执行，可参考后面 [常见问题](#常见问题) 进行解决。

1. 完整地使用分布式文件系统，需要安装 `hmdfs` 和 `filemanagement_dfs_service` 两个软件包。使用以下命令安装：

   ```shell
   sudo dnf install hmdfs filemanagement_dfs_service
   ```

2. 安装 hmdfs 文件系统。安装 hmdfs 之后会提供一个 `hmdfs.ko``文件，其存放在`/lib/modules/$(uname -r)/hmdfs\` 目录下，需要插入该 ko 来安装 hmdfs 文件系统：

   ```shell
   cd /lib/modules/$(uname -r)/hmdfs
   insmod hmdfs.ko
   ```

   > **注意**
   >
   > * 如果`/lib/modules/$(uname -r)/hmdfs/`目录下没有`hmdfs.ko`文件，是因为 hmdfs 工程构建时依赖的内核版本和当前运行环境内核版本不一致，可以在`/lib/modules`其他内核版本目录下查找`hmdfs.ko`文件。
   > * 如果是树莓派环境，请直接使用 [基于 22.03-LTS-SP4 raspberry-pi kernel 编译出来的 hmdfs.ko](https://gitee.com/heppen/hmdfs_test/blob/dev/out/hmdfs_sp3_rasp.ko)。

### 配置

1. 后续有服务启动依赖 `libsec_shared.z.so` 这个动态库，而在 openEuler 下这个动态库叫做 `libboundscheck.so`（由 `libboundscheck` 软件包提供），因此需要在 `/usr/lib64` 下软链接出一个 `libsec_shared.z.so`：

   ```shell
   ln -s /usr/lib64/libboundscheck.so /usr/lib64/libsec_shared.z.so
   ```

2. 配置每个设备的 SN 号。目前 `softbus_sever` 等服务获取设备的 `udid` 还是使用 `/etc/SN` 文件中设置的 SN 号，因此需要在每台 openEuler 设置 **不同的 SN 号**。

   ```shell
   echo "111" > /etc/SN # 注意：不同设备设置不同的数值
   ```

### 使用

分布式文件系统的使用分为两块：`挂载 hmdfs 目录` 和 `启动 distributed_file_daemon 服务`。

> **注意**
>
> 以下步骤需要在每台 openEuler 设备下执行。

#### 挂载 hmdfs 目录

1. 挂载 hmdfs 目录，可以直接使用 `mount` 命令进行挂载，保持 OpenHarmony 的目录结构一样，挂载 `/data/service/el2/100/non_account` 到 `/mnt/hmdfs/100/non_account`。

   ```shell
   mkdir -p /data/service/el2/100/non_account
   mkdir -p /mnt/hmdfs/100/non_account
   sudo mount -t hmdfs -o merge,local_dst="/mnt/hmdfs/100/non_account" "/data/service/el2/100//non_account" "/mnt/hmdfs/100/non_account"
   ```

   挂载之后，可以使用 `df -h` 命令查看新增了挂载的目录，并且挂载目录下会有 `device_view` 和 `merge_view` 两个目录。

   ```txt
   ├── device_view
   │   └── local
   └── merge_view
   ```

#### 启动 dfs\_service 服务

安装 `filemanagement_dfs_service` 及其依赖的软件包后，相关的可执行二进制会存放在 `/system/bin/` 目录下，库文件会存放在 `/system/lib64` 下。

1. 启动 `dfs_service` 分布式文件服务：

   ```shell
   cd /system/bin
   ./start_services.sh dfs
   ```

2. 停止 `dfs_service` 分布式文件服务：

   ```shell
   cd /system/bin
   ./stop_service.sh dfs
   ```

### 功能使用

每台 openEuler 设备启动完 distributed\_file\_daemon 之后，可以在挂载的 `/mnt/hmdfs/100/non_account` 下看到远端设备的目录（示例是只有两台 openEuler 设备互联）：

```txt
├── device_view
│   ├── fceda1e26c36d1dd0ba65c00d71c1ab619fcf088ad2adf33cd1e2f396dc70ee2
│   └── local
└── merge_view
```

⽬录下会有两个⽂件视图：device\_view 是分设备的视图，local 是本地⽂件视图，另外⼀个是远端设备的⽂件视图；merge\_view 是合并视图，多个设备的⽂件都在这⼀个⽬录。

后续需要跨设备进行⽂件操作，只需要操作 device\_view 下⾯远端设备⽬录下的⽂件即可。

## 常见问题

* 启动各个服务日志一直在报错 `Binder Driver died`。

  原因：说明系统未开启 binder，可以查看 `/dev/binder` 文件是否存在，如果不存在则说明未开启 binder。

  解决办法：参考 [communication\_ipc 仓 openEuler-22.03-LTS-SP4 分支的 README](https://atomgit.com/src-openeuler/communication_ipc/blob/openEuler-22.03-LTS-SP4/README.md) 开启 binder。

* 无法插入 `hmdfs.ko` 文件，报错 `insmod: ERROR: could not insert module hmdfs.ko: Invalid parameters`。

  原因：hmdfs 编译时依赖的 kernel 版本和现在运行环境不一致或者当前系统未开启。

  解决方法一：编译一个和运行环境内核匹配的 `hmdfs.ko`，然后插入该 ko 文件使用。

  解决方法二：更换和 SP4 的 kernel 版本一致的 openEuler 版本。

* 树莓派 22.03-LTS-SP4 openEuler版本下，无法插入 `hmdfs.ko` 文件，报错 `insmod: ERROR: could not insert module hmdfs.ko: Invalid module format`.

  原因：树莓派使用的 kernel 版本是树莓派特有版本，而 repo 源中 hmdfs 软件在工程构建时依赖的是服务器 22.03-LTS-SP4 版本的内核，所以无法插入。

  解决方法：使用 [基于 22.03-LTS-SP4 raspberry-pi kernel 编译出来的 hmdfs.ko](https://gitee.com/heppen/hmdfs_test/blob/dev/out/hmdfs_sp3_rasp.ko)。

* `softbus_server` 服务未成功起来，报错 `GetNetworkIfIp ifName:eth0 fail`。

  原因：使用命令 `ip a` 查看当前系统的网卡名称，查看是否有 `eth0` 有线网卡名。因为 `softbus_server` 是通过 `eth0` 这个有线网卡名来获取 ip 等信息，如果没有 `eth0` 网卡则无法启动 `softbus_server`。

  解决方法一：修改网卡名称为 `eth0`。

  解决方法二：修改 `softbus_server` 源码，将依赖的有线网卡名称改成当前系统的网卡名。

* 多台 openEuler 设备拉起 `softbus_server` 服务之后，但是在 `distributed_file_daemon` 服务的日志显示未发现上线设备。

  原因：设备之间的网络不通，或者开启了防火墙把 `softbus` 的数据拦截了。

  解决：检查网络是否通畅。（如果不影响业务，可通过 `systemctl stop firewalld.service` 暂时关闭防火墙进行测试）。

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/kernel_live_upgrade/usage_guide.md
---
# How to Run

## Command

* `nvwa help`

  Prints the help information. The printed information is as follows:

  ```text
  NAME:
  nvwa - a tool used for openEuler kernel update.

  USAGE:
  nvwa [global options] command [command options] [arguments...]

  VERSION:
  0.1

  COMMANDS:
  update   specify kernel version for nvwa to update
  init     init nvwa running environment
  help, h  Shows a list of commands or help for one command

  GLOBAL OPTIONS:
  --help, -h     show help (default: false)
  --version, -v  print the version (default: false)
  ```

* `nvwa update <kernel version>`

  When the kernel is live upgraded to a version, the NVWA searches for the kernel image and ramfs in the /boot directory. The kernel must be named in the **vmlinuz-\<kernel version>** format, and rootfs in the **initramfs-\<kernel version>.img** format.

  Note that the upgrade may fail. If the upgrade fails, some processes or services that are dumped will stop running.

* `nvwa init`

  Clears the running information generated by NVWA and modifies the systemd configuration. This command is used to clear the running information before the NVWA is executed or after the execution fails.

## Restrictions

1. For services that need to be saved using NVWA, you need to set StandardOutput and StandardError in the configuration. The following uses Redis as an example:

   ```text
   [Unit]
   Description=Redis persistent key-value database
   After=network.target
   [Service]
   ExecStart=/usr/bin/redis-server /etc/redis.conf --supervised systemd
   Type=notify
   User=redis
   Group=redis
   RuntimeDirectory=redis
   RuntimeDirectoryMode=0755
   StandardOutput=file:/root/log1.log
   StandardError=file:/root/log2.log
   [Install]
   WantedBy=multi-user.target
   ```

2. To use the acceleration feature, you need to modify the cmdline and allocate proper memory. For details, see [NVWA Acceleration Feature Description and Usage](#nvwa-acceleration-feature-description-and-usage).

3. SELINUX needs to be disabled during the running process.

   Theoretically, you need to disable the NVWA service only after you run the NVWA update command and before you restart the system to restore the process. It is recommended that SELinux be disabled during the entire process.

## NVWA Acceleration Feature Description and Usage

1. cpu park

   The cpu park command uses the kexec process to make the CPU stay busy waiting, so as to respond to the interrupt request sent by the primary core more quickly, and reduce the status changes.

   To use cpu park, you need to add "cpuparkmem=0x200000000" to cmdline. 0x200000000 is the start address of the memory that is not used by other programs. cpuparkmem occupies the memory space whose size is about 1 MB from this address.

   Note that if the memory is sufficient, it is recommended that the address range be after 4G(0x100000000). The first 4 GB is usually reserved by each system component, which is prone to conflict.

2. quick kexec

   quick kexec accelerates image loading using kexec.

   To use quick kexec, you need to enable related options in the configuration file. For more information, see "Configuration" in Installation and Deployment.

3. pin\_memory

   pin memory accelerates the storage and recovery of the CRIU.

   To use pin memory, you need to enable related options in the configuration file. For more information, see "Configuration" in Installation and Deployment.

## Generated Log Information

The logs generated by the kernel live upgrade tool consist of two parts:

* Logs generated during running

  Run the service `nvwa status` command to view logs.

* Logs generated while retaining the running information

  The logs are stored in the process/service folder in the path specified by **criu\_dir**.

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/i3/i3_user_guide.md
---
# i3 in openEuler User Guide

## What Is i3?

[i3](https://i3wm.org/) is a [tiling window manager](https://en.wikipedia.org/wiki/Tiling_window_manager).

The interface of i3 is as follows:

![i3 layout](layout.jpeg)

## Usage

1. i3 is based on the X Window protocol. Therefore, you need to install X Server first.

   ```bash
   dnf in xorg-x11-server
   ```

2. As i3 is only a window manager, it does not contain the [components](https://wiki.archlinux.org/title/desktop_environment#Custom_environments) required in a complete Linux desktop environment. You need to install some basic components.

   ```bash
   dnf in xorg-x11-drv-*  lightdm lightdm-gtk
   ```

3. Install i3 components.

   ```bash
   dnf in i3 i3status i3blocks i3lock i3blocks-contrib \
       xfce4-terminal xcompmgr acpi dmenu
   ```

4. Start lightdm after the installation is complete.

   ```bash
   sudo systemctl start lightdm
   ```

   ![](lightdm.png)

   After the session manager is displayed, enter the user name and password to log in to the i3 desktop.

### Basic Operations in i3

(**Mod** is usually mapped to the Windows key on a Windows-compatible keyboard.)

* **Mod**+**d**: Open dmenu for quickly starting processes.
* **Mod**+**Enter**: Open a terminal.
* **Mod**+**↑**/**↓**/**←**/**→**: Move focus between windows.
* **Mod**+**Shift**+**q**: Close the window of focus.
* **Mod**+**Shift**+**r**: Hot load the configuration file.
* **Mod**+**Shift**+**e**: Exit i3.
* **Mod**+**Shift**+**l**: Lock the screen.

For more operation guides, see [i3 Documentation](https://i3wm.org/docs/).

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/i3/i3_user_guide.md
---
# i3 in openEuler 用户指南

## 什么是i3

[i3wm](https://i3wm.org/)简称`i3`，是一个[平铺式窗口管理器](https://en.wikipedia.org/wiki/Tiling_window_manager)。

最终效果如下：

![使用效果](layout.jpeg)

## 如何使用

1. `i3`是基于`X Window`协议，因此需要先安装`X Server`。

   ```bash
   dnf in xorg-x11-server
   ```

2. 同时 i3 也仅仅是一个`window manager`，一个完整的 `linux桌面环境（desktop environment）`通常会包含[非常多的组件](https://wiki.archlinux.org/title/desktop_environment#Custom_environments)，因此还需要安装一些基本组件。

   ```bash
   dnf in xorg-x11-drv-*  lightdm lightdm-gtk
   ```

3. 最后，我们安装`i3`相关的组件。

   ```bash
   dnf in i3 i3status i3blocks i3lock i3blocks-contrib \
       xfce4-terminal xcompmgr acpi dmenu
   ```

4. 安装完成后，启动`lightdm`。

   ```bash
   sudo systemctl start lightdm
   ```

   ![](lightdm.png)

   在看到会话管理器后，输入相应的用户名密码登录即可看到i3下的桌面了。

### i3 的基本操作指令（下面的 mod 在通常的 PC 上是 win 键）

```bash
mod+d：调出 dmenu，用于快速启动进程。
mod+enter：启动 terminal。
mod+↑/mod+↓/mod+←/mod+→：调整 focus 的窗口。
mod+shift+q：关闭当前 focus 的窗口。
mod+shift+r：热加载配置文件。
mod+shift+e：关闭 i3。
mod+shift+l：锁屏。
```

更多的操作指导可以参考[官方文档](https://i3wm.org/docs/)。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/image_management_1.md
---
# Image Management

## Creating an Image

You can use the  **docker pull**,  **docker build**, **docker commit**,  **docker import**, or  **docker load**  command to create an image. For details about how to use these commands, see Image Management.

### Precautions

1. Do not concurrently run the  **docker load**  and  **docker rmi**  commands. If both of the following conditions are met, concurrency problems may occur:

   * An image exists in the system.
   * The docker rmi and docker load operations are concurrently performed on an image.

   Therefore, avoid this scenario. (All concurrent operations between the image creation operations such as running the  **tag**,  **build**, and  **load**, and  **rmi**  commands, may cause similar errors. Therefore, do not concurrently perform these operations with  **rmi**.)

2. If the system is powered off when docker operates an image, the image may be damaged. In this case, you need to manually restore the image.

   When the docker operates images (using the  **pull**,  **load**,  **rmi**,  **build**,  **combine**,  **commit**, or  **import**  commands), image data operations are asynchronous, and image metadata is synchronous. Therefore, if the system power is off when not all image data is updated to the disk, the image data may be inconsistent with the metadata. Users can view images (possibly none images), but cannot start containers, or the started containers are abnormal. In this case, run the  **docker rmi**  command to delete the image and perform the previous operations again. The system can be recovered.

3. Do not store a large number of images on nodes in the production environment. Delete unnecessary images in time.

   If the number of images is too large, the execution of commands such as  **docker image**  is slow. As a result, the execution of commands such as  **docker build**  or  **docker commit**  fails, and the memory may be stacked. In the production environment, delete unnecessary images and intermediate process images in time.

4. When the  **--no-parent**  parameter is used to build images, if multiple build operations are performed at the same time and the FROM images in the Dockerfile are the same, residual images may exist. There are two cases:
   * If FROM images are incomplete, the images generated when images of FROM are running may remain. Names of the residual images are similar to  **base\_v1.0.0-app\_v2.0.0**, or they are none images.
   * If the first several instructions in the Dockerfile are the same, none images may remain.

### None Image May Be Generated

1. A none image is the top-level image without a tag. For example, the image ID of  **ubuntu**  has only one tag  **ubuntu**. If the tag is not used but the image ID is still available, the image ID becomes a none image.
2. An image is protected because the image data needs to be exported during image saving. However, if a deletion operation is performed, the image may be successfully untagged and the image ID may fail to be deleted (because the image is protected). As a result, the image becomes a none image.
3. If the system is powered off when you run the  **docker pull**  command or the system is in panic, a none image may be generated. To ensure image integrity, you can run the  **docker rmi**  command to delete the image and then restart it.
4. If you run the  **docker save**  command to save an image and specify the image ID as the image name, the loaded image does not have a tag and the image name is  **none**.

### A Low Probability That Image Fails to Be Built If the Image Is Deleted When Being Built

Currently, the image build process is protected by reference counting. After an image is built, reference counting of the image is increased by 1 (holdon operation). Once the holdon operation is successful, the image will not be deleted. However, there is a low probability that before the holdon operation is performed, the image can still be deleted, causing the image build failure.

## Viewing Images

Run the following command to view the local image list:

```shell
docker images
```

## Deleting Images

Run the following command to remove an image (**image** indicates the actual image name).

```shell
docker rmi image
```

### Precautions

Do not run the  **docker rmi -f** *XXX*  command to delete images. If you forcibly delete an image, the  **docker rmi**  command ignores errors during the process, which may cause residual metadata of containers or images. If you delete an image in common mode and an error occurs during the deletion process, the deletion fails and no metadata remains.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/image_management_2.md
---
# Image Management

## build

Syntax:  **docker build \[***options***]** *path* **|** *URL* **| -**

Function: Builds an image using the Dockerfile in the specified path.

Parameter description: Common parameters are as follows. For details about more parameters, see the  **docker help build**  command section.

**Table  1**  Parameter description

**Dockerfile Command**

Dockerfile is used to describe how to build an image and automatically build a container. The format of all  **Dockerfile**  commands is  *instruction* *arguments*.

**FROM Command**

Syntax:  **FROM** *image*  or  **FROM** *image*:*tag*

Function: Specifies a basic image, which is the first command for all Dockerfile files. If the tag of a basic image is not specified, the default tag name  **latest**  is used.

**RUN Command**

Syntax:  **RUN** *command*  (for example,  **run in a shell - \`/bin/sh -c\`**) or

**RUN \[***executable*,  *param1*,  *param2*  ...  **]**  (in the  **exec**  command format)

Function: Runs any command in the image specified by the  **FROM**  command and then commits the result. The committed image can be used in later commands. The  **RUN**  command is equivalent to:

**docker run** *image* *command*

**docker commit** *container\_id*

**Remarks**

The number sign (#) is used to comment out.

**MAINTAINER Command**

Syntax:  **MAINTAINER***name*

Function: Specifies the name and contact information of the maintenance personnel.

**ENTRYPOINT Command**

Syntax:  **ENTRYPOINT cmd ***param1 param2...*  or  **ENTRYPOINT \[***"cmd", "param1", "param2"...***]**

Function: Configures the command to be executed during container startup.

**USER Command**

Syntax:  **USER** *name*

Function: Specifies the running user of memcached.

**EXPOSE Command**

Syntax:  **EXPOSE ***port*** \[***port***...]**

Function: Enables one or more ports for images.

**ENV Command**

Syntax:  **ENV** *key value*

Function: Configures environment variables. After the environment variables are configured, they can be used by subsequent **RUN** commands.

**ADD Command**

Syntax:  **ADD** *src dst*

Function: Copies a file from the  *src*  directory to the  *dest*  directory of a container.  *src*  indicates the relative path of the source directory to be built. It can be the path of a file or directory, or a remote file URL.  *dest*  indicates the absolute path of the container.

**VOLUME Command**

Syntax:  **VOLUME \["***mountpoint***"]**

Function: Creates a mount point for sharing a directory.

**WORKDIR Command**

Syntax:  **workdir** *path*

Function: Runs the  **RUN**,  **CMD**, and  **ENTRYPOINT**  commands to set the current working path. The current working path can be set multiple times. If the current working path is a relative path, it is relative to the previous  **WORKDIR**  command.

**CMD command**

Syntax:  **CMD \[***"executable","param1","param2"***]**  (This command is similar to the  **exec**  command and is preferred.)

**CMD \["***param1***","***param2***"]**  (The parameters are the default parameters for ENTRYPOINT.)

**CMD** *command* *param1* *param2*  (This command is similar to the  **shell**  command.)

Function: A Dockerfile can contain only one CMD command. If there are multiple CMD commands, only the last one takes effect.

**ONBUILD Commands**

Syntax:  **ONBUILD \[***other commands***]**

Function: This command is followed by other commands, such as the  **RUN**  and  **COPY**  commands. This command is not executed during image build and is executed only when the current image is used as the basic image to build the next-level image.

The following is a complete example of the Dockerfile command that builds an image with the sshd service installed.

Example:

1. Run the following command to build an image using the preceding Dockerfile:

   ```shell
   sudo docker build -t busybox:latest
   ```

2. Run the following command to view the generated image:

   ```shell
   docker images | grep busybox
   ```

## history

Syntax:  **docker history \[***options***]** *image*

Function: Displays the change history of an image.

Parameter description:

-H, --human=true

**--no-trunc=false**: Does not delete any output.

**-q**  and  **--quiet=false**: Display only IDs.

Example:

```shell
$ sudo docker history busybox:test
IMAGE               CREATED             CREATED BY          SIZE                COMMENT
be4672959e8b        15 minutes ago      bash                23B
21970dfada48        4 weeks ago                             128MB               Imported from -
```

## images

Syntax:  **docker images \[***options***] \[***name***]**

Function: Lists existing images. The intermediate image is not displayed if no parameter is configured.

Parameter description:

**-a**  and  **--all=false**: Display all images.

**-f**  and  **--filter=\[]**: Specify a filtering value, for example,  **dangling=true**.

**--no-trunc=false**: Does not delete any output.

**-q**  and  **--quiet=false**: Display only IDs.

Example:

```shell
$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             latest              e02e811dd08f        2 years ago         1.09MB
```

## import

Syntax:  **docker import URL|- \[***repository***\[***:tag***]]**

Function: Imports a .tar package that contains rootfs as an image. This parameter corresponds to the  **docker export**  command.

Parameter description: none.

Example:

Run the following command to generate a new image for  **busybox.tar**  exported using the  **docker export**  command:

```shell
$ sudo docker import busybox.tar busybox:test
sha256:a79d8ae1240388fd3f6c49697733c8bac4d87283920defc51fb0fe4469e30a4f
$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             test                a79d8ae12403        2 seconds ago       1.3MB
```

## load

Syntax:  **docker load \[***options***]**

Function: Reloads an image from .tar package obtained by running the  **docker save**  command. This parameter corresponds to the  **docker save**  command.

Parameter description:

**-i**  and  **--input=""**  can be used.

Example:

```shell
$ sudo docker load -i busybox.tar
Loaded image ID: sha256:e02e811dd08fd49e7f6032625495118e63f597eb150403d02e3238af1df240ba
$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             latest              e02e811dd08f        2 years ago         1.09MB
```

## login

Syntax:  **docker login \[***options***] \[***server***]**

Function: Logs in to an image server. If no server is specified, the system logs in to  **<https://index.docker.io/v1/>**  by default.

Parameter description:

**-e**  and  **--email=""**: Email address.

**-p**  and  **--password=""**: Password.

**-u**  and  **--username=""**: User name.

Example:

```shell
sudo docker login
```

## logout

Syntax:  **docker logout \[***server***]**

Function: Logs out of an image server. If no server is specified, the system logs out of  **<https://index.docker.io/v1/>**  by default.

Parameter description: none.

Example:

```shell
sudo docker logout
```

## pull

Syntax:  **docker pull \[***options***]** *name***\[***:tag***]**

Function: Pulls an image from an official or private registry.

Parameter description:

**-a**  and  **--all-tags=false**: Download all images in a registry. (A registry can be tagged with multiple tags. For example, a busybox registry may have multiple tags, such as  **busybox:14.04**,  **busybox:13.10**,  **busybox:latest**. If  **-a**  is used, all busybox images with tags are pulled.)

Example:

1. Run the following command to obtain the Nginx image from the official registry:

   ```shell
   $ sudo docker pull nginx
   Using default tag: latest
   latest: Pulling from official/nginx
   94ed0c431eb5: Pull complete
   9406c100a1c3: Pull complete
   aa74daafd50c: Pull complete
   Digest: sha256:788fa27763db6d69ad3444e8ba72f947df9e7e163bad7c1f5614f8fd27a311c3
   Status: Downloaded newer image for nginx:latest
   ```

   When an image is pulled, the system checks whether the dependent layer exists. If yes, the local layer is used.

2. Pull an image from a private registry.

   Run the following command to pull the Fedora image from the private registry, for example, the address of the private registry is  **192.168.1.110:5000**:

   ```shell
   sudo docker pull 192.168.1.110:5000/fedora
   ```

## push

Syntax:  **docker push** *name***\[***:tag***]**

Function: Pushes an image to the image registry.

Parameter description: none.

Example:

1. Run the following command to push an image to the private image registry at 192.168.1.110:5000.

2. Label the image to be pushed. (The  **docker tag**  command is described in the following section.) In this example, the image to be pushed is busybox:sshd.

   ```shell
   sudo docker tag ubuntu:sshd 192.168.1.110:5000/busybox:sshd
   ```

3. Run the following command to push the tagged image to the private image registry:

   ```shell
   sudo docker push 192.168.1.110:5000/busybox:sshd
   ```

   During the push, the system automatically checks whether the dependent layer exists in the image registry. If yes, the layer is skipped.

## rmi

Syntax:  **docker rmi \[***options***] ***image***\[***image...***]**

Function: Deletes one or more images. If an image has multiple tags in the image library, only the untag operation is performed when the image is deleted. If the image has only one tag, the dependent layers are deleted in sequence.

Parameter description:

**-f**  and  **--force=false**: Forcibly delete an image.

**--no-prune=false**: Does not delete parent images without tags.

Example:

```shell
sudo docker rmi 192.168.1.110:5000/busybox:sshd
```

## save

Syntax:  **docker save \[***options***] **\_image \_**\[***image...***]**

Function: Saves an image to a TAR package. The output is  **STDOUT**  by default.

Parameter description:

**-o**  and  **--output=""**: Save an image to a file rather than STDOUT.

Example:

```shell
$ sudo docker save -o nginx.tar nginx:latest
$ ls
nginx.tar
```

## search

Syntax:  **docker search***options* *TERM*

Function: Searches for a specific image in the image registry.

Parameter description:

**--automated=false**: Displays the automatically built image.

**--no-trunc=false**: Does not delete any output.

**-s**  and  **--stars=0**: Display only images of a specified star level or higher.

Example:

1. Run the following command to search for Nginx in the official image library:

   ```shell
   $ sudo docker search nginx
   NAME                              DESCRIPTION                                     STARS               OFFICIAL            AUTOMATED
   nginx                             Official build of Nginx.                        11873               [OK]
   jwilder/nginx-proxy               Automated Nginx reverse proxy for docker con...   1645                                    [OK]
   richarvey/nginx-php-fpm           Container running Nginx + PHP-FPM capable of...   739                                     [OK]
   linuxserver/nginx                 An Nginx container, brought to you by LinuxS...   74
   bitnami/nginx                     Bitnami nginx Docker Image                      70                                      [OK]
   tiangolo/nginx-rtmp               Docker image with Nginx using the nginx-rtmp...   51                                      [OK]
   ```

2. Run the following command to search for busybox in the private image library. The address of the private image library must be added during the search.

   ```shell
   sudo docker search 192.168.1.110:5000/busybox
   ```

## tag

Syntax:  **docker tag \[***options***] ***image***\[***:tag***] \[***registry host/***]\[***username/***]***name***\[***:tag***]**

Function: Tags an image to a registry.

Parameter description:

**-f**  or  **--force=false**: Forcibly replaces the original image when the same tag name exists.

Example:

```shell
sudo docker tag busybox:latest busybox:test
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/image_management.md
---
# Image Management

## Container Image Management

### Logging In to a Registry

#### Description

The  **isula login**  command is run to log in to a registry. After successful login, you can run the  **isula pull**  command to pull images from the registry. If the registry does not require a password, you do not need to run this command before pulling images.

#### Usage

```shell
isula login [OPTIONS] SERVER
```

#### Parameters

For details about the parameters in the **login** command, see **Appendix** > **Command Line Parameters** > **Table 1 login command parameters**.

#### Example

```shell
$ isula login -u abc my.csp-edge.com:5000

Login Succeeded
```

### Logging Out of a Registry

#### Description

The  **isula logout**  command is run to log out of a registry. If you run the  **isula pull**  command to pull images from the registry after logging out of the system, the image will fail to be pulled because you are not authenticated.

#### Usage

```shell
isula logout SERVER
```

#### Parameters

For details about the parameters in the **logout** command, see **Appendix** > **Command Line Parameters** > **Table 2 logout command parameters**.

#### Example

```shell
$ isula logout my.csp-edge.com:5000
Logout Succeeded
```

### Pulling Images from a Registry

#### Description

Pull images from a registry to the local host.

#### Usage

```shell
isula pull [OPTIONS] NAME[:TAG]
```

#### Parameters

For details about the parameters in the **pull** command, see **Appendix** > **Command Line Parameters** > **Table 3 pull command parameters**.

#### Example

```shell
$ isula pull localhost:5000/official/busybox
Image "localhost:5000/official/busybox" pulling
Image "localhost:5000/official/busybox@sha256:bf510723d2cd2d4e3f5ce7e93bf1e52c8fd76831995ac3bd3f90ecc866643aff" pulled
```

### Deleting Images

#### Description

Delete one or more images.

#### Usage

```shell
isula rmi [OPTIONS] IMAGE [IMAGE...]
```

#### Parameters

For details about the parameters in the **rmi** command, see **Appendix** > **Command Line Parameters** > **Table 4 rmi command parameters**.

#### Example

```shell
$ isula rmi rnd-dockerhub.huawei.com/official/busybox
Image "rnd-dockerhub.huawei.com/official/busybox" removed
```

### Adding an Image Tag

#### Description

Add an image tag.

#### Usage

```shell
isula tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]
```

#### Parameters

For details about the parameters in the **tag** command, see **Appendix** > **Command Line Parameters** > **Table 8 tag command parameters**.

#### Example

```shell
isula tag busybox:latest test:latest
```

### Loading Images

#### Description

Load images from a .tar package. The .tar package must be exported by using the  **docker save**  command or must be in the same format.

#### Usage

```shell
isula load [OPTIONS]
```

#### Parameters

For details about the parameters in the **load** command, see **Appendix** > **Command Line Parameters** > **Table 5 load command parameters**.

#### Example

```shell
$ isula load -i busybox.tar
Load image from "/root/busybox.tar" success
```

### Listing Images

#### Description

List all images in the current environment.

#### Usage

```shell
isula images [OPTIONS]
```

#### Parameters

For details about the parameters in the **images** command, see **Appendix** > **Command Line Parameters** > **Table 6 images command parameters**.

#### Example

```shell
$ isula images
REPOSITORY                                   TAG        IMAGE ID             CREATED              SIZE
busybox                                      latest     beae173ccac6         2021-12-31 03:19:41  1.184MB
```

### Inspecting Images

#### Description

After the configuration information of an image is returned, you can use the  **-f**  parameter to filter the information as needed.

#### Usage

```shell
isula inspect [options] CONTAINER|IMAGE [CONTAINER|IMAGE...]
```

#### Parameters

For details about the parameters in the **inspect** command, see **Appendix** > **Command Line Parameters** > **Table 7 inspect command parameters**.

#### Example

```shell
$ isula inspect -f "{{json .image.id}}" rnd-dockerhub.huawei.com/official/busybox
"e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b"
```

### Two-Way Authentication

#### Description

After this function is enabled, iSulad and image repositories communicate over HTTPS. Both iSulad and image repositories verify the validity of each other.

#### Usage

The corresponding registry needs to support this function and iSulad needs to be configured as follows:

1. Modify iSulad configuration (default path:  **/etc/isulad/daemon.json**) and set  **use-decrypted-key**  to  **false**.

2. Place related certificates in the folder named after the registry in the  **/etc/isulad/certs.d**  directory. For details about how to generate certificates, visit the official Docker website:
   * <https://docs.docker.com/engine/security/certificates/>
   * <https://docs.docker.com/engine/security/https/>

3. Run the  **systemctl restart isulad**  command to restart iSulad.

#### Parameters

Parameters can be configured in the  **/etc/isulad/daemon.json**  file or carried when iSulad is started.

```shell
isulad --use-decrypted-key=false
```

#### Example

Set  **use-decrypted-key**  to  **false**.

```shell
$ cat /etc/isulad/daemon.json
{
    "group": "isulad",
    "graph": "/var/lib/isulad",
    "state": "/var/run/isulad",
    "engine": "lcr",
    "log-level": "ERROR",
    "pidfile": "/var/run/isulad.pid",
    "log-opts": {
        "log-file-mode": "0600",
        "log-path": "/var/lib/isulad",
        "max-file": "1",
        "max-size": "30KB"
    },
    "log-driver": "stdout",
    "hook-spec": "/etc/default/isulad/hooks/default.json",
    "start-timeout": "2m",
    "storage-driver": "overlay2",
    "storage-opts": [
        "overlay2.override_kernel_check=true"
    ],
    "registry-mirrors": [
        "docker.io"
    ],
    "insecure-registries": [
        "rnd-dockerhub.huawei.com"
    ],
    "pod-sandbox-image": "",
    "image-opt-timeout": "5m",
    "native.umask": "secure",
    "network-plugin": "",
    "cni-bin-dir": "",
    "cni-conf-dir": "",
    "image-layer-check": false,
    "use-decrypted-key": false,
    "insecure-skip-verify-enforce": false
}
```

Place the certificate in the corresponding directory.

```shell
$ pwd
/etc/isulad/certs.d/my.csp-edge.com:5000
$ ls
ca.crt  tls.cert  tls.key
```

Restart iSulad.

```shell
systemctl restart isulad
```

Run the  **pull**  command to download images from the registry:

```shell
$ isula pull my.csp-edge.com:5000/busybox
Image "my.csp-edge.com:5000/busybox" pulling
Image "my.csp-edge.com:5000/busybox@sha256:f1bdc62115dbfe8f54e52e19795ee34b4473babdeb9bc4f83045d85c7b2ad5c0" pulled
```

### Importing rootfs

#### Description

Import a .tar package that contains rootfs as an image. Generally, the .tar package is exported by running the **export** command or a .tar package that contains rootfs in compatible format. Currently, the .tar, .tar.gz, .tgz, .bzip, .tar.xz, and .txz formats are supported. Do not use the TAR package in other formats for import.

#### Usage

```shell
isula import file REPOSITORY[:TAG]
```

After the import is successful, the printed character string is the image ID generated by the imported rootfs.

#### Parameters

For details about the parameters in the **import** command, see **Appendix** > **Command Line Parameters** > **Table 9 import command parameters**.

#### Example

```shell
$ isula import busybox.tar test
sha256:441851e38dad32478e6609a81fac93ca082b64b366643bafb7a8ba398301839d
$ isula images
REPOSITORY      TAG        IMAGE ID            CREATED                  SIZE
test            latest     441851e38dad        2020-09-01 11:14:35      1.168 MB
```

### Exporting rootfs

#### Description

Export the content of the rootfs of a container as a TAR package. The exported TAR package can be imported as an image by running the **import** command.

#### Usage

```shell
isula export [OPTIONS] [ID|NAME]
```

#### Parameters

For details about the parameters in the **export** command, see **Appendix** > **Command Line Parameters** > **Table 10 export command parameters**.

#### Example

```shell
$ isula run -tid --name container_test test sh
d7e601c2ef3eb8d378276d2b42f9e58a2f36763539d3bfcaf3a0a77dc668064b
$ isula export -o rootfs.tar d7e601c
$ ls
rootfs.tar
```

## Embedded Image Management

### Loading Images

#### Description

Load images based on the  **manifest**  files of embedded images. The value of  **--type**  must be set to  **embedded**.

#### Usage

```shell
isula load [OPTIONS] --input=FILE --type=TYPE
```

#### Parameters

For details about the parameters in the **load** command, see **Appendix** > **Command Line Parameters** > **Table 5 load command parameters**.

#### Example

```shell
$ isula load -i test.manifest --type embedded
Load image from "/root/work/bugfix/tmp/ci_testcase_data/embedded/img/test.manifest" success
```

### Listing Images

#### Description

List all images in the current environment.

#### Usage

```shell
isula images [OPTIONS]
```

#### Parameters

For details about the parameters in the **images** command, see **Appendix** > **Command Line Parameters** > **Table 6 images command parameters**.

#### Example

```shell
$ isula images
REPOSITORY                                   TAG        IMAGE ID             CREATED              SIZE
busybox                                      latest     beae173ccac6         2021-12-31 03:19:41  1.184MB
```

### Inspecting Images

#### Description

After the configuration information of an image is returned, you can use the  **-f**  parameter to filter the information as needed.

#### Usage

```shell
isula inspect [options] CONTAINER|IMAGE [CONTAINER|IMAGE...]
```

#### Parameters

For details about the parameters in the **inspect** command, see **Appendix** > **Command Line Parameters** > **Table 7 inspect command parameters**.

#### Example

```shell
$ isula inspect -f "{{json .created}}" test:v1
"2018-03-01T15:55:44.322987811Z"
```

### Deleting Images

#### Description

Delete one or more images.

#### Usage

```shell
isula rmi [OPTIONS] IMAGE [IMAGE...]
```

#### Parameters

For details about the parameters in the **rmi** command, see **Appendix** > **Command Line Parameters** > **Table 4 rmi command parameters**.

#### Example

```shell
$ isula rmi test:v1
Image "test:v1" removed
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/tools/community_tools/image_tailor/imagetailor_user_guide.md
---
# ImageTailor User Guide

## Introduction

In addition to the kernel, an operating system contains various peripheral packages. These peripheral packages provide functions of a general-purpose operating system but also cause the following problems:

* A large number of resources (such as memory, disks, and CPUs) are occupied, resulting in low system performance.
* Unnecessary functions increase the development and maintenance costs.

To address these problems, openEuler provides the imageTailor tool for tailoring and customization images. You can tailor unnecessary peripheral packages in the OS image or add service packages or files as required. This tool includes the following functions:

* System package tailoring: Tailors system commands, libraries, and drivers based on the list of RPM packages to be installed.
* System configuration modification: Configures the host name, startup services, time zone, network, partitions, drivers to be loaded, and kernel version.
* Software package addition: Adds custom RPM packages or files to the system.

## Installation

This section uses openEuler 22.03 LTS in the AArch64 architecture as an example to describe the installation method.

### Software and Hardware Requirements

The software and hardware requirements of imageTailor are as follows:

* The architecture is x86\_64 or AArch64.

* The OS is openEuler 22.03 LTS (the kernel version is 5.10 and the Python version is 3.9, which meet the tool requirements).

* The root directory **/** of the host to run the tool have at least 40 GB space.

* The Python version is 3.9 or later.

* The kernel version is 5.10 or later.

* The SElinux service is disabled.

  ```shell
  $ sudo setenforce 0
  $ getenforce 
  Permissive
  ```

### Obtaining the Installation Package

Download the openEuler release package to install and use imageTailor.

1. Obtain the ISO image file and the corresponding verification file.

   The image must be an everything image. Assume that the image is to be stored in the **root** directory. Run the following commands:

   ```shell
   sudo wget https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/aarch64/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso -O /root/temp/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso
   sudo wget https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/aarch64/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso.sha256sum -O /root/temp/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso.sha256sum
   ```

2. Obtain the verification value in the sha256sum verification file.

   ```shell
   sudo cat /root/temp/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso.sha256sum
   ```

3. Calculate the verification value of the ISO image file.

   ```shell
   sudo sha256sum /root/temp/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso
   ```

4. Compare the verification value in the sha256sum file with that of the ISO image. If they are the same, the file integrity is verified. Otherwise, the file integrity is damaged. You need to obtain the file again.

### Installing imageTailor

The following uses openEuler 22.03 LTS in AArch64 architecture as an example to describe how to install imageTailor.

1. Ensure that openEuler 22.03 LTS (or a running environment that meets the requirements of imageTailor) has been installed on the host.

   ```shell
   $ cat /etc/openEuler-release
   openEuler release 22.03 LTS
   ```

2. Create a **/etc/yum.repos.d/local.repo** file to configure the Yum repository. The following is an example of the configuration file. **baseurl** indicates the directory for mounting the ISO image.

   ```text
   [local]
   name=local
   baseurl=file:///root/imageTailor_mount
   gpgcheck=0
   enabled=1
   ```

3. Run the following commands as the **root** user to mount the image to the **/root/imageTailor\_mount** directory as the Yum repository (ensure that the value of **baseurl** is the same as that configured in the repo file and the disk space of the directory is greater than 20 GB):

   ```shell
   sudo mkdir /root/imageTailor_mount
   sudo mount -o loop /root/temp/openEuler-22.03-LTS-everything-aarch64-dvd.iso /root/imageTailor_mount/
   ```

4. Make the Yum repository take effect.

   ```shell
   yum clean all
   sudo yum makecache
   ```

5. Install the imageTailor tool as the **root** user.

   ```shell
   sudo yum install -y imageTailor
   ```

6. Run the following command as the **root** user to verify that the tool has been installed successfully:

   ```shell
   $ cd /opt/imageTailor/
   $ sudo ./mkdliso -h
   -------------------------------------------------------------------------------------------------------------
   Usage: mkdliso -p product_name -c configpath [--minios yes|no|force] [-h] [--sec]
   Options:
       -p,--product     Specify the product to make, check custom/cfg_yourProduct.
       -c,--cfg-path    Specify the configuration file path, the form should be consistent with custom/cfg_xxx
       --minios         Make minios: yes|no|force
       --sec            Perform security hardening
       -h,--help        Display help information

   Example:
       command:
       ./mkdliso -p openEuler -c custom/cfg_openEuler --sec
       ./mkdliso -p docker -c custom/cfg_docker
       ./mkdliso -p EMB_rootfs -c custom/cfg_EMB_rootfs
       ./mkdliso -p qcow2 -c custom/cfg_qcow2
       help:
       ./mkdliso -h
   -------------------------------------------------------------------------------------------------------------
   ```

### Directory Description

After imageTailor is installed, the directory structure of the tool package is as follows

* openEuler environment

  ```text
  [imageTailor]
      |-[custom]
          |-[cfg_openEuler]
              |-[usr_file] // Stores files to be added
              |-[usr_install] //Stores hook scripts to be added
                  |-[all]
                  |-[conf]
                  |-[hook]
              |-[cmd.conf] // Configures the default commands and libraries used by an ISO image
              |-[rpm.conf] // Configures the list of RPM packages and drivers installed by default for an ISO image
              |-[security_s.conf] // Configures security hardening policies
              |-[sys.conf] // Configures ISO image system parameters
      |-[kiwi] // Basic configurations of imageTailor
      |-[repos] //RPM sources for obtaining the RPM packages required for creating an ISO image
      |-[security-tool] // Security hardening tool
      |-mkdliso // Executable script for creating an ISO image
  ```

* Docker environment

  ```text
  [imageTailor]
      |-[custom]
          |-[cfg_docker]
              |-[config.xml]          // Configures the list of RPM packages and repositories installed by default for an ISO image
              |-[env.pm]
              |-[group]
              |-[images.sh]           // Tailoring script
              |-[passwd]
      |-[kiwi]                        // Basic configurations of imageTailor
      |-[repos]                       // RPM sources for obtaining the RPM packages required for creating an ISO image
      |-[security-tool]               // Security hardening tool
      |-mkdliso                       // Executable script for creating an ISO image
  ```

* EMB\_rootfs environment

```text
[imageTailor]
    |-[custom]
        |-[cfg_EMB_rootfs]
            |-[usr_install]         // User hook scripts
                |-[conf]
                    |-[isopackage.sdf]
                    |-[menu.lst]
                    |-[modules]
            |-[cmd.conf]            // Configures the default commands and libraries used by an ISO image
            |-[rpm.conf]            // Configures the list of RPM packages and drivers installed by default for an ISO image
            |-[security_s.conf]     // Configures security hardening policies
            |-[sys.conf]            // Configures ISO image system parameters
    |-[kiwi]                        // Basic configurations of imageTailor
    |-[repos]                       // RPM sources for obtaining the RPM packages required for creating an ISO image
    |-[security-tool]               // Security hardening tool
    |-mkdliso                       // Executable script for creating an ISO image
```

* QCOW2 environment

```text
[imageTailor]
    |-[custom]
        |-[cfg_qcow2]
            |-[bin]                 // Command scripts
                |-[create-image]    // Image creation entry
                |-[source_files]    // Script invocation entry
            |-[config]              // Configurations
                |-[grub.cfg]        // Grub configurations
                |-[repo]            // Repositories
                |-[root_pwd]        // Root password
                |-[rpmlist]         // Software package list
            |-[hooks]               // hook scripts
            |-[lib]                 // Common scripts
            |-[misc]                // Public scripts
            |-[template]            
    |-[kiwi]                        // Basic configurations of imageTailor
    |-[repos]                       // RPM sources for obtaining the RPM packages required for creating an ISO image
    |-[security-tool]               // Security hardening tool
    |-mkdliso                       // Executable script for creating an ISO image
```

## Image Customization

This section describes how to use the imageTailor tool to package the service RPM packages, custom files, drivers, commands, and libraries to the target ISO image.

### Overall Process

The following figure shows the process of using imageTailor to customize an image.

![](./figures/flowchart.png)

The steps are described as follows:

* Check software and hardware environment: Ensure that the host for creating the ISO image meets the software and hardware requirements.

* Customize service packages: Add RPM packages (including service RPM packages, commands, drivers, and library files) and files (including custom files, commands, drivers, and library files).

  * Adding service RPM packages: Add RPM packages to the ISO image as required. For details, see [Installation](#installation).
  * Adding custom files: If you want to perform custom operations such as hardware check, system configuration check, and driver installation when the target ISO system is installed or started, you can compile custom files and package them to the ISO image.
  * Adding drivers, commands, and library files: If the RPM package source of openEuler does not contain the required drivers, commands, or library files, you can use imageTailor to package the corresponding drivers, commands, or library files into the ISO image.

* Configure system parameters:

  * Configuring host parameters: To ensure that the ISO image is successfully installed and started, you need to configure host parameters.
  * Configuring partitions: You can configure service partitions based on the service plan and adjust system partitions.
  * Configuring the network: You can set system network parameters as required, such as the NIC name, IP address, and subnet mask.
  * Configuring the initial password: To ensure that the ISO image is successfully installed and started, you need to configure the initial passwords of the **root** user and GRUB.
  * Configuring kernel parameters: You can configure the command line parameters of the kernel as required.

* Configure security hardening policies.

  ImageTailor provides default security hardening policies. You can modify **security\_s.conf** (in the ISO image customization phase) to perform secondary security hardening on the system based on service requirements. For details, see the [Security Hardening Guide](https://docs.openeuler.org/en/docs/22.03_LTS/docs/SecHarden/secHarden.html).

* Create an ISO image:

  Use the imageTailor tool to create an ISO image.

### Customizing Service Packages

You can pack service RPM packages, custom files, drivers, commands, and library files into the target ISO image as required.

#### Setting a Local Repo Source

To customize an ISO image, you must set a repo source in the **/opt/imageTailor/repos/euler\_base/** directory. This section describes how to set a local repo source.

1. Download the ISO file released by openEuler. (The RPM package of the everything image released by the openEuler must be used.)

   ```shell
   cd /opt
   wget https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso
   ```

2. Create a mount directory **/opt/openEuler\_repo** and mount the ISO file to the directory.

   ```shell
   $ sudo mkdir -p /opt/openEuler_repo
   $ sudo mount openEuler-22.03-LTS-everything-aarch64-dvd.iso /opt/openEuler_repo
   mount: /opt/openEuler_repo: WARNING: source write-protected, mounted read-only.
   ```

3. Copy the RPM packages in the ISO file to the **/opt/imageTailor/repos/euler\_base/** directory.

   ```shell
   $ sudo rm -rf /opt/imageTailor/repos/euler_base && sudo mkdir -p /opt/imageTailor/repos/euler_base
   $ sudo cp -ar /opt/openEuler_repo/Packages/* /opt/imageTailor/repos/euler_base
   $ sudo chmod -R 644 /opt/imageTailor/repos/euler_base
   $ sudo ls /opt/imageTailor/repos/euler_base|wc -l
   2577
   $ sudo umount /opt/openEuler_repo && sudo rm -rf /opt/openEuler_repo
   $ cd /opt/imageTailor
   ```

#### Adding Files

You can add files to an ISO image as required. The file types include custom files, drivers, commands, or library file. Store the files to the **/opt/imageTailor/custom/cfg\_openEuler/usr\_file** directory.

##### Precautions

* The commands to be packed must be executable. Otherwise, imageTailor will fail to pack the commands into the ISO.

* The file stored in the **/opt/imageTailor/custom/cfg\_openEuler/usr\_file** directory will be generated in the root directory of the ISO. Therefore, the directory structure of the file must be a complete path starting from the root directory so that imageTailor can place the file in the correct directory.

  For example, if you want **file1** to be in the **/opt** directory of the ISO, create an **opt** directory in the **usr\_file** directory and copy **file1** to the **opt** directory. For example:

  ```shell
  $ pwd
   /opt/imageTailor/custom/cfg_openEuler/usr_file

  $ tree
    .
    ├── etc
    │   ├── default
    │   │   └── grub
    │   └── profile.d
    │       └── csh.precmd
    └── opt
        └── file1
    
    4 directories, 3 files
  ```

* The paths in **/opt/imageTailor/custom/cfg\_openEuler/usr\_file** must be real paths. For example, the paths do not contain soft links. You can run the `realpath` or `readlink -f` command to query the real path.

* If you need to invoke a custom script in the system startup or installation phase, that is, a hook script, store the file in the **hook** directory.

#### Adding RPM Packages

##### Procedure

To add RPM packages (drivers, commands, or library files) to an ISO image, perform the following steps:

> \[!NOTE] **NOTE:**
>
> * The **rpm.conf** and **cmd.conf** files are stored in the **/opt/imageTailor/custom/cfg\_openEuler/** directory.
> * The RPM package tailoring granularity below indicates **sys\_cut='no'**. For details about the cutout granularity, see [Configuring Host Parameters](#configuring-host-parameters).
> * If no local repo source is configured, configure a local repo source by referring to [Setting a Local Repo Source](#setting-a-local-repo-source).

1. Check whether the **/opt/imageTailor/repos/euler\_base/** directory contains the RPM package to be added.

   * If yes, go to step 2.
   * If no, go to step 3.

2. Configure the RPM package information in the **\<bootstrap>** section in the **rpm.conf** file.

   * For the RPM package tailoring granularity, no further action is required.
   * For other tailoring granularities, go to step 4.

3. Obtain the RPM package and store it in the **/opt/imageTailor/custom/cfg\_openEuler/usr\_rpm** directory. If the RPM package depends on other RPM packages, store the dependency packages to this directory because the added RPM package and its dependent RPM packages must be packed into the ISO image at the same time.

   * For the RPM package tailoring granularity, go to step 4.
   * For other tailoring granularities, no further action is required.

4. Configure the drivers, commands, and library files to be retained in the RPM package in the **rpm.conf** and **cmd.conf** files. If there are common files to be tailored, configure them in the **\<strip type="delete">\</strip>** section in the **cmd.conf** file.

##### Configuration File Description

| Operation        | Configuration File| Section                                                    |
| :----------- | :----------- | :----------------------------------------------------------- |
| Adding drivers    | rpm.conf     | \<drivers>      \<file name="{driver\_name}">\</drivers>Note: The **driver\_name** is the relative path of **/lib/modules/{kernel\_version\_number}/kernel/**.|
| Adding commands    | cmd.conf     | \<strip type="tools">    \<file name="{command\_name}">\</strip> |
| Adding library files  | cmd.conf     | \<strip type="libs">    \<file name="{library\_file\_name}">\</strip> |
| Deleting other files| cmd.conf     | \<strip type="delete">    \<file name="{file\_name}">\</strip>Note: The file name must be an absolute path.|

**Example**

* Adding drivers

  ```xml
  <drivers>
          <file name="arch/arm64/crypto/aes-ce-blk.ko"/>
          <file name="arch/arm64/crypto/aes-ce-ccm.ko"/>
          <file name="arch/arm64/crypto/aes-ce-cipher.ko"/>
      ......
  </drivers>
  ```

* Adding commands

  ```xml
  <strip type="tools">
      <file name="aarch64-openEuler-linux-gnu-pkg-config"/>
      <file name="accessdb"/>
      <file name="acyclic"/>
      ......
  </strip>
  ```

* Adding library files

  ```xml
  <strip type="tools">
      <file name="ANSI_X3.110"/>
      <file name="ARMSCII-8"/>
      <file name="ASMO_449"/>
  </strip>
  ```

* Deleting other files

  ```xml
  <strip type="delete">
      <file name="/usr/lib/firmware/radeon"/>
      <file name="/usr/lib/firmware/rtlwifi"/>
      <file name="/usr/lib/firmware/RTL8192E"/>
  </strip>
  ```

#### Adding Hook Scripts

A hook script is invoked by the OS during startup and installation to execute the actions defined in the script. The directory for storing hook scripts of imageTailor is **custom/cfg\_openEuler/usr\_install/hook directory**, which has different subdirectories. Each subdirectory represents an OS startup or installation phase. Store the scripts based on the phases in which the scripts are invoked. This operation is not available in the Docker environment.

##### Script Naming Rule

The script name must start with **S+number** (the number must be at least two digits). The number indicates the execution sequence of the hook script. Example: **S01xxx.sh**

> \[!NOTE] **NOTE:**
>
> The scripts in the **hook** directory are executed using the `source` command. Therefore, exercise caution when using the `exit` command in the scripts because the entire installation script exits after the `exit` command is executed.

##### Description of hook Subdirectories

| Subdirectory          | Script Example        | Time for Execution                     | Description                                                        |
| :-------------------- | :---------------------| :------------------------------- | :----------------------------------------------------------- |
| insmod\_drv\_hook       | N/A                   | After OS drivers are loaded                | N/A                                                          |
| custom\_install\_hook   | S01custom\_install.sh  | After the drivers are loaded, that is, after **insmod\_drv\_hook** is executed| You can customize the OS installation process by using a custom script.|
| env\_check\_hook        | S01check\_hw.sh        | Before the OS installation initialization                  | The script is used to check hardware specifications and types before initialization.|
| set\_install\_ip\_hook   | S01set\_install\_ip.sh  | When network configuration is being performed during OS installation initialization.    | You can customize the network configuration by using a custom script.|
| before\_partition\_hook | S01checkpart.sh       | Before partitioning                    | You can check correctness of the partition configuration file by using a custom script.|
| before\_setup\_os\_hook  | N/A                   | Before the repo file is decompressed                    | You can customize partition mounting.If the decompression path of the installation package is not the root partition specified in the partition configuration, customize partition mounting and assign the decompression path to the input global variable.|
| before\_mkinitrd\_hook  | S01install\_drv.sh     | Before the `mkinitrd` command is run          | The hook script executed before running the `mkinitrd` command when **initrd** is saved to the disk. You can add and update driver files in **initrd**.|
| after\_setup\_os\_hook   | N/A                   | After OS installation                  | After the installation is complete, you can perform custom operations on the system files, such as modifying **grub.cfg**.|
| install\_succ\_hook     | N/A                   | When the OS is successfully installed            | The scripts in this subdirectory are used to parse the installation information and send information of whether the installation succeeds.**install\_succ\_hook** cannot be set to **install\_break**.|
| install\_fail\_hook     | N/A                   | When the OS installation fails                    | The scripts in this subdirectory are used to parse the installation information and send information of whether the installation succeeds.**install\_fail\_hook** cannot be set to **install\_break**.|

### Configuring System Parameters

Before creating an ISO image, you need to configure system parameters, including host parameters, initial passwords, partitions, network, compilation parameters, and system command line parameters.

#### Configuring Host Parameters

The **\<sysconfig> \</sysconfig>** section in the **/opt/imageTailor/custom/cfg\_openEuler/sys.conf** file is used to configure common system parameters, such as the host name and kernel boot parameters. This operation is not available in the Docker environment.

The default configuration provided by openEuler is as follows. You can modify the configuration as required.

```text
<sysconfig>
    sys_service_enable='ipcc'
    sys_service_disable='cloud-config cloud-final cloud-init-local cloud-init'
    sys_utc='yes'
    sys_timezone=''
    sys_cut='no'
    sys_usrrpm_cut='no'
    sys_hostname='Euler'
    sys_usermodules_autoload=''
    sys_gconv='GBK'
</sysconfig>
```

The parameters are described as follows:

* sys\_service\_enable

  This parameter is optional. Services enabled by the OS by default. Separate multiple services with spaces. If you do not need to add a system service, use the default value **ipcc**. Pay attention to the following during the configuration:

  * Default system services cannot be deleted.
  * You can configure service-related services, but the repo source must contain the service RPM package.
  * By default, only the services configured in this parameter are enabled. If a service depends on other services, you need to configure the depended services in this parameter.

* sys\_service\_disable

  This parameter is optional. Services that are not allowed to automatically start upon system startup. Separate multiple services with spaces. If no system service needs to be disabled, leave this parameter blank.

* sys\_utc

  (Mandatory) Indicates whether to use coordinated universal time (UTC) time. The value can be **yes** or **no**. The default value is **yes**.

* sys\_timezone

  This parameter is optional. Sets the time zone. The value can be a time zone supported by openEuler, which can be queried in the **/usr/share/zoneinfo/zone.tab** file.

* sys\_cut

  (Mandatory) Indicates whether to tailor the RPM packages. The value can be **yes**, **no**, or **debug**.**yes** indicates that the RPM packages are tailored. **no** indicates that the RPM packages are not tailored (only the RPM packages in the **rpm.conf** file is installed). **debug** indicates that the RPM packages are tailored but the `rpm` command is retained for customization after installation. The default value is **no**.

  > \[!NOTE] NOTE:
  >
  > * imageTailor installs the RPM package added by the user, deletes the files configured in the **\<script type="delete">\</script>** section of the **cmd.conf** file, and then deletes the commands, libraries, and drivers that are not configured in **cmd.conf** or **rpm.conf**.
  > * When **sys\_cut='yes'** is configured, imageTailor does not support the installation of the `rpm` command. Even if the `rpm` command is configured in the **rpm.conf** file, the configuration does not take effect.

* sys\_usrrpm\_cut

  (Mandatory) Indicates whether to tailor the RPM packages added by users to the **/opt/imageTailor/custom/cfg\_openEuler/usr\_rpm** directory. The value can be **yes** or **no**. The default value is **no**.

  * **sys\_usrrpm\_cut='yes'**: imageTailor installs the RPM packages added by the user, deletes the file configured in the **\<script type="delete">\</script>** section in the **cmd.conf** file, and then deletes the commands, libraries, and drivers that are not configured in **cmd.conf** or **rpm.conf**.

  * **sys\_usrrpm\_cut='no'**: imageTailor installs the RPM packages added by the user but does not delete the files in the RPM packages.

* sys\_hostname

  (Mandatory) Host name. After the OS is deployed in batches, you are advised to change the host name of each node to ensure that the host name of each node is unique.

  The host name must be a combination of letters, digits, and hyphens (-) and must start with a letter or digit. Letters are case sensitive. The value contains a maximum of 63 characters. The default value is **Euler**.

* sys\_usermodules\_autoload

  (Optional) Driver loaded during system startup. When configuring this parameter, you do not need to enter the file extension **.ko**. If there are multiple drivers, separate them by space. By default, this parameter is left blank, indicating that no additional driver is loaded.

* sys\_gconv

  (Optional) This parameter is used to tailor **/usr/lib/gconv** and **/usr/lib64/gconv**. The options are as follows:

  * **null**/**NULL**: indicates that this parameter is not configured. If **sys\_cut='yes'** is configured, **/usr/lib/gconv** and **/usr/lib64/gconv** will be deleted.
  * **all**/**ALL**: keeps **/usr/lib/gconv** and **/usr/lib64/gconv**.
  * **xxx,xxx**: keeps the corresponding files in the **/usr/lib/gconv** and **/usr/lib64/gconv** directories. If multiple files need to be kept, use commas (,) to separate them.

* sys\_man\_cut

  (Optional) Indicates whether to tailor the man pages. The value can be **yes** or **no**. The default value is **yes**.

> \[!NOTE] NOTE:
>
> If both **sys\_cut** and **sys\_usrrpm\_cut** are configured, **sys\_cut** is used. The following rules apply:
>
> * sys\_cut='no'
>
>   No matter whether **sys\_usrrpm\_cut** is set to **yes** or **no**, the system RPM package tailoring granularity is used. That is, imageTailor installs the RPM packages in the repo source and the RPM packages in the **usr\_rpm** directory, however, the files in the RPM package are not deleted. Even if some files in the RPM packages are not required, imageTailor will delete them.
>
> * sys\_cut='yes'
>
>   * sys\_usrrpm\_cut='no'
>
>   System RPM package tailoring granularity: imageTailor deletes files in the RPM packages in the repo sources as configured.
>
>   * sys\_usrrpm\_cut='yes'
>
>   System and user RPM package tailoring granularity: imageTailor deletes files in the RPM packages in the repo sources and the **usr\_rpm** directory as configured.

#### Configuring Initial Passwords

The **root** and GRUB passwords must be configured during OS installation. Otherwise, you cannot log in to the OS as the **root** user after the OS is installed using the tailored ISO image. This section describes how to configure the initial passwords. This operation is not available in the Docker environment.

> \[!NOTE] NOTE:
>
> You must configure the initial **root** and GRUB passwords manually.

##### Configuring the Initial Password of the root User

###### Introduction

The initial password of the **root** user is stored in the **/opt/imageTailor/custom/cfg\_openEuler/rpm.conf** file. You can modify this file to set the initial password of the **root** user.

> \[!NOTE] **NOTE:**
>
> * If the `--minios yes/force` parameter is required when you run the `mkdliso` command to create an ISO image, you need to enter the corresponding information in the **/opt/imageTailor/kiwi/minios/cfg\_minios/rpm.conf** file.

The default configuration of the initial password of the **root** user in the **/opt/imageTailor/custom/cfg\_openEuler/rpm.conf** file is as follows. Add a password of your choice.

```xml
<users group="root">
    <user pwd="${pwd}" home="/root" name="root"/>
</users>
```

The parameters are described as follows:

* **group**: group to which the user belongs.
* **pwd**: ciphertext of the initial password. The encryption algorithm is SHA-512. Replace **${pwd}** with the actual ciphertext.
* **home**: home directory of the user.
* **name**: name of the user to be configured.

###### Modification Method

Before creating an ISO image, you need to change the initial password of the **root** user. The following describes how to set the initial password of the **root** user (**root** permissions are required):

1. Add a user for generating a password, for example, **testUser**.

   ```shell
   sudo useradd testUser
   ```

2. Set the password of **testUser**. Run the following command and set the password as prompted:

   ```shell
   $ sudo passwd testUser
   Changing password for user testUser.
   New password: 
   Retype new password: 
   passwd: all authentication tokens updated successfully.
   ```

3. View the **/etc/shadow** file. The content following **testUser** (string between two colons) is the ciphertext of the password.

   ```shell
   $ sudo cat /etc/shadow | grep testUser
   testUser:$6$YkX5uFDGVO1VWbab$jvbwkZ2Kt0MzZXmPWy.7bJsgmkN0U2gEqhm9KqT1jwQBlwBGsF3Z59heEXyh8QKm3Qhc5C3jqg2N1ktv25xdP0:19052:0:90:7:35::
   ```

4. Copy and paste the ciphertext to the **pwd** field in the **/opt/imageTailor/custom/cfg\_openEuler/rpm.conf** file.

   ```xml
   <users group="root">
       <user pwd="$6$YkX5uFDGVO1VWbab$jvbwkZ2Kt0MzZXmPWy.7bJsgmkN0U2gEqhm9KqT1jwQBlwBGsF3Z59heEXyh8QKm3Qhc5C3jqg2N1ktv25xdP0" home="/root" name="root"/>
   </users>
   ```

5. If the `--minios yes/force` parameter is required when you run the `mkdliso` command to create an ISO image, configure the **pwd** field of the corresponding user in **/opt/imageTailor/kiwi/minios/cfg\_minios/rpm.conf**.

   ```xml
   <users group="root">
       <user pwd="$6$YkX5uFDGVO1VWbab$jvbwkZ2Kt0MzZXmPWy.7bJsgmkN0U2gEqhm9KqT1jwQBlwBGsF3Z59heEXyh8QKm3Qhc5C3jqg2N1ktv25xdP0" home="/root" name="root"/>
   </users>
   ```

##### Configuring the Initial GRUB Password

The initial GRUB password is stored in the **/opt/imageTailor/custom/cfg\_openEuler/usr\_file/etc/default/grub** file. Modify this file to configure the initial GRUB password. If the initial GRUB password is not configured, the ISO image will fail to be created.

> \[!NOTE] NOTE:
>
> * The **root** permissions are required for configuring the initial GRUB password.
>
> * The default user corresponding to the GRUB password is **root**.
>
> * The `grub2-set-password` command must exist in the system. If the command does not exist, install it in advance.

1. Run the following command and set the GRUB password as prompted:

   ```shell
   $ sudo grub2-set-password -o ./
   Enter password: 
   Confirm password: 
   grep: .//grub.cfg: No such file or directory
   WARNING: The current configuration lacks password support!
   Update your configuration with grub2-mkconfig to support this feature.
   ```

2. After the command is executed, the **user.cfg** file is generated in the current directory. The content starting with **grub.pbkdf2.sha512** is the encrypted GRUB password.

   ```shell
   $ sudo cat user.cfg 
   GRUB2_PASSWORD=grub.pbkdf2.sha512.10000.CE285BE1DED0012F8B2FB3DEA38782A5B1040FEC1E49D5F602285FD6A972D60177C365F1
   B5D4CB9D648AD4C70CF9AA2CF9F4D7F793D4CE008D9A2A696A3AF96A.0AF86AB3954777F40D324816E45DD8F66CA1DE836DC7FBED053DB02
   4456EE657350A27FF1E74429546AD9B87BE8D3A13C2E686DD7C71D4D4E85294B6B06E0615
   ```

3. Copy the preceding ciphertext and add the following configuration to the **/opt/imageTailor/custom/cfg\_openEuler/usr\_file/etc/default/grub** file:

   ```text
   GRUB_PASSWORD="grub.pbkdf2.sha512.10000.CE285BE1DED0012F8B2FB3DEA38782A5B1040FEC1E49D5F602285FD6A972D60177C365F1
   B5D4CB9D648AD4C70CF9AA2CF9F4D7F793D4CE008D9A2A696A3AF96A.0AF86AB3954777F40D324816E45DD8F66CA1DE836DC7FBED053DB02
   4456EE657350A27FF1E74429546AD9B87BE8D3A13C2E686DD7C71D4D4E85294B6B06E0615"
   ```

#### Configuring Partitions

If you want to adjust system partitions or service partitions, modify the **\<HDpartitions >** section in the **/opt/imageTailor/custom/cfg\_openEuler/sys.conf** file. This operation is not available in the Docker environment.

> \[!NOTE] **NOTE:**
>
> * System partition: partition for storing the OS.
> * Service partition: partition for service data.
> * The type of a partition is determined by the content it stores, not the size, mount path, or file system.
> * Partition configuration is optional. You can manually configure partitions after OS installation.

The format of **\<HDpartitions>** is as follows:

disk\_ID   mount\_path   partition\_size   partition\_type   file\_system   \[Secondary formatting flag]

The default configuration is as follows:

```text
<HDpartitions>
hd0    /boot        512M      primary     ext4     yes
hd0    /boot/efi    200M      primary     vfat     yes
hd0    /            30G       primary     ext4
hd0    -            -         extended    -
hd0    /var         1536M     logical     ext4
hd0    /home        max       logical     ext4
</HDpartitions>
```

The parameters are described as follows:

* disk\_ID:
  ID of a disk. Set this parameter in the format of **hd***x*, where *x* indicates the *x*th disk.

  > \[!NOTE] **NOTE:**
  >
  > Partition configuration takes effect only when the disk can be recognized.

* mount\_path:
  Mount path to a specified partition. You can configure service partitions and adjust the default system partition. If you do not mount partitions, set this parameter to **-**.

  > \[!NOTE] **NOTE:**
  >
  > * You must configure the mount path to **/**. You can adjust mount paths to other partitions according to your needs.
  > * When the UEFI boot mode is used, the partition configuration in the x86\_64 architecture must contain the mount path **/boot**, and the partition configuration in the AArch64 architecture must contain the mount path **/boot/efi**.

* partition\_size:
  The value types are as follows:

  * G/g: The unit of a partition size is GB, for example, 2G.
  * M/m: The unit of a partition size is MB, for example, 300M.
  * T/t: The unit of a partition size is TB, for example, 1T.
  * MAX/max: The rest space of a hard disk is used to create a partition. This value can only be assigned to the last partition.

  > \[!NOTE] **NOTE:**

> - A partition size value cannot contain decimal numbers. If there are decimal numbers, change the unit of the value to make the value an integer. For example, 1.5 GB should be changed to 1536 MB.
> - When the partition size is set to **MAX**/**max**, the size of the remaining partition cannot exceed the limit of the supported file system type (the default file system type is **ext4**, and the maximum size is **16T**).

* partition\_type:
  The values of partition types are as follows:

  * primary: primary partitions
  * extended: extended partition (configure only *disk\_ID* for this partition)
  * logical: logical partitions

* file\_system:
  Currently, **ext4** and **vfat** file systems are supported.

* \[Secondary formatting flag]:
  Indicates whether to format the disk during secondary installation. This parameter is optional.

  * The value can be **yes** or **no**. The default value is **no**.

  > \[!NOTE] **NOTE:**
  >
  > Secondary formatting indicates that openEuler has been installed on the disk before this installation. If the partition table configuration (partition size, mount point, and file type) used in the previous installation is the same as that used in the current installation, this flag can be used to configure whether to format the previous partitions, except the **/boot** and **/** partitions. If the target host is installed for the first time, this flag does not take effect, and all partitions with specified file systems are formatted.

#### Configuring the Network

The system network parameters are stored in **/opt/imageTailor/custom/cfg\_openEuler/sys.conf**. You can modify the network parameters of the target ISO image, such as the NIC name, IP address, and subnet mask, by configuring **\<netconfig-*x*>\</netconfig-*x*>** in this file. This operation is not available in the Docker environment.

The default network configuration in the **sys.conf** file is as follows. **netconfig-0** indicates the **eth0** NIC. If you need to configure an additional NIC, for example, **eth1**, add **\<netconfig-1>\</netconfig-1>** to the configuration file and set the parameters of **eth1**.

```text
<netconfig-0>
BOOTPROTO="dhcp"
DEVICE="eth0"
IPADDR=""
NETMASK=""
STARTMODE="auto" 
</netconfig-0>
```

The following table describes the parameters.

* | Parameter | Mandatory or Not| Value                                           | Description                                                        |
  | :-------- | -------- | :------------------------------------------------ | :----------------------------------------------------------- |
  | BOOTPROTO | Yes      | none / static / dhcp                              | **none**: No protocol is used for boot, and no IP address is assigned.**static**: An IP address is statically assigned.**dhcp**: An IP address is dynamically obtained using the dynamic host configuration protocol (DHCP).|
  | DEVICE    | Yes      | Example: **eth1**                                         | NIC name.                                                    |
  | IPADDR    | Yes      | Example: **192.168.11.100**                               | IP address.This parameter must be configured only when the value of **BOOTPROTO** is **static**.|
  | NETMASK   | Yes      | -                                                 | Subnet mask.This parameter must be configured only when the value of **BOOTPROTO** is **static**.|
  | STARTMODE | Yes      | manual / auto / hotplug / ifplugd / nfsroot / off | NIC start mode.**manual**: A user runs the `ifup` command on a terminal to start an NIC.**auto**/**hotplug**/**ifplug**/**nfsroot**: An NIC is started when the OS identifies it.**off**: An NIC cannot be started in any situations.For details about the parameters, run the `man ifcfg` command on the host that is used to create the ISO image.|

#### Configuring Kernel Parameters

To ensure stable and efficient running of the system, you can modify kernel command line parameters as required. For an OS image created by imageTailor, you can modify the **GRUB\_CMDLINE\_LINUX** configuration in the **/opt/imageTailor/custom/cfg\_openEuler/usr\_file/etc/default/grub** file to modify the kernel command line parameters. This operation is not available in the Docker, EMB\_rootfs, or QCOW2 environment.

The default settings of the kernel command line parameters in **GRUB\_CMDLINE\_LINUX** are as follows:

```text
GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0 crashkernel=512M oops=panic softlockup_panic=1 reserve_kbox_mem=16M crash_kexec_post_notifiers panic=3 console=tty0"
```

The meanings of the configurations are as follows (for details about other common kernel command line parameters, see related kernel documents):

* net.ifnames=0 biosdevname=0

  Name the NIC in traditional mode.

* crashkernel=512M

  The memory space reserved for kdump is 512 MB.

* oops=panic panic=3

  The kernel panics when an oops error occurs, and the system restarts 3 seconds later.

* softlockup\_panic=1

  The kernel panics when a soft-lockup is detected.

* reserve\_kbox\_mem=16M

  The memory space reserved for Kbox is 16 MB.

* console=tty0

  Specifies **tty0** as the output device of the first virtual console.

* crash\_kexec\_post\_notifiers

  After the system crashes, the function registered with the panic notification chain is called first, and then kdump is executed.

### Creating an Image

After customizing the operating system, you can use the `mkdliso` script to create the OS image file. The OSimage created using imageTailor is an ISO image file.

#### Command Description

##### Syntax

```shell
mkdliso [-p openEuler|docker|EMB_rootfs|qcow2] [-c custom/cfg_openEuler|custom/cfg_docker|custom/cfg_EMB_rootfs|custom/cfg_qcow2] [--minios yes|no|force] [--sec] [-h]
```

##### Parameter Description

| Parameter| Mandatory| Description                                                     | Value Range                                                    |
| -------- | -------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| -p       | Yes      | Specifies the product name.                                                | **openEuler**, **docker**, **EMB\_rootfs**, **qcow2**                                                    |
| c        | Yes      | Specifies the relative path of the configuration file.                                      | **custom/cfg\_openEuler**, **custom/cfg\_docker**, **custom/cfg\_EMB\_rootfs**, **custom/cfg\_qcow2** |
| --minios | No      | Specifies whether to create the **initrd** file that is used to boot the system during system installation.                       | The default value is **yes**.**yes**: The **initrd** file will be created when the command is executed for the first time. When a subsequent `mkdliso` is executed, the system checks whether the **initrd** file exists in the **usr\_install/boot** directory using sha256 verification. If the **initrd** file exists, it is not created again. Otherwise, it is created.**no**: The **initrd** file is not created. The **initrd** file used for system boot and running is the same.**force**: The **initrd** file will be created forcibly, regardless of whether it exists in the **usr\_install/boot** directory or not.|
| --sec    | No      | Specifies whether to perform security hardening on the generated ISO file.If this parameter is not specified, the user should undertake the resultant security risks| N/A                                                          |
| -h       | No      | Obtains help information.                                                | N/A                                                          |

#### Image Creation Guide

To create an ISO image using`mkdliso`, perform the following steps:

> \[!NOTE] NOTE:
>
> * The absolute path to `mkdliso` must not contain spaces. Otherwise, the ISO image creation will fail.
> * In the environment for creating the ISO image, the value of **umask** must be set to **0022**.

1. Run the `mkdliso` command as the **root** user to generate the ISO image file. The following command is used for reference:

   ```shell
   sudo /opt/imageTailor/mkdliso -p openEuler -c custom/cfg_openEuler --sec
   sudo /opt/imageTailor/mkdliso -p docker -c custom/cfg_docker
   sudo /opt/imageTailor/mkdliso -p EMB_rootfs -c custom/cfg_EMB_rootfs
   sudo /opt/imageTailor/mkdliso -p qcow2 -c custom/cfg_qcow2
   ```

   After the command is executed, the created files are stored in the **/opt/imageTailor/result/{date}** directory, including:

   * **openEuler-aarch64.iso** and **openEuler-aarch64.iso.sha256** in the openEuler environment
   * **openEuler-image-qemu-\*.rootfs.cpio.gz** and **openEuler-image-qemu-\*.rootfs.cpio.gz.sha256** in the EMB\_rootfs environment
   * **docker.\*.tar.xz**, **docker.\*.tar.xz.sha256sum**, **docker\_source.rpmlist**, and **docker\_binary.rpmlist** in the Docker environment
   * **openEuler\_\*.qcow2** and **openEuler\_\*.qcow2.sha256sum** in the QCOW2 environment

2. Verify the integrity of the ISO image file. Assume that the date and time is **2022-03-21-14-48**.

   ```shell
   cd /opt/imageTailor/result/2022-03-21-14-48/
   sha256sum -c openEuler-aarch64.iso.sha256
   ```

   If the following information is displayed, the ISO image creation is complete.

   ```text
   openEuler-aarch64.iso: OK
   ```

   If the following information is displayed, the image is incomplete. The ISO image file is damaged and needs to be created again.

   ```text
   openEuler-aarch64.iso: FAILED
   sha256sum: WARNING: 1 computed checksum did NOT match
   ```

3. View the logs.

   After an image is created, you can view logs as required (for example, when an error occurs during image creation). When an image is created for the first time, the corresponding log file and security hardening log file are compressed into a TAR package (the log file is named in the format of **sys\_custom\_log\_{Date}.tar.gz**) and stored in the **result/log directory**. Only the latest 50 compressed log packages are stored in this directory. If the number of compressed log packages exceeds 50, the earliest files will be overwritten.

### Tailoring Time Zones

After the customized ISO image is installed, you can tailor the time zones supported by the openEuler system as required. This section describes how to tailor the time zones.

The information about time zones supported by openEuler is stored in the time zone folder **/usr/share/zoneinfo**. You can run the following command to view the time zone information:

```shell
$ ls /usr/share/zoneinfo/
Africa/  America/  Asia/  Atlantic/  Australia/  Etc/  Europe/
Pacific/  zone.tab
```

Each subfolder represents an area. The current areas include continents, oceans, and **Etc**. Each area folder contains the locations that belong to it. Generally, a location is a city or an island.

All time zones are in the format of *area/location*. For example, if China Standard Time is used in southern China, the time zone is Asia/Shanghai (location may not be the capital). The corresponding time zone file is **/usr/share/zoneinfo/Asia/Shanghai**.

If you want to tailor some time zones, delete the corresponding time zone files.

### Customization Example

This section describes how to use imageTailor to create an ISO image.

1. Check whether the environment used to create the ISO meets the requirements.

   ```shell
   $ cat /etc/openEuler-release
   openEuler release 22.03 LTS
   ```

2. Ensure that the root directory has at least 40 GB free space.

   ```shell
   $ df -h
   Filesystem                  Size  Used Avail Use% Mounted on
   ......
   /dev/vdb                    196G   28K  186G   1% /
   ```

3. Install the imageTailor tailoring tool. For details, see [Installation](#installation).

   ```shell
   $ sudo yum install -y imageTailor
   $ ll /opt/imageTailor/
   total 88K
   drwxr-xr-x.  3 root root 4.0K Mar  3 08:00 custom
   drwxr-xr-x. 10 root root 4.0K Mar  3 08:00 kiwi
   -r-x------.  1 root root  69K Mar  3 08:00 mkdliso
   drwxr-xr-x.  2 root root 4.0K Mar  9 14:48 repos
   drwxr-xr-x.  2 root root 4.0K Mar  9 14:48 security-tool
   ```

4. Configure a local repo source.

   ```shell
   $ wget https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso
   $ sudo mkdir -p /opt/openEuler_repo
   $ sudo mount openEuler-22.03-LTS-everything-aarch64-dvd.iso /opt/openEuler_repo
   mount: /opt/openEuler_repo: WARNING: source write-protected, mounted read-only.
   $ sudo rm -rf /opt/imageTailor/repos/euler_base && sudo mkdir -p /opt/imageTailor/repos/euler_base
   $ sudo cp -ar /opt/openEuler_repo/Packages/* /opt/imageTailor/repos/euler_base
   $ sudo chmod -R 644 /opt/imageTailor/repos/euler_base
   $ sudo ls /opt/imageTailor/repos/euler_base|wc -l
   2577
   $ sudo umount /opt/openEuler_repo && sudo rm -rf /opt/openEuler_repo
   $ cd /opt/imageTailor
   ```

5. Change the **root** and GRUB passwords.

   Replace **${pwd\*}** with the encrypted password by referring to [Configuring Initial Passwords](#configuring-initial-passwords).

   * openEuler

   ```shell
   $ cd /opt/imageTailor/
   $ sudo vi custom/cfg_openEuler/usr_file/etc/default/grub
   GRUB_PASSWORD="${pwd1}"
   $ 
   $ sudo vi kiwi/minios/cfg_minios/rpm.conf
   <users group="root">
       <user pwd="${pwd2}" home="/root" name="root"/>
   </users>
   $ 
   $ sudo vi custom/cfg_openEuler/rpm.conf
   <users group="root">
       <user pwd="${pwd2}" home="/root" name="root"/>
   </users>
   ```

   * Docker:
     There is no GRUB or root password.

   * EMB\_rootfs:

   ```shell
   $ cd /opt/imageTailor/
   $ sudo vi custom/cfg_EMB_rootfs/rpm.conf
   <users group="root">
       <user pwd="${pwd2}" home="/root" name="root"/>
   </users>
   ```

   * qcow2:

   ```shell
   $ cd /opt/imageTailor/
   $ sudo vi custom/cfg_qcow2/config/root_pwd
   ${pwd2}
   ```

6. Run the tailoring command.

   * openEuler

   ```shell
   $ sudo rm -rf /opt/imageTailor/result
   $ sudo ./mkdliso -p openEuler -c custom/cfg_openEuler --minios force
   ......
   Complete release iso file at: result/2022-03-09-15-31/openEuler-aarch64.iso
   move all mkdliso log file to result/log/sys_custom_log_20220309153231.tar.gz
   $ ll result/2022-03-09-15-31/
   total 889M
   -rw-r--r--. 1 root root 889M Mar  9 15:32 openEuler-aarch64.iso
   -rw-r--r--. 1 root root   87 Mar  9 15:32 openEuler-aarch64.iso.sha256
   ```

   * Docker:

   ```shell
   $ sudo rm -rf /opt/imageTailor/result
   $ sudo ./mkdliso -p docker -c custom/cfg_docker
   ......
   Complete release iso file at: result/2023-03-09-15-31/docker.aarch64.tar.xz
   move all mkdliso log file to result/log/sys_custom_log_20230309153231.tar.gz
   $ ls result/2023-03-09-15-31/
   docker.aarch64.tar.xz
   docker_binary.rpmlist
   docker_source.rpmlist
   docker.aarch64.tar.xz.sha256sum
   ```

   * EMB\_rootfs:

   ```shell
   $ sudo rm -rf /opt/imageTailor/result
   $ sudo ./mkdliso -p EMB_rootfs -c custom/cfg_EMB_rootfs
   ......
   Complete release iso file at: result/2023-02-20-18-13/openEuler-image-qemu-aarch64-20230220181343.rootfs.cpio.gz
   move all mkdliso log file to result/log/sys_custom_log_20230220181343.tar.gz
   $ ls result/2023-02-20-18-13/
   openEuler-image-qemu-aarch64-20230220181343.rootfs.cpio.gz
   openEuler-image-qemu-aarch64-20230220181343.rootfs.cpio.gz.sha256
   ```

   * QCOW2:

   ```shell
   $ sudo rm -rf /opt/imageTailor/result
   $ sudo ./mkdliso -p qcow2 -c custom/cfg_qcow2
   ......
   create qcow2 success
   $ ls result/2023-05-23-15-29/
   openEuler_aarch64.qcow2
   openEuler_aarch64.qcow2.sha256sum
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/community_tools/image_tailor/imagetailor_user_guide.md
---
# imageTailor 使用指南

## 简介

操作系统除内核外，还包含各种功能的外围包。通用操作系统包含较多外围包，提供了丰富的功能，但是这也带来了一些影响：

* 占用资源（内存、磁盘、CPU 等）多，导致系统运行效率低
* 很多功能用户不需要，增加了开发和维护成本

因此，openEuler 提供了 imageTailor 镜像裁剪定制工具。用户可以根据需求裁剪操作系统镜像中不需要的外围包，或者添加所需的业务包或文件。该工具主要提供了以下功能：

* 系统包裁剪定制：用户可以选择默认安装以及裁剪的rpm，也支持用户裁剪定制系统命令、库、驱动。
* 系统配置定制：用户可以配置主机名、启动服务、时区、网络、分区、加载驱动、版本号等。
* 用户文件定制：支持用户添加定制文件到系统镜像中。

## 安装工具

本节以 openEuler 22.03 LTS SP4 版本 AArch64 架构为例，说明安装方法。

### 软硬件要求

安装和运行 imageTailor 需要满足以下软硬件要求：

* 机器架构为 AArch64。

* 操作系统为 openEuler 22.03 LTS SP4（该版本内核版本为 5.10，python 版本为 3.9，满足工具要求）。

* 运行工具的机器根目录 '/' 需要 40 GB 以上空间。

* python 版本 3.9 及以上。

* kernel 内核版本 5.10 及以上。

* 关闭 SElinux 服务。

  ```shell
  $ sudo setenforce 0
  $ getenforce 
  Permissive
  ```

### 获取安装包

安装和使用 imageTailor 工具，首先需要下载 openEuler 发布件。

1. 获取 ISO 镜像文件和对应的校验文件。

   镜像必须为 everything 版本，此处假设存放在 root 目录，参考命令如下：

   ```shell
   $ sudo wget https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/aarch64/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso -O /root/temp/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso
   $ sudo wget https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/aarch64/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso.sha256sum -O /root/temp/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso.sha256sum
   ```

2. 获取 sha256sum 校验文件中的校验值。

   ```shell
   $ sudo cat /root/temp/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso.sha256sum
   ```

3. 计算 ISO 镜像文件的校验值。

   ```shell
   $ sudo sha256sum /root/temp/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso
   ```

4. 对比上述 sha256sum 文件的检验值和 ISO 镜像的校验值，如果两者相同，说明文件完整性检验成功。否则说明文件完整性被破坏，需要重新获取文件。

### 安装 imageTailor

此处以 openEuler 22.03 LTS SP4 版本的 AArch64 架构为例，介绍如何安装 imageTailor 工具。

1. 确认机器已经安装操作系统 openEuler 22.03 LTS SP4（ imageTailor 工具的运行环境）。

   ```shell
   $ cat /etc/openEuler-release
   openEuler release 22.03 LTS SP4
   ```

2. 使用 root 权限，创建文件 /etc/yum.repos.d/local.repo，配置对应 yum 源。配置内容参考如下，其中 baseurl 是用于挂载 ISO 镜像的目录：

   ```shell
   [local]
   name=local
   baseurl=file:///root/imageTailor_mount
   gpgcheck=0
   enabled=1
   ```

3. 使用 root 权限，挂载光盘镜像到 /root/imageTailor\_mount 目录（请与上述 repo 文件中配置的 baseurl 保持一致，且建议该目录的磁盘空间大于 20 GB）作为 yum 源，参考命令如下：

   ```shell
   $ sudo mkdir /root/imageTailor_mount
   $ sudo mount -o loop /root/temp/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso /root/imageTailor_mount/
   ```

4. 使 yum 源生效：

   ```shell
   $ yum clean all
   $ sudo yum makecache
   ```

5. 使用 root 权限，安装 imageTailor 裁剪工具：

   ```shell
   $ sudo yum install -y imageTailor
   ```

6. 使用 root 权限，确认工具已安装成功。

   ```shell
   $ cd /opt/imageTailor/
   $ sudo ./mkdliso -h
   -------------------------------------------------------------------------------------------------------------
   Usage: mkdliso -p product_name -c configpath [--minios yes|no|force] [-h] [--sec]
   Options:
       -p,--product     Specify the product to make, check custom/cfg_yourProduct.
       -c,--cfg-path    Specify the configuration file path, the form should be consistent with custom/cfg_xxx
       --minios         Make minios: yes|no|force
       --sec            Perform security hardening
       -h,--help        Display help information

   Example:
       command:
       ./mkdliso -p openEuler -c custom/cfg_openEuler --sec
       ./mkdliso -p docker -c custom/cfg_docker
       ./mkdliso -p EMB_rootfs -c custom/cfg_EMB_rootfs
       ./mkdliso -p qcow2 -c custom/cfg_qcow2
       help:
       ./mkdliso -h
   -------------------------------------------------------------------------------------------------------------
   ```

### 目录介绍

imageTailor 工具安装完成后，工具包的目录结构如下：

openEuler产品：

```shell
[imageTailor]
    |-[custom]
        |-[cfg_openEuler]
            |-[usr_file]            // 存放用户添加的文件
            |-[usr_install]         // 存放用户的 hook 脚本
                |-[all]
                |-[conf]
                |-[hook]
            |-[cmd.conf]            // 配置 ISO 镜像默认使用的命令和库
            |-[rpm.conf]            // 配置 ISO 镜像默认安装的 RPM 包和驱动列表
            |-[security_s.conf]     // 配置安全加固策略
            |-[sys.conf]            // 配置 ISO 镜像系统参数
    |-[kiwi]                        // imageTailor 基础配置
    |-[repos]                       // RPM 源，制作 ISO 镜像需要的 RPM 包
    |-[security-tool]               // 安全加固工具
    |-mkdliso                       // 制作 ISO 镜像的可执行脚本
```

docker产品：

```shell
[imageTailor]
    |-[custom]
        |-[cfg_docker]
            |-[config.xml]          // 配置 ISO 镜像默认安装的 RPM 包和源等配置
            |-[env.pm]
            |-[group]
            |-[images.sh]           // 裁剪定制脚本
            |-[passwd]
    |-[kiwi]                        // imageTailor 基础配置
    |-[repos]                       // RPM 源，制作 ISO 镜像需要的 RPM 包
    |-[security-tool]               // 安全加固工具
    |-mkdliso                       // 制作 ISO 镜像的可执行脚本
```

EMB\_rootfs产品：

```shell
[imageTailor]
    |-[custom]
        |-[cfg_EMB_rootfs]
            |-[usr_install]         // 存放用户的 hook 脚本
                |-[conf]
                    |-[isopackage.sdf]
                    |-[menu.lst]
                    |-[modules]
            |-[cmd.conf]            // 配置 ISO 镜像默认使用的命令和库
            |-[rpm.conf]            // 配置 ISO 镜像默认安装的 RPM 包和驱动列表
            |-[security_s.conf]     // 配置安全加固策略
            |-[sys.conf]            // 配置 ISO 镜像系统参数
    |-[kiwi]                        // imageTailor 基础配置
    |-[repos]                       // RPM 源，制作 ISO 镜像需要的 RPM 包
    |-[security-tool]               // 安全加固工具
    |-mkdliso                       // 制作 ISO 镜像的可执行脚本
```

qcow2产品：

```shell
[imageTailor]
    |-[custom]
        |-[cfg_qcow2]
            |-[bin]                 // 命令脚本
                |-[create-image]    // 镜像制作入口
                |-[source_files]    // 脚本调用入口
            |-[config]              // 配置
                |-[grub.cfg]        // Grub配置
                |-[repo]            // repo源
                |-[root_pwd]        // root密钥
                |-[rpmlist]         // 软件包列表
            |-[hooks]               // hook脚本文件
            |-[lib]                 // 通用脚本
            |-[misc]                // 公共脚本
            |-[template]            
    |-[kiwi]                        // imageTailor 基础配置
    |-[repos]                       // RPM 源，制作 ISO 镜像需要的 RPM 包
    |-[security-tool]               // 安全加固工具
    |-mkdliso                       // 制作 ISO 镜像的可执行脚本
```

## 定制系统

本章介绍使用 imageTailor 工具将业务 RPM 包、自定义文件、驱动、命令和文件打包至目标 ISO 镜像。

### 总体流程

使用 imageTailor 工具定制系统的流程请参见下图：

![](./figures/flowchart.png)

各流程含义如下：

* 检查软硬件环境：确认制作 ISO 镜像的机器满足软硬件要求。

* 定制业务包：包括添加 RPM 包（包括业务 RPM 包、命令、驱动、库文件）和添加文件（包括自定义文件、命令、驱动、库文件）

  * 添加业务 RPM 包：用户可以根据需要，添加 RPM 包到 ISO 镜像。具体要求请参见 [安装工具](#安装工具) 章节。
  * 添加自定义文件：若用户希望在目标 ISO 系统安装或启动时，能够进行自定义的硬件检查、系统配置检查、驱动安装等操作，可编写自定义文件，并打包到 ISO 镜像。
  * 添加驱动、命令、库文件：当 openEuler 的 RPM 包源未包含用户需要的驱动、命令或库文件时，可以使用 imageTailor 工具将对应驱动、命令或库文件打包至 ISO 镜像。

* 配置系统参数

  * 配置主机参数：为了确保 ISO 镜像安装和启动成功，需要配置主机参数。
  * 配置分区：用户可以根据业务规划配置业务分区，同时可以调整系统分区。
  * 配置网络：用户可以根据需要配置系统网络参数，例如：网卡名称、IP 地址、子网掩码。
  * 配置初始密码：为了确保 ISO 镜像安装和启动成功，需要配置 root 初始密码和 grub 初始密码。
  * 配置内核参数：用户可以根据需求配置内核的命令行参数。

* 配置安全加固策略

  imageTailor 提供了默认地安全加固策略。用户可以根据业务需要，通过编辑 security\_s.conf  对系统进行二次加固（仅在系统 ISO 镜像定制阶段），具体的操作方法请参见 《 [安全加固指南](https://docs.openeuler.org/zh/docs/22.03_LTS_SP4/docs/SecHarden/secHarden.html) 》。

* 制作操作系统 ISO 镜像

  使用 imageTailor 工具制作操作系统 ISO 镜像。

### 定制业务包

用户可以根据业务需要，将业务 RPM 包、自定义文件、驱动、命令和库文件打包至目标 ISO 镜像。

#### 配置本地 repo 源

定制 ISO 操作系统镜像，必须在 /opt/imageTailor/repos/euler\_base/ 目录配置 repo 源。本节主要介绍配置本地 repo 源的方法。

1. 下载 openEuler 发布的 ISO （必须使用 openEuler 发布 everything 版本镜像 的 RPM 包）。

   ```shell
   $ cd /opt
   $ wget https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/aarch64/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso
   ```

2. 创建挂载目录 /opt/openEuler\_repo ，并挂载 ISO 到该目录 。

   ```shell
   $ sudo mkdir -p /opt/openEuler_repo
   $ sudo mount openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso /opt/openEuler_repo
   mount: /opt/openEuler_repo: WARNING: source write-protected, mounted read-only.
   ```

3. 拷贝 ISO 中的 RPM 包到 /opt/imageTailor/repos/euler\_base/ 目录下。

   ```shell
   $ sudo rm -rf /opt/imageTailor/repos/euler_base && sudo mkdir -p /opt/imageTailor/repos/euler_base
   $ sudo cp -ar /opt/openEuler_repo/Packages/* /opt/imageTailor/repos/euler_base
   $ sudo chmod -R 644 /opt/imageTailor/repos/euler_base
   $ sudo ls /opt/imageTailor/repos/euler_base|wc -l
   2577
   $ sudo umount /opt/openEuler_repo && sudo rm -rf /opt/openEuler_repo
   $ cd /opt/imageTailor
   ```

#### 添加文件

用户可以根据需要添加文件到 ISO 镜像，此处的文件类型可以是用户自定义文件、驱动、命令、库文件。用户只需要将文件放至 /opt/imageTailor/custom/cfg\_openEuler/usr\_file 目录下即可。

##### 注意事项

* 命令必须具有可执行权限，否则 imageTailor 工具无法将该命令打包至 ISO 中。

* 存放在 /opt/imageTailor/custom/cfg\_openEuler/usr\_file 目录下的文件，会生成在 ISO 根目录下，所以文件的目录结构必须是从根目录开始的完整路径，以便 imageTailor 工具能够将该文件放至正确的目录下。

  例如：假设希望文件 file1 在 ISO 的 /opt 目录下，则需要在 usr\_file 目录下新建 opt 目录，再将 file1 文件拷贝至 opt 目录。如下：

  ```shell
  $ pwd
   /opt/imageTailor/custom/cfg_openEuler/usr_file

  $ tree
    .
    ├── etc
    │   ├── default
    │   │   └── grub
    │   └── profile.d
    │       └── csh.precmd
    └── opt
        └── file1
    
    4 directories, 3 files
  ```

* 存放在 /opt/imageTailor/custom/cfg\_openEuler/usr\_file 目录下的目录必须是真实路径（例如路径中不包含软链接。可在系统中使用 `realpath` 或 `readlink -f` 命令查询真实路径）。

* 如果需要在系统启动或者安装阶段调用用户提供的脚本，即 hook 脚本，则需要将该文件放在 hook 目录下。

#### 添加 RPM 包

##### 操作流程

用户可以添加 RPM 包（驱动、命令或库文件）到 ISO 镜像，操作步骤如下：

> \[!NOTE]说明
>
> * 下述 rpm.conf 和 cmd.conf 均在 /opt/imageTailor/custom/cfg\_openEuler/ 目录下。
> * 下述 RPM 包裁剪粒度是指 sys\_cut='no' 。裁剪粒度详情请参见 [配置主机参数](#配置主机参数) 。
> * 若没有配置本地 repo 源，请参见 [配置本地 repo 源](#配置本地-repo-源)进行配置。

1. 确认 /opt/imageTailor/repos/euler\_base/ 目录中是否包含需要添加的 RPM 包。

   * 是，请执行步骤 2 。
   * 否，请执行步骤 3 。

2. 在 rpm.conf 的 \<bootstrap> 字段配置该 RPM 包信息。

   * 若为 RPM 包裁剪粒度，则操作完成。
   * 若为其他裁剪粒度，请执行步骤 4 。

3. 用户自己提供 RPM 包，放至 /opt/imageTailor/custom/cfg\_openEuler/usr\_rpm 目录下。如果 RPM 包依赖于其他 RPM 包，也必须将依赖包放至该目录，因为新增 RPM 包需要和依赖 RPM 包同时打包至 ISO 镜像。

   * 若为用户 RPM 包文件裁剪，则执行 4 。
   * 其他裁剪粒度，则操作完成。

4. 请在 rpm.conf 和 cmd.conf 中配置该 RPM 包中要保留的驱动、命令和库文件。如果有要裁剪的普通文件，需要在 cmd.conf 文件中的 \<strip type="delete">\</strip>  区域配置。

##### 配置文件说明

| 对象         | 对应配置文件 | 填写区域                                                     |
| :----------- | :----------- | :----------------------------------------------------------- |
| 添加驱动     | rpm.conf     | \<drivers>      \<file name="{驱动名称}"/>\</drivers>说明：其中驱动名称所在路径为  " /lib/modules/{内核版本号}/kernel/ " 的相对路径 |
| 添加命令     | cmd.conf     | \<strip type="tools">    \<file name="{命令名称}"/>\</strip> |
| 添加库文件   | cmd.conf     | \<strip type="libs">    \<file name="{库文件名称}"/>\</strip> |
| 删除其他文件 | cmd.conf     | \<strip type="delete">    \<file name="{普通文件名称}"/>\</strip>说明：普通文件名称必须包含绝对路径 |

**示例**

* 添加驱动

  ```shell
  <drivers>
          <file name="arch/arm64/crypto/aes-ce-blk.ko"/>
          <file name="arch/arm64/crypto/aes-ce-ccm.ko"/>
          <file name="arch/arm64/crypto/aes-ce-cipher.ko"/>
      ......
  </drivers>
  ```

* 添加命令

  ```shell
  <strip type="tools">
      <file name="aarch64-openEuler-linux-gnu-pkg-config"/>
      <file name="accessdb"/>
      <file name="acyclic"/>
      ......
  </strip>
  ```

* 添加库文件

  ```shell
  <strip type="libs">
      <file name="ANSI_X3.110"/>
      <file name="ARMSCII-8"/>
      <file name="ASMO_449"/>
  </strip>
  ```

* 删除其他文件

  ```shell
  <strip type="delete">
      <file name="/usr/lib/firmware/radeon"/>
      <file name="/usr/lib/firmware/rtlwifi"/>
      <file name="/usr/lib/firmware/RTL8192E"/>
  </strip>
  ```

#### 添加 hook 脚本

hook 脚本由 OS 在启动和安装过程中调用，执行脚本中定义的动作。imageTailor 工具存放 hook 脚本的目录为 custom/cfg\_openEuler/usr\_install/hook，且其下有不同子目录，每个子目录代表 OS 启动或安装的不同阶段，用户根据脚本需要被调用的阶段存放，OS 会在对应阶段调用该脚本。用户可以根据需要存放自定义脚本到指定目录。docker产品不支持添加hook脚本。

##### **脚本命名规则**

用户可自定义脚本名称，必须 "S+数字（至少两位，个位数以0开头）" 开头，数字代表 hook 脚本的执行顺序。脚本名称示例：S01xxx.sh

> \[!NOTE]说明
>
> hook 目录下的脚本是通过 source 方式调用，所以脚本中需要谨慎使用 exit 命令，因为调用 exit 命令之后，整个安装的脚本程序也同步退出了。

##### hook 子目录说明

| hook 子目录           | hook 脚本举例         | hook 执行点                      | 说明                                                         |
| :-------------------- | :---------------------| :------------------------------- | :----------------------------------------------------------- |
| insmod\_drv\_hook       | 无                    | 加载 OS 驱动之后                 | 无                                                           |
| custom\_install\_hook   | S01custom\_install.sh  | 驱动加载完成后（即 insmod\_drv\_hook 执行后） | 用户可以自定义安装过程，不需要使用 OS 默认安装流程。 |
| env\_check\_hook        | S01check\_hw.sh        | 安装初始化之前                   | 初始化之前检查硬件配置规格、获取硬件类型。 |
| set\_install\_ip\_hook   | S01set\_install\_ip.sh  | 安装初始化过程中，配置网络时     | 用户根据自身组网，自定义网络配置。 |
| before\_partition\_hook | S01checkpart.sh       | 在分区前调用                     | 用户可以在分区之前检查分区配置文件是否正确。 |
| before\_setup\_os\_hook  | 无                    | 解压repo之前                     | 用户可以进行自定义分区挂载操作。如果安装包解压的路径不是分区配置中指定的根分区。则需要用户自定义分区挂载，并将解压路径赋值给传入的全局变量。 |
| before\_mkinitrd\_hook  | S01install\_drv.sh     | 执行 mkinitrd 操作之前           | initrd 放在硬盘的场景下，执行 mkinitrd 操作之前的 hook。用户可以进行添加、更新驱动文件等自定义操作。 |
| after\_setup\_os\_hook   | 无                    | 安装完系统之后                   | 用户可以在安装完成之后进行系统文件的自定义操作，包括修改 grub.cfg 等 |
| install\_succ\_hook     | 无                    | 系统安装流程成功结束             | 用户执行解析安装信息，回传安装是否成功等操作。install\_succ\_hook 不可以设置为 install\_break。 |
| install\_fail\_hook     | 无                    | 系统安装失败                     | 用户执行解析安装信息，回传安装是否成功等操作。install\_fail\_hook 不可以设置为 install\_break。 |

### 配置系统参数

开始制作操作系统 ISO 镜像之前，需要配置系统参数，包括主机参数、初始密码、分区、网络、编译参数和系统命令行参数。

#### 配置主机参数

/opt/imageTailor/custom/cfg\_openEuler/sys.conf 文件的 \<sysconfig> \</sysconfig> 区域用于配置系统的常用参数，例如主机名、内核启动参数等。docker产品不支持。

openEuler 提供的默认配置如下，用户可以根据需要进行修改：

```shell
<sysconfig>
    sys_service_enable='ipcc'
    sys_service_disable='cloud-config cloud-final cloud-init-local cloud-init'
    sys_utc='yes'
    sys_timezone=''
    sys_cut='no'
    sys_usrrpm_cut='no'
    sys_hostname='Euler'
    sys_usermodules_autoload=''
    sys_gconv='GBK'
</sysconfig>
```

配置中的各参数含义如下：

* sys\_service\_enable

  可选配置。OS 默认启用的服务，多个服务请以空格分开。如果用户不需要新增系统服务，请保持默认值，默认值为 ipcc 。配置时请注意：

  * 只能在默认配置的基础上增加系统服务，不能删减系统服务。
  * 可以配置业务相关的服务，但是需要 repo 源中包含业务 RPM 包。
  * 默认只开启该参数中配置的服务，如果服务依赖其他服务，需要将被依赖的服务也配置在该参数中。

* sys\_service\_disable

  可选配置。禁止服务开机自启动的服务，多个服务请以空格分开。如果用户没有需要禁用的系统服务，请修改该参数为空。

* sys\_utc

  必选配置。是否采用 UTC 时间。yes 表示采用，no 表示不采用，默认值为 yes 。

* sys\_timezone

  可选配置。设置时区，即该单板所处的时区。可配置的范围为 openEuler 支持的时区，可通过 /usr/share/zoneinfo/zone.tab 文件查询。

* sys\_cut

  必选配置。是否裁剪 RPM 包。可配置为 yes、no 或者 debug 。yes 表示裁剪，no 表示不裁剪（仅安装 rpm.conf 中的 RPM 包），debug 表示裁剪但会保留 `rpm` 命令方便安装后定制。默认值为 no 。

  > \[!NOTE]说明
  >
  > * imageTailor 工具会先安装用户添加的 RPM 包，再删除 cmd.conf
  >   中 \<script type="delete">\</script> 区域的文件，最后删除
  >   cmd.conf 和 rpm.conf 中未配置的命令、库和驱动。
  > * sys\_cut='yes' 时，imageTailor 工具不支持 `rpm` 命令的安装，即使在 rpm.conf 中配置了也不生效。

* sys\_usrrpm\_cut

  必选配置。是否裁剪用户添加到 /opt/imageTailor/custom/cfg\_openEuler/usr\_rpm 目录下的 RPM 包。yes 表示裁剪，no 表示不裁剪。默认值为 no 。

  * sys\_usrrpm\_cut='yes' ：imageTailor 工具会先安装用户添加的 RPM 包，然后删除 cmd.conf 中 \<script type="delete">\</script> 区域配置的文件，最后删除 cmd.conf 和 rpm.conf 中未配置的命令、库和驱动。

  * sys\_usrrpm\_cut='no' ：imageTailor 工具会安装用户添加的 RPM 包，不删除用户 RPM 包中的文件。

* sys\_hostname

  必选配置。主机名。大批量部署 OS 时，部署成功后，建议修改每个节点的主机名，确保各个节点的主机名不重复。

  主机名要求：字母、数字、"-" 的组合，首字母必须是字母或数字。字母支持大小写。字符个数不超过 63 。默认值为 Euler 。

* sys\_usermodules\_autoload

  可选配置。系统启动阶段加载的驱动，配置该参数时，不需要填写后缀 .ko 。如果有多个驱动，请以空格分开。默认为空，不加载额外驱动。

* sys\_gconv

  可选配置。该参数用于定制 /usr/lib/gconv, /usr/lib64/gconv ，配置取值为：

  * null/NULL：表示不配置。如果裁剪系统（sys\_cut=“yes”），则/usr/lib/gconv 和 /usr/lib64/gconv 会被删除。
  * all/ALL：不裁剪 /usr/lib/gconv 和 /usr/lib64/gconv 。
  * xxx,xxx: 保留 /usr/lib/gconv 和 /usr/lib64/gconv 目录下对应的文件。若需要保留多个文件，可用 "," 分隔。

* sys\_man\_cut

  可选配置。配置是否裁剪 man 文档。yes 表示裁剪，no 表示不裁剪。默认值为 yes 。

> \[!NOTE]说明
>
> sys\_cut 和 sys\_usrrpm\_cut 同时配置时，sys\_cut 优先级更高，即遵循如下原则：
>
> * sys\_cut='no'
>
>   无论 sys\_usrrpm\_cut='no' 还是 sys\_usrrpm\_cut='yes' ，都为系统 RPM 包裁剪粒度，即imageTailor 会安装 repo 源中的 RPM 包和 usr\_rpm 目录下的 RPM 包，但不会裁剪 RPM 包中的文件。即使用户不需要这些 RPM 包中的部分文件，imageTailor 也不会进行裁剪。
>
> * sys\_cut='yes'
>
>   1） sys\_usrrpm\_cut='no'
>
>   系统 RPM 包文件裁剪粒度：imageTailor 会根据用户配置，裁剪 repo 源中 RPM 包的文件。
>
>   2）sys\_usrrpm\_cut='yes'
>
>   系统和用户 RPM 包文件裁剪粒度：imageTailor 会根据用户的配置，裁剪 repo 源和 usr\_rpm 目录中 RPM 包的文件。

#### 配置初始密码

操作系统安装时，必须具有 root 初始密码和 grub 初始密码，否则裁剪得到的 ISO 在安装后无法使用 root 帐号进行登录。本节介绍配置初始密码的方法。docker产品不支持。

> \[!NOTE]说明
>
> root 初始密码和 grub 初始密码，必须由用户自行配置。

##### 配置 root 初始密码

###### 简介

root 初始密码保存在 "/opt/imageTailor/custom/cfg\_openEuler/rpm.conf" 中，用户通过修改该文件配置 root 初始密码。

> \[!NOTE]说明
>
> * 若使用 `mkdliso` 命令制作 ISO 镜像时需要使用  --minios yes/force 参数（制作在系统安装时进行系统引导的 initrd），则还需要在 /opt/imageTailor/kiwi/minios/cfg\_minios/rpm.conf 中填写相应信息。

/opt/imageTailor/custom/cfg\_openEuler/rpm.conf 中 root 初始密码的默认配置如下，需要用户自行添加：

```Conf
<users group="root">
    <user pwd="${pwd}" home="/root" name="root"/>
</users>
```

各参数含义如下：

* group：用户所属组。
* pwd：用户初始密码的加密密文，加密算法为 SHA-512。${pwd} 需要替换成用户实际的加密密文。
* home：用户的家目录。
* name：需要配置用户的用户名。

###### 修改方法

用户在制作 ISO 镜像前需要修改 root 用户的初始密码，这里给出设置 root 初始密码的方法（需使用 root 权限）：

1. 添加用于生成密码的用户，此处假设 testUser。

   ```shell
   $ sudo useradd testUser
   ```

2. 设置 testUser 用户的密码。参考命令如下，根据提示设置密码：

   ```shell
   $ sudo passwd testUser
   Changing password for user testUser.
   New password: 
   Retype new password: 
   passwd: all authentication tokens updated successfully.
   ```

3. 查看 /etc/shadow 文件，testUser 后的内容（两个 : 间的字符串）即为加密后的密码。

   ```shell script
   $ sudo cat /etc/shadow | grep testUser
   testUser:$6$YkX5uFDGVO1VWbab$jvbwkZ2Kt0MzZXmPWy.7bJsgmkN0U2gEqhm9KqT1jwQBlwBGsF3Z59heEXyh8QKm3Qhc5C3jqg2N1ktv25xdP0:19052:0:90:7:35::
   ```

4. 拷贝上述加密密码替换 /opt/imageTailor/custom/cfg\_openEuler/rpm.conf 中的 pwd 字段，如下所示：

   ```shell script
   <users group="root">
       <user pwd="$6$YkX5uFDGVO1VWbab$jvbwkZ2Kt0MzZXmPWy.7bJsgmkN0U2gEqhm9KqT1jwQBlwBGsF3Z59heEXyh8QKm3Qhc5C3jqg2N1ktv25xdP0" home="/root" name="root"/>
   </users>
   ```

5. 若使用 `mkdliso` 命令制作 ISO 镜像时需要使用  --minios yes/force 参数，请修改 /opt/imageTailor/kiwi/minios/cfg\_minios/rpm.conf 中对应用户的 pwd 字段。

   ```shell script
   <users group="root">
       <user pwd="$6$YkX5uFDGVO1VWbab$jvbwkZ2Kt0MzZXmPWy.7bJsgmkN0U2gEqhm9KqT1jwQBlwBGsF3Z59heEXyh8QKm3Qhc5C3jqg2N1ktv25xdP0" home="/root" name="root"/>
   </users>
   ```

##### 配置 grub 初始密码

grub 初始密码保存在 /opt/imageTailor/custom/cfg\_openEuler/usr\_file/etc/default/grub 中，用户通过修改该文件配置 grub 初始密码。如果未配置 grub 初始密码，制作 ISO 镜像会失败。

> \[!NOTE]说明
>
> * 配置 grub 初始密码需要使用 root 权限。
>
> * grub 密码对应的默认用户为 root 。
>
> * 系统中需有 grub2-set-password 命令，若不存在，请提前安装该命令。

1. 执行如下命令，根据提示设置 grub 密码：

   ```shell
   $ sudo grub2-set-password -o ./
   Enter password: 
   Confirm password: 
   grep: .//grub.cfg: No such file or directory
   WARNING: The current configuration lacks password support!
   Update your configuration with grub2-mkconfig to support this feature.
   ```

2. 命令执行完成后，会在当前目录生成 user.cfg 文件，grub.pbkdf2.sha512 开头的内容即 grub 加密密码。

   ```shell
   $ sudo cat user.cfg 
   GRUB2_PASSWORD=grub.pbkdf2.sha512.10000.CE285BE1DED0012F8B2FB3DEA38782A5B1040FEC1E49D5F602285FD6A972D60177C365F1
   B5D4CB9D648AD4C70CF9AA2CF9F4D7F793D4CE008D9A2A696A3AF96A.0AF86AB3954777F40D324816E45DD8F66CA1DE836DC7FBED053DB02
   4456EE657350A27FF1E74429546AD9B87BE8D3A13C2E686DD7C71D4D4E85294B6B06E0615
   ```

3. 复制上述密文，并在 /opt/imageTailor/custom/cfg\_openEuler/usr\_file/etc/default/grub 文件中增加如下配置：

   ```shell
   GRUB_PASSWORD="grub.pbkdf2.sha512.10000.CE285BE1DED0012F8B2FB3DEA38782A5B1040FEC1E49D5F602285FD6A972D60177C365F1
   B5D4CB9D648AD4C70CF9AA2CF9F4D7F793D4CE008D9A2A696A3AF96A.0AF86AB3954777F40D324816E45DD8F66CA1DE836DC7FBED053DB02
   4456EE657350A27FF1E74429546AD9B87BE8D3A13C2E686DD7C71D4D4E85294B6B06E0615"
   ```

#### 配置分区

若用户想调整系统分区或业务分区，可以通过修改 /opt/imageTailor/custom/cfg\_openEuler/sys.conf 文件中的 \<HDpartitions> 实现。docker产品不支持。

> \[!NOTE]说明
>
> * 系统分区：存放操作系统的分区
> * 业务分区：存放业务数据的分区
> * 差别：在于存放的内容，而每个分区的大小、挂载路径和文件系统类型都不是区分业务分区和系统分区的依据。
> * 配置分区为可选项，用户也可以在安装 OS 之后，手动配置分区

\<HDpartitions> 的配置格式为：

hd 磁盘号   挂载路径    分区大小    分区类型    文件系统类型  \[二次格式化标志位]

其默认配置如下：

```shell
<HDpartitions>
hd0    /boot        512M      primary     ext4     yes
hd0    /boot/efi    200M      primary     vfat     yes
hd0    /            30G       primary     ext4
hd0    -            -         extended    -
hd0    /var         1536M     logical     ext4
hd0    /home        max       logical     ext4
</HDpartitions>
```

各参数含义如下：

* hd 磁盘号
  磁盘的编号。请按照 hdx 的格式填写，x 指第 x 块盘。

  > \[!NOTE]说明
  >
  > 分区配置只在被安装机器的磁盘能被识别时才有效。

* 挂载路径
  指定分区挂载的路径。用户既可以配置业务分区，也可以对默认配置中的系统分区进行调整。如果不挂载，则设置为 '-'。

  > \[!NOTE]说明
  >
  > * 分区配置中必须有 '/' 挂载路径。其他的请用户自行调整。
  > * 采用 UEFI 引导时，在 x86\_64 的分区配置中必须有 '/boot' 挂载路径，在 AArch64 的分区配置中必须有 '/boot/efi' 挂载路径。

* 分区大小
  分区大小的取值有以下四种：

  * G/g：指定以 GB 为单位的分区大小，例如：2G。
  * M/m：指定以 MB 为单位的分区大小，例如：300M。
  * T/t：指定以 TB 为单位的分区大小，例如：1T。
  * MAX/max：指定将硬盘上剩余的空间全部用来创建一个分区。只能在最后一个分区配置该值。

  > \[!NOTE]说明
  >
  > * 分区大小不支持小数，如果是小数，请换算成其他单位，调整为整数的数值。例如：不能填写 1.5G，应填写为 1536M。
  > * 分区大小取 MAX/max 值时，剩余分区大小不能超过支持文件系统类型的限制（默认文件系统类型 ext4，限制大小 16T）。

* 分区类型
  分区有以下三种：

  * 主分区： primary
  * 扩展分区：extended（该分区只需配置 hd 磁盘号即可）
  * 逻辑分区：logical

* 文件系统类型
  目前支持的文件系统类型有：ext4、vfat

* 二次格式化标志位
  可选配置，表示二次安装时是否格式化：

  * 是：yes
  * 否：no 。不配置默认为 no 。

  > \[!NOTE]说明
  >
  > 二次格式化是指本次安装之前，磁盘已安装过 openEuler 系统。当前一次安装跟本次安装使用相同的分区表配置（分区大小，挂载点，文件类型）时，该标志位可以配置是否格式化之前的分区，'/boot' 和 '/' 分区除外，每次都会重新格式化。如果目标机器第一次安装，则该标志位不生效，所有指定了文件系统的分区都会进行格式化。

#### 配置网络

系统网络参数保存在 /opt/imageTailor/custom/cfg\_openEuler/sys.conf 中，用户可以通过该文件的\<netconfig-*x*>\</netconfig-*x*> 配置修改目标 ISO 镜像的网络参数，例如：网卡名称、IP地址、子网掩码。docker产品不支持。

sys.conf 中默认的网络配置如下，其中 netconfig-0 代表网卡 eth0。如果需要配置多块网卡，例如eth1，请在配置文件中增加 \<netconfig-1>\</netconfig-1>，并在其中填写网卡 eth1 的各项参数。

```shell
<netconfig-0>
BOOTPROTO="dhcp"
DEVICE="eth0"
IPADDR=""
NETMASK=""
STARTMODE="auto" 
</netconfig-0>
```

各参数含义请参见下表：

* | 参数名称  | 是否必配 | 参数值                                            | 说明                                                         |
  | :-------- | -------- | :------------------------------------------------ | :----------------------------------------------------------- |
  | BOOTPROTO | 是       | none / static / dhcp                              | none：引导时不使用协议，不配地址static：静态分配地址dhcp：使用 DHCP 协议动态获取地址 |
  | DEVICE    | 是       | 如：eth1                                          | 网卡名称                                                     |
  | IPADDR    | 是       | 如：192.168.11.100                                | IP 地址当 BOOTPROTO 参数为 static 时，该参数必配；其他情况下，该参数不用配置 |
  | NETMASK   | 是       | -                                                 | 子网掩码当 BOOTPROTO 参数为 static 时，该参数必配；其他情况下，该参数不用配置 |
  | STARTMODE | 是       | manual / auto / hotplug / ifplugd / nfsroot / off | 启用网卡的方法：manual：用户在终端执行 ifup 命令启用网卡。auto \ hotplug \ ifplugd \ nfsroot：当 OS 识别到该网卡时，便启用该网卡。off：任何情况下，网卡都无法被启用。各参数更具体的说明请在制作 ISO 镜像的机器上执行 `man ifcfg` 命令查看。 |

#### 配置内核参数

为了系统能够更稳定高效地运行，用户可以根据需要修改内核命令行参数。imageTailor 工具制作的 OS 镜像，可以通过修改 /opt/imageTailor/custom/cfg\_openEuler/usr\_file/etc/default/grub 中的 GRUB\_CMDLINE\_LINUX 配置实现内核命令行参数修改。 docker产品、EMB\_rootfs产品和qcow2产品不支持。

GRUB\_CMDLINE\_LINUX 中内核命令行参数的默认配置如下：

```shell
GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0 crashkernel=512M oops=panic softlockup_panic=1 reserve_kbox_mem=16M crash_kexec_post_notifiers panic=3 console=tty0"
```

此处各配置含义如下（其余常见的内核命令行参数请查阅内核相关文档）：

* net.ifnames=0 biosdevname=0

  以传统方式命名网卡。

* crashkernel=512M

  为 kdump 预留的内存空间大小为 512 MB。

* oops=panic panic=3

  内核 oops 时直接 panic，并且 3 秒后重启系统。

* softlockup\_panic=1

  在检测到软死锁（soft-lockup）时让内核 panic。

* reserve\_kbox\_mem=16M

  为 kbox 预留的内存空间大小为 16 MB。

* console=tty0

  指定第一个虚拟控制台的输出设备为 tty0。

* crash\_kexec\_post\_notifiers

  系统 crash 后，先调用注册到 panic 通知链上的函数，再执行 kdump。

### 制作系统

操作系统定制完成后，可以通过 mkdliso 脚本制作系统镜像文件。 imageTailor 制作的 OS 为 ISO 格式的镜像文件。

#### 命令介绍

##### 命令格式

**mkdliso \[-p openEuler|docker|EMB\_rootfs|qcow2] \[-c custom/cfg\_openEuler|custom/cfg\_docker|custom/cfg\_EMB\_rootfs|custom/cfg\_qcow2] \[--minios yes|no|force] \[--sec] \[-h]**

##### 参数说明

| 参数名称 | 是否必选 | 参数含义                                                     | 取值说明                                                     |
| -------- | -------- | ------------------------------------------------------- | ------------------------------------------------------------ |
| -p       | 是       | 设置产品名称                                                | openEuler | docker | EMB\_rootfs | qcow2                    |
| -c       | 是       | 指定配置文件的相对路径                                       | custom/cfg\_openEuler | custom/cfg\_docker | custom/cfg\_EMB\_rootfs | custom/cfg\_qcow2 |
| --minios | 否       | 制作在系统安装时进行系统引导的 initrd                        | 默认为 yesyes：第一次执行命令时会制作 initrd，之后执行命令会判断 'usr\_install/boot' 目录下是否存在 initrd（sha256 校验）。如果存在，就不重新制作 initrd，否则制作 initrd 。no：不制作 initrd，采用原有方式，系统引导和运行使用的 initrd 相同。force：强制制作 initrd，不管 'usr\_install/boot' 目录下是否存在 initrd。 |
| --sec    | 否       | 是否对生成的 ISO 进行安全加固如果用户不输入该参数，则由此造成的安全风险由用户承担 | 无                                                           |
| -h       | 否       | 获取帮助信息                                                 | 无                                                           |

#### 制作指导

使用 mkdliso 制作 ISO 镜像的操作步骤如下：

> \[!NOTE]说明
>
> * mkdliso 所在的绝对路径中不能有空格，否则会导致制作 ISO 失败。
> * 制作 ISO 的环境中，umask 的值必须设置为 0022。

1. 使用 root 权限，执行 mkdliso 命令，生成 ISO 镜像文件。参考命令如下：

   ```shell
   # sudo /opt/imageTailor/mkdliso -p openEuler -c custom/cfg_openEuler --sec
   # sudo /opt/imageTailor/mkdliso -p docker -c custom/cfg_docker
   # sudo /opt/imageTailor/mkdliso -p EMB_rootfs -c custom/cfg_EMB_rootfs
   # sudo /opt/imageTailor/mkdliso -p qcow2 -c custom/cfg_qcow2
   ```

   命令执行完成后，制作出的新文件在 /opt/imageTailor/result/{日期} 目录下，包括：\
   openEuler产品：openEuler-aarch64.iso 和 openEuler-aarch64.iso.sha256\
   EMB\_rootfs产品：openEuler-image-qemu-\*.rootfs.cpio.gz 和 openEuler-image-qemu-\*.rootfs.cpio.gz.sha256\
   docker产品：docker.\*.tar.xz 和 docker.\*.tar.xz.sha256sum 和 docker\_source.rpmlist 和 docker\_binary.rpmlist\
   qcow2产品: openEuler\_\*.qcow2 和 openEuler\_\*.qcow2.sha256sum

2. 验证 ISO 镜像文件的完整性。此处假设日期为 2022-03-21-14-48 。

   ```shell
   $ cd /opt/imageTailor/result/2022-03-21-14-48/
   $ sha256sum -c openEuler-aarch64.iso.sha256
   ```

   回显如下，表示 ISO 镜像文件完整，ISO 制作完成。

   ```text
   openEuler-aarch64.iso: OK
   ```

   若回显如下，表示镜像不完整，说明 ISO 镜像文件完整性被破坏，需要重新制作。

   ```shell
   openEuler-aarch64.iso: FAILED
   sha256sum: WARNING: 1 computed checksum did NOT match
   ```

3. 查看日志

   镜像制作完成后，可以根据需要（例如制作出错时）查看日志。第一次制作镜像时，对应的日志和安全加固日志被压缩为一个 tar 包（日志的命名格式为：sys\_custom\_log\_{*日期* }.tar.gz），存放在 result/log 目录下。该目录只保留最近时间的 50 个日志压缩包，超过 50 个时会对旧文件进行覆盖。

### 裁剪时区

定制完成的 ISO 镜像安装后，用户可以根据需求裁剪 openEuler 系统支持的时区。本节介绍裁剪时区的方法。

openEuler 操作系统支持的时区信息存放在时区文件夹 /usr/share/zoneinfo 下，可通过如下命令查看：

```shell
$ ls /usr/share/zoneinfo/
Africa/  America/  Asia/  Atlantic/  Australia/  Etc/  Europe/
Pacific/  zone.tab
```

其中每个子文件夹代表一个 Area ，当前 Area 包括：大陆、海洋以及 Etc 。每个 Area 文件夹内部则包含了隶属于其的 Location 。一个 Location 一般为一座城市或者一个岛屿。

所有时区均以 Area/Location 的形式来表示，比如中国大陆南部使用北京时间，其时区为 Asia/Shanghai（Location 并不一定会使用首都）。对应的，其时区文件为：

```text
/usr/share/zoneinfo/Asia/Shanghai
```

若用户希望裁剪某些时区，则只需将对应的时区文件删除即可。

### 定制示例

本节给出使用 imageTailor 工具定制一个 ISO 操作系统镜像的简易方案，方便用户了解制作的整体流程。

1. 检查制作 ISO 所在环境是否满足要求。

   ```shell
   $ cat /etc/openEuler-release
   openEuler release 22.03 LTS SP4
   ```

2. 确保根目录有 40 GB 以上空间。

   ```shell
   $ df -h
   Filesystem                  Size  Used Avail Use% Mounted on
   ......
   /dev/vdb                    196G   28K  186G   1% /
   ```

3. 安装 imageTailor 裁剪工具。具体安装方法请参见 [安装工具](#安装工具) 章节。

   ```shell
   $ sudo yum install -y imageTailor
   $ ll /opt/imageTailor/
   total 88K
   drwxr-xr-x.  3 root root 4.0K Mar  3 08:00 custom
   drwxr-xr-x. 10 root root 4.0K Mar  3 08:00 kiwi
   -r-x------.  1 root root  69K Mar  3 08:00 mkdliso
   drwxr-xr-x.  2 root root 4.0K Mar  9 14:48 repos
   drwxr-xr-x.  2 root root 4.0K Mar  9 14:48 security-tool
   ```

4. 配置本地 repo 源。

   ```shell
   $ wget https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/aarch64/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso
   $ sudo mkdir -p /opt/openEuler_repo
   $ sudo mount openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso /opt/openEuler_repo
   mount: /opt/openEuler_repo: WARNING: source write-protected, mounted read-only.
   $ sudo rm -rf /opt/imageTailor/repos/euler_base && sudo mkdir -p /opt/imageTailor/repos/euler_base
   $ sudo cp -ar /opt/openEuler_repo/Packages/* /opt/imageTailor/repos/euler_base
   $ sudo chmod -R 644 /opt/imageTailor/repos/euler_base
   $ sudo ls /opt/imageTailor/repos/euler_base|wc -l
   2577
   $ sudo umount /opt/openEuler_repo && sudo rm -rf /opt/openEuler_repo
   $ cd /opt/imageTailor
   ```

5. 修改 grub/root 密码

   以下 ${pwd} 的实际内容请参见 [配置初始密码](#配置初始密码) 章节生成并替换。

   openEuler:

   ```shell
   $ cd /opt/imageTailor/
   $ sudo vi custom/cfg_openEuler/usr_file/etc/default/grub
   GRUB_PASSWORD="${pwd1}"
   $ 
   $ sudo vi kiwi/minios/cfg_minios/rpm.conf
   <users group="root">
       <user pwd="${pwd2}" home="/root" name="root"/>
   </users>
   $ 
   $ sudo vi custom/cfg_openEuler/rpm.conf
   <users group="root">
       <user pwd="${pwd2}" home="/root" name="root"/>
   </users>
   ```

   docker:
   无grub/root 密码

   EMB\_rootfs:

   ```shell
   $ cd /opt/imageTailor/
   $ sudo vi custom/cfg_EMB_rootfs/rpm.conf
   <users group="root">
       <user pwd="${pwd2}" home="/root" name="root"/>
   </users>
   ```

   qcow2:

   ```shell
   $ cd /opt/imageTailor/
   $ sudo vi custom/cfg_qcow2/config/root_pwd
   ${pwd2}
   ```

6. 执行裁剪命令。

openEuler:

```shell
$ sudo rm -rf /opt/imageTailor/result
$ sudo ./mkdliso -p openEuler -c custom/cfg_openEuler --minios force
......
Complete release iso file at: result/2022-03-09-15-31/openEuler-aarch64.iso
move all mkdliso log file to result/log/sys_custom_log_20220309153231.tar.gz
$ ll result/2022-03-09-15-31/
total 889M
-rw-r--r--. 1 root root 889M Mar  9 15:32 openEuler-aarch64.iso
-rw-r--r--. 1 root root   87 Mar  9 15:32 openEuler-aarch64.iso.sha256
```

docker:

```shell
$ sudo rm -rf /opt/imageTailor/result
$ sudo ./mkdliso -p docker -c custom/cfg_docker
......
Complete release iso file at: result/2023-03-09-15-31/docker.aarch64.tar.xz
move all mkdliso log file to result/log/sys_custom_log_20230309153231.tar.gz
$ ls result/2023-03-09-15-31/
docker.aarch64.tar.xz
docker_binary.rpmlist
docker_source.rpmlist
docker.aarch64.tar.xz.sha256sum
```

EMB\_rootfs:

```shell
$ sudo rm -rf /opt/imageTailor/result
$ sudo ./mkdliso -p EMB_rootfs -c custom/cfg_EMB_rootfs
......
Complete release iso file at: result/2023-02-20-18-13/openEuler-image-qemu-aarch64-20230220181343.rootfs.cpio.gz
move all mkdliso log file to result/log/sys_custom_log_20230220181343.tar.gz
$ ls result/2023-02-20-18-13/
openEuler-image-qemu-aarch64-20230220181343.rootfs.cpio.gz
openEuler-image-qemu-aarch64-20230220181343.rootfs.cpio.gz.sha256
```

qcow2:

```shell
$ sudo rm -rf /opt/imageTailor/result
$ sudo ./mkdliso -p qcow2 -c custom/cfg_qcow2
......
create qcow2 success
$ ls result/2023-05-23-15-29/
openEuler_aarch64.qcow2
openEuler_aarch64.qcow2.sha256sum
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/maintenance/common_skills/information_collection.md
---
# Information Collection

## Querying OS Information

1. Query the OS version by running either of the following commands.
   1. `cat /etc/openEuler-latest`\
      Output:

      ```text
      openeulerversion=openEuler-22.03-LTS-SP4
      compiletime=2022-12-27-22-15-04
      gccversion=10.3.1-20
      kernelversion=5.10.0-136.12.0.86.oe2203SP3
      openjdkversion=1.8.0.352.b08-3.oe2203SP3
      ```

   2. `cat /etc/os-release`\
      Output:

      ```text
      NAME="openEuler"
      VERSION="22.03 (LTS-SP4)"
      ID="openEuler"
      VERSION_ID="22.03"
      PRETTY_NAME="openEuler 22.03 (LTS-SP4)"
      ANSI_COLOR="0;31"
      ```

   3. `cat /etc/openEuler-release`\
      Output:

      ```text
      openEuler release 22.03 (LTS-SP4)
      ```

2. Query the kernel version.

   ```shell
   uname -a
   ```

   Output:

   ```text
   Linux localhost 5.10.0-136.12.0.86.oe2203SP3.x86_64 #1 SMP Tue Dec 27 17:50:15 CST 2022 x86_64 x86_64 x86_64 GNU/Linux
   ```

## Querying Hardware Information

1. Query CPU statistics.

   ```shell
   lscpu
   ```

   ![en-us\_image\_0000001387692269](./images/en-us_image_0000001387692269.jpg)

2. View CPU parameters.

   ```shell
   cat /proc/cpuinfo
   ```

   ![en-us\_image\_0000001387293085](./images/en-us_image_0000001387293085.png)

3. View system memory information.

   ```shell
   cat /proc/meminfo
   ```

   ![en-us\_image\_0000001387692893](./images/en-us_image_0000001387692893.png)

4. View memory information.

   ```shell
   dmidecode -t memory
   ```

   ![en-us\_image\_0000001337053248](./images/en-us_image_0000001337053248.png)

5. View hard drive and partition distribution.

   ```shell
   lsblk
   ```

   ![en-us\_image\_0000001387413509](./images/en-us_image_0000001387413509.png)

6. View details about hard drives and partitions.

   ```shell
   fdisk -l
   ```

   ![en-us\_image\_0000001337533690](./images/en-us_image_0000001337533690.png)

7. View NIC information.

   ```shell
   lspci | grep -i 'eth'
   ```

   ![en-us\_image\_0000001387413793](./images/en-us_image_0000001387413793.png)

8. View all network interfaces.

   ```shell
   ip a or ifconfig -a
   ```

   ![en-us\_image\_0000001387855149](./images/en-us_image_0000001387855149.png)

9. View details about a network interface.

   ```shell
   ethtool enp7s0 (enp7s0 is used as an example.)
   ```

   ![en-us\_image\_0000001387415629](./images/en-us_image_0000001387415629.png)

10. View PCI information.

    ```shell
    lspci
    ```

    ![en-us\_image\_0000001337696078](./images/en-us_image_0000001337696078.png)

11. View the device tree.

    ```shell
    lspci -t
    ```

    ![en-us\_image\_0000001337536842](./images/en-us_image_0000001337536842.png)

12. View BIOS information.

    ```shell
    dmidecode -t bios
    ```

    ![en-us\_image\_0000001387857005](./images/en-us_image_0000001387857005.png)

## Querying Software Information

1. Query details about a software package.

   ```shell
   rpm -qi (systemd is used as an example.)
   ```

   ![en-us\_image\_0000001387755969](./images/en-us_image_0000001387755969.png)

2. View the modules provided by a software package.

   ```shell
   rpm -q --provides # (systemd is used as an example.)
   ```

   ```text
   /bin/systemctl
   /sbin/shutdown
   config(systemd) = 249-43.oe2203SP3
   libsystemd-shared-249.so()(64bit)
   libsystemd-shared-249.so(SD_SHARED)(64bit)
   pkgconfig(systemd) = 249
   pkgconfig(udev) = 249
   syslog
   system-setup-keyboard = 0.9
   systemd = 249-43.oe2203SP3
   systemd(x86-64) = 249-43.oe2203SP3
   systemd-rpm-config
   systemd-sysv = 206
   systemd-units = 249-43.oe2203SP3
   ```

3. View all installed software packages.

   ```shell
   rpm -qa # (systemd is used as an example.)
   ```

   ```text
   systemd-help-249-43.oe2203SP3.noarch
   systemd-libs-249-43.oe2203SP3.x86_64
   systemd-249-43.oe2203SP3.x86_64
   systemd-udev-249-43.oe2203SP3.x86_64
   ```

4. View the list of software packages.

   ```shell
   rpm -ql # (python3-rpm is used as an example.)
   ```

   ![en-us\_image\_0000001387780357](./images/en-us_image_0000001387780357.png)

## Viewing OS Logs

1. View the information and error logs after the system is started.

   ```shell
   cat  /var/log/messages
   ```

   ![en-us\_image\_0000001388020197](./images/en-us_image_0000001388020197.png)

2. View the security-related logs.

   ```shell
   cat /var/log/secure
   ```

   ![en-us\_image\_0000001337580216](./images/en-us_image_0000001337580216.png)

3. View the email-related logs.

   ```shell
   cat /var/log/maillog
   ```

   ![en-us\_image\_0000001337740252](./images/en-us_image_0000001337740252.png)

4. View the logs related to scheduled tasks.

   ```shell
   cat /var/log/cron
   ```

   ![en-us\_image\_0000001337420372](./images/en-us_image_0000001337420372.png)

5. View the logs related to UUCP and news devices.

   ```shell
   cat /var/log/spooler
   ```

   ![en-us\_image\_0000001337260780](./images/en-us_image_0000001337260780.png)

6. View system startup logs.

   ```shell
   cat /var/log/boot.log
   ```

   ![en-us\_image\_0000001337740540](./images/en-us_image_0000001337740540.png)

---

---
url: /en/docs/22.03_LTS_SP4/server/security/safeguard/install_safeguard.md
---
# Installation

## Requirements

* Linux kernel 5.13.0
  * BTF (`CONFIG_DEBUG_INFO_BTF`) must be enabled.
  * BPF LSM (`CONFIG_LSM` with `bpf`) must be enabled. This parameter can also be changed in the boot parameter.

### Kernel Configuration

The kernel must have been compiled with the following flags set:

```shell
CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
CONFIG_BPF_LSM=y
CONFIF_BPF_JIT=y
CONFIG_HAVE_EBPF_JIT=y
CONFIG_BPF_EVENTS=y
CONFIG_DEBUG_INTO_BTF=y
```

Kernel compile flags can usually be checked in `/proc/config.gz` or `/boot/config-<kernel-version>`.

Also, the `CONFIG_LSM` flag must contain `bpf`. This can also be controlled by the following boot parameter:

```shell
$ cat /etc/default/grub
...
GRUB_CMDLINE_LINUX="... lsm=lockdown,yama,apparmor,bpf"
...
```

Finally, run `update-grub2`.

```shell
sudo update-grub2
```

## Installation

Download the latest binary.

```shell
make libbpf-static
make build

sudo ./build/safeguard --config config/safeguard.yml #|grep BLOCK
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/security/safeguard/install_safeguard.md
---
# Installation

## Requirements

* Linux Kernel >= 5.10.0
  * BTF(`CONFIG_DEBUG_INFO_BTF`) must be enabled.
  * BPF LSM(`CONFIG_LSM` with `bpf`) must be enabled. This parameter can also be changed in the boot parameter.

### Kernel Configuration

The kernel must have been compiled with the following flags set:

```shell
CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
CONFIG_BPF_LSM=y
CONFIF_BPF_JIT=y
CONFIG_HAVE_EBPF_JIT=y
CONFIG_BPF_EVENTS=y
CONFIG_DEBUG_INTO_BTF=y
```

Kernel compile flags can usually be checked by looking at `/proc/config.gz` or `/boot/config-<kernel-version>`.

Also, the `CONFIG_LSM` flag must contain `bpf`. This can also be controlled by boot parameters as following:

```shell
$ cat /etc/default/grub
...
GRUB_CMDLINE_LINUX="... lsm=lockdown,yama,apparmor,bpf"
...
```

Finary, run `update-grub2`.

```shell
sudo update-grub2
```

## Install

Download latest released binary

```shell
$ make libbpf-static
$ make build

$ sudo ./build/safeguard --config config/safeguard.yml #|grep BLOCK
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/installation_and_configuration_3.md
---
# Installation and Configuration

This chapter describes important configurations related to the installation of the open source container Docker.

## Precautions

* The root permission is required for installing a Docker container.
* The  **docker-engine**  RPM package cannot be installed together with the  **containerd**,  **runc**, or  **podman**  RPM package. This is because the  **docker-engine**  RPM package contains all components required for Docker running, including  **containerd**,  **runc**, and  **podman**  binary files. Yet the  **containerd**,  **runc**, and  **podman**  RPM packages also contain the corresponding binary files. Software package conflicts may occur due to repeated installation.

## Basic Installation Configuration

### Daemon Parameter Configuration

You can add configuration items to the  **/etc/docker/daemon.json**  file to customize parameters. You can run the  **dockerd --help**  command to view related configuration items and their usage methods. A configuration example is as follows:

```shell
$ cat /etc/docker/daemon.json 
{        
    "debug": true,        
    "storage-driver": "overlay2",        
    "storage-opts": ["overlay2.override_kernel_check=true"] 
}
```

### Daemon Running Directory Configuration

Re-configuring various running directories and files (including  **--graph**  and  **--exec-root**) may cause directory conflicts or file attribute changes, affecting the normal use of applications.

> \[!TIP] **NOTICE:**\
> Therefore, the specified directories or files should be used only by Docker to avoid file attribute changes and security issues caused by conflicts.

* Take  **--graph**  as an example. When  **/new/path/**  is used as the new root directory of the daemon, if a file exists in  **/new/path/**  and the directory or file name conflicts with that required by Docker (for example,  **containers**,  **hooks**, and  **tmp**), Docker may update the original directory or file attributes, including the owner and permission.

> \[!TIP] **NOTICE:**\
> From Docker 17.05, the  **--graph**  parameter is marked as  **Deprecated**  and replaced with the  **--data-root**  parameter.

### Daemon Network Configuration

* After the network segment of the docker0 bridge is specified by using the  **--bip**  parameter on Docker daemon, if the  **--bip**  parameter is deleted during the next Docker daemon restart, the docker0 bridge uses the previous value of **--bip**, even if the docker0 bridge is deleted before the restart. The reason is that Docker saves the network configuration and restores the previous configuration by default during the next restart.
* When running the  **docker network create**  command to concurrently create networks, you can create two networks with the same name. The reason is that Docker networks are distinguished by IDs. The name is only an alias that is easy to identify and may not be unique.
* In the Docker bridge network mode, a Docker container establishes external communication through NAT on the host. When Docker daemon starts a Docker container, a docker-proxy process is started for each port mapped on the host to access the proxy. It is recommended that you map only the necessary ports when using userland-proxy to reduce the resources consumed by the port mapping of docker-proxy.

### Daemon umask Configuration

The default  **umask**  value of the main container process and exec process is  **0022**. To meet security specifications and prevent containers from being attacked, the default value of **umask**  is changed to  **0027**  after runC implementation is modified. After the modification, the other groups cannot access new files or directories.

The default value of **umask**  is  **0027**  when Docker starts a container. You can change the value to  **0022**  by running the  **--exec-opt native.umask=normal**  command during container startup.

> \[!TIP] **NOTICE:**\
> If  **native.umask**  is configured in  **docker create**  or  **docker run**  command, its value is used.

For details, see the parameter description in  **docker create**  and  **docker run**.

### Daemon Start Time

The Docker service is managed by systemd, which restricts the startup time of each service. If the Docker service fails to be started within the specified time, the possible causes are as follows:

* If Docker daemon is started for the first time using devicemapper, the Docker daemon needs to perform the initialization operation on the device. This operation, however, will perform a large number of disk I/O operations. When the disk performance is poor or many I/O conflicts exist, the Docker daemon startup may time out. devicemapper needs to be initialized only once and does not need to be initialized again during later Docker daemon startup.
* If the usage of the current system resources is too high, the system responses slowly, all operations in the system slow down, and the startup of the Docker service may time out.
* During the restart, a daemon traverses and reads configuration files and the init layer and writable layer configurations of each container in the Docker working directory. If there are too many containers (including the created and exited containers) in the current system and the disk read and write performance is limited, the startup of the Docker service may time out due to the long-time daemon traversing.

If the service startup times out, you are advised to rectify the fault as follows:

* Ensure that the container orchestration layer periodically deletes unnecessary containers, especially the exited containers.
* Based on performance requirements of the solution, adjust the cleanup period of the orchestration layer and the start time of the Docker service.

### Journald Component

After systemd-journald is restarted, Docker daemon needs to be restarted. Journald obtains the Docker daemon logs through a pipe. If the journald service is restarted, the pipe is disabled. The write operation of Docker logs triggers the SIGPIPE signal, which causes the Docker daemon crash. If this signal is ignored, the subsequent Docker daemon logs may fail to be recorded. Therefore, you are advised to restart Docker daemon after the journald service is restarted or becomes abnormal, ensuring that Docker logs can be properly recorded and preventing status exceptions caused by daemon crash.

### Firewalld Component

You need to restart the Docker service after restarting or starting firewalld.

* When the firewalld service is started, the iptables rules of the current system are cleared. Therefore, if the firewalld service is restarted during Docker daemon startup, the Docker service may fail to insert iptables rules, causing the Docker service startup failure.
* If the firewalld service is restarted after the Docker service is started, or the status of the firewalld service (service paused or resumed) is changed, the iptables rules of the Docker service are deleted. As a result, the container with port mapping fails to be created.

### Iptables Component

If the  **--icc=false**  option is added in Docker, the communication between containers can be restricted. However, if the OS has some rules, the communication between containers may not be restricted. For example:

```text
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) 
... 
0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0 
... 
0     0 DROP       all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
...
```

In the  **Chain FORWARD**  command, the ACCEPT icmp rule is added to DROP. As a result, after the  **--icc=false**  option is added, containers can be pinged, but the peer end is unreachable if UDP or TCP is used.

Therefore, if you want to add the  **--icc=false**  option when using Docker in a container OS, you are advised to clear iptables rules on the host first.

### Audit Component

You can configure audit for Docker. However, this configuration is not mandatory. For example:

```text
-w /var/lib/docker -k docker 
-w /etc/docker -k docker 
-w /usr/lib/systemd/system/docker.service -k docker 
-w /usr/lib/systemd/system/docker.socket -k docker 
-w /etc/sysconfig/docker -k docker 
-w /usr/bin/docker-containerd -k docker 
-w /usr/bin/docker-runc -k docker 
-w /etc/docker/daemon.json -k docker
```

Configuring audit for Docker brings certain benefits for auditing, while it does not have any substantial effects on attack defense. In addition, the audit configurations cause serious efficiency problems, for example, the system may not respond smoothly. Therefore, exercise caution in the production environment.

The following uses  **-w /var/lib/docker -k docker**  as an example to describe how to configure Docker audit.

```shell
cat /etc/audit/rules.d/audit.rules | grep docker -w /var/lib/docker/  -k docker 
auditctl -R /etc/audit/rules.d/audit.rules | grep docker 
auditctl -l | grep docker -w /var/lib/docker/ -p rwxa -k docker
```

> \[!NOTE] **NOTE:**\
> **-p \[r|w|x|a]**  and  **-w**  are used together to monitor the read, write, execution, and attribute changes (such as timestamp changes) of the directory. In this case, any file or directory operation in the  **/var/lib/docker**  directory will be recorded in the  **audit.log**  file. As a result, too many logs will be recorded in the  **audit.log**  file, which severely affects the memory or CPU usage of the auditd, and further affects the OS. For example, logs similar to the following will be recorded in the  **/var/log/audit/audit.log**  file each time the  **ls /var/lib/docker/containers**  command is executed:

```text
type=SYSCALL msg=audit(1517656451.457:8097): arch=c000003e syscall=257 success=yes exit=3 a0=ffffffffffffff9c a1=1b955b0 a2=90800 a3=0 items=1 ppid=17821 pid=1925 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts6 ses=4 comm="ls" exe="/usr/bin/ls" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="docker"type=CWD msg=audit(1517656451.457:8097):  cwd="/root"type=PATH msg=audit(1517656451.457:8097): item=0 name="/var/lib/docker/containers" inode=1049112 dev=fd:00 mode=040700 ouid=0 ogid=0 rdev=00:00 obj=unconfined_u:object_r:container_var_lib_t:s0 objtype=NORMAL 
```

### Security Configuration seccomp

During the container network performance test, it is found that the performance of Docker is lower than that of the native kernel namespace. After seccomp is enabled, system calls (such as sendto) are not performed through system\_call\_fastpath. Instead, tracesys is called, which greatly deteriorates the performance. Therefore, you are advised to disable seccomp in container scenarios where services require high performance. For example:

```shell
docker run -itd --security-opt seccomp=unconfined busybox:latest
```

### Do Not Modify Private Directory of Docker Daemon

Do not modify the root directory used by Docker (**/var/lib/docker**  by default), the directory during operation (**/run/docker**  by default), or the files or directories in the two directories. The forbidden operations include deleting files, adding files, creating soft or hard links for the directories or files, or modifying attributes, permissions, or contents of the files. If any modification is required, contact the Euler container team for review.

### Precautions for Common Users in the Scenario Where a Large Number of Containers Are Deployed

The maximum number of processes that a common user can create on an OS host can be restricted by creating the  **/etc/security/limits.d/20-nproc.conf**  file in the system. Similarly, the maximum number of processes that a common user can create in a container is determined by the value in the  **/etc/security/limits.d/20-nproc.conf**  file in the container image, as shown in the following example:

```shell
$ cat /etc/security/limits.d/20-nproc.conf
*       soft    nproc   4096
```

If an error is reported due to insufficient resources when a large number of containers are deployed by a common user, increase the value  **4096**  in the  **/etc/security/limits.d/20-nproc.conf**  file.

Configure the maximum value based on the maximum capability of the kernel, as shown in the following example:

```shell
$ sysctl -a | grep pid_max 
kernel.pid_max = 32768
```

## Storage Driver Configuration

This Docker version supports two storage drivers: overlay2 and devicemapper. Since overlay2 has better performance than devicemapper, it is recommended that overlay2 be preferentially used in the production environment.

### overlay2 Storage Driver Configuration

#### Configuration Methods

overlay2 is the default storage driver of Docker. You can also use either of the following methods to explicitly configure the driver:

* Edit the  **/etc/docker/daemon.json**  file to explicitly configure the  **storage-driver**  field.

  ```shell
  $ cat /etc/docker/daemon.json
  {
      "storage-driver": "overlay2"
  }
  ```

* Edit the  **/etc/sysconfig/docker-storage**  file and explicitly configure the Docker daemon startup parameters.

  ```shell
  $ cat /etc/sysconfig/docker-storage 
  DOCKER_STORAGE_OPTIONS="--storage-driver=overlay2"
  ```

#### Precautions

* When you perform lifecycle management operations on some containers, an error may be reported, indicating that the corresponding rootfs or executable file cannot be found.

* If the health check of a container is configured to execute executable files in the container, an error may be reported, which causes the health check failure of the container.

* When you use overlay2 as the graphdriver and modify an image file in a container for the first time, the modification fails if the file size is greater than the remaining space of the system. Even if a little modification on the file is involved, the whole file must be copied to the upper layer. If the remaining space is insufficient, the modification fails.

* Compared with common file systems, the overlay2 file system has the following behavior differences:
  * Kernel version

    overlay2 is compatible only with the native kernel 4.0 or later. You are advised to use the Ext4 file system.

  * Copy-UP performance

    Modifying files at the lower layer triggers file replication to the upper layer. Data block replication and fsync are time-consuming.

  * Rename directories
    * The rename system call is allowed only when both the source and the destination paths are at the merged layer. Otherwise, the EXDEV error is reported.
    * Kernel 4.10 introduces the redirect directory feature to fix this issue. The corresponding kernel option is  **CONFIG\_OVERLAY\_FS\_REDIRECT\_DIR**.

      When overlay2 is used, a file system directory fails to be renamed because the related feature configured in the  **/sys/module/overlay/parameters/redirect\_dir**  file has been disabled. To use this feature, you need to manually set  **/sys/module/overlay/parameters/redirect\_dir**  to  **Y**.

  * Hard link disconnection
    * If there are multiple hard links in the lower-layer directory, writing data to the merged layer will trigger Copy-UP, resulting in hard link disconnection.
    * The index feature is introduced in kernel 4.13 to fix this issue. The corresponding kernel option is  **CONFIG\_OVERLAY\_FS\_INDEX**. Note that this option is not forward compatible and does not support hot upgrade.

  * Changes of **st\_dev**  and  **st\_ino**

    After Copy-UP is triggered, you can view only new files at the merged layer, and inodes change. Although  **attr**  and  **xattr**  can be replicated,  **st\_dev**  and  **st\_ino**  are unique and cannot be replicated. As a result, the results of the **stat**  and  **ls**  commands change accordingly.

  * fd change

    Before Copy-UP is triggered, you can obtain the descriptor fd1 when opening a file in read-only mode. After Copy-UP is trigger, you can obtain the descriptor fd2 when opening the file with the same name. The two descriptors point to different files. The data written to fd2 is not displayed in fd1.

#### Abnormal Scenarios

When a container uses the overlay2 storage driver, mount points may be overwritten.

#### Abnormal Scenario: Mount Point Being Overwritten

In the faulty container, there is a mount point in  **/var/lib/docker/overlay2**.

```shell
$ mount -l | grep overlay 
overlay on /var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/merged type overlay (rw,relatime,seclabel,lowerdir=/var/lib/docker/overlay2/l/JL5PZQLNDCIBU3ZOG3LPPDBHIJ:/var/lib/docker/overlay2/l/ELRPYU4JJG4FDPRLZJCZZE4UO6,upperdir=/var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/diff,workdir=/var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/work) 
/dev/mapper/dm-root on /var/lib/docker/overlay2 type ext4 (rw,relatime,seclabel,data=ordered)
```

An error as follows may occur when some Docker commands are executed:

```shell
$ docker rm 1348136d32
docker rm: Error response from daemon: driver "overlay2" failed to remove root filesystem for 1348136d32: error while removing /var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785: invalid argument
```

You will find that the rootfs of the corresponding container cannot be found on the host. However, this does not mean that the rootfs is lost. The rootfs is overwritten by the mount point in  **/var/lib/docker/overlay2**, and services are still running properly. The solutions are as follows:

* Solution 1
  1. Run the following command to check the graphdriver used by Docker:

     ```shell
     docker info | grep "Storage Driver"
     ```

  2. Run the following commands to query the current mount point:

     ```shell
     # Devicemapper
     mount -l | grep devicemapper 
     # Overlay2
     mount -l | grep overlay2
     ```

     The output format is  *A*  on  *B*  type  *C*  (*D*).

     * *A*: block device name or  **overlay**
     * *B*: mount point
     * *C*: file system type
     * *D*: mounting attribute

  3. Run the  **umount**  command on the mount points (*B*) one by one from bottom to top.

  4. Run the  **docker restart**  command on all the containers or delete all the containers.

  5. Run the following command to restart Docker:

     ```shell
     systemctl restart docker
     ```

* Solution 2
  1. Migrate services.
  2. Restart nodes.

### devicemapper Storage Driver Configuration

If you need to set the storage driver of Docker to devicemapper, you can also use either of the following methods to explicitly configure the driver:

* Edit the  **/etc/docker/daemon.json**  file to explicitly configure the  **storage-driver**  field.

  ```shell
  $ cat /etc/docker/daemon.json
  {
      "storage-driver": "devicemapper"
  }
  ```

* Edit the  **/etc/sysconfig/docker-storage**  file and explicitly configure the Docker daemon startup parameters.

  ```shell
  $ cat /etc/sysconfig/docker-storage 
  DOCKER_STORAGE_OPTIONS="--storage-driver=devicemapper"
  ```

#### Precautions

* To use devicemapper, you must use the direct-lvm mode. For details about the configuration method, refer to  <https://docs.docker.com/storage/storagedriver/device-mapper-driver/#configure-direct-lvm-mode-for-production>.

* When configuring devicemapper, if the system does not have sufficient space for automatic capacity expansion of thinpool, disable the automatic capacity expansion function.

* Do not set both the following two parameters in the  **/etc/lvm/profile/docker-thinpool.profile**  file to  **100**:

  ```text
  activation {   
    thin_pool_autoextend_threshold=80   
    thin_pool_autoextend_percent=20 
  }
  ```

* You are advised to add  **--storage-opt dm.use\_deferred\_deletion=true**  and  **--storage-opt dm.use\_deferred\_removal=true**  when using devicemapper.

* When devicemapper is used, you are advised to use Ext4 as the container file system. You need to add  **--storage-opt dm.fs=ext4**  to the configuration parameters of Docker daemon.

* If graphdriver is devicemapper and the metadata files are damaged and cannot be restored, you need to manually restore the metadata files. Do not directly operate or tamper with metadata of the devicemapper storage driver in Docker daemon.

* When the devicemapper LVM is used, if the devicemapper thinpool is damaged due to abnormal power-off, you cannot ensure the data integrity or whether the damaged thinpool can be restored. Therefore, you need to rebuild the thinpool.

**Precautions for Switching the devicemapper Storage Pool When the User Namespace Feature Is Enabled on Docker Daemon**

* Generally, the path of the deviceset-metadata file is  **/var/lib/docker/devicemapper/metadata/deviceset-metadata**  during container startup.
* If user namespaces are used, the path of the deviceset-metadata file is  **/var/lib/docker/***userNSUID.GID***/devicemapper/metadata/deviceset-metadata**.
* When you use the devicemapper storage driver and the container is switched between the user namespace scenario and common scenario, the  **BaseDeviceUUID**  content in the corresponding deviceset-metadata file needs to be cleared. In the thinpool capacity expansion or rebuild scenario, you also need to clear the  **BaseDeviceUUID**  content in the deviceset-metadata file. Otherwise, the Docker service fails to be restarted.

## Impact of Forcibly Killing Docker Background Processes

### Semaphores May Be Residual

When the devicemapper is used as the graphdriver, forcible killing may cause residual semaphores. Docker creates semaphores when performing operations on devicemapper. If daemon is forcibly killed before the semaphores are released, the release may fail. A maximum of one semaphore can be leaked at a time, and the leakage probability is low. However, the Linux OS has an upper limit on semaphores. When the number of semaphore leakage times reaches the upper limit, new semaphores cannot be created. As a result, Docker daemon fails to be started. The troubleshooting method is as follows:

1. Check the residual semaphores in the system.

   ```shell
   $ ipcs 
   ------ Message Queues -------- 
   key        msqid      owner      perms      used-bytes   messages 
   ------ Shared Memory Segments -------- 
   key        shmid      owner      perms      bytes      nattch     status 
   ------ Semaphore Arrays -------- 
   key        semid      owner      perms      nsems 
   0x0d4d3358 238977024  root       600        1 
   0x0d4d0ec9 270172161  root       600        1 
   0x0d4dc02e 281640962  root       600        1
   ```

2. Run the  **dmsetup**  command to check semaphores created by devicemapper. The semaphore set is the subset of the system semaphores queried in the previous step.

   ```shell
   dmsetup udevcookies 
   ```

3. Check the upper limit of kernel semaphores. The fourth value is the upper limit of the current system semaphores.

   ```shell
   $ cat /proc/sys/kernel/sem 
   250     32000   32      128
   ```

   If the number of residual semaphores in step 1 is the same as the upper limit of semaphores in step 3, the number of residual semaphores reaches the upper limit. In this case, Docker daemon cannot be normally started. You can run the following command to increase the upper limit to restart Docker:

   ```shell
   echo 250 32000  32  1024 > /proc/sys/kernel/sem
   ```

   You can also run the following command to manually clear the residual devicemapper semaphores. The following describes how to clear the devicemapper semaphores applied one minute ago.

   ```shell
   $ dmsetup udevcomplete_all 1 
   This operation will destroy all semaphores older than 1 minutes with keys that have a prefix 3405 (0xd4d). 
   Do you really want to continue? [y/n]: y 
   0 semaphores with keys prefixed by 3405 (0xd4d) destroyed. 0 skipped.
   ```

### NICs May Be Residual

When a container is started in bridge mode, forcibly killing may cause residual NICs. In bridge network mode, when Docker creates a container, a pair of veths are created on the host, and then the NIC information is saved to the database. If daemon is forcibly killed before the NIC information is saved to the database of Docker, the NIC cannot be associated with Docker and cannot be deleted during the next startup because Docker deletes unused NICs from its database.

### Failed to Restart a Container

If container hook takes a long time, and containerd is forcibly killed during container startup, the container start operation may fail. When containerd is forcibly killed during container startup, an error is returned for the Docker start operation. After containerd is restarted, the last startup may still be in the  **runc create**  execution phase (executing the user-defined hook may take a long time). If you run the  **docker start**  command again to start the container, the following error message may be displayed:

```text
Error response from daemon: oci runtime error: container with id exists: xxxxxx
```

This error is caused by running  **runc create**  on an existing container (or being created). After the  **runc create**  operation corresponding to the first start operation is complete, the  **docker start**  command can be successfully executed.

The execution of hook is not controlled by Docker. In this case, if the container is recycled, the containerd process may be suspended when an unknown hook program is executed. In addition, the risk is controllable (although the creation of the current container is affected in a short period).

* After the first operation is complete, the container can be successfully started again.
* Generally, a new container is created after the container fails to be started. The container that fails to be started cannot be reused.

In conclusion, this problem has a constraint on scenarios.

### Failed to Restart the Docker Service

The Docker service cannot be restarted properly due to frequent startup in a short period The Docker system service is monitored by systemd. If the Docker service is restarted for more than five times within 10s, the systemd service detects the abnormal startup. Therefore, the Docker service is disabled. Docker can respond to the restart command and be normally restarted only when the next period of 10s starts.

## Impact of System Power-off

When a system is unexpectedly powered off or system panic occurs, Docker daemon status may not be updated to the disk in time. As a result, Docker daemon is abnormal after the system is restarted. The possible problems include but are not limited to the following:

* A container is created before the power-off. After the restart, the container is not displayed when the  **docker ps –a**  command is run, as the file status of the container is not updated to the disk. As a result, daemon cannot obtain the container status after the restart.
* Before the system power-off, a file is being written. After daemon is restarted, the file format is incorrect or the file content is incomplete. As a result, loading fails.
* As Docker database (DB) will be damaged during power-off, all DB files in  **data-root**  will be deleted during node restart. Therefore, the following information created before the restart will be deleted after the restart:
  * Network: Resources created through Docker network will be deleted after the node is restarted.
  * Volume: Resources created through Docker volume will be deleted after the node is restarted.
  * Cache construction: The cache construction information will be deleted after the node is restarted.
  * Metadata stored in containerd: Metadata stored in containerd will be recreated when a container is started. Therefore, the metadata stored in containerd will be deleted when the node is restarted.

    > \[!NOTE] **NOTE:**\
    > If you want to manually clear data and restore the environment, you can set the environment variable  **DISABLE\_CRASH\_FILES\_DELETE**  to  **true**  to disable the function of clearing DB files when the daemon process is restarted due to power-off.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/installation_configuration.md
---
# Installation and Configuration

## Installation Methods

iSulad can be installed by running the  **yum**  or  **rpm**  command. The  **yum**  command is recommended because dependencies can be installed automatically.

This section describes two installation methods.

* (Recommended) Run the following command to install iSulad:

  ```bash
  sudo yum install -y iSulad
  ```

* If the  **rpm**  command is used to install iSulad, you need to download and manually install the RMP packages of iSulad and all its dependencies. To install the RPM package of a single iSulad (the same for installing dependency packages), run the following command:

  ```bash
  # sudo rpm -ihv iSulad-xx.xx.xx-xx.xxx.aarch64.rpm
  ```

## Deployment Configuration

After iSulad is installed, you can perform related configurations as required.

### Configuration Mode

The iSulad server daemon  **isulad**  can be configured with a configuration file or by running the  **isulad --xxx**  command. The priority in descending order is as follows: CLI > configuration file > default configuration in code.

> \[!NOTE] **NOTE:**
> If systemd is used to manage the iSulad process, modify the  **OPTIONS**  field in the  **/etc/sysconfig/iSulad**  file, which functions the same as using the CLI.

* **CLI**

  During service startup, configure iSulad using the CLI. To view the configuration options, run the following command:

  ```bash
  # isulad --help
  isulad

  lightweight container runtime daemon

  Usage:  isulad [global options]

  GLOBAL OPTIONS:

        --authorization-plugin               Use authorization plugin
        --cgroup-parent                      Set parent cgroup for all containers
        --cni-bin-dir                        The full path of the directory in which to search for CNI plugin binaries. Default: /opt/cni/bin
        --cni-conf-dir                       The full path of the directory in which to search for CNI config files. Default: /etc/cni/net.d
        --container-log-driver               Set default container log driver, such as: json-file
        --container-log-opts                 Set default container log driver options, such as: max-file=7 to set max number of container log files
        --default-ulimit                     Default ulimits for containers (default [])
    -e, --engine                             Select backend engine
    -g, --graph                              Root directory of the iSulad runtime
    -G, --group                              Group for the unix socket(default is isulad)
        --help                               Show help
        --hook-spec                          Default hook spec file applied to all containers
    -H, --host                               The socket name used to create gRPC server
        --image-layer-check                  Check layer integrity when needed
        --insecure-registry                  Disable TLS verification for the given registry
        --insecure-skip-verify-enforce       Force to skip the insecure verify(default false)
        --log-driver                         Set daemon log driver, such as: file
    -l, --log-level                          Set log level, the levels can be: FATAL ALERT CRIT ERROR WARN NOTICE INFO DEBUG TRACE
        --log-opt                            Set daemon log driver options, such as: log-path=/tmp/logs/ to set directory where to store daemon logs
        --native.umask                       Default file mode creation mask (umask) for containers
        --network-plugin                     Set network plugin, default is null, support null and cni
    -p, --pidfile                            Save pid into this file
        --pod-sandbox-image                  The image whose network/ipc namespaces containers in each pod will use. (default "pause-${machine}:3.0")
        --registry-mirrors                   Registry to be prepended when pulling unqualified images, can be specified multiple times
        --selinux-enabled                    Enable selinux support
        --start-timeout                      timeout duration for waiting on a container to start before it is killed
    -S, --state                              Root directory for execution state files
        --storage-driver                     Storage driver to use(default overlay2)
    -s, --storage-opt                        Storage driver options
        --tls                                Use TLS; implied by --tlsverify
        --tlscacert                          Trust certs signed only by this CA (default "/root/.iSulad/ca.pem")
        --tlscert                            Path to TLS certificate file (default "/root/.iSulad/cert.pem")
        --tlskey                             Path to TLS key file (default "/root/.iSulad/key.pem")
        --tlsverify                          Use TLS and verify the remote
        --use-decrypted-key                  Use decrypted private key by default(default true)
        --userns-remap                       User/Group setting for user namespaces
    -V, --version                            Print the version
        --websocket-server-listening-port    CRI websocket streaming service listening port (default 10350)
  ```

  Example: Start iSulad and change the log level to **DEBUG**.

  ```bash
  # isulad -l DEBUG
  ```

* **Configuration file**

  The iSulad configuration files are **/etc/isulad/daemon.json** and **/etc/isulad/daemon\_constants.json**. The parameters in the files are described as follows.

  Configuration file **/etc/isulad/daemon\_constants.json**

  Example:

  ```bash
  # cat /etc/isulad/daemon.json
  {
      "group": "isulad",
      "default-runtime": "runc",
      "graph": "/var/lib/isulad",
      "state": "/var/run/isulad",
      "engine": "lcr",
      "log-level": "ERROR",
      "pidfile": "/var/run/isulad.pid",
      "log-opts": {
          "log-file-mode": "0600",
          "log-path": "/var/lib/isulad",
          "max-file": "1",
          "max-size": "30KB"
      },
      "log-driver": "stdout",
      "hook-spec": "/etc/default/isulad/hooks/default.json",
      "start-timeout": "2m",
      "storage-driver": "overlay2",
      "storage-opts": [
          "overlay2.override_kernel_check=true"
      ],
      "registry-mirrors": [
          "docker.io"
      ],
      "insecure-registries": [
          "rnd-dockerhub.huawei.com"
      ],
      "pod-sandbox-image": "",
      "native.umask": "secure",
      "network-plugin": "",
      "cni-bin-dir": "",
      "cni-conf-dir": "",
      "image-layer-check": false,
      "use-decrypted-key": true,
      "insecure-skip-verify-enforce": false,
      "cri-runtime": {
          "kata": "io.containerd.kata.v2"
      }
  }

  # cat /etc/isulad/daemon.json
  {
      "default-host": "docker.io",
      "registry-transformation":{
          "docker.io": "registry-1.docker.io",
          "index.docker.io": "registry-1.docker.io"
      }
  }

  ```

  > \[!TIP] **NOTICE:**
  > The default configuration file  **/etc/isulad/daemon.json**  is for reference only. Configure it based on site requirements.

### Storage Description

### Constraints

* In high concurrency scenarios (200 containers are concurrently started), the memory management mechanism of Glibc may cause memory holes and large virtual memory (for example, 10 GB). This problem is caused by the restriction of the Glibc memory management mechanism in the high concurrency scenario, but not by memory leakage. Therefore, the memory consumption does not increase infinitely. You can set  **MALLOC\_ARENA\_MAX**  to reducevirtual memory error and increase the rate of reducing physical memory. However, this environment variable will cause the iSulad concurrency performance to deteriorate. Set this environment variable based on the site requirements.

  ```bash
  To balance performance and memory usage, set MALLOC_ARENA_MAX to 4. (The iSulad performance on the ARM64 server is affected by less than 10%.)

  Configuration method:
  1. To manually start iSulad, run the export MALLOC_ARENA_MAX=4 command and then start iSulad.
  2. If systemd manages iSulad, you can modify the /etc/sysconfig/iSulad file by adding MALLOC_ARENA_MAX=4.
  ```

* Precautions for specifying the daemon running directories

  Take  **--root**  as an example. When  **/new/path/**  is used as the daemon new root directory, if a file exists in  **/new/path/**  and the directory or file name conflicts with that required by iSulad (for example,  **engines**  and  **mnt**), iSulad may update the original directory or file attributes including the owner and permission.

  Therefore, please note the impact of re-specifying various running directories and files on their attributes. You are advised to specify a new directory or file for iSulad to avoid file attribute changes and security issues caused by conflicts.

* Log file management:

  > \[!TIP] **NOTICE:**
  > Log function interconnection: logs are managed by systemd as iSulad is and then transmitted to rsyslogd. By default, rsyslog restricts the log writing speed. You can add the configuration item  **$imjournalRatelimitInterval 0**  to the  **/etc/rsyslog.conf**  file and restart the rsyslogd service.

* Restrictions on command line parameter parsing

  When the iSulad command line interface is used, the parameter parsing mode is slightly different from that of Docker. For flags with parameters in the command line, regardless of whether a long or short flag is used, only the first space after the flag or the character string after the equal sign (=) directly connected to the flag is used as the flag parameter. The details are as follows:

  1. When a short flag is used, each character in the character string connected to the hyphen (-) is considered as a short flag. If there is an equal sign (=), the character string following the equal sign (=) is considered as the parameter of the short flag before the equal sign (=).

     **isula run -du=root busybox**  is equivalent to  **isula run -du root busybox**,  **isula run -d -u=root busybox**, or  **isula run -d -u root busybox**. When  **isula run -du:root**  is used, as  **-:**  is not a valid short flag, an error is reported. The preceding command is equivalent to  **isula run -ud root busybox**. However, this method is not recommended because it may cause semantic problems.

  2. When a long flag is used, the character string connected to  **--**  is regarded as a long flag. If the character string contains an equal sign (=), the character string before the equal sign (=) is a long flag, and the character string after the equal sign (=) is a parameter.

     ```bash
     isula run --user=root busybox
     ```

     or

     ```bash
     isula run --user root busybox
     ```

* After an iSulad container is started, you cannot run the  **isula run -i/-t/-ti**  and  **isula attach/exec**  commands as a non-root user.

* The default path for storing temporary files of iSulad is **/var/lib/isulad/isulad\_tmpdir**. If the root directory of iSulad is changed, the path is **$isulad\_root/isulad\_tmpdir**. To change the directory for storing temporary files of iSulad, you can configure the **ISULAD\_TMPDIR** environment variable before starting iSulad. The **ISULAD\_TMPDIR** environment variable is checked during the iSulad startup. If the **ISULAD\_TMPDIR** environment variable is configured, the **$ISULAD\_TMPDIR/isulad\_tmpdir** directory is used as the path for storing temporary files. Do not store files or folders named **isulad\_tmpdir** in **$ISULAD\_TMPDIR** because iSulad recursively deletes the **$ISULAD\_TMPDIR/isulad\_tmpdir** directory when it is started to prevent residual data. In addition, ensure that only the **root** user can access the **$ISULAD\_TMPDIR** directory to prevent security problems caused by operations of other users.

### Daemon Multi-Port Binding

#### Description

The daemon can bind multiple UNIX sockets or TCP ports and listen on these ports. The client can interact with the daemon through these ports.

#### Port

Users can configure one or more ports in the hosts field in the  **/etc/isulad/daemon.json**  file, or choose not to specify hosts.

```json
{
    "hosts": [
        "unix:///var/run/isulad.sock",
        "tcp://localhost:5678",
        "tcp://127.0.0.1:6789"
    ]
}
```

Users can also run the  **-H**  or  **--host**  command in the  **/etc/sysconfig/iSulad**  file to configure a port, or choose not to specify hosts.

```text
OPTIONS='-H unix:///var/run/isulad.sock --host tcp://127.0.0.1:6789'
```

If hosts are not specified in the  **daemon.json**  file and iSulad, the daemon listens on  **unix:///var/run/isulad.sock**  by default after startup.

#### Restrictions

* Users cannot specify hosts in the  **/etc/isulad/daemon.json**  and  **/etc/sysconfig/iSuald**  files at the same time. Otherwise, an error will occur and iSulad cannot be started.

  ```bash
  unable to configure the isulad with file /etc/isulad/daemon.json: the following directives are specified both as a flag and in the configuration file: hosts: (from flag: [unix:///var/run/isulad.sock tcp://127.0.0.1:6789], from file: [unix:///var/run/isulad.sock tcp://localhost:5678 tcp://127.0.0.1:6789])
  ```

* If the specified host is a UNIX socket, the socket must start with  **unix://**  followed by a valid absolute path.

* If the specified host is a TCP port, the TCP port number must start with  **tcp://**  followed by a valid IP address and port number. The IP address can be that of the local host.

* A maximum of 10 valid ports can be specified. If more than 10 ports are specified, an error will occur and iSulad cannot be started.

### Configuring TLS Authentication and Enabling Remote Access

#### Description

iSulad is designed in C/S mode. By default, the iSulad daemon process listens only on the local/var/run/isulad.sock. Therefore, you can run commands to operate containers only on the local client iSula. To enable iSula's remote access to the container, the iSulad daemon process needs to listen on the remote access port using TCP/IP. However, listening is performed only by simply configuring tcp ip:port. In this case, all IP addresses can communicate with iSulad by calling  **isula -H tcp://***remote server IP address***:port**, which may cause security problems. Therefore, it is recommended that a more secure version, namely Transport Layer Security (TLS), be used for remote access.

#### Generating TLS Certificate

* Example of generating a plaintext private key and certificate

  ```bash
  #!/bin/bash
  set -e
  echo -n "Enter pass phrase:"
  read password
  echo -n "Enter public network ip:"
  read publicip
  echo -n "Enter host:"
  read HOST

  echo " => Using hostname: $publicip, You MUST connect to iSulad using this host!"

  mkdir -p $HOME/.iSulad
  cd $HOME/.iSulad
  rm -rf $HOME/.iSulad/*

  echo " => Generating CA key"
  openssl genrsa -passout pass:$password -aes256 -out ca-key.pem 4096
  echo " => Generating CA certificate"
  openssl req -passin pass:$password -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem -subj "/C=CN/ST=zhejiang/L=hangzhou/O=Huawei/OU=iSulad/CN=iSulad@huawei.com"
  echo " => Generating server key"
  openssl genrsa -passout pass:$password -out server-key.pem 4096
  echo " => Generating server CSR"
  openssl req -passin pass:$password -subj /CN=$HOST -sha256 -new -key server-key.pem -out server.csr
  echo subjectAltName = DNS:$HOST,IP:$publicip,IP:127.0.0.1 >> extfile.cnf
  echo extendedKeyUsage = serverAuth >> extfile.cnf
  echo " => Signing server CSR with CA"
  openssl x509 -req -passin pass:$password -days 365 -sha256 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile extfile.cnf
  echo " => Generating client key"
  openssl genrsa -passout pass:$password -out key.pem 4096
  echo " => Generating client CSR"
  openssl req -passin pass:$password -subj '/CN=client' -new -key key.pem -out client.csr
  echo " => Creating extended key usage"
  echo extendedKeyUsage = clientAuth > extfile-client.cnf
  echo " => Signing client CSR with CA"
  openssl x509 -req -passin pass:$password -days 365 -sha256 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cert.pem -extfile extfile-client.cnf
  rm -v client.csr server.csr extfile.cnf extfile-client.cnf
  chmod -v 0400 ca-key.pem key.pem server-key.pem
  chmod -v 0444 ca.pem server-cert.pem cert.pem
  ```

* Example of generating an encrypted private key and certificate request file

  ```bash
  #!/bin/bash

  echo -n "Enter public network ip:"
  read publicip
  echo -n "Enter pass phrase:"
  read password

  # remove certificates from previous execution.
  rm -f *.pem *.srl *.csr *.cnf

  # generate CA private and public keys
  echo 01 > ca.srl
  openssl genrsa -aes256 -out ca-key.pem -passout pass:$password 2048
  openssl req -subj '/C=CN/ST=zhejiang/L=hangzhou/O=Huawei/OU=iSulad/CN=iSulad@huawei.com' -new -x509 -days $DAYS -passin pass:$password -key ca-key.pem -out ca.pem

  # create a server key and certificate signing request (CSR)
  openssl genrsa -aes256 -out server-key.pem -passout pass:$PASS 2048
  openssl req -new -key server-key.pem -out server.csr -passin pass:$password -subj '/CN=iSulad'

  echo subjectAltName = DNS:iSulad,IP:${publicip},IP:127.0.0.1 > extfile.cnf
  echo extendedKeyUsage = serverAuth >> extfile.cnf
  # sign the server key with our CA
  openssl x509 -req -days $DAYS -passin pass:$password -in server.csr -CA ca.pem -CAkey ca-key.pem -out server-cert.pem -extfile extfile.cnf

  # create a client key and certificate signing request (CSR)
  openssl genrsa -aes256 -out key.pem -passout pass:$password 2048
  openssl req -subj '/CN=client' -new -key key.pem -out client.csr -passin pass:$password

  # create an extensions config file and sign
  echo extendedKeyUsage = clientAuth > extfile.cnf
  openssl x509 -req -days 365 -passin pass:$password -in client.csr -CA ca.pem -CAkey ca-key.pem -out cert.pem -extfile extfile.cnf

  # remove the passphrase from the client and server key
  openssl rsa -in server-key.pem -out server-key.pem -passin pass:$password
  openssl rsa -in key.pem -out key.pem -passin pass:$password

  # remove generated files that are no longer required
  rm -f ca-key.pem ca.srl client.csr extfile.cnf server.csr
  ```

#### APIs

```json
{
    "tls": true,
    "tls-verify": true,
    "tls-config": {
  "CAFile": "/root/.iSulad/ca.pem",
  "CertFile": "/root/.iSulad/server-cert.pem",
  "KeyFile":"/root/.iSulad/server-key.pem"
    }
}
```

#### Restrictions

The server supports the following modes:

* Mode 1 (client verified): tlsverify, tlscacert, tlscert, tlskey
* Mode 2 (client not verified): tls, tlscert, tlskey

The client supports the following modes:

* Mode 1 (verify the identity based on the client certificate, and verify the server based on the specified CA): tlsverify, tlscacert, tlscert, tlskey
* Mode 2 (server verified): tlsverify, tlscacert

Mode 1 is used for the server, and mode 2 for the client if the two-way authentication mode is used for communication.

Mode 2 is used for the server and the client if the unidirectional authentication mode is used for communication.

> \[!TIP] **NOTICE:**
>
> * If RPM is used for installation, the server configuration can be modified in the  **/etc/isulad/daemon.json**  and  **/etc/sysconfig/iSulad**  files.
> * Two-way authentication is recommended as it is more secure than non-authentication or unidirectional authentication.
> * GRPC open-source component logs are not taken over by iSulad. To view gRPC logs, set the environment variables  **gRPC\_VERBOSITY**  and  **gRPC\_TRACE**  as required.

#### Example

On the server:

```bash
 isulad -H=tcp://0.0.0.0:2376 --tlsverify --tlscacert ~/.iSulad/ca.pem --tlscert ~/.iSulad/server-cert.pem --tlskey ~/.iSulad/server-key.pem
```

On the client:

```bash
 isula version -H=tcp://$HOSTIP:2376 --tlsverify --tlscacert ~/.iSulad/ca.pem --tlscert ~/.iSulad/cert.pem --tlskey ~/.iSulad/key.pem
```

### devicemapper Storage Driver Configuration

To use the devicemapper storage driver, you need to configure a thinpool device which requires an independent block device with sufficient free space. Take the independent block device  **/dev/xvdf**  as an example. The configuration method is as follows:

1. Configuring a thinpool

   1. Stop the iSulad service.

      ```bash
      # systemctl stop isulad
      ```

   2. Create a logical volume manager (LVM) volume based on the block device.

      ```bash
      # pvcreate /dev/xvdf
      ```

   3. Create a volume group based on the created physical volume.

      ```bash
      # vgcreate isula /dev/xvdf
      Volume group "isula" successfully created:
      ```

   4. Create two logical volumes named  **thinpool**  and  **thinpoolmeta**.

      ```bash
      # lvcreate --wipesignatures y -n thinpool isula -l 95%VG
      Logical volume "thinpool" created.
      ```

      ```bash
      # lvcreate --wipesignatures y -n thinpoolmeta isula -l 1%VG
      Logical volume "thinpoolmeta" created.
      ```

   5. Convert the two logical volumes into a thinpool and the metadata used by the thinpool.

      ```bash
      # lvconvert -y --zero n -c 512K --thinpool isula/thinpool --poolmetadata isula/thinpoolmeta

      WARNING: Converting logical volume isula/thinpool and isula/thinpoolmeta to
      thin pool's data and metadata volumes with metadata wiping.
      THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
      Converted isula/thinpool to thin pool.
      ```

2. Modifying the iSulad configuration files

   1. If iSulad has been used in the environment, back up the running data first.

      ```bash
      # mkdir /var/lib/isulad.bk
      # mv /var/lib/isulad/* /var/lib/isulad.bk
      ```

   2. Modify configuration files.

      Two configuration methods are provided. Select one based on site requirements.

      * Edit the  **/etc/isulad/daemon.json**  file, set  **storage-driver**  to  **devicemapper**, and set parameters related to the  **storage-opts**  field. For details about related parameters, see  [Parameter Description](#parameter-description). The following lists the configuration reference:

        ```json
        {
            "storage-driver": "devicemapper"
            "storage-opts": [
             "dm.thinpooldev=/dev/mapper/isula-thinpool",
             "dm.fs=ext4",
                    "dm.min_free_space=10%"
            ]
        }
        ```

      * You can also edit  **/etc/sysconfig/iSulad**  to explicitly specify related iSulad startup parameters. For details about related parameters, see  [Parameter Description](#parameter-description). The following lists the configuration reference:

        ```text
        OPTIONS="--storage-driver=devicemapper --storage-opt dm.thinpooldev=/dev/mapper/isula-thinpool --storage-opt dm.fs=ext4 --storage-opt dm.min_free_space=10%"
        ```

3. Start iSulad for the settings to take effect.

   ```bash
   # systemctl start isulad
   ```

#### Parameter Description

For details about parameters supported by storage-opts, see  [Table 1](#en-us_topic_0222861454_table3191161993812).

**Table  1**  Parameter description

#### Precautions

* When configuring devicemapper, if the system does not have sufficient space for automatic capacity expansion of thinpool, disable the automatic capacity expansion function.

  To disable automatic capacity expansion, set both  **thin\_pool\_autoextend\_threshold**  and  **thin\_pool\_autoextend\_percent**  in the  **/etc/lvm/profile/isula-thinpool.profile**  file to  **100**.

  ```text
  activation {   
    thin_pool_autoextend_threshold=100   
    thin_pool_autoextend_percent=100 
  }
  ```

* When devicemapper is used, use Ext4 as the container file system. You need to add  **--storage-opt dm.fs=ext4**  to the iSulad configuration parameters.

* If graphdriver is devicemapper and the metadata files are damaged and cannot be restored, you need to manually restore the metadata files. Do not directly operate or tamper with metadata of the devicemapper storage driver in Docker daemon.

* When the devicemapper LVM is used, if the devicemapper thinpool is damaged due to abnormal power-off, you cannot ensure the data integrity or whether the damaged thinpool can be restored. Therefore, you need to rebuild the thinpool.

##### Precautions for Switching the devicemapper Storage Pool When the User Namespace Feature Is Enabled on iSula

* Generally, the path of the deviceset-metadata file is  **/var/lib/isulad/devicemapper/metadata/deviceset-metadata**  during container startup.
* If user namespaces are used, the path of the deviceset-metadata file is  **/var/lib/isulad/***userNSUID.GID***/devicemapper/metadata/deviceset-metadata**.
* When you use the devicemapper storage driver and the container is switched between the user namespace scenario and common scenario, the  **BaseDeviceUUID**  content in the corresponding deviceset-metadata file needs to be cleared. In the thinpool capacity expansion or rebuild scenario, you also need to clear the  **BaseDeviceUUID**  content in the deviceset-metadata file. Otherwise, the iSulad service fails to be restarted.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/secure_container/installation_and_deployment_2.md
---
# Installation and Deployment

## Installation Methods

### Prerequisites

* The root permission is required for installing a Kata container.
* For better performance experience, a Kata container needs to run on the bare metal server and cannot run on VMs.
* A Kata container depends on the following components (openEuler 1.0 version). Ensure that the required components have been installed in the environment. To install iSulad, refer to  [Installation Configuration](../../container_engine/isula_container_engine/installation_configuration.md).
  * docker-engine
  * qemu

### Installation Procedure

Released Kata container components are integrated in the  **kata-containers-***version***.rpm**  package. You can run the  **rpm**  command to install the corresponding software.

```bash
rpm -ivh kata-containers-<version>.rpm
```

## Deployment Configuration

### Configuring the Docker Engine

To enable the Docker engine to support kata-runtime, perform the following steps to configure the Docker engine:

1. Ensure that all software packages (**docker-engine**  and  **kata-containers**) have been installed in the environment.

2. Stop the Docker engine.

   ```bash
   systemctl stop docker
   ```

3. Modify the configuration file  **/etc/docker/daemon.json**  of the Docker engine and add the following configuration:

   ```json
   {
     "runtimes": {
       "kata-runtime": {
         "path": "/usr/bin/kata-runtime",
         "runtimeArgs": [
             "--kata-config",
             "/usr/share/defaults/kata-containers/configuration.toml"
           ]
       }
     }
   }
   ```

4. Restart the Docker engine.

   ```bash
   systemctl start docker
   ```

### iSulad Configuration

To enable the iSulad to support the new container runtime kata-runtime, perform the following steps which are similar to those for the container engine docker-engine:

1. Ensure that all software packages (iSulad and kata-containers) have been installed in the environment.

2. Stop iSulad.

   ```bash
   systemctl stop isulad
   ```

3. Modify the  **/etc/isulad/daemon.json**  configuration file of the iSulad and add the following configurations:

   ```json
   {
     "runtimes": {
       "kata-runtime": {
         "path": "/usr/bin/kata-runtime",
         "runtime-args": [
             "--kata-config",
             "/usr/share/defaults/kata-containers/configuration.toml"
           ]
       }
     }
   }
   ```

4. Restart iSulad.

   ```bash
   systemctl start isulad
   ```

### Configuration.toml

The Kata container provides a global configuration file configuration.toml. Users can also customize the path and configuration options of the Kata container configuration file.

In the  **runtimeArges**  field of Docker engine, you can use  **--kata-config**  to specify a private file. The default configuration file path is  **/usr/share/defaults/kata-containers/configuration.toml**.

The following lists the common fields in the configuration file. For details about the configuration file options, see [configuration.toml](appendix_2.md#configurationtoml).

1. hypervisor.qemu
   * **path**: specifies the execution path of the virtualization QEMU.
   * **kernel**: specifies the execution path of the guest kernel.
   * **initrd**: specifies the guest initrd execution path.
   * **machine\_type**: specifies the type of the analog chip. The value is  **virt**  for the ARM architecture and  **pc**  for the x86 architecture.
   * **kernel\_params**: specifies the running parameters of the guest kernel.

2. proxy.kata
   * **path**: specifies the kata-proxy running path.
   * **enable\_debug**: enables the debugging function for the kata-proxy process.

3. agent.kata
   * **enable\_blk\_mount**: enables guest mounting of the block device.
   * **enable\_debug**: enables the debugging function for the kata-agent process.

4. runtime
   * **enable\_cpu\_memory\_hotplug**: enables CPU and memory hot swap.
   * **enable\_debug**: enables debugging for the kata-runtime process.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/hybrid_deployment/rubik/installation_and_deployment.md
---
# Installation and Deployment

## Overview

This chapter describes how to install and deploy the Rubik component.

## Software and Hardware Requirements

### Hardware

* Architecture: x86 or AArch64
* Drive: 1 GB or more
* Memory: 100 MB or more

### Software

* OS: openEuler 22.03-LTS-SP4
* Kernel: openEuler 22.03-LTS-SP4 kernel

### Environment Preparation

* Install the openEuler OS.
* Install and deploy Kubernetes.
* Install the Docker or containerd container engine.

## Installing Rubik

Rubik is deployed on each Kubernetes node as a DaemonSet. Therefore, you need to perform the following steps to install the Rubik RPM package on each node.

1. Configure the Yum repositories openEuler 22.03-LTS-SP4 and openEuler 22.03-LTS-SP4:EPOL (the Rubik component is available only in the EPOL repository).

   ```text
   # openEuler 22.03-LTS-SP4 official repository
   name=openEuler22.03-LTS-SP4-Epol
   baseurl=https://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/$basearch/
   enabled=1
   gpgcheck=1
   gpgkey=https://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/RPM-GPG-KEY-openEuler
   ```

   ```text
   # openEuler 22.03-LTS-SP4:EPOL official repository
   name=Epol
   baseurl=https://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/$basearch/
   enabled=1
   gpgcheck=0
   ```

2. Install Rubik with **root** permissions.

   ```shell
   sudo yum install -y rubik
   ```

> ![](./figures/icon-note.gif)**Note**:
>
> Files related to Rubik are installed in the **/var/lib/rubik** directory.

## Deploying Rubik

Rubik runs as a container in a Kubernetes cluster in hybrid deployment scenarios. It is used to isolate and restrict resources for services with different priorities to prevent offline services from interfering with online services, improving the overall resource utilization and ensuring the quality of online services. Currently, Rubik supports isolation and restriction of CPU and memory resources, and must be used together with the openEuler 22.03-LTS-SP4 kernel. To enable or disable the memory priority feature (that is, memory tiering for services with different priorities), you need to set the value in the **/proc/sys/vm/memcg\_qos\_enable** file. The value can be **0** or **1**. The default value **0** indicates that the feature is disabled, and the value **1** indicates that the feature is enabled.

```bash
sudo echo 1 > /proc/sys/vm/memcg_qos_enable
```

### Deploying Rubik DaemonSet

1. Run the **/var/lib/rubik/build\_rubik\_image.sh** script to automatically build a Rubik image. Because the script uses the `docker build` command, make sure Docker is available. You can also use the Docker engine to build the Rubik image. Because Rubik is deployed as a DaemonSet, each node requires a Rubik image. After building an image on a node, use the **docker save** and **docker load** commands to load the Rubik image to each node of Kubernetes. Alternatively, build a Rubik image on each node. The following uses docker as an example. The command is as follows:

   ```sh
   docker build -f /var/lib/rubik/Dockerfile -t rubik:2.0.0-1 .
   ```

2. On the Kubernetes master node, change the Rubik image name in the **/var/lib/rubik/rubik-daemonset.yaml** file to the name of the image built in the previous step.

   ```yaml
   ...
   containers:
   - name: rubik-agent
   image: rubik_image_name_and_tag  # The image name must be the same as the Rubik image name built in the previous step.
   imagePullPolicy: IfNotPresent
   ...
   ```

3. On the Kubernetes master node, run the **kubectl** command to deploy the Rubik DaemonSet so that Rubik will be automatically deployed on all Kubernetes nodes.

   ```sh
   kubectl apply -f /var/lib/rubik/rubik-daemonset.yaml
   ```

4. Run the **kubectl get pods -A** command to check whether Rubik has been deployed on each node in the cluster. (The number of rubik-agents is the same as the number of nodes and all rubik-agents are in the Running status.)

   ```sh
   $ kubectl get pods -A | grep rubik
   NAMESPACE     NAME                                            READY   STATUS    RESTARTS   AGE
   ...
   kube-system   rubik-agent-76ft6                               1/1     Running   0          4s
   ...
   ```

## Common Configuration Description

The Rubik deployed using the preceding method is started with the default configurations. You can modify the Rubik configurations as required by modifying the **config.json** section in the **rubik-daemonset.yaml** file and then redeploy the Rubik DaemonSet. The following describes some common configurations. For other configurations, see [Rubik Configuration Description](./configuration.md).

### Absolute Pod Preemption

If absolute pod preemption is enabled, you only need to specify the priority using annotations in the YAML file when deploying the service pods. After being deployed successfully, Rubik automatically detects the creation and update of the pods on the current node, and sets the pod priorities based on the configured priorities. For pods that are already started or whose annotations are modified, Rubik automatically updates the pod priority configurations.

```yaml
...
  "agent": {
    "enabledFeatures": [
      "preemption"
    ]
  },
  "preemption": {
    "resource": [
      "cpu",
      "memory"
    ]
  }
...
```

> Priority configurations support only pods switching from online to offline.

## Configuring Rubik for Online and Offline Services

After Rubik is successfully deployed, you can modify the YAML file of a service to specify the service type based on the following configuration example. Then Rubik can configure the priority of the service after it is deployed to isolate resources.

The following is an example of deploying an online Nginx service:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  namespace: qosexample
  annotations:
    volcano.sh/preemptable: "false"   # If volcano.sh/preemptable is set to true, the service is an offline service. If it is set to false, the service is an online service. The default value is false.
spec:
  containers:
  - name: nginx
    image: nginx
    resources:
      limits:
        memory: "200Mi"
        cpu: "1"
      requests:
        memory: "200Mi"
        cpu: "1"
```

---

---
url: /en/docs/22.03_LTS_SP4/cloud/kubeos/kubeos/installation_and_deployment.md
---
# Installation and Deployment

This chapter describes how to install and deploy the KubeOS tool.

## Software and Hardware Requirements

### Hardware Requirements

* Currently, only the x86 and AArch64 architectures are supported.

### Software Requirements

* OS: openEuler 22.03 LTS SP4

### Environment Preparation

* Install the openEuler system. For details, see the [*openEuler Installation Guide*](../../../server/installation_upgrade/installation/installation_on_servers.md).
* Install qemu-img, bc, Parted, tar, Yum, Docker, and dosfstools.

## KubeOS Installation

To install KubeOS, perform the following steps:

1. Configure the Yum sources openEuler 22.03-LTS-SP4 and openEuler 22.03-LTS-SP4:EPOL:

   ```text
   [openEuler22.03-LTS-SP4] # openEuler 22.03-LTS-SP4 official source
   name=openEuler22.03-LTS-SP4
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/ 
   enabled=1
   gpgcheck=1
   gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/RPM-GPG-KEY-openEuler
   ```

   ```text
   [Epol] # openEuler 22.03-LTS-SP4:EPOL official source
   name=Epol
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/$basearch/
   enabled=1
   gpgcheck=1
   gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler 
   ```

2. Install KubeOS as the **root** user.

   ```shell
   # yum install KubeOS KubeOS-scripts -y
   ```

> \[!NOTE]**NOTE**:
>
> KubeOS is installed in the **/opt/kubeOS** directory, including the os-operator, os-proxy, os-agent binary files, KubeOS image build tools, and corresponding configuration files.

## KubeOS Deployment

After KubeOS is installed, you need to configure and deploy it. This section describes how to configure and deploy KubeOS.

### Building the os-operator and os-proxy Images

#### Environment Preparation

Before using Docker to create a container image, ensure that Docker has been installed and configured.

#### Procedure

1. Go to the working directory.

   ```shell
   cd /opt/kubeOS
   ```

2. Specify the image repository, name, and version for os-proxy.

   ```shell
   export IMG_PROXY=your_imageRepository/os-proxy_imageName:version
   ```

3. Specify the image repository, name, and version for os-operator.

   ```shell
   export IMG_OPERATOR=your_imageRepository/os-operator_imageName:version
   ```

4. Compile a Dockerfile to build an image. Pay attention to the following points when compiling a Dockerfile:

   * The os-operator and os-proxy images must be built based on the base image. Ensure that the base image is safe.
   * Copy the os-operator and os-proxy binary files to the corresponding images.
   * Ensure that the owner and owner group of the os-proxy binary file in the os-proxy image are **root**, and the file permission is **500**.
   * Ensure that the owner and owner group of the os-operator binary file in the os-operator image are the user who runs the os-operator process in the container, and the file permission is **500**.
   * The locations of the os-operator and os-proxy binary files in the image and the commands run during container startup must correspond to the parameters specified in the YAML file used for deployment.

   An example Dockerfile is as follows:

   ```text
   FROM your_baseimage
   COPY ./bin/proxy /proxy
   ENTRYPOINT ["/proxy"]
   ```

   ```text
   FROM your_baseimage
   COPY --chown=6552:6552 ./bin/operator /operator
   ENTRYPOINT ["/operator"]
   ```

   Alternatively, you can use multi-stage builds in the Dockerfile.

5. Build the images (the os-operator and os-proxy images) to be included in the containers OS image.

   ```shell
   # Specify the Dockerfile path of os-proxy.
   export DOCKERFILE_PROXY=your_dockerfile_proxy
   # Specify the Dockerfile path of os-operator.
   export DOCKERFILE_OPERATOR=your_dockerfile_operator
   # Build images.
   docker build -t ${IMG_OPERATOR} -f ${DOCKERFILE_OPERATOR} .
   docker build -t ${IMG_PROXY} -f ${DOCKERFILE_PROXY} .
   ```

6. Push the images to the image repository.

   ```shell
   docker push ${IMG_OPERATOR}
   docker push ${IMG_PROXY}
   ```

### Creating a KubeOS VM Image

#### Precautions

* The VM image is used as an example. For details about how to create a physical machine image, see [**KubeOS Image Creation**](./kubeos_image_creation.md).
* The root permission is required for creating a KubeOS image.
* The RPM sources of the kbimg are the **everything** and **EPOL** repositories of openEuler of a specific version. In the Repo file provided during image creation, you are advised to configure the **everything** and **EPOL** repositories of a specific openEuler version for the Yum source.
* By default, the KubeOS VM image built using the default RPM list is stored in the same path as the kbimg tool. This partition must have at least 25 GiB free drive space.
* When creating a KubeOS image, you cannot customize the file system to be mounted.

#### Procedure

Use the **kbimg.sh** script to create a KubeOS VM image. For details about the commands, see [**KubeOS Image Creation**](./kubeos_image_creation.md).

To create a KubeOS VM image, perform the following steps:

1. Go to the working directory.

   ```shell
   cd /opt/kubeOS/scripts
   ```

2. Run `kbming.sh` to create a KubeOS image. The following is a command example:

   ```shell
   bash kbimg.sh create vm-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0'''
   ```

   In the command, **xx.repo** indicates the actual Yum source file used for creating the image. You are advised to configure both the **everything** and **EPOL** repositories as Yum sources.

   After the KubeOS image is created, the following files are generated in the **/opt/kubeOS/scripts** directory:

   * **system.img**: system image in raw format. The default size is 20 GB. The size of the root file system partition is less than 2,020 MiB, and the size of the Persist partition is less than 16 GiB.
   * **system.qcow2**: system image in QCOW2 format.
   * **update.img**: partition image of the root file system that is used for upgrade.

   The created KubeOS VM image can be used only in a VM of the x86 or AArch64 architecture. KubeOS does not support legacy boot in an x86 VM

### Deploying CRD, os-operator, and os-proxy

#### Precautions

* The Kubernetes cluster must be deployed first. For details, see the [*Kubernetes Cluster Deployment Guide*](../../cluster_deployment/kubernetes/overview.md).

* The OS of the worker nodes to be upgraded in the cluster must be the KubeOS built using the method described in the previous section. If it is not, use **system.qcow2** to deploy the VM again. For details about how to deploy a VM, see the [*Virtualization User Guide*](../../../virtualization/virtualization_platform/virtualization/introduction_to_virtualization.md). Currently, KubeOS does not support the master nodes. Use openEuler 22.03-LTS-SP4 to deploy the upgrade on the master nodes.

* The YAML files for deploying CustomResourceDefinition (CRD), os-operator, os-proxy, and role-based access control (RBAC) of the OS need to be compiled.

* The os-operator and os-proxy components are deployed in the Kubernetes cluster. os-operator must be deployed as a Deployment, and os-proxy as a DaemonSet.

* Kubernetes security mechanisms, such as the RBAC, pod service account, and security policies, must be deployed.

#### Procedure

1. Prepare YAML files used for deploying CRD, RBAC, os-operator, and os-proxy of the OS. For details, see [YAML examples](https://atomgit.com/openeuler/KubeOS/tree/master/docs/example/config). The following uses **crd.yaml**, **rbac.yaml**, and **manager.yaml** as examples.

2. Deploy CRD, RBAC, os-operator, and os-proxy. Assume that the **crd.yaml**, **rbac.yaml**, and **manager.yaml** files are stored in the **config/crd**, **config/rbac**, and **config/manager** directories, respectively. Run the following commands:

   ```shell
   kubectl apply -f config/crd
   kubectl apply -f config/rbac 
   kubectl apply -f config/manager
   ```

3. After the deployment is complete, run the following command to check whether each component is started properly. If **STATUS** of all components is **Running**, the components are started properly.

   ```shell
   kubectl get pods -A
   ```

---

---
url: /en/docs/22.03_LTS_SP4/cloud/nestos/nestos/installation_and_deployment.md
---
# Installation and Deployment

## Deploying NestOS on VMware

This guide describes how to configure latest NestOS in VMware.

Currently, NestOS supports only the x86\_64 architecture.

### Before You Start

​Before deploying NestOS, make the following preparations:

* Downloading the NestOS ISO
* Preparing the **config.bu** File
* Configuring the Butane Tool (on Linux or Windows 10)
* A host machine with VMware installed

### Initial Installation and Startup

#### Starting NestOS

When NestOS is started for the first time, Ignition is not installed. You can use the nestos-installer component to install Ignition as prompted.

### Producing an Ignition File

#### Obtaining Butane

You can use Butane to convert a .bu file into an Ignition file. Ignition configurations were designed to be human readable, but difficult to write, to discourage users from attempting to write configs by hand.
Butane supports multiple environments. You can use Butane in a Linux or Windows host machines or in container environments.

```shell
docker pull quay.io/coreos/butane:release
```

#### Generating a Login Password

Run the following command on the host machine and enter the password:

```shell
# openssl passwd -1 -salt yoursalt
Password:
$1$yoursalt$1QskegeyhtMG2tdh0ldQN0
```

#### Generating an SSH Key Pair

Run the following command on the host machine to obtain the public key and private key for SSH login:

```shell
# ssh-keygen -N '' -f ./id_rsa
Generating public/private rsa key pair.
Your identification has been saved in ./id_rsa
Your public key has been saved in ./id_rsa.pub
The key fingerprint is:
SHA256:4fFpDDyGHOYEd2fPaprKvvqst3T1xBQuk3mbdon+0Xs root@host-12-0-0-141
```

```text
The key's randomart image is:
+---[RSA 3072]----+
|   ..= . o .   |
|    * = o * .  |
|     + B = *   |
|     o B O + . |
|      S O B o  |
|       * = . . |
|      . +o . . |
|     +.o . .E  |
|    o*Oo   ... |
+----[SHA256]-----+
```

You can view the **id\_rsa.pub** public key in the current directory.

```shell
# cat id_rsa.pub
ssh-rsa
AAAAB3NzaC1yc2...
```

#### Compiling a .bu File

Perform a simple initial configuration. For more details, see the description of Ignition.
A simple **config.bu** file is as follows:

```text
variant: fcos
version: 1.1.0
passwd:
  users:
    - name: nest
      password_hash: "$1$yoursalt$1QskegeyhtMG2tdh0ldQN0"
      ssh_authorized_keys:
        - "ssh-rsa
  AAAAB3NzaC1yc2EAAA..."
```

#### Generating an Ignition File

Use the Butane tool to convert the **config.bu** file to a **config.ign** file in the container environment.

```shell
# docker run --interactive --rm quay.io/coreos/butane:release \
--pretty --strict < your_config.bu > transpiled_config.ign
```

### Installing NestOS

Use SCP to copy the **config.ign** file generated by the host machine to NestOS that is initially started, which is not installed to the disk and runs in the memory.

```shell
sudo -i
scp root@your_ipAddress:/root/config.ign /root            
```

Run the following command and complete the installation as prompted:

```shell
nestos-installer install /dev/sda --ignition-file config.ign
```

After the installation is complete, restart NestOS.

```shell
systemctl reboot
```

Complete.

---

---
url: /en/docs/22.03_LTS_SP4/edge_computing/ros/installation_and_deployment.md
---
# Installation and Deployment

## Software

* OS: openEuler 22.03 LTS SP4

## Hardware

* x86\_64

## Preparing the Environment

* Install the openEuler OS by referring to the [*openEuler 22.03 LTS SP4 Installation Guide*](./../../server/installation_upgrade/installation/installation_guide.md).

## 1. ROS2

### 1. ros-humble

#### 1. Installing ros-humble

1. Install ros-humble software package

   ```shell
   yum install openeuler-ros
   yum install ros-humble-ros-base ros-humble-xxx e.g. ros-humble-turtlesim
   ```

2. Run the following command to check whether the installation is successful

   ```shell
   rpm -q ros-humble
   ```

#### 2. Test ros-humble

##### Run turtlesim

1. Run turtlesim

   ```shell
   source /opt/ros/humble/setup.bash
   ros2 run turtlesim turtlesim_node
   ```

2. Open turtlesim terminal

   ```shell
   source /opt/ros/humble/setup.bash
   ros2 run turtlesim turtle_teleop_key
   ```

3. Use the arrow keys to control the movement of the turtle
   ![ros-humble](./figures/ros-humble.png)

### 2. ros-foxy

#### 1. Installing ros-foxy-ros-base

1. Download the software package

   ```shell
   wget http://121.36.3.168:82/home:/Chenjy3_22.03/openEuler_22.03_LTS_standard_x86_64/x86_64/ros-foxy-ros-base-0.9.2-2.oe2203.x86_64.rpm
   ```

2. Install the rpm package

   ```shell
   rpm -ivh --nodeps --force ros-foxy-ros-base-0.9.2-2.oe2203.x86_64.rpm
   ```

3. Dependence installation

   ```shell
   sh /opt/ros/foxy/install_dependence.sh
   ```

4. Run the following command to check whether the installation is successful

   ```shell
   rpm -q ros-foxy-ros-base
   ```

#### 2. Test ros-foxy-ros-base

##### Run turtlesim

1. Run turtlesim

   ```shell
   source /opt/ros/foxy/setup.bash
   ros2 run turtlesim turtlesim_node
   ```

2. Open turtlesim terminal

   ```shell
   source /opt/ros/foxy/setup.bash
   ros2 run turtlesim turtle_teleop_key
   ```

3. Use the arrow keys to control the movement of the turtle

   ![ROS2-turtlesim](./figures/turtlesim.png)

## 2. ROS

### 1. ros-noetic

#### 1. Installing ros-noetic-ros-comm

1. Download the software package

   ```shell
   wget http://121.36.3.168:82/home:/davidhan:/branches:/openEuler:/22.03:/LTS:/SP1:/Epol/standard_x86_64/x86_64/ros-noetic-ros-comm-1.15.11-2.oe2203.x86_64.rpm
   ```

2. Install the rpm package

   ```shell
   rpm -ivh --nodeps --force ros-noetic-ros-comm-1.15.11-2.oe2203.x86_64.rpm
   ```

3. Dependence installation

   ```shell
   sh /opt/ros/noetic/install_dependence.sh
   ```

4. Run the following command to check whether the installation is successful

   ```shell
   rpm -q ros-noetic-ros-comm
   ```

#### 2. Test ros-noetic-ros-comm

##### Run topic\_demo

1. Create and compile workspace

   ```shell
   mkdir -p catkin_ws/src
   cd catkin_ws/src/
   source /opt/ros/noetic/setup.bash
   catkin_init_workspace
   git clone https://gitee.com/davidhan008/topic_demo.git
   cd ..
   catkin_make
   ```

2. run roscore

   ```shell
   source /opt/ros/noetic/setup.bash
   roscore
   ```

3. run topic\_demo talker

   ```shell
   source /opt/ros/noetic/setup.bash
   cd catkin_ws
   source devel/setup.bash
   rosrun topic_demo talker
   ```

4. run topic\_demo listener

   ```shell
   source /opt/ros/noetic/setup.bash
   cd catkin_ws
   source devel/setup.bash
   rosrun topic_demo listener
   ```

   ![ROS2-turtlesim](./figures/ROS-demo.png)

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/sysmaster/devmaster_install_deploy.md
---
# Installation and Deployment

Currently, devmaster can be used in the VM environment where sysmaster is used as PID 1. This section describes the requirements and procedure of devmaster installation and deployment.

## Software

* OS: openEuler 22.03 LTS SP4

## Hardware

* x86\_64 or AArch64 architecture

## Installation and Deployment

1. Run the following `yum` command to install the devmaster package:

   ```shell
   # yum install devmaster
   ```

2. The devmaster package includes a service configuration file for sysmaster. After the package is installed, devmaster overwrites udev services and will be started by sysmaster upon startup. After the devmaster package is uninstalled, udev services are restored automatically.

3. Restart the system.

4. Check the **/run/devmaster/data/** directory. If the device database file is generated, the deployment is successful.

   ```shell
   # ll /run/devmaster/data/
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/sysmaster/sysmaster_install_deploy.md
---
# Installation and Deployment

The sysmaster service can be used in containers and VMs. This document uses the AArch64 architecture as an example to describe how to install and deploy sysmaster in both scenarios.

## Software

* OS: openEuler 22.03 LTS SP4

## Hardware

* x86\_64 or AArch64 architecture

## Installation and Deployment in Containers

1. Install Docker.

   ```bash
   yum install -y docker
   systemctl restart docker
   ```

2. Load the base container image.

   Download the container image.

   ```bash
   wget https://repo.openeuler.org/openEuler-22.03-LTS-SP4/docker_img/aarch64/openEuler-docker.aarch64.tar.xz
   xz -d openEuler-docker.aarch64.tar.xz
   ```

   Load the container image.

   ```bash
   docker load --input openEuler-docker.aarch64.tar
   ```

3. Build the container.

   Create a Dockerfile based on the image name queried by the `docker images` command, for example, **openEuler-22.03-LTS-SP4**.

   ```bash
   cat << EOF > Dockerfile
   FROM openEuler-22.03-LTS-SP4
   RUN yum install -y sysmaster
   CMD ["/usr/lib/sysmaster/init"]
   EOF
   ```

   Build the container.

   ```bash
   docker build -t openEuler-22.03-LTS-SP4:latest .
   ```

4. Start and enter the container.

   Start the container.

   ```bash
   docker run -itd --privileged openEuler-22.03-LTS-SP4:latest
   ```

   Obtain the container ID.

   ```bash
   docker ps
   ```

   Use the container ID to enter the container.

   ```bash
   docker exec -it CONTAINERID /bin/bash
   ```

## Installation and Deployment in VMs

1. Create an initramfs image.\
   To avoid the impact of systemd in the initrd phase, you need to create an initramfs image with systemd removed and use this image to enter the initrd procedure. Run the following command:

   ```bash
   dracut -f --omit "systemd systemd-initrd systemd-networkd dracut-systemd" /boot/initrd_withoutsd.img
   ```

2. Add a boot item.\
   Add a boot item to **grub.cfg**, whose path is **/boot/efi/EFI/openEuler/grub.cfg** in the AArch64 architecture and **/boot/grub2/grub.cfg** in the x86\_64 architecture. Back up the original configurations and modify the configurations as follows:

   * **menuentry**: Set the item name to **openEuler sysmaster**.
   * **linux**: Change **root=/dev/mapper/openeuler-root ro** to **root=/dev/mapper/openeuler-root rw**.
   * **linux**: if Plymouth is installed in the environment, add **plymouth.enable=0** to disable it.
   * **linux**: Add **init=/usr/lib/sysmaster/init**.
   * **initrd**: Set to **/initrd\_withoutsd.img**.

3. Install sysmaster.

   ```bash
   yum install sysmaster
   ```

4. If the **openEuler sysmaster** boot item is displayed after the restart, the configuration is successful. Select it to log in to the VM.

---

---
url: /en/docs/22.03_LTS_SP4/server/development/gcc/pin_user_guide.md
---
# Installation and Deployment

## Software

* OS: openEuler 22.03 LTS SP4

## Hardware

* x86\_64
* AArch64

## Preparing the Environment

* Install the openEuler operating system. For details, see the [*openEuler Installation Guide*](./../../../server/installation_upgrade/installation/installation_guide.md).

### Install the dependency

#### Installing the Software on Which the PIN GCC Client Depends

```shell
yum install -y git
yum install -y make
yum install -y cmake
yum install -y grpc
yum install -y grpc-devel
yum install -y grpc-plugins
yum install -y protobuf-devel
yum install -y jsoncpp
yum install -y jsoncpp-devel
yum install -y gcc-plugin-devel
yum install -y llvm-mlir
yum install -y llvm-mlir-devel
yum install -y llvm-devel
```

#### Installing the Software on Which the PIN Server Depends

```shell
yum install -y git
yum install -y make
yum install -y cmake
yum install -y grpc
yum install -y grpc-devel
yum install -y grpc-plugins
yum install -y protobuf-devel
yum install -y jsoncpp
yum install -y jsoncpp-devel
yum install -y llvm-mlir
yum install -y llvm-mlir-devel
yum install -y llvm-devel
```

## Installing PIN

### Performing Installation

#### Installing the PIN GCC Client

```shell
yum install -y pin-gcc-client
```

#### Installing the PIN Server

```shell
yum install -y pin-server
```

### Build

#### Building the PIN GCC Client

```shell
git clone https://atomgit.com/openeuler/pin-gcc-client.git
cd pin-gcc-client
mkdir build
cd build
cmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH}
make
```

#### Building the PIN Server

```shell
git clone https://atomgit.com/openeuler/pin-server.git
cd pin-server
mkdir build
cd build
cmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH}
make
```

# Usage

You can use `-fplugin` and `-fplugin-arg-libpin_xxx` to enable the Plug-IN (PIN) tool.
Command:

```shell
$(TARGET): $(OBJS)
    $(CXX) -fplugin=${CLIENT_PATH}/libpin_gcc_client.so \
    -fplugin-arg-libpin_gcc_client-server_path=${SERVER_PATH}/pin_server \
    -fplugin-arg-libpin_gcc_client-log_level="1" \
    -fplugin-arg-libpin_gcc_client-arg1="xxx"
```

You can use the `${INSTALL_PATH}/bin/pin-gcc-client.json` file to configure PIN. The configuration options are as follows:

* `path`: path of the executable file of the PIN server.
* `sha256file`: path of the PIN verification file `xxx.sha256`.
* `timeout`: timeout interval for cross-process communication, in milliseconds.

Compile options:

* `-fplugin`: path of the .so file of the PIN client.
* `-fplugin-arg-libpin_gcc_client-server_path`: path of the executable program of the PIN server.
* `-fplugin-arg-libpin_gcc_client-log_level`: default log level. The value ranges from `0` to `3`. The default value is `1`.
* `-fplugin-arg-libpin_gcc_client-argN`: other parameters that can be specified as required. `argN` indicates the argument required by PIN.

# Compatibility

This section describes the compatibility issues in some special scenarios. This project is in continuous iteration and will be fixed as soon as possible. Developers are welcome to join this project.

* When PIN is enabled in the `-flto` phase, multi-process compilation using `make -j` is not supported. You are advised to use `make -j1` for compilation.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/maintenance/kernel_live_upgrade/installation_and_deployment.md
---
# Installation and Deployment

This document describes how to install and deploy the kernel live upgrade tool.

## Hardware and Software Requirements

### Hardware Requirements

* Currently, only the ARM64 architecture is supported.

### Software Requirements

* Operating system: openEuler 22.03 LTS SP4

## Environment Preparation

* Install the openEuler system. For details, see the [*openEuler 22.03 LTS SP4 Installation Guide*](./../../installation_upgrade/installation/installation_guide.md).

* The root permission is required for installing the kernel live upgrade tool.

## Installing the Kernel Live Upgrade Tool

This section describes how to install the kernel live upgrade tool.

Perform the following steps:

1. Mount the ISO file of openEuler.

   ```shell
   mount openEuler-22.03-LTS-SP4-aarch64-dvd.iso /mnt
   ```

2. Configure the local yum repository.

   ```shell
   vi /etc/yum.repos.d/local.repo
   ```

   The configurations are as follows:

   ```text
   [local]
   name=local
   baseurl=file:///mnt
   gpgcheck=1
   enabled=1
   ```

3. Import the GPG public key of the RPM digital signature to the system.

   ```shell
   rpm --import /mnt/RPM-GPG-KEY-openEuler
   ```

4. Install the kernel live upgrade tool.

   ```shell
   yum install nvwa -y
   ```

5. Check whether the installation is successful. If the command output is as follows, the installation is successful.

   ```shell
   $ rpm -qa | grep nvwa
   nvwa-xxx
   ```

## Deploying the Kernel Live Upgrade Tool

This section describes how to configure and deploy the kernel live upgrade tool.

### Configurations

The configuration files of the kernel live upgrade tool are stored in /etc/nvwa. The configuration files are as follows:

* nvwa-restore.yaml

  This configuration file is used to instruct the kernel live upgrade tool to save and recover the process during the kernel live upgrade. The configuration is as follows:

  * pids

    Specifies the processes that need to be retained and recovered during the NVWA live upgrade. The processes are identified by process ID (PID). Note that the processes managed by NVWA are automatically recovered after the NVWA service is started.

  * services

    Specifies the services that need to be retained and recovered during NVWA live upgrade. Compared to PIDs, the kernel live upgrade tool can directly save and recover the process. For services, the kernel live upgrade tool depends on the systemd to perform related operations. The service name must be the same as the service name used in systemd. Note that whether the service managed by NVWA needs to be automatically recovered when the NVWA is started depends on whether the service is enabled in the systemd. Currently, only the notify and oneshot service types are supported.

  * restore\_net

    Specifies whether the kernel live upgrade tool is required to save and recover the network configuration. If the network configuration is incorrect, the network may be unavailable after the recovery. This function is disabled by default.

  * enable\_quick\_kexec

    Used to specify whether to enable the quick kexec feature. quick kexec is a feature launched by the NVWA community to accelerate the kernel restart process. To use this feature, add "quickkexec=128M" to cmdline. 128 indicates the size of the memory allocated to the quick kexec feature. The memory is used to load the kernel and initramfs during the upgrade. Therefore, the size must be greater than the total size of the kernel and initramfs involved in the upgrade. This feature is disabled by default.

  * enable\_pin\_memory

    Used to specify whether to enable the pin memory feature. pin memory is a feature launched by the NVWA community to accelerate the process storage and recovery process. The pin\_memory feature is not supported for multi-process recovery. To use this feature, you need to add "max\_pin\_pid\_num=10 redirect\_space\_size=2M pinmemory=200M@0x640000000" to cmdline.

    max\_pin\_pid\_num indicates the maximum number of processes that support pin memory recovery. redirect\_space\_size indicates the reserved memory space required for redirecting physical pages during pin memory recovery. You are advised to set redirect\_space\_size to 1/100 of the total reserved pin memory. pinmemory indicates the start point and size of the memory segment. The 200 MB space starting from 0x640000000 is the total memory space used by the pin memory. This space should not be used by other programs.

* Configuration example of **nvwa-restore.yaml**

```yaml
pids:
  - 14109
services:
  - redis
restore_net: false
enable_quick_kexec: true
enable_pin_memory: true
```

* **nvwa-server.yaml**

  This file contains the configuration information required during the running of the kernel live upgrade tool. The details are as follows:

  * criu\_dir

    This parameter specifies the directory for storing the information generated when the kernel live upgrade tool saves the running information. Note that the information may occupy a large amount of disk space.

  * criu\_exe

    This parameter specifies the path of the CRIU executable file used by the kernel live upgrade tool. You are advised not to change the path unless you need to debug the CRIU.

  * kexec\_exe

    This parameter specifies the path of the kexec executable file used by the kernel live upgrade tool. You are advised not to change the path unless you need to debug kexec.

  * systemd\_etc

    This parameter specifies the path of the folder used to overwrite the systemd configuration file. The path is determined by the systemd. Generally, you do not need to change the path.

  * log\_dir

    This parameter stores the log information generated by the kernel live upgrade tool. The log module is not enabled currently. For details about how to view logs of the kernel live upgrade tool, see [*How to Run*](./usage_guide.md#generated-log-information).

* Configuration example of **nvwa-server.yaml**

```yaml
criu_dir: /var/nvwa/running/
criu_exe: /usr/sbin/criu
kexec_exe: /usr/sbin/kexec
systemd_etc: /etc/systemd/system/
log_dir: /etc/nvwa/log/
```

## Enabling the Kernel Live Upgrade Tool

The running of the kernel live upgrade tool depends on the configuration file. After the configuration file is modified, you need to run the kernel live upgrade tool again.

After the installation is successful, you can run the systemd commands to operate the kernel live upgrade tool.

* Enable NVWA.

  ```sh
  systemctl enable nvwa
  ```

* Start nvwa.

  ```sh
  systemctl start nvwa
  ```

* View the nvwa service status and other information.

  ```sh
  systemctl status nvwa
  ```

* For more usage, see the usage of systemd.

---

---
url: /en/docs/22.03_LTS_SP4/server/performance/atune/installation_and_deployment.md
---
# Installation and Deployment

This chapter describes how to install and deploy A-Tune.

## Software and Hardware Requirements

### Hardware Requirement

* Huawei Kunpeng 920 processor

### Software Requirement

* OS: openEuler 22.03 LTS SP4

## Environment Preparation

* For details about installing an openEuler OS, see the *openEuler 22.03 LTS SP4 Installation Guide*.
* Root permissions are required for installing A-Tune.

## A-Tune Installation

This section describes the installation modes and methods of A-Tune.

### Installation Modes

A-Tune can be installed in single-node and distributed modes.

* Single-node mode

  The client and server are installed on the same system.

* Distributed mode

  The client and server are installed on different systems.

The installation modes are as follows:

![](./figures/en-us_image_0231122163.png)

### Installation Procedure

To install A-Tune, perform the following steps:

1. Mount an openEuler ISO image.

   ```shell
   mount openEuler-22.03-LTS-SP4-aarch64-dvd.iso /mnt
   ```

2. Configure the local Yum source.

   ```shell
   vim /etc/yum.repos.d/local.repo
   ```

   The configured contents are as follows:

   ```shell
   [local]
   name=local
   baseurl=file:///mnt
   gpgcheck=1
   enabled=1
   ```

3. Import the GPG public key of the RPM digital signature to the system.

   ```shell
   rpm --import /mnt/RPM-GPG-KEY-openEuler
   ```

4. Install an A-Tune server.

   > \[!NOTE] **NOTE:**
   > In this step, both the server and client software packages are installed. For the single-node deployment, skip  **Step 5**.

   ```shell
   yum install atune -y
   yum install atune-engine -y
   ```

5. For a distributed mode, install an A-Tune client on associated server.

   ```shell
   yum install atune-client -y
   ```

6. Check whether the installation is successful.

   ```shell
   $ rpm -qa | grep atune
   atune-client-xxx
   atune-db-xxx
   atune-xxx
   atune-engine-xxx
   ```

   If the preceding information is displayed, the installation is successful.

## A-Tune Deployment

This section describes how to deploy A-Tune.

### Overview

The configuration items in the A-Tune configuration file  **/etc/atuned/atuned.cnf**  are described as follows:

* A-Tune service startup configuration (modify the parameter values as required).

  * **protocol**: Protocol used by the gRPC service. The value can be  **unix**  or  **tcp**.  **unix**  indicates the local socket communication mode, and  **tcp**  indicates the socket listening port mode. The default value is  **unix**.
  * **address**: Listening IP address of the gRPC service. The default value is  **unix socket**. If the gRPC service is deployed in distributed mode, change the value to the listening IP address.
  * **port**: Listening port of the gRPC server. The value ranges from 0 to 65535. If **protocol** is set to **unix**, you do not need to set this parameter.
  * **connect**: IP address list of the nodes where the A-Tune is located when the A-Tune is deployed in a cluster. IP addresses are separated by commas (,).
  * **rest\_host**: Listening address of the REST service. The default value is localhost.
  * **rest\_port**: Listening port of the REST service. The value ranges from 0 to 65535. The default value is 8383.
  * **engine\_host**: IP address for connecting to the A-Tune engine service of the system.
  * **engine\_port**: Port for connecting to the A-Tune engine service of the system.
  * **sample\_num**: Number of samples collected when the system executes the analysis process. The default value is 20.
  * **interval**: Interval for collecting samples when the system executes the analysis process. The default value is 5s.
  * **grpc\_tls**: Indicates whether to enable SSL/TLS certificate verification for the gRPC service. By default, this function is disabled. After grpc\_tls is enabled, you need to set the following environment variables before running the **atune-adm** command to communicate with the server:
    * export ATUNE\_TLS=yes
    * export ATUNED\_CACERT=\<Path of the client CA certificate>
    * export ATUNED\_CLIENTCERT=\<Client certificate path>
    * export ATUNED\_CLIENTKEY=\<Client key path>
    * export ATUNED\_SERVERCN=server
  * **tlsservercafile**: Path of the gPRC server's CA certificate.
  * **tlsservercertfile**: Path of the gPRC server certificate.
  * **tlsserverkeyfile**: Path of the gPRC server key.
  * **rest\_tls**: Indicates whether to enable SSL/TLS certificate verification for the REST service. This function is enabled by default.
  * **tlsrestcacertfile**: Path of the server's CA certificate of the REST service.
  * **tlsrestservercertfile**: Path of the server certificate of the REST service.
  * **tlsrestserverkeyfile**: Indicates the key path of the REST service.
  * **engine\_tls**: Indicates whether to enable SSL/TLS certificate verification for the A-Tune engine service. This function is enabled by default..
  * **tlsenginecacertfile**: Path of the client CA certificate of the A-Tune engine service.
  * **tlsengineclientcertfile**: Client certificate path of the A-Tune engine service.
  * **tlsengineclientkeyfile**: Client key path of the A-Tune engine service.

* System information

  System is the parameter information required for system optimization. You must modify the parameter information according to the actual situation.

  * **disk**: Disk information to be collected during the analysis process or specified disk during disk optimization.
  * **network**: NIC information to be collected during the analysis process or specified NIC during NIC optimization.
  * **user**: User name used for ulimit optimization. Currently, only the user  **root**  is supported.

* Log information

  Change the log level as required. The default log level is info. Log information is recorded in the **/var/log/messages** file.

* Monitor information

  Hardware information that is collected by default when the system is started.

* Tuning information

  Tuning is the parameter information required for offline tuning.

  * **noise**: Evaluation value of Gaussian noise.
  * **sel\_feature**: Indicates whether to enable the function of generating the importance ranking of offline tuning parameters. By default, this function is disabled.

#### Example

```text
#################################### server ############################### 
 # atuned config 
 [server] 
 # the protocol grpc server running on 
 # ranges: unix or tcp 
 protocol = unix 

 # the address that the grpc server to bind to
 # default is unix socket /var/run/atuned/atuned.sock
 # ranges: /var/run/atuned/atuned.sock or ip address
 address = /var/run/atuned/atuned.sock 

 # the atune nodes in cluster mode, separated by commas
 # it is valid when protocol is tcp
 # connect = ip01,ip02,ip03

 # the atuned grpc listening port
 # the port can be set between 0 to 65535 which not be used
 # port = 60001

 # the rest service listening port, default is 8383
 # the port can be set between 0 to 65535 which not be used
 rest_host = localhost
 rest_port = 8383

 # the tuning optimizer host and port, start by engine.service
 # if engine_host is same as rest_host, two ports cannot be same
 # the port can be set between 0 to 65535 which not be used
 engine_host = localhost
 engine_port = 3838

 # when run analysis command, the numbers of collected data.
 # default is 20
 sample_num = 20

 # interval for collecting data, default is 5s
 interval = 5

 # enable gRPC authentication SSL/TLS
 # default is false
 # grpc_tls = false
 # tlsservercafile = /etc/atuned/grpc_certs/ca.crt
 # tlsservercertfile = /etc/atuned/grpc_certs/server.crt
 # tlsserverkeyfile = /etc/atuned/grpc_certs/server.key

 # enable rest server authentication SSL/TLS
 # default is true
 rest_tls = true
 tlsrestcacertfile = /etc/atuned/rest_certs/ca.crt
 tlsrestservercertfile = /etc/atuned/rest_certs/server.crt
 tlsrestserverkeyfile = /etc/atuned/rest_certs/server.key

 # enable engine server authentication SSL/TLS
 # default is true
 engine_tls = true
 tlsenginecacertfile = /etc/atuned/engine_certs/ca.crt
 tlsengineclientcertfile = /etc/atuned/engine_certs/client.crt
 tlsengineclientkeyfile = /etc/atuned/engine_certs/client.key

 #################################### log ############################### 
 [log]
 # either "debug", "info", "warn", "error", "critical", default is "info"
 level = info

 #################################### monitor ############################### 
 [monitor]
 # with the module and format of the MPI, the format is {module}_{purpose}
 # the module is Either "mem", "net", "cpu", "storage"
 # the purpose is "topo"
 module = mem_topo, cpu_topo

 #################################### system ############################### 
 # you can add arbitrary key-value here, just like key = value
 # you can use the key in the profile
 [system]
 # the disk to be analysis
 disk = sda

 # the network to be analysis
 network = enp189s0f0

 user = root
 
 #################################### tuning ###############################
 # tuning configs
 [tuning]
 noise = 0.000000001
 sel_feature = false
```

The configuration items in the configuration file **/etc/atuned/engine.cnf** of the A-Tune engine are described as follows:

* Startup configuration of the A-Tune engine service (modify the parameter values as required).

  * **engine\_host**: Listening address of the A-Tune engine service. The default value is localhost.
  * **engine\_port**: Listening port of the A-Tune engine service. The value ranges from 0 to 65535. The default value is 3838.
  * **engine\_tls**: Indicates whether to enable SSL/TLS certificate verification for the A-Tune engine service. This function is enabled by default.
  * **tlsenginecacertfile**: Path of the server CA certificate of the A-Tune engine service.
  * **tlsengineservercertfile**: Path of the server certificate of the A-Tune engine service.
  * **tlsengineserverkeyfile**: Server key path of the A-Tune engine service.

* Log information

  Change the log level as required. The default log level is info. Log information is recorded in the **/var/log/messages** file.

#### Example

```text
#################################### engine ###############################
 [server]
 # the tuning optimizer host and port, start by engine.service
 # if engine_host is same as rest_host, two ports cannot be same
 # the port can be set between 0 to 65535 which not be used
 engine_host = localhost
 engine_port = 3838

 # enable engine server authentication SSL/TLS
 # default is true
 engine_tls = true
 tlsenginecacertfile = /etc/atuned/engine_certs/ca.crt
 tlsengineservercertfile = /etc/atuned/engine_certs/server.crt
 tlsengineserverkeyfile = /etc/atuned/engine_certs/server.key

 #################################### log ###############################
 [log]
 # either "debug", "info", "warn", "error", "critical", default is "info"
 level = info
```

## Starting A-Tune

After A-Tune is installed, you need to configure the A-Tune service before starting it.

* Start the atuned service.

  ```shell
  systemctl start atuned
  ```

* Query the atuned service status.

  ```shell
  systemctl status atuned
  ```

  If the following command output is displayed, the service is started successfully:

  ![](./figures/en-us_image_0214540398.png)

## Starting A-Tune Engine

To use AI functions, you need to start the A-Tune engine service.

* Start the atune-engine service.

  ```shell
  systemctl start atune-engine
  ```

* Query the atune-engine service status.

  ```shell
  systemctl status atune-engine
  ```

  If the following command output is displayed, the service is started successfully:

  ![](./figures/en-us_image_0245342444.png)

---

---
url: /en/docs/22.03_LTS_SP4/server/performance/powerapi/installation_usage.md
---
# Installation and Usage

## Installation

powerapi has been incorporated into openEuler22.03 LTS SP4. You can run the `yum` command to install powerapi:

```sh
yum install powerapi
```

## Usage

After powerapi is installed, pwrapis automatically runs to provide power consumption management services for eagle and third-party systems.
Currently, powerapi cannot be used through the CLI. To use a powerapi function, you need to install powerapi-devel and call the functions after the header file is included in the source code. For details, see [Development Using powerapi](development_using_powerapi.md).

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_guide_1.md
---
# Installation Guide

This section describes how to enable the Raspberry Pi function after [Writing Raspberry Pi Images into the SD card](./installation_modes_1.md).

## Starting the System

After an image is written into the SD card, insert the SD card into the Raspberry Pi and power on the SD card.

For details about the Raspberry Pi hardware, visit the [Raspberry Pi official website](https://www.raspberrypi.org/).

## Logging in to the System

You can log in to the Raspberry Pi in either of the following ways:

1. Local login

   Connect the Raspberry Pi to the monitor (the Raspberry Pi video output interface is Micro HDMI), keyboard, and mouse, and start the Raspberry Pi. The Raspberry Pi startup log is displayed on the monitor. After Raspberry Pi is started, enter the user name **root** and password **openeuler** to log in.

2. SSH remote login

   By default, the Raspberry Pi uses the DHCP mode to automatically obtain the IP address. If the Raspberry Pi is connected to a known router, you can log in to the router to check the IP address. The new IP address is the Raspberry Pi IP address.

   For example, the IP address of the Raspberry Pi is **192.168.31.109**. You can run the `ssh root@192.168.31.109` command and enter the password `openeuler` to remotely log in to the Raspberry Pi.

## Configuring the System

### Expanding the Root Directory Partition

The space of the default root directory partition is small. Therefore, you need to expand the partition capacity before using it.

To expand the root directory partition capacity, perform the following procedure:

1. Run the `fdisk -l` command as the root user to check the drive partition information. The command output is as follows:

   ```sh
   # fdisk -l
   Disk /dev/mmcblk0: 14.86 GiB, 15931539456 bytes, 31116288 sectors
   Units: sectors of 1 * 512 = 512 bytes
   Sector size (logical/physical): 512 bytes / 512 bytes
   I/O size (minimum/optimal): 512 bytes / 512 bytes
   Disklabel type: dos
   Disk identifier: 0xf2dc3842

   Device         Boot   Start     End Sectors  Size Id Type
   /dev/mmcblk0p1 *       8192  593919  585728  286M  c W95 FAT32 (LBA)
   /dev/mmcblk0p2       593920 1593343  999424  488M 82 Linux swap / Solaris
   /dev/mmcblk0p3      1593344 5044223 3450880  1.7G 83 Linux
   ```

   The drive letter of the SD card is **/dev/mmcblk0**, which contains three partitions:

   * **/dev/mmcblk0p1**: boot partition
   * **/dev/mmcblk0p2**: swap partition
   * **/dev/mmcblk0p3**: root directory partition

   Here, we need to expand the capacity of `/dev/mmcblk0p3`.

2. Run the `fdisk /dev/mmcblk0` command as the root user and the interactive command line interface (CLI) is displayed. To expand the partition capacity, perform the following procedure as shown in [Figure 2](#zh-cn_topic_0151920806_f6ff7658b349942ea87f4521c0256c315).

   1. Enter `p` to check the partition information.

      Record the start sector number of `/dev/mmcblk0p3`. That is, the value in the `Start` column of the `/dev/mmcblk0p3` information. In the example, the start sector number is `1593344`.

   2. Enter `d` to delete the partition.

   3. Enter `3` or press `Enter` to delete the partition whose number is `3`. That is, the `/dev/mmcblk0p3`.

   4. Enter `n` to create a partition.

   5. Enter `p` or press `Enter` to create a partition of the `Primary` type.

   6. Enter `3` or press `Enter` to create a partition whose number is `3`. That is, the `/dev/mmcblk0p3`.

   7. Enter the start sector number of the new partition. That is, the start sector number recorded in Step `1`. In the example, the start sector number is `1593344`.

      > \[!TIP] **NOTE:**\
      > Do not press **Enter** or use the default parameters.

   8. Press `Enter` to use the last sector number by default as the end sector number of the new partition.

   9. Enter `N` without changing the sector ID.

   10. Enter `w` to save the partition settings and exit the interactive CLI.

   **Figure 2** Expand the partition capacity\
   ![](./figures/Partition_expansion.png)

3. Run the `fdisk -l` command as the root user to check the drive partition information and ensure that the drive partition is correct. The command output is as follows:

   ```sh
   # fdisk -l
   Disk /dev/mmcblk0: 14.86 GiB, 15931539456 bytes, 31116288 sectors
   Units: sectors of 1 * 512 = 512 bytes
   Sector size (logical/physical): 512 bytes / 512 bytes
   I/O size (minimum/optimal): 512 bytes / 512 bytes
   Disklabel type: dos
   Disk identifier: 0xf2dc3842

   Device         Boot   Start      End  Sectors  Size Id Type
   /dev/mmcblk0p1 *       8192   593919   585728  286M  c W95 FAT32 (LBA)
   /dev/mmcblk0p2       593920  1593343   999424  488M 82 Linux swap / Solaris
   /dev/mmcblk0p3      1593344 31116287 29522944 14.1G 83 Linux
   ```

4. Run the `resize2fs /dev/mmcblk0p3` command as the root user to increase the size of the unloaded file system.

5. Run the `df -lh` command to check the drive space information and ensure that the root directory partition has been expanded.

   > \[!TIP] **NOTE:**\
   > If the root directory partition is not expanded, run the `reboot` command to restart the Raspberry Pi and then run the `resize2fs /dev/mmcblk0p3` command as the root user.

### Connecting to the Wi-Fi Network

To connect to the Wi-Fi network, perform the following procedure:

1. Check the IP address and network adapter information.

   ```sh
   ip a
   ```

   Obtain information about the wireless network adapter **wlan0**:

   ```sh
   1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
       inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
       inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
   2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
       link/ether dc:a6:32:50:de:57 brd ff:ff:ff:ff:ff:ff
       inet 192.168.31.109/24 brd 192.168.31.255 scope global dynamic noprefixroute eth0
       valid_lft 41570sec preferred_lft 41570sec
       inet6 fe80::cd39:a969:e647:3043/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
   3: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
       link/ether e2:e6:99:89:47:0c brd ff:ff:ff:ff:ff:ff
   ```

2. Scan information about available Wi-Fi networks.

   ```sh
   nmcli dev wifi
   ```

3. Connect to the Wi-Fi network.

   Run the `nmcli dev wifi connect SSID password PWD` command as the root user to connect to the Wi-Fi network.

   In the command, `SSID` indicates the SSID of the available Wi-Fi network scanned in the preceding step, and `PWD` indicates the password of the Wi-Fi network. For example, if the `SSID` is `openEuler-wifi`and the password is `12345678`, the command for connecting to the Wi-Fi network is `nmcli dev wifi connect openEuler-wifi password 12345678`. The connection is successful.

   ```sh
   Device 'wlan0' successfully activated with '26becaab-4adc-4c8e-9bf0-1d63cf5fa3f1'.
   ```

4. Check the IP address and wireless network adapter information.

   ```sh
   ip a
   ```

   ```sh
   1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
       inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
       inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
   2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
       link/ether dc:a6:32:50:de:57 brd ff:ff:ff:ff:ff:ff
       inet 192.168.31.109/24 brd 192.168.31.255 scope global dynamic noprefixroute eth0
       valid_lft 41386sec preferred_lft 41386sec
       inet6 fe80::cd39:a969:e647:3043/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
   3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
       link/ether dc:a6:32:50:de:58 brd ff:ff:ff:ff:ff:ff
       inet 192.168.31.110/24 brd 192.168.31.255 scope global dynamic noprefixroute wlan0
       valid_lft 43094sec preferred_lft 43094sec
       inet6 fe80::394:d086:27fa:deba/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/installation_guideline.md
---
# Installation Guideline

1. Install the container engine iSulad.

   ```sh
   # yum install iSulad
   ```

2. Install dependent packages of system containers.

   ```sh
   # yum install syscontainer-tools authz lxcfs-tools lxcfs
   ```

3. Run the following command to check whether iSulad is started:

   ```sh
   # systemctl status isulad
   ```

4. Enable the lxcfs and authz services.

   ```sh
   # systemctl start lxcfs
   # systemctl start authz
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_guide.md
---
# Installation Guideline

This section describes how to install openEuler using a CD/DVD-ROM. The installation process is the same for other installation modes except the boot option.

## Starting the Installation

### Booting from the CD/DVD-ROM Drive

Mount the ISO image of openEuler to the CD/DVD-ROM drive of the server and restart the server. The procedure is as follows:

> \[!NOTE] **Note**
> Before the installation, ensure that the server boots from the CD/DVD-ROM drive preferentially. The following steps describe how to use connect a virtual CD/DVD-ROM drive to the baseboard management controller (BMC) and install openEuler. The procedure for installing openEuler from a physical drive is the same as that of a virtual drive.

1. On the toolbar, click the icon shown in the following figure.

   **Figure  1**  Drive icon\
   ![](./figures/drive-icon.png)

   An image dialog box is displayed, as shown in the following figure.

   **Figure  2**  Image dialog box\
   ![](./figures/image-dialog-box.png)

2. Select **Image File** and then click **Browse**. The **Open** dialog box is displayed.

3. Select the image file and click **Open**. In the image dialog box, click **Connect**. If **Connect** changes to **Disconnect**, the virtual CD/DVD-ROM drive is connected to the server.

4. On the toolbar, click the restart icon shown in the following figure to restart the device.

   **Figure 3**  Restart icon\
   ![](./figures/restart-icon.png)

### Installation Boot Menu

A boot menu is displayed after the system is booted using the boot medium. In addition to options for starting the installation program, some other options are available on the boot menu. During system installation, the **Test this media & install openEuler 22.03-LTS-SP4** option is selected by default. If you want to select an option other than the default option, use the "↑" and "↓" arrow keys on the keyboard to make the selection and press Enter when the.

> \[!NOTE] **Note**
>
> * If you do not perform any operations within 1 minute, the system automatically selects the default option **Test this media & install openEuler 22.03-LTS-SP4** and enters the installation page.
> * During physical machine installation, if you cannot use the arrow keys to select boot options and the system does not respond after you press **Enter**, click ![](./figures/en-us_image_0229420473.png) on the BMC page and configure **Key & Mouse Reset**.

**Figure 4**  Installation boot menu\
![](./figures/Installation_wizard.png)

Installation boot options are described as follows:

* **Install openEuler 22.03-LTS-SP4**: Install openEuler on your server in GUI mode.

* **Test this media & install openEuler 22.03-LTS-SP4**: Default option. Install openEuler on your server in GUI mode. The integrity of the installation medium is checked before the installation program is started.

* **Troubleshooting**: Troubleshooting mode, which is used when the system cannot be installed properly. In troubleshooting mode, the following options are available:
  * **Install openEuler 22.03-LTS-SP4 in basic graphics mode**: Basic graphics installation mode. In this mode, the video driver is not started before the system starts and runs.
  * **Rescue the openEuler system**: Rescue mode, which is used to restore the system. In rescue mode, the installation process is printed to the VNC or BMC, and the serial port is unavailable.

On the installation boot menu screen, press **e** to go to the parameter editing screen of the selected option, and press **c** to go to the command line interface (CLI).

### Installation in GUI Mode

On the installation wizard page, select **Test this media & install openEuler 22.03-LTS-SP4** to enter the GUI installation mode.

You can perform graphical installation operations using a keyboard.

* Press **Tab** or **Shift+Tab** to move between GUI controls (such as buttons, area boxes, and check boxes).
* Press the up or down arrow key to navigate through a list.
* Press the left or right arrow key to move between the horizontal toolbars and list bars.
* Press the spacebar or **Enter** to select or delete highlighted options, expand or collapse a drop-down list.
* Press **Alt+a shortcut key** to select the control where the shortcut key is located. The shortcut key can be highlighted (underlined) by holding down **Alt**.

#### Selecting the Installation Language

After the installation starts, you are prompted to choose the language that is used during the installation process. English is configured by default, as shown in the following figure. Configure the language as required.

**Figure 5**  Selecting a language\
![](./figures/selectlanguage.png)

After the language is selected, click **Continue**. The installation page is displayed.

If you want to exit the installation, click **Exit**. The message **Are you sure you want to exit the installation program?** is displayed. Click **Yes**  in the dialog box to go back to the installation wizard page.

#### Entering the Installation Page

After the installation program starts, the installation page is displayed, as shown in the following figure. On the page, you can configure the time, language, installation source, network, and storage device.

Some configuration items are marked with alarm symbols. A alarm symbol will disappear after the item is configured. Start the installation only when all the alarm symbols disappear from the page.

If you want to exit the installation, click **Exit**. The message **Are you sure you want to exit the installation program?** is displayed. Click **Yes** in the dialog box to go back to the installation boot menu.

**Figure 6**  Installation summary\
![](./figures/installation_overview.png)

#### Setting the Keyboard Layout

On the **INSTALLATION SUMMARY**  page, click **Keyboard**. You can add or delete multiple keyboard layouts in the system.

* To view the keyboard layout: Select a keyboard layout in the left box and click the keyboard icon under the box.
* To test the keyboard layout: Select the keyboard layout in the left box and click the keyboard icon in the upper right corner to switch to the desired layout, and then type in the right box to check if the keyboard layout works properly.

**Figure 7**  Setting the keyboard layout\
![](./figures/Keyboard_layout.png)

After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page.

#### Setting the System Language

On the **INSTALLATION SUMMARY**  page, click **Language Support** to set the system language, as shown in the following figure. You can select another language as required.

> \[!NOTE] **Note**
> If you select **Chinese**, the system does not support the display of Chinese characters when you log in to the system using VNC, but supports the display of Chinese characters when you log in to the system using a serial port. When you log in to the system using SSH, whether the system supports the display of Chinese characters depends on the SSH client. If you select **English**, the display is not affected.

**Figure 8**  Setting a system language

![](./figures/languagesupport.png)

After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page.

#### Setting Date and Time

On the **INSTALLATION SUMMARY** page, click **Time & Date**. On the **TIME & DATE** page, set the system time zone, date, and time.

When setting the time zone, click a specific city on the map, or select a region from the drop-down list of **Region** and a city from the drop-down list of **City** at the top of the page, as shown in the following figure.

If your city is not displayed on the map or in the drop-down list, select the nearest city in the same time zone.

> \[!NOTE] **Note**
>
> * Before manually setting the time zone, disable the network time synchronization function in the upper right corner.
> * If you want to use the network time, ensure that the remote NTP server is reachable. For details about how to set the network, see [Setting the Network and Host Name](#setting-the-network-and-host-name).

**Figure 9**  Setting date and time

![](./figures/dateandtime.png)

After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page.

#### Setting the Installation Source

On the **INSTALLATION SUMMARY** page, click **Installation Source** to specify the installation source.

* When you use a full CD/DVD image for installation, the installation program automatically detects and displays the installation source information. You can use the default settings, as shown in the following figure.

  **Figure 10**  Installation source\
  ![](./figures/Installation_source.png)

* When the network source is used for installation, you need to set the URL of the network source.

  * HTTP or HTTPS mode

    The following figure shows the installation source in HTTP or HTTPS mode. Enter the actual installation source address, for example, **<https://repo.openeuler.org/openEuler-22.03-LTS/OS/x86_64/>**,  where **openEuler-22.03-LTS** indicates the version number, and **x86\_64** indicates the CPU architecture. Use the actual version number and CPU architecture.

    ![](./figures/sourcehttp.png)

    > \[!NOTE] **Note:**
    >
    > If the HTTPS server uses a private certificate, press **e** on the installation boot menu go to the parameter editing page of the selected option, and add the **inst.noverifyssl** parameter. In UEFI mode, add the parameter to the line starting with **linux**.

  * FTP mode

    The following figure shows the installation source in FTP mode. Enter the FTP address in the text box.

    ![](./figures/sourceftp.png)

    You need to set up an FTP server, mount the ISO image, and copy the mounted files to the shared directory on the FTP server.

  * NFS mode

    The following figure shows the installation source in NFS mode. Enter the NFS address in the text box.

    ![](./figures/sourcenfs.png)

    You need to set up an NFS server, mount the ISO image, and copy the mounted file to the shared directory on the NFS server.

During the installation, if you have any questions about configuring the installation source, see [An Exception Occurs During the Selection of the Installation Source](https://docs.openeuler.openatom.cn/en/docs/common/faq/server/installation_faq1.html#4-an-exception-occurs-during-the-selection-of-the-installation-source).

After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page.

#### Selecting Additional Software

On the **INSTALLATION SUMMARY** page, click **Software Selection** to specify the software package to be installed.

Based on the actual requirements, select **Minimal Install** in the left box and select additional software in the **Additional software for Selected Environment** area in the right box, as shown in the following figure.

**Figure 11**  Selecting additional software\
![](./figures/choosesoftware.png)

> \[!NOTE] **Note**
>
> * In **Minimal Install** mode, not all packages in the installation source are installed. If the required package is not installed, you can mount the installation source to the local host as a repo source, and use DNF to install the package.
> * If you select **Virtualization Host**, the virtualization components QEMU, libvirt, and edk2 are installed by default. You can select whether to install the OVS component in the additional software area.

After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page.

#### Setting the Installation Destination

On the **INSTALLATION SUMMARY** page, click **Installation Destination** to select the OS installation drive and partition.

You can view available local storage devices on the **INSTALLATION DESTINATION** page, as shown in the following figure.

**Figure 12**  Setting the installation destination\
![](./figures/Target_installation_position.png)

##### Storage Configuration

On the **INSTALLATION DESTINATION** page, configure the storage for partitioning. You can either manually configure partitions or select **Automatic** for automatic partitioning.

> \[!NOTE] **Note**
>
> * During partitioning, to ensure system security and performance, you are advised to divide the device into the following partitions:  **/boot**,  **/var**,  **/var/log**,  **/var/log/audit**,  **/home**, and  **/tmp**.
> * If the system is configured with the **swap** partition, the **swap** partition is used when the physical memory of the system is insufficient. Although the **swap** partition can be used to expand the physical memory, if it is used due to insufficient memory, the system response slows and the system performance deteriorates. Therefore, you are not advised to configure it in a system with sufficient physical memory or in a performance sensitive system.
> * If you need to split a logical volume group, select **Custom** to manually partition the logical volume group. On the **MANUAL PARTITIONING** page, click **Modify** in the **Volume Group** area to reconfigure the logical volume group.

**Automatic**

Select **Automatic** if openEuler is installed in a new storage device or the data in the storage device is not required. After the setting is complete, click  **Done**  in the upper left corner to go back to the **INSTALLATION SUMMARY** page.

**Custom**

If you need to manually partition the disk, click **Custom** and click **Done**  in the upper left corner. The following page is displayed.

On the **MANUAL PARTITIONING** page, you can partition the disk in either of the following ways. After the partitioning is completed, the window shown in the following figure is displayed.

* Automatic creation: Click **Click here to create them automatically**. The system automatically assigns four mount points ( **/boot**, **/**, **/home**, and **swap**) according to the available storage space.

* Manual creation: Click ![](./figures/en-us_image_0229291243.png) to add a mount point. It is recommended that the expected capacity of each mount point not exceed the available space.

  > \[!NOTE] **Note**
  > If the expected capacity of the mount point exceeds the available space, the system allocates all available space to the mount point.

**Figure 13**  MANUAL PARTITIONING page\
![](./figures/Manual_partitioning.png)

> \[!NOTE] **Note**
> The **/boot/efi** partition is required for UEFI mode only.

After the setting is complete, click **Done** in the upper left corner to go back to the **SUMMARY OF CHANGES** page.

Click **Accept Changes** to go back to the **INSTALLATION SUMMARY** page.

#### Setting the Network and Host Name

On the **INSTALLATION SUMMARY** page, select **Network & Host Name** to configure the system network functions.

The installation program automatically detects accessible local interfaces. The detected interfaces are listed in the left box, and the interface details are displayed in the right area, as shown in  [Figure 14](#zh-cn_topic_0186390264_zh-cn_topic_0122145831_fig123700157297). You can enable or disable a network interface by clicking the switch in the upper right corner of the page. The switch is turned off by default. If the installation source is set to network, turn on the switch. You can also click **Configure** to configure the selected interface. Select **Connect automatically with priority** to enable the NIC automatic startup upon system startup, as shown in [Figure 15](#zh-cn_topic_0186390264_zh-cn_topic_0122145831_fig6).

In the lower left box, enter the host name. The host name can be the fully quantified domain name (FQDN) in the format of *hostname.domain\_name* or the brief host name in the format of *hostname*.

**Figure 14**  Setting the network and host name\
![](./figures/NetworkandHostName.png)

**Figure 15**  Configuring the network\
![](./figures/confignetwork1.png)

After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page.

#### Setting the Root Password

Select **Root Password** on the **INSTALLATION SUMMARY** page. The **ROOT PASSWORD** page is displayed, as shown in the [Figure 16](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1323165793018).  Enter a password that meets the [Password Complexity](#password-complexity) requirements and confirm the password.

> \[!NOTE] **Note**
>
> * The **root** account is used to perform key system management tasks. You are not advised to use the **root** account for daily work or system access.
> * If you select **Lock root account** on the **Root Password** page, the **root** account will be disabled.

**Figure 16**  root password\
![](./figures/root_password.png)

##### Password Complexity

The password of the **root** user or the password of the new user must meet the password complexity requirements. Otherwise, the password configuration or user creation will fail. The password complexity requirements are as follows:

1. A password must contain at least eight characters.

2. A password must contain at least three of the following types: uppercase letters, lowercase letters, digits, and special characters.

3. A password must be different from the user name.

4. A password cannot contain words in the dictionary.

   > \[!NOTE] **Note**
   > In the installed openEuler environment, you can run the `cracklib-unpacker /usr/share/cracklib/pw_dict > dictionary.txt` command to export the dictionary library file **dictionary.txt**, and then check whether the password is in the dictionary.

After the settings are completed, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page.

#### Creating a User

Click **User Creation**. The **CREATE USER** is displayed, as shown in [Figure 17](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319). Enter a username and set a password. By clicking **Advanced**, you can also configure the home directory and  user group, as shown in [Figure 18](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig128716531312).

**Figure 17**  Creating a user\
![](./figures/createuser.png)

**Figure 18**  Advanced user configuration\
![](./figures/Advanced_User_Configuration.png)

##### Advanced User Configuration Requirements

When a user is created, a user group with the same name is created by default. In **ADVANCED USER CONFIGURATION**, you set the user ID and user group ID as required, but leave **Group Membership** blank unless you want to create a user group with a different name from the user. If the user group with the same name as the user is entered, the user cannot be created.

After configuration, click **Done** in the upper left corner to go back back to the **INSTALLATION SUMMARY** page.

#### Starting Installation

On the installation page, after all the mandatory items are configured, the alarm symbols will disappear. Then, you can click **Begin Installation** to install openEuler.

#### Installation Procedure

After the installation starts, the overall installation progress and the progress of writing the software package to the system are displayed. See [Figure 19](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1590863119306)

> ![](./figures/en-us_image_0213178479.png)\
> During the OS installation, if you click Exit, reset, or power off the server, or the virtual CD-ROM drive is disconnected due network faults, the installation is interrupted and the OS is unavailable. In this case, you need to reinstall the OS.

**Figure 19**  Installation progress\
![](./figures/installation_procedure.png)

#### Completing the Installation

After openEuler is installed, Click  **Reboot**  to reboot the system.

> \[!NOTE] **Note**
>
> * If a physical CD/DVD-ROM is used for installation and it is not automatically ejected during the restart, manually remove it. Then, the openEuler CLI login screen is displayed.
> * If a virtual CD/DVD-ROM is used for installation, change the server boot option to **Hard Disk** and restart the server. Then, the openEuler CLI login screen is displayed.

### Installation in CLI Mode

You can perform installation operations in CLI mode using a keyboard. See **Figure 1**.

**Figure  1** CLI mode main menu
![](figures/zh-cn_image_text_menu.png)

> \[!NOTE] **Note:**
> **\[x]** indicates that the item has been configured or is configured by default. You can modify the configuration as required. **\[!]** indicates that the item is not configured. You must configure the item before starting installation.

#### Setting the System Language

In the main menu, enter **1** to enter the language settings, and enter **1** or **2** to set the system language as required. See **Figure 2**.

**Figure  2** Selecting a Language
![](figures/zh-cn_image_text_language.png)

After completing the settings, enter **c** to go back to the main menu.

#### Setting Date and Time

In the main menu, enter **2** to enter time zone settings. See **Figure 3**.

**Figure  3** Time settings\
![](figures/zh-cn_image_text_timedate.png)

* Time zone settings\
  In **Time settings**, enter **1** to enter time zone settings. You can change regions and cities as required. See **Figure 4**.

**Figure  4** Timezone settings\
![](figures/zh-cn_image_text_timeZone.png)

Cities are displayed in pages. Press **Enter** multiple times to view all cities. See **Figure 5**.

**Figure 5** Selecting a city\
![](figures/zh-cn_image_text_timeZoneCity.png)

After completing the settings, enter **c** to go back to the main menu.

* NTP configuration
  In **Time settings**, enter **2** to enter NTP configuration. See **Figure 6**.

**Figure  6** NTP configuration\
![](figures/zh-cn_image_text_timeNTP.png)

After completing the settings, enter **c** to go back to the main menu.

#### Setting the Installation Source

In the main menu, enter **3** to enter installation source settings. See **Figure 7**. You can select a local installation source or a network location.

**Figure  7** Installation source\
![](figures/zh-cn_image_text_installSource.png)

After completing the settings, enter **c** to go back to the main menu.

> \[!NOTE] **Note:**
>
> * CD/DVD: Install from a mounted CD/DVD drive.
> * local ISO file: Install from a local ISO file.
> * Network: Install from a network location using HTTP, HTTPS, FTP, or NFS protocol.

#### Selecting Software

In the main menu, enter **4** to enter the software selection menu. See **Figure 8**.
The default **Minimal Install** indicates the minimal environment that provides basic openEuler functions. After selecting the environment, enter **c** to select additional software.

**Figure  8** Software selection\
![](figures/zh-cn_image_text_installSofware.png)

Available software varies from the selected environment:

1. Standard: standard openEuler software packages
2. Container Management: software packages for managing Linux containers
3. Development Tools: basic development environment
4. Headless Management: tools for managing non-graphical terminal systems
5. Legacy UNIX Compatibility: compatibility tools for migrating from legacy UNIX environments
6. Network Servers: network server software, such as DHCP, Kerberos, and NIS
7. Scientific Support: tools for scientific computing and parallel computing
8. Security Tools: tools for integrity and trustworthiness verification
9. System Tools: various system tools, such as SMB client and network traffic monitoring tools
10. Smart Card Support: support for smart card verification

After completing the settings, enter **c** to go back to the main menu.

#### Setting the Installation Destination

In the main menu, enter **5** to select the OS installation destination. See **Figure 9**. The installer automatically detects available locations. Generally, you can use the default configuration.

**Figure  9** Installation destination\
![](figures/zh-cn_image_text_installdest.png)

* Partitioning options
  * Replace Existing Linux system (s): Use only the space occupied by the existing OS. Data of the existing OS will be overwritten.
  * Use All Space: Delete data of the OS partition and use all space on the drive.
  * Use Free Space: Install openEuler to the free space on the drive without deleting data of the existing OS.
  * Manually assign mount points: This option is experimental and may not take effect.
* Partitioning scheme options
  * Standard Partition: A standard partition can be a file system, swap partition, or a container for software RAID or LVM physical volume.
  * LVM: Logical volume management (LVM) displays a simple bare-metal view of basic physical storage space, such as a hard disk or an LUN. Partitions that are regarded as physical volumes in physical storage can be grouped into volume groups. Each volume group can be divided into multiple logical volumes, and each logical volume simulates a standard disk partition. Therefore, an LVM logical volume can be used as a partition that contains multiple physical disks.
  * LVM Thin Provisioning: Thin provisioning allows you to manage storage pools with available space, also called thin pools, which can be allocated to any number of devices as required. The thin pools can be dynamically expanded as required to allocate storage space.
    After completing the settings, enter **c** to go back to the main menu.

> \[!NOTE] **Note:**
>
> * For system performance and security purposes, you are advised to configure the following partitions: **/boot**, **/var**, **/var/log** , **/var/log/audit**, **/home**, **/tmp**.
> * If the system is configured with the swap partition, the swap partition is used when the physical memory of the system is insufficient. Although the swap partition can be used to expand the physical memory, if the swap partition is used due to insufficient memory, the system response slows and the system performance deteriorates. Therefore, you are not advised to configure the swap partition in the system with sufficient physical memory or the performance sensitive system. In addition, unmounting of the swap partition requires the available memory (including the reclaimable memory) to be more than the size of used swap space. Otherwise, the swap partition will fail to be unmounted.
> * By default, the OS is booted from the first drive. You are advised to install the OS on the first drive. Otherwise, the OS may fail to be booted.

#### Setting the Network and Host Name

In the main menu, enter **6** to enter network configuration. See **Figure 10**. You can configure the host name and network devices. Alternatively, you can perform network configuration after the OS is installed.

**Figure  10** Network configuration\
![](figures/zh-cn_image_text_net.png)

* Set the host name.\
  Enter **1**, type in a host name, then press **Enter**.\
  Ensure that the host name does not end with a period (.) because it is not supported by the kernel and systemd.
* Configure the NIC.\
  Enter **2** to enter network device configuration. See **Figure 11**.

**Figure  11** Device configuration\
![](figures/zh-cn_image_text_networkConfig.png)

1. IPv4 address or "dhcp" for DHCP: The default value is dhcp, that is, the IP address is assigned by a DHCP server.
2. IPv4 netmask
3. IPv4 gateway
4. IPv6 address\[/prefix] or "auto" for automatic, "dhcp" for DHCP, "ignore" to turn off: The default value is **auto**.
5. IPv6 default gateway
6. Nameservers (comma separated): Domain name servers
7. Connect automatically after reboot
8. Apply configuration in installer

After completing the settings, enter **c** to go back to the main menu.

#### Setting the Root Password

In the main menu, enter **7** to enter the root password setting. See **Figure 12**.

**Figure  12** Root password setting\
![](figures/zh-cn_image_text_rootset.png)

Enter **1** to select SM3 encryption, then press **c** to start entering the password. Alternatively, you can skip SM3 encryption. See **Figure 13**.

**Figure  13** Entering root password\
![](figures/zh-cn_image_text_rootpassword.png)

After completing the settings, enter **c** to go back to the main menu.

> \[!NOTE] **Note:**
>
> * The root password is mandatory for OS installation.
>
> * The root user is used to perform key OS management tasks. You are advised not to use the root user for routine operations and OS access.
>
> * The default encryption algorithm is yescrypt. If yescrypt is not supported, SHA512 will be used.
>
> * When setting the password of the **root** user or that of a new user, you are advised to set the password according to the password complexity requirements. When you set a weak password (the password does not meet the complexity requirements), the system generates an alarm and asks you whether to use the weak password. If you enter **yes**, the weak password is forcibly set. However, the weak password poses security risks. Therefore, exercise caution when selecting a weak password. A strong password is expected to:
>   1. Contain at least 8 characters
>   2. Contain at least 3 of the following types: uppercase letters, lowercase letters, digits, and special characters.
>   3. Be different from the user name.
>   4. Not contain words in the dictionary.

#### Creating a User

In the main menu, enter **8** to enter user creation. Configure user information, such as the full name, user name, user password, whether the user is an administrator, and group of the user. See **Figure 14**.

**Figure  14** User creation\
![](./figures/zh-cn_image_text_createUsr.png)

> \[!NOTE] **Note:**
>
> * The default encryption algorithm is yescrypt. If yescrypt is not supported, SHA512 will be used.
> * Password of the new user must meet the password complexity requirements. Otherwise, user creation will fail. The password is expected to:
>   1. Contain at least 8 characters
>   2. Contain at least 3 of the following types: uppercase letters, lowercase letters, digits, and special characters.
>   3. Be different from the user name.
>   4. Not contain words in the dictionary.

After completing the settings, enter **c** to go back to the main menu.

#### Starting the Installation

After all mandatory configurations are complete, the warnings (**\[!]**) in the main menu disappear. See **Figure 15**. Enter **b** to start OS installation.

**Figure  15** Configuration complete\
![](./figures/zh-cn_image_text_startInstall.png)

Wait for the installation to complete. See **Figure 16**.

**Figure  16** Installation complete\
![](./figures/zh-cn_image_text_Installed.png)

After the installation is complete, press **Enter** to reboot the system.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_modes.md
---
# Installation Modes

> \[!TIP] **NOTICE**
>
> * Only TaiShan 200 servers and FusionServer Pro rack server are supported. For details about the supported server models, see [Hardware Compatibility](./installation_preparations.md#hardware-compatibility). Only a virtualization platform created by the virtualization components (openEuler as the host OS and QEMU and KVM provided in the release package) of openEuler and the x86 virtualization platform of Huawei public cloud are supported.
> * Currently, only installation modes such as DVD-ROM, USB flash drive, network, QCOW2 image, and private image are supported. In addition, only the x86 virtualization platform of Huawei public cloud supports the private image installation mode.

## Installation Through a DVD-ROM

This section describes how to create or use a DVD-ROM to install the openEuler.

### Preparing the Installation Source

If you have obtained a DVD-ROM, directly install the OS using the DVD-ROM. If you have obtained an ISO file, record the ISO file to a DVD and install the OS using the obtained DVD.

### Starting the Installation

Perform the following operations to start the installation:

> \[!NOTE] **NOTE**
> Set the system to preferentially boot from the DVD-ROM drive. Take BIOS as an example. You need to move the  **CD/DVD-ROM Drive**  option under  **Boot Type Order**  to the top.

1. (Optional) Disconnect all drives that are not required, such as USB drives.
2. Start your computer system.
3. Insert the installation DVD-ROM into the computer.
4. Restart the computer system.

After a short delay, a graphical wizard page is displayed, which contains different boot options. If you do not perform any operation within one minute, the installation starts automatically with the default option.

## Installation Through a USB Flash Drive

This section describes how to create or use a USB flash drive to install the openEuler.

### Preparing the Installation Source

Pay attention to the capacity of the USB flash drive. The USB flash drive must have sufficient space to store the entire image. It is recommended that the USB flash drive have more than 16 GB space.

1. Connect the USB flash drive to the system and run the  **dmesg**  command to view related log. At the end of the log, you can view the information generated by the USB flash drive that is just connected. The information is similar to the following:

   ```console
   [ 170.171135] sd 5:0:0:0: [sdb] Attached SCSI removable disk
   ```

   > \[!NOTE] **NOTE**
   > Take the  **sdb**  USB flash drive as an example.

2. Switch to user  **root**. When running the  **su**  command, you need to enter the password.

   ```shell
   su - root
   ```

3. Ensure that the USB flash drive is not mounted.

   ```shell
   findmnt /dev/sdb
   ```

   * If no command output is displayed, the file system is not mounted. Go to the next step.

   * If the following information is displayed, the USB flash drive is automatically mounted.

     ```shell
     $ findmnt /dev/sdb
     TARGET   SOURCE   FSTYPE  OPTIONS
     /mnt/iso /dev/sdb iso9660 ro,relatime
     ```

     In this case, you need to run the  **umount**  command to uninstall the device.

     ```shell
     umount /mnt/iso
     ```

4. Run the  **dd**  command to write the ISO image to the USB flash drive.

> \[!NOTE] **Note:**
> According to the ISOLINUX documentation, the ISO 9660 file system created by the `mkisofs` command will boot via BIOS firmware, but only from optical media like CD, DVD, or BD. In this case, run `isohybrid -u your.iso` to process the ISO file before running `dd` to write the ISO file into the USB flash drive. (This problem affects only the x86 architecture.)

```shell
dd if=/path/to/image.iso of=/dev/device bs=blocksize
```

Replace  **/path/to/image.iso**  with the complete path of the downloaded ISO image file, replace  **device**  with the device name provided by the  **dmesg**  command, and set a proper block size (for example, 512 KB) to replace  **blocksize**  to accelerate the write progress.

For example, if the ISO image file name is  **/home/testuser/Downloads/openEuler-21.09-aarch64-dvd.iso**  and the detected device name is  **sdb**, run the following command:

```shell
dd if=/home/testuser/Downloads/openEuler-21.09-aarch64-dvd.iso of=/dev/sdb bs=512k
```

1. After the image is written, safely eject and remove the USB flash drive.

   No progress is displayed during the image write process. When the number sign (#) appears again, run the following command to write the data to the drive. Then exit the **root** account and remove the USB flash drive. In this case, you can use the USB drive as the installation source of the system.

   ```bash
   sync
   ```

### Starting the Installation

Perform the following operations to start the installation:

> \[!NOTE] **NOTE**
> Set the system to preferentially boot from the USB flash drive. Take the BIOS as an example. You need to move the  **USB**  option under  **Boot Type Order**  to the top.

1. Disconnect all drives that are not required.
2. Open your computer system.
3. Insert the USB flash drive into the computer.
4. Restart the computer system.

After a short delay, a graphical wizard page is displayed, which contains different boot options. If you do not perform any operation within one minute, the installation program automatically starts the installation.

## Installation Through the Network Using PXE

To boot with PXE, you need to properly configure the server and your computer's network interface shall support PXE.

If the target hardware is installed with a PXE-enabled NIC, configure it to boot the computer from network system files rather than local media (such as DVD-ROMs) and execute the Anaconda installation program.

For installation through the network using PXE, the client uses a PXE-enabled NIC to send a broadcast request for DHCP information and IP address to the network. The DHCP server provides the client with an IP address and other network information, such as the IP address or host name of the DNS and FTP server (which provides the files required for starting the installation program), and the location of the files on the server.

> \[!NOTE] **NOTE**
> The TFTP, DHCP, and HTTP server configurations are not described here. For details, see  [Full-automatic Installation Guide](./using_kickstart_for_automatic_installation.md#full-automatic-installation-guide).

## Installation Through a QCOW2 Image

This section describes how to create or use a QCOW2 image to install the openEuler.

### Creating a QCOW2 Image

1. Install the  **qemu-img**  software package.

   ```shell
   dnf install -y qemu-img
   ```

2. Run the  **create**  command of the qemu-img tool to create an image file. The command format is as follows:

   ```shell
   qemu-img create -f <imgFormat> -o <fileOption> <fileName> <diskSize>
   ```

   The parameters are described as follows:

   * *imgFormat*: Image format. The value can be  **raw**  or  **qcow2**.
   * *fileOption*: File option, which is used to set features of an image file, such as specifying a backend image file, compressing, and encrypting.
   * *fileName*: File name.
   * *diskSize*: Disk size, which specifies the size of a block disk. The unit can be K, M, G, or T, indicating KiB, MiB, GiB, or TiB.

   For example, to create an image file  **openEuler-image.qcow2**  whose disk size is 32 GB and format is qcow2, the command and output are as follows:

   ```shell
   $ qemu-img create -f qcow2 openEuler-image.qcow2 32G
   Formatting 'openEuler-image.qcow2', fmt=qcow2 size=34359738368 cluster_size=65536 lazy_refcounts=off refcount_bits=16
   ```

### Starting the Installation

Perform the following operations to start the installation:

1. Prepare a QCOW2 image file.
2. Prepare the VM network.
3. Prepare the UEFI boot tool set EDK II.
4. Prepare the VM XML configuration file.
5. Create a VM.
6. Start the VM.

For details, see the  [*Virtualization User Guide*](../../../virtualization/virtualization_platform/virtualization/introduction_to_virtualization.md).

## Installation Through a Private Image

This section describes how to create or use a private image to install the openEuler.

### Creating a Private Image

For instructions about how to create a private image, see [*Image Management Service User Guide*](https://support.huaweicloud.com/intl/en-us/usermanual-ims/en-us_topic_0013901628.html).

### Starting the Installation

For details about how to start the x86 virtualization platform of Huawei public cloud, see  [Elastic Cloud Server User Guide](https://support.huaweicloud.com/intl/en-us/wtsnew-ims/index.html).

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_modes_1.md
---
# Installation Modes

> \[!TIP] **NOTE**
>
> * The hardware supports only Raspberry Pi 3B/3B+/4B/400.
> * The installation is performed by writing images to the SD card. This section describes how to write images on Windows, Linux, and Mac.
> * The image used in this section is the Raspberry Pi image of openEuler. For details about how to obtain the image, see [Installation Preparations](./installation_preparations_1.md).

## Writing Images on Windows

This section uses Windows 10 as an example to describe how to write images to the SD card in the Windows environment.

### Formatting the SD Card

To format the SD card, perform the following procedures:

1. Download and install a SD card formatting tool. The following operations use SD Card Formatter as an example.

2. Start SD Card Formatter. In **Select card**, select the drive letter of the SD card to be formatted.

   If no image has been installed in the SD card, only one drive letter exists. In **Select card**, select the drive letter of the SD card to be formatted.

   If an image has been installed in the SD card, one or more drive letters exist. For example, the SD card corresponds to three drive letters: E, G, and H. In **Select card**, you can select the drive letter E of the boot partition.

3. In **Formatting options**, select a formatting mode. The default mode is **Quick format**.

4. Click **Format** to start formatting. A progress bar is displayed to show the formatting progress.

5. After the formatting is completed, the message "Formatting was successfully completed" is displayed. Click **OK**.

### Writing Images to the SD Card

> \[!TIP] **NOTE**
> If the compressed image file **openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz** is obtained, decompress the file to obtain the **openEuler-22.03-LTS-SP4-raspi-aarch64.img** image file.

To write the **openEuler-22.03-LTS-SP4-raspi-aarch64.img** image file to the SD card, perform the following procedures:

1. Download and install a tool for writing images. The following operations use Win32 Disk Imager as an example.
2. Start Win32 Disk Imager and right-click **Run as administrator**.
3. Select the path of the image file in IMG format from the **Image File** drop-down list box.
4. In **Device**, select the drive letter of the SD card to which the image is written.
5. Click **Write**. A progress bar is displayed to show the progress of writing the image to the SD card.
6. After the write operation is completed, a dialog box is displayed, indicating that the write operation is successfully completed. Click **OK**.

## Writing Images on Linux

This section describes how to write images to the SD card in the Linux environment.

### Checking Drive Partition Information

Run the `fdisk -l` command as the  **root**  user to obtain the drive information of the SD card. For example, the drive partition corresponding to the SD card can be  **/dev/sdb**.

### Unmounting the SD Card

1. Run the `df -lh` command to check the mounted volumes.

2. If the partitions corresponding to the SD card are not mounted, skip this step. If the partitions (for example, /dev/sdb1 and /dev/sdb3) are mounted, run the following commands as the  **root**  user to unmount them:

   `umount /dev/sdb1`

   `umount /dev/sdb3`

### Writing Images to the SD Card

1. If the image obtained is compressed, run the `xz -d openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz` command to decompress the compressed file to obtain the **openEuler-22.03-LTS-SP4-raspi-aarch64.img** image file. Otherwise, skip this step.

2. Run the following command as the  **root**  user to write the `openEuler-22.03-LTS-SP4-raspi-aarch64.img` image to the SD card:

   `dd bs=4M if=openEuler-22.03-LTS-SP4-raspi-aarch64.img of=/dev/sdb`

   > \[!NOTE] **NOTE** Generally, the block size is set to 4 MB. If the write operation fails or the written image cannot be used, you can set the block size to 1 MB and try again. However, the write operation is time-consuming when the block size is set to 1 MB.

## Writing Images on Mac

This section describes how to flash images to the SD card in the Mac environment.

### Checking Drive Partition Information

Run the `diskutil list` command as the  **root**  user to obtain the drive information of the SD card. For example, the drive partition corresponding to the SD card can be  **/dev/disk3**.

### Unmounting the SD Card

1. Run the `df -lh` command to check the mounted volumes.

2. If the partitions corresponding to the SD card are not mounted, skip this step. If the partitions (for example, dev/disk3s1 and /dev/disk3s3) are mounted, run the following commands as the  **root**  user to unmount them:

   `diskutil umount /dev/disk3s1`

   `diskutil umount /dev/disk3s3`

### Writing Images to the SD Card

1. If the image obtained is compressed, run the `xz -d openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz` command to decompress the compressed file to obtain the **openEuler-22.03-LTS-SP4-raspi-aarch64.img** image file. Otherwise, skip this step.

2. Run the following command as the  **root**  user to write the image `openEuler-22.03-LTS-SP4-raspi-aarch64.img` to the SD card:

   `dd bs=4m if=openEuler-22.03-LTS-SP4-raspi-aarch64.img of=/dev/sdb`

   > \[!NOTE] **NOTE**
   >
   > Generally, the block size is set to 4 MB. If the write operation fails or the written image cannot be used, you can set the block size to 1 MB and try again. However, the write operation is time-consuming when the block size is set to 1 MB.

---

---
url: /en/docs/22.03_LTS_SP4/server/installation_upgrade/installation/install_pi.md
---
# Installation on Raspberry Pi

This section describes how to install openEuler on Raspberry Pi. Users must have basic knowledge of Linux OS management.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_preparations.md
---
# Installation Preparations

This section describes the compatibility of the hardware and software and the related configurations and preparations required for the installation.

## Obtaining the Installation Source

Obtain the openEuler release package and verification file before the installation.

Please follow the steps below to obtain the openEuler release package and verification file:

1. Visit the [openEuler](https://www.openeuler.org/en/) website.
2. Click  **Downloads**.
3. Click **Community Editions**. The version list is displayed.
4. Click **Download** on the right of **openEuler 22.03 LTS SP4**.
5. Download the required openEuler release package and the corresponding verification file based on the architecture and scenario.
   1. If the architecture is AArch64:
      1. Click **AArch64**.
      2. For local installation, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP4-(everything-)aarch64-dvd.iso** to the local host.
      3. For network installation, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP4-netinst-aarch64-dvd.iso** to the local host.
   2. If the architecture is x86\_64:
      1. Click **x86\_64**.
      2. For local installation, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP4-(everything-)x86\_64-dvd.iso** to the local host.
      3. For network installation, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP4-netinst-x86\_64-dvd.iso** to the local host.

> \[!NOTE] **Note**
> When the network is available, install openEuler through the network because the ISO release package is small.
> The release package of AArch64 architecture supports UEFI mode, while the release package of x86\_64 architecture supports UEFI mode and Legacy mode.

## Release Package Integrity Check

> \[!NOTE] **NOTE**
> This section describes how to verify the integrity of the release package for the AArch64 architecture. The procedure for verifying the integrity of the release package for the x86\_64 architecture is the same.

### Introduction

To check whether the software package is incompletely downloaded due to network or storage device faults during transmission, you need to verify the integrity of the software package after obtaining it. Only the software package that passes the verification can be installed.

Compare the verification value recorded in the verification file with the calculated verification value of the ISO file to check whether the software package is complete. If the values are consistent, the ISO file is not damaged. Otherwise, the file is damaged and you need to obtain it again.

### Prerequisites

Before verifying the integrity of the release package, you need to prepare the following files:

ISO file:  **openEuler-22.03-LTS-SP4-aarch64-dvd.iso**

Verification file:  Copy and save the **Integrity Check** SHA256 value to a local file.

### Procedures

To verify the file integrity, perform the following operations:

1. Calculate the SHA256 verification value of the file. Run the following command:

   ```sh
   sha256sum openEuler-22.03-LTS-SP4-aarch64-dvd.iso
   ```

   After the command is run, the verification value is displayed.

2. Check whether the calculated value is the same as that of the saved SHA256 value.

   If the values are consistent, the ISO file is not damaged. Otherwise, the file is damaged and you need to obtain it again.

## Installation Requirements for PMs

To install the openEuler OS on a PM, the PM must meet the following hardware compatibility and minimum hardware requirements.

### Hardware Compatibility

You need to take hardware compatibility into account before installing openEuler. The [Compatibility List](https://www.openeuler.org/en/compatibility/) describes supported servers.

### Minimum Hardware Specifications

[Table 1](#tff48b99c9bf24b84bb602c53229e2541)  lists the minimum hardware specifications supported by openEuler.

**Table 1**  Minimum hardware specifications

|  Component   | Minimum Hardware Specifications  |
|  :----  | :----  |
| Architecture  | AArch64 or x86\_64 |
| CPU  | Two single-core CPUs|
| Memory  | ≥ 4 GB (8 GB or higher recommended for better user experience) |
| Hard drive  | ≥ 32 GB (120 GB or higher recommended for better user experience) |

## Installation Requirements for VMs

To install the openEuler OS on a VM, the VM must meet the following hardware compatibility and minimum hardware requirements.

### Virtualization Platform Compatibility

You need to take the compatibility of the virtualization platform into account before installing openEuler. Currently, the following virtualization platforms are supported:

* A virtualization platform created by the virtualization components of openEuler (QEMU and KVM provided in the release package) with openEuler as the host OS
* An x86 virtualization platform of Huawei public cloud

### Minimum Virtualization Platform Specifications

[Table 2](#tff48b99c9bf24b84bb602c53229e2541)  lists the minimum virtualization platform specifications supported by openEuler.

**Table 2**  Virtualization platform specifications

|  Component   | Virtualization Platform Specifications  |
|  :----  | :----  |
| Architecture  | AArch64 or x86\_64 |
| CPU  | Two CPUs|
| Memory  | ≥ 4 GB (8 GB or higher recommended for better user experience) |
| Hard drive  | ≥ 32 GB (120 GB or higher recommended for better user experience) |

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_preparations_1.md
---
# Installation Preparations

This section describes the compatibility of the hardware and software and the related configurations and preparations required for the installation.

## Obtaining the Installation Source

Before installation, obtain the openEuler Raspberry Pi image and its verification file.

1. Visit [openEuler Repo](https://repo.openeuler.org/).
2. Choose **openEuler 22.03 LTS SP4**.
3. Click **raspi\_img**. The download list of Raspberry Pi images is displayed.
4. Click **openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz** to download the openEuler Raspberry Pi image to the local PC.
5. Click **openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz.sha256sum** to download the verification file of the openEuler Raspberry Pi image to the local PC.

## Verifying the Image Integrity

### Overview

During package transmission, to prevent software packages from being incompletely downloaded due to network or storage device problems, you need to verify the integrity of the software packages after obtaining them. Only the software packages that pass the verification can be deployed.

Compare the verification value recorded in the verification file with the verification value that is manually calculated to determine whether the software package is complete. If the two values are the same, the downloaded file is complete. Otherwise, the downloaded file is incomplete and you need to obtain the software package again.

### Prerequisites

Before verifying the integrity of the image file, ensure that the following files are available:

Image file: **openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz**

Verification file: **openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz.sha256sum**

### Procedures

To verify the file integrity, perform the following procedures:

1. Obtain the verification value from the verification file. Run the following command:

   ```shell
   cat openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz.sha256sum
   ```

2. Calculate the SHA256 verification value of the file. Run the following command:

   ```shell
   sha256sum openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz
   ```

   After the command is executed, the verification value is displayed.

3. Check whether the verification values obtained from the step 1 and step 2 are consistent.

   If they are consistent, the downloaded file is not damaged. Otherwise, the downloaded file is incomplete and you need to obtain the file again.

## Installation Requirements

If the openEuler OS is installed in the Raspberry Pi environment, the Raspberry Pi environment must meet the following requirements.

### Hardware Compatibility

Currently, the openEuler Raspberry Pi image supports the 3B, 3B+, 4B, and 400 versions.

### Minimum Hardware Specifications

[Table 1](#tff48b99c9bf24b84bb602c53229e2542) lists the minimum hardware specifications for the openEuler Raspberry Pi image.

**Table 1** Minimum hardware specifications

---

---
url: /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/installing_etcd.md
---
# Installing etcd

## Preparing the Environment

Run the following command to enable the port used by etcd:

```bash
firewall-cmd --zone=public --add-port=2379/tcp
firewall-cmd --zone=public --add-port=2380/tcp
```

## Installing the etcd Binary Package

Currently, the RPM package is used for installation.

```bash
rpm -ivh etcd*.rpm
```

Prepare the directories.

```bash
mkdir -p /etc/etcd /var/lib/etcd
cp ca.pem /etc/etcd/
cp kubernetes-key.pem /etc/etcd/
cp kubernetes.pem /etc/etcd/
# Disabling SELinux
setenforce 0
# Disabling the Default Configuration of the /etc/etcd/etcd.conf File
# Commenting Out the Line, for example, ETCD_LISTEN_CLIENT_URLS="http://localhost:2379".
```

## Compiling the etcd.service File

The following uses the `k8smaster0` machine as an example:

```bash
$ cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "ETCD_UNSUPPORTED_ARCH=arm64 /usr/bin/etcd --name=k8smaster0 --cert-file=/etc/etcd/kubernetes.pem --key-file=/etc/etcd/kubernetes-key.pem --peer-cert-file=/etc/etcd/kubernetes.pem --peer-key-file=/etc/etcd/kubernetes-key.pem --trusted-ca-file=/etc/etcd/ca.pem --peer-trusted-ca-file=/etc/etcd/ca.pem --peer-client-cert-auth --client-cert-auth --initial-advertise-peer-urls https://192.168.122.154:2380 --listen-peer-urls https://192.168.122.154:2380 --listen-client-urls https://192.168.122.154:2379,https://127.0.0.1:2379 --advertise-client-urls https://192.168.122.154:2379 --initial-cluster-token etcd-cluster-0 --initial-cluster k8smaster0=https://192.168.122.154:2380,k8smaster1=https://192.168.122.155:2380,k8smaster2=https://192.168.122.156:2380 --initial-cluster-state new --data-dir /var/lib/etcd"

Restart=always
RestartSec=10s
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
```

**Caution:**

* The boot setting `ETCD_UNSUPPORTED_ARCH=arm64` needs to be added to ARM64;
* In this document, etcd and Kubernetes control are deployed on the same machine. Therefore, the `kubernetes.pem` and `kubernetes-key.pem` certificates are used to start etcd and Kubernetes control.
* A CA certificate is used in the entire deployment process. etcd can generate its own CA certificate and use its own CA certificate to sign other certificates. However, the certificate signed by the CA certificate needs to be used when the APIServer accesses the etcd client.
* `initial-cluster` needs to be added to all configurations for deploying etcd.
* To improve the storage efficiency of etcd, you can use the directory of the SSD as `data-dir`.

Start the etcd service.

```bash
systemctl enable etcd
systemctl start etcd
```

Then, deploy other hosts in sequence.

## Verifying Basic Functions

```bash
$  ETCDCTL_API=3 etcdctl -w table endpoint status --endpoints=https://192.168.122.155:2379,https://192.168.122.156:2379,https://192.168.122.154:2379   --cacert=/etc/etcd/ca.pem   --cert=/etc/etcd/kubernetes.pem   --key=/etc/etcd/kubernetes-key.pem
+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|           ENDPOINT           |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFTAPPLIED INDEX | ERRORS |
+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://192.168.122.155:2379 | b50ec873e253ebaa |  3.4.14 |  262 kB |     false |      false |       819 |         21 |           21 |        |
| https://192.168.122.156:2379 | e2b0d126774c6d02 |  3.4.14 |  262 kB |      true |      false |       819 |         21 |           21 |        |
| https://192.168.122.154:2379 | f93b3808e944c379 |  3.4.14 |  328 kB |     false |      false |       819 |         21 |           21 |        |
+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
```

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/gnome/gnome_installation.md
---
# Installing GNOME on openEuler

GNOME is a desktop environment for Unix-like operating systems. As the officially released desktop of GNU Project, GNOME provides a comprehensive, easy-to-use, and user-friendly desktop environment for application usage and development.

For users, GNOME is a suite that integrates the desktop environment and applications. For developers, GNOME is an application development framework, consisting of a large number of function libraries. Applications written in GNOME can run properly even if users do not run the GNOME desktop environment.

GNOME includes basic software such as the file manager, app store, and text editor, and advanced applications and tools such as system sampling analysis, system logs, software engineering IDE, web browser, simple VM monitor, and developer document browser.

You are advised to create an administrator during the installation.

1. [Download](https://www.openeuler.org/en/) the openEuler ISO image, install the system, and update the software source. The Everything and EPOL sources need to be configured. The following command is used to install GNOME in minimum installation mode.

   ```sh
   sudo dnf update
   ```

2. Install a front library.

   ```sh
   sudo dnf install dejavu-fonts liberation-fonts gnu-*-fonts google-*-fonts
   ```

3. Install Xorg.

   ```sh
   sudo dnf install xorg-*
   ```

In this case, many extra packages may be installed. You can run the following commands to install the required Xorg packages:

```
sudo dnf install xorg-x11-apps xorg-x11-drivers xorg-x11-drv-ati \
 xorg-x11-drv-dummy xorg-x11-drv-evdev xorg-x11-drv-fbdev xorg-x11-drv-intel \
 xorg-x11-drv-libinput xorg-x11-drv-nouveau xorg-x11-drv-qxl \
 xorg-x11-drv-synaptics-legacy xorg-x11-drv-v4l xorg-x11-drv-vesa \
 xorg-x11-drv-vmware xorg-x11-drv-wacom xorg-x11-fonts xorg-x11-fonts-others \
 xorg-x11-font-utils xorg-x11-server xorg-x11-server-utils xorg-x11-server-Xephyr \
 xorg-x11-server-Xspice xorg-x11-util-macros xorg-x11-utils xorg-x11-xauth \
 xorg-x11-xbitmaps xorg-x11-xinit xorg-x11-xkb-utils
```

4\. Install GNOME and it's components.

````
```sh
sudo dnf install adwaita-icon-theme atk atkmm at-spi2-atk at-spi2-core baobab \
 abattis-cantarell-fonts cheese clutter clutter-gst3 clutter-gtk cogl dconf \
 dconf-editor devhelp eog epiphany evince evolution-data-server file-roller folks \
 gcab gcr gdk-pixbuf2 gdm gedit geocode-glib gfbgraph gjs glib2 glibmm24 \
 glib-networking gmime30 gnome-autoar gnome-backgrounds gnome-bluetooth \
 gnome-builder gnome-calculator gnome-calendar gnome-characters \
 gnome-clocks gnome-color-manager gnome-contacts gnome-control-center \
 gnome-desktop3 gnome-disk-utility gnome-font-viewer gnome-getting-started-docs \
 gnome-initial-setup gnome-keyring gnome-logs gnome-menus gnome-music \
 gnome-online-accounts gnome-online-miners gnome-photos gnome-remote-desktop \
 gnome-screenshot gnome-session gnome-settings-daemon gnome-shell \
 gnome-shell-extensions gnome-software gnome-system-monitor gnome-terminal \
 gnome-tour gnome-user-docs gnome-user-share gnome-video-effects \
 gnome-weather gobject-introspection gom grilo grilo-plugins \
 gsettings-desktop-schemas gsound gspell gssdp gtk3 gtk4 gtk-doc gtkmm30 \
 gtksourceview4 gtk-vnc2 gupnp gupnp-av gupnp-dlna gvfs json-glib libchamplain \
 libdazzle libgdata libgee libgnomekbd libgsf libgtop2 libgweather libgxps libhandy \
 libmediaart libnma libnotify libpeas librsvg2 libsecret libsigc++20 libsoup \
 mm-common mutter nautilus orca pango pangomm libphodav python3-pyatspi \
 python3-gobject rest rygel simple-scan sushi sysprof tepl totem totem-pl-parser \
 tracker3 tracker3-miners vala vte291 yelp yelp-tools \
 yelp-xsl zenity
```
````

5\. Enable GNOME Display Manager (GDM).

````
```sh
sudo systemctl enable gdm
```
````

6\. Set the default login mode to GUI.

````
```sh
sudo systemctl set-default graphical.target
```
````

Reboot the device for configuration verification.

```
sudo reboot
```

7\. If GDM cannot work:

Disable GDM if it is installed by default.

```
sudo systemctl disable gdm
```

Install LightDM instead.

```
sudo dnf install lightdm lightdm-gtk
```

Set the default desktop to GNOME as the root user.

```
echo 'user-session=gnome' >> /etc/lightdm/lightdm.conf.d/60-lightdm-gtk-greeter.conf
```

Enable LightDM.

```
sudo systemctl enable lightdm
```

Set the default login mode to GUI.

```
sudo systemctl set-default graphical.target
```

Reboot the device for configuration verification.

```
sudo reboot
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_on_servers.md
---
# Installing on a Server

This guide describes how to install openEuler on a server and is intended for openEuler users with a basic understanding of Linux system management.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/install_stratovirt.md
---
# Installing StratoVirt

## Software and Hardware Requirements

### Minimum Hardware Requirements

* Processor architecture: Only the AArch64 and x86\_64 processor architectures are supported. AArch64 requires ARMv8 or a later version that supports virtualization extension. x86\_64 requires VT-x support.

* 2-core CPU

* 4 GiB memory

* 16 GiB available disk space

### Software Requirements

Operating system: openEuler 22.03 LTS SP4

## Component Installation

To use StratoVirt virtualization, it is necessary to install StratoVirt. Before the installation, ensure that the openEuler Yum source has been configured.

1. Run the following command as user **root** to install the StratoVirt component:

   ```shell
   # yum install stratovirt
   ```

2. Check whether the installation is successful.

   ```shell
   $ stratovirt -version
   StratoVirt 2.1.0
   ```

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/syscare/installing_syscare.md
---
# Installing SysCare

This chapter describes how to install SysCare on openEuler.

## Installing SysCare Core Components

### Minimum Hardware Requirements

* 2 CPUs (x86\_64 or AArch64)
* 4 GB memory
* 100 GB drive

### Prerequisites

1. openEuler 22.03 LTS SP4 has been installed.
2. **root** permissions are required for patch making.

### Installing from Source

Clone the SysCare source code <https://atomgit.com/openeuler/syscare.git> and then compile and install SysCare as follows:

```shell
dnf install -y kernel-source-`uname -r` kernel-debuginfo-`uname -r` kernel-devel-`uname -r`
dnf install -y elfutils-libelf-devel openssl-devel dwarves flex python3-devel rpm-build bison cmake make gcc g++ rust cargo bpftool clang libbpf libbpf-devel llvm libbpf-static

git clone https://atomgit.com/openeuler/syscare.git
cd syscare
mkdir build_tmp
cd build_tmp
cmake -DCMAKE_INSTALL_PREFIX=/usr ..
make
make install
```

### Installing SysCare from a Repository

The repository of openEuler 22.03 LTS SP4 contains SysCare packages. You can use the `dnf` or `yum` command to download and install them.

```shell
dnf install syscare syscare-build
```

### Uninstalling SysCare

```shell
dnf remove syscare*
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/installing_the_kubernetes_software_package.md
---
# Installing the Kubernetes Software Package

```bash
dnf install -y docker conntrack-tools socat
```

In versions later than EPOL, Kubernetes can be directly installed through DNF.

```bash
dnf install kubernetes*
```

---

---
url: /en/docs/22.03_LTS_SP4/server/development/application_dev/installing_obs.md
---
# Installing the OBS Tool

## Description

Open Build Service (OBS) is a general tool for building source packages into RPM packages or Linux images.
obs-server is the software package of OBS.

## Supported Architectures

OBS supports x86\_64 and AArch64 architectures.

## OBS Installation

openEuler 22.03 LTS SP4 for the AArch64 architecture is used as an example to demonstrate how to install the multi-architecture obs-server packages.

1. Check whether the OS is openEuler 22.03 LTS SP4.

   ```shell
   $ cat /etc/openEuler-release
   openEuler release 22.03 LTS SP4
   ```

2. Configure the Yum source. The repo source for the multi-architecture obs-server must be placed before the **everything** repo source. An example Yum source configuration is as follows:

   ```shell
   [everything]
   name=everything
   baseurl=https://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/aarch64/
   enabled=1
   gpgcheck=0
   ```

   RUn the following command to open the repo source file and add the preceding content.

   ```shell
   sudo vi /etc/yum.repos.d/xxx.repo
   ```

3. Enable the Yum source.

   ```shell
   sudo yum clean all
   sudo yum makecache
   ```

4. Check whether OBS packages of other versions exist.

   ```shell
   sudo rpm -qa obs-server obs-common obs-api mod_passenger obs-api-deps obs-bundled-gems passenger ruby ruby-help ruby-irb rubygem-bundler rubygem-io-console rubygem-json rubygem-openssl rubygem-psych rubygem-rake rubygem-rdoc rubygems rubygem-bigdecimal rubygem-did_you_mean
   ```

5. (Optional) To prevent conflicts, uninstall OBS packages of other versions.

   ```shell
   sudo yum remove -y obs-server obs-common obs-api mod_passenger obs-api-deps obs-bundled-gems passenger ruby ruby-help ruby-irb rubygem-bundler rubygem-io-console rubygem-json rubygem-openssl rubygem-psych rubygem-rake rubygem-rdoc rubygems rubygem-bigdecimal rubygem-did_you_mean
   ```

   > **Note**
   >
   > * The example repo source is the multi-architecture version of obs-server released with openEuler 22.03 LTS SP4.
   > * Installation dependency packages of different versions may conflict, causing installation failure. You are advised to uninstall the preceding software packages before installation.

6. Install obs-server packages.

   ```shell
   sudo yum install -y obs-api obs-server
   ```

7. Check whether obs-server packages are successfully installed.

   ```shell
   $ rpm -qa | grep obs-server
   obs-server-2.10.11-6.oe2203.noarch
   $ rpm -qa | grep obs-api
   obs-api-2.10.11-6.oe2203.noarch
   ```

## OBS Deployment

1. Obtain the deployment script at <https://atomgit.com/openeuler/infrastructure/tree/master/obs/tf/startup>.

2. Run the **restart\_service.sh** script to deploy the OBS tool.

## Usage Instructions

You can build RPM packages using the OBS web UI or the osc CLI tool. For details, see [Building an RPM Package](./building_an_rpm_package.md).

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/virtualization_installation.md
---
# Installing Virtualization Components

This chapter describes how to install virtualization components in openEuler.

## Minimum Hardware Requirements

The minimum hardware requirements for installing virtualization components on openEuler are as follows:

* AArch64 processor architecture: ARMv8 or later, supporting virtualization expansion
* x86\_64 processor architecture, supporting VT-x
* 2-core CPU
* 4 GB memory
* 16 GB available disk space

## Installing Core Virtualization Components

### Installation Methods

#### Prerequisites

* The Yum source has been configured. For details, see the *openEuler 22.03 LTS SP4 Administrator Guide*.
* Only the administrator has permission to perform the installation.

#### Procedure

1. Install the QEMU component.

   ```shell
   yum install -y qemu
   ```

2. Install the libvirt component.

   ```shell
   yum install -y libvirt
   ```

3. Start the libvirtd service.

   ```shell
   systemctl start libvirtd
   ```

> \[!NOTE] **NOTE:**
> The KVM module is integrated in the openEuler kernel and does not need to be installed separately.

### Verifying the Installation

1. Check whether the kernel supports KVM virtualization, that is, check whether the  **/dev/kvm**  and  **/sys/module/kvm**  files exist. The command and output are as follows:

   ```shell
   $ ls /dev/kvm
   /dev/kvm
   ```

   ```shell
   $ ls /sys/module/kvm
   parameters  uevent
   ```

   If the preceding files exist, the kernel supports KVM virtualization. If the preceding files do not exist, KVM virtualization is not enabled during kernel compilation. In this case, you need to use the Linux kernel that supports KVM virtualization.

2. Check whether QEMU is successfully installed. If the installation is successful, the QEMU software package information is displayed. The command and output are as follows:

   ```shell
   $ rpm -qi qemu
   Name        : qemu
   Epoch       : 10
   Version     : 6.2.0
   Release     : 76.oe2203SP3
   Architecture: aarch64
   Install Date: Tue 15 Aug 2023 09:04:47 PM CST
   Group       : Unspecified
   Size        : 26733299
   License     : GPLv2 and BSD and MIT and CC-BY-SA-4.0
   Signature   : RSA/SHA256, Tue 01 Aug 2023 09:28:19 PM CST, Key ID 007fb747fb37bc6f
   Source RPM  : qemu-6.2.0-76.oe2203SP3.src.rpm
   Build Date  : Tue 01 Aug 2023 09:24:00 PM CST
   Build Host  : localhost
   Relocations : (not relocatable)
   URL         : http://www.qemu.org
   Summary     : QEMU is a generic and open source machine emulator and virtualizer
   Description :
   QEMU is a generic and open source processor emulator which achieves a good
   emulation speed by using dynamic translation. QEMU has two operating modes:

    * Full system emulation. In this mode, QEMU emulates a full system (for
      example a PC), including a processor and various peripherals. It can be
      used to launch different Operating Systems without rebooting the PC or
      to debug system code.
    * User mode emulation. In this mode, QEMU can launch Linux processes compiled
      for one CPU on another CPU.

   As QEMU requires no host kernel patches to run, it is safe and easy to use.
   ```

3. Check whether libvirt is successfully installed. If the installation is successful, the libvirt software package information is displayed. The command and output are as follows:

   ```shell
   $ rpm -qi libvirt
   Name        : libvirt
   Version     : 6.2.0
   Release     : 57.oe2203SP3
   Architecture: aarch64
   Install Date: Tue 30 Jul 2023 04:56:21 PM CST
   Group       : Unspecified
   Size        : 0
   License     : LGPLv2+
   Signature   : RSA/SHA256, Tue 01 Aug 2023 09:28:19 PM CST, Key ID 007fb747fb37bc6f
   Source RPM  : libvirt-6.2.0-57.oe2203SP3.src.rpm
   Build Date  : Tue 01 Aug 2023 09:24:00 PM CST
   Build Host  : 71e8c1ce149f
   Relocations : (not relocatable)
   URL         : https://libvirt.org/
   Summary     : Library providing a simple virtualization API
   Description :
   Libvirt is a C toolkit to interact with the virtualization capabilities
   of recent versions of Linux (and other OSes). The main package includes
   the libvirtd server exporting the virtualization support.
   ```

4. Check whether the libvirt service is started successfully. If the service is in the  **active**  state, the service is started successfully. You can use the virsh command line tool provided by the libvirt. The command and output are as follows:

   ```shell
   $ systemctl status libvirtd
   ● libvirtd.service - Virtualization daemon
      Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
      Active: active (running) since Tue 2019-08-06 09:36:01 CST; 5h 12min ago
        Docs: man:libvirtd(8)
              https://libvirt.org
    Main PID: 40754 (libvirtd)
       Tasks: 20 (limit: 32768)
      Memory: 198.6M
      CGroup: /system.slice/libvirtd.service
              ─40754 /usr/sbin/libvirtd
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/interconnecting_isula_shim_v2_with_stratovirt.md
---
# Interconnecting iSula shim v2 with StratoVirt

## Overview

shim v2 is a next-generation shim solution. Compared with shim v1, shim v2 features shorter call chains, clearer architecture, and lower memory overhead in multi-service container scenarios. iSula can run secure containers through isulad-shim or containerd-shim-kata-v2. The isulad-shim component is the implementation of the shim v1 solution, and the containerd-shim-kata-v2 component is the implementation of the shim v2 solution in the secure container scenario. This document describes how to interconnect iSula with containerd-shim-kata-v2.

## Interconnecting with containerd-shim-kata-v2

### Prerequisites

Before interconnecting iSula with containerd-shim-kata-v2, ensure that the following prerequisites are met:

* iSulad, lib-shim-v2, and kata-containers have been installed.
* StratoVirt supports only the devicemapper storage driver. Therefore, you need to configure the devicemapper environment and ensure that the devicemapper storage driver used by iSulad works properly.

### Environment Setup

The following describes how to install and configure iSulad and kata-containers.

#### Installing Dependencies

Configure the YUM source based on the OS version and install iSulad, lib-shim-v2, and kata-containers as the **root** user.

```shell
yum install iSulad
yum install kata-containers
yum install lib-shim-v2
```

#### Creating and Configuring a Storage Device

Prepare a drive, for example, **/dev/sdx**. The drive will be formatted. This section uses the block device **/dev/sda** as an example.

I. Creating devicemapper

1. Create a physical volume (PV).

   ```shell
   $ pvcreate /dev/sda
     Physical volume "/dev/loop0" successfully created.
   ```

2. Create a volume group (VG).

   ```shell
   $ vgcreate isula /dev/sda
     Volume group "isula" successfully created
   ```

3. Create the logical volumes **thinpool** and **thinpoolmeta**.

   ```shell
   $ lvcreate --wipesignatures y -n thinpool isula -l 95%VG
     Logical volume "thinpool" created.

   $ lvcreate --wipesignatures y -n thinpoolmeta isula -l 1%VG
     Logical volume "thinpoolmeta" created.
   ```

4. Convert the created logical volumes to a thin pool.

   ```shell
   $ lvconvert -y --zero n -c 64K \
   --thinpool isula/thinpool \
   --poolmetadata isula/thinpoolmeta
     Thin pool volume with chunk size 512.00 KiB can address at most 126.50 TiB of data.
     WARNING: Converting isula/thinpool and isula/thinpoolmeta to thin pool's data and metadata volumes with metadata wiping.
     THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
     Converted isula/thinpool and isula/thinpoolmeta to thin pool.
   ```

5. Configure automatic extension of the thin pool using lvm.

   ```shell
   $ touch /etc/lvm/profile/isula-thinpool.profile
   $ cat << EOF > /etc/lvm/profile/isula-thinpool.profile
   activation {
     thin_pool_autoextend_threshold=80
     thin_pool_autoextend_percent=20
   }
   EOF
   $ lvchange --metadataprofile isula-thinpool isula/thinpool
   Logical volume isula/thinpool changed.
   ```

II. Changing the iSulad Storage Driver Type and Setting the Default Runtime

Modify the **/etc/isulad/daemon.json** configuration file. Set **default-runtime** to **io.containerd.kata.v2** and **storage-driver** to **devicemapper**. The modification result is as follows:

```json
 {
    "default-runtime": "io.containerd.kata.v2",
    "storage-driver": "devicemapper",
    "storage-opts": [
        "dm.thinpooldev=/dev/mapper/isula-thinpool",
        "dm.fs=ext4",
        "dm.min_free_space=10%"
    ],
}
```

III. Making the Configuration Take Effect

1. Restart the iSulad for the configuration to take effect.

   ```shell
   systemctl daemon-reload
   systemctl restart isulad
   ```

2. Check whether the iSula storage driver is successfully configured.

   ```shell
   isula info
   ```

   If the following information is displayed, the configuration is successful:

   ```text
   Storage Driver: devicemapper
   ```

### Interconnection Guide

This section describes how to interconnect iSula with containerd-shim-kata-v2.

By default, containerd-shim-kata-v2 uses QEMU as the virtualization component. The following describes how to configure QEMU and StratoVirt.

#### Using QEMU

If containerd-shim-kata-v2 uses QEMU as the virtualization component, perform the following operations to interconnect iSula with containerd-shim-kata-v2:

1. Modify the kata configuration file **/usr/share/defaults/kata-containers/configuration.toml**.

   Set **sandbox\_cgroup\_with\_emulator** to **false**. Currently, shim v2 does not support this function. Other parameters are the same as the kata configuration parameters in shim v1 or use the default values.

   ```toml
   sandbox_cgroup_with_emulator = false
   ```

2. Use the BusyBox image to run the secure container and check whether the used runtime is io.containerd.kata.v2.

   ```bash
   $ id=`isula run -tid busybox /bin/sh`
   $ isula inspect -f '{{ json .HostConfig.Runtime }}' $id
   "io.containerd.kata.v2"
   ```

3. Verify that the QEMU-based VM process is started. If it is started, QEMU is successfully interconnected with the shim v2 secure container.

   ```bash
   ps -ef | grep qemu
   ```

#### Using StratoVirt

If containerd-shim-kata-v2 uses StratoVirt as the virtualization component, perform the following operations to interconnect iSula with containerd-shim-kata-v2:

1. Create the **stratovirt.sh** script in any directory (for example, **/home**) and add the execute permission to the file as the **root** user.

   ```shell
   touch /home/stratovirt.sh
   chmod +x /home/stratovirt.sh
   ```

   The content of **stratovirt.sh** is as follows, which is used to specify the path of StratoVirt:

   ```shell
   #!/bin/bash
   export STRATOVIRT_LOG_LEVEL=info  # set log level which includes trace, debug, info, warn and error.
   /usr/bin/stratovirt $@
   ```

2. Modify the kata configuration file. Set **hypervisor** of the secure container to **stratovirt**, **kernel** to the absolute path of the StratoVirt kernel image, and **initrd** to the initrd image file of kata-containers (if you use YUM to install kata-containers, the initrd image file is downloaded by default and stored in the **/var/lib/kata/** directory). StratoVirt supports only the devicemapper storage mode, prepare the environment in advance and set iSulad to the devicemapper mode.

   The configurations are as follows:

   ```shell
   [hypervisor.stratovirt]
   path = "/home/stratovirt.sh"
   kernel = "/var/lib/kata/vmlinux.bin"
   initrd = "/var/lib/kata/kata-containers-initrd.img"
   block_device_driver = "virtio-mmio"
   use_vsock = true
   enable_netmon = true
   internetworking_model="tcfilter"
   sandbox_cgroup_with_emulator = false
   disable_new_netns = false
   disable_block_device_use = false
   disable_vhost_net = true
   ```

   To use the vsock function in StratoVirt, enable the vhost\_vsock kernel module and check whether the module is successfully enabled.

   ```bash
   modprobe vhost_vsock
   lsmod |grep vhost_vsock
   ```

   Download the kernel of the required version and architecture and save it to the **/var/lib/kata/** directory. For example, download the [openeuler repo](https://repo.openeuler.org/) of the x86 architecture of openEuler 22.03 LTS.

   ```bash
   cd /var/lib/kata
   wget https://repo.openeuler.org/openEuler-22.03-LTS/stratovirt_img/x86_64/vmlinux.bin
   ```

3. Use the BusyBox image to run the secure container and check whether the used runtime is io.containerd.kata.v2.

   ```bash
   $ id=`isula run -tid busybox sh`
   $ isula inspect -f '{{ json .HostConfig.Runtime }}' $id
   "io.containerd.kata.v2"
   ```

4. Verify that the StratoVirt-based VM process is started. If it is started, StratoVirt is successfully interconnected with the shim v2 secure container.

   ```bash
   ps -ef | grep stratovirt
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/interconnect_libvirt.md
---
# Interconnecting with libvirt

## Overview

libvirt is an upper-layer software that manages different types of Hypervisors using different drivers and provides unified and stable APIs.

In cloud scenarios, libvirt is widely used to manage large numbers of VMs. To facilitate the deployment, orchestration, and management of large-scale StratoVirt VMs, StratoVirt interconnects with libvirt through the libvirt northbound interface. In this case, you can use an XML file of libvirt to describe a VM, including the VM name, CPU, and disks.

This chapter describes the XML configurations supported by the StratoVirt platform and how to use the `virsh` command to manage VMs.

## Prerequisites

To interconnect StratoVirt with libvirt, the host must meet the following requirements:

* The Yum source has been correctly configured.
* libvirt has been correctly installed and started.
* StratoVirt has been correctly installed.

## VM Configuration

The libvirt tool uses an XML file to describe features about a VM, including the VM name, CPUs, memory, disks, and NICs. You can manage the VM by modifying the XML configuration file.

Before interconnecting StratoVirt with libvirt, configure the XML file first. This section describes the supported XML configuration items and configuration methods during interconnection between StratoVirt and libvirt.

> \[!NOTE]**NOTE**
>
> Before using libvirt to manage StratoVirt VMs, pay attention to the features supported by StratoVirt, including mutually exclusive relationships between features, and feature prerequisites and specifications. For details, see [Configuring VMs](./vm_configuration.md) in CLI mode.

### VM Description

A VM XML file must contain the two basic elements that describe the VM: **domain** and **name**.

#### Elements

* **domain**: root element of the VM configuration, which is used to configure the Hypervisor type that runs the StratoVirt VM.

  Attribute **type**: type of **domain**. In StratoVirt, the value is **kvm**.

* **name**: VM name.

  A VM name contains a maximum of 255 characters, consisting of digits, letters, underscores, hyphens, and colons. Names of VMs on the same host must be unique.

#### Configuration Example

Assume that the VM name is StratoVirt. The following is the example:

```shell
<domain type='kvm'>
    <name>StratoVirt</name>
    ...
</domain>
```

### Virtual CPU and Memory

This section describes how to configure virtual CPUs and memory.

#### Elements

* **vcpu**: number of virtual processors.

* **memory**: size of the virtual memory.

  Attribute **unit**: memory unit. The value can be **KiB** (210 bytes), **MiB** (220 bytes), **GiB** (230 bytes), or **TiB** (240 bytes).

  > \[!NOTE]**NOTE**
  >
  > StratoVirt does not support the CPU topology. Do not set this item.

#### Configuration Example

The following is an example of configuring 8 GiB memory and four virtual CPUs:

```xml
<domain type='kvm'>
    ...
    <vcpu>4</vcpu>
    <memory unit='GiB'>8</memory>
    ...
</domain>
```

### VM Devices

This section describes how to use the XML file to configure VM devices, including disk, NIC, RNG (random number generator), balloon, console, and vsock devices.

#### Disks

#### Elements

* Attribute **type**: type of the backend storage medium. In StraroVirt, the value is **file**.

  Attribute **device**: type of the storage medium presented to the VM. In StraroVirt, the value is **disk**.

* **driver**: details about the backend driver.

  Attribute **type**: disk format type. In StraroVirt, the value is **raw**. Currently, StratoVirt supports only **raw** disks.

  Attribute **iothread**: iothread configured for the disk. The value is the iothread ID. Before configuring the disk iothread, use the **iothread** element to configure the iothread quantity.

* **source**: backend storage medium.

  Attribute **file**: disk path.

* **target**: details about the backend driver.

  Attribute **dev**: disk name.

  Attribute **bus**: disk device type. In StraroVirt, the value is **virtio**.

* **iotune**: disk I/O feature.

  Attribute **total\_iops\_sec**: disk IOPS.

* **address**: attribute of the bus to which the device is to be mounted.

  Attribute **type**: bus type. In StratoVirt, the value is **pci**.

  Attribute **domain**: domain of the VM.

  Attribute **bus**: ID of the bus to which the device is to be mounted.

  Attribute **slot**: ID of the slot to which the device is to be mounted. The value range is \[0, 31].

  Attribute **function**: ID of the function to which the device is to be mounted. The value range is \[0, 7].

#### Configuration Example

Set the disk path to **/home/openEuler-22.03-LTS-SP4-stratovirt.img**, iothread quantity to **1**, disk iothread to **iothread1**, and IOPS to **10000**, and mount the disk to the PCI bus whose bus ID is 1, slot ID is 0, and function ID is 0. The following is the example:

```xml
<domain type='kvm'>
    ...
    <iothreads>1</iothreads>
    <devices>
        <disk type='file' device='disk'>
     <driver name='qemu' type='raw' iothread="1"/>
     <source file='/home/openEuler-22.03-LTS-SP4-stratovirt.img'/>
     <target dev='hda' bus='virtio'/>
        <iotune>
         <total_iops_sec>10000</total_iops_sec>
        </iotune>
     <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x00' />
 </disk>
    ...
    </devices>
</domain>
```

#### Network Devices

#### Elements

* **interface**: network interface.

  Attribute **type**: network device type.

* **mac**: virtual NIC address.

  Attribute **address**: virtual NIC address.

* **source**: backend network bridge.

  Attribute **bridge**: network bridge.

* **target**: backend NIC.

  Attribute **dev**: backend tap device.

* **model**: virtual NIC type.

  Attribute **type**: virtual NIC type. In StratoVirt, the value is **virtio**.

* **driver**: whether to enable the vhost.

  Attribute **name**: If **name** is set to **qemu**, the virtio-net device is used. If **driver** is not configured or **name** is set to **vhost**, the vhost-net device is used.

#### Configuration Example

Before configuring the network, [configure the Linux bridge](.././virtualization/environment_preparation.md#setting-up-a-linux-bridge) first. Set the MAC address to **de:ad:be:ef:00:01** and network bridge to **br0**. Use the virtio-net device, and mount it to the PCI bus whose bus ID is 2, slot ID is 0, and function ID is 0. The following is the example:

```xml
<domain type='kvm'>
    ...
    <devices>
        <interface type='bridge'>
            <mac address='de:ad:be:ef:00:01' />
            <source bridge='br0'/>
            <model type='virtio'/>
            <driver name='qemu'/>
            <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x00' />
        </interface>
        ...
    </devices>
</domain>
```

#### Balloon Devices

#### Elements

* **memballoon**: balloon device type.

  Attribute **model**: type of the balloon device. In StratoVirt, the value is **virtio**.

* **alias**: alias of the balloon device.

  Attribute **name**: ID of the balloon device.

  Attribute **autodeflate**: auto deflate feature. The options are **on** and **off**.

#### Configuration Example

Configure the balloon device, enable the auto deflate feature, and mount it to the PCI bus whose bus ID is 3, slot ID is 0, and function ID is 0. The following is the example:

```xml
<domain type='kvm'>
    ...
    <devices>
        <memballoon model='virtio'>
            <alias name='balloon0' autodeflate='on'/>

            <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x00' />
        </memballoon>
        ...
    </devices>
</domain>
```

#### Console Devices

The console device is mounted to the virtio-serial bus. Therefore, you need to create a virtio-serial device when creating a console device.

> \[!NOTE]**NOTE**
>
> The console device of StratoVirt does not support the multi-port feature. Each VM can be configured with only one console device.

#### Elements

* **controller**: controller.

  Attribute **type**: controller type. The value is **virtio-serial**.

* **alias**: alias.

  Attribute **name**: device ID.

* **console**: console device.

  Attribute **type**: redirection mode of the console device. The following redirection modes are supported: **pty**, **file**, and **unix**.

* **target**: configuration of the console device.

  Attribute **type**: console device type. In StratoVirt, the value is **virtio**.

#### Configuration Example

Set the redirection mode to **pty** and mount the console device to the PCI bus whose bus ID is 4, slot ID is 0, and function ID is 0. The following is the example:

```xml
<domain type='kvm'>
    ...
    <devices>
        <controller type='virtio-serial'>
            <alias name='virt-serial0'/>
            <address type='pci' domain='0x000' bus='0x04' slot='0x00' function='0x00'/>
        </controller>
        <console type='pty'>
            <target type='virtio'/>
            <alias name='console0'/>
        </console>
        ...
    </devices>
</domain>
```

#### RNG Devices

#### Elements

* **rng**: RNG device.

  Attribute **model**: type of the RNG device. In StratoVirt, the value is **virtio**.

* **rate**: rate at which the RNG device generates random numbers.

  Attribute **period**: period of random number generation, in milliseconds. Currently, the StratoVirt does not allow you to set the period value. The default value (1000 milliseconds) is used.

  Attribute **bytes**: maximum number of bytes generated in a period.

* **backend**: RNG device backend. The value is the path of the RNG device on the host.

  Attribute **model**: type of the backend device. In StratoVirt, the value is **random**.

#### Configuration Example

Configure that a maximum of 1234 bytes are generated within 1000 ms. The path of the RNG device on the host is **/dev/random**, and the device is mounted to the PCI bus whose bus ID is 5, slot ID is 0, and function ID is 0. The following is the example:

```xml
<domain type='kvm'>
    ...
    <devices>
        <rng model='virtio'>
            <rate period='1000' bytes='1234'/>
            <backend model='random'>/dev/random</backend>
            <address type='pci' domain='0x000' bus='0x05' slot='0x00' function='0x00'/>
        </rng>
        ...
    </devices>
</domain>
```

#### vsock Devices

#### Elements

* **vsock**: vsock device.

  Attribute **model**: type of the vsock device. In StratoVirt, the value is **virtio**.

* **cid**: CID of the vsock device.

  Attribute **address**: sets the CID value.

#### Configuration Example

Set **cid** to **8** and mount the device to the PCI bus whose bus ID is 6, slot ID is 0, and function ID is 0. The following is the example:

```xml
<domain type='kvm'>
    ...
    <devices>
        <vsock model='virtio'>
            <cid auto='no' address='8'/>
            <address type='pci' domain='0x000' bus='0x06' slot='0x00' function='0x00'/>
        </vsock>
        ...
    </devices>
</domain>
```

### System Architecture Configuration

The XML file also contains some architecture-related configurations, such as the pflash and mainboard.

#### Elements

* **os**: defines VM startup parameters.

  Child element **type**: VM type. Attribute **arch** indicates the architecture and **machine** indicates the mainboard type. In StratoVirt, the AArch64 architecture supports only the virt mainboard, and the x86\_64 architecture supports only the Q35 mainboard.

  Child element **kernel**: kernel path.

  Child element **cmdline**: command line startup parameters.

  Child element **loader**: loading firmware. Attribute **readonly** indicates that the firmware is read-only and **type** indicates the firmware type. In StratoVirt, the type value is **pflash**.

* **features**: features supported by Hypervisors.

  Child element **acpi**: whether to support ACPI. The ACPI feature is used in StratoVirt, so it must be configured.

  Child element **gic**: interrupt processor specified for ARM processors. Attribute **version** indicates the GIC version. In StratoVirt, the value is **3**.

#### Configuration Example

Set the CPU architecture of the VM to ARM and the mainboard to **virt**. The startup command is `console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw`. The path of pflash is **/usr/share/edk2/aarch64/QEMU\_EFI-pflash.raw**, which is read-only. The kernel path is **/home/std-vmlinuxz**. The following is the example:

```xml
<domain type='kvm'>
    ...
    <os>
        <type arch='aarch64' machine='virt'>hvm</type>
        <kernel>/home/std-vmlinuxz</kernel>
        <cmdline>console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw</cmdline>
        <loader readonly='yes' type='pflash'>`/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw`</loader>
    </os>
    ...
</domain>
```

### Huge Page Memory

#### Elements

* **memoryBacking**: configures the memory information.

* **hugepages**: configures memory huge pages.

* **page**: configures huge pages.

  Attribute **size**: size of huge memory pages.

  Attribute **unit**: unit of the huge page size.

#### Configuration Example

The following is an example of configuring 2 MiB huge pages:

```xml
<domain type='kvm'>
    ...
    <memoryBacking>
    <hugepages>
        <page size='2' unit='MiB' />
    </hugepages>
    </memoryBacking>
    ...
</domain>
```

### Configuration Examples

#### x86 Configuration Example

Configure a server named StratoVirt with 8 GiB memory, 1 GiB huge pages, and four vCPUs. Its architecture is x86\_64 and the mainboard type is Q35. The following is a configuration example of the corresponding XML file:

```xml
<domain type='kvm'>
    <name>StratoVirt</name>
    <memory unit='GiB'>8</memory>
    <!-- hugepages -->
    <memoryBacking>
    <hugepages>
        <page size='1' unit='GiB' />
    </hugepages>
    </memoryBacking>
    <vcpu>4</vcpu>
    <!-- iothread -->
    <iothreads>1</iothreads>
    <os>
    <type arch='x86_64' machine='q35'>hvm</type>
    <kernel>/path/to/standard_vm_kernel</kernel>
    <cmdline>console=hvc0 root=/dev/vda reboot=k panic=1 rw</cmdline>
    <loader readonly='yes' type='pflash'>/path/to/pflash</loader>
    <nvram>/path/to/OVMF_VARS</nvram>
    </os>
    <features>
        <acpi/>
    </features>
    <devices>
        <emulator>/path/to/StratoVirt_binary_file</emulator>
        <controller type='pci' index='0' model='pcie-root' />
        <!-- pcie-root-port -->
        <controller type='pci' index='1' model='pcie-root-port' />
        <controller type='pci' index='2' model='pcie-root-port' />
        <controller type='pci' index='3' model='pcie-root-port' />
        <controller type='pci' index='4' model='pcie-root-port' />
        <controller type='pci' index='5' model='pcie-root-port' />
        <controller type='pci' index='6' model='pcie-root-port' />
        <controller type='pci' index='7' model='pcie-root-port' />
        <!-- block -->
        <disk type='file' device='disk'>
            <driver name='qemu'  type='raw' iothread='1'/>
            <source file='/path/to/rootfs'/>
            <target dev='hda' bus='virtio'/>
            <iotune>
                <total_iops_sec>1000</total_iops_sec>
            </iotune>
            <address type='pci' domain='0x000' bus='0x01' slot='0x00' function='0x00'/>
        </disk>
        <!-- net -->
        <interface type='bridge'>
            <mac address='de:ad:be:ef:00:01'/>
            <source bridge='qbr0'/>
            <model type='virtio'/>
            <address type='pci' domain='0x000' bus='0x02' slot='0x00' function='0x00'/>
        </interface>
        <!-- virtio-serial -->
        <controller type='virtio-serial' index='0'>
            <alias name='virt-serial0'/>
            <address type='pci' domain='0x000' bus='0x03' slot='0x00' function='0x00'/>
        </controller>
        <console type='pty'>
            <target type='virtio' port='0'/>
            <alias name='console0'/>
        </console>
        <!-- balloon -->
        <memballoon model='virtio'>
            <alias name='balloon0'/>
            <address type='pci' domain='0x000' bus='0x04' slot='0x00' function='0x00'/>
        </memballoon>
        <!-- rng -->
        <rng model='virtio'>
            <rate period='1000' bytes='1234'/>
            <backend model='random'>/path/to/random_file</backend>
            <address type='pci' domain='0x000' bus='0x05' slot='0x00' function='0x00'/>
        </rng>
        <!-- vsock -->
        <vsock model='virtio'>
            <cid auto='no' address='3'/>
            <address type='pci' domain='0x000' bus='0x06' slot='0x00' function='0x00'/>
        </vsock>
    </devices>
</domain>
```

#### ARM Configuration Example

Configure a server named StratoVirt with 8 GiB memory, 1 GiB huge pages, four vCPUs. Its architecture is AArch64 and the mainboard type is virt. The configuration example of the corresponding XML file is as follows:

```xml
<domain type='kvm'>
    <name>StratoVirt</name>
    <memory unit='GiB'>8</memory>
    <!-- hugepages -->
    <memoryBacking>
    <hugepages>
        <page size='1' unit='GiB' />
    </hugepages>
    </memoryBacking>
    <vcpu>4</vcpu>
    <!-- iothread -->
    <iothreads>1</iothreads>
    <os>
    <type arch='aarch64' machine='virt'>hvm</type>
    <kernel>/path/to/standard_vm_kernel</kernel>
    <cmdline>console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw</cmdline>
    <loader readonly='yes' type='pflash'>/path/to/pflash</loader>
    </os>
    <features>
        <acpi/>
        <gic version='3'/>
    </features>
    <devices>
        <emulator>/path/to/StratoVirt_binary_file</emulator>
        <controller type='pci' index='0' model='pcie-root'/>
        <!-- block -->
        <disk type='file' device='disk'>
            <driver name='qemu'  type='raw' iothread='1'/>
            <source file='/path/to/rootfs'/>
            <target dev='hda' bus='virtio'/>
            <address type='pci' domain='0x000' bus='0x00' slot='0x01' function='0x00'/>
            <iotune>
                <total_iops_sec>1000</total_iops_sec>
            </iotune>
        </disk>
        <!-- net -->
        <interface type='bridge'>
            <mac address='de:ad:be:ef:00:01'/>
            <source bridge='qbr0'/>
            <model type='virtio'/>
            <address type='pci' domain='0x000' bus='0x00' slot='0x02' function='0x00'/>
        </interface>
        <!-- virtio-serial -->
        <controller type='virtio-serial' index='0'>
            <alias name='virt-serial0'/>
            <address type='pci' domain='0x000' bus='0x00' slot='0x03' function='0x00'/>
        </controller>
        <console type='pty'>
            <target type='virtio' port='0'/>
            <alias name='console0'/>
        </console>
        <!-- balloon -->
        <memballoon model='virtio'>
            <alias name='balloon0'/>
            <address type='pci' domain='0x000' bus='0x00' slot='0x04' function='0x00'/>
        </memballoon>
        <!-- rng -->
        <rng model='virtio'>
            <rate period='1000' bytes='1234'/>
            <backend model='random'>/path/to/random_file</backend>
            <address type='pci' domain='0x000' bus='0x00' slot='0x05' function='0x00'/>
        </rng>
        <!-- vsock -->
        <vsock model='virtio'>
            <cid auto='no' address='3'/>
            <address type='pci' domain='0x000' bus='0x00' slot='0x06' function='0x00'/>
        </vsock>
    </devices>
</domain>
```

## VM Management

libvirt uses `virsh` commands to manage VMs. After the StratoVirt platform is interconnected with libvirt, only the following commands for interaction with StratoVirt are supported:

* `create`: creates a VM.

* `suspend`: suspends a VM.

* `resume`: resumes a VM.

* `destroy`: destroys a VM.

* `console`: logs in to a VM through the console.

> \[!NOTE]**NOTE**
>
> StratoVirt does not support commands for restarting or shutting down VMs.

### VM Lifecycle Management

If you have created a VM configuration file named **StratoVirt** in st.xml format, you can use the following commands for VM lifecycle management:

* Creating a VM.

  ```shell
  virsh create st.xml
  ```

  After the VM is created, you can run the `virsh list` command to check whether a VM named **StratoVirt** exists.

* Suspending a VM.

  ```shell
  virsh suspend StratoVirt
  ```

  After the VM is suspended, it stops running. You can run the `virsh list` command to check whether the status of VM **StratoVirt** is **paused**.

* Resuming a VM.

  ```shell
  virsh resume StratoVirt
  ```

  After the VM is resumed, you can run the `virsh list` command to check whether the status of VM **StratoVirt** is **running**.

* Destroying a VM.

  ```shell
  virsh destroy StratoVirt
  ```

  After the VM is destroyed, you can run the `virsh list` command to check that VM **StratoVirt** does not exist.

### VM Login

After the VM is created, you can run the `virsh console` command to log in to it to perform specific operations. If the VM name is **StratoVirt**, run the following command:

```shell
virsh console StratoVirt
```

> \[!NOTE]**NOTE**
>
> To use the `virsh console` command, set the redirection type of the console device to **pty** in the XML file.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/interconnection_with_the_cni_network.md
---
# Interconnection with the CNI Network

## Overview

The container runtime interface (CRI) is provided to connect to the CNI network, including parsing the CNI network configuration file and adding or removing a pod to or from the CNI network. When a pod needs to support a network through a container network plug-in such as Canal, the CRI needs to be interconnected to Canal so as to provide the network capability for the pod.

## Common CNIs

Common CNIs include CNI network configuration items in the CNI network configuration and pod configuration. These CNIs are visible to users.

* CNI network configuration items in the CNI network configuration refer to those used to specify the path of the CNI network configuration file, path of the binary file of the CNI network plug-in, and network mode. For details, see  [Table 1](#en-us_topic_0183259146_table18221919589).
* CNI network configuration items in the pod configuration refer to those used to set the additional CNI network list to which the pod is added. By default, the pod is added only to the default CNI network plane. You can add the pod to multiple CNI network planes as required.

**Table  1**  CNI network configuration items

Additional CNI network configuration mode:

Add the network plane configuration item "network.alpha.kubernetes.io/network" to annotations in the pod configuration file.

The network plane is configured in JSON format, including:

* **name**: specifies the name of the CNI network plane.
* **interface**: specifies the name of a network interface.

The following is an example of the CNI network configuration method:

```json
"annotations" : {
        "network.alpha.kubernetes.io/network": "{\"name\": \"mynet\", \"interface\": \"eth1\"}"
 }
```

### CNI Network Configuration Description

The CNI network configuration includes two types, both of which are in the .json file format.

* Single-network plane configuration file with the file name extension .conf or .json. For details about the configuration items, see  Table 1 in the appendix.
* Multi-network plane configuration file with the file name extension .conflist. For details about the configuration items, see  Table 3 in the appendix.

### Adding a Pod to the CNI Network List

If  **--network-plugin=cni**  is configured for iSulad and the default network plane is configured, a pod is automatically added to the default network plane when the pod is started. If the additional network configuration is configured in the pod configuration, the pod is added to these additional network planes when the pod is started.

**port\_mappings**  in the pod configuration is also a network configuration item, which is used to set the port mapping of the pod. To set port mapping, perform the following steps:

```json
"port_mappings":[
     { 
         "protocol": 1,
         "container_port": 80,
         "host_port": 8080
      }
]
```

* **protocol**: protocol used for mapping. The value can be  **tcp**  (identified by 0) or  **udp**  (identified by 1).
* **container\_port**: port through which the container is mapped.
* **host\_port**: port mapped to the host.

### Removing a Pod from the CNI Network List

When StopPodSandbox is called, the interface for removing a pod from the CNI network list will be called to clear network resources.

> \[!NOTE] **NOTE:**
>
> 1. Before calling the RemovePodSandbox interface, you must call the StopPodSandbox interface at least once.
> 2. If StopPodSandbox fails to call the CNI, residual network resources may exist.

## Usage Restrictions

* Currently, only CNI 0.3.0 and CNI 0.3.1 are supported. In later versions, CNI 0.1.0 and CNI 0.2.0 may need to be supported. Therefore, when error logs are displayed, the information about CNI 0.1.0 and CNI 0.2.0 is reserved.
* name: The value must contain lowercase letters, digits, hyphens (-), and periods (.) and cannot be started or ended with a hyphen or period. The value can contain a maximum of 200 characters.
* The number of configuration files cannot exceed 200, and the size of a single configuration file cannot exceed 1 MB.
* The extended parameters need to be configured based on the actual network requirements. Optional parameters do not need to be written into the netconf.json file.

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/introduction.md
---
# Introduction

openEuler is an open-source operating system. The current openEuler kernel is based on Linux and supports Kunpeng and other processors. It fully unleashes the potential of computing chips. As an efficient, stable, and secure open-source OS built by global open-source contributors, openEuler applies to database, big data, cloud computing, and artificial intelligence (AI) scenarios. In addition, openEuler community is an open-source community for global OSs. Through community cooperation, openEuler builds an innovative platform, builds a unified and open OS that supports multiple processor architectures, and promotes the prosperity of the software and hardware application ecosystem.

---

---
url: /en/docs/22.03_LTS_SP4/edge_computing/ros/getting_to_know_ros.md
---
# Introduction to ROS

## Introduction

ROS is an open source meta-operating system for robotics. It provides the services that an operating system should have, including hardware abstraction, low-level device control, implementation of common functions, inter-process message passing, and package management. It also provides the tools and library functions needed to fetch, compile, write, and run code across computers.

ROS's operating architecture is a processing architecture that uses ROS communication modules to implement loosely coupled network connections between modules [P2P](https://en.wikipedia.org/wiki/Peer-to-peer), which implements several types of communications, including:

1. Service-based synchronous [RPC](https://en.wikipedia.org/wiki/Remote_procedure_call) (remote procedure call) communication;
2. Topic-based asynchronous data flow communication, as well as data storage on the parameter server.

Since the beginning of ROS in 2007, along with the great development of robot technology, the core ideas and basic software packages of ROS have been gradually improved and different ROS distributions have been released. Below is a list of current and historical ROS releases, the rows marked in green in the table are the currently supported releases.

![ROS release](./figures/ROS-release.png)

Although ROS is still a powerful development tool in the field of robotics, due to the limitations of the initial design, many problems have gradually been exposed. For example: poor real-time performance, high system overhead, unfriendly support for Python3, no encryption mechanism and low security. Many developers and research institutions have also made improvements to address the limitations of ROS, but these local function improvements are often difficult to bring about overall performance improvements. At ROSCon 2014, the design architecture of the new generation of ROS (Next-generation ROS: Building on DDS) was officially announced. On August 31, 2015, the first alpha version of ROS2.0 was launched, and different release versions were released later. Below is a list of current and historical ROS2 releases, the rows marked in green in the table are currently supported releases.

![ROS2 release](./figures/ROS2-release.png)

## Architecture

The overall architecture of ROS is shown in the figure below:

![ROS architecture](./figures/ROS-ROS2.png)

1. OS Layer
   * ROS1 is mainly built on the Linux system, and ROS2 has brought changes. The systems supported include Linux, Windows, Mac, RTOS, and even bare metal without an operating system.
2. Middleware Layer
   * One of the most important concepts in ROS is the "node" based on the publish/subscribe model, which allows developers to develop low-coupling functional modules in parallel and facilitates secondary reuse. The communication system of ROS1 is based on TCPROS/UDPROS, while the communication system of ROS2 is based on DDS. DDS is a standard solution for data publishing/subscribing in distributed real-time systems, which will be explained in detail in the next section. ROS2 provides an abstract layer implementation of DDS internally, and users do not need to pay attention to the underlying DDS provider.
   * In the ROS1 architecture, Nodelet and TCPROS/UDPROS are parallel layers, providing a more optimized data transmission method for multiple nodes in the same process. This data transmission method is also retained in ROS2, but it is called "Intra-process", which is also independent of DDS.
3. Application Layer

   * ROS1 is strongly dependent on the ROS Master, and one can imagine what kind of dilemma the entire system will face once the Master goes down. But from the architecture of ROS2 on the right, we can find that the Master, which was a worry before, has finally disappeared, and nodes use a discovery mechanism called "Discovery" to obtain each other's information.

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secgear/introduction_to_secgear.md
---
# Introduction to secGear

## Overview

With the rapid development of cloud computing, more and more enterprises deploy computing services on the cloud. The security of user data on the third-party cloud infrastructure is facing great challenges. Confidential computing is a technology that uses hardware-based trusted execution environments (TEEs) to protect confidentiality and integrity of data in use. It relies on the bottom-layer hardware to build the minimum trust dependency, which removes the OS, hypervisor, infrastructure, system administrator, and service provider from the trusted entity list as unauthorized entities to reduce potential risks. There are various confidential computing technologies (such as Intel SGX, Arm TrustZone, and RISC-V Keystone) and software development kits (SDKs) in the industry and the application ecosystem of different TEEs are isolated from each other, which brings high development and maintenance costs to confidential computing application developers. To help developers quickly build confidential computing solutions that protect data security on the cloud, openEuler launches the unified confidential computing programming framework secGear.

## Architecture

![](./figures/secGear_arch.png)

secGear features the following benefits:

* **Architecture compatibility**: It masks differences between different SDK APIs by sharing the same set of source code across multiple architectures.
* **Easy development**: The development tools and common security components allow users to focus on services, significantly improving development efficiency.
* **High performance**: The switchless feature improves the interaction performance between the rich execution environment (REE) and TEE by more than 10-fold in typical scenarios such as frequent interactions between the REE and TEE and big data interaction.

## Key Features

### Switchless

#### Pain Points

After a conventional application is reconstructed using confidential computing, the rich execution environment (REE) logic frequently invokes the TEE logic or the REE frequently exchanges large data blocks with the TEE. Each call between the REE and TEE requires context switching among the REE user mode, REE kernel mode, driver, TEE kernel mode, and TEE user mode. When large blocks of data are exchanged during the call, multiple memory copies are generated. In addition, the interaction performance between the REE and TEE deteriorates due to factors such as the size limit of underlying data blocks, which severely affects the implementation of confidential computing applications.

#### Solution

Switchless is a technology that uses shared memory to reduce the number of context switches and data copies between the REE and TEE to optimize the interaction performance.

### Remote Attestation

#### Pain Points

Confidential computing is developed to ensure data security on the cloud. However, due to the damage of data leakage, tenants still have concerns about the theoretical security of confidential computing, which affects its promotion and application.

#### Solution

Confidential computing vendors have launched the remote attestation technology, which enables tenants to detect the trustworthiness status of the TEE and TAs on the cloud at any time.
Remote attestation is a real-time measurement technology that measures the TEE and applications running in the TEE, generates attestation reports, and uses the preset root key to sign the reports to prevent them from being tampered with or forged.
secGear encapsulates remote attestation APIs based on the remote attestation capability of each vendor's SDK. secGear must run on the Kunpeng platform.

### Secure Channel

#### Pain Points

When requesting the confidential computing service on the cloud, the data owner needs to upload the data to be processed to the TEE on the cloud for processing. Because the TEE is not connected to the network, the data needs to be transferred to the REE over the network in plaintext and then transferred to the TEE from the REE. The data plaintext is exposed in the REE memory, which poses security risks.

#### Solution

A secure channel is a technology that combines confidential computing remote attestation to implement secure key negotiation between the data owner and the TEE on the cloud. It negotiates a sessionkey owned only by the data owner and the TEE on the cloud. Then the sessionkey is used to encrypt user data transferred over the network. After receiving the ciphertext data, the REE transfers the data to the TEE for decryption and processing.

## Acronyms and Abbreviations

| Acronym/Abbreviation| Full Name                     |
| ------ | ----------------------------- |
| REE    | rich execution environment   |
| TEE    | trusted execution environment|
| EDL    | enclave description language |

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/stratovirt_introduction.md
---
# Introduction to StratoVirt

## Overview

StratoVirt is an enterprise-class Virtual Machine Monitor (VMM) oriented to cloud data centers in the computing industry. It uses a unified architecture to support VM, container, and serverless scenarios. StratoVirt has competitive advantages in key technologies such as lightweight low noise, software and hardware synergy, and Rust language-level security.
StratoVirt reserves component-based assembling capabilities and APIs in the architecture design. Advanced features can be flexibly assembled as required until they evolve to support standard virtualization. In this way, StratoVirt can strike a balance between feature requirements, application scenarios, and flexibility.

## Architecture Description

The StratoVirt core architecture consists of three layers from top to bottom:

* External API: compatible with the QEMU Monitor Protocol (QMP), has complete OCI compatibility capabilities, and supports interconnection with libvirt.
* BootLoader: discards the traditional BIOS+GRUB boot mode to achieve fast boot in lightweight scenarios, and provides UEFI boot support for standard VMs.
* Emulated mainboard:
  * MicroVM: Fully utilizes software and hardware collaboration capabilities, simplifies device models, and provides low-latency resource scaling capabilities.
  * Standard VM: implements UEFI boot with constructed ACPI tables. Virtio-pci and VFIO devices can be attached to greatly improve the VM I/O performance.

Figure 1 shows the overall architecture.

**Figure 1** Overall architecture of StratoVirt

![](./figures/StratoVirt_architecture.jpg)

## Features

* Highly isolated based on hardware.
* Fast cold boot: Benefiting from the minimalist design, a microVM can be started within 50ms.
* Low memory overhead: StratoVirt works with a memory footprint of less than 4MB.
* I/O enhancement: StratoVirt offers common I/O capabilities and minimalist I/O device emulation.
* OCI compatibility: StratoVirt works with iSula and Kata containers, and can be integrated into the Kubernetes ecosystem perfectly;
* Multi-platform support: StratoVirt fully supports Intel and Arm platforms.
* Extensibility: StratoVirt has interface and design for importing more features, and can be extended to support standard virtualization.
* Security: fewer than 46 syscalls while running;

## Implementation

### Running Architecture

* A StratoVirt VM is an independent process in Linux. The process has three types of threads: main thread, vCPU thread, and I/O thread:
  * The main thread is a cycle for asynchronously collecting and processing events from external modules, such as a vCPU thread.
  * Each vCPU has a thread to handle trap events of this vCPU.
  * I/O threads can be configured for I/O devices to improve I/O performance.

## Restrictions

* Only the Linux operating system is supported, and the recommended kernel versions are 4.19 and 5.10.
* Only Linux is supported as the operating system of the VM, and the recommended kernel versions are 4.19 and 5.10;
* A maximum of 254 CPUs are supported.

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/syscare/syscare_introduction.md
---
# Introduction to SysCare

## Overview

SysCare is an online live patching tool for both kernel and user modes. It automatically fixes bugs and vulnerabilities in OS components, such as kernels, user-mode services, and dynamic libraries.

![img](./figures/syscare_arch.png)

## SysCare Functions

SysCare supports live patching for kernels and user-mode services:

1. One-click creation\
   SysCare is a unified environment for both kernel- and user-mode live patches that ignores differences between patches, ensuring they can be created with just one click.
2. Patch lifecycle operations\
   SysCare provides a unified patch management interface for users to install, activate, uninstall, and query patches.

## SysCare Technologies

1. Unified patches: SysCare masks differences in detail when creating patches, providing a unified management tool to improve O\&M efficiency.
2. User-mode live patching: SysCare supports live patching of multi-process and multi-thread services in user mode, which takes effect when a process or thread is started or restarted.
3. Lazy mechanism: SysCare fixes the ptrace defect (all kernel calls are ended) and improves the fix success rate.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/introduction_to_virtualization.md
---
# Introduction to Virtualization

## Overview

In computer technologies, virtualization is a resource management technology that abstracts various physical resources (such as processors, memory, drives, and network adapters) of a computer, converts the resources, and presents the resources for segmentation and combination into one or more computer configuration environments. This resource management technology breaks the inseparable barrier of the physical structure, and makes these resources not restricted by the architecture, geographical or physical configuration of the existing resources after virtualization. In this way, users can better leverage the computer hardware resources and maximize the resource utilization.

Virtualization enables multiple virtual machines (VMs) to run on a physical server. The VMs share the processors, memory, and I/O device resources of the physical server, but are logically isolated from each other. In the virtualization technology, the physical server is called a host machine, the VM running on the host machine is called a guest, and the operating system (OS) running on the VM is called a guest OS. A layer of software, called the virtualization layer, exists between a host machine and a VM to simulate virtual hardware. This virtualization layer is called a VM monitor, as shown in the following figure.

**Figure  1**  Virtualization architecture
![](./figures/virtualization-architecture.png)

## Virtualization Architecture

Currently, mainstream virtualization technologies are classified into two types based on the implementation structure of the Virtual Machine Monitor (VMM):

* Hypervisor model

  In this model, VMM is considered as a complete operating system (OS) and has the virtualization function. VMM directly manages all physical resources, including processors, memory, and I/O devices.

* Host model

  In this model, physical resources are managed by a host OS, which is a traditional OS, such as Linux and Windows. The host OS does not provide the virtualization capability. The VMM that provides the virtualization capability runs on the host OS as a driver or software of the system. The VMM invokes the host OS service to obtain resources and simulate the processor, memory, and I/O devices. The virtualization implementation of this model includes KVM and Virtual Box.

Kernel-based Virtual Machine (KVM) is a kernel module of Linux. It makes Linux a hypervisor.  [Figure 2](#fig310953013541)  shows the KVM architecture. KVM does not simulate any hardware device. It is used to enable virtualization capabilities provided by the hardware, such as Intel VT-x, AMD-V, Arm virtualization extensions. The user-mode QEMU simulates the mainboard, memory, and I/O devices. The user-mode QEMU works with the kernel KVM module to simulate VM hardware. The guest OS runs on the hardware simulated by the QEMU and KVM.

**Figure  2**  KVM architecture
![](./figures/kvm-architecture.png)

## Virtualization Components

Virtualization components provided in the openEuler software package:

* KVM: provides the core virtualization infrastructure to make the Linux system a hypervisor. Multiple VMs can run on the same host at the same time.
* QEMU: simulates a processor and provides a set of device models to work with KVM to implement hardware-based virtualization simulation acceleration.
* Libvirt: provides a tool set for managing VMs, including unified, stable, and open application programming interfaces (APIs), daemon process (libvirtd), and default command line management tool (virsh).
* Open vSwitch: provides a virtual network tool set for VMs, supports programming extension and standard management interfaces and protocols (such as NetFlow, sFlow, IPFIX, RSPAN, CLI, LACP, and 802.1ag).

## Virtualization Characteristics

Virtualization has the following characteristics:

* Partition

  Virtualization can logically divide software on a physical server to run multiple VMs (virtual servers) with different specifications.

* Isolation

  Virtualization can simulate virtual hardware and provide hardware conditions for VMs to run complete OSs. The OSs of each VM are independent and isolated from each other. For example, if the OS of a VM breaks down due to a fault or malicious damage, the OSs and applications of other VMs are not affected.

* Encapsulation

  Encapsulation is performed on a per VM basis. The excellent encapsulation capability makes VMs more flexible than physical machines. Functions such as live migration, snapshot, and cloning of VMs can be realized, implementing quick deployment and automatic O\&M of data centers.

* Hardware-irrelevant

  After being abstracted by the virtualization layer, VMs are not directly bound to underlying hardware and can run on other servers without being modified.

## Virtualization Advantages

Virtualization brings the following benefits to infrastructure of the data center:

* Flexibility and scalability

  Users can dynamically allocate and reclaim resources based to meet dynamic service requirements. In addition, users can plan different VM specifications based on product requirements and adjust the scale without changing the physical resource configuration.

* Higher availability and better O\&M methods

  Virtualization provides O\&M methods such as live migration, snapshot, live upgrade, and automatic DR. Physical resources can be deleted, upgraded, or changed without affecting users, improving service continuity and implementing automatic O\&M.

* Security hardening

  Virtualization provides OS-level isolation and hardware-based processor operation privilege-level control. Compared with simple sharing mechanisms, virtualization provides higher security and implements controllable and secure access to data and services.

* High resource utilization

  Virtualization supports dynamic sharing of physical resources and resource pools, improving resource utilization.

## openEuler Virtualization

openEuler provides KVM virtualization components that support the AArch64 and x86\_64 processor architectures.

---

---
url: /en/docs/22.03_LTS_SP4/tools/community_tools/isocut/isocut_user_guide.md
---
# isocut Usage Guide

## Introduction

The size of an openEuler image is large, and the process of downloading or transferring an image is time-consuming. In addition, when an openEuler image is used to install the OS, all RPM packages contained in the image are installed. You cannot choose to install only the required software packages.

In some scenarios, you do not need to install the full software package provided by the image, or you need to install additional software packages. Therefore, openEuler provides isocut, an image tailoring and customization tool. You can use this tool to customize an ISO image that contains only the required RPM packages based on an openEuler image. The software packages can be the ones contained in an official ISO image or specified in addition to meet custom requirements.

This document describes how to install and use isocut.

## Software and Hardware Requirements

The hardware and software requirements of the computer to make an ISO file using isocut are as follows:

* CPU architecture: AArch64 or X86\_64
* OS: openEuler 22.03 LTS SP4
* 60 GB or more drive space for running isocut and storing ISO images.

## Installation

The following uses openEuler 22.03 LTS on the AArch64 architecture as an example to describe how to install isocut.

1. Ensure that openEuler 22.03 LTS has been installed on the computer.

   ```shell
   $ cat /etc/openEuler-release 
   openEuler release 22.03 LTS
   ```

2. Download the ISO image (must be an **everything** image) of the corresponding architecture and save it to any directory (it is recommended that the available space of the directory be greater than 20 GB). In this example, the ISO image is saved to the **/home/isocut\_iso** directory.

   The download address of the AArch64 image is as follows:

   <https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso>

   > **Note:**
   > The download address of the x86\_64 image is as follows:
   >
   > <https://repo.openeuler.org/openEuler-22.03-LTS/ISO/x86_64/openEuler-22.03-LTS-everything-x86_64-dvd.iso>

3. Create a **/etc/yum.repos.d/local.repo** file to configure the Yum repository. The following is an example of the configuration file. **baseurl** is the directory for mounting the ISO image.

   ```shell
   [local]
   name=local
   baseurl=file:///home/isocut_mount
   gpgcheck=0
   enabled=1
   ```

4. Run the following command as the **root** user to mount the image to the **/home/isocut\_mount** directory (ensure that the mount directory is the same as **baseurl** configured in the **repo** file) as the Yum repository:

   ```shell
   sudo mount -o loop /home/isocut_iso/openEuler-22.03-LTS-everything-aarch64-dvd.iso /home/isocut_mount
   ```

5. Make the Yum repository take effect.

   ```shell
   yum clean all
   yum makecache
   ```

6. Install isocut as the **root** user.

   ```shell
   sudo yum install -y isocut
   ```

7. Run the following command as the **root** user to verify that the tool has been installed successfully:

   ```shell
   $ sudo isocut -h
   Checking input ...
   usage: isocut [-h] [-t temporary_workspace] [-r rpm_path] [-k kickstart_file_path] [-p product_name] [-v version_number] [-i install_picture_path] [-c cut_packages] source_iso dest_iso

   Cut openEuler iso to small one

   positional arguments:
       source_iso                source iso image
       dest_iso                  destination iso image

   optional arguments:
       -h, --help                show this help message and exit
       -t temporary_workspace    temporary workspace
       -r rpm_path               extern rpm packages path
       -k kickstart_file_path    kickstart file path
       -p product_name           The product name
       -v version_number         The version number
       -i install_picture_path   The path of background pictures during the installation
       -c cut_packages           cut packages, yes/no, default is yes
   ```

## Tailoring and Customizing an Image

This section describes how to use isocut to create an image by tailoring or adding RPM packages to an openEuler image.

### Command Description

#### Format

Run the `isocut` command to use the tool. The command format is as follows:

```shell
isocut [ --help | -h ] [ -t <temporary_workspace> ] [ -r <rpm_path> ] [ -k <kickstart_file_path> ] [ -p <product_name> ] [ -v <version_number>] [ -i <install_picture_path> ] [ -c <cut_packages> ] < source_iso > < dest_iso >
```

#### Parameter Description

| Parameter| Mandatory| Description|
| ------------   | -------- | -------------------------------------------------------- |
| --help | -h | No| Queries the help information about the command.|
| -t <*temporary\_workspace*> | No| Specifies the temporary directory *temp\_path* for running the tool, which is an absolute path. The default value is **/tmp**.|
| -r <*rpm\_path*> | No| Specifies the path of the RPM packages to be added to the ISO image.|
| -k <*kickstart\_file\_path*> | No      | Specifies the kickstart template path if kickstart is used for automatic installation.   |
| -p <*product\_name*>         | No       | Product name                                                   |
| -v <*version\_number*>       | No       | Product version                                                  |
| -i <*install\_picture\_path*> | No       | Path of background pictures used during the installation                                             |
| -c <*cut\_packages*>         | No       | Specifies whether the RPM packages need to be cut. The default value is to cut RPM packages.   |
| *source\_iso*   | Yes| Path and name of the ISO source image to be tailored. If no path is specified, the current path is used by default.|
| *dest\_iso*     | Yes| Specifies the path and name of the new ISO image created by the tool. If no path is specified, the current path is used by default.|

> \[!NOTE] **Background pictures used during the installation must meet the following naming and resolution requirements:**
>
> * Left side bar background during installation: **sidebar-bg.png**, 290x780
> * Logo at the upper left corner during installation: **sidebar-logo.png**, 132x32
> * Upper tool bar background of the settings page: **topbar-bg.png**, 831x105

### Software Package Source

The RPM packages of the new image can be:

* Packages contained in an official ISO image. In this case, the RPM packages to be installed are specified in the configuration file **/etc/isocut/rpmlist**. The configuration format is *software\_package\_name.architecture*. For example, **kernel.aarch64**.

* Specified in addition. In this case, use the `-r` parameter to specify the path in which the RPM packages are stored when running the `isocut` command and add the RPM package names to the **/etc/isocut/rpmlist** configuration file. (See the name format above.)

  > \[!NOTE] **NOTE:**
  >
  > * During image customization, if an RPM package specified in the configuration file cannot be found, the RPM package will not be added to the image.
  > * If the dependency of the RPM package is incorrect, an error may be reported when running isocut.

### kickstart Functions

You can use kickstart to install images automatically by using the `-k` parameter to specify a kickstart file when running the **isocut** command.

isocut provides a kickstart template (**/etc/isocut/anaconda-ks.cfg**). You can modify the template as required.

#### Modifying the kickstart Template

If you need to use the kickstart template provided by isocut, perform the following modifications:

* Configure the root user password and the GRUB2 password in the **/etc/isocut/anaconda-ks.cfg** file. Otherwise, the automatic image installation will pause during the password setting process, waiting for you to manually enter the passwords.
* If you want to specify additional RPM packages and use kickstart for automatic installation, specify the RPM packages in the **%packages** field in both the **/etc/isocut/rpmlist** file and the kickstart file.

See the next section for details about how to modify the kickstart file.

##### Configuring Initial Passwords

###### Setting the Initial Password of the **root** User

Set the initial password of the **root** user as follows in the **/etc/isocut/anaconda-ks.cfg** file. Replace **${pwd}** with the encrypted password.

```shell
rootpw --iscrypted ${pwd}
```

Obtain the initial password of the **root** user as follows (**root** permissions are required):

1. Add a user for generating the password, for example, **testUser**.

   ```shell
   sudo useradd testUser
   ```

2. Set the password for the **testUser** user. Run the following command to set the password as prompted:

   ```shell
   $ sudo passwd testUser
   Changing password for user testUser.
   New password: 
   Retype new password: 
   passwd: all authentication tokens updated successfully.
   ```

3. View the **/etc/shadow** file to obtain the encrypted password. The encrypted password is the string between the two colons (:) following the **testUser** user name. (\*\*\*\*\*\*\* is used as an example.)

   ```shell
   $ sudo cat /etc/shadow | grep testUser
   testUser:***:19052:0:90:7:35::
   ```

4. Run the following command to replace the **pwd** field in the **/etc/isocut/anaconda-ks.cfg** file with the encrypted password (replace \*\*\*\*\*\*\* with the actual password):

   ```shell
   rootpw --iscrypted ***
   ```

###### Configuring the Initial GRUB2 Password

Add the following configuration to the **/etc/isocut/anaconda-ks.cfg** file to set the initial GRUB2 password: Replace **${pwd}** with the encrypted password.

```text
%addon com_huawei_grub_safe --iscrypted --password='${pwd}'
%end
```

> \[!NOTE] NOTE:
>
> * The **root** permissions are required for configuring the initial GRUB password.
>
> * The default user corresponding to the GRUB password is **root**.
>
> * The `grub2-set-password` command must exist in the system. If the command does not exist, install it in advance.

1. Run the following command and set the GRUB2 password as prompted:

   ```shell
   $ sudo grub2-set-password -o ./
   Enter password: 
   Confirm password: 
   grep: .//grub.cfg: No such file or directory
   WARNING: The current configuration lacks password support!
   Update your configuration with grub2-mkconfig to support this feature.
   ```

2. After the command is executed, the **user.cfg** file is generated in the current directory. The content starting with **grub.pbkdf2.sha512** is the encrypted GRUB2 password.

   ```shell
   $ sudo cat user.cfg 
   GRUB2_PASSWORD=grub.pbkdf2.sha512.***
   ```

3. Add the following information to the **/etc/isocut/anaconda-ks.cfg** file. Replace \*\*\*\*\*\*\* with the encrypted GRUB2 password.

   ```text
   %addon com_huawei_grub_safe --iscrypted --password='grub.pbkdf2.sha512.***'
   %end
   ```

##### Configuring the %packages Field

If you want to specify additional RPM packages and use kickstart for automatic installation, specify the RPM packages in the **%packages** field in both the **/etc/isocut/rpmlist** file and the kickstart file.

This section describes how to specify RPM packages in the **/etc/isocut/anaconda-ks.cfg** file.

The default configurations of **%packages** in the **/etc/isocut/anaconda-ks.cfg** file are as follows:

```text
%packages --multilib --ignoremissing
acl.aarch64
aide.aarch64
......
NetworkManager.aarch64
%end
```

Add specified RPM packages to the **%packages** configurations in the following format:

*software\_package\_name.architecture*. For example, **kernel.aarch64**.

```text
%packages --multilib --ignoremissing
acl.aarch64
aide.aarch64
......
NetworkManager.aarch64
kernel.aarch64
%end
```

### Operation Guide

> \[!NOTE] **NOTE:**
>
> * Do not modify or delete the default configuration items in the **/etc/isocut/rpmlist** file.
> * All `isocut` operations require **root** permissions.
> * The source image to be tailored can be a basic image or **everything** image. In this example, the basic image **openEuler-22.03-LTS-aarch64-dvd.iso** is used.
> * In this example, assume that the new image is named **new.iso** and stored in the **/home/result** directory, the temporary directory for running the tool is **/home/temp**, and the additional RPM packages are stored in the **/home/rpms** directory.

1. Open the configuration file **/etc/isocut/rpmlist** and specify the RPM packages to be installed (from the official ISO image).

   ```shell
   sudo vi /etc/isocut/rpmlist
   ```

2. Ensure that the space of the temporary directory for running isocut is greater than 8 GB. The default temporary directory is\*\*/tmp\*\*. You can also use the `-t` parameter to specify another directory as the temporary directory. The path of the directory must be an absolute path. In this example, the **/home/temp** directory is used. The following command output indicates that the available drive space of the **/home** directory is 38 GB, which meets the requirements.

   ```shell
   $ df -h
   Filesystem                            Size  Used Avail Use% Mounted on
   devtmpfs                              1.2G     0  1.2G   0% /dev
   tmpfs                                 1.5G     0  1.5G   0% /dev/shm
   tmpfs                                 1.5G   23M  1.5G   2% /run
   tmpfs                                 1.5G     0  1.5G   0% /sys/fs/cgroup
   /dev/mapper/openeuler_openeuler-root   69G  2.8G   63G   5% /
   /dev/sda2                             976M  114M  796M  13% /boot
   /dev/mapper/openeuler_openeuler-home   61G   21G   38G  35% /home
   ```

3. Tailor and customize the image.

   **Scenario 1**: All RPM packages of the new image are from the official ISO image.

   ```shell
   $ sudo isocut -t /home/temp /home/isocut_iso/openEuler-22.03-LTS-SP4-aarch64-dvd.iso /home/result/new.iso
     Checking input ...
     Checking user ...
     Checking necessary tools ...
     Initing workspace ...
     Copying basic part of iso image ...
     Downloading rpms ...
     Finish create yum conf
     finished
     Regenerating repodata ...
     Checking rpm deps ...
     Getting the description of iso image ...
     Remaking iso ...
     Adding checksum for iso ...
     Adding sha256sum for iso ...
     ISO cutout succeeded, enjoy your new image "/home/result/new.iso"
     isocut.lock unlocked ...
   ```

   If the preceding information is displayed, the custom image **new.iso** is successfully created.

   **Scenario 2**: The RPM packages of the new image are from the official ISO image and additional packages in **/home/rpms**.

   ```shell
   sudo isocut -t /home/temp -r /home/rpms /home/isocut_iso/openEuler-22.03-LTS-SP4-aarch64-dvd.iso /home/result/new.iso
   ```

   **Scenario 3**: The kickstart file is used for automatic installation. You need to modify the **/etc/isocut/anaconda-ks.cfg** file.

   ```shell
   sudo isocut -t /home/temp -k /etc/isocut/anaconda-ks.cfg /home/isocut_iso/openEuler-22.03-LTS-SP4-aarch64-dvd.iso /home/result/new.iso
   ```

### cut\_packages Functions

Based on the standard ISO image released by openEuler, the RPM package can be tailored on demand during the installation customization.

By specifying the `cut_packages` parameter, you can choose whether to cut RPM packages.

#### Operation Guide

**Scenario 1**: The user chooses not to cut RPM packages.

```shell
    $ sudo isocut -t /opt/tlriso/tmp -p openEuler -c no -v 22.03-LTS-SP4 openEuler-22.03-LTS-SP4-x86_64-dvd.iso openEuler-22.03-LTS-SP4-x86_64-dvd_new.iso
      Checking input ...
      Checking user ...
      Checking necessary tools ...
      Initing workspace ...
      Copying basic part of iso image ...
      Getting the description of iso image ...
      Downloading rpms ...
      Finish create yum conf
      finished
      Regenerating repodata ...
      Checking rpm deps ...
      Skip checking rpm deps!!
      Replacing install background pictures ...
      Updating EFI config file ...
      Updating legacy config file ...
      Updating treeinfo file ...
      Customizing kickstart file ...
      Remaking iso ...
      Adding checksum for iso ...
      Adding sha256sum for iso ...
      ISO cutout succeeded, enjoy your new image "openEuler-22.03-LTS-SP4-x86_64-dvd_new.iso"
      isocut.lock unlocked ...
```

If the preceding information is displayed, the custom image **new.iso** is successfully created.

**Scenario 2**: The user chooses to cut RPM packages.

```shell
sudo isocut -t /opt/tlriso/tmp -p openEuler -c yes -v 22.03-LTS-SP4 openEuler-22.03-LTS-SP4-x86_64-dvd.iso openEuler-22.03-LTS-SP4-x86_64-dvd_new.iso
```

**Scenario 3**: The RPM packages are cut by default, and the value of the parameter is empty.

```shell
sudo isocut -t /opt/tlriso/tmp -p openEuler -v 22.03-LTS-SP4 openEuler-22.03-LTS-SP4-x86_64-dvd.iso openEuler-22.03-LTS-SP4-x86_64-dvd_new.iso
```

## FAQs

### The System Fails to Be Installed Using an Image Tailored Based on the Default RPM Package List

#### Context

When isocut is used to tailor an image, the **/etc/isocut/rpmlist** configuration file is used to specify the software packages to be installed.

Images of different OS versions contain different software packages. As a result, some packages may be missing during image tailoring.
Therefore, the **/etc/isocut/rpmlist** file contains only the kernel software package by default,
ensuring that the image can be successfully tailored.

#### Symptom

The ISO image is successfully tailored using the default configuration, but fails to be installed.

An error message is displayed during the installation, indicating that packages are missing:

![](./figures/lack_pack.png)

#### Possible Cause

The ISO image tailored based on the default RPM package list lacks necessary RPM packages during installation.
The missing RPM packages are displayed in the error message, and may vary depending on the version.

#### Solution

Add the missing packages.

1. Find the missing RPM packages based on the error message.
2. Add the missing RPM packages to the **/etc/isocut/rpmlist** configuration file.
3. Tailor and install the ISO image again.

For example, if the missing packages are those in the example error message, modify the **rpmlist** configuration file as follows:

```shell
$ cat /etc/isocut/rpmlist
kernel.aarch64
lvm2.aarch64
chrony.aarch64
authselect.aarch64
shim.aarch64
efibootmgr.aarch64
grub2-efi-aa64.aarch64
dosfstools.aarch64
```

---

---
url: /zh/docs/22.03_LTS_SP4/tools/community_tools/isocut/isocut_user_guide.md
---
# isocut 使用指南

## 简介

openEuler 光盘镜像较大，下载、传输镜像很耗时。另外，使用 openEuler 光盘镜像安装操作系统时，会安装镜像所包含的全量 RPM 软件包，用户无法只安装部分所需的软件包。

在某些场景下，用户不需要安装镜像提供的全量软件包，或者需要一些额外的软件包。因此，openEuler 提供了镜像裁剪定制工具。通过该工具，用户可以基于 openEuler 光盘镜像裁剪定制仅包含所需 RPM 软件包的 ISO 镜像。这些软件包可以来自原有 ISO 镜像，也可以额外指定，从而满足用户定制需求。

本文档介绍 openEuler 镜像裁剪定制工具的安装和使用方法，以指导用户更好的完成镜像裁剪定制。

## 软硬件要求

使用 openEuler 裁剪定制工具制作 ISO 所使用的机器需要满足如下软硬件要求：

* CPU 架构为 AArch64。
* 操作系统为 openEuler 22.03 LTS SP4。
* 建议预留 60 GB 以上的磁盘空间（用于运行裁剪定制工具和存放 ISO 镜像）。

## 安装工具

此处以 openEuler 22.03 LTS SP4 版本的 AArch64 架构为例，介绍 ISO 镜像裁剪定制工具的安装操作。

1. 确认机器已安装操作系统 openEuler 22.03 LTS SP4（镜像裁剪定制工具的运行环境）。

   ```shell script
    $ cat /etc/openEuler-release 
    openEuler release 22.03 LTS SP4
   ```

2. 下载对应架构的 ISO 镜像（必须是 everything 版本），并存放在任一目录（建议该目录磁盘空间大于 20 GB），此处假设存放在 /home/isocut\_iso 目录。

   AArch64 架构的镜像下载链接为：

   <https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/aarch64/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso>

   > **说明：**
   > x86\_64 架构的镜像下载链接为：
   >
   > <https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/x86_64/openEuler-22.03-LTS-SP4-everything-x86_64-dvd.iso>

3. 创建文件 /etc/yum.repos.d/local.repo，配置对应 yum 源。配置内容参考如下，其中 baseurl 是用于挂载 ISO 镜像的目录：

   ```shell script
   [local]
   name=local
   baseurl=file:///home/isocut_mount
   gpgcheck=0
   enabled=1
   ```

4. 使用 root 权限，挂载光盘镜像到 /home/isocut\_mount 目录（请与上述 repo 文件中配置的 baseurl 保持一致）作为 yum 源，参考命令如下：

   ```shell
   sudo mount -o loop /home/isocut_iso/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso /home/isocut_mount
   ```

5. 使 yum 源生效：

   ```shell
   yum clean all
   yum makecache
   ```

6. 使用 root 权限，安装镜像裁剪定制工具：

   ```shell
   sudo yum install -y isocut
   ```

7. 使用 root 权限，确认工具已安装成功：

   ```shell
    $ sudo isocut -h
    Checking input ...
    usage: isocut [-h] [-t temporary_workspace] [-r rpm_path] [-k kickstart_file_path] [-p product_name] [-v version_number] [-i install_picture_path] [-c cut_packages] source_iso dest_iso
    
    Cut openEuler iso to small one
    
    positional arguments:
      source_iso                source iso image
      dest_iso                  destination iso image
    
    optional arguments:
      -h, --help                show this help message and exit
      -t temporary_workspace    temporary workspace
      -r rpm_path               extern rpm packages path
      -k kickstart_file_path    kickstart file path
      -p product_name           The product name
      -v version_number         The version number
      -i install_picture_path   The path of background pictures during the installation
      -c cut_packages           cut packages, yes/no, default is yes
   ```

## 裁剪定制镜像

此处介绍如何使用镜像裁剪定制工具基于 openEuler 光盘镜像裁剪或添加额外 RPM 软件包制作新镜像的方法。

### 命令介绍

#### 命令格式

镜像裁剪定制工具通过 isocut 命令执行功能。命令的使用格式为：

**isocut** \[ --help | -h ] \[ -t <*temporary\_workspace*> ] \[ -r <*rpm\_path*> ] \[ -k <*kickstart\_file\_path*> ] \[ -p <*product\_name*> ] \[ -v <*version\_number*>] \[ -i <*install\_picture\_path*> ] \[ -c <*cut\_packages*> ] < *source\_iso* > < *dest\_iso* >

#### 参数说明

| 参数                          | 是否必选 | 参数含义                                                    |
|-----------------------------| -------- |---------------------------------------------------------|
| --help | -h                | 否       | 查询命令的帮助信息。                                              |
| -t <*temporary\_workspace*>  | 否       | 指定工具运行的临时目录 *temporary\_workspace*，其中 *temporary\_workspace* 为绝对路径。默认为 /tmp 。 |
| -r  <*rpm\_path*>             | 否       | 用户需要额外添加到 ISO 镜像中的 RPM 包路径。                             |
| -k <*kickstart\_file\_path*>  | 否       | 用户需要使用 kickstart 自动安装，指定 kickstart 模板路径。                |
| -p <*product\_name*>         | 否       | 产品名称。                                                   |
| -v <*version\_number*>       | 否       | 产品版本号。                                                  |
| -i<*install\_picture\_path*> | 否       | 中度换标替换图片路径。                                             |
| -c <*cut\_packages*>        | 否        | 用户需要通过指定 cut\_packages 参数，选择是否需要裁剪rpm包。默认为裁剪rpm包。   |
| *source\_iso*                | 是       | 用于裁剪的 ISO 源镜像所在路径和名称。不指定路径时，默认当前路径。                     |
| *dest\_iso*                  | 是       | 裁剪定制生成的 ISO 新镜像存放路径和名称。不指定路径时，默认当前路径。                   |

> \[!NOTE]说明
>
> * 表中提到的路径均支持绝对路径和相对路径。
>
> **中度换标替换图片路径下图片文件命名及像素要求如下：**
>
> * 安装引导界面左侧边栏背景图： sidebar-bg.png 290\*780
> * 安装引导界面左上角logo图片： sidebar-logo.png 132\*32
> * 设置信息上侧工具栏背景图： topbar-bg.png 831\*105

### 软件包来源

新镜像的 RPM 包来源有：

* 原有 ISO 镜像。该情况通过配置文件 /etc/isocut/rpmlist 指定需要安装的 RPM 软件包，配置格式为 "软件包名.对应架构"，例如：kernel.aarch64 。

* 额外指定。执行 **isocut** 时使用 -r 参数指定软件包所在路径，并将添加的 RPM 包按上述格式添加到配置文件 /etc/isocut/rpmlist 中。

  > \[!NOTE]说明
  >
  > * 裁剪定制镜像时，若无法找到配置文件中指定的 RPM 包，则镜像中不会添加该 RPM 包。
  > * 若 RPM 包的依赖有问题，则裁剪定制镜像时可能会报错。

### kickstart 功能介绍

用户需要实现镜像自动化安装，可以通过 kickstart 的方式。在执行 **isocut** 时使用 -k 参数指定 kickstart 文件。

isocut 为用户提供了 kickstart 模板，路径是 /etc/isocut/anaconda-ks.cfg，用户可以基于该模板修改。

#### 修改 kickstart 模板

若用户需要使用 isocut 工具提供的 kickstart 模板，需要修改以下内容：

* 必须在文件 /etc/isocut/anaconda-ks.cfg 中配置 root 和 grub2 的密码。否则镜像自动化安装会卡在设置密码的环节，等待用户手动输入密码。
* 如果要添加额外 RPM 包，并使用 kickstart 自动安装，则在 /etc/isocut/rpmlist 和 kickstart 文件的 %packages 字段都要指定该 RPM 包。

接下来介绍 kickstart 文件详细修改方法。

##### 配置初始密码

###### 配置 root 初始密码

/etc/isocut/anaconda-ks.cfg 中 root 初始密码的默认配置如下，其中 ${pwd} 需要替换成用户实际的加密密文：

```shell
rootpw --iscrypted ${pwd}
```

这里给出设置 root 初始密码的方法（需使用 root 权限）：

1. 添加用于生成密码的用户，此处假设 testUser：

   ```shell script
   $ sudo useradd testUser
   ```

2. 设置 testUser 用户的密码。参考命令如下，根据提示设置密码：

   ```shell script
   $ sudo passwd testUser
   Changing password for user testUser.
   New password: 
   Retype new password: 
   passwd: all authentication tokens updated successfully.
   ```

3. 查看 /etc/shadow 文件，获取加密密码（用户 testUser 后，两个 : 间的字符串，此处使用 \*\*\* 代替）：

   ```shell script
   $ sudo cat /etc/shadow | grep testUser
   testUser:***:19052:0:90:7:35::
   ```

4. 拷贝上述加密密码替换 /etc/isocut/anaconda-ks.cfg 中的 pwd 字段，如下所示（请用实际内容替换 \*\*\* ）：

   ```shell script
   rootpw --iscrypted ***
   ```

###### 配置 grub2 初始密码

/etc/isocut/anaconda-ks.cfg 文件中添加以下配置，配置 grub2 初始密码。其中 ${pwd} 需要替换成用户实际的加密密文：

```shell
%addon com_huawei_grub_safe --iscrypted --password='${pwd}'
%end
```

> \[!NOTE]说明
>
> * 配置 grub 初始密码需要使用 root 权限。
> * grub 密码对应的默认用户为 root 。
> * 系统中需有 grub2-set-password 命令，若不存在，请提前安装该命令。

1. 执行如下命令，根据提示设置 grub2 密码：

   ```shell
   $ sudo grub2-set-password -o ./
   Enter password: 
   Confirm password: 
   grep: .//grub.cfg: No such file or directory
   WARNING: The current configuration lacks password support!
   Update your configuration with grub2-mkconfig to support this feature.
   ```

2. 命令执行完成后，会在当前目录生成 user.cfg 文件，grub.pbkdf2.sha512 开头的内容即 grub2 加密密码：

   ```shell
   $ sudo cat user.cfg 
   GRUB2_PASSWORD=grub.pbkdf2.sha512.***
   ```

3. 复制上述密文，并在 /etc/isocut/anaconda-ks.cfg 文件中增加如下配置：

   ```shell
   %addon com_huawei_grub_safe --iscrypted --password='grub.pbkdf2.sha512.***'
   %end
   ```

##### 配置 %packages 字段

如果需要添加额外 RPM 包，并使用 kickstart 自动安装，需要在 /etc/isocut/rpmlist 和 kickstart 文件的 %packages 字段都指定该 RPM 包。

此处介绍在 /etc/isocut/anaconda-ks.cfg 文件中添加 RPM 包。

/etc/isocut/anaconda-ks.cfg 文件的 %packages 默认配置如下：

```shell
%packages --multilib --ignoremissing
acl.aarch64
aide.aarch64
......
NetworkManager.aarch64
%end
```

将额外指定的 RPM 软件包添加到 %packages 配置中，需要遵循如下配置格式：

"软件包名.对应架构"，例如：kernel.aarch64

```shell
%packages --multilib --ignoremissing
acl.aarch64
aide.aarch64
......
NetworkManager.aarch64
kernel.aarch64
%end
```

### 操作指导

> \[!NOTE]说明
>
> * 请不要修改或删除 /etc/isocut/rpmlist 文件中的默认配置项。
> * isocut 的所有操作需要使用 root 权限。
> * 待裁剪的源镜像可以为基础镜像，也可以是 everything 版镜像，例子中以基础版镜像 openEuler-22.03-LTS-SP4-aarch64-dvd.iso 为例。
> * 例子中假设新生成的镜像名称为 new.iso，且存放在 /home/result 路径；运行工具的临时目录为 /home/temp；额外的 RPM 软件包存放在 /home/rpms 目录。

1. 修改配置文件 /etc/isocut/rpmlist，指定用户需要安装的 RPM 软件包（来自原有 ISO 镜像）。

   ```shell script
    sudo vi /etc/isocut/rpmlist
   ```

2. 确定运行镜像裁剪定制工具的临时目录空间大于 8 GB 。默认临时目录为 /tmp，也可以使用 -t 参数指定其他目录作为临时目录，该目录必须为绝对路径。本例中使用目录 /home/temp，由如下回显可知 /home 目录可用磁盘为 38 GB，满足要求。

   ```shell
    $ df -h
    Filesystem                            Size  Used Avail Use% Mounted on
    devtmpfs                              1.2G     0  1.2G   0% /dev
    tmpfs                                 1.5G     0  1.5G   0% /dev/shm
    tmpfs                                 1.5G   23M  1.5G   2% /run
    tmpfs                                 1.5G     0  1.5G   0% /sys/fs/cgroup
    /dev/mapper/openeuler_openeuler-root   69G  2.8G   63G   5% /
    /dev/sda2                             976M  114M  796M  13% /boot
    /dev/mapper/openeuler_openeuler-home   61G   21G   38G  35% /home
   ```

3. 执行裁剪定制。

   **场景一**：新镜像的所有 RPM 包来自原有 ISO 镜像

   ```shell script
   $ sudo isocut -t /home/temp /home/isocut_iso/openEuler-22.03-LTS-SP4-aarch64-dvd.iso /home/result/new.iso
     Checking input ...
     Checking user ...
     Checking necessary tools ...
     Initing workspace ...
     Copying basic part of iso image ...
     Downloading rpms ...
     Finish create yum conf
     finished
     Regenerating repodata ...
     Checking rpm deps ...
     Getting the description of iso image ...
     Remaking iso ...
     Adding checksum for iso ...
     Adding sha256sum for iso ...
     ISO cutout succeeded, enjoy your new image "/home/result/new.iso"
     isocut.lock unlocked ...
   ```

   回显如上，说明新镜像 new.iso 定制成功。

   **场景二**：新镜像的 RPM 包除来自原有 ISO 镜像，还包含来自 /home/rpms 的额外软件包

   ```shell
   sudo isocut -t /home/temp -r /home/rpms /home/isocut_iso/openEuler-22.03-LTS-SP4-aarch64-dvd.iso /home/result/new.iso
   ```

   **场景三**：使用 kickstart 文件实现自动化安装，需要修改 /etc/isocut/anaconda-ks.cfg 文件

   ```shell
   sudo isocut -t /home/temp -k /etc/isocut/anaconda-ks.cfg /home/isocut_iso/openEuler-22.03-LTS-SP4-aarch64-dvd.iso /home/result/new.iso
   ```

### cut packages 功能介绍

基于 openEuler 发布的标准 ISO 镜像进行最小系统定制裁剪，定制安装过程中支持按需裁剪 RPM 包。

通过指定cut\_packages参数，用户可以自行选择是否需要裁剪RPM包。

#### 操作指导

**场景一**：用户不裁剪RPM包

```shell script
 $ sudo isocut -t /opt/tlriso/tmp -p openEuler -c no -v 22.03-LTS-SP4 openEuler-22.03-LTS-SP4-x86_64-dvd.iso openEuler-22.03-LTS-SP4-x86_64-dvd_new.iso
   Checking input ...
   Checking user ...
   Checking necessary tools ...
   Initing workspace ...
   Copying basic part of iso image ...
   Getting the description of iso image ...
   Downloading rpms ...
   Finish create yum conf
   finished
   Regenerating repodata ...
   Checking rpm deps ...
   Skip checking rpm deps!!
   Replacing install background pictures ...
   Updating EFI config file ...
   Updating legacy config file ...
   Updating treeinfo file ...
   Customizing kickstart file ...
   Remaking iso ...
   Adding checksum for iso ...
   Adding sha256sum for iso ...
   ISO cutout succeeded, enjoy your new image "openEuler-22.03-LTS-SP4-x86_64-dvd_new.iso"
   isocut.lock unlocked ...
```

回显如上，说明新镜像 new.iso 定制成功。

**场景二**：用户裁剪rpm包

````
```shell
sudo isocut -t /opt/tlriso/tmp -p openEuler -c yes -v 22.03-LTS-SP4 openEuler-22.03-LTS-SP4-x86_64-dvd.iso openEuler-22.03-LTS-SP4-x86_64-dvd_new.iso
```
````

**场景三**：默认裁剪rpm包，参数取值为空

````
```shell
sudo isocut -t /opt/tlriso/tmp -p openEuler -v 22.03-LTS-SP4 openEuler-22.03-LTS-SP4-x86_64-dvd.iso openEuler-22.03-LTS-SP4-x86_64-dvd_new.iso
```
````

## FAQ

### 默认 rpm 包列表安装系统失败

#### 背景描述

用户使用 isocut 裁剪镜像时通过配置文件 /etc/isocut/rpmlist 指定需要安装的软件包。

由于不同版本会有软件包减少，可能导致裁剪镜像时出现缺包等问题。
因此 /etc/isocut/rpmlist 中默认只包含 kernel 软件包。
保证默认配置裁剪镜像必定成功。

#### 问题描述

使用默认配置裁剪出来的 iso 镜像，能够裁剪成功，但是安装可能失败。

安装报错缺包，报错截图如下：

![](./figures/lack_pack.png)

#### 原因分析

使用默认配置的 RPM 软件包列表，裁剪的 iso 镜像在安装时缺少必要的 RPM 包。
缺少的包如报错的图示，并且在不同版本中，缺少的 RPM 包也可能是不同的，以安装时实际报错为准。

#### 解决方案

增加缺少的包

````
1. 根据报错的提示整理缺少的 RPM 包列表
2. 将上述 RPM 包列表添加到配置文件 /etc/isocut/rpmlist 中
3. 再次裁剪安装 iso 镜像

以问题描述中的缺包情况为例，修改 rpmlist 配置文件如下：

```shell
$ cat /etc/isocut/rpmlist
kernel.aarch64
lvm2.aarch64
chrony.aarch64
authselect.aarch64
shim.aarch64
efibootmgr.aarch64
grub2-efi-aa64.aarch64
dosfstools.aarch64
```
````

---

---
url: /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/isulad_k8s/overview.md
---
# iSulad + Kubernetes Cluster Deployment Guide

This document outlines the process of deploying a Kubernetes cluster with kubeadm on the openEuler OS, configuring a Kubernetes + iSulad environment, and setting up gitlab-runner. It serves as a comprehensive guide for creating a native openEuler development environment cluster.

The guide addresses two primary scenarios:

**Scenario 1**: A complete walkthrough for establishing a native openEuler development CI/CD pipeline from scratch using gitlab-ci.
**Scenario 2**: Instructions for integrating an existing native openEuler development execution machine cluster into gitlab-ci.

For scenario 1, the following steps are required:

1. Set up the Kubernetes + iSulad environment.
2. Deploy GitLab.
3. Install and test gitlab-runner.

For scenario 2, where a gitlab-ci platform is already available, the process involves:

1. Configure the Kubernetes + iSulad environment.
2. Install and test gitlab-runner.

> \[!NOTE] Note
>
> All operations described in this document must be executed with root privileges.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/overview.md
---
# iSulad Container Engine

Compared with Docker, iSulad is a new container solution with a unified architecture design to meet different requirements in the CT and IT fields. Lightweight containers are implemented using C/C++. They are smart, fast, and not restricted by hardware and architecture. With less noise floor overhead, the containers can be widely used.

[Figure 1](#en-us_topic_0182207099_fig10763114141217)  shows the unified container architecture.

**Figure  1**  Unified container architecture

![](./figures/en-us_image_0183048952.png)

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/cluster_deployment/isulad_k8s/isulad_k8s_environment_deploy.md
---
# iSulad+k8s环境部署

## 准备集群服务器

需准备至少3台openEuler机器，建议在openEuler-22.03及以上版本运行。下表为示例搭建机器信息，仅供参考。

| 主机名 | IP          | 系统版本                | 角色      | 组件       |
|-------|-------------|------------------------|----------|-----------|
| lab1  | 197.xxx.xxx.xxx | openEuler 22.03 LTS SP4 | 控制节点  | iSulad/k8s |
| lab2  | 197.xxx.xxx.xxx | openEuler 22.03 LTS SP4 | 工作节点1 | iSulad/k8s |
| lab3  | 197.xxx.xxx.xxx | openEuler 22.03 LTS SP4 | 工作节点2 | iSulad/k8s |

## 镜像/软件信息

安装过程中需要用到的软件及镜像名称如下表，版本号为示例安装时用到的版本，仅供参考。

| 软件                               | 版本     |
|------------------------------------|----------|
| iSulad                             | 2.0.17-2 |
| kubernetes-client                  | 1.20.2-9 |
| kubernetes-kubeadm                 | 1.20.2-9 |
| kubernetes-kubelet                 | 1.20.2-9 |

| 镜像                               | 版本     |
|------------------------------------|----------|
| k8s.gcr.io/kube-proxy              | v1.20.2  |
| k8s.gcr.io/kube-apiserver          | v1.20.2  |
| k8s.gcr.io/kube-controller-manager | v1.20.2  |
| k8s.gcr.io/kube-scheduler          | v1.20.2  |
| k8s.gcr.io/etcd                    | 3.4.13-0 |
| k8s.gcr.io/coredns                 | 1.7.0    |
| k8s.gcr.io/pause                   | 3.2      |
| calico/node                        | v3.14.2  |
| calico/pod2daemon-flexvol          | v3.14.2  |
| calico/cni                         | v3.14.2  |
| calico/kube-controllers            | v3.14.2  |

如果在无外网环境中搭建，可以从以下链接提前下载对应版本的软件包、相关依赖软件包及镜像：

1. 软件包下载地址：<https://dl-cdn.openeuler.openatom.cn>
2. 镜像下载地址：<https://developer.aliyun.com/mirror/>

## 修改host文件

1. 修改主机名，以其中一台机器为例。

   ```shell
   # hostnamectl set-hostname lab1
   # sudo -i
   ```

2. 配置主机名解析，编辑三台服务器的/etc/hosts文件。

   ```shell
   # vim /etc/hosts
   ```

3. 在hosts文件中添加以下内容（IP+主机名）。

   ```text
   197.xxx.xxx.xxx lab1
   197.xxx.xxx.xxx lab2
   197.xxx.xxx.xxx lab3
   ```

## 环境准备

1. 关闭防火墙。

   ```shell
   # systemctl stop firewalld
   # systemctl disable firewalld
   ```

2. 禁用selinux。

   ```shell
   # setenforce 0
   ```

3. 关闭系统swap。

   ```shell
   # swapoff -a
   # sed -ri 's/.*swap.*/#&/' /etc/fstab
   ```

4. 网络配置，开启相应的转发机制。

   ```shell
   # cat > /etc/sysctl.d/kubernetes.conf <<EOF
   net.bridge.bridge-nf-call-iptables = 1
   net.ipv4.ip_forward = 1
   net.bridge.bridge-nf-call-ip6tables = 1
   vm.swappiness=0
   EOF
   ```

5. 生效规则。

   ```shell
   # modprobe overlay
   # modprobe br_netfilter
   # sysctl -p /etc/sysctl.d/kubernetes.conf
   ```

6. 配置开机启动脚本。

   ```shell
   # vim /etc/init.d/k8s.sh
   ```

   * 在k8s.sh脚本中添加以下内容。

     ```shell
     #!/bin/sh
     modprobe br_netfilter
     sysctl -w net.bridge.bridge-nf-call-ip6tables = 1
     sysctl -w net.bridge.bridge-nf-call-iptables = 1
     ```

   * 改变脚本权限。

     ```shell
     # chmod +x /etc/init.d/k8s.sh
     ```

7. 添加配置文件。

   配置文件br\_netfilter.service应该放在systemd的service目录，一般在/usr/lib/systemd/system。

   ```shell
   # vim br_netfilter.service

   [Unit]
   Description=To enable the core module br_netfilter when reboot
   After=default.target
   [Service]
   ExecStart=/etc/init.d/k8s.sh //可以自定义
   [Install]
   WantedBy=default.target
   ```

   * 添加完成后启动服务。

     ```shell
     # systemctl daemon-reload
     # systemctl enable br_netfilter.service
     ```

8. 配置sysctl。

   ```shell
   # sed -i "s/net.ipv4.ip_forward=0/net.ipv4.ip_forward=1/g" /etc/sysctl.conf
   # sed -i 12a\vm.swappiness=0 /etc/sysctl.conf
   ```

## 安装kubeadm、kubectl、kubelet组件及iSulad

1. 使用yum install安装软件包。

   kubernetes相关的组件在EPOL中，使用yum安装前需要在yum源中配置EPOL源。

   ```shell
   # yum install -y kubernetes-kubeadm
   # yum install -y kubernetes-client
   # yum install -y kubernetes-kubelet
   # yum install -y iSulad
   ```

2. 设置kubelet开机启动。

   ```shell
   # systemctl enable kubelet
   ```

## 修改isulad配置

1. 进入/etc/isulad，修改daemon.json文件。

   ```shell
   # vi /etc/isulad/daemon.json
   ```

2. 修改并添加配置文件中的内容。

   ```json
   {
      "group": "isula",
      "default-runtime": "runc",
      "graph": "/var/lib/isulad",
      "state": "/var/run/isulad",
      "engine": "lcr",
      "log-level": "ERROR",
      "pidfile": "/var/run/isulad.pid",
      "log-opts": {
         "log-file-mode": "0600",
         "log-path": "/var/lib/isulad",
         "max-file": "1",
         "max-size": "30KB"
      },
      "log-driver": "stdout",
      "container-log": {
         "driver": "json-file"
      },
      "hook-spec": "/etc/default/isulad/hooks/default.json",
      "start-timeout": "2m",
      "storage-driver": "overlay2",
      "storage-opts": [
         "overlay2.override_kernel_check=true"
      ],
      "registry-mirrors": [
                  "docker.io"
      ],
      "insecure-registries": [
                  "k8s.gcr.io",
                  "quay.io",
                  "oci.inhuawei.com",
                  "rnd-dockerhub.huawei.com",
                  "registry.aliyuncs.com",
                  "【本地私有镜像仓IP】"
      ],
      "pod-sandbox-image": "k8s.gcr.io/pause:3.2",
      "native.umask": "normal",
      "network-plugin": "cni",
      "cni-bin-dir": "/opt/cni/bin",
      "cni-conf-dir": "/etc/cni/net.d",
      "image-layer-check": false,
      "use-decrypted-key": true,
      "insecure-skip-verify-enforce": false,
      "cri-runtimes": {
         "kata": "io.containerd.kata.v2"
      }
   }
   ```

3. 重启isulad服务。

   ```shell
   # systemctl restart isulad
   ```

## 加载isulad镜像

1. 查看需要的系统镜像。

   ```shell
   # kubeadm config images list
   ```

   注意对应的版本号，输出结果如图所示。
   ![](figures/1.查看需要镜像.png)

2. 使用isula拉取所需镜像。

   > \[!NOTE]说明
   >
   > 以下所下载的镜像版本均为示例，具体版本号以上条命令返回结果为准，下同。

   ```shell
   # isula pull k8smx/kube-apiserver:v1.20.15
   # isula pull k8smx/kube-controller-manager:v1.20.15
   # isula pull k8smx/kube-scheduler:v1.20.15
   # isula pull k8smx/kube-proxy:v1.20.15
   # isula pull k8smx/pause:3.2
   # isula pull k8smx/coredns:1.7.0
   # isula pull k8smx/etcd:3.4.13-0
   ```

3. 修改已下载的镜像标签。

   ```shell
   # isula tag k8smx/kube-apiserver:v1.20.15 k8s.gcr.io/kube-apiserver:v1.20.15
   # isula tag k8smx/kube-controller-manager:v1.20.15 k8s.gcr.io/kube-controller-manager:v1.20.15
   # isula tag k8smx/kube-scheduler:v1.20.15 k8s.gcr.io/kube-scheduler:v1.20.15
   # isula tag k8smx/kube-proxy:v1.20.15 k8s.gcr.io/kube-proxy:v1.20.15
   # isula tag k8smx/pause:3.2 k8s.gcr.io/pause:3.2
   # isula tag k8smx/coredns:1.7.0 k8s.gcr.io/coredns:1.7.0
   # isula tag k8smx/etcd:3.4.13-0 k8s.gcr.io/etcd:3.4.13-0
   ```

4. 删除旧镜像。

   ```shell
   # isula rmi k8smx/kube-apiserver:v1.20.15
   # isula rmi k8smx/kube-controller-manager:v1.20.15
   # isula rmi k8smx/kube-scheduler:v1.20.15
   # isula rmi k8smx/kube-proxy:v1.20.15
   # isula rmi k8smx/pause:3.2
   # isula rmi k8smx/coredns:1.7.0
   # isula rmi k8smx/etcd:3.4.13-0
   ```

5. 查看已拉取的镜像。

   ```shell
   # isula images
   ```

## 安装crictl工具

```shell
# yum install -y cri-tools
```

## 初始化master节点

执行如下命令初始化master节点：

```shell
# kubeadm init --kubernetes-version v1.20.2 --cri-socket=/var/run/isulad.sock --pod-network-cidr=[指定pod分配IP段]

//以上参数的解释
kubernetes-version 为当前安装的版本
cri-socket 指定引擎为isulad
pod-network-cidr 指定pod分配的ip段
```

根据系统提示输入如下命令：

```shell
# mkdir -p $HOME/.kube
# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

初始化成功后，复制最后两行内容，在node节点上执行刚刚复制的命令，将节点加入master集群，如未记录上述命令可通过如下命令生成：

```shell
# kubeadm token create --print-join-command
```

## node节点添加进集群

粘贴master上初始化生成的kubeadm join ...命令，并在discovery前添加--cri-socket=/var/run/isulad.sock。

## 安装calico网络插件

1. 拉取calico镜像。

   需要在master节点配置calico网络插件，同时需要在每个节点中提前拉取需要版本的镜像。

   ```shell
   isula pull calico/node:v3.14.2
   isula pull calico/cni:v3.14.2
   isula pull calico/kube-controllers:v3.14.2
   isula pull calico/pod2daemon-flexvol:v3.14.2
   ```

2. 在master节点上获取配置文件。

   ```shell
   wget https://docs.projectcalico.org/v3.14/manifests/calico.yaml
   ```

3. 修改后创建pod。

   ```shell
   # kubectl apply -f calico.yaml
   ```

   * 如需删除使用如下命令：

     ```shell
     # kubectl delete -f calico.yaml
     ```

4. 查看pod信息。

   ```shell
   # kubectl get pod -A -o wide
   ```

## 查看master节点node信息

使用如下命令可查看节点的详细信息：

```shell
# kubectl get nodes -o wide
```

若需要重置node节点，可使用如下命令：

```shell
# kubeadm reset
```

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/cluster_deployment/isulad_k8s/overview.md
---
# iSulad+k8s集群部署指南

本文档介绍在 openEuler 操作系统上，通过 kubeadm 部署 K8S 集群，搭建 K8S+iSulad 的环境，并在该环境上部署 gitlab-runner，指导部署欧拉原生开发环境集群。

本文档主要包括以下两个场景内容：

场景一： 基于 gitlab-ci 从 “0” 开始构建欧拉原生开发CICD部署指导。\
场景二： 欧拉原生开发执行机集群被 gitlab-ci 纳管指导。

场景一中需要额外部署gitlab，步骤操作顺序为：

1. K8s+iSulad 环境部署。
2. gitlab 部署。
3. gitlab runner 部署和测试。

场景二中已有 gitlab-ci 平台，无需额外部署，步骤操作顺序为：

1. K8s+iSulad 环境部署。
2. gitlab runner 部署和测试。

> \[!NOTE]说明
>
> 本文档所有操作均使用root权限执行。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/overview.md
---
# iSula容器引擎

iSula通用容器引擎相比Docker，是一种新的容器解决方案，提供统一的架构设计来满足CT和IT领域的不同需求。相比Golang编写的Docker，轻量级容器使用C/C++实现，具有轻、灵、巧、快的特点，不受硬件规格和架构的限制，底噪开销更小，可应用领域更为广泛。

容器统一架构如[图1](#zh-cn_topic_0182207099_fig10763114141217)所示。

**图 1**  容器统一架构

![](./figures/zh-cn_image_0183048952.png)

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/interconnecting_isula_shim_v2_with_stratovirt.md
---
# iSula对接shim v2安全容器

## 概述

shim v2 是新一代 shim 架构方案，相比于 shim v1, 具有调用链更短、架构清晰的优势，在多业务容器场景，具备明显的低内存开销优势。iSula 运行安全容器可以通过 isulad-shim 或者 containerd-shim-kata-v2 来实现，其中 isulad-shim 组件是 shim v1 方案的具体实现，containerd-shim-kata-v2 组件是 shim v2 方案在安全容器场景的一种具体实现，本文介绍 iSula 与 containerd-shim-kata-v2 的对接。

## 对接 containerd-shim-kata-v2

### **前提条件**

iSula 对接 containerd-shim-kata-v2 前，需要满足如下前提：

* 已安装 iSulad和 kata-containers
* StratoVirt 仅支持 devicemapper 存储驱动，因此需要配置 devicemapper 环境并确保 iSulad 使用的 devicemapper 存储驱动正常工作

### 环境准备

此处给出安装 iSulad 和 kata-containers 并进行相应配置的参考方法。

#### 安装依赖软件

按照所使用的OS版本自行配置相应的 yum 源，使用 root 权限安装 iSulad和kata-containers ：

```shell
# yum install iSulad
# yum install kata-containers
```

#### 制作并配置存储 Storage

需要用户准备一个磁盘， 如 /dev/sdx ， 该磁盘会被格式化，本章使用块设备 /dev/sda 进行演示。

一、创建devicemapper

1. 创建 PV

   ```shell
   $ pvcreate /dev/sda
     Physical volume "/dev/loop0" successfully created.
   ```

2. 创建 VG

   ```shell
   $ vgcreate isula /dev/sda
     Volume group "isula" successfully created
   ```

3. 创建 thinpool 以及 thinpoolmeta 逻辑卷

   ```shell
   $ lvcreate --wipesignatures y -n thinpool isula -l 95%VG
     Logical volume "thinpool" created.

   $ lvcreate --wipesignatures y -n thinpoolmeta isula -l 1%VG
     Logical volume "thinpoolmeta" created.
   ```

4. 将上面创建的逻辑卷转换为 thinpool

   ```shell
   $ lvconvert -y --zero n -c 64K \
   --thinpool isula/thinpool \
   --poolmetadata isula/thinpoolmeta
     Thin pool volume with chunk size 512.00 KiB can address at most 126.50 TiB of data.
     WARNING: Converting isula/thinpool and isula/thinpoolmeta to thin pool's data and metadata volumes with metadata wiping.
     THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
     Converted isula/thinpool and isula/thinpoolmeta to thin pool.
   ```

5. 设置 lvm thinpool 自动扩展功能

   ```shell
   $ touch /etc/lvm/profile/isula-thinpool.profile
   $ cat << EOF > /etc/lvm/profile/isula-thinpool.profile
   activation {
     thin_pool_autoextend_threshold=80
     thin_pool_autoextend_percent=20
   }
   EOF
   $ lvchange --metadataprofile isula-thinpool isula/thinpool
   Logical volume isula/thinpool changed.
   ```

二、修改 iSulad 存储驱动类型并设置默认runtime

更改配置文件 /etc/isulad/daemon.json,  将 default-runtime 设置为 io.containerd.kata.v2 , 将默认存储驱动类型 overlay 配置成 devicemapper，修改后如下所示：

```json
 {
    "default-runtime": "io.containerd.kata.v2",
    "storage-driver": "devicemapper",
    "storage-opts": [
        "dm.thinpooldev=/dev/mapper/isula-thinpool",
        "dm.fs=ext4",
        "dm.min_free_space=10%"
    ],
}
```

三、使能配置

1. 重启 isulad使得配置生效 ：

   ```shell
   # systemctl daemon-reload
   # systemctl restart isulad
   ```

2. 确认 iSula 存储驱动是否配置成功：

   ```shell
   # isula info
   ```

   若回显有如下信息，说明配置成功。

   ```sh
   Storage Driver: devicemapper
   ```

### 对接指导

本章给出 iSula 对接 containerd-shim-kata-v2 的操作指导。

containerd-shim-kata-v2 默认使用 QEMU 虚拟化组件，本章分别介绍使用 QEMU 和 StratoVirt 两种虚拟化组件时的配置方法。

#### 使用 QEMU

containerd-shim-kata-v2 使用的虚拟化组件为 QEMU 时，iSula 对接 containerd-shim-kata-v2 的操作如下：

1. 修改 kata 配置文件，路径为 /usr/share/defaults/kata-containers/configuration.toml

   sandbox\_cgroup\_with\_emulator 需要设置为 false， 目前 shimv2 不支该改功能， 其他参数与 shim v1 中 kata 配置参数保持一致或者保持缺省值。

   ```sh
   sandbox_cgroup_with_emulator = false
   ```

2. 使用 busybox 镜像运行安全容器并检查使用的 runtime 为 io.containerd.kata.v2

   ```bash
   $ id=`isula run -tid busybox /bin/sh`
   $ isula inspect -f '{{ json .HostConfig.Runtime }}' $id
   "io.containerd.kata.v2"
   ```

3. 确认 qemu 虚拟机进程被拉起，说明 qemu 和 shim v2 安全容器的对接成功

   ```bash
   $ ps -ef | grep qemu
   ```

#### 使用 StratoVirt

containerd-shim-kata-v2 使用的虚拟化组件为 StratoVirt 时，iSula 对接 containerd-shim-kata-v2 的操作如下：

1. 在任一目录（例如 /home 目录）新建脚本文件 stratovirt.sh 并使用 root 权限给文件添加执行权限：

   ```shell
   # touch /home/stratovirt.sh
   # chmod +x /home/stratovirt.sh
   ```

   stratovirt.sh 内容如下，用于指定 StratoVirt 路径：

   ```shell
   #!/bin/bash
   export STRATOVIRT_LOG_LEVEL=info  # set log level which includes trace, debug, info, warn and error.
   /usr/bin/stratovirt $@
   ```

2. 修改 kata 配置文件 ，将安全容器的 hypervisor 类型配置为 stratovirt，kernel 配置 StratoVirt 的 kernel 镜像绝对路径，initrd 配置为 kata-containers 的 initrd 镜像文件（使用 yum 安装 kata-containers 时，默认会下载这个文件并存放在 /var/lib/kata/ 目录），StratoVirt 仅支持 devicemapper 存储模式，需提前准备好环境并将 iSulad 设置为 devicemapper 模式。

   配置参考如下：

   ```shell
   [hypervisor.stratovirt]
   path = "/home/stratovirt.sh"
   kernel = "/var/lib/kata/vmlinux.bin"
   initrd = "/var/lib/kata/kata-containers-initrd.img"
   block_device_driver = "virtio-mmio"
   use_vsock = true
   enable_netmon = true
   internetworking_model="tcfilter"
   sandbox_cgroup_with_emulator = false
   disable_new_netns = false
   disable_block_device_use = false
   disable_vhost_net = true
   ```

   StratoVirt 中使用 vsock 功能， 需要开启 vhost\_vsock 内核模块并确认是否开启成功

   ```bash
   $ modprobe vhost_vsock
   $ lsmod |grep vhost_vsock
   ```

   下载对应版本和架构的 kernel 并放到 /var/lib/kata/ 路径下, [openeuler repo](https://repo.openeuler.org/)：

   ```bash
   $ cd /var/lib/kata
   $ wget https://dl-cdn.openeuler.openatom.cn/openEuler-24.03-LTS-SP1/stratovirt_img/x86_64/vmlinux.bin
   ```

3. 使用 busybox 镜像运行安全容器并检查使用的 runtime 为 io.containerd.kata.v2

   ```bash
   $ id=`isula run -tid busybox sh`
   $ isula inspect -f '{{ json .HostConfig.Runtime }}' $id
   "io.containerd.kata.v2"
   ```

4. 确认 stratovirt 虚拟机进程被拉起，说明 StratoVirt 和 shim v2 安全容器的对接成功

   ```bash
   $ ps -ef | grep stratovirt
   ```

---

---
url: /en/docs/22.03_LTS_SP4/edge_computing/k3s/k3s_deployment_guide.md
---
# K3s Deployment Guide

## Introduction to K3s

K3s is a lightweight Kubernetes distribution that is optimized for edge computing and IoT scenarios. K3s provides the following enhanced features:

* Packaged as a single binary file.
* Uses an SQLite3-based lightweight storage backend as the default storage mechanism and supports etcd3, MySQL, and PostgreSQL.
* Wrapped in a simple launcher that handles complex TLS and options.
* Secure by default with reasonable defaults for lightweight environments.
* Batteries included, providing simple but powerful functions such as local storage providers, service load balancers, Helm controllers, and Traefik Ingress controllers.
* Encapsulates all operations of the Kubernetes control plane in a single binary file and process, capable of automating and managing complex cluster operations including certificate distribution.
* Minimizes external dependencies and requires only kernel and cgroup mounting.

## Application Scenarios

K3s is applicable to the following scenarios:

* Edge computing
* IoT
* Continuous integration (CI)
* Development
* Arm
* Embedded Kubernetes

The resources required for running the K3s are relatively small. Therefore, K3s is also applicable to development and testing scenarios. In these scenarios, K3s facilitates function verification and problem reproduction by shortening cluster startup time and reducing resources consumed by the cluster.

## Deploying K3s

### Step 1 Making Preparations

* Ensure that the host names of the server node and agent node are different.

You can run the `hostnamectl set-hostname "host name"` command to change the host name.

```shell
hostnamectl set-hostname agent
```

* Install K3s on each node using Yum.

  The K3s official website provides binary executable files of different architectures and the **install.sh** script for offline installation. The openEuler community migrates the compile process of the binary file to the community and releases the compiled RPM package. You can run the `yum` command to download and install K3s.

  ```shell
  yum install k3s
  ```

### Step 2 Deploying the Server Node

To install K3s on a single server, run the following command on the server node:

```shell
INSTALL_K3S_SKIP_DOWNLOAD=true k3s-install.sh
```

![1661825352724](./figures/server-install.png)

### Step 3 Checking Server Deployment

```shell
kubectl get nodes
```

### Step 4 Deploying the Agent Node

Query the token value of the server node. The token is stored in the **/var/lib/rancher/k3s/server/node-token** file on the server node.

> **Note:**
>
> Only the second half of the token is used.

![1661825538264](./figures/token.png)

Run the following command on each agent node to open required ports and add agents:

```shell
firewall-cmd --add-port=6443/tcp --zone=public --permanent
firewall-cmd --add-port=8472/udp --zone=public --permanent
firewall-cmd --reload
INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken k3s-install.sh
```

> **Note:**
>
> Replace **myserver** with the IP address of the server or a valid DNS, and replace **mynodetoken** with the token of the server node.

![1661829392357](./figures/agent-install.png)

### Step 5 Checking Agent Deployment

After the installation is complete, run `kubectl get nodes` on the server node to check whether the agent node is successfully registered.

A basic K3s cluster is set up.

## Deploying the First Nginx Service on K3s

### Step 1 Creating a Deployment

In Kubernetes, a deployment is used to deploy applications. Create and edit the **deployment.yml** file as follows:

```yml
apiVersion: apps/v1   
kind: Deployment    
metadata:            
  name: nginx-deployment    
  labels:        
    app: nginx    
spec:            
  replicas: 1    
  selector:       
    matchLabels: 
      app: nginx
  template:        
    metadata:    
      labels:    
        app: nginx
    spec:        
      containers:    
      - name: nginx
        image: nginx:alpine    
        ports:
          - containerPort: 80 

```

Configure **deployment.yml** and run the `kubectl apply` command to create a deployment.

```shell
[root@k3s-server home]# kubectl apply -f deployment.yml
```

![apply-deployment](figures/apply-deployment.png)

After the deployment is created, check whether the pods are in the running state.

```shell
[root@k3s-server home]# kubectl get pods
```

![get-pods](figures/get-pods.png)

### Step 2 Creating a Service

After the deployment is created, the Nginx service is only deployed. You need to enable the Nginx service to provide services externally. Create and edit the **service.yml** file as follows:

```yml
apiVersion: v1
kind: Service  
metadata:
  name: nginx-service  
spec:
  selector:    
    app: nginx 
  ports:  
  - protocol: TCP  
    port: 80 
    targetPort: 80 
    nodePort: 30080 
  type: NodePort

```

Configure the **service.yml** file and run the `kubectl apply` command to create a service.

```shell
[root@k3s-server home]# kubectl apply -f service.yml
```

![apply-service](figures/apply-service.png)

### Step 3 Viewing Service Information

```shell
[root@k3s-server home]# kubectl describe service nginx-service
```

![describe-service](figures/describe-service.png)

### Step 4 Accessing the Service

Run the `curl` command on the intranet to access the server. The command output shows that the Nginx service has been enabled to provide services externally.

![curl-nginx](figures/curl-nginx.png)

An Nginx service is running in the cluster.

## More

For details about how to use K3s, visit the K3s official website at <https://rancher.com/docs/k3s/latest/en>.

---

---
url: /zh/docs/22.03_LTS_SP4/edge_computing/k3s/k3s_deployment_guide.md
---
# K3s部署指南

## 什么是K3s

K3s 是一个轻量级的 Kubernetes 发行版，它针对边缘计算、物联网等场景进行了高度优化。K3s 有以下增强功能：

* 打包为单个二进制文件。
* 使用基于 sqlite3 的轻量级存储后端作为默认存储机制。同时支持使用 etcd3、MySQL 和 PostgreSQL 作为存储机制。
* 封装在简单的启动程序中，通过该启动程序处理复杂的 TLS 和选项。
* 默认情况下是安全的，对轻量级环境有合理的默认值。
* 添加了简单但功能强大的batteries-included功能，例如：本地存储提供程序，服务负载均衡器，Helm controller 和 Traefik Ingress controller。
* 所有 Kubernetes control-plane 组件的操作都封装在单个二进制文件和进程中，使 K3s 具有自动化和管理包括证书分发在内的复杂集群操作的能力。
* 最大程度减轻了外部依赖性，K3s 仅需要 kernel 和 cgroup 挂载。

## 适用场景

K3s 适用于以下场景：

* 边缘计算-Edge
* 物联网-IoT
* CI
* Development
* ARM
* 嵌入 K8s

由于运行 K3s 所需的资源相对较少，所以 K3s 也适用于开发和测试场景。在这些场景中，如果开发或测试人员需要对某些功能进行验证，或对某些问题进行重现，那么使用 K3s 不仅能够缩短启动集群的时间，还能够减少集群需要消耗的资源。

## 部署K3s

**步骤1：准备工作**

* 确保server节点及agent节点主机名不一致。

  可以通过 hostnamectl set-hostname “主机名” 进行主机名的修改。

  ```shell
  [root@agent ~]# hostnamectl set-hostname agent
  ```

* 在各节点 yum 安装 K3s。

  K3s官网采用下载对应架构二进制可执行文件的格式，通过install.sh脚本进行离线安装，openEuler社区将该二进制文件的编译过程移植到社区中，并编译出RPM包。此处可通过yum命令直接进行下载安装。

  ```shell
   [root@agent ~]# yum install k3s

  ```

**步骤2：部署server节点**

如需在单个服务器上安装 K3s，可以在 server 节点上执行如下操作：

```shell
INSTALL_K3S_SKIP_DOWNLOAD=true k3s-install.sh
```

![1661825352724](./figures/server-install.png)

**步骤3：检查server部署情况**

```shell
[root@openEuler ~]# kubectl get nodes
```

**步骤4：部署agent节点**

首先查询server节点的token值，该token可在server节点的/var/lib/rancher/k3s/server/node-token查到。

> **注意**:
>
> 后续我们只用到该token的后半部分。

![1661825538264](./figures/token.png)

选择添加其他 agent，请在每个 agent 节点上执行以下操作，执行之前放行6443/tcp和8472/udp端口。

```shell
firewall-cmd --add-port=6443/tcp --zone=public --permanent
firewall-cmd --add-port=8472/udp --zone=public --permanent
firewall-cmd --reload
INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken k3s-install.sh
```

> **注意**:
>
> 将 myserver 替换为 server 的 IP 或有效的 FQDN，并将 mynodetoken 替换 server 节点的 token。

![1661829392357](./figures/agent-install.png)

**步骤5：检查agent节点是否部署成功**

安装完毕后，回到 **server** 节点，执行 `kubectl get nodes`，可以查看agent节点是否注册成功。

至此，一个基础的k3s集群搭建完成。

## 在k3s上部署第一个nginx服务

**步骤1：创建Deployment**

在 kubernetes 中，deployment 用来部署应用，创建编辑deployment.yml文件如下：

```yaml
apiVersion: apps/v1   
kind: Deployment    
metadata:            
  name: nginx-deployment    
  labels:        
    app: nginx    
spec:            
  replicas: 1    
  selector:       
    matchLabels: 
      app: nginx
  template:        
    metadata:    
      labels:    
        app: nginx
    spec:        
      containers:    
      - name: nginx
        image: nginx:alpine    
        ports:
          - containerPort: 80 

```

按照需要配置deployment，并使用kubectl apply创建deployment。

```shell
[root@k3s-server home]# kubectl apply -f deployment.yml
```

![输入图片说明](figures/apply-deployment.png)

deployment成功创建后，查看pods状态为running。

```shell
[root@k3s-server home]# kubectl get pods
```

![输入图片说明](figures/get-pods.png)

**步骤2：创建service**

deployment创建完成后，nginx服务仅仅只是部署，故还需要让它对外公开自己的服务。创建编辑service.yml如下：

```yaml
apiVersion: v1
kind: Service  
metadata:
  name: nginx-service  
spec:
  selector:    
    app: nginx 
  ports:  
  - protocol: TCP  
    port: 80 
    targetPort: 80 
    nodePort: 30080 
  type: NodePort

```

配置 service.yml，同时使用 kubectl apply创建服务。

```shell
[root@k3s-server home]# kubectl apply -f service.yml
```

![输入图片说明](figures/apply-service.png)

**步骤3：查看服务信息**

```shell
[root@k3s-server home]# kubectl describe service nginx-service
```

![输入图片说明](figures/describe-service.png)

**步骤4：访问服务**

网内使用curl命令访问服务器，从结果可看出 nginx 服务已经对外开启。

![输入图片说明](figures/curl-nginx.png)

至此，一个 nginx 服务已在集群中运行。

## 更多用法

K3s的更多用法请参考：

* [K3s官网](https://rancher.com/docs/k3s/latest/en/)
* [K3s中文文档](https://docs.rancher.cn/k3s/)

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/nestos/nestos/usage.md
---
# K8S+iSulad 搭建

**除非特别说明，以下步骤在master节点和node节点均需执行**，本教程以master为例。

## 开始之前

需准备如下内容：

* NestOS-22.03-date.x86\_64.iso。
* 一台主机用作master，一台主机用作node。

## 组件下载

编辑源文件，添加k8s的阿里云源：

```shell
vi /etc/yum.repos.d/openEuler.repo
```

添加如下内容：

```script
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
```

下载k8s组件以及同步系统时间所用组件：

```shell
rpm-ostree install kubelet kubeadm kubectl ntp ntpdate wget
```

重启生效。

```shell
systemctl reboot
```

选择最新的版本分支进入系统。

## 配置环境

### 修改主机名，以master为例

```shell
hostnamectl set-hostname k8s-master
sudo -i
```

编辑/etc/hosts

```shell
vi /etc/hosts
```

添加如下内容，ip为主机ip

```script
192.168.237.133 k8s-master
192.168.237.135 k8s-node01
```

### 同步系统时间

```shell
ntpdate time.windows.com
systemctl enable ntpd
```

NestOS默认无swap分区，默认关闭防火墙。
关闭selinux如下：

```shell
vi /etc/sysconfig/selinux
修改为SELINUX=disabled
```

### 网络配置，开启相应的转发机制

创建配置文件。

```shell
vi /etc/sysctl.d/k8s.conf
```

添加如下内容：

```script
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
```

使配置生效：

```shell
modprobe br_netfilter
sysctl -p /etc/sysctl.d/k8s.conf
```

## 配置iSula

查看k8s需要的系统镜像，需注意pause的版本号。

```shell
kubeadm config images list
```

修改daemon配置文件。

```shell
vi /etc/isulad/daemon.json
```

```script
##关于添加项的解释说明##
registry-mirrors 设置为"docker.io"
insecure-registries 设置为"rnd-dockerhub.huawei.com"
pod-sandbox-image 设置为"registry.aliyuncs.com/google_containers/pause:3.5"(使用阿里云，pause版本可在上一步查看)
network-plugin 设置为"cni"。
cni-bin-dir 设置为"/opt/cni/bin";
cni-conf-dir 设置为"/etc/cni/net.d"
```

修改后的完整文件如下

```script
{"group": "isula",
"default-runtime": "runc",
"graph": "/var/lib/isulad",
"state": "/var/run/isulad",
"engine": "lcr",
"log-level": "ERROR",
"pidfile": "/var/run/isulad.pid",
"log-opts": {
"log-file-mode": "0600",
"log-path": "/var/lib/isulad",
"max-file": "1",
"max-size": "30KB"
},
"log-driver": "stdout",
"container-log": {
"driver": "json-file"
},
"hook-spec": "/etc/default/isulad/hooks/default.json",
"start-timeout": "2m",
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"registry-mirrors": [
"docker.io"
],
"insecure-registries": [
"rnd-dockerhub.huawei.com"
],
"pod-sandbox-image": "registry.aliyuncs.com/google_containers/pause:3.5",
"native.umask": "secure",
"network-plugin": "cni",
"cni-bin-dir": "/opt/cni/bin",
"cni-conf-dir": "/etc/cni/net.d",
"image-layer-check": false,
"use-decrypted-key": true,
"insecure-skip-verify-enforce": false
}
```

启动相关服务。

```shell
systemctl restart isulad
systemctl enable isulad
systemctl enable kubelet
```

**以上为master，node节点均需执行的操作。**

## master节点初始化

**该部分仅master节点执行。**
初始化，在这一步会拉取镜像，需等待一小段时间。也可在该步骤之前手动拉取镜像。

```shell
kubeadm init --kubernetes-version=1.22.2 --apiserver-advertise-
address=192.168.237.133 --cri-socket=/var/run/isulad.sock --image-repository
registry.aliyuncs.com/google_containers --service-cidr=10.10.0.0/16 --pod-
network-cidr=10.122.0.0/16
```

```shell
##关于初始化参数的解释说明##
kubernetes-version 为当前安装的版本
apiserver-advertise-address 为master节点ip
cri-socket 指定引擎为isulad
image-repository 指定镜像源为阿里云，可省去修改tag的步骤
service-cidr 指定service分配的ip段
pod-network-cidr 指定pod分配的ip段
```

初始化成功后，复制最后两行内容方便后续node节点加入使用。

```shell
kubeadm join 192.168.237.133:6443 --token j7kufw.yl1gte0v9qgxjzjw --discovery-
token-ca-cert-hash
sha256:73d337f5edd79dd4db997d98d329bd98020b712f8d7833c33a85d8fe44d0a4f5 --cri-
socket=/var/run/isulad.sock
```

**注意**:添加--cri-socket=/var/run/isulad.sock以使用isulad为容器引擎。

查看下载好的镜像。

```shell
isula images
```

按照初始化成功所提示，配置集群。

```shell
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
source /etc/profile
```

查看健康状态。

```shell
kubectl get cs
```

可能存在controller-manager，scheduler状态为unhealthy的情况，解决方法如下:
编辑相关配置文件。

```shell
vi /etc/kubernetes/manifests/kube-controller-manager.yaml
```

```shell
注释如下内容:
--port=0
修改hostpath:
将所有/usr/libexec/kubernetes/kubelet-plugins/volume/exec 修改为/opt/libexec/...
```

```shell
vi /etc/kubernetes/manifests/kube-scheduler.yaml
注释如下内容:
--port=0
```

修改完成后，再次查看健康状态。

## 配置网络插件

仅需要在master节点配置网络插件，但是要在**所有节点**提前拉取镜像，拉取镜像指令如下：

```shell
isula pull calico/node:v3.19.3
isula pull calico/cni:v3.19.3
isula pull calico/kube-controllers:v3.19.3
isula pull calico/pod2daemon-flexvol:v3.19.3
```

**以下步骤仅在master节点执行**
获取配置文件。

```shell
wget https://docs.projectcalico.org/v3.19/manifests/calico.yaml
```

编辑calico.yaml 修改所有/usr/libexec/... 为 /opt/libexec/...
然后执行如下命令完成calico的安装：

```shell
kubectl apply -f calico.yaml
```

通过kubectl get pod -n kube-system查看calico是否安装成功。
通过kubectl get pod -n kube-system查看是否所有pod状态都为running。

## node节点加入集群

在node节点执行如下指令，将node节点加入集群。

```shell
kubeadm join 192.168.237.133:6443 --token j7kufw.yl1gte0v9qgxjzjw --discovery-
token-ca-cert-hash
sha256:73d337f5edd79dd4db997d98d329bd98020b712f8d7833c33a85d8fe44d0a4f5 --cri-
socket=/var/run/isulad.sock
```

通过kubectl get node 查看master，node节点状态是否为ready。

至此，k8s部署成功。

# rpm-ostree使用

## rpm-ostree安装软件包

安装wget

```shell
rpm-ostree install wget
```

重启系统，可在启动时通过键盘上下按键选择rpm包安装完成后或安装前的系统状态，其中【ostree:0】为安装之后的版本。

```shell
systemctl reboot
```

查看wget是否安装成功。

```shell
rpm -qa | grep wget
```

## rpm-ostree 手动更新升级 NestOS

在NestOS中执行命令可查看当前rpm-ostree状态，可看到当前版本号。

```shell
rpm-ostree status
```

执行检查命令查看是否有升级可用，发现存在新版本。

```shell
rpm-ostree upgrade --check
```

预览版本的差异

```shell
rpm-ostree upgrade --preview
```

在最新版本中，我们将nano包做了引入。
执行如下指令会下载最新的ostree和RPM数据，不需要进行部署。

```shell
rpm-ostree upgrade --download-only
```

重启NestOS，重启后可看到系统的新旧版本两个状态，选择最新版本的分支进入。

```shell
rpm-ostree upgrade --reboot
```

## 比较NestOS版本差别

检查状态，确认此时ostree有两个版本，分别为LTS.20210927.dev.0和LTS.20210928.dev.0。

```shell
rpm-ostree status
```

根据commit号比较2个ostree的差别。

```shell
rpm-ostree db diff 55eed9bfc5ec fe2408e34148
```

## 系统回滚

当一个系统更新完成，之前的NestOS部署仍然在磁盘上，如果更新导致了系统出现问题，可以使用之前的部署回滚系统。

### 临时回滚

要临时回滚到之前的OS部署，在系统启动过程中按住shift键，当引导加载菜单出现时，在菜单中选择相关的分支。

### 永久回滚

要永久回滚到之前的操作系统部署，需要登录到目标节点，运行rpm-ostree rollback，此操作将使用之前的系统部署作为默认部署，并重新启动到其中。
执行命令，回滚到更新前的系统。

```shell
rpm-ostree rollback
```

## 切换版本

在上一步将NestOS回滚到了旧版本，可以通过命令切换当前 NestOS 使用的rpm-ostree版本，将旧版本切换为新版本。

```shell
rpm-ostree deploy -r 22.03.20220325.dev.0
```

重启后确认目前NestOS已经使用的是新版本的ostree了。

# zincati自动更新使用

zincati负责NestOS的自动更新，zincati通过cincinnati提供的后端来检查当前是否有可更新版本，若检测到有可更新版本，会通过rpm-ostree进行下载。

目前系统默认关闭zincati自动更新服务，可通过修改配置文件设置为开机自动启动自动更新服务。

```shell
vi /etc/zincati/config.d/95-disable-on-dev.toml
```

将updates.enabled设置为true，同时增加配置文件，修改cincinnati后端地址。

```shell
vi /etc/zincati/config.d/update-cincinnati.toml
```

添加如下内容：

```script
[cincinnati]
base_url="http://nestos.org.cn:8080"
```

重新启动zincati服务。

```shell
systemctl restart zincati.service
```

当有新版本时，zincati会自动检测到可更新版本，此时查看rpm-ostree状态，可以看到状态是“busy”，说明系统正在升级中。

一段时间后NestOS将自动重启，此时再次登录NestOS，可以再次确认rpm-ostree的状态，其中状态转为"idle"，而且当前版本已经是“20220325”，这说明rpm-ostree版本已经升级了。

查看zincati服务的日志，确认升级的过程和重启系统的日志。另外日志显示的"auto-updates logic enabled"也说明更新是自动的。

# 定制NestOS

我们可以使用nestos-installer 工具对原始的NestOS ISO文件进行加工，将Ignition文件打包进去从而生成定制的 NestOS ISO文件。使用定制的NestOS ISO文件可以在系统启动完成后自动执行NestOS的安装，因此NestOS的安装会更加简单。

在开始定制NestOS之前，需要做如下准备工作：

* 下载 NestOS ISO
* 准备 config.ign文件

## 生成定制NestOS ISO文件

### 设置参数变量

```shell
$ export COREOS_ISO_ORIGIN_FILE=nestos-22.03.20220324.x86_64.iso
$ export COREOS_ISO_CUSTOMIZED_FILE=my-nestos.iso
$ export IGN_FILE=config.ign
```

### ISO文件检查

确认原始的NestOS ISO文件中是没有包含Ignition配置。

```shell
$ nestos-installer iso ignition show $COREOS_ISO_ORIGIN_FILE 

Error: No embedded Ignition config.
```

### 生成定制NestOS ISO文件

将Ignition文件和原始NestOS ISO文件打包生成定制的NestOS ISO文件。

```shell
$ nestos-installer iso ignition embed $COREOS_ISO_ORIGIN_FILE --ignition-file $IGN_FILE $COREOS_ISO_ORIGIN_FILE --output $COREOS_ISO_CUSTOMIZED_FILE
```

### ISO文件检查

确认定制NestOS ISO 文件中已经包含Ignition配置了。

```shell
$ nestos-installer iso ignition show $COREOS_ISO_CUSTOMIZED_FILE
```

执行命令，将会显示Ignition配置内容。

## 安装定制NestOS ISO文件

使用定制的 NestOS ISO 文件可以直接引导安装，并根据Ignition自动完成NestOS的安装。在完成安装后，我们可以直接在虚拟机的控制台上用nest/password登录NestOS。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/maintenance/kernel_live_upgrade/kernel_live_upgrade.md
---
# Kernel Live Upgrade Guide

This document describes how to install, deploy, and use the kernel live upgrade feature on openEuler. This kernel live upgrade feature on openEuler is implemented through quick kernel restart and live program migration. A user-mode tool is provided to automate this process.

This document is intended for community developers, open-source enthusiasts, and partners who want to learn about and use the openEuler system and kernel live upgrade. The users are expected to know basics about the Linux operating system.

## Application Scenario

The kernel live upgrade is to save and restore the process running data with the second-level end-to-end latency.

The following two conditions must be met:

1. The kernel needs to be restarted due to vulnerability fixing or version update.
2. Services running on the kernel can be quickly recovered after the kernel is restarted.

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secharden/kernel_parameters.md
---
# Kernel Parameters

## Hardening the Security of Kernel Parameters

### Description

Kernel parameters specify the status of network configurations and application privileges. The kernel provides system control which can be fine-tuned or configured by users. This function can improve the security of the OS by controlling configurable kernel parameters. For example, you can fine-tune or configure network options to improve system security.

### Implementation

1. Write the hardening items in  [Table 1](#en-us_topic_0152100187_t69b5423c26644b26abe94d88d38878eb)  to the  **/etc/sysctl.conf**  file.

   > \[!NOTE] **NOTE:**\
   > Writesecurity hardening items as follows:
   >
   > ```text
   > net.ipv4.icmp_echo_ignore_broadcasts = 1  
   > net.ipv4.conf.all.rp_filter = 1  
   > net.ipv4.conf.default.rp_filter = 1  
   > ```

   **Table  1**  Policies for hardening the security of kernel parameters

2. Run the following command to load the kernel parameters set in the  **sysctl.conf**  file:

   ```shell
   sysctl -p  /etc/sysctl.conf
   ```

### Other Security Suggestions

* **net.ipv4.icmp\_echo\_ignore\_all**: ignores ICMP requests.

  For security purposes, you are advised to enable this item. The default value is  **0**. Set the value to  **1**  to enable this item.

  After this item is enabled, all incoming ICMP Echo request packets will be ignored, which will cause failure to ping the target host. Determine whether to enable this item based on your actual networking condition.

* **net.ipv4.conf.all.log\_martians/net.ipv4.conf.default.log\_martians**: logs spoofed, source routed, and redirect packets.

  For security purposes, you are advised to enable this item. The default value is  **0**. Set the value to  **1**  to enable this item.

  After this item is enabled, data from forbidden IP addresses will be logged. Too many new logs will overwrite old logs because the total number of logs allowed is fixed. Determine whether to enable this item based on your actual usage scenario.

* **net.ipv4.tcp\_timestamps**: disables tcp\_timestamps.

  For security purposes, you are advised to disable tcp\_timestamps. The default value is  **1**. Set the value to  **0**  to disable tcp\_timestamps.

  After this item is disabled, TCP retransmission timeout will be affected. Determine whether to disable this item based on the actual usage scenario.

* **net.ipv4.tcp\_max\_syn\_backlog**: determines the number of queues that is in SYN\_RECV state.

  This parameter determines the number of queues that is in SYN\_RECV state. When this number is exceeded, new TCP connection requests will not be accepted. This to some extent prevents system resource exhaustion. Configure this parameter based on your actual usage scenario.

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/key_features.md
---
# Key Features

## AI

AI is redefining OSs by powering intelligent development, deployment, and O\&M. openEuler supports general-purpose architectures like Arm, x86, and RISC-V, and next-gen AI processors like NVIDIA and Ascend. Further, openEuler is equipped with extensive AI capabilities that have made it a preferred choice for diversified computing power.

* **openEuler for AI**: openEuler offers an efficient development and runtime environment that containerizes software stacks of AI platforms with out-of-the-box availability.

  * openEuler supports TensorFlow and PyTorch frameworks and software development kits (SDKs) of major computing architectures, such as Compute Architecture for Neural Networks (CANN) and Compute Unified Architecture (CUDA), to make it easy to develop and run AI applications.

  * Environment setup is further simplified by containerizing software stacks. openEuler provides three types of container images:

  1. **SDK images**: Use openEuler as the base image and install the SDK of a computing architecture, for example, Ascend CANN and NVIDIA CUDA.

  2. **AI framework images**: Use the SDK image as the base and install AI framework software, such as PyTorch and TensorFlow.

  3. **Model application images**: Provide a complete set of toolchains and model applications.

* **AI for openEuler**: AI makes openEuler more intelligent. EulerCopilot, an intelligent Q\&A platform, is developed using foundation models and openEuler data. It assists in code generation, problem analysis, and system O\&M.

  * **EulerCopilot**: EulerCopilot is accessible via web or shell.

    1. **Web**: Provides basic OS knowledge, openEuler data, O\&M methods, and project introduction and usage guidance.

    2. **Shell**: Delivers user-friendly experience using natural languages.

## Embedded

openEuler 22.03 LTS SP4 Embedded is equipped with an embedded virtualization base that is available in the Jailhouse virtualization solution or the OpenAMP lightweight hybrid deployment solution. You can select the most appropriate solution to suit your services. openEuler 22.03 LTS SP4 Embedded supports the Robot Operating System (ROS) Humble version, which integrates core software packages such as ros-core, rosbase, and simultaneous localization and mapping (SLAM) to meet the ROS 2 runtime requirements.

* **Southbound ecosystem**: Currently, openEuler Embedded supports AArch64 and x86-64 architectures. In 22.03 LTS SP4, RK3588 chips are supported. In the future, Loongson and Phytium processors will be supported.

* **Embedded elastic virtualization base**: The elastic virtualization base of openEuler Embedded is a collection of technologies used to enable multiple OSs to run on a system-on-a-chip (SoC). These technologies include bare metal, embedded virtualization, lightweight containers, LibOS, trusted execution environment (TEE), and heterogeneous deployment.

* **Mixed criticality deployment framework**: The mixed-criticality (MICA) deployment framework is built on the converged elastic base. The unified framework masks the differences between the technologies used in the underlying elastic virtualization base, enabling Linux to be deployed together with other OSs.

* **Northbound ecosystem**: More than 350 common embedded software packages can be built using openEuler. The ROS 2 Humble version is supported, which contains core software packages such as ros-core, ros-base, and SLAM. The ROS SDK is provided to simplify embedded ROS development. The soft real-time capability allows for response to soft real-time interrupts within microseconds. DSoftBus and HiChain for point-to-point authentication of OpenHarmony have been integrated to implement interconnection between openEuler-based embedded devices and between openEuler-based embedded devices and OpenHarmony-based devices. iSulad containers are supported so that openEuler or other OS containers can be deployed on embedded devices to simplify application porting and deployment.

* **UniProton**: This hard RTOS features ultra-low latency and flexible MICA deployments. It is suited for industrial control because it supports both microcontroller units and multi-core CPUs.

## What's New in the openEuler Kernel

openEuler 22.03 LTS SP4 runs on Linux kernel 5.10. It inherits the competitive advantages of community versions and innovative features released in the openEuler community.

* **Dynamic memory isolation and release**: Memory pages are dynamically isolated and de-isolated. When isolated, the original memory is migrated safely.

* **Online CPU inspection**: To avoid silent data corruption that is a common cause of data loss, faulty cores are detected and isolated to prevent faults before they are exacerbated.

* **Adaptive provisioning of computing power**: To ensure consistency and reliability of certain applications (such as cloud desktop systems) running on multi-core servers, computing power is dynamically provisioned based on load changes.

* **Power-aware scheduling**: At the service layer, memory access bandwidth, CPU load, and other information are collected to ensure sufficient resources for critical threads. A physical topology is introduced so that the P-state control mechanism extends to new dimensions, further reducing power consumption beyond the limits of single-die frequency and voltage regulation. This feature minimizes power consumption when the service load is low

* **Enhanced core isolation**: CPUs are classified into housekeeping and non-housekeeping. The former executes background processes such as periodic system clock maintenance, while the latter executes service processes. Background processes and interrupts are all allocated to housekeeping CPUs to prevent noise from affecting service process. This enhanced core isolation improves service performance, especially needed for HPC workloads.

* **Performance monitor unit (PMU) indicators**: When multiple services share node resources, indicators such as PSI are used to measure system contention, service throughput, and delay. These indicators are essential to locating system resource bottlenecks, understanding the resource demand of specific service processes, and dynamically adjusting resource allocation. This improves the quality of online services and system health.

* **KVM TDP MMU**: In Linux kernel 5.10 and later, KVM can scale to match demand for memory virtualization. This feature is contributed by Intel to the openEuler community. Compared with the traditional KVM memory management unit (MMU), the two dimensional paging MMU, or TDP MMU, offers more efficient handling of concurrent page faults and better support for large-scale VM deployments, such as those with multiple vCPUs and large memory. In addition, the new Extended Page Tables (EPT) and Nested Page Tables (NPT) traversal interface boosts host memory utilization by removing the dependency on the rmap data structure that is typical in traditional memory virtualization solutions.

## NestOS

NestOS is a cloud OS incubated in the openEuler community. It runs rpm-ostree and Ignition technologies over a dual rootfs and atomic update design, and uses nestos-assembler for quick integration and build. NestOS is compatible with Kubernetes and OpenStack, and reduces container overheads and provides extensive cluster components in large-scale containerized environments.

* **Out-of-the-box availability**: integrates popular container engines such as iSulad, Docker, and Podman to provide lightweight and tailored OSs for the cloud.
* **Easy configuration**: uses the Ignition utility to install and configure a large number of cluster nodes with a single configuration.
* **Secure management**: runs rpm-ostree to manage software packages and works with the openEuler software package source to ensure secure and stable atomic updates.
* **Hitless node updating**: uses Zincati to provide automatic node updates and reboot without interrupting services.
* **Dual rootfs**: executes dual rootfs for active/standby switchovers, to ensure integrity and security during system running.

## SysCare

SysCare is a system-level hotfix software that provides security patches and hot fixing for OSs. It can fix system errors without restarting hosts. SysCare combines kernel-mode and user-mode hot patching to take over system repair, saving time for users to focus on other aspects of their business. It includes hot patch making, hot patch lifecycle management, and integration of user-mode hot patches for ELF files, kernel hot patches, and user-mode hot patches. The following features are added in openEuler 22.03 LTS SP4:

* Configures the dependencies of hot patches when they are created.
* Manages multiple user-mode patches.
* Detects conflicts between user-mode hot patches.
* Forcibly overwrites user-mode hot patches when conflicts occur.

## GCC for openEuler

GCC for openEuler is a high-performance compiler oriented to the openEuler ecosystem for various scenarios. It is developed on the open source GNU Compiler Collection (GCC) and inherits the capabilities of the open source GCC. GCC for openEuler optimizes C, C++, and Fortran deployments in terms of instructions, memory, and automatic vectorization, to adapt to and unleash the compute of hardware platforms, such as Kunpeng, Phytium, and LoongArch. New capabilities of GCC for openEuler include:

* Multiple GCC versions now support OpenMP, including the gcc-toolset-12 package series that run on GCC 12.3.0. Fortran supports OpenMP 4.5, while C/C++ supports some OpenMP 5.0 specifications.
* Last-level cache (LLC) allocation is optimized. By analyzing memory multiplexing on the main execution paths in a program, GCC for openEuler determines and sorts hot data. Then, prefetch instructions are inserted to pre-allocate data to the LLC, reducing LLC misses.
* Optimizations of CPUBench help intelligently identify and reduce instructions while boosting performance.

## A-Ops

A-Ops is an intelligent O\&M platform that covers data collection, health check, and fault diagnosis and rectification. Released with openEuler 22.03 LTS SP4, Apollo is an intelligent patch management framework that integrates core functions such as vulnerability scanning, CVE fixing (with cold/hot patches), and hot patching rollback. Apollo periodically downloads and synchronizes security advisories and sets scheduled tasks to scan for vulnerabilities.

Apollo enables the intelligent management of kernel patches.

* **Hot patch source management**: When openEuler vulnerabilities are released through a security advisory, the software package used for fixing the vulnerabilities is also released in the update repository. By default, after openEuler is installed, the cold patch update repository of the corresponding OS version is provided. Users can also configure the update repository of cold or hot patches.
* **Vulnerability scanning**: Manual or periodic cluster scans can be performed to check the impact of CVEs on a cluster and cold or hot patches are provided for repair.
* **Hybrid patch management**: Cold and hot patches can be applied independently or together to implement silent incorporation of hot patches on the live network and reduce hot patch maintenance costs.
* **Hot patch lifecycle management**: hot patch removal, rollback, and query

## Gazelle

Gazelle is a high-performance user-mode protocol stack. It directly reads and writes NIC packets in user mode based on the Data Plane Development Kit (DPDK), transmits the packets through shared hugepage memory, and uses the LwIP protocol stack, thereby greatly improving the network I/O throughput of applications and accelerating the network for databases. With Gazelle, high performance and universality can be achieved at the same time. In openEuler 22.03 LTS SP4, support for the UDP protocol and related interfaces is added for Gazelle to enrich the user-mode protocol stack.

* Available in single VLAN, bond4, and bond6 modes, and supports NIC self-healing after network cables are reinstalled.
* A single-instance Redis application on Kunpeng 920 VMs supports over 5,000 connections, improving performance by more than 30%.
* The TCP\_STREAM and TCP\_RR tests of netperf (packet length less than 1,463 bytes) are supported.
* Logs of the LStack, lwIP, and gazellectl modules of Gazelle are refined for more accurate fault locating.

## OCI Runtime for iSulad

Open Container Initiative (OCI) is a lightweight and open governance project dedicated to creating an open industry standard for container formats and runtimes. Developed with the support of the Linux Foundation, it aims to let any container runtimes that support OCI Runtime use OCI images to run containers. iSulad is a lightweight container engine compatible with mainstream container ecosystems, and supports standard southbound OCI APIs and can connect to multiple OCI runtimes, such as runc and kata.

As OCI has matured dramatically in the last few years, container runtimes that comply with OCI Runtime have been fitting into an expanding scope of application scenarios. runc is the first reference implementation of OCI Runtime. In the current openEuler version:

* The interconnection between iSulad and OCI Runtime is optimized, known defects are rectified, and the `isula top` and `isula attach` interfaces are added.
* runc is set as the default runtime for iSulad.
* After the default runtime is switched to runc, the dependency library of isulad-shim connected to OCI Runtime is changed to an independent and tailored static tool library.
  The switchover avoids existing process breakdowns caused by tool library upgrades, and reduces the memory overhead of containers.

## Distributed Data Management

The distributed data management system is a data management capability ported from the OpenHarmony community. This system encapsulates over 100 universal APIs that adopt DSoftBus dynamic networking to provide a range of data synchronization, such as strong and weak consistency, for each device node on the network.

* **Feature Description**

  * **Relational database**: manages data based on a relational model. It uses SQLite as the underlying persistent storage engine and supports all SQLite features.

  * **KV Store**: a key–value (KV) database that runs on SQLite. It manages KV pairs and distributes data across multiple devices and applications.

  * **Distributed Data Object**: an object-oriented in-memory data management framework that implements data object collaboration for the same application among multiple devices.

  * **Distributed Data Service**: synchronizes data between trusted devices, delivering a consistent access experience on different devices.

  * **DSoftBus**: discovers and connects devices at the network link layer.

  * **SQLite**: an open source component that provides native SQLite capabilities

* **Containerized DSoftBus**

  Migrating legacy service software to containers can remove the barriers to modernization. In openEuler 22.03 LTS SP4, DSoftBus can be deployed as a container with its dependencies and multi-client support is enabled, to greatly simplify service installation, deployment, and testing and improve compatibility with service software.

## Memory Overcommitment

Memory overcommitment is an efficient method to increase the available memory space for cloud native containers.

* **Cgroup memory policies**
  * **Proactive memory reclamation**: The type of reclaimed memory pages can be specified, for example, file pages and anonymous pages.
  * **Watermark-based reclamation**: Minimum, low, and high watermarks can be configured for passive reclamation. Asynchronous reclamation can be performed in the background to avoid impact on existing services.
  * **Memory deduplication**: All the memory space used by processes in a container can be included in KSM deduplication, without requiring applications to call the madvise API to mark memory areas beforehand.
  * **Swap space**: For each independent container, you can configure the swap backend devices (such as zram and storage devices), maximum swap space, proactive swap-in, and enable or disable swap.

* **Basic optimizations**

  * **Memory compression**: Secondary compression with zram leverages multiple compression algorithms to increase the compression ratio and compression/decompression speed.

  * **Memory reclamation**: TLB refresh is optimized in unmap and migration processes to accelerate memory reclamation and reduce lock conflicts. Transparent huge page swap is optimized as well.

* **Optimal decision-making based on the PSI mechanism**

  * PSI is available in cgroup v1 and v2.

  * Memory is proactively reclaimed using the PSI negative feedback mechanism, to improve decisions that are based on service load and cluster information. This design maintains service performance and reliability during memory overcommitment.

## DIM

Dynamic Integrity Measurement (DIM) enables timely detection and troubleshooting measures to handle attacks. It measures key memory data like code segments during program running and compares the results with the reference values to determine data tampering in the memory.

* **DIM provides the following features:**

  * Measures user-mode processes, kernel modules, and code segment in the kernel memory.
  * Extends measurements to the PCR register of the TPM 2.0 chip for remote attestation.
  * Configures measurements and verifies measurement signatures.
  * Generates and imports measurement baseline data using tools, and verifies baseline data signatures.
  * Supports SM3 algorithms.

* **DIM consists of two software packages: dim\_tools and dim.**

  * **dim\_tools**: provides the `dim_gen_baseline` command-line tool, which generates code segment measurement baseline in a specified format by parsing the Executable and Linkable Format (ELF) binary file.

  * **dim**: provides the dim\_core and dim\_monitor kernel modules. The former is the core module that parses and imports measurements and baselines configured by users, obtains target measurement data from memory, and performs measurement. The latter protects code segments and key data in dim\_core to prevent invalid measurement due to dim\_core tampering.

## Secure Boot

Secure Boot relies on public and private key pairs to sign and verify components in the boot process. A typical boot process uses the previous component to verify the digital signature of the next component. If the verification is successful, the next component runs; if the verification fails, the boot stops.

* **Feature Description**

  * The Signatrust platform generates and manages public and private key pairs and certificates, and provides the signing service for EulerMaker to build openEuler software packages.

  * The Signatrust platform signs code of the EFI components (shim, GRUB, vmlinux) for Secure Boot when the software packages are built by EulerMaker.

  * Signature verification is performed during system boot to ensure system components are safe and secure.

* **Constraints**

  * The Signatrust platform can only sign components built in the openEuler community, but cannot sign files developed by external projects or custom user files.

  * The Signatrust platform supports only the RSA algorithm.

## secDetector

secDetector is an intrusion detection system designed for OSs. It provides intrusion detection and response for critical infrastructure and reduces development costs while enhancing detection and response for third-party security tools.

secDetector consists of the detection feature cases, exception detection probes, and attack blocking module. The exception detection probes collect OS attack events that match the MITRE ATT\&CK patterns. There are eight types of exception detection probes that can detect advanced persistent threats (APTs): file operation, process management, network access, program behavior, memory tampering, resource consumption, account management, and device operation. The technical implementation architecture of secDetector consists of the SDK, service, detection feature cases, and detection framework (core).

* The secDetector SDK is provided as a user-mode dynamic link library (DLL) deployed in the security awareness services that require secDetector. The SDK communicates with the secDetector service to complete related operations (such as subscription, unsubscription, and message reading).

* The secDetector service is a user-mode service application. It manages and maintains the subscriptions of the security awareness services and maintains the probe running statuses.

* The detection feature cases correspond to a series of exception detection probes, which are in different forms. For example, each probe for detecting kernel exceptions is available in a kernel module (**.ko** file).

* The detection framework (core) is the base framework for case management, and provides common functional units required by workflows. The kernel exception detection framework is carried by a kernel module (**.ko** file).

## EulerMaker

EulerMaker is a package build system that converts source code into binary packages. It enables developers to assemble and tailor scenario-specific OSs thanks to incremental/full build, gated build, layer tailoring, and image tailoring capabilities.

* **Incremental/Full build**: Analyzes the impact of the changes to software and dependencies, obtains the list of packages to be built, and delivers parallel build tasks based on the dependency sequence.
* **Build dependency query**: Provides a software package build dependency table in a project, and collects statistics on software package dependencies.
* **Layered tailoring**: Overlays configuration layer models based on SPEC or YAML to tailor the software package version, patches, build and installation dependencies, compilation options, and build process to your project.
* **Image tailoring**: Developers can configure the repository source to generate ISO, QCOW2, and container OS images, and tailor the list of software packages for the images.
* **Local task reproduction**: Reproduces a build task locally using commands, facilitating build problem locating.
* **Easy project creation**: Creates projects based on YAML configurations, and packages can be added in batches, greatly simplifying user operations.

## DPUDirect

DPUDirect creates a collaborative operating environment for services, enabling them to be easily offloaded and ported between hosts and data processing units (DPUs). DPUDirect builds a cross-host collaboration framework at the OS layer of the host and DPU, providing a consistent runtime view for the management-plane processes offloaded to the DPU and the service processes on the host. In this way, applications are unaware of offload. Only a small amount of service code on the management plane needs to be adapted to ensure software compatibility and evolution, as well as reducing component maintenance costs.

* File system collaboration supports cross-host file system access and provides a consistent file system view for host and DPU processes. It also supports special file systems such as proc, sys, and dev.
* IPC collaboration enables imperceptible communication between host and DPU processes. It supports FIFO and UNIX domain sockets for cross-host communication.
* Mounting collaboration performs the mount operation in a specific directory on the host, which can adapt to the container image overlay scenario. The offloaded management-plane process can construct a working directory for the service process on the host, providing a unified cross-node file system view.
* epoll collaboration supports epoll operations for cross-host access of remote common files and FIFO files, and supports read and write blocking operations.
* Process collaboration uses the rexec tool to remotely start executable files. The rexec tool takes over the input and output streams of the remotely started processes and monitor the status to ensure the lifecycle consistency of the processes at both ends.

## Live VM Migration with vDPA NIC Passthrough

The kernel-mode vHost Data Path Acceleration (vDPA) framework provides a device virtualization solution that performs equivalently to passthrough. The vDPA framework unifies the architecture for diverse hardware forms, such as intelligent NICs and DPUs, and supports live migration across different hardware vendors.

Extended vDPA and vHost APIs are used for live migrating VMs across vDPA devices from the same vendor, addressing the basic live migration requirements of vDPA passthrough VMs. Further, cross-vendor live migration uses embedded code to meet future requirements.

## Lustre Server Software Package

Lustre is an open source parallel file system designed for high scalability, performance, and availability. Lustre runs on Linux and provides POSIX-compliant UNIX file system interfaces.

* **High scalability and performance**: A Lustre system is scalable in terms of the number of client nodes, drive storage capacity, and bandwidth. The scalability and performance depend on the available drives, network bandwidth, and server throughput. The following lists the main features.
  * **Client scalability**: Up to 100,000 clients are supported. A typical production environment usually has 10,000 to 20,000 clients.
  * **Client performance**: The I/O performance of a single client is 90% of the network bandwidth. The aggregated I/O performance reaches 50 million IOPS, with an I/O bandwidth of up to 50 TB/s.
  * **OSS scalability**: A single OSS can manage up to 32 OSTs, each capable of storing 500 million objects, or 1,024 TB. A maximum of 1,000 OSSs and 4,000 OSTs are supported in a Lustre system.
  * **OSS performance**: A single OSS can deliver 1.5 million IOPS, with an I/O bandwidth of 15 GB/s. The aggregated I/O performance reaches 50 million IOPS, with an I/O bandwidth of up to 50 TB/s.
  * **MDS scalability**: A single MDS can manage up to four MDTs. A single MDT supports 4 billion files of up to 16 TB when LDISKFS is used as the backend file system, or 64 billion files of up to 64 TB when ZFS is used as the backend file system.
  * **MDS performance**: 1 million creation operations or 2 million metadata stat operations can be performed within a second.
  * **File system scalability**: The maximum size of a single file in the LDISKFS backend is 32 PB. An aggregated Lustre system can contain up to 1 trillion files, or 512 PB.

## DDE

Deepin Desktop Environment (DDE) was originally developed for Uniontech OS and has been used in the desktop, server, and dedicated device versions of Uniontech OS.

DDE focuses on delivering high quality user interactions and visual design. DDE is powered by independently developed core technologies for desktop environments and provides login, screen locking, desktop, file manager, launcher, dock, window manager, control center, and additional functions. Due to its user-friendly interface, excellent interactivity, high reliability, and strong privacy protection, it is one of the most popular desktop environments among users.

## FangTian Window Engine

The FangTian window engine delivers fundamental display technologies to build a foundation for openEuler's desktop environments. FangTian hosts display services such as window management, graphic drawing and compositing, and screen delivery.

* **Feature Description**

  * **Window management** creates, moves, zooms, arranges, and destroys windows. An independent window policy module is used to support various scenarios on multiple device types, such as mobile phones and PCs.
  * **Window display** provides capabilities such as buffer allocation and swapping, vertical synchronization, rendering, compositing, and screen display. The data-driven interfaces and unified architecture realize high performance and low memory usage.
  * **FT** is a display protocol that enables the ArkUI framework to interact with FangTian. It provides unified rendering and data-driven interfaces to lower rendering load, reduce data from cross-process interactions, and enhance application animation performance.
  * **ArkUI** is a declarative UI development framework for OpenHarmony applications. It is derived from OpenHarmony and has been adapted to openEuler, allowing ArkUI-based OpenHarmony applications to run on openEuler as well.

* **Highlights**

  * **Linux application support**: Native Wayland and OpenHarmony applications can run simultaneously.

  * **High-performance display of OpenHarmony applications**: 50 application windows can be displayed at 60 FPS.

* **Constraints**

  * Only x86\_64 applications are supported. The functions of some ArkUI controls are not enabled.

  * Wayland protocol compatibility does not apply to protocol extensions.

## sysMaster

sysMaster is a collection of ultra-lightweight and highly reliable service management programs. sysMaster manages processes, containers, and VMs centrally and provides fault monitoring and self-healing mechanisms to help deal with Linux initialization and service management challenges. All these features make sysMaster an excellent choice for server, cloud computing, and embedded scenarios.

* **New features**

  * devMaster component to manage device hot swap.

  * Live updates and hot reboot operations.

  * VMs now support PID 1.

* **Constraints**

  * Only available for 64-bit OSs.

  * sysMaster configuration files must be in TOML format.

  * sysMaster can run only in system containers and VMs.

## migration-tools

migration-tools, developed by UnionTech Software Technology Co., Ltd., is positioned to meet demand for smooth, stable, and secure migration to the openEuler OS.

* **Server module**: the core of migration-tools. This module is developed on the Python Flask Web framework. It receives task requests, processes execution instructions, and distributes the instructions to each Agent.

* **Agent module**: installed in the OS to be migrated to receive task requests from the Server module and perform migration.

* **Configuration module**: reads configuration files for the Server and Agent modules.

* **Log module**: records logs during migration.

* **Migration assessment module**: provides assessment reports such as basic environment check, software package comparison and analysis, and pre-migration compatibility checks.

* **Migration function module**: provides quick migration, displays the migration progress, and checks the migration result.

## utshell

utshell is a new shell that introduces new features and inherits the usability of Bash. It enables interaction through command lines, such as responding to user operations to execute commands and providing feedback, and can execute automated scripts to facilitate O\&M.

* **Command execution**: Runs and sends return values from commands executed on user machines.

* **Job control**: Executes, manages, and controls multiple user commands as background jobs.

* **Batch processing**: Automates task execution using scripts.

* **Command aliases**: Allows users to create aliases for commands to customize their operations.

* **Historical records**: Records the commands entered by users.

## utsudo

sudo is one of the commonly used utilities for Unix-like and Linux OSs. It enables users to run specific commands with the privileges of the super user. utsudo is developed to address issues of security and reliability common in sudo. utsudo uses Rust to deliver more efficient, secure, and flexible privilege escalation. The tool uses modules such as common utility, overall framework, and function plugins.

* **Access control**: Limits the commands that can be executed by users, and specifies the required authentication method.
* **Audit log**: Records and traces all commands and tasks executed by each user.
* **Temporary privilege escalation**: Allows common users to temporarily escalate to a super user for executing privileged commands or tasks.
* **Flexible configuration**: Allows users to set arguments such as command aliases, environment variables, and execution parameters to meet system requirements.

## i3

i3 is a tiling window manager that enables the keyboard to manage the window layouts in a session or across multiple monitors. For more details, see the [upstream document](https://i3wm.org/docs/).

## Trusted Platform Control Module

The trusted platform control module (TPCM) is a base and core module that can be integrated into a trusted computing platform to establish and ensure a trust source. As one of the innovations in Trusted Computing 3.0 and the core of active immunity, TPCM implements active control over the entire platform. The overall system design consists of the protection module, computing module, and trusted management center software.

* **Overall system design**

  * **Trusted management center**: This centralized management platform, provided by a third-party vendor, formulates, delivers, maintains, and stores protection policies and reference values for trusted computing nodes.

  * **Protection module**: This module operates independently of the computing module and provides trusted computing protection functions that feature active measurement and active control to implement security protection during computing. The protection module consists of the TPCM main control firmware, TCB, and TCM.

  * **Computing module**: This module includes hardware, an OS, and application layer software.

* **Constraints**
  * Supported server: TaiShan 200 server (model 2280)

  * Supported BMC card: BC83SMMC

## safeguard

safeguard helps protect the Linux kernel and the OS based on eBPF by intercepting and auditing security operations. It uses the libbpfgo library and the Go language to implement top-level control.

* **File safeguarding**

  * Traces file system activities, including file open, close, read, write, and delete.

  * Modifies the behavior of file systems through the interception of certain file operations and custom security policies.

  * **Security policies**

    * Operations on files can be intercepted or redirected through eBPF. For example, read and write operations on sensitive files can be intercepted, and access to certain files can be redirected.

    * Access control can be customized. eBPF checks the identity, permissions, and environment of a user who requests access to a file, and allows or denies the request based on custom rules.

    * Audit and monitoring can be customized. For example, eBPF records the information about operations on certain files, such as the operator, time, and action, and outputs the information to the logs.

* **Process safeguarding**

  * Traces process life cycles, such as process creation and termination.

  * Modifies the behavior of processes, such as injecting or modifying some system calls or implementing custom scheduling policies.

* **Network safeguarding**

  * Traces network activities, such as sending, receiving, forwarding, and discarding network packets.

  * Modifies the behavior of networks through filtering and rewriting of network packets and custom routing policies.

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/kiran/kiran_user_guide.md
---
# Kiran Desktop Environment

## 1. Overview

Kiran desktop environment is a stable, efficient, and easy-to-use desktop environment oriented towards user and market requirements. It consists of the desktop, taskbar, tray, control center, and window management components. This document describes how to use the Kiran desktop.

## 2. Desktop

### 2.1. Login Screen

After the installation is complete, restart the system. After the system is started, enter the user name and password to log in to the system. The login screen displays the time, date, power button, and soft keyboard button. The adaptive UI supports screen zooming and multi-screen display. The login dialog box can be switched between screens following the mouse pointer.

![Figure 1 Login screen](figures/kiran-1.png)

### 2.2. Main Screen

Enter the correct user name and password to log in to the system. The main screen is displayed, as shown in the following figure:

![Figure 2-Main screen ](figures/kiran-2.png)

Several icons are displayed on the desktop, such as **Computer**, **Home** folder, and **Trash**. The panel, located at the bottom of the screen, allows you to launch applications and switch between virtual desktops.
A desktop is a working area of a user. You will perform operations and run applications on the desktop. You can place the files and applications on the desktop for easy access. Double-click the icons to run the corresponding applications or open the files. You can drag, add, or delete desktop icons. Desktop icons allow you to complete your work more conveniently.

![Figure 3-Computer](figures/kiran-3.png) **Computer**: Double-click to display all the local and remote disks and folders accessed from this computer.

![Figure 4-Home folder](figures/kiran-4.png) **Home** folder: Double-click to display the contents in the home directory of the current user.

![Figure 5- Trash](figures/kiran-5.png) **Trash**: Deleted files are temporarily stored in Trash.

Shortcut menu: Right-click on the desktop to display the shortcut menu, which provides shortcuts for icon management, folder creation, document creation, desktop background settings, and theme settings.

**Create Folder**: Creates a folder.

**Create Launcher...**: Creates a launcher.

**Create Document**: Creates an empty plain-text file.

**Open Terminal...**: Opens the terminal application.

**Organize Desktop by Name**: Sorts desktop files by name.

**Keep Aligned**: If this option is selected, the desktop icons are aligned to the grid.

**Change Desktop Background**: Opens Background to change the background picture of the desktop or lock screen.

### 2.3 Panel

The panel is usually located at the bottom of the screen and includes the start menu button, quick launch area, icons of frequently used applications and desktop applets, and taskbar that displays the currently running application.
When you hover the mouse pointer over an icon for several seconds, a white dialog box is displayed, describing the function of the icon.

![Figure 6-System panel](figures/kiran-6.png)

## 3. Taskbar

Taskbar: displays running applications or opened documents. You can click an item on the taskbar to maximize or minimize the selected application window. You can right-click an item and choose Maximize, Minimize, or Close the application window from the shortcut menu.

| Component                                               | Description                                                                                                         |
| :------------------------------------------------------ | :------------------------------------------------------------------------------------------------------------------ |
| ![Figure 7-Start Menu](figures/kiran-7.png)             | Start Menu button: Similar to the Start button in Windows. When you click it, the cascaded start menu is displayed. |
| ![Figure 8 Workspace button](figures/kiran-8.png)       | Click to display the workspaces.                                                                                    |
| ![Figure 9 File browser button](figures/kiran-9.png)    | Click to start the file browser to view and manage files.                                                           |
| ![Figure 10 Terminal button](figures/kiran-10.png)      | Click to start the terminal.                                                                                        |
| ![Figure 11 Web browser button](figures/kiran-11.png)   | Click to start the Firefox browser.                                                                                 |
| ![Figure 12 Network control icon](figures/kiran-12.png) | Displays the current network status. Click to modify the network configuration.                                     |
| ![Figure 13-Clock button](figures/kiran-13.png)         | Displays the current date and time. You can customize the display style as required.                                |

## 4. Control Center

### 4.1. Start Menu Settings

Choose **Start Menu** > **Control Center** > **Start Menu** Settings.
You can set the display mode and opacity of the start menu, as shown in the following figure:

![Figure 14-Start Menu Settings](figures/kiran-14.png)

The appearance start menu changes based on the opacity and display mode, as shown in the following figure:

![Figure 15-Start menu](figures/kiran-15.png)

### 4.2. Greeter Settings

Choose **Start Menu** > **Control Center** > **Greeter Settings**.
In the Kiran desktop, you can set the login screen appearance by choosing **Greeter Settings** in **Control Center**, including the background image of the login screen, whether to enable automatic login, zoom ratio, whether to allow login by entering the user name, and whether to display the user list, as shown in the following figure:

![Figure 16-Greeter Settings](figures/kiran-16.png)

You can also set automatic login. Set the user name and delay for automatic login. After the system is restarted, the user automatically logs in without entering the password.

![Figure 17 Autologin settings](figures/kiran-17.png)

### 4.3 Display Settings

Display attribute customization is required for every desktop environment. The Kiran desktop provides a powerful tool for customizing display attributes. You can choose **Start Menu** > **Control Center** > **Display Settings** to open the **Display Settings** window, as shown in the following figure:

![Figure 18-Display Settings](figures/kiran-18.png)

You can set the screen rotation, resolution, refresh rate, zoom rate, and flip. After the settings are complete, click **Apply**.

### 4.4 Mouse Settings

Configure the mouse by selecting **Kiran Cpanel Mouse** in **Control Center**. You can select left-hand or right-hand mode, adjust the mouse pointer speed, set whether to scroll naturally, and set whether to enable the middle button emulation by pressing the left and right button simultaneously. The following figure shows the normal mouse setting window:

![Figure 19-Kiran Cpanel Mouse](figures/kiran-19.png)

### 4.5. Account Manager

Account Manager is an easy-to-use tool for managing users and user groups. You can use this tool to:

1. Add users and set user attributes.
2. Modify user attributes.
3. View user attributes.
4. Delete users.

User attributes include the user name, password, and login shell. User group attribute indicates the users in the user group.

#### 4.5.1 Starting Account Manager

In **Control Center**, choose **Account Manager** to start the account management tool, as shown in the following figure:

![Figure 20 Account Manager](figures/kiran-20.png)

In the window, you can see the user list on the left and the detailed information on the right. Currently, all users in the system except the root user are listed. Click a user on the left. The detailed information about the user is displayed, including the user ID and user type.
Click **Create new user**. On the page that is displayed on the right, enter the user name, user type, and password, and change the avatar as required. After setting the attributes, click **Confirm**.

![Figure 21 Creating an account](figures/kiran-21.png)

**Note**: If you have set the minimum length of a password (for example, four digits), you must enter a password of at least four digits. Otherwise, the system will not accept the password.

Click the avatar area to change the avatar. The system has built-in avatars for you to select. You can also add your own avatar and click Confirm to save the settings.

![Figure 22-Change avatar](figures/kiran-22.png)

#### 4.5.2. Deleting a User

Click the user to be deleted in the left area and click **Delete** on the toolbar on the right, as shown in the following figures:

![Figure 23 Deleting a user](figures/kiran-23.png)
![Figure 24-Confirming the deletion](figures/kiran-24.png)

In the displayed dialog box, click **No** to cancel the deletion, or click **Yes** to confirm the deletion.

#### 4.5.3. Advanced Settings

Choose **Create new user**, enter the user name and password, and then choose **Advanced Settings**. In the displayed dialog box, set the login shell, user ID, and user home directory.

![Figure 25-Advanced Settings](figures/kiran-25.png)

### 4.6. Appearance

Display attribute customization is required for every desktop environment. The Kiran desktop provides a powerful tool for customizing display attributes. Appearance is a tool that provides unified configuration and management for the desktop background, theme, and font of the system.
Choose **Start Menu** > **Control Center** > **Appearance**. The **Appearance** window is displayed, as shown in the following figure:

![Figure 26-Appearance Preferences](figures/kiran-26.png)

#### 4.6.1. Theme

Theme can be used to set the style of the dialog boxes, menus, system panels, and icons in a unified manner or separately according to your preference.

1. Theme Settings
   The system provides multiple themes by default. You can view the theme information in the **Theme** tab page. Click the theme in the **Theme** tab page to set the system theme, as shown in the following figure:

   ![Figure 27 Theme settings](figures/kiran-27.png)

2. Customizing a Theme
   You can click Customize... to customize a theme based on your preferences, as shown in the following figure. Customization options include controls, color, window border, icons, and pointer.

   ![Figure 28-Customize theme](figures/kiran-28.png)

#### 4.6.2. Background

You can set the desktop background, including the color and style.

1. Background image settings
   As shown in the following figure, click a wallpaper in the wallpaper area to set it as the desktop wallpaper.

   ![Figure 29-Background Settings](figures/kiran-29.png)

2. Style
   You can choose how the wallpaper fits the screen by choosing a style from the drop-down list. The styles include tile, zoom, center, scale, stretch, and span.

3. Adding and removing wallpapers
   You can click **Add...** to add your own wallpaper, as shown in the following figure:

   ![Figure 30-Add wallpaper](figures/kiran-30.png)

   Click **Open** to add the wallpaper.
   You can also click **Remove** to remove wallpapers that you do not like. Simply select a wallpaper and click **Remove**.

4. Desktop background color filling settings
   You can set a color as the background. In the wallpaper tab page, choose **No Desktop Background** to use a color as the background.
   The color filling styles include solid color, horizontal gradient, and vertical gradient.

   ![Figure 31-Background color filling](figures/kiran-31.png)

#### 4.6.3. Font

1. Font Settings

You can set the fonts of the GUI of the system. The font styles include application, document, desktop, window title, and fixed width fonts.

![Figure 32-Font settings](figures/kiran-32.png)

1. Font rendering and details settings

Font rendering settings: You can choose one of the following font rendering styles: monochrome, best shapes, best contrast, and subpixel smoothing.
By default, **best shapes** is used, as shown in the following figure:

![Figure 33 Font rendering settings](figures/kiran-33.png)

1. Font Details Settings
   You can click **Details...** to set the font details. Details settings include resolution, smoothing, hinting, and subpixel order.

![Figure 34-Font details setting](figures/kiran-34.png)

You can choose whether to display icons in menus and on buttons.

![Figure 35-Icon display settings](figures/kiran-35.png)

## 5. Desktop Applications

### 5.1. Text Editor

To launch the text editor, click **Start Menu**> **All applications** > **Utilities**> **Pluma**. You can also start the text editor by entering **pluma** in the shell prompt.
A text editor is one of the most commonly used tools in all computer systems. Whether to you are creating a plain text file, data file, or source program, you need to use an editor. The text editor is used to view and modify plain text files. Plain text files, such as system logs and configuration files, are common text files that do not contain fonts or style formats.

![Figure 36-Text editor](figures/kiran-36.png)

### 5.2. Terminal

In the desktop environment, you can use the Terminal application to enter the command line interface. To start Terminal, choose **Start Menu** > **All applications** > **Utilities** > **Terminal**, or click the icon on the desktop panel.

![Figure 37-Terminal](figures/kiran-37.png)

### 5.3. Firefox

To launch Firefox, click **Start Menu** > **All applications** > **Network** > **Firefox**.
Firefox is a free and open source web browser. It uses the Gecko rendering engine and supports multiple operating systems, such as Windows, Mac OS X, and GNU/Linux. Firefox is small in size, fast in speed, and has other advanced features, such as tabbed browsing, faster loading speed, pop-up blocker, customizable toolbar, extension management, better search features, and a convenient sidebar.

![Figure 38 Firefox](figures/kiran-38.png)

### 5.4 Screenshot Tool

Choose **Start Menu** > **All applications** > **Graphics** > **Screenshot tool** to start the screenshot tool.
Screenshot tool is a small and flexible screenshot software of the Kiran desktop. The operation UI is simple and easy to use. When the software is started, the icon of the screenshot tool is added to the tray.

![Figure 39-Screenshot icon in the tray](figures/kiran-39.png)

Click the icon to display the screenshot interface. You can select the screenshot area. You can right-click the icon and choose Open Launcher to set the capture area and delay.

![Figure 40-Screenshot UI](figures/kiran-40.png)
![Figure 41-Launcher UI](figures/kiran-41.png)

In the displayed dialog box, click **√** to save the file to the desktop, or choose Options and select a custom save location, as shown in the following figure:

![Figure 42-Screenshot process](figures/kiran-42.png)

### 5.5 Network Settings

The Kiran desktop uses NetworkManager as the network configuration tool. NetworkManager can set, configure, and manage various network types, and provides advanced support for mobile broadband devices, Bluetooth, and IPv6 protocol. Choose Start Menu > **Control Center** > **Advanced Network Configuration**, or right-click the network icon in the lower right corner of the desktop and choose **Edit Connections...**, as shown in the following figure:

![Figure 43-NetworkManager](figures/kiran-43.png)

Wired connection settings:
Select the current NIC. For example, **ens33** is the NIC of the current system. Select the NIC and click the edit button. The NIC editing dialog box is displayed:

![Figure 44 Editing an NIC](figures/kiran-44.png)

IPv4 Settings are frequently used. In this example, DHCP is selected to obtain the IP address and DNS server. The system automatically obtains an IP address for the user.
When you need to manually enter the IP address, select **Manual** from the **Method** drop-down list, as shown in the following figure:

![Figure 45 IPv4 settings](figures/kiran-45.png)

Click **Add**, enter the IP address, subnet mask, gateway, and DNS server, as shown in the following figure:

![Figure 46 Setting the network IP address and DNS](figures/kiran-46.png)

Enter the IP address, subnet mask, gateway, and DNS, and Click **Save**. Click the network icon in the lower right corner of the desktop, choose **Disconnect** to disconnect from the network, and then reconnect to the network.

***

### 5.6. Time and Date Manager

To set the date and time, select **Time And Date Manager** in **Control Center**, or click the date area in the lower right corner of the desktop. The following window is displayed:

![Figure 47 Time And Date Manager](figures/kiran-47.png)

Automatic synchronization: Enable **Automatic synchronization** and connect to the Internet to automatically synchronize the date and time.
Time zone settings: Click **Change Time Zone**, select a time zone from the list on the right, and then click **Save**.

![Figure 48 Change Time Zone](figures/kiran-48.png)

Manually set the time: Disable **Automatic synchronization** and click **Set Time Manually** to manually set the year, month, day, and time. After the modification is complete, click **Save**.

![Figure 49 Set Time Manually](figures/kiran-49.png)

Modifying the date format: Click **Time date format setting** to modify the date format. You can set the long and short date display formats, time format, and whether to display seconds.

![Figure 50 Time date format setting](figures/kiran-50.png)

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/kiran/kiran_installation.md
---
# Kiran Installation

## Introduction

Kiran desktop environment, developed by Kylinsec, is a stable, efficient, and easy-to-use desktop environment oriented towards user and market requirements. Kiran supports x86 and AArch64 architectures.

## Procedure

You are advised to install Kiran as the **root** user or a newly created administrator.

1. Download the openEuler 22.03 LTS SP4 ISO file and install the OS.

2. Update the software repository.

```shell
sudo dnf update
```

1. Install kiran-desktop.

```shell
sudo dnf install kiran-desktop
```

1. Set the system to start with the graphical interface, and then restart the system using the `reboot` command.

```shell
systemctl set-default graphical.target
```

After the reboot is complete, log in to the Kiran desktop.

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/kiran/kiran_user_guide.md
---
# Kiran桌面环境用户手册

## 1.概述

Kiran桌面是一款以用户和市场需求为主导的稳定、高效、易用的桌面环境，主要包括了桌面、任务栏、托盘、控制中心和窗口管理等组件。本文介绍了Kiran桌面的使用。

## 2.桌面

### 2.1.登录界面

安装完成后重启系统，系统启动后需要输入登录的用户名和密码才能进入系统，登录界面会显示时间日期，电源按钮，软键盘按钮。界面支持自适应调整，支持屏幕放缩，支持多屏显示，登录框可以跟随鼠标进行屏幕切换。

![图1-登录界面](figures/kiran-1.png)

### 2.2.主界面

输入正确的用户名和密码后即可登录系统进入主界面，如下图所示：

![图2-主界面](figures/kiran-2.png)

桌面上放置有几个图标，如计算机、主文件夹、回收站等，位于屏幕底部的一个长条称为面板，从这里可以启动应用程序或在模拟桌面上切换。
桌面是用户的工作区域，用户操作和程序运行都是在桌面上。桌面上还有用户希望能方便访问的文件和应用程序图标，用鼠标双击可以运行相应程序或打开文件。可以拖动、添加或删除桌面图标。使用桌面图标可以更加便捷地完成工作。

![图3-计算机](figures/kiran-3.png)计算机：双击可以显示从本计算机访问的所有本地和远程磁盘和文件夹。

![图4-主文件夹](figures/kiran-4.png)主文件夹：双击可以显示/root（家目录）下的内容。

![图5-回收站](figures/kiran-5.png)回收站：暂时存放已删除文件的地方。

桌面右键：提供了创建文件夹、创建启动器、创建文档、更改桌面背景、图标保持对齐等快捷方式。

创建文件夹：可以创建新的文件夹。

创建启动器：可以创建一个新的启动器。

创建文档：可以创建空的纯文本文档。

按名称组织桌面：按名称来进行排序桌面文件。

保持对齐：勾选了保持对齐，桌面图标会按照网格对齐排列。

打开终端：直接打开终端应用。

更改桌面背景：打开“背景”，以改变桌面或锁屏的背景图片。

### 2.3.面板

面板通常位于屏幕的底部，上面包括了开始菜单按钮、快速启动区域、经常使用的应用程序与桌面小程序图标和显示当前运行应用程序的任务条。
将鼠标停在某个图标上呆几秒钟，会看到一个白色的弹出提示框，内容是对这个图标作用的描述。

![图6-系统面板](figures/kiran-6.png)

## 3.任务栏

任务栏：显示正在运行的程序或打开的文档，点击任务条上某一项可以最大化或最小化被选中的程序。可以通过在对应项上点击鼠标右键对其运行窗口进行最大化、最小化或关闭等操作。

| 组件    | 说明 |
| :------------ | :------------ |
|![图7-开始菜单](figures/kiran-7.png)|开始菜单按钮：相当于Windows中的开始按钮，单击会弹出系统级联的开始菜单|
|![图8-工作区按钮](figures/kiran-8.png)|单机此按钮启动工作区|
|![图9-文件浏览器按钮](figures/kiran-9.png)|单击此按钮启动文件浏览器，可浏览管理文件|
|![图10-终端命令按钮](figures/kiran-10.png)|单击将启动终端|
|![图11-Web浏览器按钮](figures/kiran-11.png)|单击此按钮启动firefox浏览器|
|![图12-网络控制图标](figures/kiran-12.png)|显示当前网络状态，单击可修改系统的网络配置|
|![图13-时钟按钮](figures/kiran-13.png)|显示当前日期和时间，可以根据需要定制显示的样式|

## 4.控制中心

### 4.1.开始菜单设置

选择“开始菜单”>“控制中心”>“开始菜单设置”。
开始菜单可设置开始菜单样式，根据个人喜好设置开始菜单的显示模式与不透明度，如图所示：

![图14-开始菜单设置](figures/kiran-14.png)

开始菜单根据设置的透明度、显示模式而更改，如下图：

![图15-开始菜单界面](figures/kiran-15.png)

### 4.2.登录设置

选择“开始菜单”>“控制中心”>“登录设置”打开。
在kiran桌面，用户可以通过选择控制中心中的登录设置对登录界面环境效果进行设置，其中包括登录界面背景图、是否自动登录、缩放比例、是否允许手动输入用户名登录、是否显示用户列表等，如图所示：

![图16-登录设置](figures/kiran-16.png)

还可以设置自动登录，设置自动登录的用户和延时，重启系统后会自动登录该用户，无需输入密码。

![图17-设置自动登录](figures/kiran-17.png)

### 4.3.显示设置

定制显示属性是每个桌面系统所必备的，kiran桌面提供了强大的显示属性定制工具。您可以通过选择“开始菜单”>“控制中心”>“显示设置”进入显示设置界面，如下图所示：

![图18-显示设置](figures/kiran-18.png)

这里可以设置屏幕旋转、分辨率、刷新率、缩放率和旋转，设置完成后点击“应用”即可。

### 4.4.鼠标设置

用户可以通过选择控制中心中的“鼠标设置”对鼠标进行配置，可以修改鼠标手持模式为左手和右手模式，调整鼠标移动速度，可以设置是否自动滚动，是否同时按下左右键模拟中键功能。鼠标设置常见界面如图所示：

![图19-鼠标设置](figures/kiran-19.png)

### 4.5.账户管理工具

账户管理工具工具是对用户和组进行管理的一个简单易用工具，您可以通过这个工具对用户和组群进行配置和管理，主要包括：
1\)增加用户，设置用户属性；
2\)修改用户属性；
3\)显示用户属性；
4\)删除用户；
用户属性包括：账号、口令(密码)、登录 shell，用户组属性指在该组包含哪些用户。

#### 4.5.1.启动账户管理工具

在控制中心中选择启动“账户管理工具”选项即可启动账户管理工具，如图所示：

![图20-账户管理工具](figures/kiran-20.png)

在这个界面中您可以看到有左侧菜用户栏和右侧详细信息栏两个部分。目前在列出的是系统中的所有用户（除root用户除外）。点击左侧某个用户，详细信息栏将显示用户的基本信息（用户ID、用户类型等）。
点击“创建用户”，在右侧出现页面，如下图所示，按照要求填写您要添加的用户名、用户类型、设置密码、头像。填写完毕后，单击“创建”即完成添加。

![图21-创建账户](figures/kiran-21.png)

【注】：如果您已经设置了密码允许的最小位数(例如四位)，则您在此处输入的密码位数要不小于 4 位，否则系统将不会接受该密码。

单击头像区域打开头像修改功能，系统预设了各种类型的头像供用户选择，用户也可以自己添加头像，点击“确认”后记得保存：

![图22-修改头像](figures/kiran-22.png)

#### 4.5.2.删除用户

首先在左侧信息栏里的欲删除的用户上单击，选中该用户，然后在右侧工具栏上点击“删除”按钮，如下图所示：

![图23-删除用户](figures/kiran-23.png)

![图24-删除确认提示](figures/kiran-24.png)

在弹出上图所示的对话框中单击“否”撤消删除，单击“是”确认删除。

#### 4.5.3.高级设置

选择“创建新用户”>“输入账号密码”>“高级设置”，打开一个对话框，如下图所示，可以设置用户的登录shell、指定用户ID和指定用户目录。

![图25-高级设置](figures/kiran-25.png)

### 4.6.外观

定制显示属性是每个桌面系统所必备的，Kiran桌面为您提供了强大的显示属性定制工具。外观是一个对系统的桌面背景，主题，字体三个方面提供统一配置和管理的工具。
选择“开始菜单”>“控制中心”>“外观”，显示的界面如下图所示：

![图26-外观设置](figures/kiran-26.png)

#### 4.6.1.主题

主题可以对系统的对话框风格，菜单风格，系统面板风格，图标风格进行统一设置或者也可以根据用户的喜好定制。

1\)主题设置
系统中默认已提供了多套主题，可以在主题浏览对话框中浏览主题的相关信息。点击主题浏览对话框中的主题，即可设置系统主题，如图所示：
![图27-主题设置](figures/kiran-27.png)

2\)自定义主题
用户可以通过点击“自定义”按钮，来根据用户的喜好定制系统主题，如下图所示：自定义主题包括：
a.控制；
b.色彩；
c.窗口边框；
d.图标；
e.指针；

![图28-自定义主题](figures/kiran-28.png)

#### 4.6.2.背景

用户可以对桌面背景进行设置，可以修改颜色、样式。
1\)背景图片设置
如下图所示，点击壁纸文件浏览对话框中的壁纸，即可将桌面设置为此壁纸。

![图29-背景设置](figures/kiran-29.png)

2\)样式
用户可以根据自己的喜好通过样式下拉式选择框来调整壁纸填充桌面背景时的方式。填充方式有以下五种方式：

a.平铺；
b.缩放；
c.居中；
d.比例放大；
e.伸展；
f.适合宽度。

3\)壁纸的添加与删除
用户可以通过“添加”按钮添加自己喜欢的壁纸，如下图所示：

![图30-添加壁纸](figures/kiran-30.png)

点击“打开”即可添加壁纸。
同时可以点击“删除”按钮来删除用户不喜欢的壁纸。具体步骤：选择壁纸，点击“删除”。

4\)桌面背景色彩填充设置
用户如果不喜欢用壁纸来设置桌面背景，也可以用色彩来设置背景，在壁纸选择对话框中选择无壁纸选项，即可使用色彩来填充桌面背景。
填充色彩的方式有三种：

a.纯色；
b.水平梯度；
c.垂直梯度。

![图31-背景图片色彩填充](figures/kiran-31.png)

#### 4.6.3.字体

1\)字体设置
用户可以通过字体设置来设置系统图形界面的各种类型的字体，字体类型包括以下五种类型：

a.应用程序字体；
b.文档字体；
c.桌面字体；
d.窗口标题字体；
e.等宽字体。

![图32-字体设置](figures/kiran-32.png)

2\)字体效果设置与详情设置
字体渲染效果设置
用户可以通过字体渲染效果设置来设置系统图形界面的以下四种类型的字体效果：

a.单色；
b.最佳形状；
c.最佳对比；
d.次像素平滑；
系统默认使用的最佳形状的字体渲染效果，如下图所示：

![图33-字体渲染设置](figures/kiran-33.png)

3\)字体细节设置
字体效果的一些详情设置可以通过“细节”按钮进行设置。详情设置包括：

a.字体分辨率；
b.字体平滑度；
c.字体微调；
d.字体次像素排序。

![图34-字体细节设置](figures/kiran-34.png)

4\)用户可以设置界面，选择是否在菜单显示图标和在按钮中显示图标：

![图35-显示图标与否设置](figures/kiran-35.png)

## 5.桌面应用

### 5.1.文本编辑器

要启动文本编辑器，点击“开始菜单”>“所有应用”>“工具”>“pluma”。也可以在shell提示符下键入pluma启动文本编辑器。
文本编辑器是所有计算机系统中最常用的一种工具。用户在使用计算机时，往往需要创建自己的文件，无论是一般的文字文件、资料文件，还是编写源程序，这些工作都离不开编辑器。它用于查看和修改纯文本文件，纯文本文件是不包含应用字体或风格格式的普通文本文件，如系统日志和配置文件：

![图36-文本编辑器](figures/kiran-36.png)

### 5.2.终端

在桌面环境下，可以利用终端程序进入传统的命令操作界面，启动命令行终端的方法是：选择“开始菜单”>“所有应用”>“工具”>“终端”或者桌面面板上的图标：

![图37-终端](figures/kiran-37.png)

### 5.3.Firefox火狐浏览器

要启动Firefox火狐浏览器，点击“开始菜单”>“所有应用”>“互联网”>“Firefox火狐浏览器”。
Firefox火狐浏览器，是一个自由及开放源代码网页浏览器，使用Gecko排版引擎，支持多种操作系统，如Windows、Mac OS X及GNU/Linux等。它体积小速度快，还有其他一些高级特征，主要特性有：标签式浏览、使用网上冲浪更快、可以禁止弹出式窗口、自定制工具栏、扩展管理、更好的搜索特性、快速而方便的侧栏：

![图38-firefox浏览器](figures/kiran-38.png)

### 5.4.截图工具

选择“开始菜单”>“所有应用”>“图像”>“截图工具”，可以启动截图工具。
截图工具是Kiran桌面的一款小巧灵活的屏幕捕捉软件，操作界面简洁、使用极为方便。该软件启动时会在托盘处添加截图工具图标，如下图所示：

![图39-托盘区截图图标](figures/kiran-39.png)

点击该图标后，直接弹出屏幕捕捉界面，可自行选择截图范围。可通过有击该图标打开“打开启动器”，可选择需要抓取的范围是整个桌面，或者方形区域，可设置截图延迟时间，如下图所示：

![图40-截图界面](figures/kiran-40.png)

![图41-启动器界面](figures/kiran-41.png)

在弹出的对话框中，点击“√”，即可保存至桌面，如想自定义保存位置，点击“选项”>勾选“自定义保存位置”即可。如下图所示：

![图42-截图过程](figures/kiran-42.png)

### 5.5.网络设置

Kiran桌面采用了NetworkManager作为网络配置工具，NetworkManager是用来设定、配置和管理各种网络类型的桌面工具，NetworkManager提供了对移动宽带设备、蓝牙、IPv6 提供改进的支持。通过点击“开始菜单-控制中心-网络连接”打开，或者通过点击桌面右下角网络图标选择编辑连接打开，如图所示：

![图43-网络连接工具](figures/kiran-43.png)

设置有线连接：
设备选择当前的网卡，如“ens160”是当前系统的网卡，选中该网卡，点击“编辑”按钮，弹出网卡编辑对话框：

![图44-编辑网络](figures/kiran-44.png)

“IPv4设置”是用户常用到的设置，这里选择了DHCP的方式获取IP和DNS服务器，系统会自动给用户分配IP地址。
有些时候用户会碰到需要手动填写IP地址的情况，这就需要在IPv4设置的上方“方法”下拉菜单中选择“手动”，如下图所示：

![图45-设置IPV4](figures/kiran-45.png)

接下来点击“添加”按钮依次输入IP地址、子网掩码和网关，并填写DNS服务器，如下图：

![图46-设置网络ip和dns等](figures/kiran-46.png)

填写ip地址、子网掩码、网关和DNS后保存，点击桌面右下角网络图标断开网络后重新连接。

### 5.6.时间和日期管理

要对系统的日期和时间进行设置，您可以在控制中心中选择“时间和日期管理”选项，也可以通过点击桌面右下角日期区域，系统将弹出如图所示的界面：

![图47-时间和日期管理工具](figures/kiran-47.png)

自动同步日期和时间：打开“自动同步”并连接外网可以自动同步时间。
设置时区：点击“更改时区”按钮，右侧显示如下图所示时区设置对话框，点击需要更改的时区后保存即可修改时区。

![图48-修改时区](figures/kiran-48.png)

手动设置时间：关闭自动同步按钮，点击手动设置时间，可以手动调整年份、月份、日以及时间，修改完成后保存。

![图49-手动设置时间](figures/kiran-49.png)

修改日期格式：点击日期时间格式设置可以修改显示的日期格式，可以设置长日期显示格式、短日期显示格式、时间格式、以及是否显示秒：

![图50-修改日期格式](figures/kiran-50.png)

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/known_issues.md
---
# Known Issues

There is no known issue in this version.

---

---
url: /en/docs/22.03_LTS_SP4/edge_computing/kube_edge/kube_edge_deployment_guide.md
---
# KubeEdge Deployment Guide

## Description

### KubeEdge

KubeEdge is an open source system dedicated to solving problems in edge scenarios. It extends the capabilities of containerized application orchestration and device management to edge devices. Based on Kubernetes, KubeEdge provides core infrastructure support for networks, application deployment, and metadata synchronization between the cloud and the edge. KubeEdge supports MQTT and allows for custom logic to enable communication for the resource-constrained devices at the edge. KubeEdge consists of components deployed on the cloud and edge nodes. The components are now open source.

> <https://kubeedge.io/>

### iSulad

iSulad is a lightweight container runtime daemon designed for IoT and cloud infrastructure. It is lightweight, fast, and is not restricted by hardware specifications or architectures. It is suitable for wide application in various scenarios, such as cloud, IoT, and edge computing.

> <https://atomgit.com/openeuler/iSulad>

## Cluster Overview

### Component Versions

| Component  | Version                           |
| ---------- | --------------------------------- |
| OS         | openEuler 22.03 LTS SP4           |
| Kubernetes | 1.20.2-16                         |
| iSulad     | 2.1.2                             |
| KubeEdge   | v1.8.0                            |

### Node Planning Example

| Node           | Location | Components                       |
| -------------- | -------- | -------------------------------- |
| cloud.kubeedge | Cloud    | Kubernetes (Master), iSulad, CloudCore |
| edge.kubeedge  | Edge     | iSulad, EdgeCore                 |

> Note: You can run the `hostnamectl set-hostname [cloud,edge].kubeedge` command to set the cloud and edge node names in advance.

## Preparations

### Tool Package Download

[kubeedge-tools](https://gitee.com/Poorunga/kubeedge-tools) provides complete offline installation packages and deployment scripts for easy and quick KubeEdge cluster deployment even if the node cannot access the Internet.

```bash
# Download and decompress the kubeedge-tools package on both the cloud and edge nodes.
$ wget -O kubeedge-tools.zip https://gitee.com/Poorunga/kubeedge-tools/repository/archive/master.zip
$ unzip kubeedge-tools.zip

# Go to the kubeedge-tools directory for all the subsequent operations.
$ cd kubeedge-tools-master
```

### Kubernetes Deployment

Perform the following operations on the cloud node only.

#### Initializing the Cloud Environment

```bash
$ ./setup-cloud.sh
```

#### Deploying Kubernetes

Deploy Kubernetes by referring to the [Kubernetes Cluster Deployment Guide](https://docs.openeuler.org/en/docs/21.09/docs/Kubernetes/Kubernetes.html).

> Note: Preferentially, use `kubeadm` to deploy Kubernetes if the cloud node has access to the Internet. The procedure is as follows:

```bash
$ kubeadm init --apiserver-advertise-address=[cloud_node_IP_address] --kubernetes-version v1.20.15 --pod-network-cidr=10.244.0.0/16 --upload-certs --cri-socket=/var/run/isulad.sock
...
Your Kubernetes control-plane has initialized successfully!
...

# After Kubernetes is installed, copy the specified file to the directory as prompted.
#   mkdir -p $HOME/.kube
#   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
#   sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

#### Configuring Network for the Cloud Container

Container Network Interface (CNI) software that provides network for Kubernetes nodes include [flannel](https://github.com/flannel-io/flannel), [Calico](https://github.com/projectcalico/calico), [Cilium](https://github.com/cilium/cilium), and more. If you have not decided which CNI software to use, run the following command to configure network for the cloud container:

```bash
$ ./install-flannel-cloud.sh
```

#### Checking Deployment Status

```bash
# Check whether the node status is normal (Ready)
$ kubectl get nodes
NAME             STATUS   ROLES                  AGE   VERSION
cloud.kubeedge   Ready    control-plane,master   12m   v1.20.2

# Check whether the Kubernetes components are normal (Running)
$ kubectl get pods -n kube-system
NAME                                     READY   STATUS    RESTARTS   AGE
coredns-74ff55c5b-4ptkh                  1/1     Running   0          15m
coredns-74ff55c5b-zqx5n                  1/1     Running   0          15m
etcd-cloud.kubeedge                      1/1     Running   0          15m
kube-apiserver-cloud.kubeedge            1/1     Running   0          15m
kube-controller-manager-cloud.kubeedge   1/1     Running   0          15m
kube-flannel-cloud-ds-lvh4n              1/1     Running   0          13m
kube-proxy-2tcnn                         1/1     Running   0          15m
kube-scheduler-cloud.kubeedge            1/1     Running   0          15m
```

## Deployment

### CloudCore Deployment

Perform the following operations on the cloud node only.

#### Initializing the Cluster

```bash
# Set --advertise-address to the IP address of the cloud node.
$ keadm init --advertise-address="cloud_node_IP_address" --kubeedge-version=1.8.0
...
CloudCore started
```

#### Configuring CloudCore

```bash
$ ./patch-cloud.sh
```

#### Checking Deployment Status

```bash
# active (running) indicates a normal status
$ systemctl status cloudcore | grep running
     Active: active (running) since Fri 2023-05-20 10:54:30 CST; 5min ago
```

CloudCore has been deployed on the cloud node. Then, deploy EdgeCore on the edge node.

### EdgeCore Deployment

Perform the following operations only on the edge node unless otherwise specified.

#### Initializing the Edge Environment

```bash
$ ./setup-edge.sh
```

#### Managing the Edge Node

```bash
# Run the keadm gettoken command on the cloud node.
$ keadm gettoken
96058ab80ffbeb87fe58a79bfb19ea13f9a5a6c3076a17c00f80f01b406b4f7c.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDY0NDg4NzF9.1mJegWB7SUVjgf-OvAqILgbZXeMHR9eOzMxpNFc42SI
# Save this token for subsequent steps.

# Run the keadm join command on the edge node.
# Set --cloudcore-ipport to the IP address and port number (10000) of the cloud node. Set --token to the token saved in the previous step.
$ keadm join --cloudcore-ipport=clou_node_IP_address:10000 --kubeedge-version=1.8.0 --token=96058ab80ffbeb87fe58a79bfb19ea13f9a5a6c3076a17c00f80f01b406b4f7c.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDY0NDg4NzF9.1mJegWB7SUVjgf-OvAqILgbZXeMHR9eOzMxpNFc42SI
...
KubeEdge edgecore is running...
```

#### Configuring EdgeCore

```bash
$ ./patch-edge.sh
```

#### Configuring Network for the Edge Container

If you have not decided which CNI software to use, run the following command to configure network for the edge container:

```bash
# Run the command on the cloud node.
$ ./install-flannel-edge.sh
```

#### Checking Whether the Edge Node is Managed

```bash
# Run the command on the cloud node. You can see that the edge node is added.
$ kubectl get nodes
NAME             STATUS   ROLES                  AGE     VERSION
cloud.kubeedge   Ready    control-plane,master   1h      v1.20.2
edge.kubeedge    Ready    agent,edge             10m     v1.19.3-kubeedge-v1.8.0
```

The KubeEdge cluster has been deployed. Next, let's test the task delivery from the cloud to the edge.

### Application Deployment

Perform the following operations on the cloud node only.

#### Deploying Nginx

```bash
$ kubectl apply -f yamls/nginx-deployment.yaml
deployment.apps/nginx-deployment created

# Check whether Nginx is deployed on the edge node and running.
$ kubectl get pod -owide | grep nginx
nginx-deployment-84b99f4bf-jb6sz   1/1     Running   0          30s   10.244.1.2   edge.kubeedge   <none>           <none>
```

#### Testing the Function

```bash
# Access the IP address of Nginx on the edge node.
$ curl 10.244.1.2:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
```

The deployment of KubeEdge is complete.

---

---
url: /en/docs/22.03_LTS_SP4/edge_computing/kube_edge/kube_edge_user_document.md
---
# KubeEdge Usage Guide

KubeEdge extends the capabilities of Kubernetes to edge scenarios and provides infrastructure support for the network, application deployment, and metadata synchronization between the cloud and the edge. The usage of KubeEdge is the same as that of Kubernetes. In addition, KubeEdge supports the management and control of edge devices. The following example describes how to use KubeEdge to implement edge-cloud synergy.

## 1. Preparations

**Example: KubeEdge Counter Demo**

The counter is a pseudo device. You can run this demo without any additional physical devices. The counter runs on the edge side. You can use the web interface on the cloud side to control the counter and get the counter value.

For details, see <https://github.com/kubeedge/examples/tree/master/kubeedge-counter-demo>.

**1) This demo requires the KubeEdge v1.2.1 or later. In this example, the latest KubeEdge v1.8.0 is used.**

```shell
$ kubectl get node
NAME       STATUS   ROLES        AGE   VERSION
ke-cloud   Ready    master       13h   v1.20.2
ke-edge1   Ready    agent,edge   64s   v1.19.3-kubeedge-v1.8.0

Note: In this document, the edge node ke-edge1 is used for verification. If you perform verification by referring to this document, you need to change the edge node name based on your actual deployment.
```

**2) Ensure that the following configuration items are enabled for the Kubernetes API server:**

```shell
--insecuret-port=8080
--insecure-bind-address=0.0.0.0
```

You can modify the `/etc/kubernetes/manifests/kube-apiserver.yaml` file, and then restart the Pod of the Kubernetes API server component to make the modifications take effect.

**3) Install the Go language.**

```shell
[root@ke-cloud ~]# wget https://golang.google.cn/dl/go1.14.4.linux-amd64.tar.gz
[root@ke-cloud ~]# tar -zxvf go1.14.4.linux-amd64.tar.gz -C /usr/local
```

**4) Configure the Go environment.**

```shell
[root@ke-cloud ~]# vim /etc/profile
```

Add the following to the end of the file:

```shell
export GOROOT=/usr/local/go
export GOPATH=/data/gopath
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
```

**5) Apply the modifications.**

```shell
[root@ke-cloud ~]# source /etc/profile
[root@ke-cloud ~]# mkdir -p /data/gopath && cd /data/gopath
[root@ke-cloud ~]# mkdir -p src pkg bin
```

**6) Download the sample code:**

```shell
git clone https://github.com/kubeedge/examples.git $GOPATH/src/github.com/kubeedge/examples
```

## 2. Creating the Device Model and Device

**1) Create the device model.**

```shell
cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds
kubectl create -f kubeedge-counter-model.yaml
```

**2) Create the device.**

Modify **matchExpressions** as required.

```shell
$ cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds
$ vim kubeedge-counter-instance.yaml
apiVersion: devices.kubeedge.io/v1alpha1
kind: Device
metadata:
  name: counter
  labels:
    description: 'counter'
    manufacturer: 'test'
spec:
  deviceModelRef:
    name: counter-model
  nodeSelector:
    nodeSelectorTerms:
    - matchExpressions:
      - key: 'kubernetes.io/hostname'
        operator: In
        values:
        - ke-edge1

status:
  twins:
    - propertyName: status
      desired:
        metadata:
          type: string
        value: 'OFF'
      reported:
        metadata:
          type: string
        value: '0'

$ kubectl create -f kubeedge-counter-instance.yaml
```

## 3. Deploying the Cloud Application

**1) Modify the code.**

The cloud application **web-controller-app** controls the edge application **pi-counter-app**. The default listening port of the cloud application is 80. Change the port number to 8089.

```shell
$ cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/web-controller-app
$ vim main.go
package main

import (
        "github.com/astaxie/beego"
        "github.com/kubeedge/examples/kubeedge-counter-demo/web-controller-app/controller"
)

func main() {
        beego.Router("/", new(controllers.TrackController), "get:Index")
        beego.Router("/track/control/:trackId", new(controllers.TrackController), "get,post:ControlTrack")

        beego.Run(":8089")
}
```

**2) Build the image.**

Note: When building the image, copy the source code to the path specified by **GOPATH**. Disable Go modules if they are enabled.

```shell
make all
make docker
```

**3) Deploy web-controller-app.**

```shell
cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds
kubectl apply -f kubeedge-web-controller-app.yaml
```

## 4. Deploying the Edge Application

The **pi-counter-app** application on the edge is controlled by the cloud application. The edge application communicates with the MQTT server to perform simple counting.

**1) Modify the code and build the image.**

Change the value of **GOARCH** to **amd64** in `Makefile` to run the container.

```shell
$ cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/counter-mapper
$ vim Makefile
.PHONY: all pi-execute-app docker clean
all: pi-execute-app

pi-execute-app:
        GOARCH=amd64 go build -o pi-counter-app main.go

docker:
        docker build . -t kubeedge/kubeedge-pi-counter:v1.0.0

clean:
        rm -f pi-counter-app

$ make all
$ make docker
```

**2) Deploy pi-counter-app.**

```shell
$ cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds
$ kubectl apply -f kubeedge-pi-counter-app.yaml

Note: To prevent Pod deployment from being stuck at `ContainerCreating`, run the docker save, scp, and docker load commands to release the image to the edge.

$ docker save -o kubeedge-pi-counter.tar kubeedge/kubeedge-pi-counter:v1.0.0
$ scp kubeedge-pi-counter.tar root@192.168.1.56:/root
$ docker load -i kubeedge-pi-counter.tar
```

## 5. Trying the Demo

Now, the KubeEdge Demo is deployed on the cloud and edge as follows:

```shell
$ kubectl get pods -o wide
NAME                                    READY   STATUS    RESTARTS   AGE     IP             NODE       NOMINATED NODE   READINESS GATES
kubeedge-counter-app-758b9b4ffd-f8qjj   1/1     Running   0          26m     192.168.1.66   ke-cloud   <none>           <none>
kubeedge-pi-counter-c69698d6-rb4xz      1/1     Running   0          2m      192.168.1.56   ke-edge1   <none>           <none>
```

Let's test the running effect of the Demo.

**1) Execute the ON command.**
On the web page, select **ON** and click **Execute**. You can run the following command on the edge node to view the execution result:

```shell
docker logs -f counter-container-id
```

![](./figures/en-us_image_1706077646.png)/figures/en-us\_image\_1706077646.png

![](./figures/en-us_image_1706077675.png)

![en-us\_image\_1706077688](./figures/en-us_image_1706077688.png)

**2) Check the counter's STATUS.**
On the web page, select **STATUS** and click **Execute**. The current counter status is displayed on the web page.

![en-us\_image\_1706077702](./figures/en-us_image_1706077702.png)

**3) Execute the OFF command.**
On the web page, select **OFF** and click **Execute**. You can run the following command on the edge node to view the execution result:

```shell
docker logs -f counter-container-id
```

![zh-cn\_image\_1706077716](./figures/en-us_image_1706077716.png)

![zh-cn\_image\_1706077729](./figures/en-us_image_1706077729.png)

## 6. Others

**1) For more official KubeEdge examples, visit <https://github.com/kubeedge/examples>.**

| Name                                                                                                                               | Description                                                                                   |
| ---------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| [LED-RaspBerry-Pi](https://github.com/kubeedge/examples/blob/master/led-raspberrypi/README.md)                                     | Controlling a LED light with Raspberry Pi using KubeEdge platform                             |
| [Data Analysis @ Edge](https://github.com/kubeedge/examples/blob/master/apache-beam-analysis/README.md)                            | Analyzing data at edge by using Apache Beam and KubeEdge                                      |
| [Security@Edge](https://github.com/kubeedge/examples/blob/master/security-demo/README.md)                                          | Security at edge using SPIRE for identity management                                          |
| [Bluetooth-CC2650-demo](https://github.com/kubeedge/examples/blob/master/bluetooth-CC2650-demo/README.md)                          | Controlling a CC2650 SensorTag bluetooth device using KubeEdge platform                       |
| [Play Music @Edge through WeChat](https://github.com/kubeedge/examples/blob/master/wechat-demo/README.md)                          | Play music at edge based on WeChat and KubeEdge                                               |
| [Play Music @Edge through Web](https://github.com/kubeedge/examples/blob/master/web-demo/README.md)                                | Play music at edge based on Web and KubeEdge                                                  |
| [Collecting temperature @Edge](https://github.com/kubeedge/examples/blob/master/temperature-demo/README.md)                        | Collecting temperature at edge based KubeEdge                                                 |
| [Control pseudo device counter and collect data](https://github.com/kubeedge/examples/blob/master/kubeedge-counter-demo/README.md) | Control pseudo device counter and collect data based KubeEdge                                 |
| [Play Music @Edge through Twitter](https://github.com/kubeedge/examples/blob/master/ke-twitter-demo/README.md)                     | Play music at edge based on Twitter and KubeEdge.                                             |
| [Control Zigbee @Edge through cloud](https://github.com/kubeedge/examples/blob/master/kubeedge-edge-ai-application/README.md)      | Object detection at cloud using OpenCV and using it to control zigbee on edge using Kubeedge. |

**2) Use EdgeMesh to discover edge services.**

<https://github.com/kubeedge/edgemesh>

**3) Customize the cloud-edge message route.**

<https://kubeedge.io/en/docs/developer/custom_message_deliver/>

---

---
url: /zh/docs/22.03_LTS_SP4/edge_computing/kube_edge/kube_edge_deployment_guide.md
---
# KubeEdge 部署指南

## 介绍

### KubeEdge

KubeEdge 是一个致力于解决边缘场景问题的开源系统，它将容器化应用程序编排和设备管理的能力扩展到边缘设备。基于 Kubernetes，KubeEdge 为网络、应用程序部署以及云侧与边缘侧之间的元数据同步提供核心基础设施支持。KubeEdge 支持 MQTT，并允许开发人员编写自定义逻辑，在边缘上启用资源受限的设备通信。KubeEdge 由云部分和边缘部分组成，目前均已开源。

> <https://kubeedge.io>

### iSulad

iSulad 是一个轻量级容器 runtime 守护程序，专为 IOT 和 Cloud 基础设施而设计，具有轻便、快速且不受硬件规格和体系结构限制的特性，可以被更广泛地应用在云、IoT、边缘计算等多个场景。

> <https://atomgit.com/openeuler/iSulad>

## 集群概览

### 组件版本

| 组件       | 版本                               |
| ---------- | --------------------------------- |
| OS         | openEuler 22.03 LTS SP4           |
| Kubernetes | 1.20.2-16                         |
| iSulad     | 2.1.2                             |
| KubeEdge   | v1.8.0                            |

### 节点规划（示例）

| 节点名          | 位置          | 组件                           |
| -------------- | ------------ | ------------------------------ |
| cloud.kubeedge | 云侧（cloud） | k8s（master）、isulad、cloudcore |
| edge.kubeedge  | 边缘侧（edge） | isulad、edgecore               |

> 提示：云侧和边缘侧的主机名可以使用 `hostnamectl set-hostname [cloud,edge].kubeedge` 命令提前设置好

## 准备

### 下载工具包

[kubeedge-tools](https://gitee.com/Poorunga/kubeedge-tools) 工具包提供了完备的离线安装包以及部署脚本，降低了部署复杂度并且支持在节点无法访问外网的条件下快速搭建 KubeEdge 集群。

```bash
# 下载 kubeedge-tools 工具包并解压（包括云侧和边缘侧）
$ wget -O kubeedge-tools.zip https://gitee.com/Poorunga/kubeedge-tools/repository/archive/master.zip
$ unzip kubeedge-tools.zip

# 进入 kubeedge-tools 工具包目录（后续所有操作基于此目录）
$ cd kubeedge-tools-master
```

### 部署 k8s

以下操作仅在云侧执行

#### 初始化云侧环境

```bash
$ ./setup-cloud.sh
```

#### 参考 [Kubernetes 集群部署指南](https://docs.openeuler.org/zh/docs/21.09/docs/Kubernetes/Kubernetes.html) 部署 k8s

> 提示：在云侧节点可以访问外网的条件下建议优先选用 `kubeadm` 工具部署 k8s，示例：

```bash
$ kubeadm init --apiserver-advertise-address=云侧IP --kubernetes-version v1.20.15 --pod-network-cidr=10.244.0.0/16 --upload-certs --cri-socket=/var/run/isulad.sock
...
Your Kubernetes control-plane has initialized successfully!
...

# 成功安装 k8s 后还需根据最后的提示将指定文件复制到指定目录：
#   mkdir -p $HOME/.kube
#   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
#   sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

#### 安装云侧容器网络

目前有丰富的 cni 软件可以为 k8s 节点提供容器网络功能，比如 [flannel](https://github.com/flannel-io/flannel)、[calico](https://github.com/projectcalico/calico)、[cilium](https://github.com/cilium/cilium) 等，如果你暂时不明确选用哪款 cni 软件，可以使用下方命令安装云侧容器网络：

```bash
$ ./install-flannel-cloud.sh
```

#### 检查部署情况

```bash
# 查看节点状态（Ready 即正常）
$ kubectl get nodes
NAME             STATUS   ROLES                  AGE   VERSION
cloud.kubeedge   Ready    control-plane,master   12m   v1.20.2

# 查看所有 k8s 组件运行状态（Running 即正常）
$ kubectl get pods -n kube-system
NAME                                     READY   STATUS    RESTARTS   AGE
coredns-74ff55c5b-4ptkh                  1/1     Running   0          15m
coredns-74ff55c5b-zqx5n                  1/1     Running   0          15m
etcd-cloud.kubeedge                      1/1     Running   0          15m
kube-apiserver-cloud.kubeedge            1/1     Running   0          15m
kube-controller-manager-cloud.kubeedge   1/1     Running   0          15m
kube-flannel-cloud-ds-lvh4n              1/1     Running   0          13m
kube-proxy-2tcnn                         1/1     Running   0          15m
kube-scheduler-cloud.kubeedge            1/1     Running   0          15m
```

## 部署

### 部署 cloudcore

以下操作仅在云侧执行

#### 初始化集群

```bash
# --advertise-address 填写云侧节点的主机 IP 地址
$ keadm init --advertise-address="云侧IP" --kubeedge-version=1.8.0
...
CloudCore started
```

#### 调整 cloudcore 配置

```bash
$ ./patch-cloud.sh
```

#### 检查部署情况

```bash
# active (running)即正常
$ systemctl status cloudcore | grep running
     Active: active (running) since Fri 2023-05-20 10:54:30 CST; 5min ago
```

至此，云侧的 cloudcore 已部署完成，接下来部署边缘侧 edgecore。

### 部署 edgecore

以下命令如无特殊说明则仅在边缘侧执行

#### 初始化边缘侧环境

```bash
$ ./setup-edge.sh
```

#### 纳管边缘节点

```bash
# keadm gettoken 命令需要在云侧执行
$ keadm gettoken
96058ab80ffbeb87fe58a79bfb19ea13f9a5a6c3076a17c00f80f01b406b4f7c.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDY0NDg4NzF9.1mJegWB7SUVjgf-OvAqILgbZXeMHR9eOzMxpNFc42SI
# 记录并保存此 token 值，后续步骤需要使用


# keadm join 命令在边缘侧执行
# --cloudcore-ipport 填写云侧节点的主机 IP 地址:10000，--token 填写上方 token 值
$ keadm join --cloudcore-ipport=云侧IP:10000 --kubeedge-version=1.8.0 --token=96058ab80ffbeb87fe58a79bfb19ea13f9a5a6c3076a17c00f80f01b406b4f7c.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDY0NDg4NzF9.1mJegWB7SUVjgf-OvAqILgbZXeMHR9eOzMxpNFc42SI
...
KubeEdge edgecore is running...
```

#### 调整 edgecore 配置

```bash
$ ./patch-edge.sh
```

#### 安装边缘侧容器网络

如果你暂时不明确选用哪款 cni 软件，可以使用下方命令安装边缘侧容器网络：

```bash
# 下方命令需要在云侧执行
$ ./install-flannel-edge.sh
```

#### 检查边缘节点是否纳管成功

```bash
# 下方命令需要在云侧执行（发现已经有了边缘节点）
$ kubectl get nodes
NAME             STATUS   ROLES                  AGE     VERSION
cloud.kubeedge   Ready    control-plane,master   1h      v1.20.2
edge.kubeedge    Ready    agent,edge             10m     v1.19.3-kubeedge-v1.8.0
```

至此，KubeEdge 集群部署完成，接下来我们测试一下从云侧下发应用到边缘侧。

### 部署应用

以下命令在云侧执行

#### 部署nginx

```bash
$ kubectl apply -f yamls/nginx-deployment.yaml
deployment.apps/nginx-deployment created

# 查看应用是否部署到了边缘侧（Running 即正常）
$ kubectl get pod -owide | grep nginx
nginx-deployment-84b99f4bf-jb6sz   1/1     Running   0          30s   10.244.1.2   edge.kubeedge   <none>           <none>
```

#### 测试功能

```bash
# 进入边缘侧节点，访问 nginx 应用
$ curl 10.244.1.2:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
```

至此，KubeEdge 部署已经全流程打通。

---

---
url: /zh/docs/22.03_LTS_SP4/edge_computing/kube_edge/kube_edge_user_document.md
---
# KubeEdge使用文档

KubeEdge将Kubernetes的能力延伸到了边缘场景中，为云和边缘之间的网络，应用部署和元数据同步提供基础架构支持。KubeEdge在使用上与Kubernetes保持完全一致，除此之外还扩展了对边缘设备的管理与控制。本节将通过一个简单的例子向用户演示如何通过KubeEdge完成设备边云协同任务。

## 1. 准备工作

**选用示例：KubeEdge Counter Demo**

计数器是一个伪设备，用户无需任何额外的物理设备即可运行此演示。计数器在边缘侧运行，用户可以从云侧在Web中对其进行控制，也可以从云侧在Web中获得计数器值。

详细文档参考：<https://github.com/kubeedge/examples/tree/master/kubeedge-counter-demo>

**1）本示例要求KubeEdge版本必须是v1.2.1+，此次选择最新版的KubeEdge v1.8.0**

```sh
[root@ke-cloud ~]# kubectl get node
NAME       STATUS   ROLES        AGE   VERSION
ke-cloud   Ready    master       13h   v1.20.2
ke-edge1   Ready    agent,edge   64s   v1.19.3-kubeedge-v1.8.0

说明：本文接下来的验证将使用边缘节点ke-edge1进行，如果你参考本文进行相关验证，后续边缘节点名称的配置需要根据你的实际情况进行更改。
```

**2）确保k8s apiserver开启了以下配置：**

```shell
--insecure-port=8080
--insecure-bind-address=0.0.0.0
```

可以通过修改/etc/kubernetes/manifests/kube-apiserver.yaml文件，并重启k8s-apiserver组件的pod来进行更改。

**3）安装golang：**

```sh
[root@ke-cloud ~]# wget https://golang.google.cn/dl/go1.14.4.linux-amd64.tar.gz
[root@ke-cloud ~]# tar -zxvf go1.14.4.linux-amd64.tar.gz -C /usr/local
```

**4）配置golang环境：**

```sh
[root@ke-cloud ~]# vim /etc/profile
```

文件末尾添加：

```sh
export GOROOT=/usr/local/go
export GOPATH=/data/gopath
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
```

**5）应用改变：**

```sh
[root@ke-cloud ~]# source /etc/profile
[root@ke-cloud ~]# mkdir -p /data/gopath && cd /data/gopath
[root@ke-cloud ~]# mkdir -p src pkg bin
```

**6）下载示例代码：**

```sh
[root@ke-cloud ~]# git clone https://github.com/kubeedge/examples.git $GOPATH/src/github.com/kubeedge/examples
```

## 2. 创建device model和device

**1）创建device model**

```sh
[root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds
[root@ke-cloud crds~]# kubectl create -f kubeedge-counter-model.yaml
```

**2）创建device**

根据你的实际情况修改matchExpressions：

```sh
[root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds
[root@ke-cloud crds~]# vim kubeedge-counter-instance.yaml
apiVersion: devices.kubeedge.io/v1alpha1
kind: Device
metadata:
  name: counter
  labels:
    description: 'counter'
    manufacturer: 'test'
spec:
  deviceModelRef:
    name: counter-model
  nodeSelector:
    nodeSelectorTerms:
    - matchExpressions:
      - key: 'kubernetes.io/hostname'
        operator: In
        values:
        - ke-edge1

status:
  twins:
    - propertyName: status
      desired:
        metadata:
          type: string
        value: 'OFF'
      reported:
        metadata:
          type: string
        value: '0'

[root@ke-cloud crds~]# kubectl create -f kubeedge-counter-instance.yaml
```

## 3. 部署云端应用

**1）修改代码**

云端应用web-controller-app用来控制边缘端的pi-counter-app应用，该程序默认监听的端口号为80，此处修改为8089，如下所示：

```sh
[root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/web-controller-app
[root@ke-cloud web-controller-app~]# vim main.go
package main

import (
        "github.com/astaxie/beego"
        "github.com/kubeedge/examples/kubeedge-counter-demo/web-controller-app/controller"
)

func main() {
        beego.Router("/", new(controllers.TrackController), "get:Index")
        beego.Router("/track/control/:trackId", new(controllers.TrackController), "get,post:ControlTrack")

        beego.Run(":8089")
}
```

**2）构建镜像**

注意：构建镜像时，请将源码拷贝到GOPATH对应的路径下，如果开启了go mod请关闭。

```sh
[root@ke-cloud web-controller-app~]# make all
[root@ke-cloud web-controller-app~]# make docker
```

**3）部署web-controller-app**

```sh
[root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds
[root@ke-cloud crds~]# kubectl apply -f kubeedge-web-controller-app.yaml
```

## 4. 部署边缘端应用

边缘端的pi-counter-app应用受云端应用控制，主要与mqtt服务器通信，进行简单的计数功能。

**1）修改代码与构建镜像**

需要将Makefile中的GOARCH修改为amd64才能运行该容器。

```sh
[root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/counter-mapper
[root@ke-cloud counter-mapper~]# vim Makefile
.PHONY: all pi-execute-app docker clean
all: pi-execute-app

pi-execute-app:
        GOARCH=amd64 go build -o pi-counter-app main.go

docker:
        docker build . -t kubeedge/kubeedge-pi-counter:v1.0.0

clean:
        rm -f pi-counter-app

[root@ke-cloud counter-mapper~]# make all
[root@ke-cloud counter-mapper~]# make docker
```

**2）部署Pi Counter App**

```sh
[root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds
[root@ke-cloud crds~]# kubectl apply -f kubeedge-pi-counter-app.yaml

说明：为了防止Pod的部署卡在`ContainerCreating`，这里直接通过docker save、scp和docker load命令将镜像发布到边缘端

[root@ke-cloud ~]# docker save -o kubeedge-pi-counter.tar kubeedge/kubeedge-pi-counter:v1.0.0
[root@ke-cloud ~]# scp kubeedge-pi-counter.tar root@192.168.1.56:/root
[root@ke-edge1 ~]# docker load -i kubeedge-pi-counter.tar
```

## 5. 体验Demo

现在，KubeEdge Demo的云端部分和边缘端的部分都已经部署完毕，如下：

```sh
[root@ke-cloud ~]# kubectl get pods -o wide
NAME                                    READY   STATUS    RESTARTS   AGE     IP             NODE       NOMINATED NODE   READINESS GATES
kubeedge-counter-app-758b9b4ffd-f8qjj   1/1     Running   0          26m     192.168.1.66   ke-cloud   <none>           <none>
kubeedge-pi-counter-c69698d6-rb4xz      1/1     Running   0          2m      192.168.1.56   ke-edge1   <none>           <none>
```

我们现在开始测试一下该Demo运行效果：

**1）执行ON命令**

在web页面上选择ON，并点击Execute，可以在edge节点上通过以下命令查看执行结果：

```sh
[root@ke-edge1 ~]# docker logs -f counter-container-id
```

![](./figures/zh-cn_image_1706077646.png)

![](./figures/zh-cn_image_1706077675.png)

![zh-cn\_image\_1706077688](./figures/zh-cn_image_1706077688.png)

**2）查看counter STATUS**

在web页面上选择STATUS，并点击Execute，会在Web页面上返回counter当前的status，如下所示：

![zh-cn\_image\_1706077702](./figures/zh-cn_image_1706077702.png)

**3）执行OFF命令**

在web页面上选择OFF，并点击Execute，可以在edge节点上通过以下命令查看执行结果：

```sh
[root@ke-edge1 ~]# docker logs -f counter-container-id
```

![zh-cn\_image\_1706077716](./figures/zh-cn_image_1706077716.png)

![zh-cn\_image\_1706077729](./figures/zh-cn_image_1706077729.png)

## 6. 其他

**1）更多的KubeEdge官方示例请参考 <https://github.com/kubeedge/examples>**

|名称 | 描述 |
|---|---|
|  [LED-RaspBerry-Pi](https://github.com/kubeedge/examples/blob/master/led-raspberrypi/README.md)     |Controlling a LED light with Raspberry Pi using KubeEdge platform
|[Data Analysis @ Edge](https://github.com/kubeedge/examples/blob/master/apache-beam-analysis/README.md) | Analyzing data at edge by using Apache Beam and KubeEdge
| [Security@Edge](https://github.com/kubeedge/examples/blob/master/security-demo/README.md) | Security at edge using SPIRE for identity management
[Bluetooth-CC2650-demo](https://github.com/kubeedge/examples/blob/master/bluetooth-CC2650-demo/README.md)     |Controlling a CC2650 SensorTag bluetooth device using KubeEdge platform
| [Play Music @Edge through WeChat](https://github.com/kubeedge/examples/blob/master/wechat-demo/README.md) | Play music at edge based on WeChat and KubeEdge
| [Play Music @Edge through Web](https://github.com/kubeedge/examples/blob/master/web-demo/README.md) | Play music at edge based on Web and KubeEdge
| [Collecting temperature @Edge](https://github.com/kubeedge/examples/blob/master/temperature-demo/README.md) | Collecting temperature at edge based KubeEdge
| [Control pseudo device counter and collect data](https://github.com/kubeedge/examples/blob/master/kubeedge-counter-demo/README.md) | Control pseudo device counter and collect data based KubeEdge
[Play Music @Edge through Twitter](https://github.com/kubeedge/examples/blob/master/ke-twitter-demo/README.md)| Play music at edge based on Twitter and KubeEdge.
[Control Zigbee @Edge through cloud](https://github.com/kubeedge/examples/blob/master/kubeedge-edge-ai-application/README.md) | Object detection at cloud using OpenCV and using it to control zigbee on edge using Kubeedge.

**2）使用EdgeMesh做边缘服务发现**

<https://github.com/kubeedge/edgemesh>

**3）自定义云边消息路由**

<https://kubeedge.io/en/docs/developer/custom_message_deliver>

---

---
url: /en/docs/22.03_LTS_SP4/cloud/kubeos/kubeos/kubeos_image_creation.md
---
# KubeOS Image Creation

## Introduction

kbimg is an image creation tool required for KubeOS deployment and upgrade. You can use kbimg to create KubeOS Docker, VM, and physical machine images.

## Commands

### Command Format

**bash kbimg.sh** \[ --help | -h ] create \[ COMMANDS ]  \[ OPTIONS ]

### Parameter Description

* COMMANDS

  | Parameter         | Description                                          |
  | ------------- | ---------------------------------------------- |
  | upgrade-image | Generates a Docker image for installation and upgrade.|
  | vm-image      | Generates a VM image for installation and upgrade.                |
  | pxe-image     | Generates images and files required for physical machine installation.                |

* OPTIONS

  | Option        | Description                                                        |
  | ------------ | ------------------------------------------------------------ |
  | -p           | Path of the repo file. The Yum source required for creating an image is configured in the repo file.       |
  | -v           | Version of the created KubeOS image.                                  |
  | -b           | Path of the os-agent binary file.                                        |
  | -e           | Password of the **root** user of the KubeOS image, which is an encrypted password with a salt value. You can run the OpenSSL or KIWI command to generate the password.|
  | -d           | Generated or used Docker image.                                    |
  | -h  --help | Help Information.                                                |

## Usage Description

### Precautions

* The root permission is required for executing **kbimg.sh**.
* Currently, only the x86 and AArch64 architectures are supported.
* The RPM sources of the kbimg are the **everything** and **EPOL** repositories of openEuler of a specific version. In the Repo file provided during image creation, you are advised to configure the **everything** and **EPOL** repositories of a specific openEuler version for the Yum source.

### Creating a KubeOS Docker Image

#### Precautions

* The created Docker image can be used only for subsequent VM or physical machine image creation or upgrade. It cannot be used to start containers.
* If the default RPM list is used to create a KubeOS image, at least 6 GB drive space is required. If the RPM list is customized, the occupied drive space may exceed 6 GB.

#### Example

* To configure the DNS, customize the `resolv.conf` file in the `scripts` directory.

```shell
  cd /opt/kubeOS/scripts
  touch resolv.conf
  vim resolv.conf
```

* Create a KubeOS image.

```shell
cd /opt/kubeOS/scripts
bash kbimg.sh create upgrade-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0''' -d your_imageRepository/imageName:version 
```

* After the creation is complete, view the created KubeOS image.

```shell
docker images
```

### Creating a KubeOS VM Image

#### Precautions

* To use a Docker image to create a KubeOS VM image, pull the corresponding image or create a Docker image first and ensure the security of the Docker image.
* The created KubeOS VM image can be used only in a VM of the x86 or AArch64 architecture.
* Currently, KubeOS does not support legacy boot in an x86 VM.
* If the default RPM list is used to create a KubeOS image, at least 25 GB drive space is required. If the RPM list is customized, the occupied drive space may exceed 25 GB.

#### Example

* Using the Repo Source

  * To configure the DNS, customize the `resolv.conf` file in the `scripts` directory.

  ```shell
  cd /opt/kubeOS/scripts
  touch resolv.conf
  vim resolv.conf
  ```

  * Create a KubeOS VM image.

  ```shell
  cd /opt/kubeOS/scripts
  bash kbimg.sh create vm-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0'''
  ```

* Using a Docker Image

  ```shell
  cd /opt/kubeOS/scripts
  bash kbimg.sh create vm-image -d  your_imageRepository/imageName:version
  ```

* Result Description\
  After the KubeOS image is created, the following files are generated in the **/opt/kubeOS/scripts** directory:
  * **system.qcow2**: system image in QCOW2 format. The default size is 20 GiB. The size of the root file system partition is less than 2,020 MiB, and the size of the Persist partition is less than 16 GiB.
  * **update.img**: partition image of the root file system used for upgrade.

### Creating Images and Files Required for Installing KubeOS on Physical Machines

#### Precautions

* To use a Docker image to create a KubeOS VM image, pull the corresponding image or create a Docker image first and ensure the security of the Docker image.
* The created image can only be used to install KubeOS on a physical machine of the x86 or AArch64 architecture.
* The IP address specified in the **Global.cfg** file is a temporary IP address used during installation. After the system is installed and started, configure the network by referring to **openEuler 22.03 LTS SP1 Administrator Guide** > **Configuring the Network**.
* KubeOS cannot be installed on multiple drives at the same time. Otherwise, the startup may fail or the mounting may be disordered.
* Currently, KubeOS does not support legacy boot in an x86 physical machine.
* If the default RPM list is used to create a KubeOS image, at least 5 GB drive space is required. If the RPM list is customized, the occupied drive space may exceed 5 GB.

#### Example

* Modify the `00bootup/Global.cfg` file. All parameters are mandatory. Currently, only IPv4 addresses are supported. The following is a configuration example:

  ```shell
  # rootfs file name
  rootfs_name=kubeos.tar
  # select the target disk to install kubeOS
  disk=/dev/sda
  # pxe server ip address where stores the rootfs on the http server
  server_ip=192.168.1.50
  # target machine temporary ip
  local_ip=192.168.1.100
  # target machine temporary route
  route_ip=192.168.1.1
  # target machine temporary netmask
  netmask=255.255.255.0
  # target machine netDevice name
  net_name=eth0
  ```

* Using the Repo Source

  * To configure the DNS, customize the `resolv.conf` file in the `scripts` directory.

  ```shell
    cd /opt/kubeOS/scripts
    touch resolv.conf
    vim resolv.conf
  ```

  * Create an image required for installing KubeOS on a physical machine.

  ```shell
    cd /opt/kubeOS/scripts
    bash kbimg.sh create pxe-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0'''
  ```

* Using a Docker Image

  ```shell
  cd /opt/kubeOS/scripts
  bash kbimg.sh create pxe-image -d your_imageRepository/imageName:version
  ```

* Result Description

  * **initramfs.img**: initramfs image used for boot from PXE.
  * **kubeos.tar**: OS used for installation from PXE.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/kubeos/kubeos/overview.md
---
# KubeOS Overview

This document describes how to install, deploy, and use KubeOS in the openEuler system. KubeOS connects the container OS to the scheduling system in standard extension pattern and manages the OS upgrade of nodes in the cluster through the scheduling system.

This document is intended for community developers, open source enthusiasts, and partners who use the openEuler system and want to learn and use the container OSs. Users must:

* Know basic Linux operations.
* Understand Kubernetes and Docker.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/overview.md
---
# Kubernetes Cluster Deployment Guide

This document describes how to deploy a Kubernetes cluster in binary mode on openEuler.

> \[!NOTE] **Note:**
> All operations in this document are performed using **root** permissions.

## Cluster Status

The cluster status used in this document is as follows:

* Cluster structure: six VMs running the openEuler 22.03 LTS SP4 OS, three master nodes, and three nodes.
* Physical machine: x86/Arm server running openEuler 22.03 LTS SP4.

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/overview.md
---
# Kubernetes 集群部署指南

本文档介绍在 openEuler 操作系统上，通过二进制部署 K8S 集群的一个参考方法。

说明：本文所有操作均使用 `root`权限执行。

## 集群状态

本文所使用的集群状态如下：

* 集群结构：6 个 openEuler 系统的虚拟机，3 个 master 和 3 个 node 节点
* 物理机：openEuler 的 `x86/ARM`架构服务器

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/isulad_k8s/isulad_k8s_environment_deploy.md
---
# Kubernetes+iSulad Environment Deployment

## Preparing Cluster Servers

Prepare at least 3 machines running openEuler 20.03 LTS or later versions. The following table lists information about the machines.

| Host Name | IP Address          | OS                 | Role      | Component       |
|-------|-------------|------------------------|----------|-----------|
| lab1  | 197.xxx.xxx.xxx | openEuler 20.03 LTS SP4 | Control node  | iSulad/Kubernetes |
| lab2  | 197.xxx.xxx.xxx | openEuler 20.03 LTS SP4 | Worker node 1 | iSulad/Kubernetes |
| lab3  | 197.xxx.xxx.xxx | openEuler 20.03 LTS SP4 | Worker node 2 | iSulad/Kubernetes |

## Preparing Images and Software Packages

The following table lists software packages and images used in the example. The versions are for reference only.

| Software                               | Version     |
|------------------------------------|----------|
| iSulad                             | 2.0.17-2 |
| kubernetes-client                  | 1.20.2-9 |
| kubernetes-kubeadm                 | 1.20.2-9 |
| kubernetes-kubelet                 | 1.20.2-9 |

| Image                               | Version     |
|------------------------------------|----------|
| k8s.gcr.io/kube-proxy              | v1.20.2  |
| k8s.gcr.io/kube-apiserver          | v1.20.2  |
| k8s.gcr.io/kube-controller-manager | v1.20.2  |
| k8s.gcr.io/kube-scheduler          | v1.20.2  |
| k8s.gcr.io/etcd                    | 3.4.13-0 |
| k8s.gcr.io/coredns                 | 1.7.0    |
| k8s.gcr.io/pause                   | 3.2      |
| calico/node                        | v3.14.2  |
| calico/pod2daemon-flexvol          | v3.14.2  |
| calico/cni                         | v3.14.2  |
| calico/kube-controllers            | v3.14.2  |

> If you perform the deployment in without an Internet connection, download the software packages, dependencies, and images in advance.

* Download software packages: <https://repo.openeuler.org/openEuler-22.03-LTS-SP4/>
* Download images from Docker Hub: <https://hub.docker.com>

## Modifying the hosts File

1. Change the host name of the machine, for example, **lab1**.

   ```shell
   hostnamectl set-hostname lab1
   sudo -i
   ```

2. Configure host name resolution by modifying the **/etc/hosts** file on each machine.

   ```shell
   vim /etc/hosts
   ```

3. Add the following content (IP address and host name) to the **hosts** file:

   ```text
   197.xxx.xxx.xxx lab1
   197.xxx.xxx.xxx lab2
   197.xxx.xxx.xxx lab3
   ```

## Preparing the Environment

1. Disable the firewall/

   ```shell
   systemctl stop firewalld
   systemctl disable firewalld
   ```

2. Disable SELinux.

   ```shell
   setenforce 0
   ```

3. Disable memory swapping.

   ```shell
   swapoff -a
   sed -ri 's/.*swap.*/#&/' /etc/fstab
   ```

4. Configure the network and enable forwarding.

   ```shell
   $ cat > /etc/sysctl.d/kubernetes.conf <<EOF
   net.bridge.bridge-nf-call-iptables = 1
   net.ipv4.ip_forward = 1
   net.bridge.bridge-nf-call-ip6tables = 1
   vm.swappiness=0
   EOF
   ```

5. Enable the rules.

   ```shell
   modprobe overlay
   modprobe br_netfilter
   sysctl -p /etc/sysctl.d/kubernetes.conf
   ```

6. Configure the startup script.

   ```shell
   vim /etc/init.d/k8s.sh
   ```

   * Add the following content to the k8s.sh script:

     ```shell
     #!/bin/sh
     modprobe br_netfilter
     sysctl -w net.bridge.bridge-nf-call-ip6tables = 1
     sysctl -w net.bridge.bridge-nf-call-iptables = 1
     ```

   * Change the permissions aon the script.

     ```shell
     chmod +x /etc/init.d/k8s.sh
     ```

7. Create the configuration file.

   ```shell
   $ vim br_netfilter.service

   [Unit]
   Description=To enable the core module br_netfilter when reboot
   After=default.target
   [Service]
   ExecStart=/etc/init.d/k8s.sh
   # The path can be customized.
   [Install]
   WantedBy=default.target
   ```

   * Start the service.

     ```shell
     systemctl daemon-reload
     systemctl enable br_netfilter.service
     ```

8. Configure sysctl.

   ```shell
   sed -i "s/net.ipv4.ip_forward=0/net.ipv4.ip_forward=1/g" /etc/sysctl.conf
   sed -i 12a\vm.swappiness=0 /etc/sysctl.conf
   ```

## Installing kubeadm, kubectl, kubelet, and iSulad

1. Install the software packages using Yum.

```shell
yum install -y kubernetes-kubeadm
yum install -y kubernetes-client
yum install -y kubernetes-kubelet
yum install -y iSulad
```

1. Enable kubelet to start upon system startup.

```shell
systemctl enable kubelet
```

### Modifying iSulad Configurations

1. Open the **/etc/isulad/daemon.json** file.

```shell
vi /etc/isulad/daemon.json
```

1. Modify the file as follows:

```json
{
    "group": "isula",
    "default-runtime": "runc",
    "graph": "/var/lib/isulad",
    "state": "/var/run/isulad",
    "engine": "lcr",
    "log-level": "ERROR",
    "pidfile": "/var/run/isulad.pid",
    "log-opts": {
        "log-file-mode": "0600",
        "log-path": "/var/lib/isulad",
        "max-file": "1",
        "max-size": "30KB"
    },
    "log-driver": "stdout",
    "container-log": {
        "driver": "json-file"
    },
    "hook-spec": "/etc/default/isulad/hooks/default.json",
    "start-timeout": "2m",
    "storage-driver": "overlay2",
    "storage-opts": [
        "overlay2.override_kernel_check=true"
    ],
    "registry-mirrors": [
                "docker.io"
    ],
    "insecure-registries": [
                 "k8s.gcr.io",
                 "quay.io",
                 "oci.inhuawei.com",
                 "rnd-dockerhub.huawei.com",
                 "registry.aliyuncs.com",
                 "<IP address of the local image repository>"
    ],
    "pod-sandbox-image": "k8s.gcr.io/pause:3.2",
    "native.umask": "normal",
    "network-plugin": "cni",
    "cni-bin-dir": "/opt/cni/bin",
    "cni-conf-dir": "/etc/cni/net.d",
    "image-layer-check": false,
    "use-decrypted-key": true,
    "insecure-skip-verify-enforce": false,
    "cri-runtimes": {
        "kata": "io.containerd.kata.v2"
    }
}
```

1. Restart the isulad service.

   ```shell
   systemctl restart isulad
   ```

### Loading the isulad Images

1. Check the required system images.

   ```shell
   kubeadm config images list
   ```

Pay attention to the versions in the output, as shown in the figure.\
![](figures/1.view-required-images.png)

1. Pull the images using the `isula` command.

   > **Note**:  The versions in the following commands are for reference only. Use the versions in the preceding output.

   ```shell
   isula pull k8simage/kube-apiserver:v1.20.15
   isula pull k8smx/kube-controller-manager:v1.20.15
   isula pull k8smx/kube-scheduler:v1.20.15
   isula pull k8smx/kube-proxy:v1.20.15
   isula pull k8smx/pause:3.2
   isula pull k8smx/coredns:1.7.0
   isula pull k8smx/etcd:3.4.13-0
   ```

2. Modify the tags of the pulled images.

   ```shell
   isula tag k8simage/kube-apiserver:v1.20.15 k8s.gcr.io/kube-apiserver:v1.20.15
   isula tag k8smx/kube-controller-manager:v1.20.15 k8s.gcr.io/kube-controller-manager:v1.20.15
   isula tag k8smx/kube-scheduler:v1.20.15 k8s.gcr.io/kube-scheduler:v1.20.15
   isula tag k8smx/kube-proxy:v1.20.15 k8s.gcr.io/kube-proxy:v1.20.15
   isula tag k8smx/pause:3.2 k8s.gcr.io/pause:3.2
   isula tag k8smx/coredns:1.7.0 k8s.gcr.io/coredns:1.7.0
   isula tag k8smx/etcd:3.4.13-0 k8s.gcr.io/etcd:3.4.13-0
   ```

3. Remove the old images.

   ```shell
   isula rmi k8simage/kube-apiserver:v1.20.15
   isula rmi k8smx/kube-controller-manager:v1.20.15
   isula rmi k8smx/kube-scheduler:v1.20.15
   isula rmi k8smx/kube-proxy:v1.20.15
   isula rmi k8smx/pause:3.2
   isula rmi k8smx/coredns:1.7.0
   isula rmi k8smx/etcd:3.4.13-0
   ```

4. View pulled images.

   ```shell
   isula images
   ```

### Installing crictl

```shell
yum install -y cri-tools
```

### Initializing the Master Node

Initialize the master node.

```shell
kubeadm init --kubernetes-version v1.20.2 --cri-socket=/var/run/isulad.sock --pod-network-cidr=<IP address range of the pods>
```

* `--kubernetes-version` indicates the current Kubernetes version.
* `--cri-socket` specifies the engine, that is, isulad.
* `--pod-network-cidr` specifies the IP address range of the pods.

Enter the following commands as prompted:

```shell
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

After the initialization, copy the last two lines of the output and run the copied commands on the nodes to add them to the master cluster. The commands can also be generated using the following command:

```sh
kubeadm token create --print-join-command
```

### Adding Nodes

Paste the `kubeadm join` command generated on Master, add `--cri-socket=/var/run/isulad.sock` before `--discovery-token-ca-cert-hash`, and then run the command.

```shell
kubeadm join <IP address> --token bgyis4.euwkjqb7jwuenwvs --cri-socket=/var/run/isulad.sock --discovery-token-ca-cert-hash sha256:3792f02e136042e2091b245ac71c1b9cdcb97990311f9300e91e1c339e1dfcf6
```

### Installing Calico Network Plugins

1. Pull Calico images.

   Configure the Calico network plugins on the Master node and pull the required images on each node.

   ```shell
   isula pull calico/node:v3.14.2
   isula pull calico/cni:v3.14.2
   isula pull calico/kube-controllers:v3.14.2
   isula pull calico/pod2daemon-flexvol:v3.14.2
   ```

2. Download the configuration file on Master.

   ```shell
   wget https://docs.projectcalico.org/v3.14/manifests/calico.yaml
   ```

3. Modify **calico.yaml**.

   ```yaml
   # vim calico.yaml

   # Modify the following parameters.

   - name: IP_AUTODERECTION_METHOD
   Value: ”can-reach=197.3.10.254”

   - name: CALICO_IPV4POOL_IPIP
   Value: ”CrossSubnet”
   ```

   ![](figures/2.calico-config.png)

   * If the default CNI of the pod is Flannel, add the following content to **flannel.yaml**:

     ```yaml
     --iface=enp4s0
     ```

     ![](figures/3.png)

4. Create a pod.

   ```shell
   kubectl apply -f calico.yaml
   ```

   * If you want to delete the configuration file, run the following command:

     ```shell
     kubectl delete -f calico.yaml
     ```

5. View pod information.

   ```shell
   kubectl get pod -A -o wide
   ```

### Checking the Master Node Information

```shell
kubectl get nodes -o wide
```

To reset a node, run the following command:

```shell
kubeadm reset
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/libcareplus.md
---
# LibcarePlus

## Overview

LibcarePlus is a hot patch framework for user-mode processes. It can perform hot patch operations on target processes running on the Linux system without restarting the processes. Hot patches can be used to fix CVEs and urgent bugs that do not interrupt application services.

## Hardware and Software Requirements

The following software and hardware requirements must be met to use LibcarePlus on openEuler:

* Currently, the x86 and ARM64 architectures are supported.

* LibcarePlus can run on any Linux distribution that supports **libunwind**, **elfutils**, and **binutils**.

* LibcarePlus uses the **ptrace()** system call, which requires the kernel configuration option enabled for the corresponding Linux distribution.

* LibcarePlus needs the symbol table of the original executable file when creating a hot patch. Do not strip the symbol table too early.

* On the Linux OS where SELinux is enabled, manually adapt the SELinux policies.

## Precautions and Constraints

When using LibcarePlus, comply with the following hot patch specifications and constraints:

* Only the code written in the C language is supported. The assembly language is not supported.
* Only user-mode programs are supported. Dynamic library patches are not supported.
* The code file name must comply with the C language identifier naming specifications. That is, the code file name consists of letters (A-Z and a-z), digits (0-9), and underscores (\_) but the first character cannot be a digit. Special characters such as hyphens (-) and dollar signs ($) are not allowed.
* Incremental patches are supported. Multiple patches can be installed on a process. However, you need to design the patch installation and uninstallation management. Generally, the installation and uninstallation comply with the first-in, last-out (FILO) rule.
* Automatic patch loading is not natively supported. You can design an automatic patch loading method for a specific process.
* Patch query is supported.
* The static function patch is restricted by the symbol table that can find the function in the system.
* Hot patches are process-specific. That is, a hot patch of a dynamic library can be applied only to process that invoke the dynamic library.
* The number of patches that can be applied to a process is limited by the range of the jump instruction and the size of the hole in the virtual memory address space. Generally, up to 512 patches can be applied to a process.
* Thread local storage (TLS) variables of the initial executable (IE) model can be modified.
* Symbols defined in a patch cannot be used in subsequent patches.
* Hot patches are not supported in the following scenarios:
  * Infinite loop function, non-exit function, inline function, initialization function, and non-maskable interrupt (NMI) function
  * Replacing global variables
  * Functions less than 5 bytes
  * Modifying the header file
  * Adding or deleting the input and output parameters of the target function
  * Changing (adding, deleting, or modifying) data structure members
  * Modifying the C files that contain GCC macros such as **LINE** and **FILE**
  * Modifying the Intel vector assembly instruction

## Installing LibcarePlus

### Software Installation Dependencies

The LibcarePlus running depends on **libunwind**, **elfutils**, and **binutils**. On the openEuler system configured with the Yum repo, you can run the following commands to install the software on which LibcarePlus depends:

```shell
# yum install -y binutils elfutils elfutils-libelf-devel libunwind-devel
```

#### Installing LibcarePlus

```shell
# yum install libcareplus libcareplus-devel -y
```

Check whether LibcarePlus is installed.

```shell
# libcare-ctl -h
usage: libcare-ctl [options] <cmd> [args]

Options:
  -v          - verbose mode
  -h          - this message

Commands:
  patch  - apply patch to a user-space process
  unpatch- unapply patch from a user-space process
  info   - show info on applied patches
```

## Creating LibcarePlus Hot Patches

### Introduction

LibcarePlus hot patch creation methods:

* Manual creation
* Creation through a script

The process of manually creating a hot patch is complex. For a project with a large amount of code, for example, QEMU, it is extremely difficult to manually create a hot patch. You are advised to use the script provided by LibcarePlus to generate a hot patch file with one click.

#### Manual Creation

The following takes the original file **foo.c** and the patch file **bar.c** as examples to describe how to manually create a hot patch.

1. Prepare the original file and patch file written in the C language. For example, **foo.c** and **bar.c**.

   ```c
   // foo.c
   #include <stdio.h>
   #include <time.h>

   void print_hello(void)
   {
       printf("Hello world!\n");
   }

   int main(void)
   {
       while (1) {
           print_hello();
           sleep(1);
       }
   }
   ```

   ```c
   // bar.c
   #include <stdio.h>
   #include <time.h>

   void print_hello(void)
   {
       printf("Hello world %s!\n", "being patched");
   }

   int main(void)
   {
       while (1) {
           print_hello();
           sleep(1);
       }
   }
   ```

2. Build the original file and patch file to obtain the assembly files **foo.s** and **bar.s**.

   ```shell
   # gcc -S foo.c
   # gcc -S bar.c
   # ls
   bar.c  bar.s  foo.c  foo.s
   ```

3. Run `kpatch_gensrc` to compare **foo.s** and **bar.s** and generate the **foobar.s** file that contains the assembly content of the original file and the differences.

   ```shell
   # sed -i 's/bar.c/foo.c/' bar.s
   # kpatch_gensrc --os=rhel6 -i foo.s -i bar.s -o foobar.s --force-global
   ```

   By default, `kpatch_gensrc` compares the original files in the same C language. Therefore, before the comparison, you need to run the `sed` command to change the file name **bar.c** in the patch assembly file **bar.s** to the original file name **foo.c**. Call `kpatch_gensrc` to specify the input files as **foo.s** and **bar.s** and the output file as **foobar.s**.

4. Build the assembly file **foo.s** in the original file and the generated assembly file **foobar.s** to obtain the executable files **foo** and **foobar**.

   ```shell
   # gcc -o foo foo.s
   # gcc -o foobar foobar.s -Wl,-q
   ```

   The **-Wl, -q** linker options reserve the relocation sections in **foobar**.

5. Use `kpatch_strip` to remove the duplicate content from the executables **foo** and **foobar** and reserve the content required for creating hot patches.

   ```shell
   # kpatch_strip --strip foobar foobar.stripped
   # kpatch_strip --rel-fixup foo foobar.stripped
   # strip --strip-unneeded foobar.stripped
   # kpatch_strip --undo-link foo foobar.stripped
   ```

   The options in the preceding command are described as follows:

   * **--strip** removes useless sections for patch creation from **foobar**.
   * **--rel-fixup** repairs the address of the variables and functions accessed in the patch.
   * **strip --strip-unneeded** removes the useless symbol information for hot patch relocation.
   * **--undo-link** changes the symbol address in a patch from absolute to relative.

6. Create a hot patch file.

   After the preceding operations, the contents required for creating the hot patch are obtained. Run the `kpatch_make` command to input parameters Build ID of the original executable file and **foobar.stripped** (output file of `kpatch_strip`) to `kpatch_make` to generate a hot patch file.

   ```shell
   # str=$(readelf -n foo | grep 'Build ID')
   # substr=${str##* }
   # kpatch_make -b $substr -i 0001 foobar.stripped -o foo.kpatch
   # ls
   bar.c  bar.s  foo  foobar  foobar.s  foobar.stripped  foo.c  foo.kpatch  foo.s
   ```

   The final hot patch file **foo.kpatch** whose patch ID is **0001** is obtained.

#### Creation Through a Script

This section describes how to use LibcarePlus built-in **libcare-patch-make** script to create a hot patch file. The original file **foo.c** and patch file **bar.c** are used as an example.

1. Run the `diff` command to generate the comparison file of **foo.c** and **bar.c**.

   ```shell
   # diff -up foo.c bar.c > foo.patch
   ```

   The content of the **foo.patch** file is as follows:

   ```diff
   --- foo.c 2020-12-09 15:39:51.159632075 +0800
   +++ bar.c 2020-12-09 15:40:03.818632220 +0800
   @@ -1,10 +1,10 @@
   -// foo.c
   +// bar.c
   #include <stdio.h>
   #include <time.h>

   void print_hello(void)
   {
   -    printf("Hello world!\n");
   +    printf("Hello world %s!\n", "being patched");
   }

   int main(void)
   ```

2. Write the **makefile** for building **foo.c** as follows:

   ```makefile
   all: foo

   foo: foo.c
       $(CC) -o $@ $<

   clean:
       rm -f foo

   install: foo
       mkdir $$DESTDIR || :
       cp foo $$DESTDIR
   ```

3. After the **makefile** is done, directly call `libcare-patch-make`. If `libcare-patch-make` asks you which file to install the patch, enter the original file name, as shown in the following:

   ```shell
   # libcare-patch-make --clean -i 0001 foo.patch
   rm -f foo
   BUILDING ORIGINAL CODE
   /usr/local/bin/libcare-cc -o foo foo.c
   INSTALLING ORIGINAL OBJECTS INTO /libcareplus/test/lpmake
   mkdir $DESTDIR || :
   cp foo $DESTDIR
   applying foo.patch...
   can't find file to patch at input line 3
   Perhaps you used the wrong -p or --strip option?
   The text leading up to this was:
   --------------------------
   |--- foo.c 2020-12-10 09:43:04.445375845 +0800
   |+++ bar.c 2020-12-10 09:48:36.778379648 +0800
   --------------------------
   File to patch: foo.c         
   patching file foo.c
   BUILDING PATCHED CODE
   /usr/local/bin/libcare-cc -o foo foo.c
   INSTALLING PATCHED OBJECTS INTO /libcareplus/test/.lpmaketmp/patched
   mkdir $DESTDIR || :
   cp foo $DESTDIR
   MAKING PATCHES
   Fixing up relocation printf@@GLIBC_2.2.5+fffffffffffffffc
   Fixing up relocation print_hello+0
   patch for /libcareplus/test/lpmake/foo is in /libcareplus/test/patchroot/700297b7bc56a11e1d5a6fb564c2a5bc5b282082.kpatch
   ```

   After the command is executed, the output indicates that the hot patch file is in the **patchroot** directory of the current directory, and the executable file is in the **lpmake** directory. By default, the Build ID is used to name a hot patch file generated by a script.

## Applying the LibcarePlus Hot Patch

This following uses the original file **foo.c** and patch file **bar.c** as an example to describe how to use the LibcarePlus hot patch.

### Preparation

Before using the LibcarePlus hot patch, prepare the original executable file **foo** and hot patch file **foo.kpatch**.

### Loading the Hot Patch

The procedure for applying the LibcarePlus hot patch is as follows:

1. In the first shell window, run the executable file to be patched:

   ```shell
   # ./lpmake/foo
   Hello world!
   Hello world!
   Hello world!
   ```

2. In the second shell window, run the `libcare-ctl` command to apply the hot patch:

   ```shell
   # libcare-ctl -v patch -p $(pidof foo) ./patchroot/BuildID.kpatch
   ```

   If the hot patch is applied successfully, the following information is displayed in the second shell window:

   ```shell
   1 patch hunk(s) have been successfully applied to PID '10999'
   ```

   The following information is displayed for the target process running in the first shell window:

   ```shell
   Hello world!
   Hello world!
   Hello world being patched!
   Hello world being patched!
   ```

### Querying a Hot Patch

The procedure for querying a LibcarePlus hot patch is as follows:

1. Run the following command in the second shell window:

   ```shell
   # libcare-ctl info -p $(pidof foo)

   ```

   If a hot patch is installed, the following information is displayed in the second shell window:

   ```shell
   Pid:                      551763
   Target:                   foo
   Build id:                 df05a25bdadd282812d3ee5f0a460e69038575de
   Applied patch number:     1
   Patch id:                 0001
   ```

### Uninstalling the Hot Patch

The procedure for uninstalling the LibcarePlus hot patch is as follows:

1. Run the following command in the second shell window:

   ```shell
   # libcare-ctl unpatch -p $(pidof foo) -i 0001
   ```

   If the hot patch is uninstalled successfully, the following information is displayed in the second shell window:

   ```shell
   1 patch hunk(s) were successfully cancelled from PID '10999'
   ```

2. The following information is displayed for the target process running in the first shell window:

   ```shell
   Hello world being patched!
   Hello world being patched!
   Hello world!
   Hello world!
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/libcareplus.md
---
# LibcarePlus

## 概述

LibcarePlus 是一个用户态进程热补丁框架，可以在不重启进程的情况下对 Linux 系统上运行的目标进程进行热补丁操作。热补丁可以应用于 CVE 漏洞修复，也可以应用于不中断应用服务的紧急 bug 修复。

## 软硬件要求

在 openEuler 上使用 LibcarePlus，需要满足一定的软硬件要求：

* 当前LibcarePlus支持 x86 体系架构和arm64体系架构。
* LibcarePlus 可以在任何支持安装 **libunwind**、 **elfutils** 以及 **binutils** 的 Linux 发行版系统上运行。
* LibcarePlus 使用ptrace()系统调用，需要对应Linux发行版本的相关编译选项支持。
* LibcarePlus 制作热补丁时，依赖原可执行文件的符号表，因此，请勿过早将符号表strip掉。
* 对于开启selinux的Linux系统，需要自行适配对应的selinux规则。

## 注意事项和约束

使用 LibcarePlus，需遵循以下热补丁规范和约束：

* 仅支持对 C 语言编写的代码，不支持汇编语言等。
* 代码文件名必须符合 C 语言标识符命名规范：由字母（A-Z,a-z）、数字 （0-9）、下划线“\_”组成；并且首字符不能是数字，但可以是字母或者下划线；不能包含“-”、“$”等特殊符号。
* 支持增量补丁，即支持对进程打多个补丁，但补丁加卸载管理需使用者执行设计，一般遵循FILO规则。
* 不支持补丁自动加载，对于特定进程，需使用者自行设计。
* 支持补丁查询功能。
* 静态函数补丁受限于系统中能找到该函数的符号表。
* 热补丁为进程粒度，即动态库热补丁只能对调用这个动态库的进程打补丁。
* 单个进程支持的补丁数受限于跳转指令的跳转范围和虚拟内存地址空洞大小，一般支持\[1, 512]。
* 对于TLS变量，仅支持修改IE模式的TLS变量。
* 后续补丁不能使用之前补丁中定义的符号。
* 以下场景不支持热补丁:
  * 死循环函数、不退出函数、inline 函数、初始化函数、NMI 中断处理函数。
  * 替换全局变量。
  * 小于5字节的短函数。
  * 修改头文件。
  * 增加和删除目标函数的出参和入参。
  * 数据结构成员变化（新增、删除、修改）。
  * 修改包含 **LINE** ， **FILE** 等gcc编译宏的 C 文件。
  * 修改 intel 矢量汇编指令。

## 安装 LibcarePlus

### 安装软件依赖

LibcarePlus 运行依赖于 **libunwind**、 **elfutils** 和 **binutils**，在配置了 yum 源的 openEuler 系统上，可以参考如下命令安装 LibcarePlus 的依赖软件。

```shell
# yum install -y binutils elfutils elfutils-libelf-devel libunwind-devel
```

#### 安装 LibcarePlus

```shell
# yum install libcareplus libcareplus-devel -y
```

查看安装是否成功：

```shell
# libcare-ctl -h
usage: libcare-ctl [options] <cmd> [args]

Options:
  -v          - verbose mode
  -h          - this message

Commands:
  patch  - apply patch to a user-space process
  unpatch- unapply patch from a user-space process
  info   - show info on applied patches

```

## 制作 LibcarePlus 热补丁

### 概述

LibcarePlus 支持如下方式制作热补丁：

* 手动制作
* 通过脚本制作

手动制作热补丁的过程繁琐，对于代码量较大的工程，例如QEMU，手动制作热补丁极其困难。建议使用 LibcarePlus 自带脚本一键式地生成热补丁文件。

#### 手动制作

本节以原文件 foo.c 和补丁文件 bar.c 为例，给出手动制作热补丁的指导。

1. 准备 C 语言编写的原文件和补丁文件。例如原文件 foo.c 和补丁文件 bar.c。

   ```c
   // foo.c
   #include <stdio.h>
   #include <time.h>

   void print_hello(void)
   {
       printf("Hello world!\n");
   }

   int main(void)
   {
       while (1) {
           print_hello();
           sleep(1);
       }
   }
   ```

   ```c
   // bar.c
   #include <stdio.h>
   #include <time.h>

   void print_hello(void)
   {
       printf("Hello world %s!\n", "being patched");
   }

   int main(void)
   {
       while (1) {
           print_hello();
           sleep(1);
       }
   }
   ```

2. 编译得到原文件和补丁文件的汇编文件 **foo.s** 和 **bar.s**，参考命令如下：

   ```shell
   # gcc -S foo.c
   # gcc -S bar.c
   # ls
   bar.c  bar.s  foo.c  foo.s
   ```

3. 使用 **kpatch\_gensrc** 对比 foo.s 和 bar.s 差异，生成包含原文件的汇编内容和差异内容的 foobar.s，参考命令如下：

   ```shell
   # sed -i 's/bar.c/foo.c/' bar.s
   # kpatch_gensrc --os=rhel6 -i foo.s -i bar.s -o foobar.s --force-global
   ```

   由于 **kpatch\_gensrc** 默认对同一 C 语言原文件进行对比，所以对比前需要使用 sed 命令将补丁汇编文件 bar.s 中的 bar.c 改为原文件名称 foo.c。随后调用 **kpatch\_gensrc**，指定输入文件为 foo.s 与 bar.s，输出文件为 foobar.s。

4. 编译原文件的汇编文件 foo.s 和生成的汇编文件 foobar.s，得到可执行文件 foo 和 foobar，参考命令如下：

   ```shell
   # gcc -o foo foo.s
   # gcc -o foobar foobar.s -Wl,-q
   ```

   链接选项 **-Wl, -q** 将保留foobar中的重定位节。

5. 利用 **kpatch\_strip** 去除可执行程序 foo 和 foobar 的相同内容，保留制作热补丁所需要的内容。

   ```shell
   # kpatch_strip --strip foobar foobar.stripped
   # kpatch_strip --rel-fixup foo foobar.stripped
   # strip --strip-unneeded foobar.stripped
   # kpatch_strip --undo-link foo foobar.stripped
   ```

   上述命令中的各参数含义为：

   * **--strip** 用于去除 foobar 中对于补丁制作无用的 section；
   * **--rel-fixup** 用于修复补丁内所访问的变量以及函数的地址；
   * **strip --strip-unneeded** 用于去除对于热补丁重定位操作无用的符号信息；
   * **--undo-link** 用于将补丁内符号的地址从绝对地址更改为相对地址。

6. 制作热补丁文件。

   通过以上操作，已经得到了热补丁制作所需的主要内容。接下来需要使用 **kpatch\_make** 将原可执行文件的 **Build ID** 以及 **kpatch\_strip** 的输出文件 **foobar.stripped** 作为参数传递给 **kpatch\_make**，最终生成热补丁文件，参考命令如下：

   ```shell
   # str=$(readelf -n foo | grep 'Build ID')
   # substr=${str##* }
   # kpatch_make -b $substr -i 0001 foobar.stripped -o foo.kpatch
   # ls
   bar.c  bar.s  foo  foobar  foobar.s  foobar.stripped  foo.c  foo.kpatch  foo.s
   ```

   至此，就得到了patch ID为0001的热补丁文件 foo.kpatch。

#### 通过脚本制作

本节介绍如何利用 LibcarePlus 自带的 **libcare-patch-make** 脚本制作热补丁文件，仍以原文件 foo.c 和补丁文件 bar.c 为例。

1. 利用 diff 命令生成 foo.c 和 bar.c 的对比文件，命令如下所示:

   ```shell
   # diff -up foo.c bar.c > foo.patch
   ```

   foo.patch 文件内容如下所示：

   ```diff
   --- foo.c   2020-12-09 15:39:51.159632075 +0800
   +++ bar.c   2020-12-09 15:40:03.818632220 +0800
   @@ -1,10 +1,10 @@
   -// foo.c
   +// bar.c
   #include <stdio.h>
   #include <time.h>

   void print_hello(void)
   {
   -    printf("Hello world!\n");
   +    printf("Hello world %s!\n", "being patched");
   }

   int main(void)
   ```

2. 编写编译 foo.c 的 Makefile 文件，具体如下所示：

   ```makefile
   all: foo

   foo: foo.c
       $(CC) -o $@ $<

   clean:
       rm -f foo

   install: foo
       mkdir $$DESTDIR || :
       cp foo $$DESTDIR
   ```

3. 编写好 Makefile 之后，直接调用 **libcare-patch-make** 即可。若 **libcare-patch-make** 询问选择哪个文件进行打补丁操作，输入原文件名即可，具体如下所示：

   ```shell
   # libcare-patch-make --clean -i 0001 foo.patch
   rm -f foo
   BUILDING ORIGINAL CODE
   /usr/local/bin/libcare-cc -o foo foo.c
   INSTALLING ORIGINAL OBJECTS INTO /libcareplus/test/lpmake
   mkdir $DESTDIR || :
   cp foo $DESTDIR
   applying foo.patch...
   can't find file to patch at input line 3
   Perhaps you used the wrong -p or --strip option?
   The text leading up to this was:
   --------------------------
   |--- foo.c  2020-12-10 09:43:04.445375845 +0800
   |+++ bar.c  2020-12-10 09:48:36.778379648 +0800
   --------------------------
   File to patch: foo.c
   patching file foo.c
   BUILDING PATCHED CODE
   /usr/local/bin/libcare-cc -o foo foo.c
   INSTALLING PATCHED OBJECTS INTO /libcareplus/test/.lpmaketmp/patched
   mkdir $DESTDIR || :
   cp foo $DESTDIR
   MAKING PATCHES
   Fixing up relocation printf@@GLIBC_2.2.5+fffffffffffffffc
   Fixing up relocation print_hello+0
   patch for /libcareplus/test/lpmake/foo is in /libcareplus/test/patchroot/700297b7bc56a11e1d5a6fb564c2a5bc5b282082.kpatch
   ```

   执行成功之后，输出显示：热补丁文件位于当前目录的 **patchroot** 目录下，可执行文件则在 **lpmake** 目录下。脚本生成的热补丁文件默认是采用 Build ID 作为热补丁文件的文件名。

## 应用 LibcarePlus 热补丁

本节以原文件 **foo.c** 和补丁文件 **bar.c** 为例，介绍 LibcarePlus 热补丁的应用指导。

### 前期准备

应用 LibcarePlus 热补丁之前，需要提前准备好原可执行程序 foo、以及热补丁文件 foo.kpatch。

### 加载热补丁

本节介绍应用 LibcarePlus 热补丁的具体流程。

1. 首先在第一个 shell 窗口运行需要打补丁的可执行程序，如下所示：

   ```shell
   # ./lpmake/foo
   Hello world!
   Hello world!
   Hello world!
   ```

2. 随后在第二个 shell 窗口运行 **libcare-ctl** 应用热补丁，命令如下所示：

   ```shell
   # libcare-ctl -v patch -p $(pidof foo) ./patchroot/BuildID.kpatch
   ```

   若此时热补丁应用成功，第二个 shell 窗口会有如下输出：

   ```shell
   1 patch hunk(s) have been successfully applied to PID '10999'
   ```

   而第一个 shell 窗口内运行的目标进程则会出现如下输出：

   ```shell
   Hello world!
   Hello world!
   Hello world being patched!
   Hello world being patched!
   ```

### 查询补丁

本节介绍查询LibcarePlus热补丁的具体流程。

1. 在第二个shell窗口执行如下命令：

   ```shell
   # libcare-ctl info -p $(pidof foo)

   ```

   此时若进程存在已经加载的热补丁，则第二个shell窗口会有如下输出：

   ```shell
   Pid:                      551763
   Target:                   foo
   Build id:                 df05a25bdadd282812d3ee5f0a460e69038575de
   Applied patch number:     1
   Patch id:                 0001
   ```

### 卸载热补丁

本节介绍卸载 LibcarePlus 热补丁的具体流程。

1. 在第二个 shell 窗口执行如下命令：

   ```shell
   # libcare-ctl unpatch -p $(pidof foo) -i 0001
   ```

   此时若热补丁卸载成功，第二个 shell 窗口会有如下输出：

   ```shell
   1 patch hunk(s) were successfully cancelled from PID '10999'
   ```

2. 第一个 shell 窗口内运行的目标进程则会出现如下输出：

   ```shell
   Hello world being patched!
   Hello world being patched!
   Hello world!
   Hello world!
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/fangtian/fangtian_for_linux_waylan_and_openharmony_applications.md
---
# Linux Wayland 应用及鸿蒙应用的支持

FangTian 视窗引擎融合了多个应用生态，可支持 Linux、鸿蒙应用在 openEuler 同时运行。

## Wayland应用的支持

### Wayland协议

FangTian 为了支持 Linux 原生应用，对 Wayland 应用做了兼容。由于 Wayland 协议庞杂，FangTian 当前主要兼容了 Core/Stable/Unstable 等。

### 应用运行

1. 在启动[引擎](./fangtian_environment_configuration.md#启动引擎)之后，启动 wayland 适配器的 sa。

   ```shell
   mkdir -p ~/tmp
   sa_main /system/profile/ft/ft_wl.xml > ~/tmp/ftwlsa.log 2>&1 &
   ```

2. 配置 wl 环境。

   ```shell
   export XDG_SESSION_TYPE=wayland
   export WAYLAND_DISPLAY="wayland-0"
   export QT_QPA_PLATFORMTHEME=ukui
   ```

3. Linux Wayland 应用的安装下载。

   ```shell
   sudo dnf install kylin-calculator deepin-terminal
   ```

4. 运行结果如下 。

![](./figures/wayland_apps.png)

## 鸿蒙应用的支持

### ArkUI框架

FangTian 当前支持 ArkUI 部分控件，如文本、按钮、图片等。开发者可以基于[DevEco Studio](https://developer.harmonyos.com/cn/develop/deveco-studio/)完成鸿蒙应用的开发。

### 应用代码

* [电子相册](https://gitee.com/openharmony/codelabs/tree/master/ETSUI/ElectronicAlbum)
* [简易计算器](https://gitee.com/openharmony/codelabs/tree/master/ETSUI/SimpleCalculator)

### 安装运行

1. 从 DevEco Studio 复制应用 hap 到 openEuler 目录下，如`~/apps/tmp`。

2. 解压该 hap，如`eletronicAlbum.hap`。

   ```shell
   unzip eletronicAlbum.hap
   ```

   解压之后的路径为`~/apps/tmp/eletronicAlbum`。

3. 在启动[引擎](./fangtian_environment_configuration.md#启动引擎)之后，运行 hap。

   ```shell
   hap_executor ~/apps/tmp/eletronicAlbum
   ```

4. 运行结果如下。

![](./figures/arkui_ele.png)

### 限制条件

* 当前 ArkUI 控件支持不全，web、视频类等控件不可用，napi 接口需要自行开发、迁移。

* ArkUI 在该版本版本上仅支持 x86 架构。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/local_volume_management.md
---
# Local Volume Management

## Overview

After a container managed by iSula is destroyed, all data in the container is destroyed. If you want to retain data after the container is destroyed, a data persistence mechanism is required. iSula allows files, directories, or volumes on a host to be mounted to a container at runtime. You can write the data to be persisted to the mount point in the container. After the container is destroyed, the files, directories, and volumes on the host are retained. If you need to delete a file, directory, or volume on the host, you can manually delete the file or directory, or run the iSula command to delete the volume. Currently, the iSula supports only local volume management. Local volumes are classified into named volumes and anonymous volumes. A volume whose name is specified by a user is called a named volume. If a user does not specify a name for a volume, iSula automatically generates a name (a 64-bit random number) for the volume, that is, an anonymous volume.

The following describes how to use iSula to manage local volumes.

## Precautions

* The volume name contains 2 to 64 characters and complies with the regular expression ^\[a-zA-Z0-9]\[a-zA-Z0-9\_.-]{1,63}$. That is, the first character of the volume name must be a letter or digit, and other characters can be letters, digits, underscores (\_), periods (.), and hyphens (-).
* During container creation, if data exists at the mount point of the container corresponding to the volume, the data is copied to the volume by default. If the iSula breaks down or restarts or the system is powered off during the copy process, the data in the volume may be incomplete. In this case, you need to manually delete the volume or the data in the volume to ensure that the data is correct and complete.

## Usage

### Using the -v Option to Mount Data

#### **Format**

```shell
isula run -v [SRC:]DST[:MODE,MODE...] IMAGE
```

#### **Functions**

When you create and run a container, use the -v/--volume option to mount the files, directories, or volumes on the host to the container for data persistence.

#### **Parameter Description**

* SRC: Path of the file, directory, or volume to be mounted on the host. If the value is an absolute path, a file or folder on the host is mounted. If the value is a volume name, a volume is mounted. If this parameter is not specified, an anonymous volume is mounted. If a folder or volume does not exist, iSula creates a folder or volume and then mounts it.
* DST: Mount path in the container. The value must be an absolute path.
* MODE: When the source to be mounted is a directory or file, the valid parameters are ro, rw, z, Z, private, rprivate, slave, rslave, shared, and rshared. Only one parameter of the same type can be configured. If the source is a volume, the valid parameters are ro, rw, z, Z, and nocopy. Only one parameter of the same type can be configured. Use commas (,) to separate multiple attributes. The parameters are described as follows:

| Parameter | Description |
| -------- | -----------------------------------------------|
| ro | The mount point in the container is mounted in read-only mode. |
| rw | The mount point in the container is mounted in read/write mode. |
| z | If SELinux is enabled, add the SELinux share label during mounting. |
| Z | If SELinux is enabled, add the SELinux private label during mounting. |
| private | The mount point in the container is mounted in private propagation mode. |
| rprivate | The mount point in the container is recursively mounted in private propagation mode. |
| slave | The mount point in the container is mounted in subordinate propagation mode. |
| rslave | The mount point in the container is recursively mounted in subordinate propagation mode. |
| shared | The mount point in the container is mounted in shared propagation mode. |
| rshared | The mount point in the container is recursively mounted in shared propagation mode. |
| nocopy | Data at the mount point is not copied. If this parameter is not set, data is copied by default. In addition, if data already exists in the volume, the data will not be copied. |

#### **Examples**

Run the container based on BusyBox, create or mount a volume named vol to the /vol directory of the container, and set the mount point to read-only. In addition, if data exists at the mount point in the container, the data is not copied.

```shell
isula run -v vol:/vol:ro,nocopy busybox
```

### Using the --mount Option to Mount Data

#### **Format**

```shell
isula run --mount [type=TYPE,][src=SRC,]dst=DST[,KEY=VALUE] busybox
```

#### **Functions**

When you create and run a container, use the --mount option to mount the files, directories, or volumes on the host to the container for data persistence.

#### **Parameter Description**

* type: Type of data mounted to the container. The value can be bind, volume, squashfs, or tmpfs. If this parameter is not specified, the default value is volume.
* src: Path of the file, directory, or volume to be mounted on the host. If the value is an absolute path, the file or directory on the host is mounted. If the value is a volume name, a volume is mounted. If this parameter is not specified, the volume is an anonymous volume. If a folder or volume does not exist, iSula creates a file or volume and then mounts it. The keyword src is also called source.
* dst: Mount path in the container. The value must be an absolute path. The keyword dst is also called destination or target.
* KEY=VALUE: Parameter of --mount. The values are as follows:

| KEY | VALUE |
| ------------------------------ | --------------------------------------------------------------------------- |
| selinux-opts/bind-selinux-opts | z or Z. z indicates that if SELinux is enabled, the SELinux share label is added during mounting. Z indicates that if SELinux is enabled, the SELinux private label is added during mounting.|
| ro/readonly                    | 0/false indicates that the mount is read/write. 1/true indicates that the mount is read-only. If this parameter is not specified, the mount is read-only. The parameter is supported only when type is set to bind. |
| bind-propagation                  | The value can be private, rprivate, slave, rslave, shared, or rshared. The meaning is the same as that of the -v option. This parameter is supported only when type is set to bind. |
| volume-nocopy | Data at the mount point is not copied. If this parameter is not specified, data is copied by default. In addition, if data already exists in the volume, the data will not be copied. This parameter is supported only when type is set to volume. |
| tmpfs-size                  | Maximum size of the mounted tmpfs. Be default, the size is unlimited. |
| tmpfs-mode                  | Permission on the mounted tmpfs. The default value is 1777. |

#### **Examples**

Run the container based on BusyBox, create or mount a volume named vol to the /vol directory of the container, and set the mount point to read-only. In addition, if data exists at the mount point in the container, the data is not copied.

```shell
isula run --mount type=volume,src=vol,dst=/vol,ro=true,volume-nocopy=true busybox
```

### Reusing the Mounting Configuration in Other Containers

#### **Format**

```shell
isula run --volumes-from CON1[:MODE] busybox
```

#### **Functions**

When you create and run a container, use the --volumes-from option to indicate that the mount point configuration includes that of the CON1 container. You can set multiple --volumes-from options.

#### **Parameter Description**

* CON1: Name or ID of the container whose mount point is reused.
* MODE: If the value is ro, the mount point is read-only. If the value is rw, the mount point is read/write.

#### **Examples**

Assume that a container named container1 has been configured with a volume vol1 to the container directory /vol1, and a container named container2 has been configured with a volume vol2 to the container directory /vol2. Run a new container to reuse the mounting configuration of container1 and container2. That is, volume vol1 is mounted to the /vol1 directory of the container, and volume vol2 is mounted to the /vol2 directory of the container.

```shell
isula run --volumes-from container1 --volumes-from container2 busbyox
```

### Using the Anonymous Volume in an Image

You do not need to perform any configuration to use the anonymous volume in the image. If an anonymous volume is configured in the image, iSula automatically creates an anonymous volume and mounts it to the specified path in the image at container runtime. You can write data to the mount point of an anonymous volume in a container for data persistence.

### Querying a Volume

#### **Format**

```shell
isula volume ls [OPTIONS]
```

#### **Functions**

This command is used to query all volumes managed by iSula.

#### **Parameter Description**

Option:

* -q,--quiet: If this parameter is not specified, only the volume driver information and volume name are queried by default. If this parameter is specified, only the volume name is queried.

#### **Examples**

This command is used to query all volumes managed by iSula and return only the volume name.

```shell
isula volume ls -q
```

### Deleting a Volume

#### **Format**

```shell
isula volume rm [OPTIONS] VOLUME [VOLUME...]
isula volume prune [OPTIONS]
```

#### **Functions**

* rm: deletes a specified volume. If the volume is used by a container, the volume fails to be deleted.
* prune: deletes all volumes that are not used by containers.

#### **Parameter Description**

OPTIONS in the prune command:

* -f,--force: specifies that the system does not display a message asking you whether to delete the volume. By default, a risk message is displayed. You need to enter y to continue the operation.

#### **Examples**

Delete volumes vol1 and vol2.

```shell
isula volume rm vol1 vol2
```

Delete all unused volumes in the following format. No risk message is displayed.

```shell
isula volume prune -f
```

### Precautions

#### Conflict Combination Rules

If a volume mount point conflict occurs, perform the following operations:

* If configurations of -v and --mount conflict, a failure message is returned.
* If the configuration obtained from --volumes-from conflicts with the -v or --mount configuration, the configuration is discarded.
* If the anonymous volume configuration in the image conflicts with the -v, --mount, or --volumes-from configuration, the configuration is discarded.

#### Differences Between iSula and Docker

| iSula Behavior | Docker Behavior |
| ------------------------------------------- | ------------------------------------------- |
| The volume name can contain a maximum of 64 characters. | The length of the volume name is not limited. |
| If the source to be mounted does not exist, the --mount parameter is created. | If the source to be mounted does not exist, an error is reported. |
| The --mount parameter supports the z or Z parameter configuration in bind-selinux-opts and selinux-opts. | The --mount parameter does not support the parameter configuration in the bind-selinux-opts and selinux-opts. |
| Rules for combining mount point conflicts are not processed. | The anonymous volume specified by -v is processed as the anonymous volume in the image. |
| The volume prune command displays the space that has been reclaimed. | The volume prune command does not display the space that has been reclaimed. |
| -v, --mount, and --volumes-from are configured in hostconfig, and the anonymous volume is configured in config. | The anonymous volume specified by -v is configured in config, and other configurations are configured in hostconfig. |

---

---
url: /en/docs/22.03_LTS_SP4/server/development/lustre/user_guide.md
---
# Lustre User Guide

## Overview

Lustre is an open source parallel file system designed for high scalability, performance, and availability. Lustre runs on Linux and provides POSIX-compliant UNIX file system interfaces.

An Lustre cluster contains four main components:

* Management Service (MGS): Stores configuration information for the Lustre file system.
* Metadata Service (MDS): Provides metadata service for the Lustre file systems.
* Object Storage Service (OSS): Stores file data as objects.
* Lustre clients: Mounts the Lustre file system.

These components are connected through Lustre Network(LNet), as shown below figure:

![](./figures/lustre-architecture.png)

## Environment Requirements

**Server specifications**

* One or more x86 or Arm serves installed with openEuler 22.03 LTS SP4.
* A dedicated drive is reserved for Lustre.
* An Ethernet or InfiniBand NIC is installed.

> **Notice:**
>
> In the production deployment, carefully read [Lustre manual](https://doc.lustre.org/lustre_manual.xhtml) chapters 5 and 6 for Lustre hardware configuration and storage RAID requirements.

## Installation

Install Lustre all nodes.

1. Install the Lustre RPM repository package.

   `sudo dnf install lustre-release`

2. Install the Lustre RPM packages.

   `sudo dnf install lustre lustre-tests`

> **Notice:**
>
> The current Lustre RPM packages are compiled based on the kernel in-tree IB driver for the ldiskfs backend. If you need to compile the RPM packages based on third-party IB drivers (such as the MLX IB NIC driver) or compile ZFS backend support, recompile the Lustre source RPM package.
>
> Lustre source RPM download: <https://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/update/multi_version/lustre/2.15/source/>
>
> **Install compilation dependencies.**
>
> `sudo dnf builddep --srpm lustre-2.15.3-2.oe2203SP4.src.rpm`
>
> **Recompile based on the MLX IB NIC driver.**
>
> You need to install the MLX IB NIC driver in advance.
>
> `rpmbuild --rebuild --with mofed lustre-2.15.3-2.oe2203SP4.src.rpm`
>
> **Recompile for the ZFS backend.**
>
> Use verification branch [zfs-2.1-release](https://github.com/openzfs/zfs/tree/zfs-2.1-release) to compile for the ZFS backend.
>
> `git clone -b zfs-2.1-release https://github.com/openzfs/zfs`
>
> `cd zfs && sh autogen.sh && ./configure --with-spec=redhat && make rpms`
>
> `sudo dnf install ./*$(arch).rpm`
>
> `rpmbuild --rebuild --with zfs lustre-2.15.3-2.oe2203SP4.src.rpm`

## Deployment

> **Notice:**
>
> The following steps are simplified. In the production environment, you are advised to follow the details steps in chapter 4 of the [Lustre manual](https://www.lustre.org/documentation/).

**Configure the network.**

If there are multiple NICs, specify the one(s) for Lustre to use. For example, specify one Ethernet and IB NICs for Lustre.

```bash
$ cat /etc/modprobe.d/lustre.conf 
options lnet networks="tcp(enp125s0f0),o2ib(enp133s0f0)
```

**Load the Lustre module.**

Check if the LNet is normal.

```bash
$ sudo modproe lustre
$ sudo lctl list_nids
175.200.20.14@tcp
10.20.20.14@o2ib
```

**Deploy a standalone node.**

Run the following commands to build a single-node environment for test and verification:

```bash
$ sudo /lib64/lustre/tests/llmount.sh
$ mount
...
192.168.1.203@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt)
$ lfs df -h
UUID                       bytes        Used   Available Use% Mounted on
lustre-MDT0000_UUID        95.8M        3.2M       90.5M   4% /mnt/lustre[MDT:0]
lustre-OST0000_UUID       239.0M        3.0M      234.0M   2% /mnt/lustre[OST:0]
lustre-OST0001_UUID       239.0M        3.0M      234.0M   2% /mnt/lustre[OST:1]

filesystem_summary:       478.0M        6.0M      468.0M   2% /mnt/lustre
```

**Deploy a multi-node cluster.**

On the MGS/MDS node, add an MDT whose name in the Lustre file system is **temp**.

```bash
$ sudo mkfs.lustre --fsname=temp --mgs --mdt --index=0 /dev/vdb
   Permanent disk data:
Target:     temp:MDT0000
Index:      0
Lustre FS:  temp
Mount type: ldiskfs
Flags:      0x65
              (MDT MGS first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters:

device size = 81920MB
formatting backing filesystem ldiskfs on /dev/vdb
        target name   temp:MDT0000
        kilobytes     83886080
        options        -J size=3276 -I 1024 -i 2560 -q -O dirdata,uninit_bg,^extents,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_journal_init="0",lazy_itable_init="0" -F
mkfs_cmd = mke2fs -j -b 4096 -L temp:MDT0000  -J size=3276 -I 1024 -i 2560 -q -O dirdata,uninit_bg,^extents,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_journal_init="0",lazy_itable_init="0" -F /dev/vdb 83886080k
Writing CONFIGS/mountdata

$ sudo mkdir /mnt/lustre-mdt1
$ sudo mount -t lustre /dev/vdb /mnt/lustre-mdt1
```

Add multiple MDTs in the same way with incrementing values of `--index`.

On the OSS node, add an OST:

```bash
$ sudo lctl list_nids
192.168.1.203@tcp
]$ sudo mkfs.lustre --fsname=temp --mgsnode=192.168.1.203@tcp --ost --index=0 /dev/vdc
   Permanent disk data:
Target:     temp:OST0000
Index:      0
Lustre FS:  temp
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: ,errors=remount-ro
Parameters: mgsnode=192.168.1.203@tcp

device size = 51200MB
formatting backing filesystem ldiskfs on /dev/vdc
        target name   temp:OST0000
        kilobytes     52428800
        options        -J size=1024 -I 512 -i 69905 -q -O extents,uninit_bg,dir_nlink,quota,project,huge_file,^fast_commit,flex_bg -G 256 -E resize="4290772992",lazy_journal_init="0",lazy_itable_init="0" -F
mkfs_cmd = mke2fs -j -b 4096 -L temp:OST0000  -J size=1024 -I 512 -i 69905 -q -O extents,uninit_bg,dir_nlink,quota,project,huge_file,^fast_commit,flex_bg -G 256 -E resize="4290772992",lazy_journal_init="0",lazy_itable_init="0" -F /dev/vdc 52428800k
Writing CONFIGS/mountdata
$ sudo mkdir /mnt/lustre-ost1
$ sudo mount -t lustre /dev/vdc /mnt/lustre-ost1
```

Add multiple OSTs in the same way with incrementing values of `--index`.

On the client node, mount the Lustre file system and test file read and write:

```bash
$ sudo mount -t lustre 192.168.1.203@tcp:/temp /mnt/lustre
$ mount
...
192.168.1.203@tcp:/temp on /mnt/lustre type lustre (rw,checksum,flock,nouser_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt)
$ lfs df -h
UUID                       bytes        Used   Available Use% Mounted on
temp-MDT0000_UUID          44.4G        4.8M       40.4G   1% /mnt/lustre[MDT:0]
temp-OST0000_UUID          48.2G        1.2M       45.7G   1% /mnt/lustre[OST:0]

filesystem_summary:        48.2G        1.2M       45.7G   1% /mnt/lustre

$ echo "1234asdf"|sudo tee /mnt/lustre/testfile
1234asdf
$ cat /mnt/lustre/testfile
1234asdf
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/development/lustre/user_guide.md
---
# Lustre 用户指南

## 简介

Lustre 是一个开源的分布式并行文件系统，具有高可扩展、高性能、高可用等特点。Lustre 运行于 Linux 系统之上提供符合 POSIX 标准的 UNIX 文件系统接口。

一个部署好的 Lustre 集群系统包含包括4个主要组件：

* Management Server (MGS): 存储Lustre文件系统的配置信息。
* Metadata Server (MDS): 为文件系统提供元数据服务。
* Object Storage Server (OSS): 以对象方式存储文件数据。
* Lustre clients: 挂着 Lustre 文件系统的主机。

这些组件通过 Lustre 网络（LNet）互联起来。如下图所示。

![](./figures/lustre-architecture.png)

图片来源[Lustre manual](http://lustrefs.cn/manual/) 1.2章节。

## 环境要求

**服务器配置**

* 安装 openEuler 22.03 LTS SP4 的 x86/ARM 服务器一台或多台。
* 服务器除了系统盘外，还需要配置额外的盘给 Luste 使用。
* 插有以太网或者 IB 网卡。

> **须知：**
>
> 如是生产环境，请详细参阅[Lustre manual](http://lustrefs.cn/manual/) 第五和第六章 Lustre 硬件配置和存储组RAID要求。

## 安装

在所有节点上执行以下命令安装 lustre。

1. 安装 Lustre rpm repo 包。

   ```sh
   sudo dnf install lustre-release
   ```

2. 安装 Lustre 相关 rpm 包。

   ```sh
   sudo dnf install lustre lustre-tests
   ```

> **须知：**
>
> 目前的 Lustre rpm 包是基于内核 in-tree IB 驱动编译, 而且只编译了`ldiskfs`后端，如需基于第三方IB驱动编译（例如MLX IB网卡驱动），或者编译`zfs`后端支持，请重新编译 lustre src rpm 包。
>
> lustre src rpm 下载路径: <https://repo.openeuler.org/>
>
> 目录：`openEuler-22.03-LTS-SP4/EPOL/[update/]multi_version/lustre/2.15/source/`
>
> **安装编译依赖**
>
> ```sh
> sudo dnf builddep --srpm lustre-2.15.3-2.oe2203SP4.src.rpm
> ```
>
> **基于MLX IB驱动重新编译**
>
> 需要先安装好 MLX IB 驱动。
>
> ```sh
> rpmbuild --rebuild --with mofed lustre-2.15.3-2.oe2203SP4.src.rpm
> ```
>
> **支持zfs后端重新编译**
>
> 请使用验证分支[zfs-2.1-release](https://github.com/openzfs/zfs/tree/zfs-2.1-release)来编译zfs。
>
> ```sh
> git clone -b zfs-2.1-release https://github.com/openzfs/zfs
>
> cd zfs && sh autogen.sh && ./configure --with-spec=redhat && make rpms
>
> sudo dnf install ./*$(arch).rpm
>
> rpmbuild --rebuild --with zfs lustre-2.15.3-2.oe2203SP4.src.rpm
> ```

## 部署

> **须知：**
>
> 以下是简单的部署步骤，如是生产环境，建议按照[Lustre manual](http://lustrefs.cn/manual/) 第四章“安装概述”的详细步骤进行部署。

**配置网络**

如有多块网卡或者 IB 网卡，需要配置指定哪块为 Lustre 使用，例如指定一块以太网和 IB 网卡给 lustre 使用。

```bash
$ cat /etc/modprobe.d/lustre.conf 
options lnet networks="tcp(enp125s0f0),o2ib(enp133s0f0)
```

**加载lustre模块**

并检查lnet网络是否ok。

```bash
$ sudo modproe lustre
$ sudo lctl list_nids
175.200.20.14@tcp
10.20.20.14@o2ib
```

**单机快速部署**

如需要快速拉起测试验证单机环境，可以执行以下命令。

```bash
$ sudo /lib64/lustre/tests/llmount.sh
$ mount
...
192.168.1.203@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt)
$ lfs df -h
UUID                       bytes        Used   Available Use% Mounted on
lustre-MDT0000_UUID        95.8M        3.2M       90.5M   4% /mnt/lustre[MDT:0]
lustre-OST0000_UUID       239.0M        3.0M      234.0M   2% /mnt/lustre[OST:0]
lustre-OST0001_UUID       239.0M        3.0M      234.0M   2% /mnt/lustre[OST:1]

filesystem_summary:       478.0M        6.0M      468.0M   2% /mnt/lustre
```

**多机集群部署**

在 MGS/MDS 节点上，增加一个 MDT, lustre 文件系统名为 temp。

```bash
$ sudo mkfs.lustre --fsname=temp --mgs --mdt --index=0 /dev/vdb
   Permanent disk data:
Target:     temp:MDT0000
Index:      0
Lustre FS:  temp
Mount type: ldiskfs
Flags:      0x65
              (MDT MGS first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters:

device size = 81920MB
formatting backing filesystem ldiskfs on /dev/vdb
        target name   temp:MDT0000
        kilobytes     83886080
        options        -J size=3276 -I 1024 -i 2560 -q -O dirdata,uninit_bg,^extents,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_journal_init="0",lazy_itable_init="0" -F
mkfs_cmd = mke2fs -j -b 4096 -L temp:MDT0000  -J size=3276 -I 1024 -i 2560 -q -O dirdata,uninit_bg,^extents,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_journal_init="0",lazy_itable_init="0" -F /dev/vdb 83886080k
Writing CONFIGS/mountdata

$ sudo mkdir /mnt/lustre-mdt1
$ sudo mount -t lustre /dev/vdb /mnt/lustre-mdt1
```

可以按照同样的方法增加多块 MDT, `--index`递增。

在OSS节点上，增加一块 OST。

```bash
$ sudo lctl list_nids
192.168.1.203@tcp
]$ sudo mkfs.lustre --fsname=temp --mgsnode=192.168.1.203@tcp --ost --index=0 /dev/vdc
   Permanent disk data:
Target:     temp:OST0000
Index:      0
Lustre FS:  temp
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: ,errors=remount-ro
Parameters: mgsnode=192.168.1.203@tcp

device size = 51200MB
formatting backing filesystem ldiskfs on /dev/vdc
        target name   temp:OST0000
        kilobytes     52428800
        options        -J size=1024 -I 512 -i 69905 -q -O extents,uninit_bg,dir_nlink,quota,project,huge_file,^fast_commit,flex_bg -G 256 -E resize="4290772992",lazy_journal_init="0",lazy_itable_init="0" -F
mkfs_cmd = mke2fs -j -b 4096 -L temp:OST0000  -J size=1024 -I 512 -i 69905 -q -O extents,uninit_bg,dir_nlink,quota,project,huge_file,^fast_commit,flex_bg -G 256 -E resize="4290772992",lazy_journal_init="0",lazy_itable_init="0" -F /dev/vdc 52428800k
Writing CONFIGS/mountdata
$ sudo mkdir /mnt/lustre-ost1
$ sudo mount -t lustre /dev/vdc /mnt/lustre-ost1
```

可以按照同样的方法增加多块 OST, `--index`递增。

在 client 节点上，挂着 lustre 文件系统, 测试文件读写。

```bash
$ sudo mount -t lustre 192.168.1.203@tcp:/temp /mnt/lustre
$ mount
...
192.168.1.203@tcp:/temp on /mnt/lustre type lustre (rw,checksum,flock,nouser_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt)
$ lfs df -h
UUID                       bytes        Used   Available Use% Mounted on
temp-MDT0000_UUID          44.4G        4.8M       40.4G   1% /mnt/lustre[MDT:0]
temp-OST0000_UUID          48.2G        1.2M       45.7G   1% /mnt/lustre[OST:0]

filesystem_summary:        48.2G        1.2M       45.7G   1% /mnt/lustre

$ echo "1234asdf"|sudo tee /mnt/lustre/testfile
1234asdf
$ cat /mnt/lustre/testfile
1234asdf
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/managing_devices.md
---
# Managing Devices

## Configuring a PCIe Controller for a VM

### Overview

The NIC, disk controller, and PCIe pass-through devices in a VM must be mounted to a PCIe root port. Each root port corresponds to a PCIe slot. The devices mounted to the root port support hot swap, but the root port does not support hot swap. Therefore, users need to consider the hot swap requirements and plan the maximum number of PCIe root ports reserved for the VM. Before the VM is started, the root port is statically configured.

### Configuring the PCIe Root, PCIe Root Port, and PCIe-PCI-Bridge

The VM PCIe controller is configured using the XML file. The  **model**  corresponding to PCIe root, PCIe root port, and PCIe-PCI-bridge in the XML file are  **pcie-root**,  **pcie-root-port**, and  **pcie-to-pci-bridge**, respectively.

* Simplified configuration method

  Add the following contents to the XML file of the VM. Other attributes of the controller are automatically filled by libvirt.

  ```xml
  <controller type='pci' index='0' model='pcie-root'/>
  <controller type='pci' index='1' model='pcie-root-port'/>
  <controller type='pci' index='2' model='pcie-to-pci-bridge'/>
  <controller type='pci' index='3' model='pcie-root-port'/>
  <controller type='pci' index='4' model='pcie-root-port'/>
  <controller type='pci' index='5' model='pcie-root-port'/>
  ```

  The  **pcie-root**  and  **pcie-to-pci-bridge**  occupy one  **index**  respectively. Therefore, the final  **index**  is the number of required  **root ports + 1**.

* Complete configuration method

  Add the following contents to the XML file of the VM:

  ```xml
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
      <model name='pcie-pci-bridge'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
  ```

  In the preceding contents:

  * The  **chassis**  and  **port**  attributes of the root port must be in ascending order. Because a PCIe-PCI-bridge is inserted in the middle, the  **chassis**  number skips  **2**, but the  **port**  numbers are still consecutive.
  * The  **address function**  of the root port ranges from  **0\*0**  to  **0\*7**.
  * A maximum of eight functions can be mounted to each slot. When the slot is full, the slot number increases.

  The complete configuration method is complex. Therefore, the simplified one is recommended.

## Managing Virtual Disks

### Overview

Virtual disk types include virtio-blk, virtio-scsi, and vhost-scsi. virtio-blk simulates a block device, and virtio-scsi and vhost-scsi simulate SCSI devices.

* virtio-blk: It can be used for common system disk and data disk. In this configuration, the virtual disk is presented as  **vd\[a-z]**  or  **vd\[a-z]\[a-z]**  in the VM.
* virtio-scsi: It is recommended for common system disk and data disk. In this configuration, the virtual disk is presented as  **sd\[a-z]**  or  **sd\[a-z]\[a-z]**  in the VM.
* vhost-scsi: It is recommended for the virtual disk that has high performance requirements. In this configuration, the virtual disk is presented as  **sd\[a-z]**  or  **sd\[a-z]\[a-z]**  on the VM.

### Procedure

For details about how to configure a virtual disk, see  [Storage Devices](./vm_configuration.md#storage-devices). This section uses the virtio-scsi disk as an example to describe how to attach and detach a virtual disk.

* Attach a virtio-scsi disk.

  Run the  **virsh attach-device**  command to attach the virtio-scsi virtual disk.

  ```shell
  virsh attach-device <VMInstance> <attach-device.xml>
  ```

  The preceding command can be used to attach a disk to a VM online. The disk information is specified in the  **attach-device.xml**  file. The following is an example of the  **attach-device.xml**  file:

  ```xml
  ### attach-device.xml ###
      <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2' cache='none' io='native'/>
        <source file='/path/to/another/qcow2-file'/>
        <backingStore/>
        <target dev='sdb' bus='scsi'/>
        <address type='drive' controller='0' bus='0' target='1' unit='0'/>
      </disk>
  ```

  The disk attached by running the preceding commands becomes invalid after the VM is shut down and restarted. If you need to permanently attach a virtual disk to a VM, run the  **virsh attach-device**  command with the  **--config**  parameter.

* Detach a virtio-scsi disk.

  If a disk attached online is no longer used, run the  **virsh detach-device**  command to dynamically detach it.

  ```shell
  virsh detach-device <VMInstance> <detach-device.xml>
  ```

  **detach-device.xml**  specifies the XML information of the disk to be detached, which must be the same as the XML information during dynamic attachment.

## Managing vNICs

### Overview

The vNIC types include virtio-net, vhost-net, and vhost-user. After creating a VM, you may need to attach or detach a vNIC. openEuler supports NIC hot swap, which can change the network throughput and improve system flexibility and scalability.

### Procedure

For details about how to configure a virtual NIC, see **VM Configuration** > **Network Devices**. This section uses the vhost-net NIC as an example to describe how to attach and detach a vNIC.

* Attach the vhost-net NIC.

  Run the  **virsh attach-device**  command to attach the vhost-net vNIC.

  ```shell
  virsh attach-device <VMInstance> <attach-device.xml>
  ```

  The preceding command can be used to attach a vhost-net NIC to a running VM. The NIC information is specified in the  **attach-device.xml**  file. The following is an example of the  **attach-device.xml**  file:

  ```xml
  ### attach-device.xml ###
      <interface type='bridge'>
        <mac address='52:54:00:76:f2:bb'/>
        <source bridge='br0'/>
        <virtualport type='openvswitch'/>
        <model type='virtio'/>
        <driver name='vhost' queues='2'/>
      </interface>
  ```

  The vhost-net NIC attached using the preceding commands becomes invalid after the VM is shut down and restarted. If you need to permanently attach a vNIC to a VM, run the  **virsh attach-device**  command with the  **--config**  parameter.

* Detach the vhost-net NIC.

  If a NIC attached online is no longer used, run the  **virsh detach**  command to dynamically detach it.

  ```shell
  virsh detach-device <VMInstance> <detach-device.xml>
  ```

  **detach-device.xml**  specifies the XML information of the vNIC to be detached, which must be the same as the XML information during dynamic attachment.

## Configuring a Virtual Serial Port

### Overview

In a virtualization environment, VMs and host machines need to communicate with each other to meet management and service requirements. However, in the complex network architecture of the cloud management system, services running on the management plane and VMs running on the service plane cannot communicate with each other at layer 3. As a result, service deployment and information collection are not fast enough. Therefore, a virtual serial port is required for communication between VMs and host machines. You can add serial port configuration items to the XML configuration file of a VM to implement communication between VMs and host machines.

### Procedure

The Linux VM serial port console is a pseudo terminal device connected to the host machine through the serial port of the VM. It implements interactive operations on the VM through the host machine. In this scenario, the serial port needs to be configured in the pty type. This section describes how to configure a pty serial port.

* Add the following virtual serial port configuration items under the  **devices**  node in the XML configuration file of the VM:

  ```xml
      <serial type='pty'>
      </serial>
      <console type='pty'>
        <target type='serial'/>
      </console>
  ```

* Run the  **virsh console**  command to connect to the pty serial port of the running VM.

  ```shell
  virsh console <VMInstance>
  ```

* To ensure that no serial port message is missed, use the  **--console**  option to connect to the serial port when starting the VM.

  ```shell
  virsh start --console <VMInstance>
  ```

## Managing Device Passthrough

The device passthrough technology enables VMs to directly access physical devices. The I/O performance of VMs can be improved in this way.

Currently, the VFIO passthrough is used. It can be classified into PCI passthrough and SR-IOV passthrough based on device type.

### PCI Passthrough

PCI passthrough directly assigns a physical PCI device on the host to a VM. The VM can directly access the device. PCI passthrough uses the VFIO device passthrough mode. The PCI passthrough configuration file in XML format for a VM is as follows:

```xml
<hostdev mode='subsystem' type='pci' managed='yes'>   
    <driver name='vfio'/> 
    <source>
        <address domain='0x0000' bus='0x04' slot='0x10' function='0x01'/>
    </source>
    <rom bar='off'/>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</hostdev>
```

**Table  1**  Device configuration items for PCI passthrough

> \[!NOTE] **NOTE:**
> VFIO passthrough is implemented by IOMMU group. Devices are divided to IOMMU groups based on access control services (ACS) on hardware. Devices in the same IOMMU group can be assigned to only one VM. If multiple functions on a PCI device belong to the same IOMMU group, they can be directly assigned to only one VM as well.

### SR-IOV Passthrough

#### Overview

Single Root I/O Virtualization (SR-IOV) is a hardware-based virtualization solution. With the SR-IOV technology, a physical function (PF) can provide multiple virtual functions (VFs), and each VF can be directly assigned to a VM. This greatly improves hardware resource utilization and I/O performance of VMs. A typical application scenario is SR-IOV passthrough for NICs. With the SR-IOV technology, a physical NIC (PF) can function as multiple VF NICs, and then the VFs can be directly assigned to VMs.

> \[!NOTE] **NOTE:**
>
> * SR-IOV requires the support of physical hardware. Before using SR-IOV, ensure that the hardware device to be directly assigned supports SR-IOV and the device driver on the host OS works in SR-IOV mode.
> * The following describes how to query the NIC model:\
>   In the following command output, values in the first column indicate the PCI numbers of NICs, and  **19e5:1822**  indicates the vendor ID and device ID of the NIC.
>
> ```shell
> $ lspci | grep Ether  
> 05:00.0 Ethernet controller: Device 19e5:1822 (rev 45)  
> 07:00.0 Ethernet controller: Device 19e5:1822 (rev 45)  
> 09:00.0 Ethernet controller: Device 19e5:1822 (rev 45)  
> 0b:00.0 Ethernet controller: Device 19e5:1822 (rev 45)  
> 81:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)  
> 81:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)  
> ```

#### Procedure

To configure SR-IOV passthrough for a NIC, perform the following steps:

1. Enable the SR-IOV mode for the NIC.
   1. Ensure that VF driver support provided by the NIC supplier exists on the guest OS. Otherwise, VFs in the guest OS cannot work properly.
   2. Enable the SMMU/IOMMU support in the BIOS of the host OS. The enabling method varies depending on the servers of different vendors. For details, see the help documents of the servers.
   3. Configure the host driver to enable the SR-IOV VF mode. The following uses the Hi1822 NIC as an example to describe how to enable 16 VFs.

      ```shell
      echo 16 > /sys/class/net/ethX/device/sriov_numvfs
      ```

2. Obtain the PCI BDF information of PFs and VFs.
   1. Run the following command to obtain the NIC resource list on the current board:

      ```shell
      $ lspci | grep Eth
      03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
      04:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
      05:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
      06:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
      7d:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Device a222 (rev 20)
      7d:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Device a222 (rev 20)
      7d:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Device a221 (rev 20)
      7d:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Device a221 (rev 20)
      ```

   2. Run the following command to view the PCI BDF information of VFs:

      ```shell
      $ lspci | grep "Virtual Function"
      03:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:00.4 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:00.5 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:00.6 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:00.7 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:01.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:01.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      03:01.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
      ```

   3. Select an available VF and write its configuration to the VM configuration file based on its BDF information. For example, the bus ID of the device  **03:00.1**  is  **03**, its slot ID is  **00**, and its function ID is  **1**.

3. Identify and manage the mapping between PFs and VFs.
   1. Identify VFs corresponding to a PF. The following uses PF 03.00.0 as an example:

      ```shell
      ls -l /sys/bus/pci/devices/0000\:03\:00.0/
      ```

      The following symbolic link information is displayed. You can obtain the VF IDs (virtfnX) and PCI BDF IDs based on the information.

   2. Identify the PF corresponding to a VF. The following uses VF 03:00.1 as an example:

      ```shell
      ls -l /sys/bus/pci/devices/0000\:03\:00.1/
      ```

      The following symbolic link information is displayed. You can obtain PCI BDF IDs of the PF based on the information.

      ```console
      lrwxrwxrwx 1 root root       0 Mar 28 22:44 physfn -> ../0000:03:00.0
      ```

   3. Obtain names of NICs corresponding to the PFs or VFs. For example:

      ```shell
      $ ls /sys/bus/pci/devices/0000:03:00.0/net
      eth0
      ```

   4. Set the MAC address, VLAN, and QoS information of VFs to ensure that the VFs are in the  **Up**  state before passthrough. The following uses VF 03:00.1 as an example. The PF is eth0 and the VF ID is  **0**.

      ```shell
      ip link set eth0 vf 0 mac 90:E2:BA:21:XX:XX    # Sets the MAC address.
      ifconfig eth0 up
      ip link set eth0 vf 0 rate 100                 # Sets the VF outbound rate, in Mbit/s.
      ip link show eth0                              # Views the MAC address, VLAN ID, and QoS information to check whether the configuration is successful.
      ```

4. Mount the SR-IOV NIC to the VM.

   When creating a VM, add the SR-IOV passthrough configuration item to the VM configuration file.

   ```xml
   <interface type='hostdev' managed='yes'> 
       <mac address='fa:16:3e:0a:xx:xx'/>
       <source> 
           <address type='pci' domain='0x0000' bus='0x06' slot='0x11' function='0x6'/>
       </source> 
       <vlan>
           <tag id='1'/>
       </vlan>
   </interface>
   ```

   **Table  2**  SR-IOV configuration options

Disabling the SR-IOV function:\
To disable the SR-IOV function after the VM is stopped and no VF is in use, run the following command:\
The following uses the Hi1822 NIC (corresponding network interface name: eth0) as an example:

```sh
echo 0 > /sys/class/net/eth0/device/sriov_numvfs  
```

#### SR-IOV Passthrough for the HPRE Accelerator

The accelerator engine is a hardware acceleration solution provided by TaiShan 200 server based on the Kunpeng 920 processor. The HPRE accelerator is used to accelerate SSL/TLS applications. It significantly reduces processor consumption and improves processor efficiency.\
On the Kunpeng server, the VF of the HPRE accelerator on the host needs to be passed through to the VM for internal services of the VM.

**Table 3**  HPRE accelerator description

|    Item    | Description                                                                                         |
|-------------|-----------------------------------------------------------------------------------------------------|
| Device name | Hi1620 on-chip RSA/DH security algorithm accelerator (HPRE engine)                                  |
| Function    | Modular exponentiation, RSA key pair operation, DH calculation, and auxiliary operations of large numbers (modular exponentiation, modular multiplication, modulo, modular inverse, primality test, and co-prime test) |
| VendorID    | 0x19E5                                                                                              |
| PF DeviceID | 0xA258                                                                                              |
| VF DeviceID | 0xA259                                                                                              |
| Maximum number of VF  | A maximum of 63 VFs can be created for an HPRE PF                                         |

> \[!NOTE] **Note**\
> When a VM is using a VF device, the driver on the host cannot be uninstalled, and the accelerator does not support hot swap.\
> VF operation (If VFNUMS is 0, the VF is disabled. hpre\_num is used to identify a specific accelerator device):
>
> ```shell
> echo $VFNUMS > /sys/class/uacce/hisi_hpre-$hpre_num/device/sriov_numvfs
> ```

### vDPA Passthrough

#### Overview

vDPA passthrough connects a device on a host to the vDPA framework, uses the vhost-vdpa driver to present a character device, and configures the character device for VMs to use.

vDPA passthrough provides the similar I/O performance as VFIO passthrough, provides flexibility of VirtIO devices, and supports live migration of vDPA passthrough devices.

With the SR-IOV solution, vDPA passthrough can virtualize a physical NIC (PF) into multiple NICs (VFs), and then connect the VFs to the vDPA framework for VMs to use.

#### Procedure

To configure vDPA passthrough, perform the following steps as user **root**:

1. Create and configure VFs. For details, see steps 1 to 3 in SR-IOV passthrough. The following uses **virtio-net** devices as an example (**08:00.6** and **08:00.7** are PFs, and the others are created VFs):

   ```shell
   # lspci | grep -i Eth | grep Virtio
   08:00.6 Ethernet controller: Virtio: Virtio network device
   08:00.7 Ethernet controller: Virtio: Virtio network device
   08:01.1 Ethernet controller: Virtio: Virtio network device
   08:01.2 Ethernet controller: Virtio: Virtio network device
   08:01.3 Ethernet controller: Virtio: Virtio network device
   08:01.4 Ethernet controller: Virtio: Virtio network device
   08:01.5 Ethernet controller: Virtio: Virtio network device
   08:01.6 Ethernet controller: Virtio: Virtio network device
   08:01.7 Ethernet controller: Virtio: Virtio network device
   08:02.0 Ethernet controller: Virtio: Virtio network device
   08:02.1 Ethernet controller: Virtio: Virtio network device
   08:02.2 Ethernet controller: Virtio: Virtio network device
   ```

2. Unbind the VF drivers and bind the vDPA driver of the hardware vendor.

   ```shell
   echo 0000:08:01.1 > /sys/bus/pci/devices/0000\:08\:01.1/driver/unbind
   echo 0000:08:01.2 > /sys/bus/pci/devices/0000\:08\:01.2/driver/unbind
   echo 0000:08:01.3 > /sys/bus/pci/devices/0000\:08\:01.3/driver/unbind
   echo 0000:08:01.4 > /sys/bus/pci/devices/0000\:08\:01.4/driver/unbind
   echo 0000:08:01.5 > /sys/bus/pci/devices/0000\:08\:01.5/driver/unbind
   echo -n "1af4 1000" > /sys/bus/pci/drivers/vender_vdpa/new_id
   ```

3. After vDPA devices are bound, you can run the `vdpa` command to query the list of devices managed by vDPA.

   ```shell
   # vdpa mgmtdev show
   pci/0000:08:01.1:
     supported_classes net
   pci/0000:08:01.2:
     supported_classes net
   pci/0000:08:01.3:
     supported_classes net
   pci/0000:08:01.4:
     supported_classes net
   pci/0000:08:01.5:
     supported_classes net
   ```

4. After the vDPA devices are created, create the vhost-vDPA devices.

   ```shell
   vdpa dev add name vdpa0 mgmtdev pci/0000:08:01.1
   vdpa dev add name vdpa1 mgmtdev pci/0000:08:01.2
   vdpa dev add name vdpa2 mgmtdev pci/0000:08:01.3
   vdpa dev add name vdpa3 mgmtdev pci/0000:08:01.4
   vdpa dev add name vdpa4 mgmtdev pci/0000:08:01.5
   ```

5. After the vhost-vDPA devices are created, you can run the `vdpa` command to query the vDPA device list or run the `libvirt` command to query the vhost-vDPA device information.

   ```shell
   # vdpa dev show
   vdpa0: type network mgmtdev pci/0000:08:01.1 vendor_id 6900 max_vqs 3 max_vq_size 256
   vdpa1: type network mgmtdev pci/0000:08:01.2 vendor_id 6900 max_vqs 3 max_vq_size 256
   vdpa2: type network mgmtdev pci/0000:08:01.3 vendor_id 6900 max_vqs 3 max_vq_size 256
   vdpa3: type network mgmtdev pci/0000:08:01.4 vendor_id 6900 max_vqs 3 max_vq_size 256
   vdpa4: type network mgmtdev pci/0000:08:01.5 vendor_id 6900 max_vqs 3 max_vq_size 256

   # virsh nodedev-list vdpa
   vdpa_vdpa0
   vdpa_vdpa1
   vdpa_vdpa2
   vdpa_vdpa3
   vdpa_vdpa4

   # virsh nodedev-dumpxml vdpa_vdpa0
   <device>
     <name>vdpa_vdpa0</name>
     <path>/sys/devices/pci0000:00/0000:00:0c.0/0000:08:01.1/vdpa0</path>
     <parent>pci_0000_08_01_1</parent>
     <driver>
       <name>vhost_vdpa</name>
     </driver>
     <capability type='vdpa'>
       <chardev>/dev/vhost-vdpa-0</chardev>
     </capability>
   </device>
   ```

6. Mount a vDPA device to the VM.

   When creating a VM, add the item for the vDPA passthrough device to the VM configuration file:

   ```xml
   <devices>
       <hostdev mode='subsystem' type='vdpa'>
           <source dev='/dev/vhost-vdpa-0'/>
       </hostdev>
   </devices>
   ```

   **Table 4** vDPA configuration description

   | Parameter          | Description                                          | Value             |
   | ------------------ | ---------------------------------------------------- | ----------------- |
   | hostdev.source.dev | Path of the vhost-vDPA character device on the host. | /dev/vhost-vdpa-x |

   > \[!NOTE] **NOTE:**
   > The procedures of creating and configuring VFs and binding the vDPA drivers vary with the design of hardware vendors. Follow the procedure of the corresponding vendor.

## Managing VM USB

To facilitate the use of USB devices such as USB key devices and USB mass storage devices on VMs, openEuler provides the USB device passthrough function. Through USB passthrough and hot-swappable interfaces, you can configure USB passthrough devices for VMs, or hot swap USB devices when VMs are running.

### Configuring USB Controllers

#### Overview

A USB controller is a virtual controller that provides specific USB functions for USB devices on VMs. To use USB devices on a VM, you must configure USB controllers for the VM. Currently, openEuler supports the following types of USB controllers:

* Universal host controller interface (UHCI): also called the USB 1.1 host controller specification.
* Enhanced host controller interface (EHCI): also called the USB 2.0 host controller specification.
* Extensible host controller interface (xHCI): also called the USB 3.0 host controller specification.

#### Precautions

* The host server must have USB controller hardware and modules that support USB 1.1, USB 2.0, and USB 3.0 specifications.
* You need to configure USB controllers for the VM by following the order of USB 1.1, USB 2.0, and USB 3.0.
* An xHCI controller has eight ports and can be mounted with a maximum of four USB 3.0 devices and four USB 2.0 devices. An EHCI controller has six ports and can be mounted with a maximum of six USB 2.0 devices. A UHCI controller has two ports and can be mounted with a maximum of two USB 1.1 devices.
* On each VM, only one USB controller of the same type can be configured.
* USB controllers cannot be hot swapped.
* If the USB 3.0 driver is not installed on a VM, the xHCI controller may not be identified. For details about how to download and install the USB 3.0 driver, refer to the official description provided by the corresponding OS distributor.
* To ensure the compatibility of the OS, set the bus ID of the USB controller to  **0**  when configuring a USB tablet for the VM. The tablet is mounted to the USB 1.1 controller by default.

#### Configuration Methods

The following describes the configuration items of USB controllers for a VM. You are advised to configure USB 1.1, USB 2.0, and USB 3.0 to ensure the VM is compatible with three types of devices.

The configuration item of the USB 1.1 controller (UHCI) in the XML configuration file is as follows:

```xml
<controller type='usb' index='0' model='piix3-uhci'>
</controller>
```

The configuration item of the USB 2.0 controller (EHCI) in the XML configuration file is as follows:

```xml
<controller type='usb' index='1' model='ehci'>
</controller>
```

The configuration item of the USB 3.0 controller (xHCI) in the XML configuration file is as follows:

```xml
<controller type='usb' index='2' model='nec-xhci'>
</controller>
```

### Configuring a USB Passthrough Device

#### Overview

After USB controllers are configured for a VM, a physical USB device on the host can be mounted to the VM through device passthrough for the VM to use. In the virtualization scenario, in addition to static configuration, hot swapping the USB device is supported. That is, the USB device can be mounted or unmounted when the VM is running.

#### Precautions

* A USB device can be assigned to only one VM.
* A VM with a USB passthrough device does not support live migration.
* VM creation fails if no USB passthrough devices exist in the VM configuration file.
* Forcibly hot removing a USB storage device that is performing read or write operation may damage files in the USB storage device.

#### Configuration Description

The following describes the configuration items of a USB device for a VM.

Description of the USB device in the XML configuration file:

```xml
<hostdev mode='subsystem' type='usb' managed='yes'>
    <source>        
        <address bus='m' device='n'/>
    </source>
    <address type='usb' bus='x' port='y'/>
</hostdev>
```

* **\<address bus='***m***'device='***n***'/>**:  *m*  indicates the USB bus address on the host, and  *n*  indicates the device ID.
* **\<address type='usb'bus='***x***'port='***y***'>**: indicates that the USB device is to be mounted to the USB controller specified on the VM.  *x*  indicates the controller ID, which corresponds to the index number of the USB controller configured on the VM.  *y*  indicates the port address. When configuring a USB passthrough device, you need to set this parameter to ensure that the controller to which the device is mounted is as expected.

#### Configuration Methods

To configure USB passthrough, perform the following steps:

1. Configure USB controllers for the VM. For details, see  [Configuring USB Controllers](#configuring-usb-controllers).

2. Query information about the USB device on the host.

   Run the  **lsusb**  command (the  **usbutils**  software package needs to be installed) to query the USB device information on the host, including the bus address, device address, device vendor ID, device ID, and product description. For example:

   ```shell
   lsusb
   ```

   ```console
   Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
   Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
   Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
   Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
   Bus 006 Device 002: ID 0bda:0411 Realtek Semiconductor Corp. 
   Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
   Bus 005 Device 003: ID 136b:0003 STEC 
   Bus 005 Device 002: ID 0bda:5411 Realtek Semiconductor Corp. 
   Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
   Bus 001 Device 003: ID 12d1:0003 Huawei Technologies Co., Ltd. 
   Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp. 
   Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
   Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
   ```

3. Prepare the XML description file of the USB device. Before hot removing the device, ensure that the USB device is not in use. Otherwise, data may be lost.

4. Run the hot swapping commands.

   Take a VM whose name is  **openEulerVM**  as an example. The corresponding configuration file is  **usb.xml**.

   * Hot adding of the USB device takes effect only for the current running VM. After the VM is restarted, hot add the USB device again.

     ```shell
     virsh attach-device openEulerVM usb.xml --live
     ```

   * Complete persistency configurations for hot adding of the USB device. After the VM is restarted, the USB device is automatically assigned to the VM.

     ```shell
     virsh attach-device openEulerVM usb.xml --config
     ```

   * Hot removing of the USB device takes effect only for the current running VM. After the VM is restarted, the USB device with persistency configurations is automatically assigned to the VM.

     ```shell
     virsh detach-device openEulerVM usb.xml --live
     ```

   * Complete persistency configurations for hot removing of the USB device.

     ```shell
     virsh detach-device openEulerVM usb.xml --config
     ```

## Storing Snapshots

### Overview

The VM system may be damaged due to virus damage, system file deletion by mistake, or incorrect formatting. As a result, the system cannot be started. To quickly restore a damaged system, openEuler provides the storage snapshot function. openEuler can create a snapshot that records the VM status at specific time points without informing users (usually within a few seconds). The snapshot can be used to restore the VM to the status when the snapshots were taken. For example, a damaged system can be quickly restored with the help of snapshots, which improves system reliability.

> \[!NOTE] **NOTE:**
> Currently, storage snapshots can be QCOW2 and RAW images only. Block devices are not supported.

### Procedure

To create VM storage snapshots, perform the following steps:

1. Log in to the host and run the  **virsh domblklist**  command to query the disk used by the VM.

   ```shell
   $ virsh domblklist openEulerVM
     Target   Source
    ---------------------------------------------
     vda      /mnt/openEuler-image.qcow2
   ```

2. Run the following command to create the VM disk snapshot  **openEuler-snapshot1.qcow2**:

   ```shell
   $ virsh snapshot-create-as --domain openEulerVM --disk-only --diskspec vda,snapshot=external,file=/mnt/openEuler-snapshot1.qcow2 --atomic
   Domain snapshot 1582605802 created
   ```

3. Run the following command to query disk snapshots:

   ```shell
   $ virsh snapshot-list openEulerVM
    Name         Creation Time               State
   ---------------------------------------------------------
    1582605802   2020-02-25 12:43:22 +0800   disk-snapshot
   ```

## Configuring Disk I/O Suspension

### Introduction

#### Overview

When a storage fault occurs (for example, the storage link is disconnected), the I/O error of the physical disk is sent to the VM front end through the virtualization layer. After the VM receives the I/O error, the user file system in the VM may change to the read-only state. In this case, the VM needs to be restarted or the user needs to manually recover the file system, which brings extra workload.

In this case, the virtualization platform provides the disk I/O suspension capability. When a storage fault occurs, the VM I/O being delivered to the host is suspended. During the suspension period, no I/O error is returned to the VM. In this way, the VM file system will not be in read-only state but is hung. At the same time, the VM backend retries I/Os based on the specified suspension interval. If the storage fault is rectified within the suspension time, the suspended I/O can be written to the disk. The internal file system of the VM automatically recovers and the VM does not need to be restarted. If the storage fault is not rectified within the suspension time, an error is reported to the VM and the user is notified.

#### Applicable Scenario

The cloud that may be disconnected from the storage plane is used as the backend of a virtual disk.

#### Precautions and Restrictions

* Only virtio-blk and virtio-scsi virtual drives support disk I/O suspension.

* The backend of virtual disks suspended by disk I/O is usually the cloud drive that may cause storage plane link disconnection.

* The disk I/O suspension can be enabled for read and write I/O errors. The retry interval and timeout interval for read and write I/O errors of the same disk are the same.

* The disk I/O suspension retry interval does not include the actual I/O overhead on the host. That is, the actual interval between two I/O retry operations is greater than the configured I/O error retry interval.

* The disk I/O suspension cannot identify the I/O error type (such as storage link disconnection, bad disk, and reservation conflict). As long as the hardware returns an I/O error, the disk I/O suspension is performed.

* When the disk I/O is suspended, the internal I/O of the VM is not returned. The system commands for accessing the disk, such as fdisk, are suspended. The services that depend on the returned command are also suspended.

* When the disk I/O is suspended, the I/O cannot be written to the disk. As a result, the VM may fail to be gracefully shut down. In this case, you need to forcibly shut down the VM.

* When the disk I/O is suspended, the disk data cannot be read. As a result, the VM cannot be restarted. You need to forcibly shut down the VM, wait until the storage fault is rectified, and then restart the VM.

* After a storage fault occurs, the following problems cannot be solved even though disk I/O suspension exists:

  1. Failed to execute advanced storage features.

     Advanced features include virtual disk hot swapping, virtual disk creation, VM startup, VM shutdown, forcible VM shutdown, VM hibernation and wakeup, VM storage hot migration, VM storage hot migration cancellation, VM storage snapshot creation, VM storage snapshot combination, and VM disk capacity query, VM online scale-out, virtual CD-ROM drive insertion and ejection.

  2. Failed to execute the VM life cycle.

* When a VM configured with disk I/O suspension initiates hot migration, the XML configuration of the destination disk must contain the same disk I/O suspension configuration as that of the source disk.

### Disk I/O Suspension Configuration

#### Qemu Command Line Configuration

The disk I/O suspension function is enabled by specifying `werror=retry` and `rerror=retry` on the virtual disk device and using `retry_interval` and `retry_timeout` to configure the retry policy. `retry_interval` indicates the I/O error retry interval. The value ranges from 0 to MAX\_LONG, in milliseconds. If this parameter is not set, the default value 1000 ms is used. `retry_timeout` indicates the I/O retry timeout interval. The value ranges from 0 to MAX\_LONG. The value 0 indicates that no timeout occurs. The unit is millisecond. If this parameter is not set, the default value is 0.

The I/O suspension configuration of the virtio-blk disk is as follows:

```shell
-drive file=/path/to/your/storage,format=raw,if=none,id=drive-virtio-disk0,cache=none,aio=native \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,\
drive=drive-virtio-disk0,id=virtio-disk0,write-cache=on,\
werror=retry,rerror=retry,retry_interval=2000,retry_timeout=10000
```

The I/O suspension configuration of the virtio-scsi disk is as follows:

```shell
-drive file=/path/to/your/storage,format=raw,if=none,id=drive-scsi0-0-0-0,cache=none,aio=native \
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\
device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,write-cache=on,\
werror=retry,rerror=retry,retry_interval=2000,retry_timeout=10000
```

#### XML Configuration

The disk I/O suspension function is enabled by specifying `error_policy='retry'` and `rerror_policy='retry'`in the disk XML configuration file. Configure the values of `retry_interval` and `retry_timeout`. `retry_interval` indicates the I/O error retry interval. The value ranges from 0 to MAX\_LONG, in milliseconds. If this parameter is not set, the default value 1000 ms is used. `retry_timeout` indicates the I/O retry timeout interval. The value ranges from 0 to MAX\_LONG. The value 0 indicates that no timeout occurs. The unit is millisecond. If this parameter is not set, the default value is 0.

The disk I/O suspension XML configuration of the virtio-blk disk is as follows:

```xml
<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='none' io='native' error_policy='retry' rerror_policy='retry' retry_interval='2000' retry_timeout='10000'/>
  <source file='/path/to/your/storage'/>
  <target dev='vdb' bus='virtio'/>
  <backingStore/>
</disk>
```

The disk I/O suspension XML configuration of the virtio-scsi disk is as follows:

```xml
<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='none' io='native' error_policy='retry' rerror_policy='retry' retry_interval='2000' retry_timeout='10000'/>
  <source file='/path/to/your/storage'/>
  <target dev='sdb' bus='scsi'/>
  <backingStore/>
  <address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/memory_storage/lvm/managing_drives_through_lvm.md
---
# Managing Drives Through LVM

## LVM Overview

Logical Volume Manager (LVM) is a mechanism used for managing drive partitions in Linux. By adding a logical layer between drives and file systems, LVM shields the drive partition layout for file systems, thereby improving flexibility in managing drive partitions.

The procedure of managing a drive through LVM is as follows:

1. Create physical volumes for a drive.
2. Combine several physical volumes into a volume group.
3. Create logical volumes in the volume group.
4. Create file systems on logical volumes.

When drives are managed using LVM, file systems are distributed on multiple drives and can be easily resized as needed. Therefore, file system space will no longer be limited by drive capacities.

### Basic Terms

* Physical media: refers to physical storage devices in the system, such as drives (**/dev/hda**  and  **/dev/sda**). It is the storage unit at the lowest layer of the storage system.

* Physical volume (PV): refers to a drive partition or device (such as a RAID) that has the same logical functions as a drive partition. PVs are basic logical storage blocks of LVM. A PV contains a special label that is stored in the second 512-byte sector by default. It can also be stored in one of the first four sectors. A label contains the universal unique identifier (UUID) of the PV, size of the block device, and the storage location of LVM metadata in the device.

* Volume group (VG): consists of PVs and shields the details of underlying PVs. You can create one or more logical volumes within a VG without considering detailed PV information.

* Logical volume (LV): A VG cannot be used directly. It can be used only after being partitioned into LVs. LVs can be formatted into different file systems and can be directly used after being mounted.

* Physical extent (PE): A PE is a small storage unit in a PV. The PE size is the same as the size of the logical extent in the VG.

* Logical extent (LE): An LE is a small storage unit in an LV. In one VG, the LEs of all the LVs have the same size.

## Installing the LVM

> \[!NOTE] **NOTE:**
> The LVM has been installed on the openEuler OS by default. You can run the  **rpm -qa | grep lvm2**  command to check whether it is installed. If the command output contains "lvm2", the LVM has been installed. In this case, skip this section. If no information is output, the LVM is not installed. Install it by referring to this section.

1. Configure the local yum source. For details, see  [Configuring the Repo Server](./../../administration/administrator/configuring_the_repo_server.md).

2. Clear the cache.

   ```bash
   dnf clean all
   ```

3. Create a cache.

   ```bash
   dnf makecache
   ```

4. Install the LVM as the **root** user.

   ```bash
   dnf install lvm2
   ```

5. Check the installed RPM package.

   ```bash
   rpm -qa | grep lvm2
   ```

## Managing PVs

### Creating a PV

Run the  **pvcreate**  command as the **root** user to create a PV.

```bash
pvcreate [option] devname ...
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-f**: forcibly creates a PV without user confirmation.
  * **-u**: specifies the UUID of the device.
  * **-y**: answers yes to all questions.

* *devname*: specifies the name of the device corresponding to the PV to be created. If multiple PVs need to be created in batches, set this option to multiple device names and separate the names with spaces.

Example 1: Create PVs based on  **/dev/sdb**  and  **/dev/sdc**.

```bash
pvcreate /dev/sdb /dev/sdc
```

Example 2: Create PVs based on  **/dev/sdb1**  and  **/dev/sdb2**.

```bash
pvcreate /dev/sdb1 /dev/sdb2
```

### Viewing a PV

Run the  **pvdisplay**  command as the **root** user to view PV information, including PV name, VG to which the PV belongs, PV size, PE size, total number of PEs, number of available PEs, number of allocated PEs, and UUID.

```bash
pvdisplay [option] devname
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-s**: outputs information in short format.
  * **-m**: displays the mapping from PEs to LEs.

* *devname*: indicates the device corresponding to the PV to be viewed. If no PVs are specified, information about all PVs is displayed.

Example: Run the following command to display the basic information about the PV  **/dev/sdb**:

```bash
pvdisplay /dev/sdb
```

### Modifying PV Attributes

Run the  **pvchange**  command as the **root** user to modify the attributes of a PV.

```bash
pvchange [option] pvname ...
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-u**: generates a new UUID.
  * **-x**: indicates whether PE allocation is allowed.

* *pvname*: specifies the name of the device corresponding to the PV to be modified. If multiple PVs need to be modified in batches, set this option to multiple device names and separate the names with spaces.

Example: Run the following command to prohibit PEs on the PV  **/dev/sdb**  from being allocated. Running `pvdisplay` for a PV that is not added to a VG will return the **Allocatable** attribute with the value **NO**. You need to add the PV to a VG before you can change the attribute.

```bash
pvchange -x n /dev/sdb
```

### Deleting a PV

Run the  **pvremove**  command as the **root** user to delete a PV.

```bash
pvremove [option] pvname ...
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-f**: forcibly deletes a PV without user confirmation.
  * **-y**: answers yes to all questions.

* *pvname*: specifies the name of the device corresponding to the PV to be deleted. If multiple PVs need to be deleted in batches, set this option to multiple device names and separate the names with spaces.

Example: Run the following command to delete the PV  **/dev/sdb**. If the PV has been added to a VG, you need to delete the VG or remove the PV from the VG in advance.

```bash
pvremove /dev/sdb
```

## Managing VGs

### Creating a VG

Run the  **vgcreate**  command as the **root** user to create a VG.

```bash
vgcreate [option] vgname pvname ...
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-l**: specifies the maximum number of LVs that can be created on the VG.
  * **-p**: specifies the maximum number of PVs that can be added to the VG.
  * **-s**: specifies the PE size of a PV in the VG.

* *vgname*: name of the VG to be created.

* *pvname*: name of the PV to be added to the VG.

Example: Run the following command to create VG  **vg1**  and add the PVs  **/dev/sdb**  and  **/dev/sdc**  to the VG.

```bash
vgcreate vg1 /dev/sdb /dev/sdc  
```

### Viewing a VG

Run the  **vgdisplay**  command as the **root** user to view VG information.

```bash
vgdisplay [option] [vgname]
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-s**: outputs information in short format.
  * **-A**: displays only attributes of active VGs.

* *vgname*: name of the VG to be viewed. If no VGs are specified, information about all VGs is displayed.

Example: Run the following command to display the basic information about VG  **vg1**:

```bash
vgdisplay vg1
```

### Modifying VG Attributes

Run the  **vgchange**  command as the **root** user to modify the attributes of a VG.

```bash
vgchange [option] vgname
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-a**: sets the active status of the VG.

* *vgname*: name of the VG whose attributes are to be modified.

Example: Run the following command to change the status of  **vg1**  to active.

```bash
vgchange -ay vg1
```

### Extending a VG

Run the  **vgextend**  command as the **root** user to dynamically extend a VG. In this way, the VG size is extended by adding PVs to the VG.

```bash
vgextend [option] vgname pvname ...
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **dev**: debugging mode.
  * **-t**: test only.

* *vgname*: name of the VG whose size is to be extended.

* *pvname*: name of the PV to be added to the VG.

Example: Run the following command to add PV  **/dev/sdb**  to VG  **vg1**:

```bash
vgextend vg1 /dev/sdb
```

### Shrinking a VG

Run the  **vgreduce**  command as the **root** user to delete PVs from a VG to reduce the VG size. A VG must contain at least one PV.

```bash
vgreduce [option] vgname pvname ...
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-a**: If no PVs are specified in the command, all empty PVs are deleted.
  * **--removemissing**: deletes lost PVs in the VG to restore the VG to the normal state.

* *vgname*: name of the VG to be shrunk.

* *pvname*: name of the PV to be deleted from the VG.

Example: Run the following command to remove PV  **/dev/sdb2**  from VG  **vg1**:

```bash
vgreduce vg1 /dev/sdb2
```

### Deleting a VG

Run the  **vgremove**  command as the **root** user to delete a VG.

```bash
vgremove [option] vgname
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-f**: forcibly deletes a VG without user confirmation.

* *vgname*: name of the VG to be deleted.

Example: Run the following command to delete VG  **vg1**.

```bash
vgremove vg1
```

## Managing LVs

### Creating an LV

Run the  **lvcreate**  command as the **root** user to create an LV.

```bash
lvcreate [option] vgname
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-L**: specifies the size of the LV in kKmMgGtT.
  * **-l**: specifies the size of the LV (number of LEs).
  * **-n**: specifies the name of the LV to be created.
  * **-s**: creates a snapshot.

* *vgname*: name of the VG to be created.

Example 1: Run the following command to create a 10 GB LV in VG  **vg1**.

```bash
lvcreate -L 10G vg1
```

Example 2: Run the following command to create a 200 MB LV in VG  **vg1**  and name the LV  **lv1**.

```bash
lvcreate -L 200M -n lv1 vg1
```

### Viewing an LV

Run the  **lvdisplay**  command as the **root** user to view the LV information, including the size of the LV, its read and write status, and snapshot information.

```bash
lvdisplay [option] [lvname]
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-v**: displays the mapping from LEs to PEs.

* *lvname*: device file corresponding to the LV whose attributes are to be displayed. If this option is not set, attributes of all LVs are displayed.

  > \[!NOTE] **NOTE:**
  > Device files corresponding to LVs are stored in the VG directory. For example, if LV  **lv1**  is created in VG  **vg1**, the device file corresponding to  **lv1**  is  **/dev/vg1/lv1**.

Example: Run the following command to display the basic information about LV  **lv1**:

```bash
lvdisplay /dev/vg1/lv1
```

### Adjusting the LV Size

Run the  **lvresize**  command as the **root** user to increase or reduce the size of an LVM LV. This may cause data loss. Therefore, exercise caution when running this command.

```bash
lvresize [option] vgname
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-L**: specifies the size of the LV in kKmMgGtT.
  * **-l**: specifies the size of the LV (number of LEs).
  * **-f**: forcibly adjusts the size of the LV without user confirmation.

* *lvname*: name of the LV to be adjusted.

Example 1: Run the following command to increase the size of LV  **/dev/vg1/lv1**  by 200 MB.

```bash
lvresize -L +200 /dev/vg1/lv1
```

Example 2: Run the following command to reduce the size of LV  **/dev/vg1/lv1**  by 200 MB.

```bash
lvresize -L -200 /dev/vg1/lv1
```

### Extending an LV

Run the  **lvextend**  command as the **root** user to dynamically extend the size of an LV online without interrupting the access of applications to the LV.

```bash
lvextend [option] lvname
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-L**: specifies the size of the LV in kKmMgGtT.
  * **-l**: specifies the size of the LV (number of LEs).
  * **-f**: forcibly adjusts the size of the LV without user confirmation.

* *lvname*: device file of the LV whose size is to be extended.

Example: Run the following command to increase the size of LV  **/dev/vg1/lv1**  by 100 MB.

```bash
lvextend -L +100M /dev/vg1/lv1
```

### Shrinking an LV

Run the  **lvreduce**  command as the **root** user to reduce the size of an LV. This may delete existing data on the LV. Therefore, confirm whether the data can be deleted before running the command.

```bash
lvreduce [option] lvname
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-L**: specifies the size of the LV in kKmMgGtT.
  * **-l**: specifies the size of the LV (number of LEs).
  * **-f**: forcibly adjusts the size of the LV without user confirmation.

* *lvname*: device file of the LV whose size is to be extended.

Example: Run the following command to reduce the space of LV  **/dev/vg1/lvl**  by 100 MB:

```bash
lvreduce -L -100M /dev/vg1/lv1
```

### Deleting an LV

Run the  **lvremove**  command as the **root** user to delete an LV. If the LV has been mounted by running the  **mount**  command, you need to run the  **umount**  command to unmount the LV before running the  **lvremove**  command.

```bash
lvremove [option] lvname
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-f**: forcibly deletes an LV without user confirmation.

* *lvname*: device name of the LV to be deleted.

Example: Run the following command to delete LV  **/dev/vg1/lv1**.

```bash
lvremove /dev/vg1/lv1
```

## Creating and Mounting a File System

After creating an LV, you need to create a file system on the LV and mount the file system to the corresponding directory.

### Creating a File System

Run the  **mkfs**  command as the **root** user to create a file system.

```bash
mkfs [option] lvname
```

In the preceding information:

* *option*: command parameter options. Common parameter options are as follows:
  * **-t**: specifies the type of the Linux file system to be created, such as  **ext2**,  **ext3**, and  **ext4**. The default type is  **ext2**.

* *lvname*: name of the LV device file corresponding to the file system to be created.

Example: Run the following command to create the  **ext4**  file system on LV  **/dev/vg1/lv1**:

```bash
mkfs -t ext4 /dev/vg1/lv1
```

### Manually Mounting a File System

The file system that is manually mounted is not valid permanently. It does not exist after the OS is restarted.

Run the  **mount**  command as the **root** user to mount a file system.

```bash
mount lvname mntpath
```

In the preceding information:

* *lvname*: name of the LV device file corresponding to the file system to be mounted.
* *mntpath*: mount path.

Example: Run the following command to mount LV  **/dev/vg1/lv1**  to the directory  **/mnt/data**.

```bash
mount /dev/vg1/lv1 /mnt/data
```

### Automatically Mounting a File System

A file system that is automatically mounted does not exist after the OS is restarted. You need to manually mount the file system again. If you perform the following steps as the **root** user after manually mounting the file system, the file system can be automatically mounted after the OS is restarted.

1. Run the  **blkid**  command to query the UUID of an LV. The following uses LV  **/dev/vg1/lv1**  as an example:

   ```bash
   blkid /dev/vg1/lv1
   ```

   Check the command output. It contains the following information in which  *uuidnumber*  is a string of digits, indicating the UUID, and  *fstype*  indicates the file system type.

   /dev/vg1/lv1: UUID="  *uuidnumber*  " TYPE="  *fstype*  "

2. Run the  **vi /etc/fstab**  command to edit the  **fstab**  file and add the following content to the end of the file:

   ```vim
   UUID=uuidnumber  mntpath                   fstype    defaults        0 0
   ```

   In the preceding information:

   * Column 1: indicates the UUID. Enter  *uuidnumber*  obtained in  [1](#li65701520154311).
   * Column 2: indicates the mount directory of the file system. Replace  *mntpath*  with the actual value.
   * Column 3: indicates the file system format. Enter  *fstype*  obtained in  [1](#li65701520154311).
   * Column 4: indicates the mount option. In this example,  **defaults**  is used.
   * Column 5: indicates the backup option. Enter either  **1**  (the system automatically backs up the file system) or  **0**  (the system does not back up the file system). In this example,  **0**  is used.
   * Column 6: indicates the scanning option. Enter either  **1**  (the system automatically scans the file system during startup) or  **0**  (the system does not scan the file system). In this example,  **0**  is used.

3. Verify the automatic mounting function.
   1. Run the  **umount**  command to unmount the file system. The following uses LV  **/dev/vg1/lv1**  as an example:

      ```bash
      umount /dev/vg1/lv1
      ```

   2. Run the following command to reload all content in the  **/etc/fstab**  file:

      ```bash
      mount -a
      ```

   3. Run the following command to query the file system mounting information (**/mnt/data**  is used as an example):

      ```bash
      mount | grep /mnt/data
      ```

      Check the command output. If the command output contains the following information, the automatic mounting function takes effect:

      ```text
      /dev/vg1/lv1 on /mnt/data
      ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/secure_container/managing_the_lifecycle_of_a_secure_container.md
---
# Managing the Lifecycle of a Secure Container

## Starting a Secure Container

You can use the Docker engine or iSulad as the container engine of the secure container. The invoking methods of the two engines are similar. You can select either of them to start a secure container.

To start a secure container, perform the following steps:

1. Ensure that the secure container component has been correctly installed and deployed.

2. Prepare the container image. If the container image is busybox, run the following commands to download the container image using the Docker engine or iSulad:

   ```sh
   docker pull busybox
   ```

   ```sh
   isula pull busybox
   ```

3. Start a secure container. Run the following commands to start a secure container using the Docker engine and iSulad:

   ```sh
   docker run -tid --runtime kata-runtime --network none busybox <command>
   ```

   ```sh
   isula run -tid --runtime kata-runtime --network none busybox <command>
   ```

   > \[!NOTE] **NOTE:**\
   > The secure container supports the CNI network only and does not support the CNM network. The  **-p**  and  **--expose**  options cannot be used to expose container ports. When using a secure container, you need to specify the  **--net=none**  option.

4. Start a pod.
   1. Start the pause container and obtain the sandbox ID of the pod based on the command output. Run the following commands to start a pause container using the Docker engine and iSulad:

      ```sh
      docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=podsandbox <pause-image> <command>
      ```

      ```sh
      isula run -tid --runtime kata-runtime --network none --annotation io.kubernetes.cri.container-type=sandbox <pause-image> <command>
      ```

   2. Create a service container and add it to the pod. Run the following commands to create a service container using the Docker engine and iSulad:

      ```sh
      docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=container --annotation io.kubernetes.sandbox.id=<sandbox-id> busybox <command>
      ```

      ```sh
      isula run -tid --runtime kata-runtime --network none --annotation io.kubernetes.cri.container-type=container --annotation io.kubernetes.cri.sandbox-id=<sandbox-id> busybox <command>
      ```

      **--annotation**  is used to mark the container type, which is provided by the Docker engine and iSulad, but not provided by the open-source Docker engine in the upstream community.

## Stopping a Secure Container

* Run the following command to stop a secure container:

  ```sh
  docker stop <container-id>
  ```

* Stop a pod.

  When stopping a pod, note that the lifecycle of the pause container is the same as that of the pod. Therefore, stop service containers before the pause container.

## Deleting a Secure Container

Ensure that the container has been stopped.

```sh
docker rm <container-id>
```

To forcibly delete a running container, run the  **-f**  command.

```sh
docker rm -f <container-id>
```

## Running a New Command in the Container

The pause container functions only as a placeholder container. Therefore, if you start a pod, run a new command in the service container. The pause container does not execute the corresponding command. If only one container is started, run the following command directly:

```sh
docker exec -ti <container-id> <command>
```

> \[!NOTE] **NOTE:**
>
> 1. If the preceding command has no response because another host runs the  **docker restart**  or  **docker stop**  command to access the same container, you can press  **Ctrl**+**P**+**Q**  to exit the operation.
> 2. If the  **-d**  option is used, the command is executed in the background and no error information is displayed. The exit code cannot be used to determine whether the command is executed correctly.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/vm_management.md
---
# Managing VMs

## Overview

StratoVirt allows you to query VM information and manage VM resources and lifecycle with QMP. To query the information about a VM, connect to the VM first.

## Querying VM Information

### Introduction

StratoVirt can be used to query the VM status, vCPU topology, and vCPU online status.

### Querying VM Status

Run the **query-status** command to query the running status of a VM.

* Usage:

  **{ "execute": "query-status" }**

* Example:

```text
<- { "execute": "query-status" }
-> { "return": { "running": true,"singlestep": false,"status": "running" } 
```

### Querying Topology Information

Run the **query-cpus** command to query the topologies of all CPUs.

* Usage:

  **{ "execute": "query-cpus" }**

* Example:

```text
<- { "execute": "query-cpus" }
-> {"return":[{"CPU":0,"arch":"x86","current":true,"halted":false,"props":{"core-id":0,"socket-id":0,"thread-id":0},"qom_path":"/machine/unattached/device[0]","thread_id":8439},{"CPU":1,"arch":"x86","current":true,"halted":false,"props":{"core-id":0,"socket-id":1,"thread-id":0},"qom_path":"/machine/unattached/device[1]","thread_id":8440}]}
```

### Querying vCPU Online Status

Run the **query-hotpluggable-cpus** command to query the online/offline statuses of all vCPUs.

* Usage:
  **{ "execute": "query-hotpluggable-cpus" }**

* Example:

```text
<- { "execute": "query-hotpluggable-cpus" }
-> {"return":[{"props":{"core-id":0,"socket-id":0,"thread-id":0},"qom-path":"/machine/unattached/device[0]","type":"host-x86-cpu","vcpus-count":1},{"props":{"core-id":0,"socket-id":1,"thread-id":0},"qom-path":"/machine/unattached/device[1]","type":"host-x86-cpu","vcpus-count":1}]}
```

Online vCPUs have the `qom-path` item, while offline vCPUs do not.

## Managing VM Lifecycle

### Introduction

StratoVirt can manage the lifecycle of a VM, including starting, stopping, resuming, and exiting the VM.

### Creating and Starting a VM

Use the command line parameters to specify the VM configuration, and create and start a VM.

* When using the command line parameters to specify the VM configuration, run the following command to create and start the VM:

```shell
$/path/to/stratovirt - *[Parameter 1] [Parameter option] - [Parameter 2] [Parameter option]*...
```

> \[!NOTE]
>
> After the lightweight VM is started, there are two NICs: eth0 and eth1. The two NICs are reserved for hot plugging: eth0 first and then eth1. Currently, only two virtio-net NICs can be hot plugged.

### Connecting to a VM

StratoVirt uses QMP to manage VMs. To stop, resume, or exit a VM, connect it the StratoVirt through QMP first.

Open a new CLI (CLI B) on the host and run the following command to connect to the api-channel as the **root** user:

```shell
ncat -U /path/to/socket
```

After the connection is set up, you will receive a greeting message from StratoVirt, as shown in the following:

```text
{"QMP":{"version":{"qemu":{"micro":1,"minor":0,"major":4},"package":""},"capabilities":[]}}
```

You can now manage the VM by entering the QMP commands in CLI B.

> \[!NOTE]
>
> QMP provides **stop**, **cont**, **quit**, and **query-status** commands to manage and query VM statuses.
>
> All QMP commands for managing VMs are entered in CLI B. `<-` indicates the command input, and `->` indicates the QMP returned result.

### Stopping a VM

QMP provides the **stop** command to stop a VM, that is, to stop all vCPUs of the VM. The command syntax is as follows:

**{"execute":"stop"}**

**Example:**

The **stop** command and the command output are as follows:

```text
<- {"execute":"stop"}
-> {"event":"STOP","data":{},"timestamp":{"seconds":1583908726,"microseconds":162739}}
-> {"return":{}}
```

### Resuming a VM

QMP provides the **cont** command to resume a stopped VM, that is, to resume all vCPUs of the VM. The command syntax is as follows:

**{"execute":"cont"}**

**Example:**

The **cont** command and the command output are as follows:

```text
<- {"execute":"cont"}
-> {"event":"RESUME","data":{},"timestamp":{"seconds":1583908853,"microseconds":411394}}
-> {"return":{}}
```

### Exiting a VM

QMP provides the **quit** command to exit a VM, that is, to exit the StratoVirt process. The command syntax is as follows:

**{"execute":"quit"}**

**Example:**

```text
<- {"execute":"quit"}
-> {"return":{}}
-> {"event":"SHUTDOWN","data":{"guest":false,"reason":"host-qmp-quit"},"timestamp":{"ds":1590563776,"microseconds":519808}}
```

## Managing VM Resources

### Hot-Pluggable Disks

StratoVirt allows you to adjust the number of disks when a VM is running. That is, you can add or delete VM disks without interrupting services.

**Note**

* For a standard VM, the **CONFIG\_HOTPLUG\_PCI\_PCIE=y** configuration must be enabled for the VM kernel.

* For a standard VM, devices can be hot added to the root port. The root port device must be configured before the VM is started.

* You are not advised to hot swap a device when the VM is being started, stopped, or under high internal pressure. Otherwise, the VM may become abnormal because the drivers on the VM cannot respond in a timely manner.

#### Hot Adding Disks

**Usage:**

Lightweight VM:

```text
{"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}}
{"execute": "device_add", "arguments": {"id": "drive-0", "driver": "virtio-blk-mmio", "addr": "0x1"}}
```

Standard VM:

```text
{"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}}
{"execute":"device_add", "arguments":{"id":"drive-0", "driver":"virtio-blk-pci", "drive": "drive-0", "addr":"0x0", "bus": "pcie.1"}}
```

**Parameters:**

* For a lightweight VM, the value of **node-name** in **blockdev-add** must be the same as that of **id** in **device\_add**. For example, the values of **node-name** and **id** are both **drive-0** as shown above.

* For a standard VM, the value of **drive** must be the same as that of **node-name** in **blockdev-add**.

* **/path/to/block** is the image path of the hot added disks. It cannot be the path of the disk image that boots the rootfs.

* For a lightweight VM, the value of **addr**, starting from **0x0**, is mapped to a virtio device on the VM. **0x0** is mapped to **vda**, **0x1** is mapped to **vdb**, and so on. To be compatible with the QMP protocol, **addr** can be replaced by **lun**, but **lun=0** is mapped to the **vdb** of the guest machine. For a standard VM, the value of **addr** must be **0x0**.

* For a standard VM, **bus** indicates the name of the bus to mount the device. Currently, the device can be hot added only to the root port device. The value of **bus** must be the ID of the root port device.

* For a lightweight VM, StratoVirt supports a maximum of six virtio-blk disks. Note this when hot adding disks. For a standard VM, the maximum number of hot added disks depends on the number of root port devices.

**Example:**

Lightweight VM:

```text
<- {"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}}
-> {"return": {}}
<- {"execute": "device_add", "arguments": {"id": "drive-0", "driver": "virtio-blk-mmio", "addr": "0x1"}}
-> {"return": {}}
```

Standard VM:

```text
<- {"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}}
-> {"return": {}}
<- {"execute":"device_add", "arguments":{"id":"drive-0", "driver":"virtio-blk-pci", "drive": "drive-0", "addr":"0x0", "bus": "pcie.1"}}
-> {"return": {}}
```

#### Hot Removing Disks

**Usage:**

Lightweight VM:

```text
{"execute": "device_del", "arguments": {"id":"drive-0"}}
```

Standard VM:

```text
{"execute": "device_del", "arguments": {"id":"drive-0"}}
{"execute": "blockdev-del", "arguments": {"node-name": "drive-0"}}
```

**Parameters:**

**id** indicates the ID of the disk to be hot removed.

* **node-name** indicates the backend name of the disk.

**Example:**

Lightweight VM:

```text
<- {"execute": "device_del", "arguments": {"id": "drive-0"}}
-> {"event":"DEVICE_DELETED","data":{"device":"drive-0","path":"drive-0"},"timestamp":{"seconds":1598513162,"microseconds":367129}}
-> {"return": {}}
```

Standard VM:

```text
<- {"execute": "device_del", "arguments": {"id":"drive-0"}}
-> {"return": {}}
-> {"event":"DEVICE_DELETED","data":{"device":"drive-0","path":"drive-0"},"timestamp":{"seconds":1598513162,"microseconds":367129}}
<- {"execute": "blockdev-del", "arguments": {"node-name": "drive-0"}}
-> {"return": {}}
```

A **DEVICE\_DELETED** event indicates that the device is removed from StratoVirt.

### Hot-Pluggable NICs

StratoVirt allows you to adjust the number of NICs when a VM is running. That is, you can add or delete VM NICs without interrupting services.

**Note**

* For a standard VM, the **CONFIG\_HOTPLUG\_PCI\_PCIE=y** configuration must be enabled for the VM kernel.

* For a standard VM, devices can be hot added to the root port. The root port device must be configured before the VM is started.

* You are not advised to hot swap a device when the VM is being started, stopped, or under high internal pressure. Otherwise, the VM may become abnormal because the drivers on the VM cannot respond in a timely manner.

#### Hot Adding NICs

**Preparations (Requiring the root Permission)**

1. Create and enable a Linux bridge. For example, if the bridge name is **qbr0**, run the following command:

   ```shell
   brctl addbr qbr0
   ifconfig qbr0 up
   ```

2. Create and enable a tap device. For example, if the tap device name is **tap0**, run the following command:

   ```shell
   ip tuntap add tap0 mode tap
   ifconfig tap0 up
   ```

3. Add the tap device to the bridge.

   ```shell
   brctl addif qbr0 tap0
   ```

**Usage:**

Lightweight VM:

```text
{"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}}
{"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-mmio", "addr":"0x0"}}
```

Standard VM:

```text
{"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}}
{"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-pci", "addr":"0x0", "netdev": "net-0", "bus": "pcie.1"}}
```

**Parameters:**

* For a lightweight VM, **id** in **netdev\_add** must be the same as that in **device\_add**. **ifname** is the name of the backend tap device.

* For a standard VM, the value of **netdev** must be the value of **id** in **netdev\_add**.

* For a lightweight VM, the value of **addr**, starting from **0x0**, is mapped to an NIC on the VM. **0x0** is mapped to **eth0**, **0x1** is mapped to **eth1**. For a standard VM, the value of **addr** must be **0x0**.

* For a standard VM, **bus** indicates the name of the bus to mount the device. Currently, the device can be hot added only to the root port device. The value of **bus** must be the ID of the root port device.

* For a lightweight VM, StratoVirt supports a maximum of two virtio-net NICs. Therefore, pay attention to the specification restrictions when hot adding in NICs. For a standard VM, the maximum number of hot added disks depends on the number of root port devices.

**Example:**

Lightweight VM:

```text
<- {"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}}
-> {"return": {}}
<- {"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-mmio", "addr":"0x0"}} 
-> {"return": {}}
```

**addr:0x0** corresponds to **eth0** in the VM.

Standard VM:

```text
<- {"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}}
-> {"return": {}}
<- {"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-pci", "addr":"0x0", "netdev": "net-0", "bus": "pcie.1"}}
-> {"return": {}}
```

#### Hot Removing NICs

**Usage:**

Lightweight VM:

```text
{"execute": "device_del", "arguments": {"id": "net-0"}}
```

Standard VM:

```text
{"execute": "device_del", "arguments": {"id":"net-0"}}
{"execute": "netdev_del", "arguments": {"id": "net-0"}}
```

**Parameters:**

**id**: NIC ID, for example, **net-0**.

* **id** in **netdev\_del** indicates the backend name of the NIC.

**Example:**

Lightweight VM:

```text
<- {"execute": "device_del", "arguments": {"id": "net-0"}}
-> {"event":"DEVICE_DELETED","data":{"device":"net-0","path":"net-0"},"timestamp":{"seconds":1598513339,"microseconds":97310}}
-> {"return": {}}
```

Standard VM:

```text
<- {"execute": "device_del", "arguments": {"id":"net-0"}}
-> {"return": {}}
-> {"event":"DEVICE_DELETED","data":{"device":"net-0","path":"net-0"},"timestamp":{"seconds":1598513339,"microseconds":97310}}
<- {"execute": "netdev_del", "arguments": {"id": "net-0"}}
-> {"return": {}}
```

A **DEVICE\_DELETED** event indicates that the device is removed from StratoVirt.

### Hot-swappable Pass-through Devices

You can add or delete the passthrough devices of a StratoVirt standard VM when it is running.

**Note**

* The **CONFIG\_HOTPLUG\_PCI\_PCIE=y** configuration must be enabled for the VM kernel.

* Devices can be hot added to the root port. The root port device must be configured before the VM is started.

* You are not advised to hot swap a device when the VM is being started, stopped, or under high internal pressure. Otherwise, the VM may become abnormal because the drivers on the VM cannot respond in a timely manner.

#### Hot Adding Pass-through Devices

**Usage:**

```text
{"execute":"device_add", "arguments":{"id":"vfio-0", "driver":"vfio-pci", "bus": "pcie.1", "addr":"0x0", "host": "0000:1a:00.3"}}
```

**Parameters:**

* **id** indicates the ID of the hot added device.

* **bus** indicates the name of the bus to mount the device.

* **addr** indicates the slot and function numbers to mount the device. Currently, **addr** must be set to **0x0**.

* **host** indicates the domain number, bus number, slot number, and function number of the passthrough device on the host machine.

**Example:**

```text
<- {"execute":"device_add", "arguments":{"id":"vfio-0", "driver":"vfio-pci", "bus": "pcie.1", "addr":"0x0", "host": "0000:1a:00.3"}}
-> {"return": {}}
```

#### Hot Removing Pass-through Devices

**Usage:**

```text
{"execute": "device_del", "arguments": {"id": "vfio-0"}}
```

**Parameters:**

* **id** indicates the ID of the device to be hot removed, which is specified when the device is hot added.

**Example:**

```text
<- {"execute": "device_del", "arguments": {"id": "vfio-0"}}
-> {"return": {}}
-> {"event":"DEVICE_DELETED","data":{"device":"vfio-0","path":"vfio-0"},"timestamp":{"seconds":1614310541,"microseconds":554250}}
```

A **DEVICE\_DELETED** event indicates that the device is removed from StratoVirt.

## Using Balloon Devices

The balloon device is used to reclaim idle memory from a VM. It called by running the QMP command.

**Usage:**

```text
{"execute": "balloon", "arguments": {"value": 2147483648}}
```

**Parameters:**

* **value**: size of the guest memory to be set. The unit is byte. If the value is greater than the memory value configured during VM startup, the latter is used.

**Example:**

The memory size configured during VM startup is 4 GiB. If the idle memory of the VM queried by running the free command is greater than 2 GiB, you can run the QMP command to set the guest memory size to 2147483648 bytes.

```text
<- {"execute": "balloon", "arguments": {"value": 2147483648}}
-> {"return": {}}
```

Query the actual memory of the VM:

```text
<- {"execute": "query-balloon"}
-> {"return":{"actual":2147483648}}
```

## Using VM Memory Snapshots

### Introduction

A VM memory snapshot stores the device status and memory information of a VM in a snapshot file. If the VM is damaged, you can use the snapshot to restore it to the time when the snapshot was created, improving system reliability.

StratoVirt allows you to create snapshots for stopped VMs and create VMs in batches with a snapshot file as the VM template. As long as a snapshot is created after a VM is started and enters the user mode, the quick startup can skip the kernel startup and user-mode service initialization phases and complete the VM startup in milliseconds.

### Mutually Exclusive Features

Memory snapshots cannot be created or used for VMs that are configured with the following devices or use the following features:

* vhost-net device
* VFIO passthrough device
* Balloon device
* Huge page memory feature
* mem-shared feature
* memory backend file **mem-path**

### Creating a Snapshot

For StratoVirt VMs, perform the following steps to create a storage snapshot:

1. Create and start a VM.

2. Run the QMP command on the host to stop the VM.

   ```text
   <- {"execute":"stop"}
   -> {"event":"STOP","data":{},"timestamp":{"seconds":1583908726,"microseconds":162739}}
   -> {"return":{}}
   ```

3. Confirm that the VM is stopped.

   ```text
   <- {"execute":"query-status"}
   -> {"return":{"running":true,"singlestep":false,"status":"paused"}}
   ```

4. Run the following QMP command to create a VM snapshot in a specified absolute path, for example, **/path/to/template**:

   ```text
   <- {"execute":"migrate", "arguments":{"uri":"file:/path/to/template"}}
   -> {"return":{}}
   ```

5. Check whether the snapshot is successfully created.

   ```text
   <- {"execute":"query-migrate"}
   ```

   If "{"return":{"status":"completed"}}" is displayed, the snapshot is successfully created.

   If the snapshot is created successfully, the `memory` and `state` directories are generated in the specified path **/path/to/template**. The `state` file contains VM device status information, and the `memory` file contains VM memory data. The size of the `memory` file is close to the configured VM memory size.

### Querying Snapshot Status

There are five statuses in the snapshot process.

* `None`: The snapshot resource is not ready.
* `Setup`: The snapshot resource is ready. You can create a snapshot.
* `Active`: The snapshot is being created.
* `Completed`: The snapshot is created successfully.
* `Failed`: The snapshot fails to be created.

You can run the `query-migrate` QMP command on the host to query the status of the current snapshot. For example, if the VM snapshot is created successfully, the following output is displayed:

```text
<- {"execute":"query-migrate"}
-> {"return":{"status":"completed"}}
```

### Restoring a VM

#### Precautions

* The following models support the snapshot and boot from snapshot features:
  * microvm
  * Q35 (x86\_64)
  * virt (AArch64)
* When a snapshot is used for restoration, the configured devices must be the same as those used when the snapshot is created.
* If a microVM is used and the disk/NIC hot plugging-in feature is enabled before the snapshot is taken, you need to configure the hot plugged-in disks or NICs in the startup command line during restoration.

#### Restoring a VM from a Snapshot File

**Command Format**

```shell
stratovirt -incoming URI
```

**Parameters**

**URI**: snapshot path. The current version supports only the `file` type, followed by the absolute path of the snapshot file.

**Example**

Assume that the VM used for creating a snapshot is created by running the following command:

```shell
$ stratovirt \
    -machine microvm \
    -kernel /path/to/kernel \
    -smp 1 -m 1024 \
    -append "console=ttyS0 pci=off reboot=k quiet panic=1 root=/dev/vda" \
    -drive file=/path/to/rootfs,id=rootfs,readonly=off,direct=off \
    -device virtio-blk-device,drive=rootfs \
    -qmp unix:/path/to/socket,server,nowait \
    -serial stdio
```

Then, the command for restoring the VM from the snapshot (assume that the snapshot storage path is **/path/to/template**) is as follows:

```shell
$ stratovirt \
    -machine microvm \
    -kernel /path/to/kernel \
    -smp 1 -m 1024 \
    -append "console=ttyS0 pci=off reboot=k quiet panic=1 root=/dev/vda" \
    -drive file=/path/to/rootfs,id=rootfs,readonly=off,direct=off \
    -device virtio-blk-device,drive=rootfs \
    -qmp unix:/path/to/another_socket,server,nowait \
    -serial stdio \
    -incoming file:/path/to/template
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/managing_vms.md
---
# Managing VMs

## VM Life Cycle

### Introduction

#### Overview

To leverage hardware resources and reduce costs, users need to properly manage VMs. This section describes basic operations during the VM lifecycle, such as creating, using, and deleting VMs.

#### VM Status

A VM can be in one of the following status:

* **undefined**: The VM is not defined or created. That is, libvirt considers that the VM does not exist.
* **shut off**: The VM has been defined but is not running, or the VM is terminated.
* **running**: The VM is running.
* **paused**: The VM is suspended and its running status is temporarily stored in the memory. The VM can be restored to the running status.
* **saved**: Similar to the  **paused**  status, the running state is stored in a persistent storage medium and can be restored to the running status.
* **crashed**: The VM crashes due to an internal error and cannot be restored to the running status.

#### Status Transition

VMs in different status can be converted, but certain rules must be met.  [Figure 1](#fig671014583483)  describes the common rules for transiting the VM status.

**Figure  1**  Status transition diagram\
![](./figures/status-transition-diagram.png)

#### VM ID

In libvirt, a created VM instance is called a  **domain**, which describes the configuration information of resources such as the CPU, memory, network device, and storage device of the VM. On a host, each domain has a unique ID, which is represented by the VM  **Name**,  **UUID**, and  **Id**. For details, see  [Table 1](#table84397266483). During the VM lifecycle, an operation can be performed on a specific VM by using a VM ID.

**Table  1**  Domain ID description

> \[!NOTE] **NOTE:**\
> Run the  **virsh**  command to query the VM ID and UUID. For details, see  [Querying VM Information](#querying-vm-information).

### Management Commands

#### Overview

You can use the  **virsh**  command tool to manage the VM lifecycle. This section describes the commands related to the lifecycle.

#### Prerequisites

* Before performing operations on a VM, you need to query the VM status to ensure that the operations can be performed. For details about the conversion between status, see  [Status Transition](#status-transition).
* You have administrator rights.
* The VM XML configuration files are prepared.

#### Command Usage

You can run the  **virsh**  command to manage the VM lifecycle. The command format is as follows:

```shell
virsh <operate> <obj> <options>
```

The parameters are described as follows:

* *operate*: manages VM lifecycle operations, such as creating, deleting, and starting VMs.
* *obj*: specifies the operation object, for example, the VM to be operated.
* *options*: command option. This parameter is optional.

[Table 2](#table389518422611)  describes the commands used for VM lifecycle management.  *VMInstance*  indicates the VM name, VM ID, or VM UUID,  *XMLFile*  indicates the XML configuration file of the VM, and  *DumpFile*  indicates the dump file. Change them based on the site requirements.

**Table  2**  VM Lifecycle Management Commands

### Example

This section provides examples of commands related to VM life cycle management.

* Create a VM.

  The VM XML configuration file is  **openEulerVM.xml**. The command and output are as follows:

  ```shell
  $ virsh define openEulerVM.xml
  Domain openEulerVM defined from openEulerVM.xml
  ```

* Start a VM.

  Run the following command to start the  *openEulerVM*:

  ```shell
  $ virsh start openEulerVM
  Domain openEulerVM started
  ```

* Reboot a VM.

  Run the following command to reboot the  *openEulerVM*:

  ```shell
  $ virsh reboot openEulerVM
  Domain openEulerVM is being rebooted
  ```

* Shut down a VM.

  Run the following command to shut down the  *openEulerVM*:

  ```shell
  $ virsh shutdown openEulerVM
  Domain openEulerVM is being shutdown
  ```

* Destroy a VM.
  * If the  **nvram**  file is not used during the VM startup, run the following command to destroy the VM:

    ```shell
    virsh undefine <VMInstance>
    ```

  * If the  **nvram**  file is used during the VM startup, run the following command to specify the  **nvram**  processing policy when destroying the VM:

    ```shell
    virsh undefine <VMInstance> <strategy>
    ```

    *strategy*  indicates the policy for destroying a VM. The values can be:

    \--**nvram**: delete the corresponding  **nvram**  file when destroying a VM.

    \--**keep-nvram**: destroy a VM but retain the corresponding  **nvram**  file.

    For example, to delete the  *openEulerVM*  and its  **nvram**  file, run the following command:

    ```shell
    $ virsh undefine openEulerVM --nvram
    Domain openEulerVM has been undefined
    ```

## Modifying VM Configurations Online

### Overview

After a VM is created, users can modify VM configurations. This process is called online modification of VM configuration. After the configuration is modified online, the new VM configuration file is persistent and takes effect after the VM is shut down and restarted.

The format of the command for modifying VM configuration is as follows:

```shell
virsh edit <VMInstance>
```

The  **virsh edit**  command is used to edit the XML configuration file corresponding to  **domain**  to update VM configuration.  **virsh edit**  uses the  **vi**  program as the default editor. You can specify the editor type by modifying the environment variable  *EDITOR*  or  *VISUAL*. By default,  **virsh edit**  preferentially uses the text editor specified by the environment variable  *VISUAL*.

### Procedure

1. (Optional) Set the editor of the  **virsh edit**  command to  **vim**.

   ```shell
   export VISUAL=vim
   ```

2. Run the  **virsh edit**  command to open the XML configuration file of the  *openEulerVM*.

   ```shell
   virsh edit openEulerVM
   ```

3. Modify the VM configuration file.

4. Save the VM configuration file and exit.

5. Shut down the VM.

   ```shell
   virsh shutdown openEulerVM
   ```

6. Start the VM for the modification to take effect.

   ```shell
   virsh start openEulerVM
   ```

## Querying VM Information

### Overview

The libvirt provides a set of command line tools to query VM information. This section describes how to use commands to obtain VM information.

### Prerequisites

To query VM information, the following requirements must be met:

* The libvirtd service is running.

* Only the administrator has the permission to execute command line.

### Querying VM Information on a Host

* Query the list of running and paused VMs on a host.

  ```shell
  virsh list
  ```

  For example, the following command output indicates that three VMs exist on the host.  **openEulerVM01**  and  **openEulerVM02**  are running, and  **openEulerVM03**  is paused.

  ```text
   Id    Name                           State
  ----------------------------------------------------
   39    openEulerVM01                   running
   40    openEulerVM02                   running
   69    openEulerVM03                   paused
  ```

* Query the list of VM information defined on a host.

  ```shell
  virsh list --all
  ```

  For example, the following command output indicates that four VMs are defined on the current host.  **openEulerVM01**  is running,  **openEulerVM02**  is paused, and  **openEulerVM03**  and  **openEulerVM04**  are shut down.

  ```text
   Id    Name                           State
  ----------------------------------------------------
   39    openEulerVM01                  running
   69    openEulerVM02                  paused
   -     openEulerVM03                  shut off
   -     openEulerVM04                  shut off
  ```

### Querying Basic VM Information

Libvirt component provides a group of commands for querying the VM status, including the VM running status, device information, and scheduling attributes. For details, see  [Table 3](#table10582103963816).

**Table  3**  Querying basic VM information

### Example

* Run the  **virsh dominfo**  command to query the basic information about a created VM. The query result shows that the VM ID is  **5**, UUID is  **ab472210-db8c-4018-9b3e-fc5319a769f7**, memory size is 8 GiB, and the number of vCPUs is 4.

  ```shell
  $ virsh dominfo openEulerVM
  Id:             5
  Name:           openEulerVM
  UUID:           ab472210-db8c-4018-9b3e-fc5319a769f7
  OS Type:        hvm
  State:          running
  CPU(s):         4
  CPU time:       6.8s
  Max memory:     8388608 KiB
  Used memory:    8388608 KiB
  Persistent:     no
  Autostart:      disable
  Managed save:   no
  Security model: none
  Security DOI:   0
  ```

* Run the  **virsh domstate**  command to query the VM status. The query result shows that VM  **openEulerVM**  is running.

  ```shell
  $ virsh domstate openEulerVM
  running
  ```

* Run  **virsh schedinfo**  to query the VM scheduling information. The query result shows that the CPU reservation share of the VM is 1024.

  ```shell
  $ virsh schedinfo openEulerVM
  Scheduler      : posix
  cpu_shares     : 1024
  vcpu_period    : 100000
  vcpu_quota     : -1
  emulator_period: 100000
  emulator_quota : -1
  global_period  : 100000
  global_quota   : -1
  iothread_period: 100000
  iothread_quota : -1
  ```

* Run the  **virsh vcpucount**  command to query the number of vCPUs. The query result shows that the VM has four CPUs.

  ```shell
  $ virsh vcpucount openEulerVM
  maximum      live           4
  current      live           4
  ```

* Run the  **virsh domblklist**  command to query the VM disk information. The query result shows that the VM has two disks. sda is a virtual disk in qcow2 format, and sdb is a cdrom device.

  ```shell
   $ virsh domblklist openEulerVM
   Target   Source
  ---------------------------------------------------------------------
   sda      /home/openeuler/vm/openEuler_aarch64.qcow2
   sdb      /home/openeuler/vm/openEuler-22.03-LTS-SP4-aarch64-dvd.iso
  ```

* Run the  **virsh domiflist**  command to query the VM NIC information. The query result shows that the VM has one NIC, the backend is vnet0, which is on the br0 bridge of the host. The MAC address is 00:05:fe:d4:f1:cc.

  ```shell
  $ virsh domiflist openEulerVM
  Interface  Type       Source     Model       MAC
  -------------------------------------------------------
  vnet0      bridge     br0        virtio      00:05:fe:d4:f1:cc
  ```

* Run the  **virsh iothreadinfo**  command to query the VM I/O thread information. The query result shows that the VM has five I/O threads, which are scheduled on physical CPUs 7-10.

  ```shell
  $ virsh iothreadinfo openEulerVM
   IOThread ID     CPU Affinity
  ---------------------------------------------------
   3               7-10
   4               7-10
   5               7-10
   1               7-10
   2               7-10
  ```

## Logging In to a VM

This section describes how to log in to a VM using VNC.

### Logging In Using VNC Passwords

#### Overview

After the OS is installed on a VM, you can remotely log in to the VM using VNC to manage the VM.

#### Prerequisites

Before logging in to a VM using a client, such as RealVNC or TightVNC, ensure that:

* You have obtained the IP address of the host where the VM resides.
* The environment where the client resides can access the network of the host.
* You have obtained the VNC listening port of the VM. This port is automatically allocated when the client is started. Generally, the port number is  **5900 + x**  (*x*  is a positive integer and increases in ascending order based on the VM startup sequence.  **5900**  is invisible to users.)
* If a password has been set for the VNC, you also need to obtain the VNC password of the VM.

  > \[!NOTE] **NOTE:**\
  > To set a password for the VM VNC, edit the XML configuration file of the VM. That is, add the  **passwd**  attribute to the  **graphics**  element and set the attribute value to the password to be configured. For example, to set the VNC password of the VM to  **n8VfjbFK**, configure the XML file as follows:
  >
  > ```xml
  >    <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0' keymap='en-us' passwd='n8VfjbFK'>  
  >      <listen type='address' address='0.0.0.0'/>  
  >    </graphics>  
  > ```

#### Procedure

#### Procedure

1. Query the VNC port number used by the VM. For example, if the VM name is  *openEulerVM*, run the following command:

   ```shell
   $ virsh vncdisplay openEulerVM
   :3
   ```

   > \[!NOTE] **NOTE:**\
   > To log in to the VNC, you need to configure firewall rules to allow the connection of the VNC port. The reference command is as follows, where  *X*  is  **5900 + Port number**, for example,  **5903**.
   >
   > ```shell
   > firewall-cmd --zone=public --add-port=X/tcp  
   > ```

2. Start the VncViewer software and enter the IP address and port number of the host. The format is  **host IP address:port number**, for example,  **10.133.205.53:3**.

3. Click  **OK**  and enter the VNC password (optional) to log in to the VM VNC.

### Configuring VNC TLS Login

#### Overview

By default, the VNC server and client transmit data in plaintext. Therefore, the communication content may be intercepted by a third party. To improve security, openEuler allows the VNC server to configure the Transport Layer Security (TLS) mode for encryption and authentication. TLS implements encrypted communication between the VNC server and client to prevent communication content from being intercepted by third parties.

> \[!NOTE] **NOTE:**
>
> * To use the TLS encryption authentication mode, the VNC client must support the TLS mode (for example, TigerVNC). Otherwise, the VNC client cannot be connected.
> * The TLS encryption authentication mode is configured at the host level. After this feature is enabled, the TLS encryption authentication mode is enabled for the VNC clients of all VMs running on the host.

#### Procedure

To enable the TLS encryption authentication mode for the VNC, perform the following steps:

1. Log in to the host where the VNC server resides, and edit the corresponding configuration items in the  **/etc/libvirt/qemu.conf**  configuration file of the server. The configuration is as follows:

   ```text
   vnc_listen = "x.x.x.x"                    # "x.x.x.x" indicates the listening IP address of the VNC. Set this parameter based on the site requirements. The VNC server allows only the connection requests from clients whose IP addresses are in this range.
   vnc_tls = 1                        # If this parameter is set to 1, VNC TLS is enabled.
   vnc_tls_x509_cert_dir = "/etc/pki/libvirt-vnc"                           # Specify /etc/pki/libvirt-vnc as the path for storing the certificate.
   vnc_tls_x509_verify = 1                              #If this parameter is set to 1, the X509 certificate is used for TLS authentication.
   ```

2. Create a certificate and a private key file for the VNC. The following uses GNU TLS as an example.

   > \[!NOTE] **NOTE:**\
   > To use GNU TLS, install the gnu-utils software package in advance.

   1. Create a certificate file issued by the Certificate Authority (CA).

      ```shell
      certtool --generate-privkey > ca-key.pem
      ```

   2. Create a self-signed public and private key for the CA certificate.  *Your organization name*  indicates the organization name, which is specified by the user.

      ```shell
      $ cat > ca.info<<EOF
      cn = Your organization name 
      ca
      cert_signing_key
      EOF
      ```

      ```shell
      certtool --generate-self-signed \
               --load-privkey ca-key.pem \
               --template ca.info \
               --outfile ca-cert.pem
      ```

      In the preceding generated file,  **ca-cert.pem**  is the generated CA public key, and**ca-key.pem**  is the generated CA private key. The CA must keep them properly to prevent disclosure.

   3. Issue a certificate to the VNC server.  **Server Organization Name**  indicates the actual service name, for example,  **server.foo.com**. Set this parameter based on the site requirements.

      ```shell
      $ cat > server.info<<EOF
      cn = Server Organization Name
      tls_www_server
      encryption_key
      signing_key
      EOF
      ```

      ```shell
      certtool --generate-privkey > server-key.pem
      certtool --generate-certificate \
                 --load-ca-certificate ca-cert.pem \
                 --load-ca-privkey ca-key.pem \
                 --load-privkey server-key.pem \
                 --template server.info \
                 --outfile server-cert.pem
      ```

      In the preceding generated file,  **server-key.pem**  is the private key of the VNC server, and  **server-cert.pem**  is the public key of the VNC server.

   4. Issue a certificate to the VNC client.

      ```shell
      $ cat > client.info<<EOF
      cn = Client Organization Name
      tls_www_client
      encryption_key
      signing_key
      EOF
      ```

      ```shell
      certtool --generate-privkey > client-key.pem
      certtool --generate-certificate \
                 --load-ca-certificate ca-cert.pem \
                 --load-ca-privkey ca-key.pem \
                 --load-privkey client-key.pem \
                 --template client.info \
                 --outfile client-cert.pem
      ```

      In the preceding generated file,  **client-key.pem**  is the private key of the VNC client, and  **client-cert.pem**  is the public key of the VNC client. The generated public and private key pairs need to be copied to the VNC client.

3. Shut down the VM to be logged in to and restart the libvirtd service on the host where the VNC server resides.

   ```shell
   systemctl restart libvirtd
   ```

4. Save the generated server certificate to the specified directory on the VNC server and grant the read and write permissions on the certificate only to  the current user.

   ```shell
   sudo mkdir -m 750 /etc/pki/libvirt-vnc
   cp ca-cert.pem /etc/pki/libvirt-vnc/ca-cert.pem
   cp server-cert.pem /etc/pki/libvirt-vnc/server-cert.pem
   cp server-key.pem /etc/pki/libvirt-vnc/server-key.pem
   chmod 0600 /etc/pki/libvirt-vnc/*
   ```

5. Copy the generated client certificates  **ca-cert.pem**,  **client-cert.pem**, and  **client-key.pem**  to the VNC client. After the TLS certificate of the VNC client is configured, you can use VNC TLS to log in to the VM.

   > \[!NOTE] **NOTE:**
   >
   > * For details about how to configure the VNC client certificate, see the usage description of each client.
   > * For details about how to log in to the VM, see Logging In Using VNC Passwords.

## VM Secure Boot

### General Introduction

#### Overview

Secure boot uses public and private key pairs to sign and validate boot components. During the startup, the previous component validates the digital signature of the next component. If the validation is successful, the next component starts. If the validation fails, the startup fails. Secure boot is used to detect whether the firmware and software during startup of the device are tampered with to prevent malware from intrusion and modification. Secure boot ensures the integrity of each component during system startup and prevents unauthorized components from being loaded and running, thereby preventing security threats to the system and user data. Secure boot is implemented based on the UEFI boot mode. It is not supported by the legacy boot mode. According to UEFI specifications, some reliable public keys can be built in the mainboard before delivery. Any operating system or hardware drivers that you want to load on this mainboard must be authenticated by these public keys. The secure boot of a physical machine is implemented by the physical BIOS, while the secure boot of a VM is simulated by software. The process of the VM secure boot is the same as that of the host secure boot, both complying with the open-source UEFI specifications. The UEFI on the virtualization platform is provided by the edk component. When a VM starts, QEMU maps the UEFI image to the memory to simulate the firmware startup process for the VM. Secure boot is a security protection capability provided by edk during the VM startup to protect the OS kernel of the VM from being tampered with. The sequence of signature validation for the secure boot is as follows: UEFI BIOS->shim->GRUB->vmlinuz (signature validation is passed and loaded in sequence).

| English | Acronyms and Abbreviations | Description |
| :----- | :----- | :----- |
| Secure boot | - | Secure boot indicates that a component validates the digital signature of the next component during startup. If the validation is successful, the component runs. If the validation fails, the component stops running. It ensures the integrity of each component during system startup. |
| Platform key | PK | Platform key is owned by the OEM vendor and must be RSA2048 or stronger. The PK establishes a trusted relationship between the platform owner and the platform firmware. The platform owner registers the PKpub, public key of the PK, with the platform firmware. The platform owner can use the PKpriv, private part of the PK, to change the ownership of the platform or register the KEK key. |
| Key exchange key | KEK | Key exchange key creates a trusted relationship between the platform firmware and the OS. Each OS and third-party application that communicates with the platform firmware register the KEKpub, public part of the KEK key, in the platform firmware. |
| Database trustlist | DB | Database trustlist stores and validates the keys of components such as shim, GRUB, and vmlinuz. |
| Database blocklist | DBx | Database blocklist stores revoked keys. |

#### Function Description

The VM secure boot feature is implemented based on the edk open-source project. In non-secure boot mode, the basic Linux process is as follows:

**Figure 1** System startup process

![](./figures/OSBootFlow.png)

In secure boot mode, the first component loaded after UEFI BIOS starts is shim in the system image. By interacting with UEFI BIOS, shim obtains the key stored in the variable DB of UEFI BIOS to validate GRUB. After GRUB is loaded, the key and the authentication API are also called to validate the kernel. The Linux boot process is as follows:

**Figure 2** Secure boot process

![](./figures/SecureBootFlow.png)

The secure boot feature involves multiple key scenarios. Based on the scenario analysis and system breakdown, the secure boot feature involves the following subsystems: UEFI BIOS validating shim, shim validating GRUB, and GRUB validating kernel. When UEFI BIOS validates shim, if the validation is successful, shim is started. If the validation fails, an error message is displayed and shim fails to start. Shim needs to use the private key for signature during image compilation and creation, and the public key certificate needs to be imported to the variable area DB of UEFI BIOS. After shim is started, validate the startup of GRUB. If the validation is successful, GRUB is started. If the validation fails, an error message is displayed and GRUB fails to start. GRUB needs to be signed during image compilation and creation. The public and private key pairs are the same as those of shim. After GRUB is started, it calls the key and the authentication API key registered in UEFI BIOS to validate the kernel. If the validation is successful, GRUB starts the kernel. If the validation fails, an error message is displayed. GRUB needs to sign the image during compilation and creation and uses the public and private key pair that is the same as that of shim.

#### Constraints

* Running on the UEFI BIOS that does not support secure boot does not affect existing functions and services.
* The secure boot feature depends on the UEFI BIOS and takes effect only when the UEFI supports this feature.
* When secure boot is enabled in the UEFI BIOS, the system cannot be started if the related components have no signature or the signature is incorrect.
* If secure boot is disabled in the UEFI BIOS, the validation function during the boot process is disabled.
* The second half of the secure boot validation chain, that is, shim->GRUB->kernel, guides the kernel to start. This part of the validation chain is implemented by the OS image. If the OS does not support guiding the kernel for secure boot, the VM secure boot fails.
* Currently, the x86 architecture do not provide nvram file configuration to configure VM secure boot.

### Secure Boot Practice

VM secure boot depends on UEFI BIOS. The UEFI BIOS image is installed using the edk rpm package. This section uses AArch64 as an example to describe how to configure VM secure boot.

#### Configuring VM

The components in the edk rpm package are installed in the /usr/share/edk2/aarch64 directory, including `QEMU_EFI-pflash.raw` and `vars-template-pflash.raw`. The following describes the XML configuration of the UEFI BIOS during VM startup.

```xml
<os>
    <type arch='aarch64' machine='virt'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader>
    <nvram template='/usr/share/edk2/aarch64/vars-template-pflash.raw'>/path/to/QEMU-VARS.fd</nvram>
</os>
```

In the preceding configuration, /usr/share/edk2/aarch64/QEMU\_EFI-pflash.raw indicates the path of the UEFI BIOS image. /usr/share/edk2/aarch64/vars-template-pflash.raw is the path of the NVRAM image template, and /path/to/QEMU-VARS.fd is the path of the NVRAM image file of the current VM, which is used to store environment variables in the UEFI BIOS.

#### Importing Certificate

The certificate for VM secure boot is imported from the BIOS page. Before importing the certificate, you need to import the certificate file to the VM. You can mount the directory where the certificate file is located to the VM by mounting a disk. For example, you can create an image that contains the certificate and mount the image in the XML configuration file of the VM.

Create a certificate file image.

```shell
dd of='/path/to/data.img' if='/dev/zero' bs=1M count=64
mkfs.vfat -I /path/to/data.img
mkdir /path/to/mnt
mount path/to/data.img /path/to/mnt/
cp -a /path/to/certificates/* /path/to/mnt/
umount /path/to/mnt/
```

In the preceding command, /path/to/certificates/ indicates the path where the certificate file is located, /path/to/data.img indicates the path where the certificate file image is located, and /path/to/mnt/ indicates the image mounting path.

Mount the image in the XML file of the VM.

```xml
<devices>
 <disk type='file' device='disk'>
  <driver name='qemu' type='raw' cache='none' io='native'/>
  <source file='/path/to/data.img'/>
  <target dev='sdc' bus='scsi'/>
  <boot order='2'/>
  <readonly/>
 </disk>
</devices>
```

Start the VM and import the PK certificate. The procedure is as follows (the procedure for importing the KEK certificate is the same as that for importing the DB certificate):

After the VM is started, press F2 to go to the BIOS screen.

**Figure 1** BIOS screen

![](./figures/CertEnrollP1.png)

**Figure 2** Device Manager

![](./figures/CertEnrollP2.png)

**Figure 3** Custom Secure Boot Options

![](./figures/CertEnrollP3.png)

**Figure 4** PK Options

![](./figures/CertEnrollP4.png)

**Figure 5** Enrolling PK

![](./figures/CertEnrollP5.png)

In the File Explorer window, many disk directories are displayed, including the certificate file directory mounted through the disk.

**Figure 6** File Explorer

![](./figures/CertEnrollP6.png)

Select the PK certificate to be imported in the disk directory.

**Figure 7** Disk where the certificate is stored

![](./figures/CertEnrollP7.png)

**Figure 8** Selecting Commit Changes and Exit to save the imported certificate

![](./figures/CertEnrollP8.png)

After the certificate is imported, the UEFI BIOS writes the certificate information and secure boot attributes to the NVRAM configuration file /path/to/QEMU-VARS.fd. Upon the next startup, the VM reads related configurations from the /path/to/QEMU-VARS.fd file, initializes certificate information and secure boot attributes, automatically imports the certificate, and enables secure boot. Similarly, you can use /path/to/QEMU-VARS.fd as the UEFI BIOS boot configuration template file of other VMs with the same configuration. Modify the nvram template field so that the certificate is automatically imported and the secure boot option is enabled when other VMs are started. The VM XML configuration is modified as follows:

```xml
<os>
    <type arch='aarch64' machine='virt'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader>
    <nvram template='/path/to/QEMU-VARS.fd'></nvram>
</os>
```

#### Secure Boot Observation

After the VM is correctly configured and the PK, KEK, and DB certificates are imported, the VM runs in secure boot mode. You can configure the serial port log file in the VM configuration file in XML format to check whether the VM is in the secure boot mode. The following figure shows how to configure the serial port log file.

```xml
<serial type='file'>
 <source path='/path/to/log_file' append='on'/>
</serial>
```

After the OS image is successfully loaded to the VM, if "UEFI Secure Boot is enabled" is displayed in the serial port log file, the VM is in the secure boot state.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/maximum_number_of_handles.md
---
# Maximum Number of Handles

## Function Description

System containers support limit on the number of file handles. File handles include common file handles and network sockets. When starting a container, you can specify the  **--files-limit**  parameter to limit the maximum number of handles opened in the container.

## Parameter Description

## Constraints

* If the value of  **--files-limit**  is too small, the system container may fail to run the  **exec**  command and the error "open temporary files" is reported. Therefore, you are advised to set the parameter to a large value.
* File handles include common file handles and network sockets.

## Example

To use  **--files-limit**  to limit the number of file handles opened in a container, run the following command to check whether the kernel supports files cgroup:

```sh
[root@localhost ~]# cat /proc/1/cgroup | grep files
10:files:/
```

If  **files**  is displayed, files cgroup is supported.

Start the container, specify the  **--files-limit**  parameter, and check whether the  **files.limit**  parameter is successfully written.

```sh
[root@localhost ~]# isula run  -tid --files-limit 1024 --system-container  --external-rootfs /tmp/root-fs  empty init 01e82fcf97d4937aa1d96eb8067f9f23e4707b92de152328c3fc0ecb5f64e91d 
[root@localhost ~]# isula exec -it 01e82fcf97d4 bash 
[root@localhost ~]# cat /sys/fs/cgroup/files/files.limit 
1024 

```

The preceding information indicates that the number of file handles is successfully limited in the container.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/community_tools/migration_tools/migration_tools_user_guide.md
---
# migration-tools

## 介绍

本文主要介绍服务器迁移软件（以下简称“migration-tools”）的使用方法，帮助用户顺利从原系统（CentOS 7、CentOS 8）迁移到统信服务器操作系统。
migration-tools 工具提供网页界面方式进行操作，以供使用者在图形化界面便捷的进行迁移操作。

## 部署方式

在安装 openeuler 22.03-LTS-SP4 服务器上部署服务端（server），在需要迁移的 CentOS 7/CentOS 8 服务器上部署客户端（agent）。

### 支持迁移的系统

1. 支持将 AMD64 和 ARM64 架构的 CentOS 系列系统迁移到 UOS 系统，迁移前需自行准备目标系统的全量源。

2. openeuler 迁移：目前仅支持 centos 7.4 cui 系统迁移至 openeuler 20.03-LTS-SP1。

3. 不建议对安装了 i686 架构的 rpm 包的原系统进行迁移，如果对这种原系统进行迁移会出现迁移失败的结果。

|原系统|目标系统|使用的软件源|
|---|---|---|
|centos 7.4 cui|openeuler 20.03-LTS-SP1|使用 openeuler 外网源|
|centos 7.0~7.7|UOS 1002a|UOS 1002a（全量源）|
|centos 8.0~8.2|UOS 1050a|UOS 1050a（全量源）|

## 使用方法

### 安装与配置

#### 安装 migration-tools-server 端

* 关闭防火墙。

  ```shell

  systemctl stop firewalld

  ```

* 安装 migration-tools-server。

  ```shell
  yum install migration-tools-server -y
  ```

* 修改配置文件。

  ```shell

  vim /etc/migration-tools/migration-tools.conf

  ```

  ![配置文件](./figures/migration-tools-conf.png)

* 重启 migration-tools-server 服务。

  ```shell

  systemctl restart migration-tools-server

  ```

* 分发 agent 软件包。

* 根据迁移系统的版本选择分发的软件包。

  CentOS 7 系列：

  xx.xx.xx.xx表示迁移机器IP。

  ```shell

  scp -r /usr/lib/migration-tools-server/agent-rpm/el7 root@xx.xx.xx.xx:/root

  ```

  CentOS 8 系列：

  ```shell

  scp -r /usr/lib/migration-tools-server/agent-rpm/el8 root@xx.xx.xx.xx:/root

  ```

### 迁移 openeuler 系统

> **注意:** openeuler 系统目前仅支持单独使用脚本迁移。

* 从 server 端分发迁移脚本至 agent 端。

  ```shell

  cd /usr/lib/migration-tools-server/ut-Migration-tools-0.1/centos7/
  scp openeuler/centos72openeuler.py root@10.12.23.106:/root

  ```

* 安装迁移所需依赖。

  ```shell

  yum install python3 dnf rsync yum-utils -y

  ```

* 开始迁移。

  ```shell

  python3 centos7/openeuler/centos72openeuler.py

  ```

* 迁移完成后系统会自动重启，重启完成后即迁移完成。

  ![openeuler迁移完成](./figures/openeuler-migration-complete.png)

### 迁移 UOS 系统

#### 安装 migration-tools-agent 端

在准备迁移的 centos 机器上执行以下步骤：

> **注意:** 目前 migration-tools 仅支持 centos7.4 cui 迁移至 openeuler 20.03-LTS-SP1。

* 关闭防火墙。

  ```shell

  systemctl stop firewalld

  ```

* 安装 epel-release（部分依赖包含在 epel 源中）。

  ```shell

  yum install epel-release -y

  ```

* 安装 migration-tools-agent 软件包（CentOS 7 系列需安装对应架构的软件包）。

  CentOS 7：

  ```shell

  cd /root/el7/x86_64
  yum install ./* -y

  ```

  CentSO 8：

  ```shell

  cd /root/el8/
  yum install ./* -y

  ```

* 修改配置文件。

  ```shell

  vim /etc/migration-tools/migration-tools.conf

  ```

  ![配置文件](./figures/migration-tools-conf.png)

* 重启 migration-tools-agent 服务。

  ```shell

  systemctl restart migration-tools-agent

  ```

#### UOS 系统迁移步骤

* 登录 web 端

  在 server 端和 agent 端服务均启动后，打开浏览器（建议使用：Chrome），在浏览器导航栏中输入`https://SERVER_IP:9999`即可。

  ![首页](./figures/首页.png)

* 点击“我已阅读并同意此协议”，然后点击“下一步”。
  ![许可协议](./figures/许可协议.png)

* 迁移提示页面内容如下，点击“下一步”。
  ![提示](./figures/提示.png)

* 环境检测页面会检查系统版本和系统剩余空间大小，在检测完成后点击“下一步”。

> **注意:** 如果出现检测长时间无反应，请检查 agent 防火墙是否关闭，server 与 agent 服务是否开启。
>
> 如需重新检测，在浏览器中刷新即可。

![环境检测](./figures/环境检测.png)

* 用户检测页面会检查用户名以及密码，推荐使用 root 用户，点击“下一步”开始检测，检测完成后自动进入 repo 源配置页面。

  ![用户检测](./figures/用户检测.png)

repo 源配置页面：

* 请根据要迁移的系统输入对应的 repo 源。

  `centos7:1002a，centos8:1050a`

* 确保使用的软件源为全量源，否则迁移会失败。

* 输入栏中只需输入1个软件仓库路径即可。

![repo](./figures/repo.png)

* 输入完成后点击“下一步”，等待软件源连通性检测完毕后，进入 kernel 版本选择页面，选择 4.19 内核，点击“下一步”。

  ![kernel](./figures/kernel.png)

* 迁移环境检查界面可以对比迁移前后的软件包差异，并输出检测报告，检查完成后可以导出检测报告。

  > **注意：** 检测时间大约为1个小时，请耐心等待。

  ![迁移检查](./figures/迁移检查.png)

* 检测完成后，点击“下一步”会弹出系统迁移“确认”窗口，请确保系统已做好备份，准备完成后点击确认开始系统迁移。

  ![迁移确认](./figures/迁移确认.png)

* 点击“确认”后，进入系统迁移页面。

  ![迁移开始](./figures/迁移开始.png)

* 可以点击“查看详情”，来查看迁移情况。

  ![迁移中](./figures/迁移中.png)

* 迁移完成后，页面会跳转至迁移完成页面，可在该页面导出迁移分析报告及迁移日志。

* 导出后，可在 server 端 /var/tmp/uos-migration/ 目录下找到报告和日志的压缩包，解压后即可查看。

  ![迁移完成](./figures/迁移完成.png)

* 迁移完成后，需手动重启 agent 机器，并验证是否迁移完成。

##### 验证步骤

执行以下命令，检查操作系统版本是否已迁移至目标操作系统。

```shell

uosinfo

```

如显示以下信息表示迁移成功。

1002a：

```shell

  #################################################
  Release:  UnionTech OS Server release 20 (kongli)
  Kernel :  4.19.0-91.77.97.uelc20.x86_64
  Build  :  UnionTech OS Server 20 1002c 20211228 x86_64
  #################################################

```

1050a：

```shell

  #################################################
  Release:  UnionTech OS Server release 20 (kongzi)
  Kernel :  4.19.0-91.82.88.uelc20.x86_64
  Build  :  UnionTech OS Server 20 1050a 20220214 x86_64
  #################################################
  
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/secure_container/monitoring_secure_containers.md
---
# Monitoring Secure Containers

## Description

The  **kata events**  command is used to view the status information of a specified container. The information includes but is not limited to the container memory, CPU, PID, Blkio, hugepage memory, and network information.

## Usage

```sh
kata-runtime metrics <sandbox id>
```

## Prerequisites

*sandbox id* is an long ID. The container to be queried must be in the  **running**  state. Otherwise, the following error message will be displayed: `Get "<http://shim/metrics":dial> unix /run/vc/\<id\>/shim-monitor : connect : connection refused`

When an annotation specifies a container  running within a specific sandbox, attempting to query the container using **kata-runtime metrics** will fail. Instead, you must query the corresponding sandbox for the container.

This command can be used to query the status of only one sandbox.

## Example

```shell
$ kata-runtime metrics e2270357d23f9d3dd424011e1e70aa8defb267d813c3d451db58f35aeac97a04

# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 2.656e-05
go_gc_duration_seconds{quantile="0.25"} 3.345e-05
go_gc_duration_seconds{quantile="0.5"} 3.778e-05
go_gc_duration_seconds{quantile="0.75"} 4.657e-05
go_gc_duration_seconds{quantile="1"} 0.00023001
go_gc_duration_seconds_sum 0.00898126
go_gc_duration_seconds_count 195
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 27
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.17.3"} 1
# HELP kata_hypervisor_netdev Net devices statistics.
# TYPE kata_hypervisor_netdev gauge
kata_hypervisor_netdev{interface="lo",item="recv_bytes"} 0
kata_hypervisor_netdev{interface="lo",item="recv_compressed"} 0
kata_hypervisor_netdev{interface="lo",item="recv_drop"} 0
kata_hypervisor_netdev{interface="lo",item="recv_errs"} 0
kata_hypervisor_netdev{interface="lo",item="recv_fifo"} 0
kata_hypervisor_netdev{interface="lo",item="recv_frame"} 0
kata_hypervisor_netdev{interface="lo",item="recv_multicast"} 0
kata_hypervisor_netdev{interface="lo",item="recv_packets"} 0
kata_hypervisor_netdev{interface="lo",item="sent_bytes"} 0
kata_hypervisor_netdev{interface="lo",item="sent_carrier"} 0
kata_hypervisor_netdev{interface="lo",item="sent_colls"} 0
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/system_container/writable_namespace_kernel_parameters.md
---
# namespace化内核参数可写

## 功能描述

对于运行在容器内的业务，如数据库，大数据，包括普通应用，有对部分内核参数进行设置和调整的需求，以满足最佳的业务运行性能和可靠性。内核参数要么不允许修改，要么全部允许修改（特权容器）：

在不允许用户在容器内修改时，只提供了--sysctl外部接口，而且容器内不能灵活修改参数值。

在允许用户在容器内修改时，部分内核参数是全局有效的，某个容器修改后，会影响主机上所有的程序，安全性降低。

系统容器提供--ns-change-opt参数，可以指定namespace化的内核参数在容器内动态设置，当前仅支持net、ipc。

## 参数说明

## 约束限制

* 如果容器启动同时指定了--privileged（特权容器）和--ns-change-opt，则--ns-change-opt不生效。

## 使用示例

启动容器， 指定--ns-change-opt=net：

```sh
[root@localhost ~]# isula run -tid --ns-change-opt net --system-container --external-rootfs /root/myrootfs none init
4bf44a42b4a14fdaf127616c90defa64b4b532b18efd15b62a71cbf99ebc12d2
[root@localhost ~]# isula exec -it 4b mount | grep /proc/sys
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime)
```

可以看到容器内/proc/sys/net挂载点为rw，说明net相关的namespace化的内核参数具有读写权限。

再启动一个容器，指定--ns-change-opt=ipc：

```sh
[root@localhost ~]# isula run -tid --ns-change-opt ipc --system-container --external-rootfs /root/myrootfs none init
c62e5e5686d390500dab2fa76b6c44f5f8da383a4cbbeac12cfada1b07d6c47f
[root@localhost ~]# isula exec -it c6 mount | grep /proc/sys
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/shmmax type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/shmmni type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/shmall type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/shm_rmid_forced type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/msgmax type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/msgmni type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/msgmnb type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/sem type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/fs/mqueue type proc (rw,nosuid,nodev,noexec,relatime)
```

可以看到容器内ipc相关的内核参数挂载点为rw，说明ipc相关的namespace化的内核参数具有读写权限。

---

---
url: /en/docs/22.03_LTS_SP4/cloud/nestos/nestos/overview.md
---
# NestOS User Guide

This document describes the installation, deployment, features, and usage of the NestOS cloud-based operating system. NestOS runs common container engines, such as Docker, iSula, PodMan, and CRI-O, and integrates technologies such as Ignition, rpm-ostree, OCI runtime, and SELinux. NestOS adopts the design principles of dual-system partitions, container technology, and cluster architecture. It can adapt to multiple basic running environments in cloud scenarios.In addition, NestOS optimizes Kubernetes and provides support for platforms such as OpenStack and oVirt for IaaS ecosystem construction. In terms of PaaS ecosystem construction, platforms such as OKD and Rancher are supported for easy deployment of clusters and secure running of large-scale containerized workloads.

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/nestos/nestos/overview.md
---
# NestOS用户指南

本文介绍云底座操作系统NestOS的安装部署与各个特性说明和使用方法，使用户能够快速了解并使用NestOS。NestOS搭载了docker、iSulad、podman、cri-o等常见容器引擎，将ignition配置、rpm-ostree、OCI支持、SElinux强化等技术集成在一起，采用基于双系统分区、容器技术和集群架构的设计思路，可以适配云场景下多种基础运行环境。同时NestOS针对Kubernetes进行优化，在IaaS生态构建方面，针对openStack、oVirt等平台提供支持；在PaaS生态构建方面，针对OKD、Rancher等平台提供支持，使系统具备十分便捷的集群组件能力，可以更安全的运行大规模的容器化工作负载。

---

---
url: /en/docs/22.03_LTS_SP4/tools/maintenance.md
---


---

---
url: /en/docs/22.03_LTS_SP4/server/performance/oeaware/oeaware_user_guide.md
---
# oeAware User Guide

## Overview

oeAware is a framework that provides low-load collection, sensing, and tuning upon detecting defined system behaviors on openEuler. The framework divides the tuning process into three layers: collection, sensing, and tuning. The three layers are developed as plugins and associated with each other through subscription, overcoming the limitations of traditional tuning features that run independently and are statically enabled or disabled.

## Installation

Configure the openEuler Yum repository and run the `yum` commands to install oeAware. oeAware is installed by default on openEuler 22.03 LTS SP4.

```shell
yum install oeAware-manager
```

## How to Use

Start the oeAware service and then run the `oeawarectl` command to use it.

### Service Startup

Run the `systemd` command to start the service. oeAware is started by default after the installation.

```shell
systemctl start oeaware
```

### Configuration File

The configuration file is stored in `/etc/oeAware/config.yaml`.

```yaml
log_path: /var/log/oeAware # Log storage path.
log_level: 1 # Log level. 1: DEBUG; 2: INFO; 3: WARN; 4: ERROR
enable_list: # The plugin is enabled by default.
   - name: libtest.so # Configure the plugin and enable all instances of the plugin.
   - name: libtest1.so # Configure plugin instances and enable these plugin instances.
     instances:
      - instance1
      - instance2
      ...
   ...
plugin_list: # Plugins you can download.
  - name: test # The name must be unique. If duplicated, the first entry is used.
    description: hello world
    url: https://gitee.com/openeuler/oeAware-manager/raw/master/README.md # url cannot be empty.
  ...
```

After modifying the configuration file, run the following command to restart the service:

```shell
systemctl restart oeaware
```

### Plugin Description

**Plugin definition**: Each plugin corresponds to a .so file. Plugins are classified into collection plugins, sensing plugins, and tuning plugins.

**Instance definition**: Instances are basic units of service scheduling. A plugin contains multiple instances. For example, a collection plugin includes multiple collection items, and each collection item is an instance.

### Plugin Loading

By default, the service loads the plugins from the plugin storage path.

The plugin path is `/usr/lib64/oeAware-plugin/`.

You can also manually load the plugins.

```shell
oeawarectl -l | --load <plugin name>
```

Example:

```shell
[root@localhost ~]# oeawarectl -l libthread_collect.so
Plugin loaded successfully.
```

If the operation fails, an error description is returned.

### Plugin Uninstallation

```shell
oeawarectl -r <plugin name> | --remove <plugin name>
```

Example:

```shell
[root@localhost ~]# oeawarectl -r libthread_collect.so
Plugin remove successfully.
```

If the operation fails, an error description is returned.

### Plugin Query

#### Querying the Plugin Status

```shell
oeawarectl -q # Query all loaded plugins.
oeawarectl --query <plugin name> # Query a specified plugin.
```

Example:

```shell
Show plugins and instances status.
------------------------------------------------------------
libthread_scenario.so
        thread_scenario(available, close, count: 0)
libanalysis_oeaware.so
        hugepage_analysis(available, close, count: 0)
        dynamic_smt_analysis(available, close, count: 0)
        smc_d_analysis(available, close, count: 0)
        xcall_analysis(available, close, count: 0)
        net_hirq_analysis(available, close, count: 0)
        numa_analysis(available, close, count: 0)
        docker_coordination_burst_analysis(available, close, count: 0)
        microarch_tidnocmp_analysis(available, close, count: 0)
libscenario_numa.so
        scenario_numa(available, close, count: 12)
libsystem_tune.so
        stealtask_tune(available, close, count: 0)
        dynamic_smt_tune(available, close, count: 0)
        smc_tune(available, close, count: 0)
        xcall_tune(available, close, count: 0)
        transparent_hugepage_tune(available, close, count: 0)
        seep_tune(available, close, count: 0)
        preload_tune(available, close, count: 0)
        binary_tune(available, close, count: 0)
        numa_sched_tune(available, close, count: 0)
        net_hard_irq_tune(available, close, count: 0)
        multi_net_path_tune(available, close, count: 0)
libdocker_tune.so
        docker_cpu_burst(available, close, count: 0)
        docker_burst(available, close, count: 0)
        load_based_scheduling_tune(available, close, count: 0)
libpmu.so
        pmu_counting_collector(available, close, count: 0)
        pmu_sampling_collector(available, close, count: 12)
        pmu_spe_collector(available, close, count: 12)
        pmu_uncore_collector(available, close, count: 12)
libdocker_collector.so
        docker_collector(available, close, count: 0)
libtune_numa.so
        tune_numa_mem_access(available, close, count: 12)
libub_tune.so
        unixbench_tune(available, close, count: 0)
libsystem_collector.so
        thread_collector(available, close, count: 0)
        kernel_config(available, close, count: 0)
        command_collector(available, close, count: 0)
        env_info_collector(available, close, count: 0)
        net_interface_info(available, close, count: 0)
------------------------------------------------------------
format:
[plugin]
        [instance]([dependency status], [running status], [enable cnt])
dependency status: available means satisfying dependency, otherwise unavailable.
running status: running means that instance is running, otherwise close.
enable cnt: number of instances enabled.
```

If the operation fails, an error description is returned.

#### Querying Tuning Instance Information

```shell
oeawarectl --info
```

Displays the description information and running status of the tunning instance.

#### Querying the Subscription Relationship of Running Instances

```shell
oeawarectl -Q # Query the subscription relationship diagram of all running instances.
oeawarectl --query-dep= <plugin instance> # Query the subscription relationship diagram of the running instances.
```

The `dep.png` file is generated in the current directory, showing the subscription relationship.

The subscription relationship is displayed only when the instances are running.

Example:

```sh
oeawarectl -e thread_scenario
oeawarectl -Q
```

![img](./figures/dep.png)

### Plugin Instance Enablement

#### Enabling a Plugin Instance

```shell
oeawarectl -e | --enable <plugin instance>
```

If a plugin instance is enabled, the topic instance subscribed by the plugin instance is also enabled.

If the operation fails, an error description is returned.

You are advised to enable the following plugins:

* libsystem\_tune.so: stealtask\_tune, smc\_tune, xcall\_tune, seep\_tune
* libub\_tune.so: unixbench\_tune
* libtune\_numa.so: tune\_numa\_mem\_access

Other plugins are mainly used to provide data. You can obtain plugin data through the SDK.

#### Disabling a Plugin Instance

```shell
oeawarectl -d | --disable <plugin instance>
```

If a plugin instance is disabled, the topic instance subscribed by the plugin instance is also disabled.

If the operation fails, an error description is returned.

### Plugin Download and Installation

Run the `--list` command to query the installed plugins and the RPM packages that can be downloaded.

```shell
oeawarectl --list
```

The query result is as follows:

```shell
Supported Packages: # Packages that can be downloaded
[name1] # A plugin listed in the plugin_list in config
[name2]
...
Installed Plugins: # Installed plugins
[name1]
[name2]
...
```

Run the `--install` command to download and install the RPM package.

```shell
oeawarectl -i | --install <RPM package name> # Specify a package name that can be queried using --list (that is, a package listed under Supported Packages).
```

If the operation fails, an error description is returned.

### Analysis Mode

```sh
oeawarectl analysis -h
usage: oeawarectl analysis [options]...
  options
   -t|--time <s>                    set analysis duration in seconds(default 30s), range from 1 to 100.
   -r|--realtime                    show real time report.
   -v|--verbose                     show verbose information.
   -h|--help                        show this help message.
   --l1-miss-threshold              set l1 tlbmiss threshold.
   --l2-miss-threshold              set l2 tlbmiss threshold.
   --out-path                       set the path of the analysis report.
   --dynamic-smt-threshold          set dynamic smt cpu threshold.
   --pid                            set the pid to be analyzed.
   --numa-thread-threshold          set numa sched thread creation threshold.
   --smc-change-rate                set smc connections change rate threshold.
   --smc-localnet-flow              set smc local net flow threshold.
   --host-cpu-usage-threshold       set host cpu usage threshold.
   --docker-cpu-usage-threshold     set docker cpu usage threshold.
```

\--`l1-miss-threshold` is used to set the threshold for L1 TLB miss. If the miss rate exceeds this threshold, it is considered high.

\--`l2-miss-threshold` is used to set the threshold for L2 TLB miss. If the miss rate exceeds this threshold, it is considered high.

Example:

Run the following command to generate the system analysis report:

```sh
oeawarectl analysis -t 10
```

The report consists of three parts:

* Data Analysis: analyzes the system performance data based on the system running status.
* Analysis Conclusion: provides the system analysis conclusion.
* Analysis Suggestion: provides the tuning suggestions.

### Help

Run the `--help` command for help information.

```shell
usage: oeawarectl [options]...
  options
    analysis                run analysis mode.
    -l|--load [plugin]      load plugin.
    -r|--remove [plugin]    remove plugin from system.
    -e|--enable [instance]  enable the plugin instance.
    -d|--disable [instance] disable the plugin instance.
    -q                      query all plugins information.
    --query [plugin]        query the plugin information.
    -Q                      query all instances dependencies.
    --query-dep [instance]  query the instance dependency.
    --list                  the list of supported plugins.
    --info                  the list of InfoCmd plugins.
    -i|--install [plugin]   install plugin from the list.
    --reload-conf           reload config file(now only support log level).
    --help                  show this help message.
```

## Plugin Development Description

### Basic Data Structure

```c++
typedef struct {
    char *instanceName; // Instance name
    char *topicName; // Topic name
    char *params; // Parameters
} CTopic;

typedef struct {
    CTopic topic;
    unsigned long long len; // Length of the data array
    void **data; // Stored data
} DataList;

const int OK = 0;
const int FAILED = -1;

typedef struct {
    int code; // If the operation is successful, OK is returned. If the operation fails, FAILED is returned.
    char *payload; // Additional information
} Result;

```

### Instance Base Class

```c++
namespace oeaware {
// Instance type.
const int TUNE = 0b10000;
const int SCENARIO = 0b01000;
const int RUN_ONCE = 0b00010;
class Interface {
public:
    virtual Result OpenTopic(const Topic &topic) = 0;
    virtual void CloseTopic(const Topic &topic) = 0;
    virtual void UpdateData(const DataList &dataList) = 0;
    virtual Result Enable(const std::string &param = "") = 0;
    virtual void Disable() = 0;
    virtual void Run() = 0;
protected:
    std::string name;
    std::string version;
    std::string description;
    std::vector<Topic> supportTopics;
    int priority;
    int type;
    int period;
}
}
```

Each instance is developed by inheriting from the instance base class, implementing six virtual functions, and assigning values to seven class attributes.

The instance uses a Publish-Subscribe pattern, obtaining data through a Subscribe API and publishing data through a Publish API.

### Attribute Description

| Attribute| Type| Description|
| --- | --- | --- |
| name | string | Instance name.|
| version | string | Instance version (reserved).|
| description | string | Instance description.|
| supportTopics | vector\<Topic> | Supported topics.|
| priority | int | Instance execution priority (tuning > awareness > collection).|
| type | int | Instance type, which is identified by bits. The second bit indicates a single execution instance, the third bit indicates a collection instance, the fourth bit indicates an awareness instance, and the fifth bit indicates a tuning instance.|
| period | int | Instance execution period, in milliseconds. The value is a multiple of 10.|

### API Description

| Function Name| Parameter| Return Value| Description|
| --- | --- | --- | --- |
|Result OpenTopic(const Topic \&topic) | topic: topic to be opened| | Open the specified topic.|
| void CloseTopic(const Topic \&topic) | topic: topic to be closed| |Close the specified topic.|
| void UpdateData(const DataList \&dataList) | dataList: subscribed data| | When a topic is subscribed to, this topic updates data through UpdateData every period.|
| Result Enable(const std::string \&param = "") | param: reserved for future use| | Enable this instance.|
| void Disable() | | | Disable the instance.|
| void Run() | | | Execute the run function in every period.|

### Instance Example

```C++
#include <oeaware/interface.h>
#include <oeaware/data/thread_info.h>

class Test : public oeaware::Interface {
public:
    Test() {
        name = "TestA";
        version = "1.0";
        description = "this is a test plugin";
        supportTopics;
        priority = 0;
        type = 0;
        period = 20;
    }
    oeaware::Result OpenTopic(const oeaware::Topic &topic) override {
        return oeaware::Result(OK);
    }
    void CloseTopic(const oeaware::Topic &topic) override {

    }
    void UpdateData(const DataList &dataList) override {
        for (int i = 0; i < dataList.len; ++i) {
            ThreadInfo *info = static_cast<ThreadInfo*>(dataList.data[i]);
            INFO(logger, "pid: " << info->pid << ", name: " << info->name);
        }
    }
    oeaware::Result Enable(const std::string &param = "") override {
        Subscribe(oeaware::Topic{"thread_collector", "thread_collector", ""});
        return oeaware::Result(OK);
    }
    void Disable() override {

    }
    void Run() override {
        DataList dataList;
        oeaware::SetDataListTopic(&dataList, "test", "test", "");
        dataList.len = 1;
        dataList.data = new void* [1];
        dataList.data[0] = &pubData;
        Publish(dataList);
    }
private:
    int pubData = 1;
};

extern "C" void GetInstance(std::vector<std::shared_ptr<oeaware::Interface>> &interfaces)
{
    interfaces.emplace_back(std::make_shared<Test>());
}
```

## Internal Plugins

### libpmu.so

| Instance Name| Architecture| Description| Topic|
| --- | --- | --- | --- |
| pmu\_counting\_collector | AArch64| Collect count events.|cycles, net:netif\_rx, L1-dcache-load-misses, L1-dcache-loads, L1-icache-load-misses, L1-icache-loads, branch-load-misses, branch-loads, dTLB-load-misses, dTLB-loads, iTLB-load-misses, iTLB-loads, cache-references, cache-misses, l2d\_tlb\_refill, l2d\_cache\_refill, l1d\_tlb\_refill, l1d\_cache\_refill, l1d\_tlb, l1i\_tlb, l1i\_tlb\_refill, l2d\_tlb, l2i\_tlb, l2i\_tlb\_refill, inst\_retired, instructions, sched:sched\_process\_fork, sched:sched\_process\_exit|
| pmu\_sampling\_collector | AArch64| Collect sample events.| cycles, skb:skb\_copy\_datagram\_iovec, net:napi\_gro\_receive\_entry|
| pmu\_spe\_collector | AArch64| Collect SPE events.| spe |
| pmu\_uncore\_collector | AArch64| Collect uncore events.| uncore |

#### Restrictions

The collection of SPE events depends on the hardware capability. This plugin relies on the BIOS SPE feature. Before running the plugin, you need to enable the SPE.

Run `perf list | grep arm_spe` to check whether the SPE is enabled. If it is enabled, the following information is displayed:

```sh
arm_spe_0//                                      [Kernel PMU event]
```

If not, perform the following steps to enable it:

Go to MISC Config --> SPE in the BIOS. If the SPE is set to `Disable`, switch it to `Enable`. If you cannot find this option, the BIOS version may be outdated.

Access `vim /boot/efi/EFI/openEuler/grub.cfg` of the system, locate the startup item corresponding to the kernel version, and add `kpti=off` to the end of the startup item. Example:

```sh
linux   /vmlinuz-4.19.90-2003.4.0.0036.oe1.aarch64 root=/dev/mapper/openeuler-root ro rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap video=VGA-1:640x480-32@60me rhgb quiet  smmu.bypassdev=0x1000:0x17 smmu.bypassdev=0x1000:0x15 crashkernel=1024M,high video=efifb:off video=VGA-1:640x480-32@60me kpti=off
```

Press **Esc**, enter `:wq`, and press **Enter** to save the change and exit. Run the `reboot` command to restart the server.

### libsystem\_collector.so

System information collection plugin

| Instance Name| Architecture| Description| Topic|
| --- | --- | --- | --- |
| thread\_collector | AArch64/x86| Collect system thread information.| thread\_collector |
| kernel\_config | AArch64/x86| Collect kernel parameters, including all sysctl parameters, lscpu, and meminfo.| get\_kernel\_config, get\_cmd, set\_kernel\_config|
| command\_collector | AArch64/x86| Collect sysstat data.| mpstat, iostat, vmstat, sar, pidstat|

### libdocker\_collector.so

Docker information collection plugin

| Instance Name| Architecture| Description| Topic|
| --- | --- | --- | --- |
| docker\_collector | AArch64/x86| Collect Docker information.| docker\_collector |

### libthread\_scenario.so

Thread sensing plugin

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| thread\_scenario | AArch64/x86| Obtain the thread information from the configuration file.| thread\_collector::thread\_collector |

#### Configuration File

thread\_scenario.conf

```sh
redis
fstime
fsbuffer
fsdisk
```

### libanalysis\_oeaware.so

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| analysis\_aware | AArch64| Analyze service characteristics in the current environment and provide tuning suggestions.| pmu\_spe\_collector::spe, pmu\_counting\_collector::net:netif\_rx, pmu\_sampling\_collector::cycles, pmu\_sampling\_collector::skb:skb\_copy\_datagram\_iovec, pmu\_sampling\_collector::net:napi\_gro\_receive\_entry |

### libsystem\_tune.so

System tuning plugin

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| stealtask\_tune | AArch64| In high-load scenarios, the lightweight search algorithm quickly balances loads across multiple cores, optimizing CPU efficiency.| None|
| smc\_tune | AArch64| Enable SMC acceleration to provide transparent acceleration for TCP connections.| None|
| xcall\_tune | AArch64| Reduce system call noise to improve system performance.| thread\_collector::thread\_collector |
| seep\_tune | AArch64| Enable the intelligent power mode to reduce system power consumption.| None|
| transparent\_hugepage\_tune | AArch64/x86| Enable transparent huge pages to reduce the tlb-miss rate.| None|
| preload\_tune | AArch64| Load dynamic libraries seamlessly.| None|
| binary\_tune | AArch64| Bind special binary files running inside the container to physical CPU cores. Programs that require tuning are identified by parsing specific sections of their ELF files, and CPU affinity is set according to the configuration to improve performance.| env\_info::static, env\_info::realtime, thread\_collector::thread\_collector, docker\_collector::docker\_collector |
| cluster\_tune | AArch64| Enable CPU cluster scheduling to optimize performance.| None|
| dynamic\_smt\_tune | AArch64| In low-load scenarios, physical cores are preferentially allocated to minimize inter-core interference of from hyper-threading.| None|
| numa\_sched\_tune | AArch64| In scenarios with NUMA bottlenecks, try to schedule threads on the same NUMA node throughout their lifecycle.| None|
| hardirq\_tune | AArch64| Bind NIC queue interrupts to the NUMA node where the corresponding workload runs, minimizing cross-NUMA access.| None|
| multi\_net\_path | AArch64| Perform NIC multipath tunning, ensuring each interrupt processes only the workload on its own NUMA node.| None|

#### Configuration File

##### xcall.yaml

```yaml
redis: # Thread name
    - xcall_1: 1 # xcall_1 indicates the xcall tunning method. Currently, only xcall_1 is supported, where 1 indicates the system call to be optimized.
mysql:
    - xcall_1: 1
node:
    - xcall_1: 1
```

**Restrictions**: `xcall_tune` depends on kernel features. You need to enable `FAST_SYSCALL` to compile the kernel and add the `xcall` field to the command line.

##### preload.yaml

Path: `/etc/oeAware/preload.yaml`

```yaml
- appname: ""
  so: ""
```

Run the `oeawarectl -e preload_tune` command to load the .so file to the corresponding process based on the configuration file.

### libub\_tune.so

UnixBench tuning plugin

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| unixbench\_tune | AArch64/x86| Reduce remote memory access to optimize the UnifiedBus performance.| thread\_collector::thread\_collector |

### libdocker\_tune.so

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| docker\_cpu\_burst | AArch64| CPUBurst can temporarily provide additional CPU resources for containers to alleviate performance bottlenecks caused by CPU limits when burst loads occur.| pmu\_counting\_collector::cycles, docker\_collector::docker\_collector|
| docker\_coordination\_burst\_tune | AArch64| Detect the CPU quotas of multiple containers and allocate idle CPU resources to containers with insufficient computing power. | None|
| load\_based\_scheduling\_tune | AArch64| For containers whose load exceeds the threshold, load-based scheduling is automatically enabled to distribute resources more evenly across containers.| docker\_collector::docker\_collector, env\_info\_collector::static, pmu\_sampling\_collector::cycles |
| docker\_cluster\_affinity | AArch64| In a system with a cluster architecture, containers are aware of the cluster architecture and perform scheduling accordingly. They can also monitor CPU load across multiple containers and dynamically adjust quotas between containers to handle resource imbalance.| l3c\_hit, docker\_collector::docker\_collector |

## External Plugins

You can use the following command to install an external plugin, for example, the numafast plugin.

```sh
oeawarectl -i numafast
```

### libscenario\_numa.so

| Instance Name| Architecture| Description| Subscription| Topic|
| --- | --- | --- | --- | --- |
| scenario\_numa | AArch64| Obtain the cross-NUMA memory access ratio in the current environment. It is used by instances or SDKs through subscription (and cannot be enabled independently).| pmu\_uncore\_collector::uncore | system\_score |

### libtune\_numa.so

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| tune\_numa\_mem\_access | AArch64| Periodically migrate threads and memory to reduce cross-NUMA memory access.| scenario\_numa::system\_score, pmu\_spe\_collector::spe, pmu\_counting\_collector::cycles |

#### tune\_numa\_mem\_access Usage

You can run the `--help` command to view all parameters and their functions of tune\_numa\_mem\_access.

```shell
[root@localhost ~]# oeawarectl -e tune_numa_mem_access -cmd "--help cmd"
Instance enabled failed, because show help message:
Usage: oeaware -e tune_numa_mem_access -cmd "[options][<param>]"
   or vim /etc/numafast.yaml and set options
 attr:c => support conf by cmdline, y => support conf by yaml, r => support reload yaml online
Options:
    -i, --sampling-interval <n>    attr:cy, every sampling interval n msec, range is [100, 100000], default is 100
    -t, --sampling-times <n>       attr:cy, every optimizing have n times sampling, range is [1, 1000] default is 10
    -m, --tune-mode <mode>         attr:cy, tune mode, mode can be [b, t, p], default is b
                                              b: migrate page and thread
                                              t: migrate thread only
                                              p: migrate page only
    -w, --load-way <alg>           attr:cy, load way, can be [b, c], default is b
                                              b: balance the load of threads on all numa nodes
                                              c: centralize processes to fewer numas based on load
        --smt <alg>                attr:cy, smt mode, can be [off, phy-first], default is phy-first
                                              off: disable smt
                                              phy-first: migrate threads to physical cores first, may limit load
    -h, --help <type>              attr:c, show help info, type can be [cmd, yaml], default is cmd
    -v, --version                  attr:c, show version info
    -W, --whitelist <process name list>
                                   attr:cy, only migrate process in the list, regexp list split by comma, if not set, migrate all process.
    -b, --blacklist <process name list>
                                   attr:cy, do not migrate process in the list, regexp list split by comma, priority higher than whitelist.
        --precise-load             attr:cy, load control precisely
        --mem-numa-aggregation     attr:cy, process memory aggregate by numa
        --mem-balance              attr:cy, process memory average by numa
 other options refer to /etc/numafast.yaml

[root@localhost format]# oeawarectl -e tune_numa_mem_access -cmd "--help yaml"
Instance enabled failed, because show help message:
Usage: vim /etc/numafast.yaml and set options
sampling-interval: <n> # every sampling interval n msec, range is [100, 100000], default is 100
sampling-times: <n> # every optimizing have n times sampling, range is [1, 1000] default is 10
tune-mode: <mode> # tune mode, mode can be [b, t, p], default is b
    # b: migrate page and thread
    # t: migrate thread only
    # p: migrate page only
load-way: <alg> # load way, can be [b, c], default is b
    # b: balance the load of threads on all numa nodes
    # c: centralize processes to fewer numas based on load
smt: <alg> # smt mode, can be [off, phy-first, load-first], default is phy-first
    # off: disable smt
    # phy-first: migrate threads to physical cores first, may limit load
    # load-first: migrate threads to physical cores based on load, limit load
whitelist: [] # only migrate process in the list, regexp list split by comma, if not set, migrate all process.
group: # process affinity group
    # - [process1, process2, ...]
min-numa-score: <n> # min numa score, range is [0 ,1000], default is 955
max-numa-score: <n> # max numa score, range is [0, 1000], default is 975
min-rx-ops-per-ms: <n> # min rx ops per ms, default is 10000
numa-ratio: [] # process initial load distribution for each node
page-reserve: <n> # page reserve, range is [0, 4294967295], default is 100000
precise-load: <true|false> # load control precisely
mem-numa-aggregation: <true|false> # process memory aggregate by numa
process: # process config
    # - name: process1 # process name, /proc/pid/comm
    #   params-regex: "" # process params regex, /proc/pid/cmdline
    #   algorithm: "" # process algorithm, support [MigrateThreadsToOneNode, BalanceProcNum]
    #   migrate-all-memory: "" # migrate all memory, support [true, false]
    #   default-mig-mem-node: "" # default migrate memory node, support [0, numa_node_num - 1]
    #   net-affinity: "" # process net affinity, set net interface name

```

## SDK Instructions

```C
typedef int(*Callback)(const DataList *);
int OeInit(); // Initialize resources and establish a connection with the server.
int OeSubscribe(const CTopic *topic, Callback callback); // Subscribe to a topic and execute the callback asynchronously.
int OeUnsubscribe(const CTopic *topic); // Unsubscribe from a topic.
int OePublish(const DataList *dataList); // Publish data to the server.
void OeClose(); // Release resources.
```

**Example**

```C
#include "oe_client.h"
#include "command_data.h"
int f(const DataList *dataList)
{
    int i = 0;
    for (; i < dataList->len; i++) {
        CommandData *data = (CommandData*)dataList->data[i];
        for (int j = 0; j < data->attrLen; ++j) {
            printf("%s ", data->itemAttr[j]);
        }
        printf("\n");
    }
    return 0;
}
int main() {
    OeInit();
    CTopic topic = {
        "command_collector",
        "sar",
        "-q 1",
    };
    if (OeSubscribe(&topic, f) < 0) {
        printf("failed\n");
    } else {
        printf("success\n");
    }
    sleep(10);
    OeClose();
}
```

## Constraints

### Function Constraints

By default, oeAware integrates the Arm microarchitecture profiling module libkperf. This module can only be accessed by one process at a time. If other processes or tools (such as perf) attempt to use it simultaneously, conflicts may occur.

### Operation Constraints

oeAware only allows operations by users in the root group, while the SDK allows operations by users in both the root and oeaware groups.

## Precautions

oeAware performs strict validation on the configuration files, plugin user groups, and permissions. Do not modify the permissions or user group settings of any oeAware-related file.

Permission description:

* Plugin file: 440

* Client executable file: 750

* Server executable file: 750

* Service configuration file: 640

---

---
url: >-
  /en/docs/22.03_LTS_SP4/tools/community_tools/performance/oeaware/oeaware_user_guide.md
---
# oeAware User Guide

## Overview

oeAware is a framework that provides low-load collection, sensing, and tuning upon detecting defined system behaviors on openEuler. The framework divides the tuning process into three layers: collection, sensing, and tuning. The three layers are developed as plugins and associated with each other through subscription, overcoming the limitations of traditional tuning features that run independently and are statically enabled or disabled.

## Installation

Configure the openEuler Yum repository and run the `yum` commands to install oeAware. oeAware is installed by default on openEuler 22.03 LTS SP4.

```shell
yum install oeAware-manager
```

## How to Use

Start the oeAware service and then run the `oeawarectl` command to use it.

### Service Startup

Run the `systemd` command to start the service. oeAware is started by default after the installation.

```shell
systemctl start oeaware
```

### Configuration File

The configuration file is stored in `/etc/oeAware/config.yaml`.

```yaml
log_path: /var/log/oeAware # Log storage path.
log_level: 1 # Log level. 1: DEBUG; 2: INFO; 3: WARN; 4: ERROR
enable_list: # The plugin is enabled by default.
   - name: libtest.so # Configure the plugin and enable all instances of the plugin.
   - name: libtest1.so # Configure plugin instances and enable these plugin instances.
     instances:
      - instance1
      - instance2
      ...
   ...
plugin_list: # Plugins you can download.
  - name: test # The name must be unique. If duplicated, the first entry is used.
    description: hello world
    url: https://gitee.com/openeuler/oeAware-manager/raw/master/README.md # url cannot be empty.
  ...
```

After modifying the configuration file, run the following command to restart the service:

```shell
systemctl restart oeaware
```

### Plugin Description

**Plugin definition**: Each plugin corresponds to a .so file. Plugins are classified into collection plugins, sensing plugins, and tuning plugins.

**Instance definition**: Instances are basic units of service scheduling. A plugin contains multiple instances. For example, a collection plugin includes multiple collection items, and each collection item is an instance.

### Plugin Loading

By default, the service loads the plugins from the plugin storage path.

The plugin path is `/usr/lib64/oeAware-plugin/`.

You can also manually load the plugins.

```shell
oeawarectl -l | --load <plugin name>
```

Example:

```shell
[root@localhost ~]# oeawarectl -l libthread_collect.so
Plugin loaded successfully.
```

If the operation fails, an error description is returned.

### Plugin Uninstallation

```shell
oeawarectl -r <plugin name> | --remove <plugin name>
```

Example:

```shell
[root@localhost ~]# oeawarectl -r libthread_collect.so
Plugin remove successfully.
```

If the operation fails, an error description is returned.

### Plugin Query

#### Querying the Plugin Status

```shell
oeawarectl -q # Query all loaded plugins.
oeawarectl --query <plugin name> # Query a specified plugin.
```

Example:

```shell
Show plugins and instances status.
------------------------------------------------------------
libthread_scenario.so
        thread_scenario(available, close, count: 0)
libanalysis_oeaware.so
        hugepage_analysis(available, close, count: 0)
        dynamic_smt_analysis(available, close, count: 0)
        smc_d_analysis(available, close, count: 0)
        xcall_analysis(available, close, count: 0)
        net_hirq_analysis(available, close, count: 0)
        numa_analysis(available, close, count: 0)
        docker_coordination_burst_analysis(available, close, count: 0)
        microarch_tidnocmp_analysis(available, close, count: 0)
libscenario_numa.so
        scenario_numa(available, close, count: 12)
libsystem_tune.so
        stealtask_tune(available, close, count: 0)
        dynamic_smt_tune(available, close, count: 0)
        smc_tune(available, close, count: 0)
        xcall_tune(available, close, count: 0)
        transparent_hugepage_tune(available, close, count: 0)
        seep_tune(available, close, count: 0)
        preload_tune(available, close, count: 0)
        binary_tune(available, close, count: 0)
        numa_sched_tune(available, close, count: 0)
        net_hard_irq_tune(available, close, count: 0)
        multi_net_path_tune(available, close, count: 0)
libdocker_tune.so
        docker_cpu_burst(available, close, count: 0)
        docker_burst(available, close, count: 0)
        load_based_scheduling_tune(available, close, count: 0)
libpmu.so
        pmu_counting_collector(available, close, count: 0)
        pmu_sampling_collector(available, close, count: 12)
        pmu_spe_collector(available, close, count: 12)
        pmu_uncore_collector(available, close, count: 12)
libdocker_collector.so
        docker_collector(available, close, count: 0)
libtune_numa.so
        tune_numa_mem_access(available, close, count: 12)
libub_tune.so
        unixbench_tune(available, close, count: 0)
libsystem_collector.so
        thread_collector(available, close, count: 0)
        kernel_config(available, close, count: 0)
        command_collector(available, close, count: 0)
        env_info_collector(available, close, count: 0)
        net_interface_info(available, close, count: 0)
------------------------------------------------------------
format:
[plugin]
        [instance]([dependency status], [running status], [enable cnt])
dependency status: available means satisfying dependency, otherwise unavailable.
running status: running means that instance is running, otherwise close.
enable cnt: number of instances enabled.
```

If the operation fails, an error description is returned.

#### Querying Tuning Instance Information

```shell
oeawarectl --info
```

Displays the description information and running status of the tunning instance.

#### Querying the Subscription Relationship of Running Instances

```shell
oeawarectl -Q # Query the subscription relationship diagram of all running instances.
oeawarectl --query-dep= <plugin instance> # Query the subscription relationship diagram of the running instances.
```

The `dep.png` file is generated in the current directory, showing the subscription relationship.

The subscription relationship is displayed only when the instances are running.

Example:

```sh
oeawarectl -e thread_scenario
oeawarectl -Q
```

![img](./figures/dep.png)

### Plugin Instance Enablement

#### Enabling a Plugin Instance

```shell
oeawarectl -e | --enable <plugin instance>
```

If a plugin instance is enabled, the topic instance subscribed by the plugin instance is also enabled.

If the operation fails, an error description is returned.

You are advised to enable the following plugins:

* libsystem\_tune.so: stealtask\_tune, smc\_tune, xcall\_tune, seep\_tune
* libub\_tune.so: unixbench\_tune
* libtune\_numa.so: tune\_numa\_mem\_access

Other plugins are mainly used to provide data. You can obtain plugin data through the SDK.

#### Disabling a Plugin Instance

```shell
oeawarectl -d | --disable <plugin instance>
```

If a plugin instance is disabled, the topic instance subscribed by the plugin instance is also disabled.

If the operation fails, an error description is returned.

### Plugin Download and Installation

Run the `--list` command to query the installed plugins and the RPM packages that can be downloaded.

```shell
oeawarectl --list
```

The query result is as follows:

```shell
Supported Packages: # Packages that can be downloaded
[name1] # A plugin listed in the plugin_list in config
[name2]
...
Installed Plugins: # Installed plugins
[name1]
[name2]
...
```

Run the `--install` command to download and install the RPM package.

```shell
oeawarectl -i | --install <RPM package name> # Specify a package name that can be queried using --list (that is, a package listed under Supported Packages).
```

If the operation fails, an error description is returned.

### Analysis Mode

```sh
oeawarectl analysis -h
usage: oeawarectl analysis [options]...
  options
   -t|--time <s>                    set analysis duration in seconds(default 30s), range from 1 to 100.
   -r|--realtime                    show real time report.
   -v|--verbose                     show verbose information.
   -h|--help                        show this help message.
   --l1-miss-threshold              set l1 tlbmiss threshold.
   --l2-miss-threshold              set l2 tlbmiss threshold.
   --out-path                       set the path of the analysis report.
   --dynamic-smt-threshold          set dynamic smt cpu threshold.
   --pid                            set the pid to be analyzed.
   --numa-thread-threshold          set numa sched thread creation threshold.
   --smc-change-rate                set smc connections change rate threshold.
   --smc-localnet-flow              set smc local net flow threshold.
   --host-cpu-usage-threshold       set host cpu usage threshold.
   --docker-cpu-usage-threshold     set docker cpu usage threshold.
```

\--`l1-miss-threshold` is used to set the threshold for L1 TLB miss. If the miss rate exceeds this threshold, it is considered high.

\--`l2-miss-threshold` is used to set the threshold for L2 TLB miss. If the miss rate exceeds this threshold, it is considered high.

Example:

Run the following command to generate the system analysis report:

```sh
oeawarectl analysis -t 10
```

The report consists of three parts:

* Data Analysis: analyzes the system performance data based on the system running status.
* Analysis Conclusion: provides the system analysis conclusion.
* Analysis Suggestion: provides the tuning suggestions.

### Help

Run the `--help` command for help information.

```shell
usage: oeawarectl [options]...
  options
    analysis                run analysis mode.
    -l|--load [plugin]      load plugin.
    -r|--remove [plugin]    remove plugin from system.
    -e|--enable [instance]  enable the plugin instance.
    -d|--disable [instance] disable the plugin instance.
    -q                      query all plugins information.
    --query [plugin]        query the plugin information.
    -Q                      query all instances dependencies.
    --query-dep [instance]  query the instance dependency.
    --list                  the list of supported plugins.
    --info                  the list of InfoCmd plugins.
    -i|--install [plugin]   install plugin from the list.
    --reload-conf           reload config file(now only support log level).
    --help                  show this help message.
```

## Plugin Development Description

### Basic Data Structure

```c++
typedef struct {
    char *instanceName; // Instance name
    char *topicName; // Topic name
    char *params; // Parameters
} CTopic;

typedef struct {
    CTopic topic;
    unsigned long long len; // Length of the data array
    void **data; // Stored data
} DataList;

const int OK = 0;
const int FAILED = -1;

typedef struct {
    int code; // If the operation is successful, OK is returned. If the operation fails, FAILED is returned.
    char *payload; // Additional information
} Result;

```

### Instance Base Class

```c++
namespace oeaware {
// Instance type.
const int TUNE = 0b10000;
const int SCENARIO = 0b01000;
const int RUN_ONCE = 0b00010;
class Interface {
public:
    virtual Result OpenTopic(const Topic &topic) = 0;
    virtual void CloseTopic(const Topic &topic) = 0;
    virtual void UpdateData(const DataList &dataList) = 0;
    virtual Result Enable(const std::string &param = "") = 0;
    virtual void Disable() = 0;
    virtual void Run() = 0;
protected:
    std::string name;
    std::string version;
    std::string description;
    std::vector<Topic> supportTopics;
    int priority;
    int type;
    int period;
}
}
```

Each instance is developed by inheriting from the instance base class, implementing six virtual functions, and assigning values to seven class attributes.

The instance uses a Publish-Subscribe pattern, obtaining data through a Subscribe API and publishing data through a Publish API.

### Attribute Description

| Attribute| Type| Description|
| --- | --- | --- |
| name | string | Instance name.|
| version | string | Instance version (reserved).|
| description | string | Instance description.|
| supportTopics | vector\<Topic> | Supported topics.|
| priority | int | Instance execution priority (tuning > awareness > collection).|
| type | int | Instance type, which is identified by bits. The second bit indicates a single execution instance, the third bit indicates a collection instance, the fourth bit indicates an awareness instance, and the fifth bit indicates a tuning instance.|
| period | int | Instance execution period, in milliseconds. The value is a multiple of 10.|

### API Description

| Function Name| Parameter| Return Value| Description|
| --- | --- | --- | --- |
|Result OpenTopic(const Topic \&topic) | topic: topic to be opened| | Open the specified topic.|
| void CloseTopic(const Topic \&topic) | topic: topic to be closed| |Close the specified topic.|
| void UpdateData(const DataList \&dataList) | dataList: subscribed data| | When a topic is subscribed to, this topic updates data through UpdateData every period.|
| Result Enable(const std::string \&param = "") | param: reserved for future use| | Enable this instance.|
| void Disable() | | | Disable the instance.|
| void Run() | | | Execute the run function in every period.|

### Instance Example

```C++
#include <oeaware/interface.h>
#include <oeaware/data/thread_info.h>

class Test : public oeaware::Interface {
public:
    Test() {
        name = "TestA";
        version = "1.0";
        description = "this is a test plugin";
        supportTopics;
        priority = 0;
        type = 0;
        period = 20;
    }
    oeaware::Result OpenTopic(const oeaware::Topic &topic) override {
        return oeaware::Result(OK);
    }
    void CloseTopic(const oeaware::Topic &topic) override {

    }
    void UpdateData(const DataList &dataList) override {
        for (int i = 0; i < dataList.len; ++i) {
            ThreadInfo *info = static_cast<ThreadInfo*>(dataList.data[i]);
            INFO(logger, "pid: " << info->pid << ", name: " << info->name);
        }
    }
    oeaware::Result Enable(const std::string &param = "") override {
        Subscribe(oeaware::Topic{"thread_collector", "thread_collector", ""});
        return oeaware::Result(OK);
    }
    void Disable() override {

    }
    void Run() override {
        DataList dataList;
        oeaware::SetDataListTopic(&dataList, "test", "test", "");
        dataList.len = 1;
        dataList.data = new void* [1];
        dataList.data[0] = &pubData;
        Publish(dataList);
    }
private:
    int pubData = 1;
};

extern "C" void GetInstance(std::vector<std::shared_ptr<oeaware::Interface>> &interfaces)
{
    interfaces.emplace_back(std::make_shared<Test>());
}
```

## Internal Plugins

### libpmu.so

| Instance Name| Architecture| Description| Topic|
| --- | --- | --- | --- |
| pmu\_counting\_collector | AArch64| Collect count events.|cycles, net:netif\_rx, L1-dcache-load-misses, L1-dcache-loads, L1-icache-load-misses, L1-icache-loads, branch-load-misses, branch-loads, dTLB-load-misses, dTLB-loads, iTLB-load-misses, iTLB-loads, cache-references, cache-misses, l2d\_tlb\_refill, l2d\_cache\_refill, l1d\_tlb\_refill, l1d\_cache\_refill, l1d\_tlb, l1i\_tlb, l1i\_tlb\_refill, l2d\_tlb, l2i\_tlb, l2i\_tlb\_refill, inst\_retired, instructions, sched:sched\_process\_fork, sched:sched\_process\_exit|
| pmu\_sampling\_collector | AArch64| Collect sample events.| cycles, skb:skb\_copy\_datagram\_iovec, net:napi\_gro\_receive\_entry|
| pmu\_spe\_collector | AArch64| Collect SPE events.| spe |
| pmu\_uncore\_collector | AArch64| Collect uncore events.| uncore |

#### Restrictions

The collection of SPE events depends on the hardware capability. This plugin relies on the BIOS SPE feature. Before running the plugin, you need to enable the SPE.

Run `perf list | grep arm_spe` to check whether the SPE is enabled. If it is enabled, the following information is displayed:

```sh
arm_spe_0//                                      [Kernel PMU event]
```

If not, perform the following steps to enable it:

Go to MISC Config --> SPE in the BIOS. If the SPE is set to `Disable`, switch it to `Enable`. If you cannot find this option, the BIOS version may be outdated.

Access `vim /boot/efi/EFI/openEuler/grub.cfg` of the system, locate the startup item corresponding to the kernel version, and add `kpti=off` to the end of the startup item. Example:

```sh
linux   /vmlinuz-4.19.90-2003.4.0.0036.oe1.aarch64 root=/dev/mapper/openeuler-root ro rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap video=VGA-1:640x480-32@60me rhgb quiet  smmu.bypassdev=0x1000:0x17 smmu.bypassdev=0x1000:0x15 crashkernel=1024M,high video=efifb:off video=VGA-1:640x480-32@60me kpti=off
```

Press **Esc**, enter `:wq`, and press **Enter** to save the change and exit. Run the `reboot` command to restart the server.

### libsystem\_collector.so

System information collection plugin

| Instance Name| Architecture| Description| Topic|
| --- | --- | --- | --- |
| thread\_collector | AArch64/x86| Collect system thread information.| thread\_collector |
| kernel\_config | AArch64/x86| Collect kernel parameters, including all sysctl parameters, lscpu, and meminfo.| get\_kernel\_config, get\_cmd, set\_kernel\_config|
| command\_collector | AArch64/x86| Collect sysstat data.| mpstat, iostat, vmstat, sar, pidstat|

### libdocker\_collector.so

Docker information collection plugin

| Instance Name| Architecture| Description| Topic|
| --- | --- | --- | --- |
| docker\_collector | AArch64/x86| Collect Docker information.| docker\_collector |

### libthread\_scenario.so

Thread sensing plugin

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| thread\_scenario | AArch64/x86| Obtain the thread information from the configuration file.| thread\_collector::thread\_collector |

#### Configuration File

thread\_scenario.conf

```sh
redis
fstime
fsbuffer
fsdisk
```

### libanalysis\_oeaware.so

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| analysis\_aware | AArch64| Analyze service characteristics in the current environment and provide tuning suggestions.| pmu\_spe\_collector::spe, pmu\_counting\_collector::net:netif\_rx, pmu\_sampling\_collector::cycles, pmu\_sampling\_collector::skb:skb\_copy\_datagram\_iovec, pmu\_sampling\_collector::net:napi\_gro\_receive\_entry |

### libsystem\_tune.so

System tuning plugin

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| stealtask\_tune | AArch64| In high-load scenarios, the lightweight search algorithm quickly balances loads across multiple cores, optimizing CPU efficiency.| None|
| smc\_tune | AArch64| Enable SMC acceleration to provide transparent acceleration for TCP connections.| None|
| xcall\_tune | AArch64| Reduce system call noise to improve system performance.| thread\_collector::thread\_collector |
| seep\_tune | AArch64| Enable the intelligent power mode to reduce system power consumption.| None|
| transparent\_hugepage\_tune | AArch64/x86| Enable transparent huge pages to reduce the tlb-miss rate.| None|
| preload\_tune | AArch64| Load dynamic libraries seamlessly.| None|
| binary\_tune | AArch64| Bind special binary files running inside the container to physical CPU cores. Programs that require tuning are identified by parsing specific sections of their ELF files, and CPU affinity is set according to the configuration to improve performance.| env\_info::static, env\_info::realtime, thread\_collector::thread\_collector, docker\_collector::docker\_collector |
| cluster\_tune | AArch64| Enable CPU cluster scheduling to optimize performance.| None|
| dynamic\_smt\_tune | AArch64| In low-load scenarios, physical cores are preferentially allocated to minimize inter-core interference of from hyper-threading.| None|
| numa\_sched\_tune | AArch64| In scenarios with NUMA bottlenecks, try to schedule threads on the same NUMA node throughout their lifecycle.| None|
| hardirq\_tune | AArch64| Bind NIC queue interrupts to the NUMA node where the corresponding workload runs, minimizing cross-NUMA access.| None|
| multi\_net\_path | AArch64| Perform NIC multipath tunning, ensuring each interrupt processes only the workload on its own NUMA node.| None|

#### Configuration File

##### xcall.yaml

```yaml
redis: # Thread name
    - xcall_1: 1 # xcall_1 indicates the xcall tunning method. Currently, only xcall_1 is supported, where 1 indicates the system call to be optimized.
mysql:
    - xcall_1: 1
node:
    - xcall_1: 1
```

**Restrictions**: `xcall_tune` depends on kernel features. You need to enable `FAST_SYSCALL` to compile the kernel and add the `xcall` field to the command line.

##### preload.yaml

Path: `/etc/oeAware/preload.yaml`

```yaml
- appname: ""
  so: ""
```

Run the `oeawarectl -e preload_tune` command to load the .so file to the corresponding process based on the configuration file.

### libub\_tune.so

UnixBench tuning plugin

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| unixbench\_tune | AArch64/x86| Reduce remote memory access to optimize the UnifiedBus performance.| thread\_collector::thread\_collector |

### libdocker\_tune.so

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| docker\_cpu\_burst | AArch64| CPUBurst can temporarily provide additional CPU resources for containers to alleviate performance bottlenecks caused by CPU limits when burst loads occur.| pmu\_counting\_collector::cycles, docker\_collector::docker\_collector|
| docker\_coordination\_burst\_tune | AArch64| Detect the CPU quotas of multiple containers and allocate idle CPU resources to containers with insufficient computing power. | None|
| load\_based\_scheduling\_tune | AArch64| For containers whose load exceeds the threshold, load-based scheduling is automatically enabled to distribute resources more evenly across containers.| docker\_collector::docker\_collector, env\_info\_collector::static, pmu\_sampling\_collector::cycles |
| docker\_cluster\_affinity | AArch64| In a system with a cluster architecture, containers are aware of the cluster architecture and perform scheduling accordingly. They can also monitor CPU load across multiple containers and dynamically adjust quotas between containers to handle resource imbalance.| l3c\_hit, docker\_collector::docker\_collector |

## External Plugins

You can use the following command to install an external plugin, for example, the numafast plugin.

```sh
oeawarectl -i numafast
```

### libscenario\_numa.so

| Instance Name| Architecture| Description| Subscription| Topic|
| --- | --- | --- | --- | --- |
| scenario\_numa | AArch64| Obtain the cross-NUMA memory access ratio in the current environment. It is used by instances or SDKs through subscription (and cannot be enabled independently).| pmu\_uncore\_collector::uncore | system\_score |

### libtune\_numa.so

| Instance Name| Architecture| Description| Subscription|
| --- | --- | --- | --- |
| tune\_numa\_mem\_access | AArch64| Periodically migrate threads and memory to reduce cross-NUMA memory access.| scenario\_numa::system\_score, pmu\_spe\_collector::spe, pmu\_counting\_collector::cycles |

#### tune\_numa\_mem\_access Usage

You can run the `--help` command to view all parameters and their functions of tune\_numa\_mem\_access.

```shell
[root@localhost ~]# oeawarectl -e tune_numa_mem_access -cmd "--help cmd"
Instance enabled failed, because show help message:
Usage: oeaware -e tune_numa_mem_access -cmd "[options][<param>]"
   or vim /etc/numafast.yaml and set options
 attr:c => support conf by cmdline, y => support conf by yaml, r => support reload yaml online
Options:
    -i, --sampling-interval <n>    attr:cy, every sampling interval n msec, range is [100, 100000], default is 100
    -t, --sampling-times <n>       attr:cy, every optimizing have n times sampling, range is [1, 1000] default is 10
    -m, --tune-mode <mode>         attr:cy, tune mode, mode can be [b, t, p], default is b
                                              b: migrate page and thread
                                              t: migrate thread only
                                              p: migrate page only
    -w, --load-way <alg>           attr:cy, load way, can be [b, c], default is b
                                              b: balance the load of threads on all numa nodes
                                              c: centralize processes to fewer numas based on load
        --smt <alg>                attr:cy, smt mode, can be [off, phy-first], default is phy-first
                                              off: disable smt
                                              phy-first: migrate threads to physical cores first, may limit load
    -h, --help <type>              attr:c, show help info, type can be [cmd, yaml], default is cmd
    -v, --version                  attr:c, show version info
    -W, --whitelist <process name list>
                                   attr:cy, only migrate process in the list, regexp list split by comma, if not set, migrate all process.
    -b, --blacklist <process name list>
                                   attr:cy, do not migrate process in the list, regexp list split by comma, priority higher than whitelist.
        --precise-load             attr:cy, load control precisely
        --mem-numa-aggregation     attr:cy, process memory aggregate by numa
        --mem-balance              attr:cy, process memory average by numa
 other options refer to /etc/numafast.yaml

[root@localhost format]# oeawarectl -e tune_numa_mem_access -cmd "--help yaml"
Instance enabled failed, because show help message:
Usage: vim /etc/numafast.yaml and set options
sampling-interval: <n> # every sampling interval n msec, range is [100, 100000], default is 100
sampling-times: <n> # every optimizing have n times sampling, range is [1, 1000] default is 10
tune-mode: <mode> # tune mode, mode can be [b, t, p], default is b
    # b: migrate page and thread
    # t: migrate thread only
    # p: migrate page only
load-way: <alg> # load way, can be [b, c], default is b
    # b: balance the load of threads on all numa nodes
    # c: centralize processes to fewer numas based on load
smt: <alg> # smt mode, can be [off, phy-first, load-first], default is phy-first
    # off: disable smt
    # phy-first: migrate threads to physical cores first, may limit load
    # load-first: migrate threads to physical cores based on load, limit load
whitelist: [] # only migrate process in the list, regexp list split by comma, if not set, migrate all process.
group: # process affinity group
    # - [process1, process2, ...]
min-numa-score: <n> # min numa score, range is [0 ,1000], default is 955
max-numa-score: <n> # max numa score, range is [0, 1000], default is 975
min-rx-ops-per-ms: <n> # min rx ops per ms, default is 10000
numa-ratio: [] # process initial load distribution for each node
page-reserve: <n> # page reserve, range is [0, 4294967295], default is 100000
precise-load: <true|false> # load control precisely
mem-numa-aggregation: <true|false> # process memory aggregate by numa
process: # process config
    # - name: process1 # process name, /proc/pid/comm
    #   params-regex: "" # process params regex, /proc/pid/cmdline
    #   algorithm: "" # process algorithm, support [MigrateThreadsToOneNode, BalanceProcNum]
    #   migrate-all-memory: "" # migrate all memory, support [true, false]
    #   default-mig-mem-node: "" # default migrate memory node, support [0, numa_node_num - 1]
    #   net-affinity: "" # process net affinity, set net interface name

```

## SDK Instructions

```C
typedef int(*Callback)(const DataList *);
int OeInit(); // Initialize resources and establish a connection with the server.
int OeSubscribe(const CTopic *topic, Callback callback); // Subscribe to a topic and execute the callback asynchronously.
int OeUnsubscribe(const CTopic *topic); // Unsubscribe from a topic.
int OePublish(const DataList *dataList); // Publish data to the server.
void OeClose(); // Release resources.
```

**Example**

```C
#include "oe_client.h"
#include "command_data.h"
int f(const DataList *dataList)
{
    int i = 0;
    for (; i < dataList->len; i++) {
        CommandData *data = (CommandData*)dataList->data[i];
        for (int j = 0; j < data->attrLen; ++j) {
            printf("%s ", data->itemAttr[j]);
        }
        printf("\n");
    }
    return 0;
}
int main() {
    OeInit();
    CTopic topic = {
        "command_collector",
        "sar",
        "-q 1",
    };
    if (OeSubscribe(&topic, f) < 0) {
        printf("failed\n");
    } else {
        printf("success\n");
    }
    sleep(10);
    OeClose();
}
```

## Constraints

### Function Constraints

By default, oeAware integrates the Arm microarchitecture profiling module libkperf. This module can only be accessed by one process at a time. If other processes or tools (such as perf) attempt to use it simultaneously, conflicts may occur.

### Operation Constraints

oeAware only allows operations by users in the root group, while the SDK allows operations by users in both the root and oeaware groups.

## Precautions

oeAware performs strict validation on the configuration files, plugin user groups, and permissions. Do not modify the permissions or user group settings of any oeAware-related file.

Permission description:

* Plugin file: 440

* Client executable file: 750

* Server executable file: 750

* Service configuration file: 640

---

---
url: /zh/docs/22.03_LTS_SP4/server/performance/oeaware/oeaware_user_guide.md
---
# oeAware用户指南

## 简介

oeAware是在openEuler上实现低负载采集感知调优的框架，目标是动态感知系统行为后智能使能系统的调优特性。传统调优特性都以独立运行且静态打开关闭为主，oeAware将调优拆分采集、感知和调优三层，每层通过订阅方式关联，各层采用插件式开发尽可能复用。

## 安装

配置openEuler的yum源，使用yum命令安装。在openEuler-22.03-LTS-SP4版本中会默认安装。

```shell
yum install oeAware-manager
```

## 使用方法

首先启动oeaware服务，然后通过`oeawarectl`命令进行使用。

### 服务启动

通过systemd服务启动。安装完成后会默认启动。

```shell
systemctl start oeaware
```

### 配置文件

配置文件路径：`/etc/oeAware/config.yaml`。

```yaml
log_path: /var/log/oeAware #日志存储路径
log_level: 1 #日志等级 1：DEBUG 2：INFO 3：WARN 4：ERROR
enable_list: #默认使能插件
   - name: libtest.so #只配置插件，使能本插件的所有实例
   - name: libtest1.so #配置插件实例，使能配置的插件实例
     instances:
      - instance1
      - instance2
      ...
   ...
plugin_list: #可支持下载的包
  - name: test #名称需要唯一，如果重复取第一个配置
    description: hello world
    url: https://gitee.com/openeuler/oeAware-manager/raw/master/README.md #url非空
  ...
```

修改配置文件后，通过以下命令重启服务。

```shell
systemctl restart oeaware
```

### 插件说明

**插件定义**：一个插件对应一个`.so`文件，插件分为采集插件、感知插件和调优插件。

**实例定义**：服务中的调度单位是实例，一个插件中包括多个实例。例如，一个采集插件包括多个采集项，每个采集项是一个实例。

### 插件加载

服务会默认加载插件存储路径下的插件。

插件路径：`/usr/lib64/oeAware-plugin/`。

另外也可以通过手动加载的方式加载插件。

```shell
oeawarectl -l | --load <插件名>
```

示例：

```shell
[root@localhost ~]# oeawarectl -l libthread_collect.so
Plugin loaded successfully.
```

失败返回错误说明。

### 插件卸载

```shell
oeawarectl -r <插件名> | --remove <插件名>
```

示例：

```shell
[root@localhost ~]# oeawarectl -r libthread_collect.so
Plugin remove successfully.
```

失败返回错误说明。

### 插件查询

#### 查询插件状态信息

```shell
oeawarectl -q  #查询系统中已经加载的所有插件
oeawarectl --query <插件名> #查询指定插件
```

示例：

```shell
Show plugins and instances status.
------------------------------------------------------------
libthread_scenario.so
        thread_scenario(available, close, count: 0)
libanalysis_oeaware.so
        hugepage_analysis(available, close, count: 0)
        dynamic_smt_analysis(available, close, count: 0)
        smc_d_analysis(available, close, count: 0)
        xcall_analysis(available, close, count: 0)
        net_hirq_analysis(available, close, count: 0)
        numa_analysis(available, close, count: 0)
        docker_coordination_burst_analysis(available, close, count: 0)
        microarch_tidnocmp_analysis(available, close, count: 0)
libscenario_numa.so
        scenario_numa(available, close, count: 12)
libsystem_tune.so
        stealtask_tune(available, close, count: 0)
        dynamic_smt_tune(available, close, count: 0)
        smc_tune(available, close, count: 0)
        xcall_tune(available, close, count: 0)
        transparent_hugepage_tune(available, close, count: 0)
        seep_tune(available, close, count: 0)
        preload_tune(available, close, count: 0)
        binary_tune(available, close, count: 0)
        numa_sched_tune(available, close, count: 0)
        net_hard_irq_tune(available, close, count: 0)
        multi_net_path_tune(available, close, count: 0)
libdocker_tune.so
        docker_cpu_burst(available, close, count: 0)
        docker_burst(available, close, count: 0)
        load_based_scheduling_tune(available, close, count: 0)
libpmu.so
        pmu_counting_collector(available, close, count: 0)
        pmu_sampling_collector(available, close, count: 12)
        pmu_spe_collector(available, close, count: 12)
        pmu_uncore_collector(available, close, count: 12)
libdocker_collector.so
        docker_collector(available, close, count: 0)
libtune_numa.so
        tune_numa_mem_access(available, close, count: 12)
libub_tune.so
        unixbench_tune(available, close, count: 0)
libsystem_collector.so
        thread_collector(available, close, count: 0)
        kernel_config(available, close, count: 0)
        command_collector(available, close, count: 0)
        env_info_collector(available, close, count: 0)
        net_interface_info(available, close, count: 0)
------------------------------------------------------------
format:
[plugin]
        [instance]([dependency status], [running status], [enable cnt])
dependency status: available means satisfying dependency, otherwise unavailable.
running status: running means that instance is running, otherwise close.
enable cnt: number of instances enabled.
```

失败返回错误说明。

#### 查询调优实例信息

```shell
oeawarectl --info
```

显示调优实例描述信息及运行状态。

#### 查询运行实例订阅关系

```shell
oeawarectl -Q  #查询所有运行实例的订阅关系图
oeawarectl --query-dep= <插件实例>  #查询运行实例订阅关系图
```

在当前目录下生成dep.png，显示订阅关系。

实例未运行，不会显示订阅关系。

示例：

```sh
oeawarectl -e thread_scenario
oeawarectl -Q
```

![img](./figures/dep.png)

### 插件实例使能

#### 使能插件实例

```shell
oeawarectl -e | --enable <插件实例>
```

使能某个插件实例，会将其订阅的topic实例一起使能。

失败返回错误说明。

推荐使能插件列表：

* libsystem\_tune.so：stealtask\_tune，smc\_tune，xcall\_tune，seep\_tune。
* libub\_tune.so：unixbench\_tune。
* libtune\_numa.so：tune\_numa\_mem\_access。

其他插件主要用来提供数据，可通过sdk获取插件数据。

#### 关闭插件实例

```shell
oeawarectl -d | --disable <插件实例>
```

关闭某个插件实例，会将其订阅的topic实例一起关闭。

失败返回错误说明。

### 插件下载安装

通过`--list`命令查询支持下载的rpm包和已安装的插件。

```shell
oeawarectl --list
```

查询结果如下。

```shell
Supported Packages: #可下载的包
[name1] #config中配置的plugin_list
[name2]
...
Installed Plugins: #已安装的插件
[name1]
[name2]
...
```

通过`--install`命令下载安装rpm包。

```shell
oeawarectl -i | --install <rpm包名> #指定--list下查询得到的包名称（Supported Packages下的包）
```

失败返回错误说明。

### 分析模式

```sh
oeawarectl analysis -h
usage: oeawarectl analysis [options]...
  options
   -t|--time <s>                    set analysis duration in seconds(default 30s), range from 1 to 100.
   -r|--realtime                    show real time report.
   -v|--verbose                     show verbose information.
   -h|--help                        show this help message.
   --l1-miss-threshold              set l1 tlbmiss threshold.
   --l2-miss-threshold              set l2 tlbmiss threshold.
   --out-path                       set the path of the analysis report.
   --dynamic-smt-threshold          set dynamic smt cpu threshold.
   --pid                            set the pid to be analyzed.
   --numa-thread-threshold          set numa sched thread creation threshold.
   --smc-change-rate                set smc connections change rate threshold.
   --smc-localnet-flow              set smc local net flow threshold.
   --host-cpu-usage-threshold       set host cpu usage threshold.
   --docker-cpu-usage-threshold     set docker cpu usage threshold.
```

\--l1-miss-threshold用于设置l1-tlb—miss阈值，超过这个阈值miss率为high。

\--l2-miss-threshold用于设置l2-tlb—miss阈值，超过这个阈值miss率为high。

示例：

执行以下命令，输出系统分析报告。

```sh
oeawarectl analysis -t 10
```

报告分为三部分：

* Data Analysis：根据系统运行状态，给出系统性能数据分析。
* Analysis Conclusion：给出系统分析结论。
* Analysis Suggestion：给出具体调优方法。

### 帮助

通过`--help`查看帮助。

```shell
usage: oeawarectl [options]...
  options
    analysis                run analysis mode.
    -l|--load [plugin]      load plugin.
    -r|--remove [plugin]    remove plugin from system.
    -e|--enable [instance]  enable the plugin instance.
    -d|--disable [instance] disable the plugin instance.
    -q                      query all plugins information.
    --query [plugin]        query the plugin information.
    -Q                      query all instances dependencies.
    --query-dep [instance]  query the instance dependency.
    --list                  the list of supported plugins.
    --info                  the list of InfoCmd plugins.
    -i|--install [plugin]   install plugin from the list.
    --reload-conf           reload config file(now only support log level).
    --help                  show this help message.
```

## 插件开发说明

### 基础数据结构

```c++
typedef struct {
    char *instanceName; // 实例名称
    char *topicName; // 主题名称
    char *params; // 参数
} CTopic;

typedef struct {
    CTopic topic;
    unsigned long long len; // data数组的长度
    void **data; // 存储的数据
} DataList;

const int OK = 0;
const int FAILED = -1;

typedef struct {
    int code; // 成功返回OK，失败返回FAILED
    char *payload; // 附带信息
} Result;

```

### 实例基类

```c++
namespace oeaware {
// Instance type.
const int TUNE = 0b10000;
const int SCENARIO = 0b01000;
const int RUN_ONCE = 0b00010;
class Interface {
public:
    virtual Result OpenTopic(const Topic &topic) = 0;
    virtual void CloseTopic(const Topic &topic) = 0;
    virtual void UpdateData(const DataList &dataList) = 0;
    virtual Result Enable(const std::string &param = "") = 0;
    virtual void Disable() = 0;
    virtual void Run() = 0;
protected:
    std::string name;
    std::string version;
    std::string description;
    std::vector<Topic> supportTopics;
    int priority;
    int type;
    int period;
}
}
```

实例开发继承实例基类，实现6个虚函数，并对类的7个属性赋值。

实例采用订阅发布模式，通过Subscribe获取数据，通过Publish接口发布数据。

### 属性说明

| 属性 | 类型 | 说明 |
| --- | --- | --- |
| name | string | 实例名称 |
| version | string | 实例版本（预留） |
| description | string | 实例描述 |
| supportTopics | vector\<Topic> | 支持的topic |
| priority | int | 实例执行的优先级（调优 > 感知 > 采集）|
| type | int | 实例类型，通过比特位标识，第二位表示单次执行实例，第三位表示采集实例，第四位表示感知实例，第5位表示调优实例|
| period | int | 实例执行周期，单位ms，period为10的倍数 |

### 接口说明

| 函数名 | 参数 | 返回值 | 说明 |
| --- | --- | --- | --- |
|Result OpenTopic(const Topic \&topic) | topic：打开的主题 | | 打开对应的topic |
| void CloseTopic(const Topic \&topic) | topic：关闭的主题| |关闭对应的topic |
| void UpdateData(const DataList \&dataList) | dataList：订阅的数据 | | 当订阅topic时，被订阅的topic每周期会通过UpdateData更新数据 |
| Result Enable(const std::string \&param = "") | param：预留 | | 使能本实例 |
| void Disable() | | | 关闭本实例 |
| void Run() | | | 每周期会执行run函数 |

### 实例示例

```C++
#include <oeaware/interface.h>
#include <oeaware/data/thread_info.h>

class Test : public oeaware::Interface {
public:
    Test() {
        name = "TestA";
        version = "1.0";
        description = "this is a test plugin";
        supportTopics;
        priority = 0;
        type = 0;
        period = 20;
    }
    oeaware::Result OpenTopic(const oeaware::Topic &topic) override {
        return oeaware::Result(OK);
    }
    void CloseTopic(const oeaware::Topic &topic) override {

    }
    void UpdateData(const DataList &dataList) override {
        for (int i = 0; i < dataList.len; ++i) {
            ThreadInfo *info = static_cast<ThreadInfo*>(dataList.data[i]);
            INFO(logger, "pid: " << info->pid << ", name: " << info->name);
        }
    }
    oeaware::Result Enable(const std::string &param = "") override {
        Subscribe(oeaware::Topic{"thread_collector", "thread_collector", ""});
        return oeaware::Result(OK);
    }
    void Disable() override {

    }
    void Run() override {
        DataList dataList;
        oeaware::SetDataListTopic(&dataList, "test", "test", "");
        dataList.len = 1;
        dataList.data = new void* [1];
        dataList.data[0] = &pubData;
        Publish(dataList);
    }
private:
    int pubData = 1;
};

extern "C" void GetInstance(std::vector<std::shared_ptr<oeaware::Interface>> &interfaces)
{
    interfaces.emplace_back(std::make_shared<Test>());
}
```

## 内部插件

### libpmu.so

| 实例名称 | 架构 | 说明 | topic |
| --- | --- | --- | --- |
| pmu\_counting\_collector | aarch64 | 采集count相关事件 |cycles，net:netif\_rx，L1-dcache-load-misses，L1-dcache-loads，L1-icache-load-misses，L1-icache-loads，branch-load-misses，branch-loads，dTLB-load-misses，dTLB-loads，iTLB-load-misses，iTLB-loads，cache-references，cache-misses，l2d\_tlb\_refill，l2d\_cache\_refill，l1d\_tlb\_refill，l1d\_cache\_refill，l1d\_tlb，l1i\_tlb，l1i\_tlb\_refill，l2d\_tlb，l2i\_tlb，l2i\_tlb\_refill，inst\_retired，instructions，sched:sched\_process\_fork，sched:sched\_process\_exit |
| pmu\_sampling\_collector | aarch64 | 采集sample相关事件 | cycles，skb:skb\_copy\_datagram\_iovec，net:napi\_gro\_receive\_entry |
| pmu\_spe\_collector | aarch64 | 采集spe事件 | spe |
| pmu\_uncore\_collector | aarch64 | 采集uncore事件 | uncore |

#### 限制条件

采集spe事件需要依赖硬件能力，此插件运行依赖 BIOS 的 SPE，运行前需要将 SPE 打开。

运行perf list | grep arm\_spe查看是否已经开启SPE，如果开启，则有如下显示：

```sh
arm_spe_0//                                      [Kernel PMU event]
```

如果没有开启，则按下述步骤开启。

检查BIOS配置项 MISC Config --> SPE 的状态，如果状态为 Disable，则需要更改为 Enable。如果找不到这个选项，可能是BIOS版本过低。

进入系统`vim /boot/efi/EFI/openEuler/grub.cfg`，定位到内核版本对应的开机启动项，在末尾增加`kpti=off`。例如：

```sh
linux   /vmlinuz-4.19.90-2003.4.0.0036.oe1.aarch64 root=/dev/mapper/openeuler-root ro rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap video=VGA-1:640x480-32@60me rhgb quiet  smmu.bypassdev=0x1000:0x17 smmu.bypassdev=0x1000:0x15 crashkernel=1024M,high video=efifb:off video=VGA-1:640x480-32@60me kpti=off
```

按**ESC**，输入“:wq”，按**Enter**保存并退出。执行reboot命令重启服务器。

### libsystem\_collector.so

系统信息采集插件。

| 实例名称 | 架构 | 说明 | topic |
| --- | --- | --- | --- |
| thread\_collector | aarch64/x86 | 采集系统中的线程信息 | thread\_collector |
| kernel\_config | aarch64/x86| 采集内核相关参数，包括sysctl所有参数、lscpu、meminfo等 | get\_kernel\_config，get\_cmd，set\_kernel\_config |
| command\_collector | aarch64/x86 | 采集sysstat相关数据 | mpstat，iostat，vmstat，sar，pidstat |

### libdocker\_collector.so

docker信息采集插件。

| 实例名称 | 架构 | 说明 | topic |
| --- | --- | --- | --- |
| docker\_collector | aarch64/x86 | 采集docker相关信息 | docker\_collector |

### libthread\_scenario.so

线程感知插件。

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| thread\_scenario | aarch64/x86 | 通过配置文件获取对应线程信息 | thread\_collector::thread\_collector |

#### 配置文件

thread\_scenario.conf

```sh
redis
fstime
fsbuffer
fsdisk
```

### libanalysis\_oeaware.so

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| analysis\_aware | 分析当前环境的业务特征，并给出优化建议 | aarch64 | pmu\_spe\_collector::spe, pmu\_counting\_collector::net:netif\_rx, pmu\_sampling\_collector::cycles, pmu\_sampling\_collector::skb:skb\_copy\_datagram\_iovec, pmu\_sampling\_collector::net:napi\_gro\_receive\_entry |

### libsystem\_tune.so

系统调优插件。

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| stealtask\_tune | aarch64 | 高负载场景下，通过轻量级搜索算法，实现多核间快速负载均衡，最大化cpu资源利用率 | 无 |
| smc\_tune | aarch64 | 使能smc加速，对使用tcp协议的连接无感加速 | 无 |
| xcall\_tune | aarch64 | 通过减少系统调用底噪，提升系统性能 | thread\_collector::thread\_collector |
| seep\_tune | aarch64 | 使能智能功耗模式，降低系统能耗 | 无 |
| transparent\_hugepage\_tune | aarch64/x86 | 开启透明大页，降低tlbmiss | 无 |
| preload\_tune | aarch64 | 无感加载动态库 | 无 |
| binary\_tune | aarch64 | 将容器中运行的特殊二进制文件绑定到物理核心，通过解析ELF文件中的特殊段识别需要调优的程序，并根据配置进行CPU亲和性绑定，提升程序性能 | env\_info::static, env\_info::realtime, thread\_collector::thread\_collector, docker\_collector::docker\_collector |
| cluster\_tune | aarch64 | 启用CPU cluster调度来优化性能 | 无 |
| dynamic\_smt\_tune | aarch64 | 低负载场景优先分配物理核，减少超线程的核间干扰 | 无 |
| numa\_sched\_tune | aarch64 | 针对有numa瓶颈的场景，让线程在整个生命周期尽可能在同numa内调度 | 无 |
| hardirq\_tune | aarch64 | 将网卡队列对应的中断尽量和使用该中断的业务绑定在相同numa上，减少跨numa访问 | 无 |
| multi\_net\_path | aarch64 | 网卡多路径调优，每个中断只处理所在numa上的业务 | 无 |
| soft\_domain\_tune | aarch64 | 分域调度调优，多实例业务单个实例尽量在独立的调度域内调度，更加亲和 | env\_info::static, thread\_collector::thread\_collector, docker\_collector::docker\_collector |

#### 配置文件

##### xcall.yaml

```yaml
redis: # 线程名称
    - xcall_1: 1 #xcall_1表示xcall优化方式，目前只有xcall_1; 1表示需要优化系统调用号
mysql:
    - xcall_1: 1
node:
    - xcall_1: 1
```

**限制说明**：xcall\_tune依赖内核特性，需要开启FAST\_SYSCALL编译内核，并且在cmdline里增加xcall字段。

##### preload.yaml

路径：`/etc/oeAware/preload.yaml`

```yaml
- appname: ""
  so: ""
```

通过执行`oeawarectl -e preload_tune`命令，根据配置文件给对应进程加载so。

##### soft\_domain.yaml

配置文件路径: /etc/oeAware/plugin/soft\_domain.yaml

配置说明：

```yaml
- type: 配置类型，支持 "docker" 或 "process"
  * docker: 对Docker容器进行分域调度
  * process: 对进程进行分域调度（默认不生效，仅对配置的进程进行分域调度）     
- whitelist: 白名单列表，支持通配符匹配（如 "mysql*"）
- cpu_num: CPU配额，字符串格式，不能超过单个NUMA节点的CPU数量
```

注意：

1. cpu\_num 不能超过单个NUMA节点的CPU数量，否则配置校验会失败
2. 如果配置文件不存在或为空，则默认什么也不配置
3. 容器一旦绑定NUMA后，就不会再修改
4. 如果不需要任何配置，可以保留为空或删除所有配置项

Docker容器分域配置示例

```yaml
- type: "docker"
  whitelist: ["mysql*", "redis*"]
  cpu_num: "16"
```

进程分域配置示例

```yaml
- type: "process"
  whitelist: ["mysqld", "redis-server"]
  cpu_num: "8"
```

### libub\_tune.so

unixbench调优插件。

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| unixbench\_tune | aarch64/x86 | 通过减少远端内存访问，优化ub性能 | thread\_collector::thread\_collector |

### libdocker\_tune.so

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| docker\_cpu\_burst | aarch64 | 在出现突发负载时，CPUBurst可以为容器临时提供额外的CPU资源，缓解CPU限制带来的性能瓶颈 | pmu\_counting\_collector::cycles，docker\_collector::docker\_collector |
| docker\_coordination\_burst\_tune | aarch64 | 感知多容器的CPU配额，划分空闲CPU算力给算力不足的容器  | 无 |
| load\_based\_scheduling\_tune | aarch64 | 针对超过负载超过阈值的容器，自动使能潮汐调度，使资源在容器间更均匀 | docker\_collector::docker\_collector, env\_info\_collector::static, pmu\_sampling\_collector::cycles |
| docker\_cluster\_affinity | aarch64 | 在系统存在cluster架构是，容器感知cluster架构进行调度，并感知多容器间CPU负载，在容器与容器之间进行调整quota资源（针对多容器资源负载不均衡场景） | l3c\_hit, docker\_collector::docker\_collector |

## 外部插件

外部插件需要通过以下命令安装，例如安装numafast相关插件。

```sh
oeawarectl -i numafast
```

### libscenario\_numa.so

| 实例名称 | 架构 | 说明 | 订阅 | topic |
| --- | --- | --- | --- | --- |
| scenario\_numa | aarch64 | 感知当前环境跨NUMA访存比例，用于实例或sdk订阅（无法单独使能） | pmu\_uncore\_collector::uncore | system\_score |

### libtune\_numa.so

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| tune\_numa\_mem\_access | aarch64 | 周期性迁移线程和内存，减少跨NUMA内存访问 | scenario\_numa::system\_score, pmu\_spe\_collector::spe, pmu\_counting\_collector::cycles |

#### tune\_numa\_mem\_access使用说明

tune\_numa\_mem\_access可以通过 `--help`命令查看所有的参数及其作用

```shell
[root@localhost ~]# oeawarectl -e tune_numa_mem_access -cmd "--help cmd"
Instance enabled failed, because show help message:
Usage: oeaware -e tune_numa_mem_access -cmd "[options][<param>]"
   or vim /etc/numafast.yaml and set options
 attr:c => support conf by cmdline, y => support conf by yaml, r => support reload yaml online
Options:
    -i, --sampling-interval <n>    attr:cy, every sampling interval n msec, range is [100, 100000], default is 100
    -t, --sampling-times <n>       attr:cy, every optimizing have n times sampling, range is [1, 1000] default is 10
    -m, --tune-mode <mode>         attr:cy, tune mode, mode can be [b, t, p], default is b
                                              b: migrate page and thread
                                              t: migrate thread only
                                              p: migrate page only
    -w, --load-way <alg>           attr:cy, load way, can be [b, c], default is b
                                              b: balance the load of threads on all numa nodes
                                              c: centralize processes to fewer numas based on load
        --smt <alg>                attr:cy, smt mode, can be [off, phy-first], default is phy-first
                                              off: disable smt
                                              phy-first: migrate threads to physical cores first, may limit load
    -h, --help <type>              attr:c, show help info, type can be [cmd, yaml], default is cmd
    -v, --version                  attr:c, show version info
    -W, --whitelist <process name list>
                                   attr:cy, only migrate process in the list, regexp list split by comma, if not set, migrate all process.
    -b, --blacklist <process name list>
                                   attr:cy, do not migrate process in the list, regexp list split by comma, priority higher than whitelist.
        --precise-load             attr:cy, load control precisely
        --mem-numa-aggregation     attr:cy, process memory aggregate by numa
        --mem-balance              attr:cy, process memory average by numa
 other options refer to /etc/numafast.yaml

[root@localhost format]# oeawarectl -e tune_numa_mem_access -cmd "--help yaml"
Instance enabled failed, because show help message:
Usage: vim /etc/numafast.yaml and set options
sampling-interval: <n> # every sampling interval n msec, range is [100, 100000], default is 100
sampling-times: <n> # every optimizing have n times sampling, range is [1, 1000] default is 10
tune-mode: <mode> # tune mode, mode can be [b, t, p], default is b
    # b: migrate page and thread
    # t: migrate thread only
    # p: migrate page only
load-way: <alg> # load way, can be [b, c], default is b
    # b: balance the load of threads on all numa nodes
    # c: centralize processes to fewer numas based on load
smt: <alg> # smt mode, can be [off, phy-first, load-first], default is phy-first
    # off: disable smt
    # phy-first: migrate threads to physical cores first, may limit load
    # load-first: migrate threads to physical cores based on load, limit load
whitelist: [] # only migrate process in the list, regexp list split by comma, if not set, migrate all process.
group: # process affinity group
    # - [process1, process2, ...]
min-numa-score: <n> # min numa score, range is [0 ,1000], default is 955
max-numa-score: <n> # max numa score, range is [0, 1000], default is 975
min-rx-ops-per-ms: <n> # min rx ops per ms, default is 10000
numa-ratio: [] # process initial load distribution for each node
page-reserve: <n> # page reserve, range is [0, 4294967295], default is 100000
precise-load: <true|false> # load control precisely
mem-numa-aggregation: <true|false> # process memory aggregate by numa
process: # process config
    # - name: process1 # process name, /proc/pid/comm
    #   params-regex: "" # process params regex, /proc/pid/cmdline
    #   algorithm: "" # process algorithm, support [MigrateThreadsToOneNode, BalanceProcNum]
    #   migrate-all-memory: "" # migrate all memory, support [true, false]
    #   default-mig-mem-node: "" # default migrate memory node, support [0, numa_node_num - 1]
    #   net-affinity: "" # process net affinity, set net interface name

```

## SDK使用说明

```C
typedef int(*Callback)(const DataList *);
int OeInit(); // 初始化资源，与server建立链接
int OeSubscribe(const CTopic *topic, Callback callback); // 订阅topic，异步执行callback
int OeUnsubscribe(const CTopic *topic); // 取消订阅topic
int OePublish(const DataList *dataList); // 发布数据到server
void OeClose(); // 释放资源
```

**示例**

```C
#include "oe_client.h"
#include "command_data.h"
int f(const DataList *dataList)
{
    int i = 0;
    for (; i < dataList->len; i++) {
        CommandData *data = (CommandData*)dataList->data[i];
        for (int j = 0; j < data->attrLen; ++j) {
            printf("%s ", data->itemAttr[j]);
        }
        printf("\n");
    }
    return 0;
}
int main() {
    OeInit();
    CTopic topic = {
        "command_collector",
        "sar",
        "-q 1",
    };
    if (OeSubscribe(&topic, f) < 0) {
        printf("failed\n");
    } else {
        printf("success\n");
    }
    sleep(10);
    OeClose();
}
```

## 约束限制

### 功能约束

oeAware默认集成了arm的微架构采集libkperf模块，该模块同一时间只能有一个进程进行调用，如其他进程调用或者使用perf命令可能存在冲突。

### 操作约束

当前oeAware仅支持root组用户进行操作，sdk支持root组和oeaware组用户使用。

## 注意事项

oeAware的配置文件和插件用户组和权限有严格校验，不要对oeAware的相关文件进行权限和用户组进行修改。

权限说明：

* 插件文件：440

* 客户端执行文件：750

* 服务端执行文件：750

* 服务配置文件：640

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/community_tools/performance/oeaware/oeaware_user_guide.md
---
# oeAware用户指南

## 简介

oeAware是在openEuler上实现低负载采集感知调优的框架，目标是动态感知系统行为后智能使能系统的调优特性。传统调优特性都以独立运行且静态打开关闭为主，oeAware将调优拆分采集、感知和调优三层，每层通过订阅方式关联，各层采用插件式开发尽可能复用。

## 安装

配置openEuler的yum源，使用yum命令安装。在openEuler-22.03-LTS-SP4版本中会默认安装。

```shell
yum install oeAware-manager
```

## 使用方法

首先启动oeaware服务，然后通过`oeawarectl`命令进行使用。

### 服务启动

通过systemd服务启动。安装完成后会默认启动。

```shell
systemctl start oeaware
```

### 配置文件

配置文件路径：`/etc/oeAware/config.yaml`。

```yaml
log_path: /var/log/oeAware #日志存储路径
log_level: 1 #日志等级 1：DEBUG 2：INFO 3：WARN 4：ERROR
enable_list: #默认使能插件
   - name: libtest.so #只配置插件，使能本插件的所有实例
   - name: libtest1.so #配置插件实例，使能配置的插件实例
     instances:
      - instance1
      - instance2
      ...
   ...
plugin_list: #可支持下载的包
  - name: test #名称需要唯一，如果重复取第一个配置
    description: hello world
    url: https://gitee.com/openeuler/oeAware-manager/raw/master/README.md #url非空
  ...
```

修改配置文件后，通过以下命令重启服务。

```shell
systemctl restart oeaware
```

### 插件说明

**插件定义**：一个插件对应一个`.so`文件，插件分为采集插件、感知插件和调优插件。

**实例定义**：服务中的调度单位是实例，一个插件中包括多个实例。例如，一个采集插件包括多个采集项，每个采集项是一个实例。

### 插件加载

服务会默认加载插件存储路径下的插件。

插件路径：`/usr/lib64/oeAware-plugin/`。

另外也可以通过手动加载的方式加载插件。

```shell
oeawarectl -l | --load <插件名>
```

示例：

```shell
[root@localhost ~]# oeawarectl -l libthread_collect.so
Plugin loaded successfully.
```

失败返回错误说明。

### 插件卸载

```shell
oeawarectl -r <插件名> | --remove <插件名>
```

示例：

```shell
[root@localhost ~]# oeawarectl -r libthread_collect.so
Plugin remove successfully.
```

失败返回错误说明。

### 插件查询

#### 查询插件状态信息

```shell
oeawarectl -q  #查询系统中已经加载的所有插件
oeawarectl --query <插件名> #查询指定插件
```

示例：

```shell
Show plugins and instances status.
------------------------------------------------------------
libthread_scenario.so
        thread_scenario(available, close, count: 0)
libanalysis_oeaware.so
        hugepage_analysis(available, close, count: 0)
        dynamic_smt_analysis(available, close, count: 0)
        smc_d_analysis(available, close, count: 0)
        xcall_analysis(available, close, count: 0)
        net_hirq_analysis(available, close, count: 0)
        numa_analysis(available, close, count: 0)
        docker_coordination_burst_analysis(available, close, count: 0)
        microarch_tidnocmp_analysis(available, close, count: 0)
libscenario_numa.so
        scenario_numa(available, close, count: 12)
libsystem_tune.so
        stealtask_tune(available, close, count: 0)
        dynamic_smt_tune(available, close, count: 0)
        smc_tune(available, close, count: 0)
        xcall_tune(available, close, count: 0)
        transparent_hugepage_tune(available, close, count: 0)
        seep_tune(available, close, count: 0)
        preload_tune(available, close, count: 0)
        binary_tune(available, close, count: 0)
        numa_sched_tune(available, close, count: 0)
        net_hard_irq_tune(available, close, count: 0)
        multi_net_path_tune(available, close, count: 0)
libdocker_tune.so
        docker_cpu_burst(available, close, count: 0)
        docker_burst(available, close, count: 0)
        load_based_scheduling_tune(available, close, count: 0)
libpmu.so
        pmu_counting_collector(available, close, count: 0)
        pmu_sampling_collector(available, close, count: 12)
        pmu_spe_collector(available, close, count: 12)
        pmu_uncore_collector(available, close, count: 12)
libdocker_collector.so
        docker_collector(available, close, count: 0)
libtune_numa.so
        tune_numa_mem_access(available, close, count: 12)
libub_tune.so
        unixbench_tune(available, close, count: 0)
libsystem_collector.so
        thread_collector(available, close, count: 0)
        kernel_config(available, close, count: 0)
        command_collector(available, close, count: 0)
        env_info_collector(available, close, count: 0)
        net_interface_info(available, close, count: 0)
------------------------------------------------------------
format:
[plugin]
        [instance]([dependency status], [running status], [enable cnt])
dependency status: available means satisfying dependency, otherwise unavailable.
running status: running means that instance is running, otherwise close.
enable cnt: number of instances enabled.
```

失败返回错误说明。

#### 查询调优实例信息

```shell
oeawarectl --info
```

显示调优实例描述信息及运行状态。

#### 查询运行实例订阅关系

```shell
oeawarectl -Q  #查询所有运行实例的订阅关系图
oeawarectl --query-dep= <插件实例>  #查询运行实例订阅关系图
```

在当前目录下生成dep.png，显示订阅关系。

实例未运行，不会显示订阅关系。

示例：

```sh
oeawarectl -e thread_scenario
oeawarectl -Q
```

![img](./figures/dep.png)

### 插件实例使能

#### 使能插件实例

```shell
oeawarectl -e | --enable <插件实例>
```

使能某个插件实例，会将其订阅的topic实例一起使能。

失败返回错误说明。

推荐使能插件列表：

* libsystem\_tune.so：stealtask\_tune，smc\_tune，xcall\_tune，seep\_tune。
* libub\_tune.so：unixbench\_tune。
* libtune\_numa.so：tune\_numa\_mem\_access。

其他插件主要用来提供数据，可通过sdk获取插件数据。

#### 关闭插件实例

```shell
oeawarectl -d | --disable <插件实例>
```

关闭某个插件实例，会将其订阅的topic实例一起关闭。

失败返回错误说明。

### 插件下载安装

通过`--list`命令查询支持下载的rpm包和已安装的插件。

```shell
oeawarectl --list
```

查询结果如下。

```shell
Supported Packages: #可下载的包
[name1] #config中配置的plugin_list
[name2]
...
Installed Plugins: #已安装的插件
[name1]
[name2]
...
```

通过`--install`命令下载安装rpm包。

```shell
oeawarectl -i | --install <rpm包名> #指定--list下查询得到的包名称（Supported Packages下的包）
```

失败返回错误说明。

### 分析模式

```sh
oeawarectl analysis -h
usage: oeawarectl analysis [options]...
  options
   -t|--time <s>                    set analysis duration in seconds(default 30s), range from 1 to 100.
   -r|--realtime                    show real time report.
   -v|--verbose                     show verbose information.
   -h|--help                        show this help message.
   --l1-miss-threshold              set l1 tlbmiss threshold.
   --l2-miss-threshold              set l2 tlbmiss threshold.
   --out-path                       set the path of the analysis report.
   --dynamic-smt-threshold          set dynamic smt cpu threshold.
   --pid                            set the pid to be analyzed.
   --numa-thread-threshold          set numa sched thread creation threshold.
   --smc-change-rate                set smc connections change rate threshold.
   --smc-localnet-flow              set smc local net flow threshold.
   --host-cpu-usage-threshold       set host cpu usage threshold.
   --docker-cpu-usage-threshold     set docker cpu usage threshold.
```

\--l1-miss-threshold用于设置l1-tlb—miss阈值，超过这个阈值miss率为high。

\--l2-miss-threshold用于设置l2-tlb—miss阈值，超过这个阈值miss率为high。

示例：

执行以下命令，输出系统分析报告。

```sh
oeawarectl analysis -t 10
```

报告分为三部分：

* Data Analysis：根据系统运行状态，给出系统性能数据分析。
* Analysis Conclusion：给出系统分析结论。
* Analysis Suggestion：给出具体调优方法。

### 帮助

通过`--help`查看帮助。

```shell
usage: oeawarectl [options]...
  options
    analysis                run analysis mode.
    -l|--load [plugin]      load plugin.
    -r|--remove [plugin]    remove plugin from system.
    -e|--enable [instance]  enable the plugin instance.
    -d|--disable [instance] disable the plugin instance.
    -q                      query all plugins information.
    --query [plugin]        query the plugin information.
    -Q                      query all instances dependencies.
    --query-dep [instance]  query the instance dependency.
    --list                  the list of supported plugins.
    --info                  the list of InfoCmd plugins.
    -i|--install [plugin]   install plugin from the list.
    --reload-conf           reload config file(now only support log level).
    --help                  show this help message.
```

## 插件开发说明

### 基础数据结构

```c++
typedef struct {
    char *instanceName; // 实例名称
    char *topicName; // 主题名称
    char *params; // 参数
} CTopic;

typedef struct {
    CTopic topic;
    unsigned long long len; // data数组的长度
    void **data; // 存储的数据
} DataList;

const int OK = 0;
const int FAILED = -1;

typedef struct {
    int code; // 成功返回OK，失败返回FAILED
    char *payload; // 附带信息
} Result;

```

### 实例基类

```c++
namespace oeaware {
// Instance type.
const int TUNE = 0b10000;
const int SCENARIO = 0b01000;
const int RUN_ONCE = 0b00010;
class Interface {
public:
    virtual Result OpenTopic(const Topic &topic) = 0;
    virtual void CloseTopic(const Topic &topic) = 0;
    virtual void UpdateData(const DataList &dataList) = 0;
    virtual Result Enable(const std::string &param = "") = 0;
    virtual void Disable() = 0;
    virtual void Run() = 0;
protected:
    std::string name;
    std::string version;
    std::string description;
    std::vector<Topic> supportTopics;
    int priority;
    int type;
    int period;
}
}
```

实例开发继承实例基类，实现6个虚函数，并对类的7个属性赋值。

实例采用订阅发布模式，通过Subscribe获取数据，通过Publish接口发布数据。

### 属性说明

| 属性 | 类型 | 说明 |
| --- | --- | --- |
| name | string | 实例名称 |
| version | string | 实例版本（预留） |
| description | string | 实例描述 |
| supportTopics | vector\<Topic> | 支持的topic |
| priority | int | 实例执行的优先级（调优 > 感知 > 采集）|
| type | int | 实例类型，通过比特位标识，第二位表示单次执行实例，第三位表示采集实例，第四位表示感知实例，第5位表示调优实例|
| period | int | 实例执行周期，单位ms，period为10的倍数 |

### 接口说明

| 函数名 | 参数 | 返回值 | 说明 |
| --- | --- | --- | --- |
|Result OpenTopic(const Topic \&topic) | topic：打开的主题 | | 打开对应的topic |
| void CloseTopic(const Topic \&topic) | topic：关闭的主题| |关闭对应的topic |
| void UpdateData(const DataList \&dataList) | dataList：订阅的数据 | | 当订阅topic时，被订阅的topic每周期会通过UpdateData更新数据 |
| Result Enable(const std::string \&param = "") | param：预留 | | 使能本实例 |
| void Disable() | | | 关闭本实例 |
| void Run() | | | 每周期会执行run函数 |

### 实例示例

```C++
#include <oeaware/interface.h>
#include <oeaware/data/thread_info.h>

class Test : public oeaware::Interface {
public:
    Test() {
        name = "TestA";
        version = "1.0";
        description = "this is a test plugin";
        supportTopics;
        priority = 0;
        type = 0;
        period = 20;
    }
    oeaware::Result OpenTopic(const oeaware::Topic &topic) override {
        return oeaware::Result(OK);
    }
    void CloseTopic(const oeaware::Topic &topic) override {

    }
    void UpdateData(const DataList &dataList) override {
        for (int i = 0; i < dataList.len; ++i) {
            ThreadInfo *info = static_cast<ThreadInfo*>(dataList.data[i]);
            INFO(logger, "pid: " << info->pid << ", name: " << info->name);
        }
    }
    oeaware::Result Enable(const std::string &param = "") override {
        Subscribe(oeaware::Topic{"thread_collector", "thread_collector", ""});
        return oeaware::Result(OK);
    }
    void Disable() override {

    }
    void Run() override {
        DataList dataList;
        oeaware::SetDataListTopic(&dataList, "test", "test", "");
        dataList.len = 1;
        dataList.data = new void* [1];
        dataList.data[0] = &pubData;
        Publish(dataList);
    }
private:
    int pubData = 1;
};

extern "C" void GetInstance(std::vector<std::shared_ptr<oeaware::Interface>> &interfaces)
{
    interfaces.emplace_back(std::make_shared<Test>());
}
```

## 内部插件

### libpmu.so

| 实例名称 | 架构 | 说明 | topic |
| --- | --- | --- | --- |
| pmu\_counting\_collector | aarch64 | 采集count相关事件 |cycles，net:netif\_rx，L1-dcache-load-misses，L1-dcache-loads，L1-icache-load-misses，L1-icache-loads，branch-load-misses，branch-loads，dTLB-load-misses，dTLB-loads，iTLB-load-misses，iTLB-loads，cache-references，cache-misses，l2d\_tlb\_refill，l2d\_cache\_refill，l1d\_tlb\_refill，l1d\_cache\_refill，l1d\_tlb，l1i\_tlb，l1i\_tlb\_refill，l2d\_tlb，l2i\_tlb，l2i\_tlb\_refill，inst\_retired，instructions，sched:sched\_process\_fork，sched:sched\_process\_exit |
| pmu\_sampling\_collector | aarch64 | 采集sample相关事件 | cycles，skb:skb\_copy\_datagram\_iovec，net:napi\_gro\_receive\_entry |
| pmu\_spe\_collector | aarch64 | 采集spe事件 | spe |
| pmu\_uncore\_collector | aarch64 | 采集uncore事件 | uncore |

#### 限制条件

采集spe事件需要依赖硬件能力，此插件运行依赖 BIOS 的 SPE，运行前需要将 SPE 打开。

运行perf list | grep arm\_spe查看是否已经开启SPE，如果开启，则有如下显示：

```sh
arm_spe_0//                                      [Kernel PMU event]
```

如果没有开启，则按下述步骤开启。

检查BIOS配置项 MISC Config --> SPE 的状态，如果状态为 Disable，则需要更改为 Enable。如果找不到这个选项，可能是BIOS版本过低。

进入系统`vim /boot/efi/EFI/openEuler/grub.cfg`，定位到内核版本对应的开机启动项，在末尾增加`kpti=off`。例如：

```sh
linux   /vmlinuz-4.19.90-2003.4.0.0036.oe1.aarch64 root=/dev/mapper/openeuler-root ro rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap video=VGA-1:640x480-32@60me rhgb quiet  smmu.bypassdev=0x1000:0x17 smmu.bypassdev=0x1000:0x15 crashkernel=1024M,high video=efifb:off video=VGA-1:640x480-32@60me kpti=off
```

按**ESC**，输入“:wq”，按**Enter**保存并退出。执行reboot命令重启服务器。

### libsystem\_collector.so

系统信息采集插件。

| 实例名称 | 架构 | 说明 | topic |
| --- | --- | --- | --- |
| thread\_collector | aarch64/x86 | 采集系统中的线程信息 | thread\_collector |
| kernel\_config | aarch64/x86| 采集内核相关参数，包括sysctl所有参数、lscpu、meminfo等 | get\_kernel\_config，get\_cmd，set\_kernel\_config |
| command\_collector | aarch64/x86 | 采集sysstat相关数据 | mpstat，iostat，vmstat，sar，pidstat |

### libdocker\_collector.so

docker信息采集插件。

| 实例名称 | 架构 | 说明 | topic |
| --- | --- | --- | --- |
| docker\_collector | aarch64/x86 | 采集docker相关信息 | docker\_collector |

### libthread\_scenario.so

线程感知插件。

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| thread\_scenario | aarch64/x86 | 通过配置文件获取对应线程信息 | thread\_collector::thread\_collector |

#### 配置文件

thread\_scenario.conf

```sh
redis
fstime
fsbuffer
fsdisk
```

### libanalysis\_oeaware.so

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| analysis\_aware | 分析当前环境的业务特征，并给出优化建议 | aarch64 | pmu\_spe\_collector::spe, pmu\_counting\_collector::net:netif\_rx, pmu\_sampling\_collector::cycles, pmu\_sampling\_collector::skb:skb\_copy\_datagram\_iovec, pmu\_sampling\_collector::net:napi\_gro\_receive\_entry |

### libsystem\_tune.so

系统调优插件。

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| stealtask\_tune | aarch64 | 高负载场景下，通过轻量级搜索算法，实现多核间快速负载均衡，最大化cpu资源利用率 | 无 |
| smc\_tune | aarch64 | 使能smc加速，对使用tcp协议的连接无感加速 | 无 |
| xcall\_tune | aarch64 | 通过减少系统调用底噪，提升系统性能 | thread\_collector::thread\_collector |
| seep\_tune | aarch64 | 使能智能功耗模式，降低系统能耗 | 无 |
| transparent\_hugepage\_tune | aarch64/x86 | 开启透明大页，降低tlbmiss | 无 |
| preload\_tune | aarch64 | 无感加载动态库 | 无 |
| binary\_tune | aarch64 | 将容器中运行的特殊二进制文件绑定到物理核心，通过解析ELF文件中的特殊段识别需要调优的程序，并根据配置进行CPU亲和性绑定，提升程序性能 | env\_info::static, env\_info::realtime, thread\_collector::thread\_collector, docker\_collector::docker\_collector |
| cluster\_tune | aarch64 | 启用CPU cluster调度来优化性能 | 无 |
| dynamic\_smt\_tune | aarch64 | 低负载场景优先分配物理核，减少超线程的核间干扰 | 无 |
| numa\_sched\_tune | aarch64 | 针对有numa瓶颈的场景，让线程在整个生命周期尽可能在同numa内调度 | 无 |
| hardirq\_tune | aarch64 | 将网卡队列对应的中断尽量和使用该中断的业务绑定在相同numa上，减少跨numa访问 | 无 |
| multi\_net\_path | aarch64 | 网卡多路径调优，每个中断只处理所在numa上的业务 | 无 |
| soft\_domain\_tune | aarch64 | 分域调度调优，多实例业务单个实例尽量在独立的调度域内调度，更加亲和 | env\_info::static, thread\_collector::thread\_collector, docker\_collector::docker\_collector |

#### 配置文件

##### xcall.yaml

```yaml
redis: # 线程名称
    - xcall_1: 1 #xcall_1表示xcall优化方式，目前只有xcall_1; 1表示需要优化系统调用号
mysql:
    - xcall_1: 1
node:
    - xcall_1: 1
```

**限制说明**：xcall\_tune依赖内核特性，需要开启FAST\_SYSCALL编译内核，并且在cmdline里增加xcall字段。

##### preload.yaml

路径：`/etc/oeAware/preload.yaml`

```yaml
- appname: ""
  so: ""
```

通过执行`oeawarectl -e preload_tune`命令，根据配置文件给对应进程加载so。

##### soft\_domain.yaml

配置文件路径: /etc/oeAware/plugin/soft\_domain.yaml

配置说明：

```yaml
- type: 配置类型，支持 "docker" 或 "process"
  * docker: 对Docker容器进行分域调度
  * process: 对进程进行分域调度（默认不生效，仅对配置的进程进行分域调度）     
- whitelist: 白名单列表，支持通配符匹配（如 "mysql*"）
- cpu_num: CPU配额，字符串格式，不能超过单个NUMA节点的CPU数量
```

注意：

1. cpu\_num 不能超过单个NUMA节点的CPU数量，否则配置校验会失败
2. 如果配置文件不存在或为空，则默认什么也不配置
3. 容器一旦绑定NUMA后，就不会再修改
4. 如果不需要任何配置，可以保留为空或删除所有配置项

Docker容器分域配置示例

```yaml
- type: "docker"
  whitelist: ["mysql*", "redis*"]
  cpu_num: "16"
```

进程分域配置示例

```yaml
- type: "process"
  whitelist: ["mysqld", "redis-server"]
  cpu_num: "8"
```

### libub\_tune.so

unixbench调优插件。

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| unixbench\_tune | aarch64/x86 | 通过减少远端内存访问，优化ub性能 | thread\_collector::thread\_collector |

### libdocker\_tune.so

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| docker\_cpu\_burst | aarch64 | 在出现突发负载时，CPUBurst可以为容器临时提供额外的CPU资源，缓解CPU限制带来的性能瓶颈 | pmu\_counting\_collector::cycles，docker\_collector::docker\_collector |
| docker\_coordination\_burst\_tune | aarch64 | 感知多容器的CPU配额，划分空闲CPU算力给算力不足的容器  | 无 |
| load\_based\_scheduling\_tune | aarch64 | 针对超过负载超过阈值的容器，自动使能潮汐调度，使资源在容器间更均匀 | docker\_collector::docker\_collector, env\_info\_collector::static, pmu\_sampling\_collector::cycles |
| docker\_cluster\_affinity | aarch64 | 在系统存在cluster架构是，容器感知cluster架构进行调度，并感知多容器间CPU负载，在容器与容器之间进行调整quota资源（针对多容器资源负载不均衡场景） | l3c\_hit, docker\_collector::docker\_collector |

## 外部插件

外部插件需要通过以下命令安装，例如安装numafast相关插件。

```sh
oeawarectl -i numafast
```

### libscenario\_numa.so

| 实例名称 | 架构 | 说明 | 订阅 | topic |
| --- | --- | --- | --- | --- |
| scenario\_numa | aarch64 | 感知当前环境跨NUMA访存比例，用于实例或sdk订阅（无法单独使能） | pmu\_uncore\_collector::uncore | system\_score |

### libtune\_numa.so

| 实例名称 | 架构 | 说明 | 订阅 |
| --- | --- | --- | --- |
| tune\_numa\_mem\_access | aarch64 | 周期性迁移线程和内存，减少跨NUMA内存访问 | scenario\_numa::system\_score, pmu\_spe\_collector::spe, pmu\_counting\_collector::cycles |

#### tune\_numa\_mem\_access使用说明

tune\_numa\_mem\_access可以通过 `--help`命令查看所有的参数及其作用

```shell
[root@localhost ~]# oeawarectl -e tune_numa_mem_access -cmd "--help cmd"
Instance enabled failed, because show help message:
Usage: oeaware -e tune_numa_mem_access -cmd "[options][<param>]"
   or vim /etc/numafast.yaml and set options
 attr:c => support conf by cmdline, y => support conf by yaml, r => support reload yaml online
Options:
    -i, --sampling-interval <n>    attr:cy, every sampling interval n msec, range is [100, 100000], default is 100
    -t, --sampling-times <n>       attr:cy, every optimizing have n times sampling, range is [1, 1000] default is 10
    -m, --tune-mode <mode>         attr:cy, tune mode, mode can be [b, t, p], default is b
                                              b: migrate page and thread
                                              t: migrate thread only
                                              p: migrate page only
    -w, --load-way <alg>           attr:cy, load way, can be [b, c], default is b
                                              b: balance the load of threads on all numa nodes
                                              c: centralize processes to fewer numas based on load
        --smt <alg>                attr:cy, smt mode, can be [off, phy-first], default is phy-first
                                              off: disable smt
                                              phy-first: migrate threads to physical cores first, may limit load
    -h, --help <type>              attr:c, show help info, type can be [cmd, yaml], default is cmd
    -v, --version                  attr:c, show version info
    -W, --whitelist <process name list>
                                   attr:cy, only migrate process in the list, regexp list split by comma, if not set, migrate all process.
    -b, --blacklist <process name list>
                                   attr:cy, do not migrate process in the list, regexp list split by comma, priority higher than whitelist.
        --precise-load             attr:cy, load control precisely
        --mem-numa-aggregation     attr:cy, process memory aggregate by numa
        --mem-balance              attr:cy, process memory average by numa
 other options refer to /etc/numafast.yaml

[root@localhost format]# oeawarectl -e tune_numa_mem_access -cmd "--help yaml"
Instance enabled failed, because show help message:
Usage: vim /etc/numafast.yaml and set options
sampling-interval: <n> # every sampling interval n msec, range is [100, 100000], default is 100
sampling-times: <n> # every optimizing have n times sampling, range is [1, 1000] default is 10
tune-mode: <mode> # tune mode, mode can be [b, t, p], default is b
    # b: migrate page and thread
    # t: migrate thread only
    # p: migrate page only
load-way: <alg> # load way, can be [b, c], default is b
    # b: balance the load of threads on all numa nodes
    # c: centralize processes to fewer numas based on load
smt: <alg> # smt mode, can be [off, phy-first, load-first], default is phy-first
    # off: disable smt
    # phy-first: migrate threads to physical cores first, may limit load
    # load-first: migrate threads to physical cores based on load, limit load
whitelist: [] # only migrate process in the list, regexp list split by comma, if not set, migrate all process.
group: # process affinity group
    # - [process1, process2, ...]
min-numa-score: <n> # min numa score, range is [0 ,1000], default is 955
max-numa-score: <n> # max numa score, range is [0, 1000], default is 975
min-rx-ops-per-ms: <n> # min rx ops per ms, default is 10000
numa-ratio: [] # process initial load distribution for each node
page-reserve: <n> # page reserve, range is [0, 4294967295], default is 100000
precise-load: <true|false> # load control precisely
mem-numa-aggregation: <true|false> # process memory aggregate by numa
process: # process config
    # - name: process1 # process name, /proc/pid/comm
    #   params-regex: "" # process params regex, /proc/pid/cmdline
    #   algorithm: "" # process algorithm, support [MigrateThreadsToOneNode, BalanceProcNum]
    #   migrate-all-memory: "" # migrate all memory, support [true, false]
    #   default-mig-mem-node: "" # default migrate memory node, support [0, numa_node_num - 1]
    #   net-affinity: "" # process net affinity, set net interface name

```

## SDK使用说明

```C
typedef int(*Callback)(const DataList *);
int OeInit(); // 初始化资源，与server建立链接
int OeSubscribe(const CTopic *topic, Callback callback); // 订阅topic，异步执行callback
int OeUnsubscribe(const CTopic *topic); // 取消订阅topic
int OePublish(const DataList *dataList); // 发布数据到server
void OeClose(); // 释放资源
```

**示例**

```C
#include "oe_client.h"
#include "command_data.h"
int f(const DataList *dataList)
{
    int i = 0;
    for (; i < dataList->len; i++) {
        CommandData *data = (CommandData*)dataList->data[i];
        for (int j = 0; j < data->attrLen; ++j) {
            printf("%s ", data->itemAttr[j]);
        }
        printf("\n");
    }
    return 0;
}
int main() {
    OeInit();
    CTopic topic = {
        "command_collector",
        "sar",
        "-q 1",
    };
    if (OeSubscribe(&topic, f) < 0) {
        printf("failed\n");
    } else {
        printf("success\n");
    }
    sleep(10);
    OeClose();
}
```

## 约束限制

### 功能约束

oeAware默认集成了arm的微架构采集libkperf模块，该模块同一时间只能有一个进程进行调用，如其他进程调用或者使用perf命令可能存在冲突。

### 操作约束

当前oeAware仅支持root组用户进行操作，sdk支持root组和oeaware组用户使用。

## 注意事项

oeAware的配置文件和插件用户组和权限有严格校验，不要对oeAware的相关文件进行权限和用户组进行修改。

权限说明：

* 插件文件：440

* 客户端执行文件：750

* 服务端执行文件：750

* 服务配置文件：640

---

---
url: /en/docs/22.03_LTS_SP4/tools/community_tools/oemaker/oemaker_user_guide.md
---
# oemaker User Guide

## Overview

This document describes how to install and use oemaker, the openEuler image creation tool.

## Software and Hardware Requirements

The hardware and software requirements of the computer to create an ISO file using oemaker are as follows:

* CPU architecture: AArch64 or X86\_64
* OS: openEuler 22.03 LTS SP4
* 60 GB or more drive space for running oemaker and storing ISO images.

## Installation

The following uses openEuler 22.03 LTS SP4 on the AArch64 architecture as an example to describe how to install oemaker.

1. Ensure that openEuler 22.03 LTS SP4 has been installed on the computer.

   ```shell
    cat /etc/openEuler-release 
    openEuler release 22.03 LTS SP4
   ```

2. Download the ISO image (must be an **everything** image) of the corresponding architecture and save it to any directory (it is recommended that the available space of the directory be greater than 20 GB). In this example, the ISO image is saved to the **/home** directory.

   AArch64 image download:

   <https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/aarch64/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso>

   > \[!NOTE] **NOTE:**
   >
   > x86\_64 image download:
   >
   > <https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/x86_64/openEuler-22.03-LTS-SP4-everything-x86_64-dvd.iso>

3. Create a **/etc/yum.repos.d/local.repo** file to configure the Yum source. The following is an example of the configuration file. **baseurl** is the directory for mounting the ISO image.

   ```shell
   [local]
   name=local
   baseurl=file:///home/oemaker_iso
   gpgcheck=0
   enabled=1
   ```

4. Run the following command as the **root** user to mount the image to the **/home/oemaker\_iso** directory (ensure that the mount directory is the same as **baseurl** configured in the **repo** file) as the Yum repository:

   ```shell
   sudo mount -o loop /home/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso /home/oemaker_iso
   ```

5. Make the Yum repository take effect.

   ```shell
   yum clean all
   yum makecache
   ```

6. Install oemaker as the **root** user.

   ```shell
   sudo yum install -y oemaker
   ```

7. Run the following command as the **root** user to verify that the tool has been installed successfully:

   ```shell
    sudo oemaker -h
    Usage: oemaker [-h] [-t Type] [-p Product] [-v Version] [-r RELEASE] [-s REPOSITORY]
    
    optional arguments:
        -t Type        ISO Type, include standard debug source everything everything_debug everything_src livecd and netinst
        -p Product     Product Name, such as: openEuler
        -v Version     version identifier
        -r RELEASE     release information
        -s REPOSITORY  source dnf repository address link(may be listed multiple times)
        -h             show the help message and exit
   ```

## Image Creation

This section describes how to use oemaker to create an openEuler image.

### Command Description

#### Syntax

Run the **oemaker** command to use the tool. The command syntax is as follows:

```shell
oemaker [ --help | -h ] [ -t <ISO type> ] [ -p <Product name> ] [ -v <Version number> ] [-r <Release information> ] [-s <Source dnf repository address link>]
```

#### Command Options

| Option                         | Mandatory| Description                                                   |
|-----------------------------| -------- |---------------------------------------------------------|
| --help \ -h                | No      | Query the help information the command.                                             |
| -t \<ISO type>  | Yes      | Specify the image type. The value can be **standard**, **debug**, **source**, **everything**, **everything\_debug**, **everything\_src**, **livecd**, or **netinst**.|
| -p  \<Product name>             | No      | Product name.                            |
| -v \<Version number>  | No      | Product version.               |
| -r \<Release information>         | No      | Release information.                                                  |
| -s \<Source dnf repository address link>       | No      | Software repository.                                                 |

### Software Repository

The RPM packages of the new image can be:

* Packages of the original ISO image: The RPM packages to be installed are specified in the configuration file **rpmlist**. The configuration format is *software\_package\_name*. For example, **kernel**.

* Extra packages: Extra packages can be added to **/home/oemaker\_iso** and run the `createrepo` command based on **normal.xml** of the original image to regenerate the repository.

  > \[!NOTE] **NOTE:**
  >
  > * During image creation, if an RPM package specified in the configuration file cannot be found, the RPM package will not be added to the image.
  > * If the dependency of the RPM package is incorrect, an error may be reported during image creation.

### Operation Guide

The following uses the **livecd** image as an example.

1. Modify the configuration file **/opt/oemaker/config/aarch64/livecd/rpmlist** to specify the RPM software packages to be installed.

   ```shell
    sudo vi /opt/oemaker/config/aarch64/livecd/rpmlist
   ```

2. Ensure that the space of the temporary directory for running oemaker is greater than 60 GB.

   ```shell
    df -h
    Filesystem                            Size  Used Avail Use% Mounted on
    devtmpfs                              1.2G     0  1.2G   0% /dev
    tmpfs                                 1.5G     0  1.5G   0% /dev/shm
    tmpfs                                 1.5G   23M  1.5G   2% /run
    tmpfs                                 1.5G     0  1.5G   0% /sys/fs/cgroup
    /dev/mapper/openeuler_openeuler-root   69G  2.8G   63G   5% /
    /dev/sda2                             976M  114M  796M  13% /boot
    /dev/mapper/openeuler_openeuler-home   61G   21G   38G  35% /home
   ```

3. Perform the creation.

   Run the following commands:

   ```shell
   cd /opt/oemaker
   ./oemaker.sh -t livecd -p openEuler -v 22.03-LTS-SP4 -r '' -s "file:///home/oemaker_iso"
   ```

   The result is stored in the **/result** directory.

   ```shell
   ls /result/ -l
   total 549052
   -rw-r--r-- 1 root root     20244 Nov 17 15:24 openEuler-livecd-22.03-LTS-SP4-aarch64_binary.rpmlist
   -rw-r--r-- 1 root root 562188288 Nov 17 15:24 openEuler-livecd-22.03-LTS-SP4-aarch64.iso
   -rw-r--r-- 1 root root     15736 Nov 17 15:24 openEuler-livecd-22.03-LTS-SP4-aarch64_source.rpmlist
   ```

---

---
url: /zh/docs/22.03_LTS_SP4/tools/community_tools/oemaker/oemaker_user_guide.md
---
# oemaker 使用指南

## 简介

本文档介绍 openEuler 镜像制作工具的安装和使用方法，以指导用户更好的完成镜像制作。

## 软硬件要求

使用 openEuler 制作工具制作 ISO 所使用的机器需要满足如下软硬件要求：

* CPU 架构为 AArch64 或 X86\_64。
* 操作系统为 openEuler 22.03 LTS SP4。
* 建议预留 60 GB 以上的磁盘空间（用于运行镜像制作工具和存放 ISO 镜像）。

## 安装工具

此处以 openEuler 22.03 LTS SP4 版本的 AArch64 架构为例，介绍 ISO 镜像制作工具的安装操作。

1. 确认机器已安装操作系统 openEuler 22.03 LTS SP4（镜像制作工具的运行环境）。

   ```shell
    cat /etc/openEuler-release 
    openEuler release 22.03 LTS SP4
   ```

2. 下载对应架构的 ISO 镜像（必须是 everything 版本），并存放在任一目录（建议该目录磁盘空间大于 20 GB），此处假设存放在 /home/ 目录。

   AArch64 架构的镜像下载链接为：

   <https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/aarch64/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso>

   > \[!NOTE]说明
   >
   > x86\_64 架构的镜像下载链接为：
   >
   > <https://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/x86_64/openEuler-22.03-LTS-SP4-everything-x86_64-dvd.iso>

3. 创建文件 /etc/yum.repos.d/local.repo，配置对应 yum 源。配置内容参考如下，其中 baseurl 是用于挂载 ISO 镜像的目录：

   ```shell
   [local]
   name=local
   baseurl=file:///home/oemaker_iso
   gpgcheck=0
   enabled=1
   ```

4. 使用 root 权限，挂载光盘镜像到 /home/oemaker\_iso 目录（请与上述 repo 文件中配置的 baseurl 保持一致）作为 yum 源，参考命令如下：

   ```shell
   sudo mount -o loop /home/openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso /home/oemaker_iso
   ```

5. 使 yum 源生效：

   ```shell
   yum clean all
   yum makecache
   ```

6. 使用 root 权限，安装镜像制作工具：

   ```shell
   sudo yum install -y oemaker
   ```

7. 使用 root 权限，确认工具已安装成功：

   ```shell
    sudo oemaker -h
    Usage: oemaker [-h] [-t Type] [-p Product] [-v Version] [-r RELEASE] [-s REPOSITORY]
    
    optional arguments:
        -t Type        ISO Type, include standard debug source everything everything_debug everything_src livecd and netinst
        -p Product     Product Name, such as: openEuler
        -v Version     version identifier
        -r RELEASE     release information
        -s REPOSITORY  source dnf repository address link(may be listed multiple times)
        -h             show the help message and exit
   ```

## 制作镜像

此处介绍如何使用镜像制作工具基于 openEuler 光盘镜像制作新镜像的方法。

### 命令介绍

#### 命令格式

镜像制作工具通过 oemaker 命令执行功能。命令的使用格式为：

```shell
oemaker [ --help | -h ] [ -t <ISO type> ] [ -p <Product name> ] [ -v <Version number> ] [-r <Release information> ] [-s <Source dnf repository address link>]
```

#### 参数说明

| 参数                          | 是否必选 | 参数含义                                                    |
|-----------------------------| -------- |---------------------------------------------------------|
| --help \ -h                | 否       | 查询命令的帮助信息。                                              |
| -t \<ISO type>  | 是       | 镜像制作类型:standard、debug、 source、 everything、 everything\_debug、 everything\_src、 livecd 和 netinst。 |
| -p  \<Product name>             | 否       | 产品名称。                             |
| -v \<Version number>  | 否       | 产品版本号。                |
| -r \<Release information>         | 否       | 发布信息。                                                   |
| -s \<Source dnf repository address link>       | 否       | 软件安装源。                                                  |

### 软件包来源

镜像的 RPM 包来源有：

* 原有 ISO 镜像：该情况通过配置文件 rpmlist 指定需要安装的 RPM 软件包，配置格式为 "软件包名"，例如：kernel。

* 额外定制包：添加额外包到/home/oemaker\_iso中，通过原镜像normal.xml使用createrepo命令重新生成源。

  > \[!NOTE]说明
  >
  > * 制作镜像时，若无法找到配置文件中指定的 RPM 包，则镜像中不会添加该 RPM 包。
  > * 若 RPM 包的依赖有问题，则制作镜像时可能会报错。

### 操作指导

以livecd镜像制作为例

1. 修改配置文件 /opt/oemaker/config/aarch64/livecd/rpmlist，指定用户需要安装的 RPM 软件包。

   ```shell
    sudo vi /opt/oemaker/config/aarch64/livecd/rpmlist
   ```

2. 确定运行镜像制作工具的临时目录或根目录空间大于 60 GB 。

   ```shell
    df -h
    Filesystem                            Size  Used Avail Use% Mounted on
    devtmpfs                              1.2G     0  1.2G   0% /dev
    tmpfs                                 1.5G     0  1.5G   0% /dev/shm
    tmpfs                                 1.5G   23M  1.5G   2% /run
    tmpfs                                 1.5G     0  1.5G   0% /sys/fs/cgroup
    /dev/mapper/openeuler_openeuler-root   69G  2.8G   63G   5% /
    /dev/sda2                             976M  114M  796M  13% /boot
    /dev/mapper/openeuler_openeuler-home   61G   21G   38G  35% /home
   ```

3. 执行制作。

   **执行制作命令**，示例：

   ```shell
   cd /opt/oemaker
   ./oemaker.sh -t livecd -p openEuler -v 22.03-LTS-SP4 -r '' -s "file:///home/oemaker_iso"
   ```

   结果输出在/result/目录下：

   ```shell
   ls /result/ -l
   total 549052
   -rw-r--r-- 1 root root     20244 Nov 17 15:24 openEuler-livecd-22.03-LTS-SP4-aarch64_binary.rpmlist
   -rw-r--r-- 1 root root 562188288 Nov 17 15:24 openEuler-livecd-22.03-LTS-SP4-aarch64.iso
   -rw-r--r-- 1 root root     15736 Nov 17 15:24 openEuler-livecd-22.03-LTS-SP4-aarch64_source.rpmlist
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/tools/community_tools/oepkgs/oepkgs_image_source_configuration_and_usage.md
---
# oepkgs Image Source Configuration and Usage

## Configuration

Currently, the oepkgs image source provides more than 30,000 software packages. You can download the software packages by following the installation guide on the oepkgs search page. Alternatively, you can download the oepkgs-release package to the server and run the `yum search` command to search for the desired software package, and download and use it.

> \[!NOTE] **NOTE:**
>
> * The address of the oepkgs search page is <https://search.oepkgs.net/>. The method of downloading and installing software packages is the same as that of downloading and installing the oepkgs-release package. The following describes the detailed procedure.
> * Download the oepkgs-release package to the server. By default, the `priority` field in the .repo configuration file is used to change the priority of the oepkgs image source to the lowest to ensure that the image source provided by the OS is preferentially used. You can change the priority of the oepkgs image source as required.

1. Log in to the [oepkgs community](https://oepkgs.net/) website.
2. Click the **Search** tab. The software package search page is displayed.
3. Enter **oepkgs-release** in the search box and click **Search**. The download list of oepkgs-release packages of different versions is displayed.
   * oepkgs-release
4. Select the oepkgs-release package to be downloaded based on the openEuler version in the environment to be configured.
5. Click **View Details** to view details about the oepkgs-release package and configure the image source based on the installation guide.
   1. Add the source.

      ```shell
      dnf config-manager --add-repo https://repo.oepkgs.net/openeuler/rpm/openEuler-xxx/extras/noarch/
      ```

   2. Update the source index.

      ```shell
      dnf update
      ```

   3. Install the **oepkgs-release** software package.

      ```shell
      dnf install oepkgs-release
      ```

   4. Check the image source configured on the service.

      ```shell
      dnf repolist
      ```

> \[!NOTE] **NOTE:**
>
> * After the oepkgs-release package is installed, the oepkgs image source has been configured on the service. You can view the new .repo file in the **/etc/yum.repos.d/** directory.

## Usage Guide

If the oepkgs image source is configured in the environment by downloading the oepkgs-release package to the server, run the following command to download the software package from the oepkgs image source:

1. Query the software package.

   ```shell
   dnf search *** 
   ```

2. Install the software package.

   ```shell
   dnf install *** 
   ```

---

---
url: /en/docs/22.03_LTS_SP4/tools/community_tools/oepkgs/overview.md
---
# oepkgs User Guide

Open External Packages Service [oepkgs](https://oepkgs.net/en/) is a third-party community that provides software packages and container images for openEuler and other Linux distributions. Currently, the oepkgs [image source](https://repo.oepkgs.net/openEuler/rpm/) provides more than 30,000 software packages. The oepkgs community performs build tests and compatibility tests on software packages, and manages the lifecycle of the [source code repository](https://gitee.com/src-oepkgs) of software packages on oepkgs.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/community_tools/oepkgs/co_construction_and_future_of_oepkgs.md
---
# oepkgs共建与未来

## 个人如何贡献

![](./public_sys_resources/contrib-oepkgs.png)

1. 基于PR，创建仓库

   在 [oepkgs-management](https://gitee.com/oepkgs/oepkgs-management) 仓库提 PR，填写两个配置文件，PR 合入之后，创仓机器人 ci-robot 会在 [src-oepkgs](https://gitee.com/src-oepkgs) 下面自动创建仓库。

   > \[!NOTE]说明
   >
   > * oepkgs 仓库将软件包按照领域、类别划分，不同领域及类别的软件包由**不同的 sig 组**进行维护。
   > * 开源软件引入 **oepkgs 已有 sig 组**，提交申请创仓 PR，可基于对应 sig 组的 sig-info.yaml 文件进行修改，不强制要求开源软件引入 oepkgs 仓并新建 sig 组。

   oepkgs-management 仓库中的两个配置文件，以 nginx 为例，分别是:

   * sig-info.yaml
   * nginx.yaml

   sig-info.yaml 字段解释:

   | 字段 | 解释 | 是否必填 |
   |---|---|---|
   | name | sig 组名称，一般跟软件包领域相关 | √ |
   | description | 对该 sig 的描述 | √ |
   | mailing\_list | sig 组的订阅邮箱地址 | × |
   | meeting\_url | sig 组会议链接 | × |
   | maintainers | sig 组的管理者，负责该 sig 组下源码仓 PR 的检视与合入 | √ |
   | repositories | sig 组下面的源码仓 | √ |
   | committers | sig 组下面某些源码仓的 committers，负责对应仓库 PR 的检视与合入 | √ |

   nginx.yaml (仓库配置文件)字段解释：

   | 字段 | 解释 | 是否必填 |
   |---|---|---|
   | name | 包名（源码仓名） | √ |
   | description | 对软件包的描述 | √ |
   | upstream | 软件包上游仓库地址 | √ |
   | branches | 仓库分支，oepkgs 镜像源分支管理详见：[《oepkgs 分支管理文档》](https://atomgit.com/openeuler/oec-application/blob/master/doc/software-compatibility/oepkgs%E5%88%86%E6%94%AF%E7%AE%A1%E7%90%86.md) | √ |

2. 补充源码文件

   * 完成步骤一之后，5分钟内会生成 <https://gitee.com/src-oepkgs/nginx> 仓库，通过 PR 往这个仓库中补充源码文件，分别是可用于支撑生成 rpm 包的 nginx.spec 文件、软件包源码包 nginx-2.12.0.tar.bz2，详见：<https://gitee.com/src-oepkgs/nginx> 。

   > \[!NOTE]说明
   >
   > * 提了 PR 之后，在 5~30 分钟时间内，会进行 PR 门禁构建测试，PR 会评论出 PR 构建结果，建议在 **Build\_Result** 显示为 **SUCCESS** 之后合入 PR。
   > * 前面配置文件 oepkgs-management/sig/virtual/sig-info.yaml 中指定的 maintainer，可通过在 PR 下面评论 /lgtm 及 /approve 合入 PR。

3. 构建

   oepkgs 提供一个成熟的 CICD 体系，支撑软件包源码构建，二进制扫描，基本功能验证，保障软件仓库质量可靠及持续演进。

### 参考资料

* [如何贡献软件包到 oepkgs](https://atomgit.com/openeuler/oec-application/blob/master/doc/software-compatibility/rpm%E6%9E%84%E5%BB%BA%E4%BB%A5%E5%8F%8A%E5%BB%BA%E4%BB%93%E6%B5%81%E7%A8%8B.md)

## 未来规划

为更多普通用户和开发者提供服务

* 向用户和开发者开放服务，为更多的人参与及使用。反向促进 oepkgs 服务更加完善。

软件包 patch 管理

* 规划建设软件包 patch 管理系统，增强二进制包安全加固信息展示。检索平台将利用软件包 patch 管理，提供更加全面的二进制包信息，给用户更多检索可能。

持续建设

* 持续建设 openEuler 扩展仓，不断补全生态软件。协同 openEuler 官方仓，共同促进 openEuler 生态发展。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/community_tools/oepkgs/oepkgs_image_source_configuration_and_usage.md
---
# oepkgs镜像源配置与使用

## 配置方法

目前oepkgs 镜像源中已有3w+款软件包，用户可以通过 oepkgs 的检索页面，按照安装指引下载使用软件包，也可以通过将 oepkgs-release 包下载到服务器上，使用 yum search命令行方式查找软件包，并下载使用。

> \[!NOTE]说明
>
> * oepkgs 的检索页面地址：<https://search.oepkgs.net> ，所有软件包的下载安装方式与 oepkgs-release 包的下载安装方式一致，下面给出了详细的操作步骤。
> * 通过将 oepkgs-release 包下载到服务器上的方式，默认会通过 .repo 配置文件的 priority 字段调整 oepkgs 镜像源的优先级为最低，保证优先使用 os 自带的镜像源，可以通过调整 priority 的级别更改 oepkgs 的镜像源。

1. 登录[oepkgs社区](https://oepkgs.net/)网站。
2. 点击“检索”页签，进入软件包检索页面。
3. 在搜索框中输入“oepkgs-release”，点击搜索，显示 oepkgs 的不同版本源的下载列表。
   * oepkgs-release
4. 根据实际待配置环境的 openEuler 版本选择需要下载的 oepkgs 的发布包。
5. 点击“查看”按钮，查看oepkgs 发布包的详情，根据安装指引，完成镜像源配置：
   1. 添加源

      ```shell
      dnf config-manager --add-repo https://repo.oepkgs.net/openeuler/rpm/openEuler-xxx/extras/noarch/
      ```

   2. 更新源索引

      ```shell
      dnf update
      ```

   3. 安装 oepkgs-release 软件包

      ```shell
      dnf install oepkgs-release
      ```

   4. 查看服务上配置的镜像源

      ```shell
      dnf repolist
      ```

> \[!NOTE]说明
>
> * 安装上 oepkgs 的发布包，服务上就已经配置上了 oepkgs 镜像源，用户可以在 /etc/yum.repos.d/下面看见新增的后缀为 .repo 的文件。

## 使用指导

如是通过将 oepkgs-release 包下载到服务器上的方式将 oepkgs 镜像源配置到环境中，通过如下命令下载使用 oepkgs 镜像源中软件包：

1. 查询软件包

   ```shell
   dnf search *** 
   ```

2. 安装软件包

   ```shell
   dnf install *** 
   ```

---

---
url: /en/docs/22.03_LTS_SP4/cloud/hybrid_deployment/oncn_bwm/overview.md
---
# oncn-bwm User Guide

## Introduction

With the rapid development of technologies such as cloud computing, big data, artificial intelligence, 5G, and the Internet of Things (IoT), data center construction becomes more and more important. However, the server resource utilization of the data center is very low, resulting in a huge waste of resources. To improve the utilization of server resources, oncn-bwm emerges.

oncn-bwm is a pod bandwidth management tool applicable to hybrid deployment of offline services. It properly schedules network resources for nodes based on QoS levels to ensure online service experience and greatly improve the overall network bandwidth utilization of nodes.

The oncn-bwm tool supports the following functions:

* Enabling/Disabling/Querying pod bandwidth management
* Setting the pod network priority
* Setting the offline service bandwidth range and online service waterline
* Querying internal statistics

## Installation

### Environmental Requirements

* Operating system: openEuler 22.03 LTS SP4 with the Yum repository of openEuler 22.03 LTS SP4

### Installation Procedure

Run the following command:

```shell
yum install oncn-bwm
```

## How to Use

The oncn-bwm tool provides the `bwmcli` command line tool to enable pod bandwidth management or perform related configurations. The overall format of the `bwmcli` command is as follows:

**bwmcli**  < option(s) >

> Note:
>
> The root permission is required for running the `bwmcli` command.
>
> Pod bandwidth management is supported only in the outbound direction of a node (packets are sent from the node to other nodes).
>
> Pod bandwidth management cannot be enabled for NICs for which tc qdisc rules have been configured.
>
> Upgrading the oncn-bwm package does not affect the enabling status before the upgrade. Uninstalling the oncn-bwm package disables pod bandwidth management for all NICs.

### Command Interfaces

#### Pod Bandwidth Management

##### Commands and Functions

| Command Format                   | Function                                                        |
| --------------------------- | ------------------------------------------------------------ |
| **bwmcli -e** \<NIC name >   | Enables pod bandwidth management for a specified NIC.|
| **bwmcli -d** \<NIC name >   | Disables pod bandwidth management for a specified NIC.|
| **bwmcli -p devs**          | Queries pod bandwidth management of all NICs on a node.|

> Note:
>
> * If no NIC name is specified, the preceding commands take effect for all NICs on a node.
>
> * Enable pod bandwidth management before running other `bwmcli` commands.

##### Examples

* Enable pod bandwidth management for NICs eth0 and eth1.

  ```shell
  # bwmcli -e eth0 -e eth1
  enable eth0 success
  enable eth1 success
  ```

* Disable pod bandwidth management for NICs eth0 and eth1.

  ```shell
  # bwmcli -d eth0 -d eth1
  disable eth0 success
  disable eth1 success
  ```

* Query pod bandwidth management of all NICs on a node.

  ```shell
  # bwmcli -p devs
  eth0            : enabled
  eth1            : disabled
  eth2            : disabled
  docker0         : disabled
  lo              : disabled
  ```

#### Pod Network Priority

##### Commands and Functions

| Command Format                                                    | Function                                                        |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| **bwmcli -s** *path* \<prio>                                  | Sets the network priority of a pod. *path* indicates the cgroup path corresponding to the pod, and *prio* indicates the priority. The value of *path* can be a relative path or an absolute path. The default value of *prio* is **0**. The optional values are **0** and **-1**. The value **0** indicates online services, and the value **-1** indicates offline services.|
| **bwmcli -p** *path*                                         | Queries the network priority of a pod.                                         |

> Note:
>
> Online and offline network priorities are supported. The oncn-bwm tool controls the bandwidth of pods in real time based on the network priority. The specific policy is as follows: For online pods, the bandwidth is not limited. For offline pods, the bandwidth is limited within the offline bandwidth range.

##### Examples

* Set the priority of the pod whose cgroup path is **/sys/fs/cgroup/net\_cls/test\_online** to **0**.

  ```shell
  # bwmcli -s /sys/fs/cgroup/net_cls/test_online 0
  set prio success
  ```

* Query the priority of the pod whose cgroup path is **/sys/fs/cgroup/net\_cls/test\_online**.

  ```shell
  # bwmcli -p /sys/fs/cgroup/net_cls/test_online
  0
  ```

#### Offline Service Bandwidth Range

| Command Format                          | Function                                                        |
| ---------------------------------- | ------------------------------------------------------------ |
| **bwmcli -s bandwidth** \<low,high> | Sets the offline bandwidth for a host or VM. **low** indicates the minimum bandwidth, and **high** indicates the maximum bandwidth. The unit is KB, MB, or GB, and the value range is \[1 MB, 9999 GB].|
| **bwmcli -p bandwidth**            | Queries the offline bandwidth of a host or VM.                         |

> Note:
>
> * All NICs with pod bandwidth management enabled on a host are considered as a whole, that is, the configured online service waterline and offline service bandwidth range are shared.
>
> * The pod bandwidth configured using `bwmcli` takes effect for all offline services on a node. The total bandwidth of all offline services cannot exceed the bandwidth range configured for the offline services. There is no network bandwidth limit for online services.
>
> * The offline service bandwidth range and online service waterline are used together to limit the offline service bandwidth. When the online service bandwidth is lower than the configured waterline, the offline services can use the configured maximum bandwidth. When the online service bandwidth is higher than the configured waterline, the offline services can use the configured minimum bandwidth.

##### Examples

* Set the offline bandwidth to 30 Mbit/s to 100 Mbit/s.

  ```shell
  # bwmcli -s bandwidth 30mb,100mb
  set bandwidth success
  ```

* Query the offline bandwidth range.

  ```shell
  # bwmcli -p bandwidth
  bandwidth is 31457280(B),104857600(B)
  ```

#### Online Service Waterline

##### Commands and Functions

| Command Format                                      | Function                                                        |
| ---------------------------------------------- | ------------------------------------------------------------ |
| **bwmcli -s waterline** \<val>                  | Sets the online service waterline for a host or VM. *val* indicates the waterline value. The unit is KB, MB, or GB, and the value range is \[20 MB, 9999 GB].|
| **bwmcli -p waterline**                        | Queries the online service waterline of a host or VM.                         |

> Note:
>
> * When the total bandwidth of all online services on a host is higher than the waterline, the bandwidth that can be used by offline services is limited. When the total bandwidth of all online services on a host is lower than the waterline, the bandwidth that can be used by offline services is increased.
> * The system determines whether the total bandwidth of online services exceeds or is lower than the configured waterline every 10 ms. Then the system determines the bandwidth limit for offline services based on whether the online bandwidth collected within each 10 ms is higher than the waterline.

##### Examples

* Set the online service waterline to 20 MB.

  ```shell
  # bwmcli -s waterline 20mb
  set waterline success
  ```

* Query the online service waterline.

  ```shell
  # bwmcli -p waterline
  waterline is 20971520(B)
  ```

#### Statistics

##### Commands and Functions

| Command Format           | Function              |
| ------------------- | ------------------ |
| **bwmcli -p stats** | Queries internal statistics.|

> Note:
>
> * **offline\_target\_bandwidth**: target bandwidth for offline services.
>
> * **online\_pkts**: total number of online service packets after pod bandwidth management is enabled.
>
> * **offline\_pkts**: total number of offline service packets after pod bandwidth management is enabled.
>
> * **online\_rate**: current online service rate.
>
> * **offline\_rate**: current offline service rate.

##### Examples

Query internal statistics.

```shell
# bwmcli -p stats
offline_target_bandwidth: 2097152
online_pkts: 2949775
offline_pkts: 0
online_rate: 602
offline_rate: 0
```

### Typical Use Case

To configure pod bandwidth management on a node, perform the following steps:

```shell
bwmcli -p devs #Query the pod bandwidth management status of the NICs in the system.
bwmcli -e eth0 # Enable pod bandwidth management for the eth0 NIC.
bwmcli -s /sys/fs/cgroup/net_cls/online 0 # Set the network priority of the online service pod to 0
bwmcli -s /sys/fs/cgroup/net_cls/offline -1 # Set the network priority of the offline service pod to -1.
bwmcli -s bandwidth 20mb,1gb # Set the bandwidth range for offline services.
bwmcli -s waterline 30mb # Set the waterline for online services.
```

### Constraints

1. Only the **root** user is allowed to run the bwmcli command.
2. Currently, this feature supports only two network QoS priorities: offline and online.
3. If the tc qdisc rules have been configured for a NIC, the network QoS function will fail to be enabled for the NIC.
4. After a NIC is removed and then inserted, the original QoS rules will be lost. In this case, you need to manually reconfigure the network QoS function.
5. When you run one command to enable or disable multiple NICs at the same time, if any NIC fails to be operated, operations on subsequent NICs will be stopped.
6. When SELinux is enabled in the environment, if the SELinux policy is not configured for the bwmcli program, some commands (such as setting or querying the waterline, bandwidth, and priority) may fail. You can confirm the failure in SELinux logs. To solve this problem, disable SELinux or configure the SELinux policy for the bwmcli program.
7. Upgrading the software package does not change the enabling status before the upgrade. Uninstalling the software package disables the function for all devices.
8. The NIC name can contain only digits, letters, hyphens (-), and underscores (\_). NICs whose names contain other characters cannot be identified.
9. In actual scenarios, bandwidth limiting may cause protocol stack memory overstock. In this case, backpressure depends on transport-layer protocols. For protocols that do not have backpressure mechanisms, such as UDP, packet loss, ENOBUFS, and rate limiting deviation may occur.

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/hybrid_deployment/oncn_bwm/overview.md
---
# oncn-bwm用户指南

## 简介

随着云计算、大数据、人工智能、5G、物联网等技术的迅速发展，数据中心的建设越来越重要。然而，数据中心的服务器资源利用率很低，造成了巨大的资源浪费。为了提高服务器资源利用率，oncn-bwm应运而生。

oncn-bwm是一款适用于在、离线业务混合部署场景的Pod带宽管理工具，它会根据QoS分级对节点内的网络资源进行合理调度，保障在线业务服务体验的同时，大幅提升节点整体的网络带宽利用率。

oncn-bwm工具支持如下功能：

* 使能/去除/查询Pod带宽管理
* 设置Pod网络优先级
* 设置离线业务带宽范围和在线业务水线
* 内部统计信息查询

## 安装

### 环境要求

操作系统为openEuler 22.03-LTS-SP4，且配置了22.03-LTS-SP4的yum源。

### 安装步骤

使用以下命令直接安装：

```shell
yum install oncn-bwm
```

## 使用方法

oncn-bwm工具提供了`bwmcli`命令行工具来使能Pod带宽管理或进行相关配置。`bwmcli`命令的整体格式如下：

**bwmcli**  < option(s) >

> 说明：
>
> 使用`bwmcli`命令需要root权限。
>
> 仅支持节点上出方向（报文从节点内发往其他节点）的Pod带宽管理。
>
> 已设置tc qdisc规则的网卡，不支持使能Pod带宽管理。
>
> 升级oncn-bwm包不会影响升级前的使能状态；卸载oncn-bwm包会关闭所有网卡的Pod带宽管理。

### 命令接口

#### Pod带宽管理

##### 命令和功能

| 命令格式                    | 功能                                                         |
| --------------------------- | ------------------------------------------------------------ |
| **bwmcli -e** <网卡名称>    | 使能指定网卡的Pod带宽管理。 |
| **bwmcli -d** <网卡名称>    | 去除指定网卡的Pod带宽管理。 |
| **bwmcli -p devs**          | 查询节点所有网卡的Pod带宽管理。 |

> 说明：
>
> * 不指定网卡名时，上述命令会对节点上的所有的网卡生效。
>
> * 执行 `bwmcli` 其他命令前需要开启Pod带宽管理。

##### 使用示例

* 使能网卡eth0和eth1的Pod带宽管理

  ```shell
  # bwmcli -e eth0 -e eth1
  enable eth0 success
  enable eth1 success
  ```

* 取消网卡eth0和eth1的Pod带宽管理

  ```shell
  # bwmcli -d eth0 -d eth1
  disable eth0 success
  disable eth1 success
  ```

* 查询节点所有网卡的Pod带宽管理

  ```shell
  # bwmcli -p devs
  eth0            : enabled
  eth1            : disabled
  eth2            : disabled
  docker0         : disabled
  lo              : disabled
  ```

#### Pod网络优先级

##### 命令和功能

| 命令格式                                                     | 功能                                                         |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| **bwmcli -s** *path* *\<prio>*                                | 设置Pod网络优先级。其中*path*为Pod对应的cgroup路径，*prio*为优先级。*path*取相对路径或者绝对路径均可。 *prio*默认值为0，可选值为0和-1，0标识为在线业务，-1标识为离线业务。 |
| **bwmcli -p** *path*                                         | 查询Pod网络优先级。                                          |

> 说明：
>
> 支持在线或离线两种网络优先级，oncn-bwm工具会按照网络优先级实时控制Pod的带宽，具体策略为：对于在线类型的Pod，不会限制其带宽；对于离线类型的Pod，会将其带宽限制在离线带宽范围内。

##### 使用示例

* 设置cgroup路径为/sys/fs/cgroup/net\_cls/test\_online的Pod的优先级为0

  ```shell
  # bwmcli -s /sys/fs/cgroup/net_cls/test_online 0
  set prio success
  ```

* 查询cgroup路径为/sys/fs/cgroup/net\_cls/test\_online的Pod的优先级

  ```shell
  # bwmcli -p /sys/fs/cgroup/net_cls/test_online
  0
  ```

#### 离线业务带宽范围

| 命令格式                             | 功能                                                         |
| ------------------------------------ | ------------------------------------------------------------ |
| **bwmcli -s bandwidth** *\<low,high>* | 设置一个主机/虚拟机的离线带宽。其中*low*表示最低带宽，*high*表示最高带宽，其单位可取值为kb/mb/gb，有效范围为\[1mb, 9999gb]。|
| **bwmcli -p bandwidth**              | 查询设置一个主机/虚拟机的离线带宽。                          |

> 说明：
>
> * 一个主机上所有使能Pod带宽管理的网卡在实现内部被当成一个整体看待，也就是共享设置的在线业务水线和离线业务带宽范围。
>
> * 使用 `bwmcli` 设置Pod带宽对此节点上所有离线业务生效，所有离线业务的总带宽不能超过离线业务带宽范围。在线业务没有网络带宽限制。
>
> * 离线业务带宽范围与在线业务水线共同完成离线业务带宽限制，当在线业务带宽低于设置的水线时：离线业务允许使用设置的最高带宽；当在线业务带宽高于设置的水线时，离线业务允许使用设置的最低带宽。

##### 使用示例

* 设置离线带宽范围在30mb到100mb

  ```shell
  # bwmcli -s bandwidth 30mb,100mb
  set bandwidth success
  ```

* 查询离线带宽范围

  ```shell
  # bwmcli -p bandwidth
  bandwidth is 31457280(B),104857600(B)
  ```

#### 在线业务水线

##### 命令和功能

| 命令格式                                       | 功能                                                         |
| ---------------------------------------------- | ------------------------------------------------------------ |
| **bwmcli -s waterline** *\<val>*                | 设置一个主机/虚拟机的在线业务水线，其中*val*为水线值，单位可取值为kb/mb/gb ，有效范围为\[20mb, 9999gb]。 |
| **bwmcli -p waterline**                        | 查询一个主机/虚拟机的在线业务水线。                          |

> \[!NOTE]说明
>
> * 当一个主机上所有在线业务的总带宽高于水线时，会限制离线业务可以使用的带宽，反之当一个主机上所有在线业务的总带宽低于水线时，会提高离线业务可以使用的带宽。
> * 判断在线业务的总带宽是否超过/低于设置的水线的时机：每10ms判断一次，根据每个10ms内统计的在线带宽是否高于水线来决定对离线业务采用的带宽限制。

##### 使用示例

* 设置在线业务水线为20mb

  ```shell
  # bwmcli -s waterline 20mb
  set waterline success
  ```

* 查询在线业务水线

  ```shell
  # bwmcli -p waterline
  waterline is 20971520(B)
  ```

#### 统计信息

##### 命令和功能

| 命令格式            | 功能               |
| ------------------- | ------------------ |
| **bwmcli -p stats** | 查询内部统计信息。 |

> \[!NOTE]说明
>
> * offline\_target\_bandwidth 表示离线业务目标带宽
>
> * online\_pkts 表示开启Pod带宽管理后在线业务总包数
>
> * offline\_pkts 表示开启Pod带宽管理后离线业务总包数
>
> * online\_rate 表示当前在线业务速率
>
> * offline\_rate 表示当前离线业务速率

##### 使用示例

查询内部统计信息

```shell
# bwmcli -p stats
offline_target_bandwidth: 2097152
online_pkts: 2949775
offline_pkts: 0
online_rate: 602
offline_rate: 0
```

### 典型使用案例

完整配置一个节点上的Pod带宽管理可以按照如下步骤顺序操作：

```shell
bwmcli -p devs # 查询系统当前网卡Pod带宽管理状态
bwmcli -e eth0 # 使能eth0的网卡Pod带宽管理
bwmcli -s /sys/fs/cgroup/net_cls/online 0 # 设置在线业务Pod的网络优先级为0
bwmcli -s /sys/fs/cgroup/net_cls/offline -1 # 设置离线业务Pod的网络优先级为-1
bwmcli -s bandwidth 20mb,1gb # 配置离线业务带宽范围
bwmcli -s waterline 30mb # 配置在线业务的水线
```

### 约束限制

1. 仅允许root用户执行bwmcli命令行。
2. 本特性当前仅支持设置两档网络QoS优先级：离线和在线。
3. 某个网卡上已经设置过tc qdisc规则的情况下，对此网卡使能网络QoS功能会失败。
4. 网卡被插拔重新恢复后，原来设置的QoS规则会丢失，需要手动重新配置网络QoS功能。
5. 用一条命令同时使能/去使能多张网卡的时候，如果中间有网卡执行失败，则终止对后面网卡的执行。
6. 环境上开启SELinux的情况下，未对bwmcli程序配置SELinux策略可能导致部分命令（例如水线，带宽，优先级的设置或查询）失败，可在SELinux日志中确认。此情况可以通过关闭SELinux或对bwmcli程序配置SELinux策略解决。
7. 升级包不会影响升级前的使能状态，卸载包会关闭对所有设备的使能。
8. 网卡名仅支持数字、英文字母、中划线“-” 和下划线“\_”这四类字符类型，包含其他字符类型的网卡不被识别。
9. 实际使用过程中，带宽限速有可能造成协议栈内存积压，此时依赖传输层协议自行反压，对于udp等无反压机制的协议场景，可能出现丢包、ENOBUFS、限速有偏差等问题。

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/aops/quick_deployment_of_aops.md
---
# One-Click Deployment of A-Ops

One-click deployment of A-Ops is based on Docker and docker-compose to simply deployment and implement one-click start and stop.

## Environment Requirements

You are advised to use two or more machines with 8 GB or more memory running openEuler 22.03 LTS SP1 or later. Assume that the machines are host A and B.

* MySQL, Elasticsearch, Kafka, Redis, and Prometheus are deployed on host A, which provides data services.
* The A-Ops server and A-Ops frontend are deployed on host B to provide service functions as well as display and operations.

| Host | IP Address      | Services                                     |
| -------- | ----------- | -------------------------------------------- |
| Host A    | 192.168.1.1 | MySQL, Elasticsearch, Redis, Kafka, Prometheus   |
| Host B    | 192.168.1.2 | aops-zeus, aops-diana, aops-apollo, aops-hermes |

## Environment Configuration

### Disabling the Firewall on Host A

```shell
systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld
```

### Installing Docker and docker-compose

```shell
dnf install docker docker-compose
# Set Docker to start upon system startup.
systemctl enable docker
```

### Installing aops-vulcanus and aops-tools

```shell
dnf install aops-vulcanus aops-tools
```

### Perform One-Click Deployment

* Execute the deployment script.

```shell
cd /opt/aops/scripts/deploy/container
# Execute run.sh.
bash run.sh
```

> Enter the interactive CLI.
>
> ```console
> 1. Build the docker container (build).
> 2. Start the container orchestration service (start-service/start-env).
> 3. Stop all container services (stop-service/stop-env).
> run.sh: line 74: read: `Enter to exit the operation (Q/q).': not a valid identifier
> Select an operation procedure to continue:
>
> ```
>
> **build**: Deployment of basic services (such as MySQL and Kafka) does not need the build operation.
>
> **start-service**: Start the service and frontend of A-Ops.
>
> **start-env**: Start basic service including MySQL, Redis, and Kafka.
>
> **stop-service**: Stop the service and frontend of A-Ops.
>
> **stop-env**: Stop basic services. The data is retained.
>
> **Q/q**: Exit the interactive CLI.

* Deploy the A-Ops server.

```shell
# Execute the deployment script on host B.
cd /opt/aops/scripts/deploy/container
bash run.sh
# Run start-service in the interactive CLI.
```

* Modify service configuration files.

> **Note: If the A-Ops service and basic services are deployed on the same host, you do not need to modify the configuration files. In this example, set the IP addresses for connecting to basic services to the IP address of host A in all configuration files.**
>
> **Password-free mode is used in the default MySQL connection string. The MySQL basic service is configured with the default password "123456". Change the configurations as required.**

```shell
# Modify the IP addresses for connecting to mysql, elasticsearch, kafka, and redis in apollo.ini, diana.ini, and zeus.ini.
cd /etc/aops/
```

* **FAQ**

**1. The Elasticsearch basic service cannot be started normally.**

Check whether the permission on the **/opt/es** directory is **777**. You can run `chmod -R 777 /opt/es` to modify the permission.

**2. The Prometheus basic service cannot be started normally.**

Check whether the configuration file **prometheus.yml** exists in **/etc/prometheus**. If not, create it.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/ai/euler-copilot-framework/witty_assistant/witty_shell/opencode_guide/agent_introduce.md
---
# OpenCode Agent 介绍

本节介绍 OpenCode 的 Agent / Skill / MCP 体系。这些能力由 OpenCode 提供，Witty CLI 通过托管配置注册并复用。

## Agent 体系

### Witty 内置 Agent：`witty-builtin-agent`

`witty-builtin-agent` 是 Witty 随 `witty-builtin-agent` 子包提供的默认 primary agent，定义在 `/usr/share/witty/opencode/config.d/witty-builtin-agent.json`（由 loader 合入 `/etc/opencode/opencode.json`）。

| 字段 | 值 |
| ---- | -- |
| `mode` | `primary` |
| 描述 | Witty Assistant，提供知识问答、命令查询、故障诊断、方案规划与可视化报告 |
| 颜色 | `#5F87FF` |
| 提示词 | `{file:/usr/share/witty/opencode/agents/witty-builtin-agent/witty-builtin-agent.md}` |

### OpenCode 内置 Agent

除 `witty-builtin-agent` 外，OpenCode 还提供多个内置 agent（`opencode agent list` 实测）：

| Agent | 模式 | 说明 |
| ---- | ---- | ---- |
| `build` | primary | 默认主 agent |
| `plan` | primary | 规划型 agent |
| `summary` / `title` | primary | 摘要 / 标题生成 |
| `explore` / `general` / `baize` / `dayu` / `kuafu` / `nuwa` | subagent | 各类子 agent |
| `fuxi` | all | 主 agent + 子 agent |

### 诊断能力扩展包：`witty-diagnosis-agent`

`witty-diagnosis-agent` 是一个**可选的诊断能力扩展包**，它并不注册为独立 agent，而是通过 `config.d/witty-diagnosis-agent.json` 声明 Plugin，并在 `/usr/share/witty/opencode/skills/witty-diagnosis-agent/` 下安装大量诊断类 Skill。安装后，这些 Skill 与插件上下文会注入到当前 agent。

**使用方式**：

```bash
sudo dnf install witty-diagnosis-agent
```

安装后 OpenCode / Witty 会自动加载新增 Skill；若需让已运行的 server 生效，可执行 `witty server restart`。

## Skill 体系

Skill 是 OpenCode 的场景化能力包，通过 `skills.paths`（Witty 托管为 `/usr/share/witty/opencode/skills`）加载。

### Witty 内置 Skill（`witty-builtin-agent` 子包）

| Skill | 用途 |
| ----- | ---- |
| `manpage-skill` | 查询 Linux / openEuler 命令用法、选项与示例 |
| `log-anomaly-detector` | 分析系统日志与性能指标，进行故障初步定位 |
| `html-report-generator` | 将诊断报告与方案渲染为网页 |
| `brainstorm-beagle` | 当目标不明确时生成完整可执行的方案 |
| `plantuml-skill` | 绘制流程图、时序图与架构图 |

### 经验技能管理：`experience-skill`

`experience-skill` 是 Witty 的核心知识引擎，支持 **Skill**（工作流程技能）与 **Wiki**（资料文档提炼）两类经验的创建、评估、检索、合并与优化。其 CLI 提供 `sync`、`add-experiences`、`list-experiences`、`search-experiences`、`delete-*`、`web` 等子命令（`web` 为经验管理界面，非 Witty 的 Web 前端）。

### 诊断 Skill（`witty-diagnosis-agent` 子包）

安装 `witty-diagnosis-agent` 后包含数十个诊断 Skill，例如：

* `linux-oom-analyzer`、`memory-leak-diagnosis`、`process-hang-diagnosis`
* `network-diagnosis`、`dns-resolution-diagnosis`、`tls-certificate-diagnosis`
* `disk-health-diagnosis`、`coredump_diagnose`、`kernel-io-uring-diagnosis`
* `root-cause-analysis`、`root-cause-localization`、`fault-rca-report-generation` 等

## MCP 体系

### `openeuler_portal`

`openeuler_portal` 是 Witty 内置的本地 MCP，通过 `npx -y openeuler-portal-mcp` 提供 openEuler 官网数据查询（兼容性、CVE、软件包、文档、SIG、Issue/PR 等）。其 token（`OPENEULER_TOKEN`、`GITCODE_TOKEN`、`FORUM_TOKEN`）通过环境变量注入。

```json
"mcp": {
  "openeuler_portal": {
    "type": "local",
    "command": ["npx", "-y", "openeuler-portal-mcp"],
    "environment": {
      "OPENEULER_TOKEN": "${OPENEULER_TOKEN}",
      "GITCODE_TOKEN": "${GITCODE_TOKEN}",
      "FORUM_TOKEN": "${FORUM_TOKEN}"
    },
    "enabled": true,
    "timeout": 30000
  }
}
```

## Permission 策略

`witty-builtin-agent` 默认 permission 策略（节选）：

```json
"permission": {
  "*": "ask",
  "read": { "*": "allow", "*.env": "ask" },
  "edit": "allow",
  "glob": "allow",
  "grep": "allow",
  "webfetch": "allow",
  "websearch": "allow",
  "skill": "allow",
  "task": "allow",
  "openeuler_portal_*": "allow",
  "bash": "allow",
  "external_directory": { "*": "ask", "/tmp/**": "allow", "/usr/share/witty/**": "allow" }
}
```

* **只读操作**（查系统信息、读文件、检查状态）默认允许；
* **危险操作**（改配置、装软件、重启服务、删文件）默认 `ask`，需用户确认；
* 敏感数据默认本地处理，未经授权不上传。

> `permission` 各字段语义以 [opencode.ai/docs](https://opencode.ai/docs/) 为准。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/ai/euler-copilot-framework/witty_assistant/witty_shell/social_software_guide/bridge_introduce.md
---
# OpenCode Bridge 桥接器安装教程

## 一、产品概述

OpenCode Bridge 是一款**通用型智能桥接服务**，可快速打通各类主流通讯平台，实现消息互通、指令转发与统一管理。

### 支持平台

飞书、个人微信、企业微信、QQ、钉钉、Discord、WhatsApp、Telegram

### 核心能力

**特色增强能力**

* Cron 定时计划任务
* 主动心跳保活
* 可视化后台配置

**原生基础能力**

* 权限闭环管理
* 问答卡片推送
* 对话上下文隔离
* 文件发送
* 会话绑定续连
* 权限透传、指令透传

***

## 二、一键安装部署教程

> 环境要求：已安装 `Git`、`Node.js`（推荐 v16+）
> 全流程采用自动化脚本，无需手动配置复杂参数

### 1. 拉取代码并安装依赖

```sh
# 克隆项目源码
git clone https://github.com/HNGM-HP/opencode-bridge.git
cd opencode-bridge

# 【推荐】使用国内镜像加速安装依赖
npm install --include=dev --registry=https://registry.npmmirror.com

# 赋予部署脚本执行权限
chmod +x ./scripts/deploy.sh
```

**可选优化（大幅加快安装速度）**

跳过不必要的二进制文件下载：

```sh
export ELECTRON_SKIP_BINARY_DOWNLOAD=1
export PUPPETEER_SKIP_DOWNLOAD=1
```

### 2. 执行自动化部署

```sh
./scripts/deploy.sh
```

脚本会自动完成环境检查、配置初始化、端口分配等操作。

### 3. 启动桥接器服务

```sh
./scripts/start.sh
```

启动成功后，服务将在后台持续运行。

### 4. 访问可视化配置后台

* **本地访问**：`http://localhost:4098`
* **远程访问**：`http://[你的服务器IP]:4098`

### 5. 部署成功验证

打开地址后，出现**桥接器可视化配置页面**，即代表安装启动完成。

![桥接器配置页面](./pictures/桥接器配置页面.png)

### 6. 基础配置（必做）

> 说明：使用新版 Witty CLI 时，`witty` 会自动管理 `opencode serve`（默认监听 `127.0.0.1:4096`）。桥接器既可对接该服务，也可通过「配置 OpenCode 启动命令」让桥接器自行拉起 OpenCode。

1. 配置 OpenCode 服务 IP 与端口
   ![配置openCode的ip和端口](./pictures/配置openCode的ip和端口.png)

2. 配置 OpenCode 启动命令
   ![配置openCode的启动命令](./pictures/配置openCode的启动命令.png)

配置保存后，桥接器即可正常对接服务运行。

### 7. 特殊配置

进入项目目录并编辑环境变量配置文件：

```sh
cd opencode-bridge
vim .env
```

在 `.env` 文件中添加如下配置并保存：

```sh
OPENCODE_AUTO_START=false
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/ai/euler-copilot-framework/witty_assistant/witty_shell/opencode_guide/user_guide.md
---
# OpenCode 使用说明

本节介绍在终端中**直接使用 OpenCode** 的常用方式。OpenCode 是驱动 Witty CLI 的智能体框架，提供了完整的 CLI 与无头 Server 能力。

> 一般用户请使用 Witty CLI 的 Shell 直输或 REPL（见[使用智能助手 CLI](../user_guide/user_guide.md)），无需直接调用 OpenCode 命令。

## 常用命令

| 命令 | 用途 |
| ---- | ---- |
| `opencode` | 在当前目录启动交互式 TUI |
| `opencode serve` | 启动无头 opencode server（HTTP + SSE） |
| `opencode run [message..]` | 用单条消息运行 opencode（非交互） |
| `opencode agent list` | 列出所有可用 agent，含 primary / subagent 标记 |
| `opencode agent create` | 创建新 agent |
| `opencode providers` | 管理 AI 提供商与凭据（别名 `opencode auth`） |
| `opencode models [provider]` | 列出可用模型 |
| `opencode mcp` | 管理 MCP 服务器 |
| `opencode session` | 会话管理 |
| `opencode plugin <module>` | 安装插件并更新配置 |
| `opencode upgrade` | 升级到最新版本 |

## 启动交互式 TUI

```bash
opencode
```

TUI 默认加载 `/etc/opencode/opencode.json` 中的 agent / skill / mcp 配置。在 TUI 中可通过 `/agent` 等命令切换 agent；具体可用的 `/` 命令以 OpenCode 官方文档为准。

## 无头 Server

```bash
opencode serve --port 4096
```

`opencode serve` 运行后提供以下端点（Witty 即依赖这些端点）：

* `GET /global/health` — 健康检查
* `GET /session`、`POST /session` — 会话管理
* `POST /session/{id}/message` — 发送消息
* `GET /event?directory=<cwd>` — SSE 事件流（所有会话共享，需按 sessionID 过滤）
* `GET /agent`、`GET /provider`、`GET /mcp` — 元信息
* `GET /doc` — OpenAPI 文档（唯一事实来源）

## 查看与使用 Agent

```bash
opencode agent list
```

输出示例（节选，openEuler 24.03 实测）：

```text
build (primary)
explore (subagent)
general (subagent)
plan (primary)
summary (primary)
title (primary)
baize (subagent)
witty-builtin-agent (primary)
```

其中 `witty-builtin-agent` 为 Witty 内置的默认 primary agent；`build` / `plan` 等为 OpenCode 内置 agent。

## Agent 配置

OpenCode 的 agent 通过 `/etc/opencode/opencode.json`（或用户级 `~/.config/opencode/opencode.json`、项目级 `opencode.json`）配置。典型字段：

```json
{
  "$schema": "https://opencode.ai/config.json",
  "agent": {
    "witty-builtin-agent": {
      "description": "Witty Assistant",
      "mode": "primary",
      "prompt": "{file:/usr/share/witty/opencode/agents/witty-builtin-agent/witty-builtin-agent.md}",
      "color": "#5F87FF",
      "permission": {
        "*": "ask",
        "read": { "*": "allow" },
        "bash": "allow"
      }
    }
  }
}
```

* `mode`：`primary`（默认主 agent）、`subagent`（子 agent）、`all`（主 agent + 子 agent）；
* `prompt`：角色提示词，支持 `{file:...}` 引用本地文件；
* `permission`：工具 / 文件 / shell 授权策略，行为 `allow` / `ask` / `deny`。

> 详细字段语义以 [opencode.ai/docs](https://opencode.ai/docs/) 为准。修改 `mode`、`permission` 等字段前务必查阅官方文档，避免凭猜测修改外部配置。

## Skill 使用

Skill 通过 `skills.paths` 指定搜索目录。Witty 托管时固定为 `/usr/share/witty/opencode/skills`。具体加载方式与命令以 OpenCode 官方文档为准。

## MCP 使用

MCP 通过 `mcp` 字段配置，支持 `local`（本地命令）与 `remote`（SSE/HTTP）两种类型。Witty 内置的 `openeuler_portal` 即通过本地 `npx -y openeuler-portal-mcp` 提供 openEuler 官网数据查询。

```bash
opencode mcp
```

## 权限交互

当 Agent 请求调用工具、读取文件或执行 shell 命令时，OpenCode 会根据 `permission` 策略发起 `permission.asked` 事件；在 TUI 中会弹出确认请求。Witty 在终端中接管这一交互，通过 `witty` 的授权提示完成回复。

## 相关文档

* [OpenCode 部署](./deployment.md)
* [OpenCode Agent 介绍](./agent_introduce.md)

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/ai/euler-copilot-framework/witty_assistant/witty_shell/opencode_guide/deployment.md
---
# OpenCode 部署（直接使用）

本手册介绍在 openEuler 上**直接部署 / 使用 OpenCode** 的步骤。通常安装 `witty` 后即会自动完成 OpenCode 的部署与配置，本节适用于需要绕过 Witty CLI、自行管理 OpenCode 的进阶场景。

> 一般用户请直接使用 [Witty CLI 一键部署](../deploy_guide/deployment.md)，无需手动部署 OpenCode。

## 环境要求

* **操作系统**：openEuler 24.03 LTS SP3 或更高版本
* **依赖**：
  * `nodejs >= 20`（来自 `everything` 仓库）
  * `opencode`（来自 `update` 仓库）
* **系统权限**：需具备 sudo 权限

## 安装 OpenCode

```bash
sudo dnf install nodejs opencode
```

验证安装：

```bash
opencode --version
```

## Witty 托管配置

安装 Witty 后，`witty-agent-loader` 负责在 `/etc/opencode/opencode.json` 中聚合生成 OpenCode 的托管配置。该文件为**生成产物**，不建议手工维护。

### 托管目录

| 目录 | 说明 |
| ---- | ---- |
| `/usr/share/witty/opencode/config.d/` | 各子包提供的标准 `opencode.json` 配置片段 |
| `/usr/share/witty/opencode/agents/<agent>/` | Agent 角色提示词等资源 |
| `/usr/share/witty/opencode/skills/` | Skill 包（含 `experience-skill`、内置 5 个 Skill、诊断 Skill 等） |
| `/usr/share/witty/opencode/plugins/` | OpenCode Plugin |
| `/etc/opencode/opencode.json` | 由 loader 生成的托管主配置 |

### 生成配置要点

生成的 `/etc/opencode/opencode.json` 通常包含：

```json
{
  "$schema": "https://opencode.ai/config.json",
  "skills": {
    "paths": ["/usr/share/witty/opencode/skills"]
  },
  "agent": {
    "witty-builtin-agent": {
      "description": "Witty Assistant，提供知识问答、命令查询、故障诊断、方案规划与可视化报告",
      "mode": "primary",
      "prompt": "{file:/usr/share/witty/opencode/agents/witty-builtin-agent/witty-builtin-agent.md}",
      "color": "#5F87FF",
      "permission": { "*": "ask", "read": { "*": "allow", "*.env": "ask" }, "bash": "allow" }
    }
  },
  "mcp": {
    "openeuler_portal": {
      "type": "local",
      "command": ["npx", "-y", "openeuler-portal-mcp"],
      "enabled": true
    }
  }
}
```

* `skills.paths` 固定指向 `/usr/share/witty/opencode/skills`，安装或卸载 Skill 子包后目录集合自动变化；
* agent / mcp / permission 等字段来自 `config.d/` 片段，loader 会合并且把 `{file:...}` 相对引用改写为绝对路径；
* `plugin` 字段来自各子包声明，多个子包的 plugin 条目会被去重合并。

## 直接启动 OpenCode

### 无头服务

```bash
opencode serve --port 4096
```

Witty 默认即通过这种方式连接 OpenCode；在开放了 `server.auto_start = false` 时，用户可手动启动该服务。

### 交互式 TUI

```bash
opencode
```

`opencode` 默认会在当前目录启动交互式终端界面，并自动加载 `/etc/opencode/opencode.json` 中的 agent / skill / mcp 配置。

## 如何扩展

### 安装内置 Agent 与 Skill

内置智能体与技能随 Witty 一并安装，无需额外操作。安装额外的诊断能力：

```bash
sudo dnf install witty-diagnosis-agent
```

安装后需让 OpenCode 重新加载配置：

```bash
opencode agent list     # 确认 agent 可见
```

## 附录

### 手动部署（不通过 witty）

若不想依赖 Witty 的托管 loader，可自行维护 `/etc/opencode/opencode.json`，但需自行处理配置片段合并与 `{file:...}` 路径改写。建议仅在测试环境中使用，正式环境推荐保留 Witty 的托管部署。

### 常见问题

**Q：`opencode` 安装在哪？**

`opencode` 来自 `update` 仓库。若安装失败，请确认 `/etc/yum.repos.d/` 已启用 `update` 仓库。

**Q：如何确认 OpenCode 配置已被 Witty 托管生成？**

查看 `/etc/opencode/opencode.json` 是否存在，以及 `witty doctor` 输出的 `/doc endpoint` 与 `server management` 检查项。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/installation_upgrade/upgrade/openeuler_22.03_lts_upgrade_and_downgrade_guide.md
---
# openEuler 22.03 LTS Upgrade and Downgrade Guide

This document describes how to upgrade openEuler 22.03 LTS to openEuler 22.03 LTS SP4. The operations for other versions are similar.

## 1. OS Installation

Obtain an openEuler 22.03 LTS SP4 image and install the OS by referring to the installation guide.

View the versions of openEuler and the kernel in the current environment.

```sh
cat /etc/openEuler-latest
```

## 2. Compatibility Upgrade

### 2.1 Adding the openEuler 22.03 LTS SP4 Repositories (openEuler-22.03-LTS-SP4.repo)

```sh
vi /etc/yum.repos.d/openEuler-22.03-LTS-SP4.repo
```

Add information about the following openEuler 22.03 LTS SP4 repositories and save and exit.

```conf
[SP4_OS]
name=SP4_OS
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/
metalink=https://mirrors.openeuler.org/metalink?repo=$releasever/OS&arch=$basearch
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler

[SP4_everything]
name=SP4_everything
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/
metalink=https://mirrors.openeuler.org/metalink?repo=$releasever/everything&arch=$basearch
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/RPM-GPG-KEY-openEuler

[SP4_EPOL]
name=EPOL
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/$basearch/
metalink=https://mirrors.openeuler.org/metalink?repo=$releasever/EPOL/main&arch=$basearch
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler

[SP4_debuginfo]
name=debuginfo
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/debuginfo/$basearch/
metalink=https://mirrors.openeuler.org/metalink?repo=$releasever/debuginfo&arch=$basearch
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/debuginfo/$basearch/RPM-GPG-KEY-openEuler

[SP4_source]
name=source
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/source/
metalink=https://mirrors.openeuler.org/metalink?repo=$releasever&arch=source
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/source/RPM-GPG-KEY-openEuler

[SP4_update]
name=SP4_update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/update/$basearch/
metalink=https://mirrors.openeuler.org/metalink?repo=$releasever/update&arch=$basearch
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler

[SP4_update-source]
name=SP4_update-source
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/update/source/
metalink=https://mirrors.openeuler.org/metalink?repo=$releasever/update&arch=source
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/source/RPM-GPG-KEY-openEuler
```

### 2.2 Performing the Upgrade

```sh
dnf update | tee update_log
```

Note:

1. If an error is reported during the upgrade, run `dnf update --skip-broken -x conflict_pkg1 |tee update_log` to avoid the problem. If multiple packages conflict, use the `-x conflict_pkg1 -x conflict_pkg2 -x conflict_pkg3` options to skip the packages and analyze, validate, and update the conflicted packages after the upgrade.
2. Options:\
   `--allowerasing`: Allow erasing of installed packages to resolve dependencies.\
   `--skip-broken`: Resolve dependency problems by skipping packages.\
   `-x`: Used with `--skip-broken` to specify the packages to be skipped.

### 2.3 Rebooting the OS

```sh
reboot
```

## 3. Upgrade Verification

View the versions of openEuler and the kernel in the current environment.

```sh
cat /etc/openEuler-latest
```

## 4. Compatibility Downgrade

### 4.1 Performing the Downgrade

```sh
dnf downgrade | tee downgrade_log
```

### 4.2 Rebooting the OS

```sh
reboot
```

## 5. Downgrade Verification

View the versions of openEuler and the kernel in the current environment.

```sh
cat /etc/openEuler-latest
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/ai/euler-copilot-framework/ai_full_stack/ai_container_image_userguide/ai_container_image_user_guide.md
---
# openEuler AI 容器镜像用户指南

## 简介

openEuler AI 容器镜像封装了不同硬件算力的 SDK 以及 AI 框架、大模型应用等软件，用户只需要在目标环境中加载镜像并启动容器，即可进行 AI 应用开发或使用，大大减少了应用部署和环境配置的时间，提升效率。

## 获取镜像

目前，openEuler 已发布支持 Ascend 和 NVIDIA 平台的容器镜像，获取路径如下：

* `docker.io/openeuler/cann` 存放 SDK 类镜像，在 openEuler 基础镜像之上安装 CANN 系列软件，适用于 Ascend 环境。
* `docker.io/openeuler/cuda` 存放 SDK 类镜像，在 openEuler 基础镜像之上安装 CUDA 系列软件，适用于 NVIDIA 环境。
* `docker.io/openeuler/pytorch` 存放 AI 框架类镜像，在 SDK 镜像基础之上安装 PyTorch，根据安装的 SDK 软件内容区分适用平台。
* `docker.io/openeuler/tensorflow` 存放 AI 框架类镜像，在 SDK 镜像基础之上安装 TensorFlow，根据安装的 SDK 软件内容区分适用平台。
* `docker.io/openeuler/llm` 存放模型应用类镜像，在 AI 框架镜像之上包含特定大模型及工具链，根据安装的 SDK 软件内容区分适用平台。

详细的 AI 容器镜像分类和镜像 tag 的规范说明见[oEEP-0014](https://gitee.com/openeuler/TC/blob/master/oEEP/oEEP-0014%20openEuler%20AI容器镜像软件栈规范.md)。

由于 AI 容器镜像的体积一般较大，推荐用户在启动容器前先通过如下命令将镜像拉取到开发环境中。

```sh
docker pull image:tag
```

其中，`image`为仓库名，如`openeuler/cann`，`tag`为目标镜像的 TAG，待镜像拉取完成后即可启动容器。注意，使用`docker pull`命令需按照下文方法安装`docker`软件。

## 启动容器

1. 在环境中安装`docker`，官方安装方法见 `https://docs.docker.com/engine/install/`，也可直接通过如下命令进行安装。

   ```sh
   yum install -y docker
   ```

   或

   ```sh
   apt-get install -y docker
   ```

2. NVIDIA环境安装`nvidia-container`

   1）配置yum或apt repo

   * 使用yum安装时，执行：

   ```sh
   curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
   sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
   ```

   * 使用apt安装时，执行：

   ```sh
   curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
   ```

   ```sh
   curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
   sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
   sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
   ```

   2）安装`nvidia-container-toolkit`,`nvidia-container-runtime`，执行：

   ```sh
   # yum安装
   yum install -y nvidia-container-toolkit nvidia-container-runtime
   ```

   ```sh
   # apt安装
   apt-get install -y nvidia-container-toolkit nvidia-container-runtime
   ```

   3）配置docker

   ```sh
   nvidia-ctk runtime configure --runtime=docker
   systemctl restart docker
   ```

   非NVIDIA环境不执行此步骤。

3. 确保环境中安装`driver`及`firmware`，用户可从[NVIDIA](https://www.nvidia.com/)或[Ascend](https://www.hiascend.com/)官网获取正确版本进行安装。安装完成后 Ascend 平台使用`npu-smi`命令、NVIDIA 平台使用`nvidia-smi`进行测试，正确显示硬件信息则说明安装正常。

4. 完成上述操作后，即可使用`docker run`命令启动容器。

```sh
# Ascend环境启动容器
docker run --rm --network host \
           --device /dev/davinci0:/dev/davinci0 \
           --device /dev/davinci_manager --device /dev/devmm_svm --device /dev/hisi_hdc \
           -v /usr/local/dcmi:/usr/local/dcmi -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
           -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
           -ti image:tag
```

```sh
# NVIDIA环境启动容器
docker run --gpus all -d -ti image:tag
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/driver_dev/openeuler_driver_development_specifications.md
---
# openEuler Driver Development Specifications

## Purpose

The openEuler driver development specifications are formulated to standardize and unify the submission process and mode of developed openEuler drivers and enable drivers on openEuler.

## Application Scope

The openEuler driver development specifications apply to the development of openEuler and all released versions.

## Basic Requirements for Drivers

Objectives of openEuler:

* Becomes a platform that accelerates technological innovation, maturity, and application.
* Maintains the secure, stable, and reliable kernel with optimal performance boosted by an extensive ecosystem, facilitating quick application in the industry.

Drivers that meet the preceding principles can be submitted to openEuler.

### Contributor License Agreement (CLA)

Contributors must sign the [CLA](https://www.openeuler.org/en/community/contribution/) before contributing to the openEuler community.

> **Note**: It takes about one week for the CLA to take effect after it is signed.

### Driver Requirements

The driver must meet the following requirements:

1. Have a unique name in the system.
2. Pass the Kernel Application Binary Interface (kABI) check of the openEuler community.
3. Provide correct driver version information.
4. Provide a description of driver module parameters.
5. Provide required auxiliary tools.
6. Declare the license information.
7. It is recommended that the coupling mode between the driver and the OS release be added. For example, directly check the **/etc/openEuler-release** file or other technical roadmaps, and do not couple the driver with the specific release information.
8. Provide the driver installation guide in the community repository.

## Reference

* [Kernel SIG | openEuler Kernel Patch Incorporation Specifications](https://mp.weixin.qq.com/s/rSH79v7btJfsdivC2mki1w)
* [How to Participate in openEuler Kernel Development](https://mp.weixin.qq.com/s/a42a5VfayFeJgWitqbI8Qw)

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/security/secharden/security_configuration_benchmark.md
---
# openEuler Security Configuration Description

For details, see the [openEuler security configuration description](https://atomgit.com/openeuler/security-committee/tree/master/sub-projects/secure-configuration-benchmark).

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/security/secharden/security_configuration_benchmark.md
---
# openEuler安全配置说明

详细内容请参考[openEuler安全配置说明](https://atomgit.com/openeuler/security-committee/tree/master/sub-projects/secure-configuration-benchmark)。

---

---
url: /zh/docs/22.03_LTS_SP4/server/releasenotes/account_list.md
---
# openEuler账号清单

| 用户名 | 默认密码 | 用户用途 | 用户状态 | 登录方式 | 备注 |
|---  |---  | ---  | ---  |---  |---  |
| root | openEuler12#$| 虚拟机镜像默认用户 | 启用 | 远程登录 | 登录使用openEuler虚拟机镜像安装的虚拟机。 |
| root | openEuler#12 | 登录GRUB2 | 启用 | 本地登录、远程登录 | GRUB (GRand UnifiedBootloader) 是操作系统启动管理器，用来引导不同系统（如Windows、Linux）。GRUB2是GRUB的升级版。系统启动时，可以通过GRUB2界面修改启动参数。为了确保系统的启动参数不被任意修改，需要对GRUB2界面进行加密，仅在输入正确的GRUB2口令时才能修改。 |

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/driver_dev/openeuler_driver_development_specifications.md
---
# openEuler驱动开发规范

## 目的

为规范和统一 openEuler 驱动开发的提交流程及方式，使驱动在 openEuler上使能，特制定 openEuler 驱动开发规范。

## 适用范围

openEuler 驱动开发规范适用于 openEuler 在开发以及已发布的所有版本的开发过程。

## 对驱动的基本要求

openEuler的目标：

* 成为技术创新的平台，加速技术创新、成熟及落地应用。
* 维护安全稳定、可靠、性能领先以及生态丰富的稳定内核，方便产业界快速应用。

因此，满足以上原则的驱动可以提交至openEuler。

### 签署贡献者协议（CLA）

贡献者贡献openEuler社区前，需签署贡献者协议[CLA](https://openeuler.org/zh/community/contribution/)。

> **说明** ：CLA签署后大约需要一周时间生效。

### 驱动要求

驱动需满足如下要求：

1. 名称不能与系统已有名称发生冲突。
2. 对照openEuler社区KABI检测。
3. 正确的驱动版本信息。
4. 驱动模块参数需要解释说明。
5. 如有配套工具一并提供。
6. 声明 license 信息。
7. 建议增加驱动与操作系统发行版耦合方式的规范，如直接检查 /etc/openEuler-release 文件或者其他作为技术路线的判断，不再与具体的发行版信息耦合。
8. 在社区建仓开发时同步提供驱动安装指导。

## 参考资料

* [如何参与 openEuler 内核开发](https://mp.weixin.qq.com/s?__biz=MzkyMjYzNjU0Ng==\&mid=2247506938\&idx=1\&sn=28dd447090d9d3ddeef3970975df3210\&source=41#wechat_redirect)

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secharden/os_hardening_overview.md
---
# OS Hardening Overview

This chapter describes the purpose and solution of openEuler system hardening.

## Notice

Security hardening is crucial for system security. Therefore, only the **root** user is allowed to change and apply security hardening policies.

## Security Hardening Purpose

The OS, as the core of the information system, manages hardware and software resources and is the basis of information system security. Applications must depend on the OS to ensure the integrity, confidentiality, availability, and controllability of information. Without the OS security protection, protective methods against hackers and virus attacks at other layers cannot meet the security requirements.

Therefore, security hardening is essential for an OS. Security hardening helps build a dynamic and complete security system, enhance product security, and improve product competitiveness.

## Security Hardening Solution

This section describes the openEuler security hardening solution, including the hardening methods and items.

### Security Hardening Method

You can manually modify security hardening configurations, run commands to harden the system, or use a security hardening tool to modify security hardening configurations in batches. security-tool runs as openEuler-security.service. When the system is started for the first time, the system automatically runs the service to execute the default hardening policy, and sets the service not to start as the system starts.

You can modify the  **/etc/openEuler\_security/security.conf**  file and use the security hardening tool to implement customized security hardening.

## Security Hardening Impacts

Security hardening on file permissions and account passwords may affect user habits and system usability. For details about common hardening items that affect system usability, see  [Table 1](#en-us_topic_0152100325_ta4a48f54ff2849ada7845e2380209917).

**Table  1**  Security hardening impacts

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/os_installation.md
---
# OS Installation

## Release Files

The openEuler release files include [ISO release packages](http://repo.openeuler.org/openEuler-22.03-LTS-SP4/ISO/), [VM images](http://repo.openeuler.org/openEuler-22.03-LTS-SP4/virtual_machine_img/), [container images](http://repo.openeuler.org/openEuler-22.03-LTS-SP4/docker_img/), [embedded images](http://repo.openeuler.org/openEuler-22.03-LTS-SP4/embedded_img/), and [repo sources](http://repo.openeuler.org/openEuler-22.03-LTS-SP4/).

Table 1 ISO release packages

| Name                                       | Description                                                  |
| ------------------------------------------ | ------------------------------------------------------------ |
| openEuler-22.03-LTS-SP4-aarch64-dvd.iso            | Base installation ISO file for the AArch64 architecture, including the core components for running the minimum system. |
| openEuler-22.03-LTS-SP4-everything-aarch64-dvd.iso | Full installation ISO file for the AArch64 architecture, including all components for running the entire system. |
| openEuler-22.03-LTS-SP4-everything-debug-aarch64-dvd.iso  | ISO file for openEuler debugging in the AArch64 architecture, including the symbol table information required for debugging. |
| openEuler-22.03-LTS-SP4-x86\_64-dvd.iso             | Base installation ISO file for the x86\_64 architecture, including the core components for running the minimum system. |
| openEuler-22.03-LTS-SP4-everything-x86\_64-dvd.iso  | Full installation ISO file for the x86\_64 architecture, including all components for running the entire system. |
| openEuler-22.03-LTS-SP4-everything-debuginfo-x86\_64-dvd.iso   | ISO file for openEuler debugging in the x86\_64 architecture, including the symbol table information required for debugging. |
| openEuler-22.03-LTS-SP4-source-dvd.iso             | ISO file of the openEuler source code.                       |
| openEuler-22.03-LTS-SP4-edge-aarch64-dvd.iso       | Edge ISO file for the AArch64 architecture, including the core components for running the minimum system. |
| openEuler-22.03-LTS-SP4-edge-x86\_64-dvd.iso        | Edge ISO file for the x86\_64 architecture, including the core components for running the minimum system. |
| openEuler-22.03-LTS-loongarch64-dvd-beta4.iso  | Base installation ISO file for the LoongArch architecture, including the core components for running the minimum system. |
| openEuler-22-03-LTS-ppc64le-dvd-alpha.iso      | Base installation ISO file for the ppc64le architecture, including the core components for running the minimum system. |
| openEuler-Server-OS-isoe-sw\_64-20221227.iso    | Base installation ISO file for the sw\_64 architecture, including the core components for running the minimum system. |

Table 2 VM images

| Name                             | Description                                        |
| -------------------------------- | -------------------------------------------------- |
| openEuler-22.03-LTS-SP4-aarch64.qcow2.xz | VM image of openEuler in the AArch64 architecture. |
| openEuler-22.03-LTS-SP4-x86\_64.qcow2.xz  | VM image of openEuler in the x86\_64 architecture.  |

Note: The default password of **root** user of the VM image is **openEuler12#$**. Change the password upon the first login.

Table 3 Container images

| Name                            | Description                                               |
| ------------------------------- | --------------------------------------------------------- |
| openEuler-docker.aarch64.tar.xz | Container image of openEuler in the AArch64 architecture. |
| openEuler-docker.x86\_64.tar.xz  | Container image of openEuler in the x86\_64 architecture.  |
| openEuler-22.03-LTS-SP4-stratovirt-aarch64.img.xz  | StratoVirt container image of openEuler in the AArch64 architecture.  |
| openEuler-22.03-LTS-SP4-stratovirt-x86\_64.img.xz  | StratoVirt container image of openEuler in the x86\_64 architecture.  |

Table 4 Embedded images

| Name                                   | Description                            |
| -------------------------------------- | ------------------------------- |
| arm64/aarch64-std/zImage               | Kernel image that supports QEMU in the AArch64 architecture. |
| arm64/aarch64-std/\*toolchain-22.03.sh | Development and compilation toolchain in the AArch64 architecture.   |
| arm64/aarch64-std/\*rootfs.cpio.gz     | File system that supports QEMU in the AArch64 architecture. |
| arm32/arm-std/zImage                   | Kernel image that supports QEMU in the ARM architecture.     |
| arm32/arm-std/\*toolchain-22.03.sh     | Development and compilation toolchain in the ARM architecture.       |
| arm32/arm-std/\*rootfs.cpio.gz         | File system that supports QEMU in the ARM architecture.     |
| source-list/manifest.xml               | Manifest of source code used for building.              |

Table 5 Repo sources

| Name                | Description                                |
| ------------------- | ------------------------------------------ |
| ISO                 | Stores ISO images.                         |
| OS                  | Stores basic software package sources.     |
| debuginfo           | Stores debugging package sources.          |
| docker\_img          | Stores container images.                   |
| virtual\_machine\_img | Stores VM images.                          |
| embedded\_img        | Stores embedded images.                    |
| everything          | Stores full software package sources.      |
| extras              | Stores extended software package sources.  |
| source              | Stores source code software package.       |
| update              | Stores update software package sources.    |
| EPOL                | Stores extended openEuler package sources. |

## Minimum Hardware Specifications

The following table lists the minimum hardware specifications for openEuler 22.03 LTS SP4.

Table 6 Minimum hardware requirements

| Component  | Minimum Hardware Specification                      |
| ---------- | --------------------------------------------------- |
| CPU        | Kunpeng 920 (AArch64) / x86\_64 (later than Skylake) |
| Memory     | ≥ 4 GB (8 GB or more for better experience)         |
| Hard drive | ≥ 120 GB                                            |

## Hardware Compatibility

The following table describes the servers and configurations supported by openEuler. openEuler will support more servers in the future. Partners and developers are welcome to participate in the contribution and verification. For details about the servers supported by openEuler, see the [Compatibility List](https://www.openeuler.org/en/compatibility/).

Table 7 Supported servers and configurations

---

---
url: /en/docs/22.03_LTS_SP4/server/development/gcc/kernel_fdo_user_guide.md
---
# Overview

The feedback-directed optimization (FDO) of the kernel allows users to build optimized kernels for different applications to improve the application performance in single-application scenarios. In addition, FDO is integrated GCC for openEuler, and A-FOT provides automatic optimization, enabling users to easily enable FDO.

# Installation and Deployment

## Software Requirements

* OS: openEuler 22.03 LTS SP4

## Hardware Requirements

* Architecture: AArch64 or x86\_64

## Software Installation

### Downloading the Kernel Source Code

```shell
yum install -y kernel-source
cp -r /usr/src/linux-5.10.0-153.12.0.89.oe2203SP4.aarch64 .
```

**Note: Change the version number as required.**

### (Optional) Installing GCC

GCC of openEuler 22.03 LTS SP4 can compile kernels with PGO. Alternatively, you can perform the following steps to manually build a customized GCC based on other GCC versions (GCC 10 or later). The key is to add `--disable-tls --disable-libsanitizer` during configuration.

```shell
cd ${GCC_DIR}
mkdir build
cd build
../configure --prefix=${GCC_INSTALL_PREFIX} --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,fortran,lto --enable-plugin --enable-initfini-array --disable-libgcj --without-isl --without-cloog --enable-gnu-indirect-function --build=aarch64-linux-gnu --with-stage1-ldflags='-Wl,-z,relro,-z,now' --with-boot-ldflags='-Wl,-z,relro,-z,now' --disable-bootstrap --with-multilib-list=lp64 --enable-bolt --disable-tls --disable-libsanitizer
make -j 96 && make install -j 96
```

### Installing A-FOT

```shell
git clone https://atomgit.com/openeuler/A-FOT.git
```

# Usage

You can use A-FOT to enable kernel FDO and obtain the optimized kernel by specifying **opt\_mode** as **Auto\_kernel\_PGO**. Other configuration items can be specified on the CLI, for example, `./a-fot --pgo_phase 1`. `-s` and `-n` options can be specified on CLI only. Options related to kernel FDO are as follows.

| No.| Option (Configuration File)| Description                                                    | Default Value                  |
| ---- | -------------------- | ------------------------------------------------------------ | ------------------------ |
| 1    | config\_file          | Path of the configuration file. User configurations are read from this file.              | ${afot\_path}/a-fot.ini   |
| 2    | opt\_mode             | Optimization mode to be executed by the tool. The value can be **AutoFDO**, **AutoPrefetch**, **AutoBOLT**, or **Auto\_kernel\_PGO**.| AutoPrefetch             |
| 3    | pgo\_mode             | Kernel FDO mode, which can be GCOV or full PGO.                | all                      |
| 4    | pgo\_phase            | FDO execution phase.          | 1                        |
| 5    | kernel\_src           | Kernel source code directory. If this option is not specified, the tool automatically downloads the source code.  | None (optional)              |
| 6    | kernel\_name          | File name of the kernel build. The tool will add the **-pgoing** or **-pgoed** suffix depending on the phase. | kernel                   |
| 7    | work\_path            | Script working directory, which is used to store log files, wrappers, and profiles.      | **/opt** (**/tmp** cannot be used.)|
| 8    | run\_script           | Application execution script. The user needs to write the script, which will be used by the tool to execute the target application.| /root/run.sh             |
| 9    | gcc\_path             | GCC path.                        | /usr                     |

After configuring the compilation options, run the following command to use A-FOT to automatically optimize the kernel:

```shell
a-fot --config_file ./a-fot.ini -s
```

**Note: The `-s` option instructs A-FOT to automatically reboot into the compiled kernel. If you do not want the tool to automatically perform this sensitive operation, omit this option. However, you need to manually reboot and perform the second phase (`--pgo_phase 2`).**

# Compatibility

This section describes the compatibility issues in some special scenarios. This project is in continuous iteration and issues will be fixed as soon as possible. Developers are welcome to join this project.

* The implementation of FDO in the GCC varies with the version. Therefore, GCC 10 or later is required.

---

---
url: /en/docs/22.03_LTS_SP4/server/performance/oeaware/plugin_user_guide.md
---
# Overview

This section summarizes the information about the oeAware plugins and instances, and describes how to use the tuning instances.
The collection and sensing plugins are used to collect and integrate data and provide the data for the tuning plugins. When the tuning plugins are used, the collection and awareness instances on which they depend are automatically started. The following mainly describes how to use the tuning plugins.

# Collection Plugins

## Overview

|Collection Plugin |Instance Name| Description|2203-LTS-SP4 |2403-LTS-SP1 |
|:---:|:---:|:---:|:---:|:---:|
|libpmu.so| pmu\_counting\_collector|Collect the system PMU performance counters (core).|AArch64|AArch64|
|libpmu.so| pmu\_uncore\_collector|Collect the system PMU performance counters (uncore).|AArch64|AArch64|
|libpmu.so| pmu\_sampling\_collector|Collect system PMU-related behavior records.|AArch64|AArch64|
|libpmu.so| pmu\_spe\_collector|Collect system SPE records.|AArch64|AArch64|
| libdocker\_collector.so |docker\_cpu\_collector|Collect Docker container information in the current environment.|Supported|Supported|
|libsystem\_collector.so|thread\_collector|Collect thread information in the current environment.|Supported|Supported|
|libsystem\_collector.so|kernel\_config|Collect or configure kernel parameters.|Supported|Supported|
|libsystem\_collector.so|command\_collector|Use sysstat-related collection commands to collect system information.|Supported|Supported|

# Sensing Plugins

## Overview

|Sensing Plugin |Instance Name| Description|2203-LTS-SP4 |2403-LTS-SP1 |
|:---:|:---:|:---:|:---:|:---:|
| libthread\_scenario.so |thread\_scenario|Obtain key thread information in the current environment.|Supported|Supported|
| libscenario\_numa.so (external plugin)|scenario\_numa|Obtain the cross-NUMA memory access ratio in the current environment.|AArch64| AArch64|
|libanalysis\_aware.so |analysis\_aware|Analyze service characteristics in the current environment and provide tuning suggestions.|AArch64|AArch64|

# Tuning Plugins

## Overview

|Tuning Plugin |Instance Name| Description|2203-LTS-SP4 |2403-LTS-SP1 |
|:---:|:---:|:---:|:---:|:---:|
|libsystem\_tune.so |stealtask\_tune|Optimize CPU scheduling, reduce CPU idling, and improve CPU utilization.|AArch64|AArch64|
|libsystem\_tune.so | smc\_tune|Improve the network throughput and reduce the latency based on shared memory communication in the kernel space.|Not supported|Supported|
|libsystem\_tune.so | xcall\_tune|Bypass non-essential code paths to minimize system call processing overhead.|Not supported|AArch64|
|libsystem\_tune.so |seep\_tune|Enable dynamic frequency scaling to reduce overall system power consumption.|AArch64|AArch64|
|libub\_tune.so |unixbench\_tune|Optimize the UnixBench test.|Supported|Supported|
|libdocker\_tune.so |docker\_cpu\_burst|CPUBurst can temporarily provide additional CPU resources for containers to alleviate performance bottlenecks caused by CPU limits when burst loads occur. This ensures and improves the service quality of applications (especially latency-sensitive applications).|AArch64|AArch64|
|libtune\_numa.so (external plugin)| tune\_numa\_mem\_access|Periodically migrate threads and memory to reduce cross-NUMA memory access.|AArch64|AArch64|
| Gazelle| Not integrated into oeAware|The high-performance user-mode protocol stack greatly improves the network I/O throughput of applications and focuses on database network performance acceleration.|-|-|
| libdfot.so |dfot\_tuner\_sysboost|Dynamic feedback optimization (optimization at startup and runtime). Currently, the optimization at startup is implemented.|AArch64|Not supported|

## Tuning Instance Usage

### tune\_numa\_mem\_access

#### Application Scenarios

* Frequent memory access
* Performance gains from manual core or NUMA affinity setting

#### Prerequisites

##### Operating Environment

* AArch64
* Physical machine
* openEuler kernel (4.19, 5.10, 6.6)

##### Enabling SPE

This plugin relies on the BIOS SPE feature. Before running the plugin, you need to enable the SPE. Run `perf list | grep arm_spe` to check whether the SPE is enabled. If it is enabled, the following information is displayed:

```shell
arm_spe_0//                                      [Kernel PMU event]
```

If not, perform the following steps to enable it:

1. Go to MISC Config --> SPE in the BIOS. If the SPE is set to `Disable`, switch it to `Enable`. If you cannot find this option, the BIOS version may be outdated.

2. Access `vim /etc/grub2-efi.cfg` of the system, locate the startup item corresponding to the kernel version, and add `kpti=off` to the end of the startup item. Example:

   ```shell
           linux   /vmlinuz-4.19.90-2003.4.0.0036.oe1.aarch64 root=/dev/mapper/openeuler-root ro rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap video=VGA-1:640x480-32@60me rhgb quiet  smmu.bypassdev=0x1000:0x17 smmu.bypassdev=0x1000:0x15 crashkernel=1024M,high video=efifb:off video=VGA-1:640x480-32@60me kpti=off
   ```

3. Press `Esc`, enter `:wq`, and press `Enter` to save the change and exit.

4. Run the `reboot` command to restart the server.

##### Installing the Plugin

This plugin is an external plugin and is not installed together with oeAware. You need to install it separately.

1. Check whether the plugin is installed.

   ```shell
   oeawarectl -q | grep tune_numa_mem_access
   ```

2. If the plugin does not exist, install it as follows:

   Installation method 1: `oeawarectl -i numafast`

   Installation method 2: Select the RPM package of numafast corresponding to the current system kernel version from [OEPKGS](https://search.oepkgs.net/en-US/list?s=numafast\&exactSearch=exact\&searchType=default) and manually install it. You are advised to use the latest version of numafast.

   After installation, load the plugin by running `oeawarectl -l tune_numa_mem_access`.

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e tune_numa_mem_access
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d tune_numa_mem_access
   ```

### docker\_cpu\_burst

#### Application Scenarios

* Common containers, not applicable to K8s
* High service load in the container

#### Prerequisites

* Operating Environment
  * openEuler kernel (5.10, 6.6)

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e docker_cpu_burst
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d docker_cpu_burst
   ```

### unixbench\_tune

#### Application Scenarios

UnixBench test

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e unixbench_tune
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d unixbench_tune
   ```

### stealtask\_tune

#### Application Scenarios

High service load

#### Prerequisites

* Operating Environment
  * openEuler kernel (5.10, 6.6)

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e stealtask_tune
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d stealtask_tune
   ```

### xcall\_tune

#### Application Scenarios

System call overhead reduction

#### Prerequisites

To be added.

#### How to Use

```shell
oeawarectl -e xcall_tune
```

### seep\_tune

#### Application Scenarios

* Energy saving

#### Prerequisites

* AArch64 physical machine
* XXX enabled on the BIOS (to be added)

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e seep_tune
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d seep_tune
   ```

### smc\_tune

#### Application Scenarios

* Local network communication

#### Prerequisites

* openEuler kernel (6.6)

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e smc_tune
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d smc_tune
   ```

### dfot\_tuner\_sysboost

#### Prerequisites

##### Operating Environment

* openEuler kernel (5.10)

##### Installing the Plugin

This plugin is an external plugin and is not installed together with oeAware. You need to install it separately.

1. Check whether the plugin is installed.

   ```shell
   oeawarectl -q | grep dfot_tuner_sysboost
   ```

2. If the plugin does not exist, install it.

   ```shell
   yum install D-FOT
   ```

   Load the plugin after installation.

   ```shell
   oeawarectl -l tune_numa_mem_access
   ```

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e dfot_tuner_sysboost
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d dfot_tuner_sysboost
   ```

### Gazelle

#### Application Scenarios

* Service performance affected by the network
* Low latency and high throughput

#### How to Use

This tuning capability has not been integrated into oeAware. For details, see [Gazelle User Guide](https://gitcode.com/openeuler/gazelle/blob/master/doc/en/user-guide_en.md).

---

---
url: >-
  /en/docs/22.03_LTS_SP4/tools/community_tools/performance/oeaware/plugin_user_guide.md
---
# Overview

This section summarizes the information about the oeAware plugins and instances, and describes how to use the tuning instances.
The collection and sensing plugins are used to collect and integrate data and provide the data for the tuning plugins. When the tuning plugins are used, the collection and awareness instances on which they depend are automatically started. The following mainly describes how to use the tuning plugins.

# Collection Plugins

## Overview

|Collection Plugin |Instance Name| Description|2203-LTS-SP4 |2403-LTS-SP1 |
|:---:|:---:|:---:|:---:|:---:|
|libpmu.so| pmu\_counting\_collector|Collect the system PMU performance counters (core).|AArch64|AArch64|
|libpmu.so| pmu\_uncore\_collector|Collect the system PMU performance counters (uncore).|AArch64|AArch64|
|libpmu.so| pmu\_sampling\_collector|Collect system PMU-related behavior records.|AArch64|AArch64|
|libpmu.so| pmu\_spe\_collector|Collect system SPE records.|AArch64|AArch64|
| libdocker\_collector.so |docker\_cpu\_collector|Collect Docker container information in the current environment.|Supported|Supported|
|libsystem\_collector.so|thread\_collector|Collect thread information in the current environment.|Supported|Supported|
|libsystem\_collector.so|kernel\_config|Collect or configure kernel parameters.|Supported|Supported|
|libsystem\_collector.so|command\_collector|Use sysstat-related collection commands to collect system information.|Supported|Supported|

# Sensing Plugins

## Overview

|Sensing Plugin |Instance Name| Description|2203-LTS-SP4 |2403-LTS-SP1 |
|:---:|:---:|:---:|:---:|:---:|
| libthread\_scenario.so |thread\_scenario|Obtain key thread information in the current environment.|Supported|Supported|
| libscenario\_numa.so (external plugin)|scenario\_numa|Obtain the cross-NUMA memory access ratio in the current environment.|AArch64| AArch64|
|libanalysis\_aware.so |analysis\_aware|Analyze service characteristics in the current environment and provide tuning suggestions.|AArch64|AArch64|

# Tuning Plugins

## Overview

|Tuning Plugin |Instance Name| Description|2203-LTS-SP4 |2403-LTS-SP1 |
|:---:|:---:|:---:|:---:|:---:|
|libsystem\_tune.so |stealtask\_tune|Optimize CPU scheduling, reduce CPU idling, and improve CPU utilization.|AArch64|AArch64|
|libsystem\_tune.so | smc\_tune|Improve the network throughput and reduce the latency based on shared memory communication in the kernel space.|Not supported|Supported|
|libsystem\_tune.so | xcall\_tune|Bypass non-essential code paths to minimize system call processing overhead.|Not supported|AArch64|
|libsystem\_tune.so |seep\_tune|Enable dynamic frequency scaling to reduce overall system power consumption.|AArch64|AArch64|
|libub\_tune.so |unixbench\_tune|Optimize the UnixBench test.|Supported|Supported|
|libdocker\_tune.so |docker\_cpu\_burst|CPUBurst can temporarily provide additional CPU resources for containers to alleviate performance bottlenecks caused by CPU limits when burst loads occur. This ensures and improves the service quality of applications (especially latency-sensitive applications).|AArch64|AArch64|
|libtune\_numa.so (external plugin)| tune\_numa\_mem\_access|Periodically migrate threads and memory to reduce cross-NUMA memory access.|AArch64|AArch64|
| Gazelle| Not integrated into oeAware|The high-performance user-mode protocol stack greatly improves the network I/O throughput of applications and focuses on database network performance acceleration.|-|-|
| libdfot.so |dfot\_tuner\_sysboost|Dynamic feedback optimization (optimization at startup and runtime). Currently, the optimization at startup is implemented.|AArch64|Not supported|

## Tuning Instance Usage

### tune\_numa\_mem\_access

#### Application Scenarios

* Frequent memory access
* Performance gains from manual core or NUMA affinity setting

#### Prerequisites

##### Operating Environment

* AArch64
* Physical machine
* openEuler kernel (4.19, 5.10, 6.6)

##### Enabling SPE

This plugin relies on the BIOS SPE feature. Before running the plugin, you need to enable the SPE. Run `perf list | grep arm_spe` to check whether the SPE is enabled. If it is enabled, the following information is displayed:

```shell
arm_spe_0//                                      [Kernel PMU event]
```

If not, perform the following steps to enable it:

1. Go to MISC Config --> SPE in the BIOS. If the SPE is set to `Disable`, switch it to `Enable`. If you cannot find this option, the BIOS version may be outdated.

2. Access `vim /etc/grub2-efi.cfg` of the system, locate the startup item corresponding to the kernel version, and add `kpti=off` to the end of the startup item. Example:

   ```shell
           linux   /vmlinuz-4.19.90-2003.4.0.0036.oe1.aarch64 root=/dev/mapper/openeuler-root ro rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap video=VGA-1:640x480-32@60me rhgb quiet  smmu.bypassdev=0x1000:0x17 smmu.bypassdev=0x1000:0x15 crashkernel=1024M,high video=efifb:off video=VGA-1:640x480-32@60me kpti=off
   ```

3. Press `Esc`, enter `:wq`, and press `Enter` to save the change and exit.

4. Run the `reboot` command to restart the server.

##### Installing the Plugin

This plugin is an external plugin and is not installed together with oeAware. You need to install it separately.

1. Check whether the plugin is installed.

   ```shell
   oeawarectl -q | grep tune_numa_mem_access
   ```

2. If the plugin does not exist, install it as follows:

   Installation method 1: `oeawarectl -i numafast`

   Installation method 2: Select the RPM package of numafast corresponding to the current system kernel version from [OEPKGS](https://search.oepkgs.net/en-US/list?s=numafast\&exactSearch=exact\&searchType=default) and manually install it. You are advised to use the latest version of numafast.

   After installation, load the plugin by running `oeawarectl -l tune_numa_mem_access`.

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e tune_numa_mem_access
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d tune_numa_mem_access
   ```

### docker\_cpu\_burst

#### Application Scenarios

* Common containers, not applicable to K8s
* High service load in the container

#### Prerequisites

* Operating Environment
  * openEuler kernel (5.10, 6.6)

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e docker_cpu_burst
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d docker_cpu_burst
   ```

### unixbench\_tune

#### Application Scenarios

UnixBench test

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e unixbench_tune
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d unixbench_tune
   ```

### stealtask\_tune

#### Application Scenarios

High service load

#### Prerequisites

* Operating Environment
  * openEuler kernel (5.10, 6.6)

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e stealtask_tune
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d stealtask_tune
   ```

### xcall\_tune

#### Application Scenarios

System call overhead reduction

#### Prerequisites

To be added.

#### How to Use

```shell
oeawarectl -e xcall_tune
```

### seep\_tune

#### Application Scenarios

* Energy saving

#### Prerequisites

* AArch64 physical machine
* XXX enabled on the BIOS (to be added)

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e seep_tune
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d seep_tune
   ```

### smc\_tune

#### Application Scenarios

* Local network communication

#### Prerequisites

* openEuler kernel (6.6)

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e smc_tune
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d smc_tune
   ```

### dfot\_tuner\_sysboost

#### Prerequisites

##### Operating Environment

* openEuler kernel (5.10)

##### Installing the Plugin

This plugin is an external plugin and is not installed together with oeAware. You need to install it separately.

1. Check whether the plugin is installed.

   ```shell
   oeawarectl -q | grep dfot_tuner_sysboost
   ```

2. If the plugin does not exist, install it.

   ```shell
   yum install D-FOT
   ```

   Load the plugin after installation.

   ```shell
   oeawarectl -l tune_numa_mem_access
   ```

#### How to Use

1. Enable this instance.

   ```shell
   oeawarectl -e dfot_tuner_sysboost
   ```

2. Stop this instance.

   ```shell
   oeawarectl -d dfot_tuner_sysboost
   ```

### Gazelle

#### Application Scenarios

* Service performance affected by the network
* Low latency and high throughput

#### How to Use

This tuning capability has not been integrated into oeAware. For details, see [Gazelle User Guide](https://gitcode.com/openeuler/gazelle/blob/master/doc/en/user-guide_en.md).

---

---
url: /en/docs/22.03_LTS_SP4/tools/devops/eulermaker/merge_configs.md
---
# Overview

This feature allows users to modify, customize, and iterate build files of software packages to manage macro definition differences between build files of different versions and packages.

## Installation and Uninstallation

### Installation

```shell
pip install merge_configs-0.0.6-py3-none-any.whl
```

### Uninstallation

```shell
pip uninstall merge-configs
```

## Usage

### Command Options

```shell
merge-configs --help
-p PACKAGES, --packages PACKAGES: Specifies the software packages to be merged. Separate multiple software packages by spaces.
-The c CONFIG_FILE, --config_file CONFIG_FILE: Sets the hierarchical root directory file config.yaml.
-o OUTPUT, --output OUTPUT: Sets the output file path.
-d --debug: Indicates whether to set the log mode to debug.
-l LIST_FEATURES, --list-features LIST_FEATURES: If not empty, displays the user configuration information and sets the software packages in the value of -p. Use commas (,) to separate multiple software packages.
-a TARGET_ARCH, --arch TARGET_ARCH: Sets the target architecture for merge, for example, x86_64 or aarch64.
```

Frequently used command:

```shell
merge-configs -p \${package} -c \${config_path}/config.yaml -o \${output_path} -a \${target_arch_name} -l \${package}
```

The common YAML structure is as follows:

![](./figures/image.png)

After the conversion:

![](./figures/1686189862936_image.png)

### Software Package Tailoring

The software package compilation information is stored separately in a hierarchical architecture, including the main YAML configuration, **files.yaml** file configuration, compilation execution script, runtime execution script, and changelog. The customized content in each file is parsed and converted by `merge-configs` and takes effect during compilation.

#### Parameter Customization

1. Parameter name customization:
   The parameter name can be modified. Generally, change only the source and patch numbers. A random parameter name may conflict with the SPEC file syntax.

2. Parameter value customization:
   The customization scope of parameter values is large. You can modify the content as required. However, do not change the value type. For example, if the value type is changed from string to list, conversion errors may occur.

Patch number and value modification:

![](./figures/1686190779219_image.png)

After the conversion:

![](./figures/1686190839529_image.png)

#### Conditional Customization

Add **when** conditions to the keys at the YAML configuration layer to add conditional customization.

```text
Source:
    0: http://ftp.gnu.org/gnu/libtool/libtool-%{version}.tar.xz
source when arch in aarch64:
    100: libtool-aarch-%{version}.tar.xz
```

There are three customization modes:

1. Architecture customization

   ```text
   buildRequires:
       - "gcc"
   buildRequires when arch in x86_64:
       - "gcc-c++"
   buildRequires when arch not in x86_64:
       - "gzip"
   ```

2. Flag customization:
   The **defineFlags** field will be converted to **bcond\_with** or **bcond\_without**.

   ```text
   defineFlags:
       +auto_compile: ""
   patchset when +auto_compile:
       1001: libtool-0.0.1-auto_compile.patch
   ```

3. Macro customization:
   **%%{rpmGlobal.}** indicates the macro defined in the package information, and **%%%{rpmGlobal.}** indicates the macro defined in the RPM system.

   ```text
   rpmGloal:
       posttest: 0
   source when %%{rpmGlobal.posttest}:
       1: posttest.sh
   source when %%%{rpmGlobal._debugsource_packages}:
       2: openEuler_setup.py
   ```

After customization:

![](./figures/1686194042686_image.png)

After customization and conversion:

![](./figures/1686194008501_image.png)

### Conversion

Currently, EulerMaker supports only conversion YAML to SPEC and supports only RPM package build using `rpmbuild`.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/security/cert_signature/overview_of_certificates_and_signatures.md
---
# Overview of Certificates and Signatures

## Overview

Digital signature is an important technology for protecting the integrity of OSs. By adding signatures to key system components and verifying the signatures in subsequent processes such as component loading and running, you can effectively check component integrity and prevent security problems caused by component tampering. Multiple system integrity protection mechanisms are supported in the industry to protect the integrity of different types of components in each phase of system running. Typical technical mechanisms include:

* Secure boot
* Kernel module signing
* Integrity measurement architecture (IMA)
* RPM signature verification

The preceding integrity protection security mechanisms depend on signatures (usually integrated in the component release phase). However, open source communities generally lack signature private keys and certificate management mechanisms. Therefore, Linux distributions released by open source communities generally do not provide default signatures or use only private keys temporarily generated in the build phase for signatures. Usually, these integrity protection security mechanisms can be enabled only after users or downstream OSVs perform secondary signing, which increases the cost of security functions and reduces usability.

## Solution

The openEuler community infrastructure supports the signature service. The signature platform manages signature private keys and certificates in a unified manner and works with the EulerMaker build platform to automatically sign key files during the software package build process of the community edition. Currently, the following file types are supported:

* EFI files
* Kernel module files
* IMA digest lists
* RPM software packages

## Constraints

The signature service of the openEuler community has the following constraints:

* Currently, only official releases of the openEuler community can be signed. Private builds cannot be signed.
* Currently, only EFI files related to OS secure boot can be signed, including shim, GRUB, and kernel files.
* Currently, only the kernel module files provided by the kernel software package can be signed.

---

---
url: /en/docs/22.03_LTS_SP4/tools/devops/patch_tracking/patch_tracking.md
---
# patch-tracking

## Overview

During the development of the openEuler release, the latest code of each software package in the upstream community needs to be updated in a timely manner to fix function bugs and security issues, preventing the openEuler release from defects and vulnerabilities.

This tool manages the patches for software packages, proactively monitors the patches submitted by the upstream community, automatically generates patches, submits issues to the corresponding Maintainer, and verifies basic patch functions to reduce the verification workload and help the Maintainer make decisions quickly.

## Architecture

### C/S Architecture

The patch-tracking uses the C/S architecture.

The patch-tracking is located in the server. It executes patch tracking tasks, including maintaining tracking items, identifying branch code changes in the upstream repository and generating patch files, and submitting issues and PRs to Gitee. In addition, the patch-tracking provides RESTful APIs for adding, deleting, modifying, and querying tracking items.

The patch-tracking-cli is a command line tool located in the client. It invokes the RESTful APIs of the patch-tracking to add, delete, modify, and query tracking items.

### Core Procedure

I. Patch tracking service procedure

The procedure for handling the submitted patch is as follows:

1. Add the tracking item using the command line tool.
2. Automatically obtain patch files from the upstream repository (for example, GitHub) that is configured for the tracking item.
3. Create a temporary branch and submit the obtained patch file to the temporary branch.
4. Automatically submit an issue to the corresponding repository and generate the PR associated with the issue.

![PatchTracking](./images/PatchTracking.jpg)

II. Procedure for the Maintainer to handle the submitted patch

The procedure for handling the submitted patch is as follows:

1. The Maintainer analyzes the PR.
2. Execute the continuous integration (CI). After the CI is successfully executed, determine whether to merge the PR.

![Maintainer](./images/Maintainer.jpg)

### Data structure

* Tracking table

| No.  | Name            | Description                                                  | Type    | Key     | Is Null Allowed |
| :--: | --------------- | ------------------------------------------------------------ | ------- | ------- | --------------- |
|  1   | id              | Sequence number of the tracking item of the self-added patch | int     | -       | No              |
|  2   | version\_control | Version control system type of the upstream SCM              | String  | -       | No              |
|  3   | scm\_repo        | Upstream SCM repository address                              | String  | -       | No              |
|  4   | scm\_branch      | Upstream SCM tracking branch                                 | String  | -       | No              |
|  5   | scm\_commit      | Latest Commit ID processed by the upstream code              | String  | -       | Yes             |
|  6   | repo            | Address of the Gitee repository where the package source code is stored | String  | Primary | No              |
|  7   | branch          | Branch of the Gitee repository where the package source code is stored | String  | Primary | No              |
|  8   | enabled         | Indicating whether to start tracking                         | Boolean | -       | No              |

* Issue table

| No.  | Name   | Description                                                  | Type   | Key     | Is Null Allowed |
| :--: | ------ | ------------------------------------------------------------ | ------ | ------- | --------------- |
|  1   | issue  | Issue No.                                                    | String | Primary | No              |
|  2   | repo   | Address of the Gitee repository where the package source code is stored | String | -       | No              |
|  3   | branch | Branch of the Gitee repository where the package source code is stored | String | -       | No              |

## Tool Deployment

### Downloading Software

The repo source is officially released at <https://repo.openeuler.org/>.

The RPM package can be obtained from \[<https://build.openeuler.openatom.cn/package/show/openEuler:22.03:LTS:SP4/patch-tracking>).

### Installing the Tool

Method 1: Install the patch-tracking from the repo source.

1. Use DNF to mount the repo source (The repo source of 22.03 LTS SP4 or later is required. For details, see the [Application Development Guide](../../../server/development/application_dev/application_development.md)). Run the following command to download and install the patch-tracking and its dependencies.

2. Run the following command to install the `patch-tracking`:

   ```shell
   dnf install patch-tracking
   ```

Method 2: Install the patch-tracking using the RPM package.

1. Install the required dependencies.

   ```shell
   dnf install python3-uWSGI python3-flask python3-Flask-SQLAlchemy python3-Flask-APScheduler python3-Flask-HTTPAuth python3-requests python3-pandas
   ```

2. `patch-tracking-1.0.0-1.oe1.noarch.rpm` is used as an example. Run the following command to install the patch-tracking.

   ```shell
   rpm -ivh patch-tracking-1.0.0-1.oe1.noarch.rpm
   ```

### Generating a Certificate

Run the following command to generate a certificate:

```shell
openssl req -x509 -days 3650 -subj "/CN=self-signed" \
-nodes -newkey rsa:4096 -keyout self-signed.key -out self-signed.crt
```

Copy the generated `self-signed.key` and `self-signed.crt` files to the **/etc/patch-tracking** directory.

### Configuring Parameters

Configure the corresponding parameters in the configuration file. The path of the configuration file is `/etc/patch-tracking/settings.conf`.

1. Configure the service listening address.

   ```text
   LISTEN = "127.0.0.1:5001"
   ```

2. GitHub Token is used to access the repository information hosted in the upstream open source software repository of GitHub. For details about how to create a GitHub token, see [Creating a personal access token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token).

   ```text
   GITHUB_ACCESS_TOKEN = ""
   ```

3. For a repository that is hosted on Gitee and needs to be tracked, configure a Gitee Token with the repository permission to submit patch files, issues, and PRs.

   ```text
   GITEE_ACCESS_TOKEN = ""
   ```

4. Scan the database as scheduled to detect whether new or modified tracking items exist and obtain upstream patches for the detected tracking items. Set the interval of scanning and the unit is second.

   ```text
   SCAN_DB_INTERVAL = 3600
   ```

5. When the command line tool is running, you need to enter the user name and password hash value for the authentication for the POST interface.

   ```text
   USER = "admin"

   PASSWORD = ""
   ```

> The default value of `USER` is `admin`.

Run the following command to obtain the password hash value. **Test@123** is the configured password.

```shell
$ generate_password Test@123
pbkdf2:sha256:150000$w38eLeRm$ebb5069ba3b4dda39a698bd1d9d7f5f848af3bd93b11e0cde2b28e9e34bfbbae
```

> The password hash value must meet the following complexity requirements:
>
> * The length is more than or equal to 6 bytes.
> * The password must contain uppercase letters, lowercase letters, digits, and special characters (~!@#%^\*-\_=+).

Add the password hash value `pbkdf2:sha256:150000$w38eLeRm$ebb5069ba3b4dda39a698bd1d9d7f5f848af3bd93b11e0cde2b28e9e34bfbbae` to the quotation marks of `PASSWORD = ""`.

### Starting the Patch Tracking Service

You can use either of the following methods to start the service:

* Use the systemd mode.

  ```shell
  systemctl start patch-tracking
  ```

* Run the executable program.

  ```shell
  /usr/bin/patch-tracking
  ```

## Tool Usage

### Adding a Tracking Item

You can associate the software repository and branch to be tracked with the corresponding upstream open source software repository and branch in any of the following ways:

* Using the CLI

  Parameter description:

  > \--user: User name to be authenticated for the POST interface. It is the same as the USER parameter in the **settings.conf** file.
  > \--password: Password to be authenticated for the POST interface. It is the password string corresponding to the PASSWORD hash value in the **settings.conf** file.
  > \--server: URL for starting the patch tracking service, for example, 127.0.0.1:5001.
  > \--version\_control: Control tool of the upstream repository version. Only GitHub is supported.
  > \--repo: Name of the repository to be tracked, in the format of organization/repository.
  >
  > \--branch: Branch name of the repository to be tracked.
  > \--scm\_repo: Name of the upstream repository to be tracked, in the GitHub format of organization/repository.
  > \--scm\_branch: Branch of the upstream repository to be tracked.
  >
  > \--enabled: Indicates whether to automatically track the repository.

  For example:

  ```shell
  patch-tracking-cli add --server 127.0.0.1:5001 --user admin --password Test@123 --version_control github --repo testPatchTrack/testPatch1 --branch master --scm_repo BJMX/testPatch01 --scm_branch test  --enabled true
  ```

* Using a specified file

  Parameter description:

  > \--server: URL for starting the patch tracking service, for example, 127.0.0.1:5001.
  > \--user: User name to be authenticated for the POST interface. It is the same as the USER parameter in the **settings.conf** file.
  > \--password: Password to be authenticated for the POST interface. It is the password string corresponding to the PASSWORD hash value in the **settings.conf** file.
  > \--file: YAML file path.

  Add the information about the repository, branch, version management tool, and whether to enable monitoring to the YAML file (for example, **tracking.yaml**). The file path is used as the command of the `--file` to invoke the input parameters.

  For example:

  ```shell
  patch-tracking-cli add --server 127.0.0.1:5001 --user admin --password Test@123 --file tracking.yaml
  ```

  The format of the YAML file is as follows. The content on the left of the colon (:) cannot be modified, and the content on the right of the colon (:) needs to be set based on the site requirements.

  ```shell
  version_control: github
  scm_repo: xxx/xxx
  scm_branch: master
  repo: xxx/xxx
  branch: master
  enabled: true
  ```

  > version\_control: Control tool of the upstream repository version. Only GitHub is supported.
  > scm\_repo: Name of the upstream repository to be tracked, in the GitHub format of organization/repository.
  > scm\_branch: Branch of the upstream repository to be tracked.
  > repo: Name of the repository to be tracked, in the format of organization/repository.
  > branch: Branch name of the repository to be tracked.
  > enabled: Indicates whether to automatically track the repository.

* Using a specified directory

  Place multiple `xxx.yaml` files in a specified directory, such as the `test_yaml`, and run the following command to record the tracking items of all YAML files in the specified directory.

  Parameter description:

  > \--user: User name to be authenticated for the POST interface. It is the same as the USER parameter in the **settings.conf** file.
  > \--password: Password to be authenticated for the POST interface. It is the password string corresponding to the PASSWORD hash value in the **settings.conf** file.
  > \--server: URL for starting the patch tracking service, for example, 127.0.0.1:5001.
  > \--dir: Path where the YAML file is stored.

  ```shell
  patch-tracking-cli add --server 127.0.0.1:5001 --user admin --password Test@123 --dir /home/Work/test_yaml/
  ```

### Querying a Tracking Item

Parameter description:

> \--server: (Mandatory) URL for starting the patch tracking service, for example, 127.0.0.1:5001.
> \--table: (Mandatory) Table to be queried.
> \--Repo: (Optional) repo to be queried. Query all content in the table if this parameter is not configured.
> \--branch: (Optional) Branch to be queried.

```shell
patch-tracking-cli query --server <SERVER> --table tracking
```

The website can be accessed properly.

```shell
patch-tracking-cli query --server 127.0.0.1:5001 --table tracking
```

### Querying the Generated Issue

```shell
patch-tracking-cli query --server <SERVER> --table issue
```

For example:

```shell
patch-tracking-cli query --server 127.0.0.1:5001 --table issue
```

### Deleting a Tracking Item

```shell
patch-tracking-cli delete --server SERVER --user USER --password PWD --repo REPO [--branch BRANCH]
```

For example:

```shell
patch-tracking-cli delete --server 127.0.0.1:5001 --user admin --password Test@123 --repo testPatchTrack/testPatch1 --branch master
```

> You can delete a single piece of data from a specified repo or branch. You can also delete data of all branches in a specified repo.

### Checking Issues and PRs on Gitee

Log in to Gitee and check the software project to be tracked. On the Issues and Pull Requests tab pages of the project, you can see the item named in `[patch tracking] TIME`, for example, the `[patch tracking] 20200713101548`. This item is the issue and PR of the patch file that is just generated.

## FAQ

### When I Access api.github.com, the Connection Is Refused

#### Symptom

During the operation of the patch-tracking, the following error message may occur:

```text
 Sep 21 22:00:10 localhost.localdomain patch-tracking[36358]: 2020-09-21 22:00:10,812 - patch_tracking.util.github_api - WARNING - HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /user (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0xfffe19d35820>: Failed to establish a new connection: [Errno 111] Connection refused'))   
```

#### Possible Cause

The preceding problem is caused by the unstable network access between the patch-tracking and GitHub API. Ensure that the patch-tracking is operating in a stable network environment (for example, Huawei Cloud Hong Kong).

---

---
url: /zh/docs/22.03_LTS_SP4/tools/devops/patch_tracking/patch_tracking.md
---
# patch-tracking

## 简介

在 openEuler 发行版开发过程中，需要及时更新上游社区各个软件包的最新代码，修改功能 bug 及安全问题，确保发布的 openEuler 发行版尽可能避免缺陷和漏洞。

本工具对软件包进行补丁管理，主动监控上游社区提交，自动生成补丁，并自动提交 issue 给对应的 maintainer，同时自动验证补丁基础功能，减少验证工作量支持 maintainer 快速决策。

## 架构

### C/S架构

patch-tracking采用 C/S 架构。

服务端(patch-tracking) ：负责执行补丁跟踪任务，包括：维护跟踪项，识别上游仓库分支代码变更并形成补丁文件，向 Gitee 提交 issue 及 PR，同时 patch-tracking 提供 RESTful 接口，用于对跟踪项进行增删改查操作。

客户端：即命令行工具（patch-tracking-cli），通过调用 patch-tracking 的 RESTful 接口，实现对跟踪项的增删改查操作。

### 核心流程

1. 补丁跟踪服务流程。

   主要步骤：

   1. 通过命令行工具添加跟踪项。
   2. 自动从跟踪项配置的上游仓库（例如GitHub）获取补丁文件。
   3. 创建临时分支，将获取到的补丁文件提交到临时分支。
   4. 自动提交 issue 到对应仓库，并生成关联 issue 的 PR。

   ![PatchTracking](./images/PatchTracking.jpg)

2. Maintainer对提交的补丁处理流程。

   主要步骤：

   1. Maintainer 分析 PR。
   2. 执行 CI，执行成功后判断是否合入 PR。

   ![Maintainer](./images/Maintainer.jpg)

### 数据结构

* Tracking表

| 序号 | 名称 | 说明 | 类型 | 键 | 允许空 |
|:----:| ----| ----| ----| ----| ----|
| 1 | id | 自增补丁跟踪项序号 | int | - | NO |
| 2 | version\_control | 上游SCM的版本控制系统类型 | String | - | NO |
| 3 | scm\_repo | 上游SCM仓库地址 | String | - | NO |
| 4 | scm\_branch | 上游SCM跟踪分支 | String | - | NO |
| 5 | scm\_commit | 上游代码最新处理过的Commit ID | String | - | YES |
| 6 | repo | 包源码在Gitee的仓库地址 | String | Primary | NO |
| 7 | branch | 包源码在Gitee的仓库分支 | String | Primary | NO |
| 8 | enabled | 是否启动跟踪 | Boolean | -| NO |

* Issue表

| 序号 | 名称 | 说明 | 类型 | 键 | 允许空 |
|:----:| ----| ----| ----| ----| ----|
| 1 | issue | issue编号 | String | Primary | NO |
| 2 | repo | 包源码在Gitee的仓库地址 | String | - | NO |
| 3 | branch | 包源码在Gitee的仓库分支 | String | - | NO |

## 工具部署

### 软件下载

Repo 源地址：<https://repo.openeuler.org/>

rpm 包获取地址：<https://build.openeuler.openatom.cn/package/show/openEuler:22.03:LTS:SP4/patch-tracking>

### 安装工具

方法1：从repo源安装。

1. 使用 dnf 挂载 repo源（需要 22.03-LTS-SP4 或更新的 repo 源，具体方法参考[应用开发指南](https://openeuler.org/zh/docs/22.03_LTS_SP1/docs/ApplicationDev/application-development.html)），然后执行如下指令下载以及安装 patch-tracking 及其依赖。

2. 执行以下命令安装`patch-tracking`。

   ```shell script
   dnf install patch-tracking
   ```

方法2：直接使用rpm安装

1. 首先安装相关依赖。

   ```shell script
   dnf install python3-uWSGI python3-flask python3-Flask-SQLAlchemy python3-Flask-APScheduler python3-Flask-HTTPAuth python3-requests python3-pandas
   ```

2. 以`patch-tracking-1.0.0-1.oe1.noarch.rpm`为例，执行如下命令安装。

   ```shell script
   rpm -ivh patch-tracking-1.0.0-1.oe1.noarch.rpm
   ```

### 生成证书

执行如下命令生成证书。

```shell script
openssl req -x509 -days 3650 -subj "/CN=self-signed" \
-nodes -newkey rsa:4096 -keyout self-signed.key -out self-signed.crt
```

将生成的 `self-signed.key` 和 `self-signed.crt` 文件拷贝到 **/etc/patch-tracking** 目录。

### 配置参数

在配置文件中对相应参数进行配置，配置文件路径为 `/etc/patch-tracking/settings.conf`。

1. 配置服务监听地址。

   ```text
   LISTEN = "127.0.0.1:5001"
   ```

2. GitHub Token， 用于访问托管在 GitHub 上游开源软件仓的仓库信息 ， 生成 GitHub Token 的方法参考 [Creating a personal access token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) 。

   ```text
   GITHUB_ACCESS_TOKEN = ""
   ```

3. 对于托管在gitee上的需要跟踪的仓库，配置一个有该仓库权限的gitee的token，用于提交patch文件，提交issue，提交PR等操作。

   ```text
   GITEE_ACCESS_TOKEN = ""
   ```

4. 定时扫描数据库中是否有新增或修改的跟踪项，对扫描到的跟踪项执行获取上游补丁任务，在这里配置扫描的时间间隔，数字单位是秒。

   ```text
   SCAN_DB_INTERVAL = 3600
   ```

5. 命令行工具运行过程中，POST接口需要填写进行认证的用户名和口令哈希值。

   ```text
   USER = "admin"

   PASSWORD = ""
   ```

> **说明：**\
> `USER`默认值为`admin`。

​     执行如下指令，获取口令的哈希值，其中Test@123为设置的口令。

```shell
[root]# generate_password Test@123
pbkdf2:sha256:150000$w38eLeRm$ebb5069ba3b4dda39a698bd1d9d7f5f848af3bd93b11e0cde2b28e9e34bfbbae
```

> **说明：**
>
> `口令值`需要满足如下复杂度要求：
>
> * 长度大于等于6个字符
> * 必须包含大写字母、小写字母、数字、特殊字符（~!@#%^\*-\_=+）

将口令的哈希值`pbkdf2:sha256:150000$w38eLeRm$ebb5069ba3b4dda39a698bd1d9d7f5f848af3bd93b11e0cde2b28e9e34bfbbae`配置到`PASSWORD = ""`引号中。

### 启动补丁跟踪服务

可以使用以下两种方式启动服务。

* 使用systemd方式。

  ```shell
  systemctl start patch-tracking
  ```

* 直接执行可执行程序。

  ```shell
  /usr/bin/patch-tracking
  ```

## 工具使用

### 添加跟踪项

将需要跟踪的软件仓库和分支与其上游开源软件仓库与分支关联起来，可以通过以下三种方式实现。

* 命令行直接添加。

参数含义：

> \--user ：POST接口需要进行认证的用户名，同settings.conf中的USER参数 \
> \--password ：POST接口需要进行认证的口令，为settings.conf中的PASSWORD哈希值对应的实际的口令字符串 \
> \--server ：启动Patch Tracking服务的URL，例如：127.0.0.1:5001 \
> \--version\_control ：上游仓库版本的控制工具，只支持github \
> \--repo： 需要进行跟踪的仓库名称，格式：组织/仓库 \
> \--branch ：需要进行跟踪的仓库的分支名称 \
> \--scm\_repo ：被跟踪的上游仓库的仓库名称，github格式：组织/仓库 \
> \--scm\_branch： 被跟踪的上游仓库的仓库的分支 \
> \--enabled ：是否自动跟踪该仓库

例如：

```shell script
patch-tracking-cli add --server 127.0.0.1:5001 --user admin --password Test@123 --version_control github --repo testPatchTrack/testPatch1 --branch master --scm_repo BJMX/testPatch01 --scm_branch test  --enabled true
```

* 指定文件添加

参数含义：

> \--server ：启动Patch Tracking服务的URL，例如：127.0.0.1:5001 \
> \--user ：POST接口需要进行认证的用户名，同settings.conf中的USER参数 \
> \--password ：POST接口需要进行认证的口令，为settings.conf中的PASSWORD哈希值对应的实际的口令字符串 \
> \--file ：yaml文件路径

将仓库、分支、版本管理工具、是否启动监控等信息写入yaml文件（例如tracking.yaml），文件路径作为--file的入参调用命令。

例如：

```shell script
patch-tracking-cli add --server 127.0.0.1:5001 --user admin --password Test@123 --file tracking.yaml
```

yaml文件内容格式如下，冒号左边的内容不可修改，右边内容根据实际情况填写。

```shell script
version_control: github
scm_repo: xxx/xxx
scm_branch: master
repo: xxx/xxx
branch: master
enabled: true
```

> version\_control ：上游仓库版本的控制工具，只支持github \
> scm\_repo ：被跟踪的上游仓库的仓库名称，github格式：组织/仓库 \
> scm\_branch ：被跟踪的上游仓库的仓库的分支 \
> repo ：需要进行跟踪的仓库名称，格式：组织/仓库 \
> branch ：需要进行跟踪的仓库的分支名称 \
> enabled ：是否自动跟踪该仓库

* 指定目录添加

在指定的目录，例如`test_yaml`下放入多个`xxx.yaml`文件，执行如下命令，记录指定目录下所有yaml文件的跟踪项。

参数含义：

> \--user ：POST接口需要进行认证的用户名，同settings.conf中的USER参数 \
> \--password ：POST接口需要进行认证的口令，为settings.conf中的PASSWORD哈希值对应的实际的口令字符串 \
> \--server ：启动Patch Tracking服务的URL，例如：127.0.0.1:5001 \
> \--dir ：存放yaml文件目录的路径

```shell script
patch-tracking-cli add --server 127.0.0.1:5001 --user admin --password Test@123 --dir /home/Work/test_yaml/
```

### 查询跟踪项

参数含义：

> \--server ：必选参数，启动Patch Tracking服务的URL，例如：127.0.0.1:5001 \
> \--table ：必选参数，需要查询的表 \
> \--repo ：可选参数，需要查询的repo；如果没有该参数查询表中所有内容 \
> \--branch ：可选参数，需要查询的branch

```shell script
patch-tracking-cli query --server SERVER --table tracking
```

例如：

```shell script
patch-tracking-cli query --server 127.0.0.1:5001 --table tracking
```

### 查询生成的 Issue

```shell script
patch-tracking-cli query --server SERVER --table issue
```

例如：

```shell script
patch-tracking-cli query --server 127.0.0.1:5001 --table issue
```

### 删除跟踪项

```shell script
patch-tracking-cli delete --server SERVER --user USER --password PWD --repo REPO [--branch BRANCH]
```

例如：

```shell script
patch-tracking-cli delete --server 127.0.0.1:5001 --user admin --password Test@123 --repo testPatchTrack/testPatch1 --branch master
```

> **说明：**
>
> 可以删除指定repo和branch的单条数据；也可直接删除指定repo下所有branch的数据。

### 码云查看 issue 及 PR

登录Gitee上进行跟踪的软件项目，在该项目的Issues和Pull Requests页签下，可以查看到名为`[patch tracking] TIME`，例如`[patch tracking] 20200713101548`的条目，该条目即是刚生成的补丁文件的issue和对应PR。

## FAQ

### 访问 api.github.com Connection refused 异常

#### 问题描述

patch-tracking 运行过程中，可能会出现如下报错：

```text
 9月 21 22:00:10 localhost.localdomain patch-tracking[36358]: 2020-09-21 22:00:10,812 - patch_tracking.util.github_api - WARNING - HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /user (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0xfffe19d35820>: Failed to establish a new connection: [Errno 111] Connection refused'))   
```

#### 原因分析

以上问题是 patch-tracking 与 GitHub API 服务之间网络访问不稳定导致，请确保在与 GitHub API 服务之间网络稳定的环境中（如华为云任一国际站点）运行 patch-tracking。

---

---
url: /en/docs/22.03_LTS_SP4/tools/devops/pkgship/pkgship.md
---
# pkgship

## Introduction

The pkgship is a query tool used to manage the dependency of OS software packages and provide a complete dependency graph. The pkgship provides functions such as software package dependency query, lifecycle management, and patch query.

1. Software package dependency query: Allow community personnel to understand the impact on software when software packages are introduced, updated, or deleted.
2. Patch query: Allow community personnel to learn about the patches in the openEuler software package and obtain the patch information. For details, see [patch-tracking](./../patch_tracking/patch_tracking.md).

## Architecture

The system uses the Flask-RESTful development mode. The following figure shows the architecture:

![](images/pkgship.png)

## Downloading the Software

* The repo source is officially released at: <https://repo.openeuler.org/>
* You can obtain the source code at: <https://atomgit.com/openeuler/pkgship>
* You can obtain the RPM package at: <https://117.78.1.88/project/show/openEuler:Mainline>

## Operating Environment

* Hardware configuration:

| Item| Recommended Specification|
|----------|----------|
| CPU| 8 cores|
| Memory| 32 GB (minimum: 4 GB)|
| Network bandwidth| 300 Mbit/s|
| I/O| 375 MB/s|

* Software configuration:

| Name| Specifications|
|----------|----------|
| Elasticsearch| 7.10.1. Single-node and cluster deployment is available.|
| Redis| 5.0.4 or later is recommended. You are advised to set the size to 3/4 of the memory.|
| Python| 3.8 or later.|

## Installing the Tool

> Note: The software can run in Docker. In openEuler 21.09, due to environment restrictions, use the `--privileged` parameter when creating a Docker. Otherwise, the software fails to be started. This document will be updated after the adaptation.

### 1. Installing the pkgship

You can use either of the following methods to install the pkgship:

* Method 1: Mount the repo source using DNF.
  Use DNF to mount the repo source where the pkgship is located (for details, see the [Application Development Guide](https://docs.openeuler.org/en/docs/22.03_LTS_SP1/docs/ApplicationDev/application-development.html). Then run the following command to download and install the pkgship and its dependencies:

  ```bash
  dnf install pkgship
  ```

* Method 2: Install the RPM package. Download the RPM package of the pkgship and run the following command to install the pkgship (x.x-x indicates the version number and needs to be replaced with the actual one):

  ```bash
  rpm -ivh pkgship-x.x-x.oe1.noarch.rpm
  ```

  Or

  ```bash
  dnf install pkgship-x.x-x.oe1.noarch.rpm
  ```

### 2. Installing Elasticsearch and Redis

If Elasticsearch or Redis is not installed in the environment, you can execute the automatic installation script after the pkgship is installed.

The default script path is as follows:

```bash
/etc/pkgship/auto_install_pkgship_requires.sh
```

Run the following command:

```bash
/bin/bash auto_install_pkgship_requires.sh elasticsearch
```

Or

```bash
 /bin/bash auto_install_pkgship_requires.sh redis
```

### 3. Adding a User After the Installation

After the pkgship software is installed, the system automatically creates a user named **pkgshipuser** and a user group named **pkgshipuser**. They will be used when the service is started and running.

## Configuring Parameters

1\. Configure the parameters in the configuration file. The default configuration file of the system is stored in **/etc/pkgship/package.ini**. Modify the configuration file as required.

```bash
vim /etc/pkgship/package.ini
```

```ini
[SYSTEM-System Configuration]
; Path for storing the .yaml file imported during database initialization. The .yaml file records the location of the imported .sqlite file.
init_conf_path=/etc/pkgship/conf.yaml

; Service query port
query_port=8090

; Service query IP address
query_ip_addr=127.0.0.1

; Address of the remote service. The command line can directly call the remote service to complete the data request.
remote_host=https://api.openeuler.org/pkgmanage

; Directory for storing temporary files during initialization and download. The directory will not be occupied for a long time. It is recommended that the available space be at least 1 GB.
temporary_directory=/opt/pkgship/tmp/

[LOG-Logs]
; Service log storage path
log_path=/var/log/pkgship/

; Log level. The options are as follows:
; INFO DEBUG WARNING ERROR CRITICAL
log_level=INFO

; Maximum size of a service log file. If the size of a service log file exceeds the value of this parameter, the file is automatically compressed and dumped. The default value is 30 MB.
max_bytes=31457280

; Maximum number of backup log files. The default value is 30.
backup_count=30

[UWSGI-Web Server Configuration]
; Operation log path
daemonize=/var/log/pkgship-operation/uwsgi.log
; Size of data transmitted between the front end and back end
buffer-size=65536
; Network connection timeout interval
http-timeout=600
; Service response time
harakiri=600

[REDIS-Cache Configuration]
; The address of the Redis cache server can be the released domain or IP address that can be accessed.
; The default link address is 127.0.0.1.
redis_host=127.0.0.1

; Port number of the Redis cache server. The default value is 6379.
redis_port=6379

; Maximum number of connections allowed by the Redis server at a time.
redis_max_connections=10

[DATABASE-Database]
; Database access address. The default value is the IP address of the local host.
database_host=127.0.0.1

; Database access port. The default value is 9200.
database_port=9200

```

2\. Create a YAML configuration file to initialize the database. The **conf.yaml** file is stored in the **/etc/pkgship/** directory by default. The pkgship reads the name of the database to be created and the SQLite file to be imported based on this configuration. You can also configure the repo address of the SQLite file. An example of the **conf.yaml** file is as follows:

```yaml
dbname: oe22.03   #Database name
src_db_file: /etc/pkgship/repo/openEuler-20.09/src  #Local path of the source package
bin_db_file: /etc/pkgship/repo/openEuler-20.09/bin  #Local path of the binary package
priority: 1 #Database priority

dbname: oe20.09
src_db_file: https://repo.openeuler.org/openEuler-20.09/source  #Repo source of the source package
bin_db_file: https://repo.openeuler.org/openEuler-20.09/everything/aarch64 #Repo source of the binary package
priority: 2
```

> To change the storage path, change the value of **init\_conf\_path** in the **package.ini** file.
>
> The SQLite file path cannot be configured directly.
>
> The value of **dbname** can contain only lowercase letters and digits.

## Starting and Stopping the Service

The pkgship can be started and stopped in two modes: systemctl mode and pkgshipd mode. In systemctl mode, the automatic startup mechanism can be stopped when an exception occurs. You can run any of the following commands:

```shell
systemctl start pkgship.service # Start the service.

systemctl stop pkgship.service # Stop the service.

systemctl restart pkgship.service # Restart the service.
```

```sh
pkgshipd start # Start the service.

pkgshipd stop # Stop the service.
```

> Only one mode is supported in each start/stop period. The two modes cannot be used at the same time.
>
> The pkgshipd startup mode can be used only by the **pkgshipuser** user.
>
> If the **systemctl** command is not supported in the Docker environment, run the **pkgshipd** command to start or stop the service.

## Using the Tool

1. Initialize the database.

   > Application scenario: After the service is started, to query the package information and dependency in the corresponding database (for example, oe22.03 and oe20.09), you need to import the SQLite (including the source code library and binary library) generated by the **createrepo** to the service. Then insert the generated JSON body of the package information into the corresponding database of Elasticsearch. The database name is the value of d**bname-source/binary** generated based on the value of **dbname** in the **conf.yaml** file.

   ```bash
   pkgship init [-filepath path]
   ```

   > Parameter description:
   > **-filepath**: (Optional) Specifies the path of the initialization configuration file **config.yaml.** You can use either a relative path or an absolute path. If no parameter is specified, the default configuration is used for initialization.

2. Query a single package.

   You can query details about a source package or binary package (**packagename**) in the specified **database** table.

   > Application scenario: You can query the detailed information about the source package or binary package in a specified database.

   ```bash
   pkgship pkginfo $packageName $database [-s]
   ```

   > Parameter description:
   > **packagename**: (Mandatory) Specifies the name of the software package to be queried.
   > **database**: (Mandatory) Specifies the database name.
   >
   > **-s**: (Optional) Specifies that the source package `src` is to be queried by `-s`. If this parameter is not specified, the binary package information of `bin` is queried by default.

3. Query all packages.

   Query information about all packages in the database.

   > Application scenario: You can query information about all software packages in a specified database.

   ```bash
   pkgship list $database [-s]
   ```

   > Parameter description:
   > **database**: (Mandatory) Specifies the database name.
   > **-s**: (Optional) Specifies that the source package `src` is to be queried by `-s`. If this parameter is not specified, the binary package information of `bin` is queried by default.

4. Query the installation dependency.

   Query the installation dependency of the binary package (**binaryName**).

   > Application scenario: When you need to install the binary package A, you need to install B, the installation dependency of A, and C, the installation dependency of B, etc. A can be installed only after all the installation dependencies are installed in the system. Therefore, before installing the binary package A, you may need to query all installation dependencies of A. You can run the following command to query multiple databases based on the default priority of the platform, and to customize the database query priority.

   ```bash
   pkgship installdep [$binaryName $binaryName1 $binaryName2...] [-dbs] [db1 db2...] [-level] $level
   ```

   > Parameter description:
   > **binaryName**: (Mandatory) Specifies the name of the dependent binary package to be queried. Multiple packages can be transferred.
   >
   > **-dbs:** (Optional) Specifies the priority of the database to be queried. If this parameter is not specified, the database is queried based on the default priority.
   >
   > **-level**: (Optional) Specifies the dependency level to be queried. If this parameter is not specified, the default value **0** is used, indicating that all levels are queried.

5. Query the compilation dependency.

   Query all compilation dependencies of the source code package (**sourceName**).

   > Application scenario: To compile the source code package A, you need to install B, the compilation dependency package of A. To install B, you need to obtain all installation dependency packages of B. Therefore, before compiling the source code package A, you need to query the compilation dependencies of A and all installation dependencies of these compilation dependencies. You can run the following command to query multiple databases based on the default priority of the platform, and to customize the database query priority.

   ```bash
   pkgship builddep [$sourceName $sourceName1 $sourceName2..] -dbs [db1 db2 ..] [-level] $level
   ```

   > Parameter description:
   > **sourceName**: (Mandatory) Specifies the name of the source package on which the compilation depends. Multiple packages can be queried.
   >
   > **-dbs:** (Optional) Specifies the priority of the database to be queried. If this parameter is not specified, the database is queried based on the default priority.
   >
   > **-level**: (Optional) Specifies the dependency level to be queried. If this parameter is not specified, the default value **0** is used, indicating that all levels are queried.

6. Query the self-compilation and self-installation dependencies.

   Query the installation and compilation dependencies of a specified binary package (**binaryName**) or source package (**sourceName**). In the command, **\[pkgName]** indicates the name of the binary package or source package to be queried. When querying a binary package, you can query all installation dependencies of the binary package, and the compilation dependencies of the source package corresponding to the binary package, as well as all installation dependencies of these compilation dependencies. When querying a source package, you can query its compilation dependency, and all installation dependencies of the compilation dependency, as well as all installation dependencies of the binary packages generated by the source package. In addition, you can run this command together with the corresponding parameters to query the self-compilation dependency of a software package and the dependency of a subpackage.

   > Application scenario: If you want to introduce a new software package based on the existing version library, you need to introduce all compilation and installation dependencies of the software package. You can run this command to query these two dependency types at the same time to know the packages introduced by the new software package, and to query binary packages and source packages.

   ```bash
    pkgship selfdepend [$pkgName1 $pkgName2 $pkgName3 ..] [-dbs] [db1 db2..] [-b] [-s] [-w]
   ```

   > Parameter description:
   >
   > **pkgName**: (Mandatory) Specifies the name of the software package on which the installation depends. Multiple software packages can be transferred.
   >
   > **-dbs:** (Optional) Specifies the priority of the database to be queried. If this parameter is not specified, the database is queried based on the default priority.
   >
   > **-b**: (Optional) Specifies that the package to be queried is a binary package. If this parameter is not specified, the source package is queried by default.
   >
   > **-s**: (Optional) If **-s** is specified, all installation dependencies, compilation dependencies (that is, compilation dependencies of the source package on which compilation depends), and installation dependencies of all compilation dependencies of the software package are queried. If **-s** is not added, all installation dependencies and layer-1 compilation dependencies of the software package, as well as all installation dependencies of layer-1 compilation dependencies, are queried.
   >
   > **-w**: (Optional) If **-w** is specified, when a binary package is introduced, the query result displays the source package corresponding to the binary package and all binary packages generated by the source package. If **-w** is not specified, only the corresponding source package is displayed in the query result when a binary package is imported.

7. Query dependency.
   Query the packages that depend on the software package (**pkgName**) in a database (**dbName**).

   > Application scenario: You can run this command to query the software packages that will be affected by the upgrade or deletion of the software source package A. This command displays the source packages (for example, B) that depend on the binary packages generated by source package A (if it is a source package or the input binary package for compilation). It also displays the binary packages (for example, C1) that depend on A for installation. Then, it queries the source package (for example, D) that depend on the binary package generated by B C1 for compilation and the binary package (for example E1) for installation. This process continues until it traverses the packages that depend on the binary packages.

   ```bash
    pkgship bedepend dbName [$pkgName1 $pkgName2 $pkgName3] [-w] [-b] [-install/build]
   ```

   > Parameter description:
   >
   > **dbName**: (Mandatory) Specifies the name of the repository whose dependency needs to be queried. Only one repository can be queried each time.
   >
   > **pkgName**: (Mandatory) Specifies the name of the software package to be queried. Multiple software packages can be queried.
   >
   > **-w**: (Optional) If **-w** is not specified, the query result does not contain the subpackages of the corresponding source package by default. If **\[-w]** is specified after the command, not only the dependency of binary package C1 is queried, but also the dependency of other binary packages (such as C2 and C3) generated by source package C corresponding to C1 is queried.
   >
   > **-b**: (Optional) Specifies `-b` and indicates that the package to be queried is a binary package. By default, the source package is queried.
   >
   > **-install/build**: (Optional) `-install` indicates that installation dependencies are queried. `-build` indicates that build dependencies are queried. By default, all dependencies are queried. `-install` and `-build` are exclusive to each other.

8. Query the database information.

   > Application scenario: Check which databases are initialized in Elasticsearch. This function returns the list of initialized databases based on the priority.

   ```bash
    pkgship dbs
   ```

9. Obtain the version number.

   > Application scenario: Obtain the version number of the pkgship software.

   ```bash
    pkgship -v
   ```

## Viewing and Dumping Logs

### Viewing Logs

When the pkgship service is running, two types of logs are generated: service logs and operation logs.

1\. Service logs:

Path: **/var/log/pkgship/log\_info.log**. You can customize the path through the **log\_path** field in the **package.ini** file.

Function: This log records the internal running of the code to facilitate fault locating.

Permission: The permissions on the path and the log file are 755 and 644, respectively. Common users can view the log file.

2\. Operation logs:

Path: **/var/log/pkgship-operation/uwsgi.log**. You can customize the path through the **daemonize** field in the **package.ini** file.

Function: This log records user operation information, including the IP address, access time, URL, and result, to facilitate subsequent queries and record attacker information.

Permission: The permissions on the path and the log file are 700 and 644, respectively. Only the **root** and **pkgshipuser** users can view the log file.

### Dumping Logs

1\. Service log dumping:

* Dumping mechanism

  Use the dumping mechanism of the logging built-in function of Python to back up logs based on the log size.

> The items are used to configure the capacity and number of backups of each log in the **package.ini** file.
>
> ```ini
> ; Maximum capacity of each file, the unit is byte, default is 30M
> max_bytes=31457280
>
> ; Number of old logs to keep;default is 30
> backup_count=30
> ```

* Dumping process

  After a log is written, if the size of the log file exceeds the configured log capacity, the log file is automatically compressed and dumped. The compressed file name is **log\_info.log.***x***.gz**, where *x* is a number. A smaller number indicates a later backup.

  When the number of backup log files reaches the threshold, the earliest backup log file is deleted and the latest compressed log file is backed up.

2\. Operation log dumping:

* Dumping mechanism

  A script is used to dump data by time. Data is dumped once a day and is retained for 30 days. Customized configuration is not supported.

  > The script is stored in **/etc/pkgship/uwsgi\_logrotate.sh**.

* Dumping process

  When the pkgship is started, the script for dumping data runs in the background. From the startup, dumping and compression are performed every other day. A total of 30 compressed files are retained. The compressed file name is **uwsgi.log-20201010***x*\*\*.zip\*\*, where *x* indicates the hour when the file is compressed.

  After the pkgship is stopped, the script for dumping data is stopped and data is not dumped . When the pkgship is started again, the script for dumping data is executed again.

---

---
url: /zh/docs/22.03_LTS_SP4/tools/devops/pkgship/pkgship.md
---
# pkgship

## 介绍

pkgship是一款管理OS软件包依赖关系，提供依赖和被依赖关系完整图谱的查询工具，pkgship提供软件包依赖查询、生命周期管理、补丁查询等功能。

1. 软件包依赖查询：方便社区人员在软件包引入、更新和删除的时候了解软件的影响范围。
2. 补丁查询：方便社区人员了解openEuler软件包的补丁情况以及提取补丁内容，详细内容请参见[patch-tracking](./../patch_tracking/patch_tracking.md)。

## 架构

系统采用flask-restful开发，架构如下图所示。

![](images/pkgship3.png)

## 软件下载

* Repo源挂载正式发布地址：<https://repo.openeuler.org/>
* 源码获取地址：<https://atomgit.com/openeuler/pkgship>
* RPM包版本获取地址：<https://build.openeuler.openatom.cn/project/show/openEuler:Mainline>

## 运行环境

* 硬件配置：

| 配置项   | 推荐规格    |
| -------- | ----------- |
| CPU      | 8核         |
| 内存     | 32G，最小4G |
| 网络带宽 | 300M        |
| I/O      | 375MB/sec   |

* 软件配置：

| 软件名        | 版本和规格                                 |
| ------------- | ------------------------------------------ |
| Elasticsearch | 版本7.10.1；单机部署可用；有能力可部署集群 |
| Redis         | 建议5.0.4及以上；建议大小配置为内存的3/4   |
| Python        | 版本 3.8及以上                             |

## 安装工具

> **说明:**
>
> 该软件支持在docker下运行。目前在openEuler 22.03 LTS SP4版本下，由于环境条件限制，创建docker时请使用--privileged参数，不使用--privileged参数将会导致软件启动失败，后续适配后将更新该文档。

**1、pkgship工具安装**

工具安装可通过以下两种方式中的任意一种实现。

* 方法一，通过dnf挂载repo源实现。\
  先使用dnf挂载pkgship软件在所在repo源（具体方法可参考[应用开发指南](https://docs.openeuler.org/zh/docs/22.03_LTS_SP1/docs/ApplicationDev/application-development.html)），然后执行如下指令下载以及安装pkgship及其依赖。

  ```bash
  dnf install pkgship
  ```

* 方法二，通过安装rpm包实现。
  先下载pkgship的rpm包，然后执行如下命令进行安装（其中“x.x-x”表示版本号，请用实际情况代替）。

  ```bash
  rpm -ivh pkgship-x.x-x.oe1.noarch.rpm
  ```

  或者

  ```bash
  dnf install pkgship-x.x-x.oe1.noarch.rpm
  ```

**2、Elasticsearch和Redis安装**

如果环境没有安装Elasticsearch或者Redis，可以在pkgship安装之后执行自动化安装脚本。

脚本路径默认为：

```sh
/etc/pkgship/auto_install_pkgship_requires.sh
```

执行方法为

```sh
/bin/bash auto_install_pkgship_requires.sh elasticsearch
```

或者

```sh
 /bin/bash auto_install_pkgship_requires.sh redis
```

> **说明:**
>
> 以rpm包方式安装Elasticsearch默认为无密码模式，且pkgship需使用无密码设置的Elasticsearch，因此，当前建议Elasticsearch和pkgship需安装在同一服务器，通过网络隔离提高安全性。后续版本将支持Elasticsearch设置用户名密码。

**3、安装后添加用户**

在安装pkgship软件后，会自动创建名为pkgshipuser的用户和名为pkgshipuser的用户组，无需手动创建，后续服务启动和运行时，都会以该用户角色操作。

## 配置参数

1.在配置文件中对相应参数进行配置，系统的默认配置文件存放在 /etc/pkgship/package.ini，请根据实际情况进行配置更改。

```sh
vim /etc/pkgship/package.ini
```

```ini
[SYSTEM-系统配置]
; 初始化数据库时导入的yaml文件存放位置，该yaml中记录导入的sqlite文件位置
init_conf_path=/etc/pkgship/conf.yaml

; 若部署为客户端-服务端方式，服务端需保证query_ip_addr为本机ip或者（0.0.0.0），
; 客户端可通过query_ip_addr和query_port访问服务端，或者通过设置映射的remote_host访问服务端

; 服务查询端口
query_port=8090

; 服务查询ip
query_ip_addr=127.0.0.1

; 远程服务的地址，命令行可以直接调用远程服务来完成数据请求
remote_host=https://api.openeuler.org/pkgmanage

; 初始化和下载临时文件存放目录，不会长时间占用，建议可用空间至少1G
temporary_directory=/opt/pkgship/tmp/

[LOG-日志]
; 业务日志存放路径
log_path=/var/log/pkgship/

; 打印日志级别，支持如下：
; INFO DEBUG WARNING ERROR CRITICAL
log_level=INFO

; 单个业务日志文件最大容量，超过该值会自动压缩转储，默认为30M
max_bytes=31457280

; 备份日志保留的最大数量，默认为30
backup_count=30

[UWSGI-Web服务器配置]
; 操作日志路径
daemonize=/var/log/pkgship-operation/uwsgi.log
; 前后端传输数据大小
buffer-size=65536
; 网络连接超时时间
http-timeout=600
; 服务响应时间
harakiri=600

[REDIS-缓存配置]
; Redis缓存服务器的地址可以是已发布的可以正常访问的域或IP地址
;链接地址默认为127.0.0.1
redis_host=127.0.0.1

;Redis缓存服务器的端口，默认为6379
redis_port=6379

;Redis服务器一次允许的最大连接数
redis_max_connections=10

[DATABASE-数据库]
;数据库访问地址，建议设置为本机地址
database_host=127.0.0.1

;数据库访问端口，默认为9200
database_port=9200

```

2.创建初始化数据库的yaml配置文件：
conf.yaml 文件默认存放在 /etc/pkgship/ 路径下，pkgship会通过该配置读取要建立的数据库名称以及需要导入的sqlite文件，也支持配置sqlite文件所在的repo地址。conf.yaml 示例如下所示。

```yaml
dbname: oe22.03   #数据库名称
src_db_file: /etc/pkgship/repo/openEuler-20.09/src  #源码包所在的本地路径
bin_db_file: /etc/pkgship/repo/openEuler-20.09/bin  #二进制包所在的本地路径
priority: 1 #数据库优先级

dbname: oe20.09
src_db_file: https://repo.openeuler.org/openEuler-20.09/source  #源码包所在的repo源
bin_db_file: https://repo.openeuler.org/openEuler-20.09/everything/aarch64 #二进制包所在的repo源
priority: 2
```

> **说明:**
>
> 如需更改存放路径，请更改package.ini下的 init\_conf\_path 选项。
>
> 不支持直接配置sqlite文件路径。
>
> dbname请使用小写字母或者数字，不支持大写字母。

## 服务启动和停止

pkgship启动和停止方式有两种，systemctl方式和pkgshipd方式，其中systemctl方式启动可以有异常停止自启动的机制。两种方式的执行命令为：

```shell
systemctl start pkgship.service 启动服务

systemctl stop pkgship.service 停止服务

systemctl restart pkgship.service 重启服务
```

```sh
pkgshipd start 启动服务

pkgshipd stop 停止服务
```

> **说明:**
>
> 每次启停周期内仅支持一种方式，不允许两种操作同时使用。
>
> pkgshipd启动方式只允许在pkgshipuser用户下操作。
>
> docker环境下如果不支持systemctl命令，请使用pkgshipd启停方式。

## 工具使用

1. 数据库初始化。

   > 使用场景：服务启动后，为了能查询对应的数据库（比如oe22.03，oe20.09）中的包信息及包依赖关系，需要将这些数据库通过createrepo生成的sqlite（分为源码库和二进制库）导入进服务内，生成对应的包信息json体然后插入Elasticsearch对应的数据库中。数据库名为根据conf.yaml中配置的dbname生成的dbname-source/binary。

   ```bash
   pkgship init [-filepath path]
   ```

   > 参数说明：\
   > -filepath：指定初始化配置文件config.yaml的路径，可以使用相对路径和绝对路径，不带参数则使用默认配置初始化，可选参数。

2. 单包查询。

   用户可查询源码包或者二进制包(packagename)在指定数据库表（database）中的具体信息。

   > 使用场景：用户可查询源码包或者二进制包在指定数据库中的具体信息。

   ```bash
   pkgship pkginfo $packagename $database [-s]
   ```

   > 参数说明：\
   > packagename：指定要查询的软件包名，必传参数。
   > database：指定具体的数据库名称，必传参数。
   >
   > -s: 指定`-s`将查询的是`src`源码包信息；若未指定，默认查询`bin`二进制包信息，可选参数。

3. 所有包查询。

   查询数据库下包含的所有包的信息。

   > 使用场景：用户可查询指定数据库下包含的所有软件包信息。

   ```bash
   pkgship list $database [-s]
   ```

   > 参数说明：\
   > database：指定具体的数据库名称，必传参数。\
   > -s: 指定`-s`将查询的是`src`源码包信息；若未指定，默认查询`bin`二进制包信息，可选参数。

4. 安装依赖查询。

   查询二进制包(binaryName)的安装依赖。

   > 使用场景：用户需要安装某个二进制包A时，需要安装该二进制包A的安装依赖B，及B的安装依赖C等等，直至所有的安装依赖全部安装到系统才能成功安装二进制包A。因此，在用户安装二进制包A之前，可能会需要查询二进制包A的所有安装依赖。该命令提供了此功能，允许用户根据平台默认的优先级在多个数据库之间进行查询；同时也支持用户自定义数据库查询优先级。

   ```bash
   pkgship installdep [$binaryName $binaryName1 $binaryName2...] [-dbs] [db1 db2...] [-level] $level
   ```

   > 参数说明：\
   > binaryName：需要查询安装的依赖的二进制包名字,支持传多个；必传参数。
   >
   > -dbs： 指定需要查询的database优先级，不传按照系统默认优先级搜索；可选参数。
   >
   > -level：指定需要查询的依赖层级，不传默认为0，查询所有层级；可选参数。

5. 编译依赖查询。

   查询源码包(sourceName)的所有编译依赖。

   > 使用场景：用户要编译某个源码包A的时候，需要安装源码包A的编译依赖B, 要成功安装编译依赖B需要获取B的所有安装依赖。因此，在用户编译源码包A之前，可能会需要查询源码包的编译依赖以及这些编译依赖的所有安装依赖。该命令提供了此功能，允许用户根据平台默认的优先级在多个数据库之间进行查询;同时也支持用户自定义数据库查询优先级。

   ```bash
   pkgship builddep [$sourceName $sourceName1 $sourceName2..] -dbs [db1 db2 ..] [-level] $level
   ```

   > 参数说明：\
   > sourceName：需要查询编译依赖的源码包名字,支持多个查询；必传参数。
   >
   > -dbs： 指定需要查询的database优先级,不传按照系统默认优先级搜索；可选参数。
   >
   > -level：指定需要查询的依赖层级,不传默认为0，查询所有层级；可选参数。

6. 自编译自安装依赖查询。

   查询指定二进制包(binaryName)或源码包(sourceName )的安装及编译依赖，其中\[pkgName]为查询的二进制包或者源码包的名称。当查询二进制包时，可以查询到该二进制包的所有安装依赖以及该二进制包对应的源码包的编译依赖，及这些编译依赖的所有安装依赖；当查询源码包时，可以查询该源码包的编译依赖，及这些编译依赖的所有安装依赖，并且查询该源码包生成的所有二进制包的所有安装依赖。同时，配合对应参数使用，该命令也支持查询软件包的自编译依赖查询，和包含子包的依赖查询。

   > 使用场景：如果开发者想在现有的版本库的基础上引入新的软件包，应同时引入该软件包的所有编译、安装依赖。该命令提供开发者一个同时查询这两种依赖关系的功能，能让开发者知晓该软件包会引入哪些其他的包，该命令支持查询二进制包和源码包。

   ```bash
    pkgship selfdepend [$pkgName1 $pkgName2 $pkgName3 ..] [-dbs] [db1 db2..] [-b] [-s] [-w]
   ```

   > 参数说明：
   >
   > pkgName：需要查询安装的依赖的软件包名字,支持传多个；必传参数。
   >
   > -dbs： 指定需要查询的database优先级,不传按照系统默认优先级搜索；可选参数。
   >
   > -b：指定`-b`表示查询的包是二进制,不指定默认查询源码包；可选参数。
   >
   > -s： 指定-s表示查询软件包的所有安装依赖和所有编译依赖（即编译依赖的源码包的编译依赖），以及所有编译依赖的安装依赖；如果不增加-s参数表示只查询软件包的所有安装依赖和一层编译依赖，以及一层编译依赖的所有安装依赖；可选参数。
   >
   > -w：指定-w表示引入某个二进制包的时候，查询结果会显示出该二进制包对应的源码包以及该源码包生成的所有二进制包；如果不指定-w参数表示引入某个二进制包的时候，查询结果只显示对应的源码包；可选参数。

7. 被依赖查询。\
   查询软件包(pkgName)在某数据库(dbName)中被哪些包所依赖。

   > 使用场景：针对软件包A，在升级或删除的情况下会影响哪些软件包，可通过该命令查询。该命令会显示源码包A(若为源码包)生成的所有二进制包（若输入为二进制包，那此处即为输入的二进制包）被哪些源码包（比如B）编译依赖，被哪些二进制包（比如C1）安装依赖；以及B生成的二进制包及C1被哪些源码包（比如D）编译依赖，被哪些二进制包（比如E1）安装依赖，以此类推，遍历这些二进制包的被依赖。

   ```bash
    pkgship bedepend dbName [$pkgName1 $pkgName2 $pkgName3] [-w] [-b] [-install/build]
   ```

   > 参数说明:
   >
   > dbName：需要查询依赖关系的仓库，不支持多个；必选参数。
   >
   > pkgName：待查询的软件包名称，支持多个；必选参数。
   >
   > -w ：当不指定-w 时，查询结果默认不包含对应源码包的子包；当命令后指定配置参数\[-w] 时，不仅会查询二进制包C1的被依赖关系，还会进一步去查询C1对应的源码包C生成的其他二进制包（比如：C2,C3）的被依赖关系；可选参数。
   >
   > -b：指定`-b`表示查询的包是二进制,默认查询源码包；可选参数。
   >
   > -install/build：指定`-install`表示查询的是安装被依赖,指定`-build`表示查询的是编译被依赖，默认查询全部, 但`-install`和`-build`不能同时存在；可选参数。

8. 数据库信息。

   > 使用场景：查看Elasticsearch中初始化了哪些数据库，该功能会按照优先级顺序返回已经初始化的数据库列表。

   ```bash
   pkgship dbs
   ```

9. 获取版本号。

   > 使用场景：获取pkgship软件的版本号。

   ```bash
   pkgship -v
   ```

## 日志查看和转储

**日志查看**

pkgship服务在运行时会产生两种日志，业务日志和操作日志。

1、业务日志:

路径：/var/log/pkgship/log\_info.log（支持在package.ini中通过log\_path字段自定义路径）。

功能：主要记录代码内部运行的日志，方便问题定位。

权限：路径权限755，日志文件权限644，普通用户可以查看。

2、操作日志：

路径：/var/log/pkgship-operation/uwsgi.log （支持在package.ini中通过daemonize字段自定义路径）。

功能：记录使用者操作信息，包括ip，访问时间，访问url，访问结果等，方便后续查阅以及记录攻击者信息。

权限：路径权限700，日志文件权限644，只有root和pkgshipuser可以查看。

**日志转储**

1、业务日志转储：

* 转储机制

  使用python自带的logging内置函数的转储机制，按照日志大小来备份。

> 配置项，package.ini中配置每个日志的容量和备份数量
>
> ```ini
> ; Maximum capacity of each file, the unit is byte, default is 30M
> max_bytes=31457280
>
> ; Number of old logs to keep;default is 30
> backup_count=30
> ```

* 转储过程

  当某次日志写入后，日志文件大小超过配置的日志容量时，会自动压缩转储，压缩后文件名为log\_info.log.x.gz， x是数字，数字越小为越新的备份。

  当备份日志数量到达配置的备份数量之后，最早的备份日志会被删除掉，然后备份一个最新的压缩日志文件。

2、操作日志转储：

* 转储机制

  使用脚本进行转储，按照时间转储，每日转储一次，共保留30天，不支持自定义配置。

  > 脚本位置：/etc/pkgship/uwsgi\_logrotate.sh

* 转储过程

  pkgship启动时转储脚本后台运行，从启动时，每隔1天进行转储压缩，共保留30份压缩文件，压缩文件名称为uwsgi.log-20201010x.zip， x为压缩时的小时数。

  pkgship停止后转储脚本停止，不再进行转储，再次启动时，转储脚本重新执行。

---

---
url: /en/docs/22.03_LTS_SP4/server/performance/powerapi/powerapi.md
---
# powerapi User Guide

This document describes how to install and use powerapi, the interface for controlling the power consumption of openEuler.

This article is intended for community developers, open source enthusiasts, and partners who use the openEuler OS and want to learn and use powerapi. Users must have basic knowledge of the Linux OS.

---

---
url: /zh/docs/22.03_LTS_SP4/server/performance/powerapi/powerapi.md
---
# powerapi 用户指南

本文档介绍 openEuler 系统功耗控制接口 powerapi 的安装和使用方法，以指导用户快速了解并使用 powerapi。

本文档适用于使用 openEuler 系统并希望了解和使用 powerapi 的社区开发者、开源爱好者以及相关合作伙伴。使用人员需要具备基本的 Linux 操作系统知识。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/preparing_certificates.md
---
# Preparing Certificates

**Statement: The certificate used in this document is self-signed and cannot be used in a commercial environment.**

Before deploying a cluster, you need to generate certificates required for communication between components in the cluster. This document uses the open-source CFSSL as the verification and deployment tool to help users understand the certificate configuration and the association between certificates of cluster components. You can select a tool based on the site requirements, for example, OpenSSL.

## Building and Installing CFSSL

The following commands for building and installing CFSSL are for your reference (the CFSSL website access permission is required, and the proxy must be configured first):

```bash
wget --no-check-certificate  https://github.com/cloudflare/cfssl/archive/v1.5.0.tar.gz
tar -zxf v1.5.0.tar.gz
cd cfssl-1.5.0/
yum -y install git go
make -j6
cp bin/* /usr/local/bin/
```

## Generating a Root Certificate

Compile the CA configuration file, for example, ca-config.json:

```bash
$ cat ca-config.json | jq
{
  "signing": {
    "default": {
      "expiry": "8760h"
    },
    "profiles": {
      "kubernetes": {
        "usages": [
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ],
        "expiry": "8760h"
      }
    }
  }
}
```

Compile a CA CSR file, for example, ca-csr.json:

```bash
$ cat ca-csr.json  | jq
{
  "CN": "Kubernetes",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "openEuler",
      "OU": "WWW",
      "ST": "BinJiang"
    }
  ]
}
```

Generate the CA certificate and key:

```bash
cfssl gencert -initca ca-csr.json | cfssljson -bare ca
```

The following certificates are obtained:

```bash
ca.csr  ca-key.pem  ca.pem
```

## Generating the admin Account Certificate

admin is an account used by K8S for system management. Compile the CSR configuration of the admin account, for example, admin-csr.json:

```bash
cat admin-csr.json | jq
{
  "CN": "admin",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:masters",
      "OU": "Containerum",
      "ST": "BinJiang"
    }
  ]
}
```

Generate a certificate:

```bash
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin
```

The result is as follows:

```bash
admin.csr  admin-key.pem  admin.pem
```

## Generating a service-account Certificate

Compile the CSR configuration file of the service-account account, for example, service-account-csr.json:

```bash
cat service-account-csr.json | jq
{
  "CN": "service-accounts",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "Kubernetes",
      "OU": "openEuler k8s install",
      "ST": "BinJiang"
    }
  ]
}
```

Generate a certificate:

```bash
cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes service-account-csr.json | cfssljson -bare service-account
```

The result is as follows:

```bash
service-account.csr  service-account-key.pem  service-account.pem
```

## Generating the kube-controller-manager Certificate

Compile the CSR configuration of kube-controller-manager:

```bash
{
  "CN": "system:kube-controller-manager",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:kube-controller-manager",
      "OU": "openEuler k8s kcm",
      "ST": "BinJiang"
    }
  ]
}
```

Generate a certificate:

```bash
cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager
```

The result is as follows:

```bash
kube-controller-manager.csr  kube-controller-manager-key.pem  kube-controller-manager.pem
```

## Generating the kube-proxy Certificate

Compile the CSR configuration of kube-proxy:

```bash
{
  "CN": "system:kube-proxy",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:node-proxier",
      "OU": "openEuler k8s kube proxy",
      "ST": "BinJiang"
    }
  ]
}
```

Generate a certificate:

```bash
cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
```

The result is as follows:

```bash
kube-proxy.csr  kube-proxy-key.pem  kube-proxy.pem
```

## Generating the kube-scheduler Certificate

Compile the CSR configuration of kube-scheduler:

```bash
{
  "CN": "system:kube-scheduler",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:kube-scheduler",
      "OU": "openEuler k8s kube scheduler",
      "ST": "BinJiang"
    }
  ]
}
```

Generate a certificate:

```bash
cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler
```

The result is as follows:

```bash
kube-scheduler.csr  kube-scheduler-key.pem  kube-scheduler.pem
```

## Generating the kubelet Certificate

The certificate involves the host name and IP address of the server where kubelet is located. Therefore, the configuration of each node is different. The script is compiled as follows:

```bash
$ cat node_csr_gen.bash

#!/bin/bash

nodes=(k8snode1 k8snode2 k8snode3)
IPs=("192.168.122.157" "192.168.122.158" "192.168.122.159")

for i in "${!nodes[@]}"; do

cat > "${nodes[$i]}-csr.json" <<EOF
{
  "CN": "system:node:${nodes[$i]}",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:nodes",
      "OU": "openEuler k8s kubelet",
      "ST": "BinJiang"
    }
  ]
}
EOF

 # generate ca
 echo "generate: ${nodes[$i]} ${IPs[$i]}"
 cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -hostname=${nodes[$i]},${IPs[$i]}-profile=kubernetes ${nodes[$i]}-csr.json | cfssljson -bare ${nodes[$i]}
done
```

Note: If a node has multiple IP addresses or other aliases, you can add other IP addresses or host names to -hostname.

The result is as follows:

```bash
k8snode1.csr       k8snode1.pem       k8snode2-key.pem  k8snode3-csr.json
k8snode1-csr.json  k8snode2.csr       k8snode2.pem      k8snode3-key.pem
k8snode1-key.pem   k8snode2-csr.json  k8snode3.csr      k8snode3.pem
```

CSR configuration information. The following uses k8snode1 as an example:

```bash
$ cat k8snode1-csr.json
{
  "CN": "system:node:k8snode1",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:nodes",
      "OU": "openEuler k8s kubelet",
      "ST": "BinJiang"
    }
  ]
}
```

Note: The account group to which each node belongs is `system:node`. Therefore, the CN field of CSR is `system:node` plus `hostname`.

## Generating the kube-apiserver Certificate

Compile the CSR configuration file of kube api server:

```bash
$ cat kubernetes-csr.json | jq
{
  "CN": "kubernetes",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "Kubernetes",
      "OU": "openEuler k8s kube api server",
      "ST": "BinJiang"
    }
  ]
}
```

Generate the certificate and key:

```bash
cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -hostname=10.32.0.1,192.168.122.154,192.168.122.155,192.168.122.156,127.0.0.1,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.svc.cluster.local -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes
```

The result is as follows:

```bash
kubernetes.csr  kubernetes-key.pem  kubernetes.pem
```

*Note: 10.32.0.1 is an IP address within the range used by internal services. You can set it to other values. This parameter will be set when the apiserver service is started.*

## Generating the ETCD Certificate

ETCD can be deployed in either of the following ways:

* Start an ETCD service on each api-server.
* Independently deploy an ETCD cluster service.

If the certificate is deployed together with api-server, use the `kubernetes-key.pem` and `kubernetes.pem` certificates generated in the preceding steps.

For an independent ETCD cluster, you need to create the following certificates:

Compile the CSR configuration of ETCD:

```bash
cat etcd-csr.json | jq
{
  "CN": "ETCD",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "ETCD",
      "OU": "openEuler k8s etcd",
      "ST": "BinJiang"
    }
  ]
}
```

Generate a certificate:

```bash
cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -hostname=192.168.122.154,192.168.122.155,192.168.122.156,127.0.0.1 -profile=kubernetes etcd-csr.json | cfssljson -bare etcd
```

Note: Assume that the IP address of the ETCD cluster is *192.168.122.154,192.168.122.155,192.168.122.156*.

The result is as follows:

```bash
etcd.csr  etcd-key.pem  etcd.pem
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/application_dev/preparations_for_development_environment.md
---
# Preparing the Development Environment

## Environment Requirements

* If physical machines (PMs) are used, the minimum hardware specifications of the development environment are listed in [Table 1](#table154419352610).

  **Table 1**  Minimum hardware specifications

* If virtual machines (VMs) are used, the minimum virtualization space required by the development environment is listed in [Table 2](#table780410493819).

  **Table 2**  Minimum virtualization space specifications

### OS Requirements

The openEuler OS is required.

For details about how to install the openEuler OS, see the [Installation Guide](./../../installation_upgrade/installation/installation_guide.md). On the **SOFTWARE SELECTION** page, select **Development Tools** in the **Add-Ons for Selected Environment** area.

## Configuring the openEuler Yum Source

Configure an online Yum source using the online openEuler repo source. Alternatively, configure a local Yum source by mounting an ISO file and creating a local openEuler repo source.

### Configuring an Online Yum Source by Obtaining the Online openEuler Repo Source

> \[!NOTE] **NOTE:**
> openEuler provides multiple repo sources for users to use online. For details about the repo sources, see [OS Installation](./../../releasenotes/os_installation.md). This section uses the OS repo source of the AArch64 architecture as an example to describe how to configure it as a Yum source.

1. Go to the Yum source directory and view the .repo configuration file in the directory.

   ```shell
   $ cd /etc/yum.repos.d
   $ ls
   openEuler.repo
   ```

2. Edit the **openEuler.repo** file as the **root** user and configure the online openEuler repo source as the Yum source.

   ```shell
   vi openEuler.repo
   ```

   Edit the **openEuler.repo** file as follows:

```text
   [osrepo]
   name=osrepo
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/aarch64/
   enabled=1

   gpgcheck=1
   gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/aarch64/RPM-GPG-KEY-openEuler
```

> \[!NOTE] **NOTE:**
>
> * *repoid* indicates the ID of the software repository. *repoid* in all .repo configuration files must be unique. In the example, *repoid* is set to **osrepo**.
>
> * **name** indicates the character string of the software repository description.
>
> * **baseurl** indicates the address of the software repository.
>
> * **enabled** indicates whether to enable the software source repository. The value can be **1** or **0**. The default value is **1**, indicating that the software source repository is enabled.
>
> * **gpgcheck** can be set to **1** or **0**. Value **1** indicates that the GNU Private Guard (GPG) check is enabled, while value **0** indicates that the GPG check is disabled. **gpgcheck** checks whether the source of the RPM package is valid and secure. If this option is not specified, the GPG check is enabled by default.
>
> * **gpgkey** indicates the public key used to verify the signature.

### Configuring a Local Yum Source by Mounting an ISO File

> \[!NOTE] **NOTE:**
> openEuler provides multiple ISO release packages. For details about the ISO release packages, see [OS Installation](./../../releasenotes/os_installation.md). This section uses the **openEuler-22.03-LTS-SP4-aarch64-dvd.iso** release package and **openEuler-22.03-LTS-SP4-aarch64-dvd.iso.sha256sum** verification file as examples. Modify them based on the actual requirements.

1. Download the ISO release package.
   * Download an ISO image using a cross-platform file transfer tool.
     1. Visit the [openEuler community](https://www.openeuler.org/en/).

     2. Choose **Downloads** > **Community Editions**.

     3. Locate the target version, for example, **openEuler 22.03-LTS-SP4**. Then, click **Download**. The download list is displayed.

     4. The download list includes the following architectures and scenarios:
        Architectures:
        * **x86\_64**: ISO of the x86\_64 architecture.
        * **AArch64**: ISO of the AArch64 architecture.
        * **ARM32**: ISO for embedded devices.
          Scenarios:
        * Server: ISO for the server scenario.
        * Edge computing: ISO for the edge computing scenario.
        * Cloud computing: ISO for the cloud computing scenario.
        * Embedded: ISO for the embedded scenario.

     5. Click **AArch64**.

     6. Click **Server**.

     7. Choose **Offline Standard ISO** and click **Download** to download the openEuler release package to the local host.

     8. Click **SHA256** to copy the checksum. Save the checksum as a local verification file.

     9. Log in to the openEuler OS and create a directory for storing the release package and verification file, for example, **~/iso**.

        ```shell
        mkdir ~/iso
        ```

     10. Use a cross-platform file transfer tool (such as WinSCP) to upload the local openEuler release package and verification file to the openEuler OS.

   * Run the **wget** command to download the ISO image.
     1. Visit the [openEuler community](https://www.openeuler.org/en/).

     2. Choose **Downloads** > **Community Editions**.

     3. Locate the target version, for example, **openEuler 22.03-LTS-SP4**. Then, click **Download**. The download list is displayed.

     4. The download list includes the following architectures and scenarios:
        Architectures:
        * **x86\_64**: ISO of the x86\_64 architecture.
        * **AArch64**: ISO of the AArch64 architecture.
        * **ARM32**: ISO for embedded devices.
          Scenarios:
        * Server: ISO for the server scenario.
        * Edge computing: ISO for the edge computing scenario.
        * Cloud computing: ISO for the cloud computing scenario.
        * Embedded: ISO for the embedded scenario.

     5. Click **AArch64**.

     6. Click **Server**.

     7. Choose **Offline Standard ISO**, right-click **Download**, and copy the link address.

     8. Right-click **SHA256** and copy the link address.

     9. Log in to the openEuler OS, create a directory for storing the release package and verification file, for example, **~/iso**. Then switch to the directory.

        ```shell
        mkdir ~/iso
        cd ~/iso
        ```

     10. Run the **wget** command to remotely download the release package and verification file. In the command, replace **ipaddriso** and **ipaddrisosum** with the addresses copied in steps 7 and 8.

         ```shell
         wget ipaddriso
         wget ipaddrisosum
         ```

2. Verify the integrity of the release package.
   1. Obtain the verification value in the verification file.

      ```shell
      cat openEuler-22.03-LTS-SP4-aarch64-dvd.iso.sha256sum
      ```

   2. Calculate the SHA256 verification value of the openEuler release package.

      ```shell
      sha256sum openEuler-22.03-LTS-SP4-aarch64-dvd.iso 
      ```

      After the command is executed, the verification value is displayed.

   3. Check whether the verification values calculated in step 1 and step 2 are the same.

      If the verification values are the same, the integrity of the ISO file is not damaged. If the verification values are different, the integrity of the ISO file is damaged and you need to obtain the ISO file again.

3. Mount the ISO file and create a repo source.

   Run the `mount` command to mount the image file as the **root** user.

   Example:

   ```shell
   mount /home/iso/openEuler-22.03-LTS-SP4-aarch64-dvd.iso /mnt/
   ```

   The structure of the mounted **/mnt** directory is as follows:

   ```console
   .
   │── boot.catalog
   │── docs
   │── EFI
   │── images
   │── Packages
   │── repodata
   │── TRANS.TBL
   └── RPM-GPG-KEY-openEuler
   ```

   In the directory, **Packages** indicates the directory where the RPM package is stored, **repodata** indicates the directory where the repo source metadata is stored, and **RPM-GPG-KEY-openEuler** indicates the public key for signing openEuler.

4. Go to the Yum source directory and view the .repo configuration file in the directory.

   ```shell
   $ cd /etc/yum.repos.d

   $ ls
   openEuler.repo
   ```

5. Edit the **openEuler.repo** file as the **root** user. Configure the local openEuler repo source created in step [3](#li6236932222) as the local Yum source.

   ```shell
   vi openEuler.repo
   ```

   Edit the **openEuler.repo** file as follows:

   ```text
   [localosrepo]
   name=localosrepo
   baseurl=file:///mnt
   enabled=1
   gpgcheck=1
   gpgkey=file:///mnt/RPM-GPG-KEY-openEuler
   ```

## Installing Software Packages

Install the software required for development. The software required varies in different development environments, but the installation methods are the same. This section describes how to install common software packages (JDK and rpm-build). Some development software, such as GCC and GNU make, is provided by the openEuler OS by default.

### Installing the JDK Software Package

1. Run the `dnf list installed | grep jdk` command to check whether JDK has been installed.

   ```shell
   dnf list installed | grep jdk
   ```

   Check the command output. If the command output contains **jdk**, the software has been installed and does not need to be installed again. If no information is displayed, the software is not installed.

2. Clear the cache.

   ```shell
   dnf clean all
   ```

3. Create a cache.

   ```shell
   dnf makecache
   ```

4. Query the JDK software packages that can be installed.

   ```shell
   dnf search jdk | grep jdk
   ```

   View the command output and install the **java-x.x.x-openjdk-devel.aarch64** software package. *x.x.x* indicates the version number.

5. Install the JDK software package as the **root** user. The following uses the **java-1.8.0-openjdk-devel-1.8.0.372.b07-1.oe2203SP3.aarch64** software package as an example.

   ```shell
   dnf install java-1.8.0-openjdk-devel-1.8.0.372.b07-1.oe2203SP3.aarch64
   ```

6. Query the JDK version.

   ```shell
   java -version
   ```

   If the command output contains **openjdk version "1.8.0\_232"**, JDK has been correctly installed. **1.8.0\_232** indicates the JDK version.

### Installing the rpm-build Software Package

1. Run the `dnf list installed | grep rpm-build` command to check whether the rpm-build software has been installed.

   ```shell
   dnf list installed | grep rpm-build
   ```

   Check the command output. If the command output contains **rpm-build**, the software has been installed and does not need to be installed again. If no information is displayed, the software is not installed.

2. Clear the cache.

   ```shell
   dnf clean all
   ```

3. Create a cache.

   ```shell
   dnf makecache
   ```

4. Install the rpm-build software package as the **root** user.

   ```shell
   dnf install rpm-build
   ```

5. Query the rpm-build version.

   ```shell
   rpmbuild --version
   ```

## Using the IDE for Java Development

For small-sized Java applications, you can directly use JDK to compile them to run Java applications. However, for medium- and large-sized Java applications, this method cannot meet developers' requirements. You can perform the following operations to install and use the development environment (IDE) to facilitate Java development on the openEuler OS.

### Overview

IntelliJ IDEA is a popular Java IDE. You can download and use the community edition of IntelliJ IDEA free of charge. Currently, openEuler supports Java program development using the IntelliJ IDEA, which improves the work efficiency of developers.

### Logging In to the Server Using MobaXterm

MobaXterm is an excellent SSH client. It has a built-in X Server and can easily solve the remote GUI display problems.

You need to download and install MobaXterm in advance, start it, log in to your server in SSH mode, and perform the following operations.

### Setting the JDK Environment

Before setting **JAVA\_HOME**, you need to find the JDK installation path. If you have not installed JDK, install it by referring to the preceding section "Installing the JDK Software Package."

Run the following commands to view the Java path:

```shell
$ which java
/usr/bin/java
```

Run the following commands to check the directory to which the soft link points:

```shell
$ ls -la /usr/bin/java
lrwxrwxrwx. 1 root root  22 Mar 6 20:28 /usr/bin/java -> /etc/alternatives/java
$ ls -la /etc/alternatives/java
lrwxrwxrwx. 1 root root  83 Mar 6 20:28 /etc/alternatives/java -> /usr/lib/jvm/java-1.8.0-openjdk-devel-1.8.0.372.b07-1.oe2203SP3.aarch64/jre/bin/java
```

The actual path of JDK is **/usr/lib/jvm/java-1.8.0-openjdk-devel-1.8.0.372.b07-1.oe2203SP3.aarch64**. Run the following commands to set **JAVA\_HOME** and **PATH**:

```shell
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-devel-1.8.0.372.b07-1.oe2203SP3.aarch64
export PATH=$JAVA_HOME/bin:$PATH
```

### Downloading and Installing the GTK Library

Run the following command:

```shell
dnf list installed | grep gtk
```

If **gtk2** or **gtk3** is displayed, the GTK library has been installed. In this case, skip this step. Otherwise, run the following command as the **root** user to automatically download and install the GTK library:

```shell
dnf -y install gtk2 libXtst libXrender  xauth
```

### Setting X11 Forwarding

Switch to the SSHD configuration directory.

```shell
cd ~/.ssh
```

If the directory does not exist, run the following command to create it and then switch to it:

```shell
mkdir ~/.ssh
```

Edit the **config** file in the .ssh directory and save the file.

1. Run the **vim** command to open the **config** file.

   ```shell
   vim config
   ```

2. Add the following content to the end of the file and save the file:

   ```shell
   Host *
       ForwardAgent yes
       ForwardX11 yes
   ```

### Downloading and Running IntelliJ IDEA

After performing the preceding environment configuration, you can download and use IntelliJ IDEA. The latest IntelliJ IDEA is incompatible with openEuler in some functions. You are advised to download the [Linux package of the 2018 version](https://www.jetbrains.com/idea/download/other.html). Move the downloaded package to the directory where you want to install the software and decompress the package.

```shell
tar xf ideaIC-2018.3.tar.gz
```

Decompress the package, switch to the IntelliJ IDEA directory, and run IntelliJ IDEA.

```shell
cd ./idea-IC-183.4284.148
bin/idea.sh &
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/prepare_env.md
---
# Preparing the Environment

## Usage

* StratoVirt can run on VMs with the x86\_64 or AArch64 processor architecture.
* You are advised to compile, debug, and deploy StratoVirt on openEuler 22.03 LTS SP4.
* StratoVirt can run with non-root permissions.

## Environment Requirements

The following are required in the environment for running StratoVirt:

* /dev/vhost-vsock device (for implementing MMIO)
* nmap tool
* Kernel and rootfs images

## Preparing Devices and Tools

* To run StratoVirt, the MMIO device must be implemented. Therefore, before running StratoVirt, ensure that the **/dev/vhost-vsock** device exists.

  Check whether the device exists.

  ```sh
  $ ls /dev/vhost-vsock
  /dev/vhost-vsock
  ```

  If the device does not exist, run the following command to generate it:

  ```sh
  modprobe vhost_vsock
  ```

* To use QMP commands, install the nmap tool first. After configuring the Yum source, run the following command to install the tool:

  ```sh
  # yum install nmap
  ```

## Preparing Images

### Creating the Kernel Image

StratoVirt of the current version supports only the PE kernel image of the x86\_64 and AArch64 platforms. The kernel image in PE format can be generated by using the following method:

1. Run the following commands to obtain the kernel source code of openEuler:

   ```sh
   git clone https://atomgit.com/openeuler/kernel.git
   cd kernel
   ```

2. Run the following command to check and switch to the kernel version openEuler-22.03-LTS-SP4:

   ```sh
   git checkout openEuler-22.03-LTS-SP4
   ```

3. Configure and compile the Linux kernel. You are advised to use the [recommended configuration file](https://atomgit.com/openeuler/stratovirt/tree/master/docs/kernel_config)). Copy the file to the kernel directory, rename it to **.config**, and run the `make olddefconfig` command to update to the latest default configuration (otherwise, you may need to manually select options for subsequent compilation). Alternatively, you can run the following command to configure the kernel as prompted. The system may display a message indicating that specific dependencies are missing. Run the `yum install` command to install the dependencies as prompted.

   ```sh
   make menuconfig
   ```

4. Run the following command to create and convert the kernel image to the PE format. The converted image is **vmlinux.bin**.

   ```sh
   make -j vmlinux && objcopy -O binary vmlinux vmlinux.bin
   ```

5. If you want to use the kernel in bzImzge format on the x86 platform, run the following command:

   ```sh
   make -j bzImage
   ```

## Creating the Rootfs Image

The rootfs image is a file system image. When StratoVirt is started, the ext4 image with **init** can be loaded. To create an ext4 rootfs image, perform the following steps:

1. Prepare a file with a proper size (for example, create a file with the size of 10 GB in **/home**).

   ```sh
   cd /home
   dd if=/dev/zero of=./rootfs.ext4 bs=1G count=10
   ```

2. Create an empty ext4 file system on this file.

   ```sh
   mkfs.ext4 ./rootfs.ext4
   ```

3. Mount the file image. Create the **/mnt/rootfs** directory and mount **rootfs.ext4** to the directory as user **root**.

   ```sh
   $ mkdir /mnt/rootfs
   # Return to the directory where the file system is created, for example, **/home**.
   $ cd /home
   $ sudo mount ./rootfs.ext4 /mnt/rootfs && cd /mnt/rootfs
   ```

4. Obtain the latest alpine-mini rootfs of the corresponding processor architecture.

   * If the AArch64 processor architecture is used, you can get the latest rootfs from the [alpine](http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/). For example, alpine-minirootfs-3.16.0-aarch64.tar.gz, the reference commands are as follows:

   ```sh
   wget http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/aarch64/alpine-minirootfs-3.16.0-aarch64.tar.gz
   tar -zxvf alpine-minirootfs-3.16.0-aarch64.tar.gz
   rm alpine-minirootfs-3.16.0-aarch64.tar.gz
   ```

   * If the x86\_64 processor architecture is used, you can get the latest rootfs from the [alpine](http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/). For example, alpine-minirootfs-3.16.0-x86\_64.tar.gz, the reference commands are as follows:

   ```sh
   wget http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/x86_64/alpine-minirootfs-3.16.0-x86_64.tar.gz
   tar -zxvf alpine-minirootfs-3.16.0-x86_64.tar.gz
   rm alpine-minirootfs-3.16.0-x86_64.tar.gz
   ```

5. Run the following commands to create a simple **/sbin/init** for the ext4 file image:

   ```sh
   $ rm sbin/init; touch sbin/init && cat > sbin/init <<EOF
   #! /bin/sh
   mount -t devtmpfs dev /dev
   mount -t proc proc /proc
   mount -t sysfs sysfs /sys
   ip link set up dev lo

   exec /sbin/getty -n -l /bin/sh 115200 /dev/ttyS0
   poweroff -f
   EOF

   sudo chmod +x sbin/init
   ```

6. Unmount the rootfs image.

   ```sh
   cd /home; umount /mnt/rootfs
   ```

   The rootfs is created successfully. You can use the ext4 rootfs image file **rootfs.ext4**, which is stored in the **/home** directory.

## Obtaining Firmware Required for Standard Boot

Firmware refers to the device driver stored in a device. An OS can be booted in standard boot mode only through firmware. Currently, StratoVirt supports standard boot based on the Unified Extensible Firmware Interface (UEFI) only in the x86\_64 and AArch64 architectures.

EDK II is open source software that implements the UEFI standard. StratoVirt uses EDK II as the standard boot firmware, and therefore you need to obtain the EDK II firmware binary, which can be installed by yum commands.

Run the following command in the x86\_64 architecture:

```shell
sudo yum install -y edk2-ovmf
```

Run the following command in the AArch64 architecture:

```shell
sudo yum install -y edk2-aarch64
```

The EDK II firmware binary contains two files: one is used to save executable code, and the other is used to save boot configuration information. After the installation is complete, in the x86\_64 architecture, the firmware file **OVMF\_CODE.fd** and firmware configuration file **OVMF\_VARS.fd** are saved in the **/usr/share/edk2/ovmf** directory. In the AArch64 architecture, the firmware file **QEMU\_EFI-pflash.raw** and firmware configuration file **vars-template-pflash.raw** are saved in the **/usr/share/edk2/aarch64** directory.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/preparing_vms.md
---
# Preparing VMs

This document describes how to use virt manager to install a VM. Ignore if your VM is prepared.

## Installing Dependency Tools

VM installation depends on related tools. The following command is an example for installing the dependency and enabling the libvirtd service. (If a proxy is required, configure the proxy first.)

```bash
dnf install virt-install virt-manager libvirt-daemon-qemu edk2-aarch64.noarch virt-viewer
systemctl start libvirtd
systemctl enable libvirtd
```

## Preparing VM Disk Files

```bash
dnf install -y qemu-img
virsh pool-define-as vmPool --type dir --target /mnt/vm/images/
virsh pool-build vmPool
virsh pool-start vmPool
virsh pool-autostart  vmPool
virsh vol-create-as --pool vmPool --name master0.img --capacity 200G --allocation 1G --format qcow2
virsh vol-create-as --pool vmPool --name master1.img --capacity 200G --allocation 1G --format qcow2
virsh vol-create-as --pool vmPool --name master2.img --capacity 200G --allocation 1G --format qcow2
virsh vol-create-as --pool vmPool --name node1.img --capacity 300G --allocation 1G --format qcow2
virsh vol-create-as --pool vmPool --name node2.img --capacity 300G --allocation 1G --format qcow2
virsh vol-create-as --pool vmPool --name node3.img --capacity 300G --allocation 1G --format qcow2
```

## Enabling Firewall Ports

**Method 1**

1. Query a port.

   ```shell
   netstat -lntup | grep qemu-kvm
   ```

2. Enable the VNC firewall port. For example, if the port number starts from 5900, run the following commands:

   ```shell
   firewall-cmd --zone=public --add-port=5900/tcp
   firewall-cmd --zone=public --add-port=5901/tcp
   firewall-cmd --zone=public --add-port=5902/tcp
   firewall-cmd --zone=public --add-port=5903/tcp
   firewall-cmd --zone=public --add-port=5904/tcp
   firewall-cmd --zone=public --add-port=5905/tcp
   ```

**Method 2**

Disable the firewall.

```shell
systemctl stop firewalld
```

## Preparing the VM Configuration File

A VM configuration file is required for creating a VM. For example, if the configuration file is master.xml and the host name of the VM is k8smaster0, the configuration is as follows:

```bash
 cat master.xml

<domain type='kvm'>
    <name>k8smaster0</name>
    <memory unit='GiB'>8</memory>
    <vcpu>8</vcpu>
    <os>
 <type arch='aarch64' machine='virt'>hvm</type>
 <loader readonly='yes' type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader>
 <nvram>/var/lib/libvirt/qemu/nvram/k8smaster0.fd</nvram>
    </os>
    <features>
 <acpi/>
 <gic version='3'/>
    </features>
    <cpu mode='host-passthrough'>
        <topology sockets='2' cores='4' threads='1'/>
    </cpu>
    <iothreads>1</iothreads>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>restart</on_crash>
    <devices>
 <emulator>/usr/libexec/qemu-kvm</emulator>
 <disk type='file' device='disk'>
     <driver name='qemu' type='qcow2' iothread="1"/>
     <source file='/mnt/vm/images/master0.img'/>
     <target dev='vda' bus='virtio'/>
     <boot order='1'/>
 </disk>
 <disk type='file' device='cdrom'>
     <driver name='qemu' type='raw'/>
     <source file='/mnt/openEuler-21.09-everything-aarch64-dvd.iso'/>
     <readonly/>
     <target dev='sdb' bus='scsi'/>
     <boot order='2'/>
 </disk>
        <interface type='network'>
           <mac address='52:54:00:00:00:80'/>
           <source network='default'/>
           <model type='virtio'/>
        </interface>
 <console type='pty'/>
        <video>
           <model type='virtio'/>
        </video>
        <controller type='scsi' index='0' model='virtio-scsi'/>
 <controller type='usb' model='ehci'/>
 <input type='tablet' bus='usb'/>
 <input type='keyboard' bus='usb'/>
 <graphics type='vnc' listen='0.0.0.0'/>
    </devices>
    <seclabel type='dynamic' model='dac' relabel='yes'/>
</domain>
```

The VM configuration must be unique. Therefore, you need to modify the following to ensure that the VM is unique:

* name: host name of the VM. You are advised to use lowercase letters. In this example, the value is `k8smaster0`.
* nvram: handle file path of the NVRAM, which must be globally unique. In this example, the value is `/var/lib/libvirt/qemu/nvram/k8smaster0.fd`.
* disk source file: VM disk file path. In this example, the value is `/mnt/vm/images/master0.img`.
* mac address of the interface: MAC address of the interface. In this example, the value is `52:54:00:00:00:80`.

## Installing a VM

1. Create and start a VM.

   ```shell
   virsh define master.xml
   virsh start k8smaster0
   ```

2. Obtain the VNC port number of the VM.

   ```shell
   virsh vncdisplay k8smaster0
   ```

3. Use a VM connection tool, such as VNC Viewer, to remotely connect to the VM and perform configurations as prompted.

4. Set the host name of the VM, for example, k8smaster0.

   ```shell
   hostnamectl set-hostname k8smaster0
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/privileged_container.md
---
# Privileged Container

## Scenarios

By default, iSulad starts common containers that are suitable for starting common processes. However, common containers have only the default permissions defined by capabilities in the  **/etc/default/isulad/config.json**  directory. To perform privileged operations (such as use devices in the  **/sys**  directory), a privileged container is required. By using this feature, user  **root**  in the container has  **root**  permissions of the host. Otherwise, user  **root**  in the container has only common user permissions of the host.

## Usage Restrictions

Privileged containers provide all functions for containers and remove all restrictions enforced by the device cgroup controller. A privileged container has the following features:

* Secomp does not block any system call.

* The  **/sys**  and  **/proc**  directories are writable.

* All devices on the host can be accessed in the container.

* All system capabilities will be enabled.

Default capabilities of a common container are as follows:

When a privileged container is enabled, the following capabilities are added:

## Usage Guide

iSulad runs the  **--privileged**  command to enable the privilege mode for containers. Do not add privileges to containers unless necessary. Comply with the principle of least privilege to reduce security risks.

```sh
isula run --rm -it --privileged busybox
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/administrator/process_management.md
---
# Process Management

The operating system (OS) manages multiple user requests and tasks. In most cases, the OS comes with only one CPU and one main memory, but multiple tier-2 disks and input/output (I/O) devices. Therefore, users have to share resources, but it appears to users that they are exclusively occupying resources. The OS places user tasks, OS tasks, mailing, print tasks, and other pending tasks in a queue and schedules the tasks according to predefined rules. This topic describes how the OS manages processes.

## Viewing Processes

Linux is a multi-task system and needs to get process information during process management. To manage processes, you need to know the number of processes and their statuses. Multiple commands are available to view processes.

### who Command

The `who` command is used to display system user information. For example, before running the `talk` command to establish instant communication with another user, you need to run the `who` command to determine whether the target user is online. In another example, the system administrator can run the `who` command to learn what each login user is doing at the current time. The `who` command is widely seen in system administration since it is easy to use and can return a comprehensive set of accurate user information.

The following is an example output of the `who` command, where system users and their status are displayed: The use of the `who` command is as follows:

```shell
$ who
admin    tty1         2023-07-28 15:55
admin    pts/0        2023-08-05 15:46 (192.168.0.110)
admin    pts/2        2023-07-29 19:52 (192.168.0.110)
root     pts/3        2023-07-30 12:07 (192.168.0.110)
root     pts/4        2023-07-31 10:29 (192.168.0.144)
root     pts/5        2023-07-31 14:52 (192.168.0.11)
root     pts/6        2023-08-06 10:12 (192.168.0.234)
root     pts/8        2023-08-06 11:34 (192.168.0.234)
```

### ps Command

The  **ps**  command is the most basic and powerful command to view process information. The ps command is used to display process information, including which processes are running, terminated, resource-hungry, or stay as zombies.

The `ps` command is the most basic and powerful command to view process information, including which processes are running, terminated, resource-hungry, or stay as zombies.

A common scenario is to monitor background processes, which do not interact with your screen, keyboard, and other I/O devices.  [Table 1](#en-us_topic_0151921029_t34619d964a3d41ad8694189ec383359c)  lists the common `ps` command options.

**Table  1**  Common ps command options

For example, to list all processes on a terminal, run the following command:

```shell
$ ps -a
  PID TTY          TIME CMD
12175 pts/6    00:00:00 bash
24526 pts/0    00:00:00 vsftpd
29478 pts/5    00:00:00 ps
32461 pts/0    1-01:58:33 sh
```

### top Command

Both the `top` and `ps` commands can display a list of currently running processes, but the `top` command allows you to update the displayed list of processes by pressing a button repeatedly. If the `top` command is executed in foreground, it exclusively occupies foreground until it is terminated. The `top` command provides real-time visibility into system processor status. You can sort the list of CPU tasks by CPU usage, memory usage, or task execution time. Extensive display customization, such as choosing the columns or sorting method, can be achieved using interactive commands or the customization file.

[Figure 1](#en-us_topic_0151921029_f289234fcdbac453796200d80e9889cd1)  provides an example output of the `top` command.

**Figure  1**  Example command output\
![](./figures/example-command-output.png)

### kill Command

The `kill` command is used to terminate a process regardless of whether the process is running in foreground or background. It differs from the combo key **Ctrl+C**, which can terminate only foreground processes. The reason for terminating a background process can be heavy use of CPU resources or deadlock.

The `kill` command sends a signal to terminate running processes. By default, the `TERM` signal is used, terminating all processes incapable of capturing it. To terminate a process capable of capturing the `TERM` signal, use the `KILL` signal (signal ID: 9) instead.

Two types of syntax of the `kill` command:

```shell
kill [-s signal | -p] [-a] PID…
kill -l [signal]
```

The process ID can be retrieved by running the `ps` command. The `-s` option indicates the signal sent to the specified program. The signal details can be viewed by running the `kill -l` command. The `-p` option indicates the specified process ID.

For example, to terminate the process whose ID is 1409, run the following command as the **root** user:

```shell
kill -9 1409
```

Example output of the `kill` command with the `-l` option

```shell
$ kill -l
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM
16) SIGSTKFLT   17) SIGCHLD     18) SIGCONT     19) SIGSTOP     20) SIGTSTP
21) SIGTTIN     22) SIGTTOU     23) SIGURG      24) SIGXCPU     25) SIGXFSZ
26) SIGVTALRM   27) SIGPROF     28) SIGWINCH    29) SIGIO       30) SIGPWR
31) SIGSYS      34) SIGRTMIN    35) SIGRTMIN+1  36) SIGRTMIN+2  37) SIGRTMIN+3
38) SIGRTMIN+4  39) SIGRTMIN+5  40) SIGRTMIN+6  41) SIGRTMIN+7  42) SIGRTMIN+8
43) SIGRTMIN+9  44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13
48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12
53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9  56) SIGRTMAX-8  57) SIGRTMAX-7
58) SIGRTMAX-6  59) SIGRTMAX-5  60) SIGRTMAX-4  61) SIGRTMAX-3  62) SIGRTMAX-2
63) SIGRTMAX-1  64) SIGRTMAX
```

## Scheduling a Process

The time-consuming and resource-demanding part of maintenance work is often performed at late night. You can schedule relevant processes to get started at the scheduled time instead of staying up all night. The following describes the process scheduling commands.

### Using the at Command to Run Processes at the Scheduled Time

#### Function

The `at` command is used to run a batch of processes (a series of commands) at the scheduled time or time and date.

Syntax of the `at` command:

```shell
at [-V] [-q queue] [-f filename] [-mldbv] time
at -c job [job...]
```

#### Time Format

The scheduled time can be in any of the following formats:

* *hh:mm* today: If *hh:mm* is earlier than the current time, the selected commands will be run at *hh:mm* the next day.
* midnight, noon, teatime (typically at 16:00), or the like
* 12-hour format followed by am or pm
* Time + date (*month day*, *mm/dd/yy*, or *dd.mm.yy*). The scheduled date must follow the scheduled time.

The scheduled time can also be relative time, which is suitable for scheduling commands that are going to be executed soon. For example, now+*N* minutes, hours, days, or weeks. *N* indicates the specified time, which may be a few days or hours. Further, the scheduled time can be words like today, tomorrow, or the like. Here are some examples of the scheduled time.

Assume that the current time is 12:30 June 7 2019 and you want to run a command at 4:30 pm. The time scheduled by the `at` command can be any of the following:

```shell
 at 4:30pm
 at 16:30
 at 16:30 today
 at now+4 hours
 at now+ 240 minutes
 at 16:30 7.6.19
 at 16:30 6/7/19
 at 16:30 Jun 7
```

Although you can select any of the preceding examples according to your preference, absolute time in 24-hour format, such as `at 16:30 6/7/19`, is recommended.

#### Privileges

Only commands from standard input or from the file specified by the **-f** option can be scheduled by the `at` command. If the `su` command is executed to switch the OS from user A to user B and then the `at` command is executed at the shell prompt of user B, the `at` command execution result is sent to user B, whereas emails (if any) are sent to user A.

For example, to run the `slocate -u` command at 10 am on June 8, 2019, run the following commands as the **root** user:

```shell
$ at  10:00  6/8/19
at> slocate -u
at>
[1]+   Stopped    at  10:00  6/8/19
```

When the **at>** prompt appears, type `slocate -u` and press **Enter**. Repeat the step to add other commands that need to be run at 10 am on 8 June 2019. Then, press **Ctrl+D** to exit the `at` command.

The administrator is authorized to run the `at` command unconditionally. For other users, their privileges to run the `at` command is defined in the **/etc/at.allow** and **/etc/at.deny** files.

### Using the cron Service to Run Commands Periodically

The `at` command can run commands at the scheduled time, but only once. It means that after the commands to be run is specified, the system completes the task at the specified time. If you need to run the commands repeatedly, the **cron** service is a good choice.

#### Cron Service

The **cron** service searches the **/var/spool/cron** directory for the **crontab** files named by the user name in the **/etc/passwd** file and loads the search results into memory to execute the commands in the **crontab** files. Each user has a **crontab** file with the same name as the user name. For example, the **crontab** file of the **userexample** user is **/var/spool/cron/userexample**.

The **cron** service also reads the cron configuration file **/etc/crontab** every minute, which can be edited in various formats. If no **crontab** files are found, the **cron** service enters sleep mode and releases system resources. One minute later, the **cron** service is waken up to repeat the search work and command execution. Therefore, the background process occupies few resources and is wakened up every minute to check whether there are commands to be executed.

Command execution results are then mailed to users specified by the environment variable `MAILTO` in the **/etc/crontab** file. The **cron** service, once started, does not require manual intervention except when you need to replace the scheduled commands with new ones.

#### crontab Command

The `crontab` command is used to install, edit, remove, list, and perform other operations on **crontab** files. Each user has its own **crontab** files and can add commands to be executed to the files.

Here are common `crontab` command options:

* crontab -u   //Set the **cron** service of a user. This option is required only when the `crontab` command is run by the **root** user.
* crontab -l   //List details about the **cron** service of a user.
* crontab -r   //Remove the **cron** service of a user.
* crontab -e   //Edit the **cron** service of a user.

For example, to list the **cron** service settings of the **root** user, run the following command:

```shell
crontab -u root -l
```

#### crontab Files

Enter the commands to be executed and their scheduled time in **crontab** files. Each line in the files contains six fields. The first five fields are the time when the specified command is executed, and the last field is the command to be executed. Fields are separated by spaces or tabs. The format is as follows:

```shell
minute hour day-of-month month-of-year day-of-week commands
```

The following table describes the fields in each line.

**Table  2**  Parameter description

The fields cannot be left unspecified. In addition to numerical values, the following special characters are allowed: asterisk (\*), indicating a wildcard value; forward slash (/), followed by a numeral value *N* to indicate that commands will be executed at a regular interval of *N*; hyphen (-), used with a range; and comma (,), used to separate discrete values. A complete path to the commands must be provided.

For example, to allow the OS to add **sleepy** to the **/tmp/test.txt** file every two hours from 18 pm to 22 pm, add the following line to a **crontab** file:

```shell
* 18-22/2 * * * echo "sleepy" >> /tmp/test.txt
```

Each time the **cron** service settings of a user are edited, the **cron** service generates a **crontab** file with the same name as the user in the **/var/spool/cron directory**. The **crontab** file can be edited only using the `crontab -e` command. Alternatively, the user can create a file and run the `crontab _filename_` command to import its **cron** settings to the new file.

For example, to create a **crontab** file for the **userexample** user, perform the following steps:

1. Create a file using any text editor. Add the commands that need to be executed periodically and the command execution interval to the new file. In this example, the new file is **~/userexample.cron**.
2. Run the following command as the **root** user to install the new file as the **crontab** file of the **userexample** user:

   ```shell
   crontab -u userexample ~/userexample.cron
   ```

After the new file is installed, you will find a file named **userexample** in the **/var/spool/cron** directory. This file is the required **crontab** file.

> \[!NOTE] **NOTE:**
> Do not restart the **cron** service after a **crontab** file is modified, because the **cron** service, once started, reads the **crontab** file every minute to check whether there are commands that need to be executed periodically.

#### /etc/crontab File

The **cron** service reads all files in the **/var/spool/cron** directory and the **/etc/crontab** file every minute. Therefore, you can use the **cron** service by configuring the **/etc/crontab** file. A **crontab** file contains user-specific commands, whereas the **/etc/crontab** file contains system-wide commands. The following is an example of the **/etc/crontab** file.

```text
SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root //If an error occurs or data is output, the data is sent to the account by email.
HOME=/
#  run-parts
01 * * * * root run-parts /etc/cron.hourly //Run scripts in the /etc/cron.hourly directory once an hour.
02 4 * * *   root run-parts /etc/cron.daily    //Run scripts in the /etc/cron.daily directory once a day.
22 4 * * 0  root run-parts /etc/cron.weekly     //Run scripts in the /etc/cron.weekly directory once a week.
42 4 1  * *  root run-parts /etc/cron.monthly     //Run scripts in the /etc/cron.monthly directory once a month.
```

> \[!NOTE] **NOTE:**
> If the **run-parts** parameter is deleted, a script name instead of a directory name is used.

## Suspending/Resuming a Process

A process can be suspended or resumed by job control, and the process will continue to work from the suspended point after being resumed. To suspend a foreground process, press **Ctrl+Z**. After you press **Ctrl+Z**, the `cat` command is suspended together with the foreground process you want to suspend. You can use the `jobs` command instead to display a list of shell jobs, including their names, IDs, and status.

To resume a process in foreground or background, run the `fg` or `bg` command, respectively. The process then starts from where it was suspended previously.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/diversified_computing/dpu_offload/qtfs_architecture_and_usage.md
---
# qtfs

## 介绍

qtfs是一个共享文件系统项目，可部署在host-dpu的硬件架构上，也可以部署在2台服务器之间。它以客户端服务器的模式工作，使客户端能通过qtfs访问服务端的指定文件系统，得到本地文件访问一致的体验。

qtfs的特性：

* 支持挂载点传播；

* 支持proc、sys、cgroup等特殊文件系统的共享；

* 支持远程文件读写的共享；

* 支持在客户端对服务端的文件系统进行远程挂载；

* 支持特殊文件的定制化处理；

* 支持远端fifo、unix-socket等，并且支持epoll，使客户端和服务端像本地通信一样使用这些文件；

* 支持基于host-dpu架构通过PCIe协议底层通信，性能大大优于网络；

* 支持内核模块形式开发，无需对内核进行侵入式修改。

## 软件架构

软件大体框架图：

![qtfs-arch](./figures/qtfs-arch.png)

## 安装教程

目录说明：

* **rexec**：跨主机二进制生命周期管理组件，在该目录下编译rexec和rexec\_server。

* **ipc**: 跨主机unix domain socket协同组件，在该目录下编译udsproxyd二进制和libudsproxy.so库。

* **qtfs**： 客户端内核模块相关代码，直接在该目录下编译客户端ko。

* **qtfs\_server**： 服务端内核模块相关代码，直接在该目录下编译服务端ko和相关程序。

* **qtinfo**：诊断工具，支持查询文件系统的工作状态以及修改log级别等。

* **demo**、**test**、**doc**：测试程序、演示程序以及项目资料等。

* 根目录： 客户端与服务端通用的公共模块代码。

### VSOCK通信模式

如有DPU硬件支持通过vsock与host通信，可选择此方法。

如果没有硬件，也可以选择host-vm作为qtfs的client与server进行模拟测试，通信通道为vsock：

启动vm时为vm配置vsock通道，vm可参考如下配置，增加vsock段置：

```text
    <devices>
      ...
      <vsock model='virtio'>
        <cid auto='no' address='10'/>
        <alias name='vsock0'/>
        <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
      </vsock>
      ...
    </devices>
```

其他依赖：

1. 要求内核版本在5.10或更高版本。
2. 安装内核开发包：yum install kernel-devel json-c-devel。

服务端编译安装：

```bash
    1. cd qtfs_server
    2. make clean && make -j
    3. insmod qtfs_server.ko qtfs_server_vsock_cid=2 qtfs_server_vsock_port=12345 qtfs_log_level=WARN
    4. 配置白名单，将qtfs/config/qtfs/whitelist文件拷贝至/etc/qtfs/下，请手动配置需要的白名单选项，至少需要配置一个Mount白名单才能启动后续服务，任何文件或目录的增删改查操作都需要在白名单中增加对应权限才能正常工作。
    Tips: whitelist文件可从https://atomgit.com/openeuler/dpu-utilities/blob/master/qtfs/config/qtfs/whitelist获取
    5. nohup ./engine 16 1 2 12121 10 12121 2>&1 &
    6. engine参数解释：engine ${number_of_threads} ${uds_proxy_thread_number} ${server_cid_or_ip} ${server_uds_proxy_port} ${client_cid_or_ip} ${client_uds_proxy_port}
    Tips: 这里的cid需要根据配置决定，如果host作为server端，则cid固定配置为2，如果vm作为server端，则需要配置为前面xml中的cid字段，本例中为10。
```

客户端安装：

```bash
    1. cd qtfs
    2. make clean && make -j
    3. insmod qtfs.ko qtfs_server_vsock_cid=2 qtfs_server_vsock_port=12345 qtfs_log_level=WARN
    4. cd ../ipc/
    5. make clean && make && make install
    6. nohup udsproxyd 1 10 12121 2 12121 2>&1 &
    7. udsproxyd参数解释：udsproxyd ${uds_proxy_thread_number} ${client_cid_or_ip} ${client_uds_proxy_port} ${server_cid_or_ip} ${server_uds_proxy_port}
    Tips：这里插入ko的cid和port配置为与server端一致即可，udsproxyd的cid + port与server端交换位置。
```

其他注意事项：

1. udsproxyd目前也支持vsock和测试模式两种，使用vsock模式时，不能带UDS\_TEST\_MODE=1进行编译。
2. 如果vsock不通，需要检查host是否插入了vhost\_vsock内核模块：modprobe vhost\_vsock。

### 测试模式 - 网络通信通道

首先找两台服务器（或虚拟机）配置内核编译环境：

1. 要求内核版本在5.10或更高版本。
2. 安装内核开发包：yum install kernel-devel。
3. 假设host服务器ip为192.168.10.10，dpu为192.168.10.11。

服务端安装：

```bash
    1. cd qtfs_server
    2. make clean && make -j QTFS_TEST_MODE=1
    3. 指定测试服务端的ip，ip a a ip_server(例：192.168.10.10)/port dev network（例:ens32），防止机器重启造成的ip变更问题，方便测试
    4. insmod qtfs_server.ko qtfs_server_ip=x.x.x.x qtfs_server_port=12345 qtfs_log_level=WARN
    5. 配置白名单，将qtfs/config/qtfs/whitelist文件拷贝至/etc/qtfs/下，请手动配置需要的白名单选项，至少需要配置一个Mount白名单才能启动后续服务。任何文件或目录的增删改查操作都需要在白名单中增加对应权限才能正常工作。
    Tips: whitelist文件可从https://atomgit.com/openeuler/dpu-utilities/blob/master/qtfs/config/qtfs/whitelist获取
    6. nohup ./engine 16 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 &
    7. engine参数解释：engine ${number_of_threads} ${uds_proxy_thread_number} ${server_cid_or_ip} ${server_uds_proxy_port} ${client_cid_or_ip} ${client_uds_proxy_port}
```

Tips: 该模式暴露网络端口，有可能造成安全隐患，仅能用于功能验证测试，勿用于实际生产环境。

客户端安装：

```bash
    1. cd qtfs
    2. make clean && make -j QTFS_TEST_MODE=1
    3. 指定测试用户端的ip，ip a a ip_client(例：192.168.10.11)/port dev network（例:ens32），防止机器重启造成的ip变更问题，方便测试
    3. insmod qtfs.ko qtfs_server_ip=x.x.x.x qtfs_server_port=12345 qtfs_log_level=WARN
    4. cd ../ipc/
    5. make clean && make UDS_TEST_MODE=1 && make install
    6. nohup udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 &
    7. udsproxyd参数解释：udsproxyd ${uds_proxy_thread_number} ${client_cid_or_ip} ${client_uds_proxy_port} ${server_cid_or_ip} ${server_uds_proxy_port}
Tips：这里插入ko的cid和port配置为与server端一致即可，udsproxyd的cid + port与server端交换位置。
```

Tips: 该模式暴露网络端口，有可能造成安全隐患，仅能用于功能验证测试，勿用于实际生产环境。

## 使用说明

安装完成后，客户端通过挂载把服务端的文件系统让客户端可见，例如：

```bash
    mount -t qtfs / /root/mnt/
```

客户端进入"/root/mnt"后便可查看到server端/目录下的所有文件，以及对其进行相关操作。此操作受到白名单的控制，需要挂载路径在server端白名单的Mount列表，或者在其子目录下，且后续的查看或读写操作都需要开放对应的白名单项才能进行。

Tips：若完成测试环境的配置后，无法通过客户端访问所挂载的客户端文件，可检查是否由防火墙的阻断导致。

## qtfs查询及控制工具

源码qtinfo目录下提供了qtfs的查询及控制工具qtinfo和qtcfg，该工具的编译过程如下：

```bash
1. cd qtfs/qtinfo

2. make role=client 或 make role=server，其中role按照当前节点属性进行设置
```

编译完成后生成qtinfo和qtcfg二进制，可以通过不加任何参数执行该二进制查看其用法。

## rexec使用

1. rexec工具依赖上述udsproxyd服务，使用udsproxyd提供的uds协同进行通信，请确认udsproxyd正常启动。

2. 为rexec\_server配置白名单，将qtfs/config/rexec/whitelist文件拷贝至/etc/rexec/下，请手动配置需要的白名单选项，在其中增加允许执行的二进制命令，请注意该白名单应该配置在rexec\_server运行的系统上，如果双向运行，则两侧都需要配置，rexec\_server服务只接受该白名单列出的二进制拉起执行，不在白名单中的请求会被rexec\_server拒绝拉起。

3. 为rexec端配置uds白名单（这个白名单只在需要调用rexec二进制的系统中配置，如果是双向则都配置），在udsconnect中增加rexec通信socket所在目录白名单，增加方式有两种：

   1. 在qtfs\_server端可以将/etc/qtfs/whitelist的`[udsconnect]`表项中增加`/var/run/rexec`，修改后需要重新启动engine进程使其生效。

   2. 使用前述的qtcfg进行配置：`qtcfg -w udsconnect -x /var/run/rexec`，配置完成后可通过`qtcfg -w udsconnect -z`查询是否生效。qtcfg可以在qtfs client或者server端动态添加白名单，qtfs server端应该在engine拉起后执行。

4. 拉起rexec\_server作为服务端，无需参数。

5. 通过rexec ${your\_cmd} 验证rexec功能是否正常。

   Tips：rexec的远程执行功能支持双向，需要在client和server都拉起rexec\_server服务。

## 无感卸载场景

qtfs可用于DPU管理面无感卸载场景，通过qtfs为卸载进程准备运行时工作目录、系统工作目录，并为卸载后管理进程与主机侧业务进程提供透明的本地通信接口。

qtfs提供的协同文件系统、协同IPC等能力尽可能做到系统通用，管理软件仍需要进行少量适配工作。相比通过拆分方案进行管理软件卸载，qtfs提供的无感卸载可以大幅降低业务修改适配的工作量（可将代码修改控制在几百行，拆分方案代码修改可达千行甚至万行级别）。

通过这种方案可大幅提升管理面卸载的兼容性，方便后续版本升级；另外该方案具备一定通用性，可适配虚拟化管理面卸载和容器管理面卸载等场景。

Tips：不同场景管理面进程使用qtfs进行无感卸载时，仍需要进行代码适配，用户需要具备场景专业知识，对管理代码工具进行适配，并使用合适的qtfs配置。

本文档后续章节介绍两个管理面工具无感卸载的推荐场景：[虚拟化管理面DPU无感卸载](./libvirt_direct_connection_aggregation_environment_establishment.md)及[容器管理面DPU无感卸载](./container_management_plane_direct_connection_aggregation_environment_establishment.md)。

Tips：上述无感卸载指导文档中提供的libvirt及docker适配patch仅供参考，不可用于商用环境；用户应基于自己的实际场景进行对应代码修改适配。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/diversified_computing/dpu_offload/qtfs_architecture_and_usage.md
---
# qtfs Shared File System Architecture and Usage

## Introduction

qtfs is a shared file system project. It can be deployed on either a host-DPU hardware architecture or on two hosts. qtfs works in client-server mode, allowing the client to access specified file systems on the server in the same way that local files are accessed.

qtfs provides the following features:

* Mount point propagation

* Sharing of special file systems such as proc, sys, and cgroup

* Shared read and write of remote files

* Remote mounting of server file systems on the client

* Customized processing of special files

* Remote FIFO, Unix sockets, and epoll that allow the client and server to access the files as if they were like local

* Bottom-layer host-DPU communication over the PCIe protocol, outperforming the network

* Kernel module development, preventing intrusive modification to the kernel

## Software Architecture

![qtfs-arch](./figures/qtfs-arch.png)

## Installation

Perform operations in the following qtfs-related directories:

* **qtfs**: code of the client kernel module. Compile the client **.ko** file in this directory.

* **qtfs\_server**: code of the server kernel module. Compile the server **.ko** files and related programs in this directory.

* **qtinfo**: diagnosis tool that is used to check the status of file systems and change the log level.

* **demo**, **test**, and **doc**: demo programs, test programs, and project documents.

* Root directory: code of common modules used by the client and server.

Configure the kernel compilation environment on two servers (or VMs).

1. The kernel version must be 5.10 or later.
2. Install the kernel development package by running `yum install kernel-devel`.
3. Assume that the host IP address is 192.168.10.10 and the DPU IP address is 192.168.10.11.

Install the qtfs server.

```bash
    1. cd qtfs_server
    2. make clean && make
    3. insmod qtfs_server.ko qtfs_server_ip=192.168.10.10 qtfs_server_port=12345 qtfs_log_level=WARN
    4. nohup ./engine 16 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 &
```

Install the qtfs client.

```bash
    1. cd qtfs
    2. make clean && make
    3. insmod qtfs.ko qtfs_server_ip=192.168.10.10 qtfs_server_port=12345 qtfs_log_level=WARN
    4. cd ../ipc/
    5. make clean && make && make install
    6. nohup udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 &
```

## Usage

After the installation is complete, mount the server file system to the client. For example:

```bash
    mount -t qtfs / /root/mnt/
```

The file system is visible to the client. Access **/root/mnt** on the client to view and operate files on the server.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/querying_information.md
---
# Querying Information

## Querying the Service Version

### Description

The  `isula version`  command is run to query the version of the iSulad service.

### Usage

```shell
isula version
```

### Example

Query the version information.

```shell
isula version
```

If the iSulad service is running properly, you can view the information about versions of the client, server, and  **OCI config**.

```text
Client:
  Version:      2.1.2
  Git commit:   cecc8ca30fde7700e97cea3151d2a7fee9d02b07
  Built:        2023-07-30T04:21:48.521198248-04:00

Server:
  Version:      2.1.2
  Git commit:   cecc8ca30fde7700e97cea3151d2a7fee9d02b07
  Built:        2023-07-30T04:21:48.521198248-04:00

OCI config:
  Version:      1.0.0-rc5-dev
  Default file: /etc/default/isulad/config.json
```

If the iSulad service is not running, only the client information is queried and a message is displayed indicating that the connection times out.

```text
Client:
  Version:      2.1.2
  Git commit:   cecc8ca30fde7700e97cea3151d2a7fee9d02b07
  Built:        2023-07-30T04:21:48.521198248-04:00

Can not connect with server.Is the iSulad daemon running on the host?
```

Therefore, the  `isula version`  command is often used to check whether the iSulad service is running properly.

## Querying System-level Information

### Description

The  `isula info`  command is run to query the system-level information, number of containers, and number of images.

### Usage

```shell
isula info
```

### Example

Query system-level information, including the number of containers, number of images, kernel version, and operating system (OS).

```shell
$ isula info
Containers: 2
 Running: 0
 Paused: 0
 Stopped: 2
Images: 8
Server Version: 2.1.2
Logging Driver: json-file
Cgroup Driverr: cgroupfs
Hugetlb Pagesize: 2MB
Kernel Version: 5.10.0-153.12.0.92.oe2203SP3.aarch64
Operating System: openEuler 22.03 (LTS-SP4)
OSType: Linux
Architecture: aarch64
CPUs: 4
Total Memory: 2 GB
Name: openEuler
iSulad Root Dir: /var/lib/isulad
```

---

---
url: /en/docs/22.03_LTS_SP4/server/quickstart/quick_start.md
---
# Quick Start

This document uses openEuler 22.03 LTS SP4 installed on the TaiShan 200 server as an example to describe how to quickly install and use openEuler OS. For details about the installation requirements and methods, see the [Installation Guide](./../installation_upgrade/installation/installation_guide.md).

## Making Preparations

* Hardware Compatibility

  [Table 1](#table14948632047)  describes the types of supported servers.

  **Table  1**  Supported servers

  | Server Type | Server Name | Server Model |
  | :----  | :---- |:---- |
  | Rack server| TaiShan 200 | 2280 balanced model |
  | Rack server | FusionServer Pro | FusionServer Pro 2288H V5  NOTE: The server must be configured with the Avago SAS3508 RAID controller card and the LOM-X722 NIC.|

* Minimum Hardware Specifications

  [Table 2](#tff48b99c9bf24b84bb602c53229e2541)  lists the minimum hardware specifications supported by openEuler.

  **Table  2**  Minimum hardware requirements

  | Component    | Minimum Hardware Specifications                                             | Description                                                                                                                                                                                                  |
  | :----------- | :-------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
  | Architecture | AArch64x86\_64                                    | 64-bit Arm architecture64-bit Intel x86 architecture                                                                                                                              |
  | CPU          | Huawei Kunpeng 920 seriesIntel ® Xeon® processor | -                                                                                                                                                                                                            |
  | Memory       | ≥ 4 GB (8 GB or higher recommended for better user experience)              | -                                                                                                                                                                                                            |
  | Drive        | ≥ 120 GB (for better user experience)                                       | IDE, SATA, and SAS drives are supported.A driver software is required to use the NVMe drive with the DIF feature. Contact the drive manufacturer if the feature is not available. |

## Obtaining the Installation Source

Perform the following operations to obtain the openEuler release package:

1. Visit the [openEuler](https://www.openeuler.org/en/) website.
2. Click  **Downloads**.
3. Click **Community Editions**. The version list is displayed.
4. Click **Download** on the right of **openEuler 22.03-LTS-SP4**.
5. Download the required openEuler release package and the corresponding verification file based on the architecture and scenario.
   1. If the architecture is AArch64:
      1. Click **AArch64**.
      2. For local installation, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP4-(everything-)aarch64-dvd.iso** to the local host.
      3. For network installation, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP4-netinst-aarch64-dvd.iso** to the local host.
   2. If the architecture is x86\_64:
      1. Click **x86\_64**.
      2. For local installation, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP4-(everything-)x86\_64-dvd.iso** to the local host.
      3. For network installation, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP4-netinst-x86\_64-dvd.iso** to the local host.

## Checking the Release Package Integrity

To prevent incomplete download of the software package due to network or storage device problems during the transmission, you need to check the integrity of the obtained openEuler software package.

### Prerequisites

The following files need to be prepared:

ISO file: openEuler-22.03-LTS-SP4-aarch64-dvd.iso

Verification file: Copy and save the SHA256 value corresponding to the ISO file.

### Verification Procedure

1. Calculate the SHA256 verification value of the file. Run the following command:

   ```shell
   sha256sum openEuler-22.03-LTS-SP4-aarch64-dvd.iso
   ```

   After the command is run, the verification value is displayed.

2. Check whether the verification value is the same as that in the local verification file.

   If the verification values are the same, the .iso file is not damaged. If they are not the same, the file is damaged and you need to obtain the file again.

## Starting Installation

1. Log in to the iBMC WebUI.

   For details, see  [TaiShan 200 Server User Guide (Model 2280)](https://support.huawei.com/enterprise/en/doc/EDOC1100093459).

2. Choose  **Configuration**  from the main menu, and select  **Boot Device**  from the navigation tree. The  **Boot Device**  page is displayed.

   Set  **Effective**  and  **Boot Medium**  to  **One-time**  and  **DVD-ROM**, respectively, and click  **Save**, as shown in  [Figure 1](#fig1011938131018).

   **Figure  1**  Setting the boot device&#x20;
   ![](./figures/setting-the-boot-device.png)

3. Choose  **Remote Console**  from the main menu. The  **Remote Console**  page is displayed.

   Select an integrated remote console as required to access the remote virtual console, for example,  **Java Integrated Remote Console (Shared)**.

4. On the toolbar, click the icon shown in the following figure.

   **Figure  2**  Drive icon\
   ![](./figures/drive-icon.png)

   An image dialog box is displayed, as shown in the following figure.

   **Figure  3**  Image dialog box\
   ![](./figures/image-dialog-box.png)

5. Select  **Image File**  and then click  **Browse**. The  **Open**  dialog box is displayed.

6. Select the image file and click  **Open**. In the image dialog box, click  **Connect**. If  **Connect**  changes to  **Disconnect**, the virtual CD/DVD-ROM drive is connected to the server.

7. On the toolbar, click the restart icon shown in the following figure to restart the device.

   **Figure  4**  Restart icon\
   ![](./figures/restart-icon.png)

8. A boot menu is displayed after the system restarts, as shown in  [Figure 5](#fig1648754873314).

   > \[!NOTE] **NOTE:**
   >
   > * If you do not perform any operations within 1 minute, the system automatically selects the default option  **Test this media & install openEuler 22.03\_LTS\_SP4**  and enters the installation page.
   > * During physical machine installation, if you cannot use the arrow keys to select boot options and the system does not respond after you press  **Enter**, click  ![](./figures/en-us_image_0229420473.png)  on the BMC page and configure  **Key & Mouse Reset**.

   **Figure  5**  Installation wizard\
   ![](./figures/Installation_wizard.png)

9. On the installation wizard page, press  **Enter**  to select the default option  **Test this media & install openEuler 22.03\_LTS\_SP4**  to enter the GUI installation page.

## Performing Installation

After entering the GUI installation page, perform the following operations to install the system:

1. Set an installation language. The default language is English. You can change the language based on the site requirements, as shown in  [Figure 6](#fig874344811484).

   **Figure  6**  Selecting a language\
   ![](./figures/selecting-a-language.png)

2. On the  **INSTALLATION SUMMARY**  page, set configuration items based on the site requirements.

   * A configuration item with an alarm symbol must be configured. When the alarm symbol disappears, you can perform the next operation.
   * A configuration item without an alarm symbol is configured by default.
   * You can click  **Begin Installation**  to install the system only when all alarms are cleared.

   **Figure  7**  Installation summary\
   ![](./figures/installation-summary.png)

   1. Select  **Software Selection**  to set configuration items.

      Based on the site requirements, select  **Minimal Install**  on the left box and select an add-on in the  **Add-Ons for Selected Environment**  area on the right, as shown in  [Figure 8](#fig1133717611109).

      **Figure  8**  Selecting installation software\
      ![](./figures/selecting-installation-software.png)

      > \[!NOTE] **NOTE:**
      >
      > * In  **Minimal Install**  mode, not all packages in the installation source are installed. If a required package is not installed, you can mount the installation source to the local host as a repo source, and use DNF to install the package.
      > * If you select  **Virtual Host**, the virtualization components QEMU, libvirt, and edk2 are installed by default. You can select whether to install components such as OVS in the add-on area.

      After the setting is complete, click  **Done**  in the upper left corner to go back to the  **INSTALLATION SUMMARY**  page.

   2. Select  **Installation Destination**  to set configuration items.

      On the  **INSTALLATION DESTINATION**  page, select a local storage device.

      > \[!TIP] **NOTICE:**
      >
      > * The NVMe data protection feature is not supported because the NVMe drivers built in the BIOSs of many servers are of earlier versions. (Data protection: Format disk sectors to 512+N or 4096+N bytes.) Therefore, when selecting a proper storage medium, do not select an NVMe SSD with data protection enabled as the system disk. Otherwise, the OS may fail to boot.
      > * You can consult the server vendor about whether the BIOS supports NVMe disks with data protection enabled as system disks. If you cannot confirm whether the BIOS supports NVMe disks with data protection enabled as system disks, you are not advised to use an NVMe disk to install the OS, or you can disable the data protection function of an NVMe disk to install the OS.

      You also need to configure the storage to partition the system. You can either manually configure partitions or select  **Automatic**  for automatic partitioning. Select  **Automatic**  if the system is installed in a new storage device or the data in the storage device is not required, as shown in  [Figure 9](#fig153381468101).

      **Figure  9**  Setting the installation destination\
      ![](./figures/setting-the-installation-destination.png)

      > \[!NOTE] **NOTE:**
      >
      > * During partitioning, to ensure system security and performance, you are advised to configure the following partitions:  **/boot**,  **/var**,  **/var/log**,  **/var/log/audit**,  **/home**, and  **/tmp**.
      > * If the system is configured with the **swap** partition, the **swap** partition is used when the physical memory of the system is insufficient. Although the **swap** partition can be used to expand the physical memory, when the **swap** partition is used due to insufficient memory, the system responds slowly and the system performance deteriorates. Therefore, you are advised not to configure the **swap** partition in a system with sufficient physical memory or a performance-sensitive system.
      > * If you need to split a logical volume group, select  **Custom**  to manually partition the logical volume group. On the  **MANUAL PARTITIONING**  page, click  **Modify**  in the  **Volume Group**  area to reconfigure the logical volume group.

      After the setting is complete, click  **Done**  in the upper left corner to go back to the  **INSTALLATION SUMMARY**  page.

   3. Select **Root Password** and set the root password.

      On the **ROOT PASSWORD** page, enter a password that meets the **Password Complexity** requirements and confirm the password, as shown in [Figure 10](#fig_root_password).

      > \[!NOTE] **NOTE:**
      >
      > * The **root** account is used to perform key system management tasks. You are not advised to use the **root** account for daily work or system access.
      >
      > * If you select **Lock root account** on the **Root Password** page, the **root** account will be disabled.

      **Password Complexity**&#x20;

      The password of the **root** user or a new user must meet the password complexity requirements. Otherwise, the password setting or user creation will fail. The password must meet the following requirements:

      1. Contains at least eight characters.

      2. Contains at least three of the following: uppercase letters, lowercase letters, digits, and special characters.

      3. Different from the user name.

      4. Not allowed to contain words in the dictionary.

      > \[!NOTE] **NOTE:**
      > In the openEuler environment, you can run the `cracklib-unpacker /usr/share/cracklib/pw_dict > dictionary.txt` command to export the dictionary library file **dictionary.txt**. You can check whether the password is in this dictionary.

      **Figure 10** Root password
      ![](./figures/password-of-the-root-account.png)

      After the settings are completed, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page.

   4. Select **Create a User** and set the parameters.

      [Figure 11](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319) shows the page for creating a user. Enter the user name and set the password. The password complexity requirements are the same as those of the root password. In addition, you can set the home directory and user group by clicking **Advanced**, as shown in [Figure 12](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319).

      **Figure 11** Creating a use\
      ![](./figures/creating-a-user.png)

      **Figure 12** Advanced user configuration\
      ![](./figures/advanced-user-configuration.png)

      After the settings are completed, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page.

   5. Set other configuration items. You can use the default values for other configuration items.

3. Click **Start the Installation** to install the system, as shown in [Figure 13](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319).

   **Figure 13** Starting the installation\
   ![](./figures/installation-process.png)

4. After the installation is completed, restart the system.

   openEuler has been installed. Click  **Reboot**  to restart the system.

## Viewing System Information

After the system is installed and restarted, the system CLI login page is displayed. Enter the username and password set during the installation to log in to openEuler and view the following system information. For details about system management and configuration, see the  [openEuler 22.03\_LTS\_SP4 Administrator Guide](https://openeuler.org/en/docs/22.03_LTS_SP4/docs/Administration/administration.html))).

* View the system information:

  ```shell
  $ cat /etc/os-release
  NAME="openEuler"
  VERSION="22.03 (LTS-SP4)"
  ID="openEuler"
  VERSION_ID="22.03"
  PRETTY_NAME="openEuler 22.03 (LTS-SP4)"
  ANSI_COLOR="0;31"
  ```

* View system resource information.

  View the CPU information.

  ```shell
  lscpu
  ```

  View the memory information.

  ```shell
  free
  ```

  View the drive information.

  ```shell
  fdisk -l
  ```

* View the IP addresses.

  ```shell
  ip addr
  ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/reboot_or_shutdown_in_a_container.md
---
# Reboot or Shutdown in a Container

## Function Description

The  **reboot**  and  **shutdown**  commands can be executed in a system container. You can run the  **reboot**  command to restart a container, and run the  **shutdown**  command to stop a container.

## Parameter Description

## Constraints

* The shutdown function relies on the actual OS of the container running environment.
* When you run the  **shutdown -h now**  command to shut down the system, do not open multiple consoles. For example, if you run the  **isula run -ti**  command to open a console and run the  **isula attach**  command for the container in another host bash, another console is opened. In this case, the  **shutdown**  command fails to be executed.

## Example

* Specify the  **--restart on-reboot**  parameter when starting a container. For example:

  ```sh
  [root@localhost ~]# isula run -tid --restart on-reboot --system-container --external-rootfs /root/myrootfs none init
  106faae22a926e22c828a0f2b63cf5c46e5d5986ea8a5b26de81390d0ed9714f
  ```

* In the container, run the  **reboot**  command.

  ```sh
  [root@localhost ~]# isula exec -it 10 bash
  [root@localhost /]# reboot
  ```

  Check whether the container is restarted.

  ```sh
  [root@localhost ~]# isula exec -it 10 ps aux
  USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
  root         1  0.1  0.0  21588  9504 ?        Ss   12:11   0:00 init
  root        14  0.1  0.0  27024  9376 ?        Ss   12:11   0:00 /usr/lib/system
  root        17  0.0  0.0  18700  5876 ?        Ss   12:11   0:00 /usr/lib/system
  dbus        22  0.0  0.0   9048  3624 ?        Ss   12:11   0:00 /usr/bin/dbus-d
  root        26  0.0  0.0   8092  3012 ?        Rs+  12:13   0:00 ps aux
  ```

* In the container, run the  **shutdown**  command.

  ```sh
  [root@localhost ~]# isula exec -it 10 bash
  [root@localhost /]# shutdown -h now
  [root@localhost /]# [root@localhost ~]#
  ```

  Check whether the container is stopped.

  ```sh
  [root@localhost ~]# isula exec -it 10 bash
  Error response from daemon: Exec container error;Container is not running:106faae22a926e22c828a0f2b63cf5c46e5d5986ea8a5b26de81390d0ed9714f
  ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/installation_upgrade/installation/more_resources.md
---
# References

* How to Create a Raspberry Pi Image File
* How to Use Raspberry Pi

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/resolved_issues.md
---
# Resolved Issues

For the complete issue list, see [Issues](https://gitee.com/organizations/src-openeuler/issues).

For the complete list of kernel related commits, see [Commits](https://gitee.com/openeuler/kernel/commits/openEuler-22.03-LTS-SP4).

For details about resolved issues, see [Table 1](#table2204014971491143).

**Table 1** Resolved issues

|ISSUE ID|Issue|Description|Repository|
|-|-|-|-|
| I9RF7L | <https://gitee.com/open_euler/dashboard?issue_id=I9RF7L> | \[EulerMaker] octave fails to be built in the openEuler-22.03-LTS-SP4:epol project. | octave |
| I9S7JR | <https://gitee.com/open_euler/dashboard?issue_id=I9S7JR> | \[EulerMaker] dde-network-core fails to be built in the openEuler-22.03-LTS-SP4:epol project. | dde-network-core |
| I9S7JY | <https://gitee.com/open_euler/dashboard?issue_id=I9S7JY> | \[EulerMaker] glib2 build error in the openEuler-22.03-LTS-SP4:everything project. | glib2 |
| I9SJPV | <https://gitee.com/open_euler/dashboard?issue_id=I9SJPV> | \[22.03-LTS-SP4-rc1] The dpu-utilities package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dpu-utilities |
| I9SJPX | <https://gitee.com/open_euler/dashboard?issue_id=I9SJPX> | \[22.03-LTS-SP4-rc1] The oemaker package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | oemaker |
| I9SJPZ | <https://gitee.com/open_euler/dashboard?issue_id=I9SJPZ> | \[22.03-LTS-SP4-rc1] The llvm-bolt package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | llvm-bolt |
| I9SJQ0 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ0> | \[22.03-LTS-SP4-rc1] The hadoop package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | hadoop |
| I9SJQ2 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ2> | \[22.03-LTS-SP4-rc1] The openEuler-release package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | openEuler-release |
| I9SJQ3 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ3> | \[22.03-LTS-SP4-rc1] The dde-session-shell package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-session-shell |
| I9SJQ4 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ4> | \[22.03-LTS-SP4-rc1] The dde package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde |
| I9SJQ5 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ5> | \[22.03-LTS-SP4-rc1] The dtkgui package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dtkgui |
| I9SJQ6 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ6> | \[22.03-LTS-SP4-rc1] The dtkcore package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dtkcore |
| I9SJQ9 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ9> | \[22.03-LTS-SP4-rc1] The deepin-anything package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-anything |
| I9SJQB | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQB> | \[22.03-LTS-SP4-rc1] The dde-kwin package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-kwin |
| I9SJQD | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQD> | \[22.03-LTS-SP4-rc1] The libmetal package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | libmetal |
| I9SJQE | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQE> | \[22.03-LTS-SP4-rc1] The dde-qt-dbus-factory package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-qt-dbus-factory |
| I9SJQF | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQF> | \[22.03-LTS-SP4-rc1] The deepin-compressor package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-compressor |
| I9SJQH | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQH> | \[22.03-LTS-SP4-rc1] The dde-clipboard package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-clipboard |
| I9SJQI | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQI> | \[22.03-LTS-SP4-rc1] The deepin-image-editor package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-image-editor |
| I9SJQL | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQL> | \[22.03-LTS-SP4-rc1] The dde-control-center package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-control-center |
| I9SJQN | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQN> | \[22.03-LTS-SP4-rc1] The deepin-wallpapers package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-wallpapers |
| I9SJQO | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQO> | \[22.03-LTS-SP4-rc1] The dde-session-ui package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-session-ui |
| I9SJQP | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQP> | \[22.03-LTS-SP4-rc1] The deepin-gtk-theme package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-gtk-theme |
| I9SJQQ | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQQ> | \[22.03-LTS-SP4-rc1] The deepin-log-viewer package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-log-viewer |
| I9SJQS | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQS> | \[22.03-LTS-SP4-rc1] The deepin-devicemanager package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-devicemanager |
| I9SJQU | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQU> | \[22.03-LTS-SP4-rc1] The deepin-system-monitor package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-system-monitor |
| I9SJQW | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQW> | \[22.03-LTS-SP4-rc1] The deepin-screen-recorder package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-screen-recorder |
| I9SJQY | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQY> | \[22.03-LTS-SP4-rc1] The dtkcommon package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dtkcommon |
| I9SJQZ | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQZ> | \[22.03-LTS-SP4-rc1] The deepin-default-settings package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-default-settings |
| I9SJR1 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR1> | \[22.03-LTS-SP4-rc1] The dde-api package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-api |
| I9SJR2 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR2> | \[22.03-LTS-SP4-rc1] The deepin-gettext-tools package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-gettext-tools |
| I9SJR3 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR3> | \[22.03-LTS-SP4-rc1] The startdde package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | startdde |
| I9SJR5 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR5> | \[22.03-LTS-SP4-rc1] The deepin-icon-theme package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-icon-theme |
| I9SJR6 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR6> | \[22.03-LTS-SP4-rc1] The deepin-terminal package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-terminal |
| I9SJR8 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR8> | \[22.03-LTS-SP4-rc1] The dde-daemon package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-daemon |
| I9SJRA | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRA> | \[22.03-LTS-SP4-rc1] The deepin-desktop-schemas package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-desktop-schemas |
| I9SJRB | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRB> | \[22.03-LTS-SP4-rc1] The dde-file-manager package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-file-manager |
| I9SJRC | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRC> | \[22.03-LTS-SP4-rc1] The deepin-editor package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-editor |
| I9SJRD | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRD> | \[22.03-LTS-SP4-rc1] The dde-launcher package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-launcher |
| I9SJRF | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRF> | \[22.03-LTS-SP4-rc1] The deepin-menu package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-menu |
| I9SJRG | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRG> | \[22.03-LTS-SP4-rc1] The dde-polkit-agent package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-polkit-agent |
| I9SJRH | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRH> | \[22.03-LTS-SP4-rc1] The dde-dock package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-dock |
| I9SJRJ | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRJ> | \[22.03-LTS-SP4-rc1] The deepin-pw-check package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-pw-check |
| I9SJRL | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRL> | \[22.03-LTS-SP4-rc1] The dtkwidget package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dtkwidget |
| I9SJRM | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRM> | \[22.03-LTS-SP4-rc1] The dde-calendar package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-calendar |
| I9SJZK | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZK> | \[22.03-LTS-SP4-rc1] The ctags package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ctags |
| I9SJZN | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZN> | \[22.03-LTS-SP4-rc1] The protobuf2 package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | protobuf2 |
| I9SJZP | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZP> | \[22.03-LTS-SP4-rc1] The exempi package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | exempi |
| I9SJZR | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZR> | \[22.03-LTS-SP4-rc1] The virt-manager package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | virt-manager |
| I9SJZS | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZS> | \[22.03-LTS-SP4-rc1] The python-mako package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | python-mako |
| I9SJZT | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZT> | \[22.03-LTS-SP4-rc1] The gcc package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | gcc |
| I9SJZU | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZU> | \[22.03-LTS-SP4-rc1] The python-bottle package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | python-bottle |
| I9SJZV | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZV> | \[22.03-LTS-SP4-rc1] The dbus package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dbus |
| I9SJZW | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZW> | \[22.03-LTS-SP4-rc1] The libcxxabi package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | libcxxabi |
| I9SJZX | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZX> | \[22.03-LTS-SP4-rc1] The sqlite package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | sqlite |
| I9SJZY | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZY> | \[22.03-LTS-SP4-rc1] The linuxdoc-tools package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | linuxdoc-tools |
| I9SK02 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK02> | \[22.03-LTS-SP4-rc1] The groovy package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | groovy |
| I9SK03 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK03> | \[22.03-LTS-SP4-rc1] The libwd package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | libwd |
| I9SK04 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK04> | \[22.03-LTS-SP4-rc1] The python-imagesize package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | python-imagesize |
| I9SK05 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK05> | \[22.03-LTS-SP4-rc1] The storm package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | storm |
| I9SK06 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK06> | \[22.03-LTS-SP4-rc1] The perl package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | perl |
| I9SK08 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK08> | \[22.03-LTS-SP4-rc1] The scap-security-guide package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | scap-security-guide |
| I9SK0A | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0A> | \[22.03-LTS-SP4-rc1] The redis6 package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | redis6 |
| I9SK0B | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0B> | \[22.03-LTS-SP4-rc1] The gradle package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | gradle |
| I9SK0D | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0D> | \[22.03-LTS-SP4-rc1] The libkae package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | libkae |
| I9SK0E | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0E> | \[22.03-LTS-SP4-rc1] The sysget package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | sysget |
| I9SK0G | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0G> | \[22.03-LTS-SP4-rc1] The libsrtp package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | libsrtp |
| I9SK0I | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0I> | \[22.03-LTS-SP4-rc1] The python-beaker package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | python-beaker |
| I9SK0J | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0J> | \[22.03-LTS-SP4-rc1] The unixODBC package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | unixODBC |
| I9SK0K | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0K> | \[22.03-LTS-SP4-rc1] The pytorch package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | pytorch |
| I9SK0O | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0O> | \[22.03-LTS-SP4-rc1] The groovy18 package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | groovy18 |
| I9SK0Q | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0Q> | \[22.03-LTS-SP4-rc1] The llvm-libunwind package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | llvm-libunwind |
| I9SK0S | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0S> | \[22.03-LTS-SP4-rc1] The criu package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | criu |
| I9SK0T | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0T> | \[22.03-LTS-SP4-rc1] The sysmaster package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | sysmaster |
| I9SK0U | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0U> | \[22.03-LTS-SP4-rc1] The python-flask-restful package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | python-flask-restful |
| I9SK0V | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0V> | \[22.03-LTS-SP4-rc1] The man-db package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | man-db |
| I9SK0Y | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0Y> | \[22.03-LTS-SP4-rc1] The python-sphinx-theme-alabaster package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | python-sphinx-theme-alabaster |
| I9SK0Z | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0Z> | \[22.03-LTS-SP4-rc1] The osinfo-db package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | osinfo-db |
| I9SK11 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK11> | \[22.03-LTS-SP4-rc1] The python-configshell package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | python-configshell |
| I9SK13 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK13> | \[22.03-LTS-SP4-rc1] The libcxx package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | libcxx |
| I9SK14 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK14> | \[22.03-LTS-SP4-rc1] The secGear package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | secGear |
| I9SK15 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK15> | \[22.03-LTS-SP4-rc1] The redis5 package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | redis5 |
| I9SK17 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK17> | \[22.03-LTS-SP4-rc1] The openEuler -repos package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | openEuler-repos |
| I9SK18 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK18> | \[22.03-LTS-SP4-rc1] The ft\_utils package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ft\_utils |
| I9SK1A | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1A> | \[22.03-LTS-SP4-rc1] The dde-introduction package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dde-introduction |
| I9SK1C | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1C> | \[22.03-LTS-SP4-rc1] The deepin-clone package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-clone |
| I9SK1D | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1D> | \[22.03-LTS-SP4-rc1] The shotwell package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | shotwell |
| I9SK1E | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1E> | \[22.03-LTS-SP4-rc1] The communication\_ipc package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | communication\_ipc |
| I9SK1F | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1F> | \[22.03-LTS-SP4-rc1] The ft\_wl\_fwk package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ft\_wl\_fwk |
| I9SK1I | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1I> | \[22.03-LTS-SP4-rc1] The ft\_mmi package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ft\_mmi |
| I9SK1J | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1J> | \[22.03-LTS-SP4-rc1] The k3s-containerd package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | k3s-containerd |
| I9SK1K | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1K> | \[22.03-LTS-SP4-rc1] The redshift package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | redshift |
| I9SK1M | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1M> | \[22.03-LTS-SP4-rc1] The ft\_multimedia package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ft\_multimedia |
| I9SK1N | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1N> | \[22.03-LTS-SP4-rc1] The k3s package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | k3s |
| I9SK1O | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1O> | \[22.03-LTS-SP4-rc1] The deepin-desktop-base package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-desktop-base |
| I9SK1P | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1P> | \[22.03-LTS-SP4-rc1] The ukui-settings-daemon package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ukui-settings-daemon |
| I9SK1S | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1S> | \[22.03-LTS-SP4-rc1] The systemabilitymgr\_safwk package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | systemabilitymgr\_safwk |
| I9SK1U | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1U> | \[22.03-LTS-SP4-rc1] The arkui-linux package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | arkui-linux |
| I9SK1V | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1V> | \[22.03-LTS-SP4-rc1] The filemanagement\_dfs\_service package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | filemanagement\_dfs\_service |
| I9SK1W | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1W> | \[22.03-LTS-SP4-rc1] The migration-tools package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | migration-tools |
| I9SKC6 | <https://gitee.com/open_euler/dashboard?issue_id=I9SKC6> | \[22.03-LTS-SP4-rc1] The pyflakes package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | pyflakes |
| I9SKC7 | <https://gitee.com/open_euler/dashboard?issue_id=I9SKC7> | \[22.03-LTS-SP4-rc1] The openjdk-latest package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | openjdk-latest |
| I9SKC9 | <https://gitee.com/open_euler/dashboard?issue_id=I9SKC9> | \[22.03-LTS-SP4-rc1] The qt5integration package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | qt5integration |
| I9SKCB | <https://gitee.com/open_euler/dashboard?issue_id=I9SKCB> | \[22.03-LTS-SP4-rc1] The aops-diana package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | aops-diana |
| I9SKCC | <https://gitee.com/open_euler/dashboard?issue_id=I9SKCC> | \[22.03-LTS-SP4-rc1] The qt5dxcb-plugin package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | qt5dxcb-plugin |
| I9SL18 | <https://gitee.com/open_euler/dashboard?issue_id=I9SL18> | \[22.03-LTS-SP4-rc1] The oec-hardware package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | oec-hardware |
| I9SL90 | <https://gitee.com/open_euler/dashboard?issue_id=I9SL90> | \[22.03-LTS-SP4-rc1] \[Am/x86] The PWR\_SYS\_GetRtPowerInfo interface is not open. | powerapi |
| I9SMBB | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBB> | \[22.03-LTS-SP4-rc1] The aops-ceres package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | aops-ceres |
| I9SMBC | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBC> | \[22.03-LTS-SP4-rc1] The busybox package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | busybox |
| I9SMBD | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBD> | \[22.03-LTS-SP4-rc1] The containers-common package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | containers-common |
| I9SMBF | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBF> | \[22.03-LTS-SP4-rc1] The e2fsprogs package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | e2fsprogs |
| I9SMBH | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBH> | \[22.03-LTS-SP4-rc1] The ebtables package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ebtables |
| I9SMBI | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBI> | \[22.03-LTS-SP4-rc1] The firewalld package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | firewalld |
| I9SMBJ | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBJ> | \[22.03-LTS-SP4-rc1] The gcc-cross package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | gcc-cross |
| I9SMBK | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBK> | \[22.03-LTS-SP4-rc1] The gdk-pixbuf2 package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | gdk-pixbuf2 |
| I9SMBL | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBL> | \[22.03-LTS-SP4-rc1] The imageTailor package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | imageTailor |
| I9SMBM | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBM> | \[22.03-LTS-SP4-rc1] The iproute package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | iproute |
| I9SMBO | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBO> | \[22.03-LTS-SP4-rc1] The iSulad package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | iSulad |
| I9SMBP | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBP> | \[22.03-LTS-SP4-rc1] The KubeOS package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | KubeOS |
| I9SMBQ | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBQ> | \[22.03-LTS-SP4-rc1] The libsndfile package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | libsndfile |
| I9SMBR | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBR> | \[22.03-LTS-SP4-rc1] The openstack-releases package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | openstack-releases |
| I9SQWH | <https://gitee.com/open_euler/dashboard?issue_id=I9SQWH> | \[22.03-LTS-SP4-rc1] \[Am/x86] The PWR\_CreateDcTask interface is called to create a task of an unsupported data type. | powerapi |
| I9SRCC | <https://gitee.com/open_euler/dashboard?issue_id=I9SRCC> | \[22.03\_SP4\_RC1\_everything] \[Arm\x86] An error message is displayed during the mandoc installation. | mandoc |
| I9STHL | <https://gitee.com/open_euler/dashboard?issue_id=I9STHL> | \[EulerMaker] python-beaker fails to be built in the openEuler-22.03-LTS-SP4:everything project. | python-beaker |
| I9SURK | <https://gitee.com/open_euler/dashboard?issue_id=I9SURK> | \[22.03\_SP4\_RC1\_epol] \[Arm\x86] An error message is displayed during the afterburn uninstallation. | afterburn |
| I9SUVR | <https://gitee.com/open_euler/dashboard?issue_id=I9SUVR> | \[openEuler-22.03-LTS-SP4] Failed to stop the storm-nimbus.service service. | storm |
| I9SUXW | <https://gitee.com/open_euler/dashboard?issue_id=I9SUXW> | \[22.03-LTS-SP4-rc1] The \[Compilation alarm check] \[Arm/x86] An alarm is generated during compilation. | powerapi |
| I9SV5C | <https://gitee.com/open_euler/dashboard?issue_id=I9SV5C> | \[22.03\_SP4\_RC1\_everything] \[Arm\x86] An error message is displayed during the devmaster installation. | sysmaster |
| I9SV9F | <https://gitee.com/open_euler/dashboard?issue_id=I9SV9F> | \[openEuler-22.03-LTS-SP4] Failed to stop the storm-supervisor.service service. | storm |
| I9T040 | <https://gitee.com/open_euler/dashboard?issue_id=I9T040> | \[22.03-LTS-SP4-rc1] \[x86/Arm] The luarocks source package fails to be compiled locally because the openresty and openresty-openssl111-devel dependencies are missing. | luarocks |
| I9T0OH | <https://gitee.com/open_euler/dashboard?issue_id=I9T0OH> | \[22.03-LTS-SP4-rc1] \[x86/Arm] The perl-Compress-Raw-Zlib source package fails to be compiled locally, and the check phase fails. | perl-Compress-Raw-Zlib |
| I9T5DF | <https://gitee.com/open_euler/dashboard?issue_id=I9T5DF> | \[22.03-LTS-SP4-rc1] \[Arm/x86] Keyword-based search by invoking the PWR\_PROC\_QueryProcs interface does not meet the expectation. | powerapi |
| I9T5Q8 | <https://gitee.com/open_euler/dashboard?issue_id=I9T5Q8> | \[22.03-LTS-SP4-rc1] \[x86] netdata -D reports core dump. | netdata |
| I9T5TN | <https://gitee.com/open_euler/dashboard?issue_id=I9T5TN> | \[22.03\_SP4\_RC1\_epol] \[Arm/x86] An exception occurs during the migration-tools-server upgrade. | migration-tools |
| I9T6FB | <https://gitee.com/open_euler/dashboard?issue_id=I9T6FB> | \[22.03\_SP4\_RC1\_everything] \[Arm/x86] obs-server fails to be uninstalled after the OS upgrades to SP4. | obs-server |
| I9T7M8 | <https://gitee.com/open_euler/dashboard?issue_id=I9T7M8> | \[22.03-LTS-SP4-rc1] \[Arm/x86] Fails to invoke the PWR\_PROC\_GetWattState interface for the first time, and error code 13 is returned. | powerapi |
| I9TNRP | <https://gitee.com/open_euler/dashboard?issue_id=I9TNRP> | \[22.03-LTS-SP4-rc1] \[Address sanitizer test] \[Arm/x86] The PWR\_CreateDcTask interface is called to create a task whose data type is 2. The task callback function is triggered, causing the service to break down. | powerapi |
| I9U0YX | <https://gitee.com/open_euler/dashboard?issue_id=I9U0YX> | \[openEuler -22.03-LTS-SP4 rc1] An error is reported after the sysmonitor.service service is started. | sysmonitor |
| I9U163 | <https://gitee.com/open_euler/dashboard?issue_id=I9U163> | \[22.03-LTS-SP4-rc1] \[x86/Arm] glassfish-jsp source package fails to be compiled locally. | glassfish-jsp |
| I9ULPM | <https://gitee.com/open_euler/dashboard?issue_id=I9ULPM> | \[22.03\_SP4\_RC2\_everything] \[Arm\x86] An exception message is displayed during obs-server downgrade. | obs-server |
| I9UPLT | <https://gitee.com/open_euler/dashboard?issue_id=I9UPLT> | \[22.03-LTS-SP4-rc2] \[Address sanitizer test] \[Arm/x86] When the TEST\_PWR\_PROC\_QueryProcs interface is called, the keyword is an empty string and **num** is **100** (construct 5,000 processes for cyclic printing). The interface calling times out, and error code 1 is returned. | powerapi |
| I9UQHL | <https://gitee.com/open_euler/dashboard?issue_id=I9UQHL> | \[22.03-LTS-SP4 RC2] \[deja] Some cases of aarch64-sve-acle.exp fail to be executed. | gcc |
| I9UVJP | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJP> | \[22.03-LTS-SP4-rc2] The abseil-cpp package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | abseil-cpp |
| I9UVJQ | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJQ> | \[22.03-LTS-SP4-rc2] The shim package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | shim |
| I9UVJR | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJR> | \[22.03-LTS-SP4-rc2] The spdk package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | spdk |
| I9UVJS | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJS> | \[22.03-LTS-SP4-rc2] The syscare package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | syscare |
| I9UVJT | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJT> | \[22.03-LTS-SP4-rc2] The libsepol package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | libsepol |
| I9UVJU | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJU> | \[22.03-LTS-SP4-rc2] The libxml2 package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | libxml2 |
| I9UVJV | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJV> | \[22.03-LTS-SP4-rc2] The kernel package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | kernel |
| I9UVJW | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJW> | \[22.03-LTS-SP4-rc2] The openssh package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | openssh |
| I9UVJY | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJY> | \[22.03-LTS-SP4-rc2] The ghostscript package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ghostscript |
| I9UVJZ | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJZ> | \[22.03-LTS-SP4-rc2] The libvirt package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | libvirt |
| I9UVK0 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK0> | \[22.03-LTS-SP4-rc2] The skopeo package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | skopeo |
| I9UVK2 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK2> | \[22.03-LTS-SP4-rc2] The dwarves package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | dwarves |
| I9UVK3 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK3> | \[22.03-LTS-SP4-rc2] The openjdk-11 is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | openjdk-11 |
| I9UVK4 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK4> | \[22.03-LTS-SP4-rc2] The bind package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | bind |
| I9UVK5 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK5> | \[22.03-LTS-SP4-rc2] The pcs package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | pcs |
| I9UVK6 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK6> | \[22.03-LTS-SP4-rc2] The bash package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | bash |
| I9UVK8 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK8> | \[22.03-LTS-SP4-rc2] The lwip package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | lwip |
| I9UVK9 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK9> | \[22.03-LTS-SP4-rc2] The lsof package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | lsof |
| I9UVKA | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKA> | \[22.03-LTS-SP4-rc2] The nautilus package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | nautilus |
| I9UVKB | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKB> | \[22.03-LTS-SP4-rc2] The util-linux package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | util-linux |
| I9UVKC | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKC> | \[22.03-LTS-SP4-rc2] The python-jinja2 package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | python-jinja2 |
| I9UVKD | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKD> | \[22.03-LTS-SP4-rc2] The deepin-turbo package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-turbo |
| I9UVKE | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKE> | \[22.03-LTS-SP4-rc2] The ovirt-engine package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ovirt-engine |
| I9UVKF | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKF> | \[22.03-LTS-SP4-rc2] The deepin-graphics-driver-manager package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | deepin-graphics-driver-manager |
| I9UVKG | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKG> | \[22.03-LTS-SP4-rc2] The ignition package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ignition |
| I9UVMX | <https://gitee.com/open_euler/dashboard?issue_id=I9UVMX> | \[22.03-LTS-SP4-rc2] The docker package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | docker |
| I9UVOU | <https://gitee.com/open_euler/dashboard?issue_id=I9UVOU> | \[22.03-LTS-SP4-rc2] The poissonsearch-oss package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | poissonsearch-oss |
| I9UVOV | <https://gitee.com/open_euler/dashboard?issue_id=I9UVOV> | \[22.03-LTS-SP4-rc2] The openjdk-17 package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | openjdk-17 |
| I9UWOZ | <https://gitee.com/open_euler/dashboard?issue_id=I9UWOZ> | \[22.03-LTS-SP4-rc2] The gazelle package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | gazelle |
| I9UWPT | <https://gitee.com/open_euler/dashboard?issue_id=I9UWPT> | \[22.03-LTS-SP4-rc2] The openjdk-1.8.0 package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | openjdk-1.8.0 |
| I9UWQK | <https://gitee.com/open_euler/dashboard?issue_id=I9UWQK> | \[22.03-LTS-SP4-rc2] "ICE:during GIMPLE pass: unswitch: internal compiler error: Segmentation fault" is displayed during the application compilation using auto-bolt and NPB. | gcc |
| I9V325 | <https://gitee.com/open_euler/dashboard?issue_id=I9V325> | \[22.03-LTS-SP4-rc2] \[Arm/x86] The PWR\_PROC\_SetWattAttrs interface is called with **domainMask** set to **1**. The query result is **10**. | powerapi |
| I9V34B | <https://gitee.com/open_euler/dashboard?issue_id=I9V34B> | \[22.03-LTS-SP4-rc2] "internal compiler error: Aborted : during GIMPLE pass: llc\_allocate" is reported during the postgresql-11.3 compilation using **\[codedb]-O2 -fllc-allocate**. | gcc |
| I9V3TI | <https://gitee.com/open_euler/dashboard?issue_id=I9V3TI> | \[22.03-LTS-SP4-rc2] **-O3 -fwhole-program -fllc-allocate** reports "Segmentation fault: during GIMPLE pass: llc\_allocate". | gcc |
| I9V738 | <https://gitee.com/open_euler/dashboard?issue_id=I9V738> | \[22.03\_SP4\_RC2\_everything] \[Arm\x86] The obs-api version conflict causes upgrade failures. | obs-server |
| I9V75L | <https://gitee.com/open_euler/dashboard?issue_id=I9V75L> | \[22.03-LTS-SP4-rc2] The host registration template does not contain the **ssh\_pkey** field. | aops-ceres |
| I9V7BM | <https://gitee.com/open_euler/dashboard?issue_id=I9V7BM> | \[22.03-LTS-SP4-rc2] \[Arm/x86] The PWR\_PROC\_SetSmartGridState interface is called, and error code 502 is returned. | powerapi |
| I9VAXG | <https://gitee.com/open_euler/dashboard?issue_id=I9VAXG> | \[22.03-LTS-SP4-rc2] \[Arm/x86] The PWR\_PROC\_AddWattProcs interface is called to add a non-existent process. The execution is successful, and the queried process is a random value. | powerapi |
| I9VPAX | <https://gitee.com/open_euler/dashboard?issue_id=I9VPAX> | \[22.03-LTS-SP4-rc2] \[x86] Check whether the libomp-test binary package needs to be deleted. | libomp |
| I9VPPE | <https://gitee.com/open_euler/dashboard?issue_id=I9VPPE> | \[22.03-LTS-SP4-rc2] \[Arm/x86] The repository lacks the arkui-linux binary package, and only the arkui-linux-devel binary package of the Arm architecture is available, which is inconsistent with that of EBS. | arkui-linux |
| I9VR7L | <https://gitee.com/open_euler/dashboard?issue_id=I9VR7L> | \[22.03-LTS-SP4-rc2] The message for creating a hot patch removal task is incorrect. | aops-hermes |
| I9VVTI | <https://gitee.com/open_euler/dashboard?issue_id=I9VVTI> | \[22.03-LTS-SP4-rc2] The message for creating a REPO setting task is incorrect. | aops-hermes |
| I9W0J1 | <https://gitee.com/open_euler/dashboard?issue_id=I9W0J1> | \[22.03-LTS-SP4-rc2] \[Arm/x86] After the eagle service is started, the log contains the error message "/etc/eagle/plugin/libidle\_service.so: No such file or directory". | eagle |
| IA4DBK | <https://gitee.com/open_euler/dashboard?issue_id=IA4DBK> | \[22.03-LTS-SP4-rc3] Whether to name the utshell software package in the same way as other packages in the openEuler community. | utshell |
| IA4MIA | <https://gitee.com/open_euler/dashboard?issue_id=IA4MIA> | \[22.03-LTS-SP4-rc3] \[x86/Arm] Failed to execute the **luarocks** command. | luarocks |
| IA4MKD | <https://gitee.com/open_euler/dashboard?issue_id=IA4MKD> | \[22.03-LTS-SP4-rc3] \[Arm/x86] Install the software package and run the **oeawarectl --help** command. Error message "error while loading shared libraries: libyaml-cpp.so.0.6: cannot open shared object file: No such file or directory" is displayed. | oeAware-manager |
| IA4QCT | <https://gitee.com/open_euler/dashboard?issue_id=IA4QCT> | \[22.03-LTS-SP4-rc3] \[Arm/x86] Install the software package, enable plugin instances with dependencies, and disable one of the dependent instances. The command is executed successfully, but the instance fails to be disabled. | oeAware-manager |
| IA4R47 | <https://gitee.com/open_euler/dashboard?issue_id=IA4R47> | \[22.03-LTS-SP4-rc3] \[Arm/x86] Install the software package, set **enable\_list**, set **name** to plugin that does not exist, set **instances** to an existing instance, and restart the service. The plugin is enabled successfully. | oeAware-manager |
| IA55IU | <https://gitee.com/open_euler/dashboard?issue_id=IA55IU> | \[22.03-LTS-SP4-rc3] The docker-runc package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | runc |
| IA57HR | <https://gitee.com/open_euler/dashboard?issue_id=IA57HR> | \[22.03-LTS-SP4-rc3] \[Arm/x86] Service logs need to be optimized. | oeAware-manager |
| IA57S9 | <https://gitee.com/open_euler/dashboard?issue_id=IA57S9> | \[22.03-LTS-SP4-rc3] \[Arm/x86] **/etc/eagle/eagle\_config.ini** does not contain the parameter configuration description. | eagle |
| IA58XV | <https://gitee.com/open_euler/dashboard?issue_id=IA58XV> | \[22.03-LTS-SP4-rc3] \[Arm/x86] yaml-cpp fails to parse the YAML configuration file. | yaml-cpp |
| IA5BWS | <https://gitee.com/open_euler/dashboard?issue_id=IA5BWS> | \[22.03-LTS-SP4-rc3] \[Arm/x86] The oeAware series software packages are installed, **libthread\_collector.so** is deleted, the service is started, and the dependencies become abnormal. | oeAware-manager |
| IA5EDP | <https://gitee.com/open_euler/dashboard?issue_id=IA5EDP> | \[EulermMaker] openEuler 22.03-LTS-SP4 stratovirt image build fails. | kernel |
| IA5MCM | <https://gitee.com/open_euler/dashboard?issue_id=IA5MCM> | \[22.03-LTS-SP4-rc3] \[Arm/x86] The **eagle\_policy.ini** file configuration does not take effect. | eagle |
| IA5NUJ | <https://gitee.com/open_euler/dashboard?issue_id=IA5NUJ> | \[22.03-LTS-SP4-rc3] \[Arm/x86] Change **watt\_threshold** and **watt\_domain\_mask** to **0** in the policy configuration file and restart the service. The two parameters do not restore to the default values. | eagle |
| IA5Z8U | <https://gitee.com/open_euler/dashboard?issue_id=IA5Z8U> | \[EulerMaker] k3s fails to be built in the openEuler-22.03-LTS-SP4:epol project. | k3s |
| IA60UN | <https://gitee.com/open_euler/dashboard?issue_id=IA60UN> | \[22.03-LTS-SP4-rc4] The python-pip package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | python-pip |
| IA60UO | <https://gitee.com/open_euler/dashboard?issue_id=IA60UO> | \[22.03-LTS-SP4-rc4] The ghostscript package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ghostscript |
| IA60UP | <https://gitee.com/open_euler/dashboard?issue_id=IA60UP> | \[22.03-LTS-SP4-rc4] The bash package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | bash |
| IA60UQ | <https://gitee.com/open_euler/dashboard?issue_id=IA60UQ> | \[22.03-LTS-SP4-rc4] The kbd package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | kbd |
| IA60UR | <https://gitee.com/open_euler/dashboard?issue_id=IA60UR> | \[22.03-LTS-SP4-rc4] The ignition package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | ignition |
| IA613G | <https://gitee.com/open_euler/dashboard?issue_id=IA613G> | \[22.03-LTS-SP4-rc4] The xorg-x11-server package is downgraded in 22.03 LTS SP4 compared to 22.03 LTS SP3. | xorg-x11-server |
| IA6148 | <https://gitee.com/open_euler/dashboard?issue_id=IA6148> | \[22.03-LTS-SP4 RC4] **-O3 -march=armv8.2-a+sve -fllc-allocate --param branch-prob-threshold=50**, but the assembly file does not generate the prf(x) instruction. | gcc |
| IA68T7 | <https://gitee.com/open_euler/dashboard?issue_id=IA68T7> | \[22.03-LTS-SP4-rc4] \[Arm/x86] After the eagle service is started, an additional setting is added before eagle in the admin configuration of **/etc/sysconfig/pwrapis/pwrapis\_config.ini**. | eagle |

---

---
url: /en/docs/22.03_LTS_SP4/edge_computing/ros/ros_user_guide.md
---
# ROS User Guide

This document introduces the installation, deployment, and usage of Robot Operating System (ROS) on openEuler.

This document is intended for developers, open-source enthusiasts, and partners who use the openEuler system and want to know and use ROS. You need to have basic knowledge of the Linux OS.

---

---
url: /zh/docs/22.03_LTS_SP4/edge_computing/ros/ros_user_guide.md
---
# ROS用户指南

本文档介绍openEuler系统上ROS（英语：Robot Operating System，一般译为机器人操作系统）的安装部署与使用方法，以指导用户快速了解并使用ROS。

本文档适用于使用openEuler系统并希望了解和使用ROS的社区开发者、开源爱好者以及相关合作伙伴。使用人员需要具备基本的Linux操作系统知识。

---

---
url: /en/docs/22.03_LTS_SP4/cloud/hybrid_deployment/rubik/configuration.md
---
# Rubik Configuration Description

The Rubik program is written in Go and compiled into a static executable file to minimize the coupling with the system.

## Commands

Besides the `-v` option for querying version information, Rubik does not support other options. The following is an example of version query output:

```bash
$ ./rubik -v
Version:       2.0.0
Release:       3.oe2203SP3
Go Version:    go1.18.8
Git Commit:    bcaace8
Built:         2023-03-30
OS/Arch:       linux/amd64
```

## Configuration

When the Rubik binary file is executed, Rubik parses configuration file **/var/lib/rubik/config.json**.

> Custom configuration file path is currently not supported to avoid confusion.
> When Rubik runs as a Daemonset in a Kubernetes cluster, modify the ConfigMap in the **hack/rubik-daemonset.yaml** file to configure Rubik.

The configuration file is in JSON format and keys are in lower camel case.

An example configuration file is as follows:

```json
{
  "agent": {
    "logDriver": "stdio",
    "logDir": "/var/log/rubik",
    "logSize": 2048,
    "logLevel": "info",
    "cgroupRoot": "/sys/fs/cgroup",
    "enabledFeatures": [
      "preemption",
      "dynCache",
      "ioLimit",
      "ioCost",
      "quotaBurst",
      "quotaTurbo",
      "psi"
    ]
  },
  "preemption": {
    "resource": [
      "cpu",
      "memory"
    ]
  },
  "quotaTurbo": {
    "highWaterMark": 50,
    "syncInterval": 100
  },
  "dynCache": {
    "defaultLimitMode": "static",
    "adjustInterval": 1000,
    "perfDuration": 1000,
    "l3Percent": {
      "low": 20,
      "mid": 30,
      "high": 50
    },
    "memBandPercent": {
      "low": 10,
      "mid": 30,
      "high": 50
    }
  },
  "ioCost": [
    {
      "nodeName": "k8s-single",
      "config": [
        {
          "dev": "sdb",
          "enable": true,
          "model": "linear",
          "param": {
            "rbps": 10000000,
            "rseqiops": 10000000,
            "rrandiops": 10000000,
            "wbps": 10000000,
            "wseqiops": 10000000,
            "wrandiops": 10000000
          }
        }
      ]
    }
  ],
  "psi": {
    "interval": 10,
    "resource": [
      "cpu",
      "memory",
      "io"
    ],
    "avg10Threshold": 5.0
  }
}
```

Rubik configuration items include common items and feature items. Common items are under the **agent** section and are applied globally. Feature items are applied to sub-features that are enabled in the **enabledFeatures** field under **agent**.

### agent

The **agent** section stores common configuration items related to Rubik running, such as log configurations and cgroup mount points.

| Key\[=Default Value]       | Type   | Description                                                                 | Example Value        |
| ------------------------- | ---------- | -------------------------------------- | --------------------------- |
| logDriver=stdio           | string     | Log driver, which can be the standard I/O or file           | stdio, file                 |
| logDir=/var/log/rubik     | string     | Log directory                           | Anu readable and writable directory              |
| logSize=1024              | int        | Total size of logs in MB when logDriver=file | \[10, $2^{20}$]              |
| logLevel=info             | string     | Log level                           | debug,info,warn,error       |
| cgroupRoot=/sys/fs/cgroup | string     | Mount point of the system cgroup                   | Mount point of the system cgroup        |
| enabledFeatures=\[]        | string array | List of Rubik features to be enabled                | Rubik features. See [Feature Introduction](./feature_introduction.md) for details. |

### preemption

The **preemption** field stores configuration items of the absolute preemption feature, including CPU and memory preemption. You can configure this field to use either or both of CPU and memory preemption.

| Key\[=Default Value]       | Type   | Description                                                                 | Example Value        |
| --------------- | ---------- | -------------------------------- | ----------- |
| resource=\[]     | string array | Resource type to be accessed | cpu, memory |

### dynCache

The **dynCache** field stores configuration items related to pod memory bandwidth and last-level cache (LLC) limits. **l3Percent** indicates the watermarks of each LLC level. **memBandPercent** indicates watermarks of memory bandwidth in MB.

| Key\[=Default Value]       | Type   | Description                                                                 | Example Value        |
| ----------------------- | ------ | ------------------ | --------------- |
| defaultLimitMode=static | string |  dynCache control mode | static, dynamic |
| adjustInterval=1000      | int    | Interval for dynCache control, in milliseconds| \[10, 10000] |
| perfDuration=1000        | int    | perf execution duration for dynCache, in milliseconds | \[10, 10000] |
| l3Percent                | map    | Watermarks of each L3 cache level of dynCache in percents|      |
| .low=20                  | int    |  Watermark of the low L3 cache level | \[10, 100]     |
| .mid=30                  | int    | Watermark of the middle L3 cache level  | \[low, 100]   |
| .high=50                 | int    | Watermark of the high L3 cache level  | \[mid, 100]   |
| memBandPercent           | map    | Watermarks of each memory bandwidth level of dynCache in percents|   |
| .low=10                  | int    | Watermark of the low bandwidth level in MB | \[10, 100]  |
| .mid=30                  | int    | Watermark of the middle bandwidth level in MB  | \[low, 100]   |
| .high=50                 | int    | Watermark of the high bandwidth level in MB | \[mid, 100]   |

### quotaTurbo

The **quotaTurbo** field stores configuration items of the user-mode elastic traffic limiting feature.

| Key\[=Default Value]       | Type   | Description                                                                 | Example Value        |
| ----------------- | ------ | -------------------------------- | -------------------- |
| highWaterMark=60  | int | High watermark of CPU load         |\[0, alarmWaterMark) |
| alarmWaterMark=80 | int | Alarm watermark of CPU load | (highWaterMark,100]            |
| syncInterval=100  | int | Interval for triggering container quota updates, in milliseconds | \[100,10000] |

### ioCost

The **ioCost** field stores configuration items of the iocost-based I/O weight control feature. The field is an array whose elements are names of nodes (**nodeName**) and their device configuration arrays (**config**).

| Key       | Type   | Description                                                                 | Example Value        |
| ----------------- | ------ | -------------------------------- | -------------------- |
| nodeName  | string | Node name         | Kubernetes cluster node name |
| config | array | Configurations of a block device |   /       |

**config** parameters of a block device:

| Key\[=Default Value] | Type   | Description                                                                 | Example Value        |
| --------------- | ------ | --------------------------------------------- | -------------- |
| dev             | string | Physical block device name                    | /              |
| model           | string | iocost model                                | linear         |
| param           | /      | Device parameters specific to the model               | /              |

For the **linear** model, the **param** field supports the following parameters:

| Key\[=Default Value] | Type   | Description                                                                 | Example Value        |
| --------------- | ---- | ---- | ------ |
| rbps            | int64  | Maximum read bandwidth     | (0, $2^{63}$) |
| rseqiops        | int64  | Maximum sequential read IOPS  | (0, $2^{63}$) |
| rrandiops       | int64  | Maximum random read IOPS | (0, $2^{63}$) |
| wbps            | int64  | Maximum write bandwidth     | (0, $2^{63}$) |
| wseqiops        | int64  | Maximum sequential write IOPS | (0, $2^{63}$) |
| wrandiops       | int64  | Maximum random write IOPS | (0, $2^{63}$) |

### psi

The **psi** field stores configuration items of the PSI-based interference detection feature. This feature can monitor CPUs, memory, and I/O resources.You can configure this field to monitor the PSI of any or all of the resources.

| Key\[=Default Value] | Type   | Description                                                                 | Example Value        |
| --------------- | ---------- | -------------------------------- | ----------- |
| interval=10 |int|Interval for PSI monitoring, in seconds| \[10,30]|
| resource=\[]     | string array | Resource type to be accessed | cpu, memory, io |
| avg10Threshold=5.0     | float | Average percentage of blocking time of a job in 10 seconds. If this threshold is reached, offline services are evicted. | \[5.0,100]|

---

---
url: /en/docs/22.03_LTS_SP4/cloud/hybrid_deployment/rubik/overview.md
---
# Rubik User Guide

## Overview

Low server resource utilization has always been a recognized challenge in the industry. With the development of cloud native technologies, hybrid deployment of online (high-priority) and offline (low-priority) services becomes an effective means to improve resource utilization.

In hybrid service deployment scenarios, Rubik can properly schedule resources based on Quality if Service (QoS) levels to greatly improve resource utilization while ensuring the quality of online services.

Rubik supports the following features:

* [Absolute preemption](./feature_introduction.md#absolute-preemption)
  * [CPU absolute preemption](./feature_introduction.md#cpu-absolute-preemption)
  * [Memory absolute preemption](./feature_introduction.md#memory-absolute-preemption)
* [dynCache memory bandwidth and L3 cache access limit](./feature_introduction.md#dyncache-memory-bandwidth-and-l3-cache-access-limit)
* [dynMemory tiered memory reclamation](./feature_introduction.md#dynmemory-tiered-memory-reclamation)
* [Flexible bandwidth](./feature_introduction.md#flexible-bandwidth)
  * [quotaBurst kernel-mode solution](./feature_introduction.md#quotaburst-kernel-mode-solution)
  * [quotaTurbo user-mode solution](./feature_introduction.md#quotaturbo-user-mode-solution)
* [I/O weight control based on ioCost](feature_introduction.md#io-weight-control-based-on-iocost)
* [Interference detection based on pressure stall information metrics](./feature_introduction.md#interference-detection-based-on-pressure-stall-information-metrics)

This document is intended for community developers, open source enthusiasts, and partners who use the openEuler system and want to learn and use Rubik. Users must:

* Know basic Linux operations.
* Be familiar with basic operations of Kubernetes and Docker/iSulad.

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/hybrid_deployment/rubik/overview.md
---
# rubik 使用指南

## 概述

如何改善服务器资源利用率低的现状一直是业界公认的难题，随着云原生技术的发展，将在线（高优先级）、离线（低优先级）业务混合部署成为了当下提高资源利用率的有效手段。

rubik 容器调度在业务混合部署的场景下，根据 QoS 分级，对资源进行合理调度，从而实现在保障在线业务服务质量的前提下，大幅提升资源利用率。

rubik 当前支持如下特性：

* [preemption 绝对抢占](./feature_introduction.md#preemption-绝对抢占)
  * [CPU绝对抢占](./feature_introduction.md#cpu绝对抢占)
  * [内存绝对抢占](./feature_introduction.md#内存绝对抢占)
* [dynCache 访存带宽和LLC限制](./feature_introduction.md#dyncache-访存带宽和llc限制)
* [dynMemory 内存异步分级回收](./feature_introduction.md#dynmemory-内存异步分级回收)
* [支持弹性限流](./feature_introduction.md#支持弹性限流)
  * [quotaBurst 支持弹性限流内核态解决方案](./feature_introduction.md#quotaburst-内核态解决方案)
  * [quotaTurbo 支持弹性限流用户态解决方案](./feature_introduction.md#quotaturbo-用户态解决方案)
* [ioCost 支持iocost对IO权重控制](./feature_introduction.md#iocost-支持iocost对io权重控制)
* [PSI 支持基于PSI指标的干扰检测](./feature_introduction.md#psi-支持基于psi指标的干扰检测)
* [CPU驱逐水位线控制](./feature_introduction.md#cpu驱逐水位线控制)
* [内存驱逐水位线控制](./feature_introduction.md#内存驱逐水位线控制)

本文档适用于使用 openEuler 系统并希望了解和使用 rubik 的社区开发者、开源爱好者以及相关合作伙伴。使用人员需要具备以下经验和技能：

* 熟悉 Linux 基本操作
* 熟悉 kubernetes 和 docker/iSulad 基本操作

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/hybrid_deployment/rubik/configuration.md
---
# Rubik配置说明

rubik执行程序由Go语言实现，并编译为静态可执行文件，以便尽可能与系统依赖解耦。

## 命令

Rubik仅支持 使用`-v` 参数查询版本信息，不支持其他参数。
版本信息输出示例如下所示，该信息中的内容和格式可能随着版本发生变化。

```bash
$ ./rubik -v
Version:       2.0.1
Release:       2.oe2403sp1
Go Version:    go1.22.1
Git Commit:    bcaace8
Built:         2024-12-10
OS/Arch:       linux/amd64
```

## 配置

执行rubik二进制时，rubik首先会解析配置文件，配置文件的路径固定为`/var/lib/rubik/config.json`。

> \[!NOTE]说明
>
> * 为避免配置混乱，暂不支持指定其他路径。
> * ubik支持以daemonset形式运行在kubernetes集群中。我们提供了yaml脚本（`hack/rubik-daemonset.yaml`），并定义了`ConfigMap`作为配置。因此，以daemonset形式运行rubik时，应修改`hack/rubik-daemonset.yaml`中的相应配置。

配置文件采用json格式，字段键采用驼峰命名规则，且首字母小写。
配置文件示例内容如下：

```json
{
  "agent": {
    "logDriver": "stdio",
    "logDir": "/var/log/rubik",
    "logSize": 2048,
    "logLevel": "info",
    "cgroupRoot": "/sys/fs/cgroup",
    "enabledFeatures": [
      "preemption",
      "dynCache",
      "ioLimit",
      "ioCost",
      "quotaBurst",
      "quotaTurbo",
      "psi",
      "cpuevict",
      "memoryevict"
    ]
  },
  "preemption": {
    "resource": [
      "cpu",
      "memory"
    ]
  },
  "quotaTurbo": {
    "highWaterMark": 50,
    "syncInterval": 100
  },
  "dynCache": {
    "defaultLimitMode": "static",
    "adjustInterval": 1000,
    "perfDuration": 1000,
    "l3Percent": {
      "low": 20,
      "mid": 30,
      "high": 50
    },
    "memBandPercent": {
      "low": 10,
      "mid": 30,
      "high": 50
    }
  },
  "ioCost": [
    {
      "nodeName": "k8s-single",
      "config": [
        {
          "dev": "sdb",
          "enable": true,
          "model": "linear",
          "param": {
            "rbps": 10000000,
            "rseqiops": 10000000,
            "rrandiops": 10000000,
            "wbps": 10000000,
            "wseqiops": 10000000,
            "wrandiops": 10000000
          }
        }
      ]
    }
  ],
  "psi": {
    "interval": 10,
    "resource": [
      "cpu",
      "memory",
      "io"
    ],
    "avg10Threshold": 5.0
  },
  "cpuevict": {
    "threshold": 60,
    "interval": 1,
    "windows": 2,
    "cooldown": 20
  },
  "memoryevict": {
    "threshold": 60,
    "interval": 1,
    "cooldown": 4
  }
}
```

Rubik配置分为两类：通用配置和特性配置。通用配置由agent关键字标识，用于保存全局的配置。特性配置按服务类型区分，应用于各个子特性。特性配置必须在通用配置的`enabledFeatures`字段中声明方可使用。

### agent

`agent`配置用于记录保存rubik运行的通用配置，例如日志、cgroup挂载点等信息。

| 配置键\[=默认值]           | 类型       | 描述                                   | 可选值                      |
| ------------------------- | ---------- | -------------------------------------- | --------------------------- |
| logDriver=stdio           | string     | 日志驱动，支持标准输出和文件           | stdio, file                 |
| logDir=/var/log/rubik     | string     | 日志保存目录                           | 可读可写的目录              |
| logSize=1024              | int        | 日志限额，单位MB，仅logDriver=file生效 | \[10, $2^{20}$]              |
| logLevel=info             | string     | 输出日志级别                           | debug,info,warn,error       |
| cgroupRoot=/sys/fs/cgroup | string     | 系统cgroup挂载点路径                   | 系统cgroup挂载点路径        |
| enabledFeatures=\[]        | string数组 | 需要使能的rubik特性列表                | rubik支持特性，参见特性介绍 |

### preemption

`preemption`字段用于标识绝对抢占特性配置。目前，Preemption特性支持CPU和内存的绝对抢占，用户可以按需配置该字段，单独或组合使用资源的绝对抢占。

| 配置键\[=默认值] | 类型       | 描述                             | 可选值      |
| --------------- | ---------- | -------------------------------- | ----------- |
| resource=\[]     | string数组 | 资源类型，声明何种资源需要被访问 | cpu, memory |

### dynCache

`dynCache`字段用于标识支持Pod访存带宽和LLC限制特性配置。`l3Percent`字段用于标识最后一级缓存（LLC）水位控制线，`memBandPercent`字段用于标识访存带宽（MB）水位控制线。

| 配置键\[=默认值]         | 类型   | 描述               | 可选值          |
| ----------------------- | ------ | ------------------ | --------------- |
| defaultLimitMode=static | string | dynCache的控制模式 | static, dynamic |
| adjustInterval=1000     | int | dynCache动态控制间隔时间，单位ms| \[10, 10000] |
| perfDuration=1000       | int | dynCache性能perf执行时长，单位ms | \[10, 10000] |
| l3Percent               | map | dynCache控制中L3各级别对应水位（%）| / |
| .low=20                 | int | L3 Cache低水位组控制线 | \[10, 100] |
| .mid=30                 | int | L3 Cache中水位组控制线 | \[low, 100] |
| .high=50                | int | L3 Cache高水位组控制线 | \[mid, 100]|
| memBandPercent          | map | dynCache控制中MB各级别对应水位（%）|/|
| .low=10                 | int | MB（访存带宽）低水位组控制线 | \[10, 100]|
| .mid=30                 | int | MB中水位组控制线 | \[low, 100] |
| .high=50                | int | MB高水位组控制线 | \[mid, 100] |

### quotaTurbo

`quotaTurbo`字段用于标识支持弹性限流技术（用户态）配置。

| 配置键\[=默认值]   | 类型   | 描述                             | 可选值               |
| ----------------- | ------ | -------------------------------- | -------------------- |
| highWaterMark=60  | int | CPU负载的高水位值         |\[0,警戒水位) |
| alarmWaterMark=80 | int | CPU负载的警戒水位 | (高水位,100]            |
| syncInterval=100  | int | 触发容器quota值更新的间隔（单位：毫秒） | \[100,10000] |

### ioCost

`ioCost`字段用于标识支持iocost对IO权重控制特性配置。其类型为数组，数组中的每一个元素由节点名称`nodeName`和设备参数数组`config`组成。

| 配置键  | 类型   | 描述                             | 可选值               |
| ----------------- | ------ | -------------------------------- | -------------------- |
| nodeName  | string | 节点名称         | kubernetes中节点名称 |
| config | 数组 | 单个设备的配置信息 |   /       |

单个块设备配置`config`参数：

| 配置键\[=默认值] | 类型   | 描述                                          | 可选值         |
| --------------- | ------ | --------------------------------------------- | -------------- |
| dev             | string | 块设备名称，仅支持物理设备                    | /              |
| model           | string | iocost模型名                                | linear         |
| param           | /      | 设备参数，根据不同模型有不同参数               | /              |

模型为linear时，`param`字段支持如下参数：

| 配置键\[=默认值] | 类型 | 描述 | 可选值 |
| --------------- | ---- | ---- | ------ |
|rbps            | int64  | 块设备最大读带宽     | （0, $2^{63}$) |
| rseqiops        | int64  | 块设备最大顺序读iop  | （0, $2^{63}$) |
| rrandiops       | int64  | 块设备最大随机读iops | （0, $2^{63}$) |
| wbps            | int64  | 块设备最大写带宽     | （0, $2^{63}$) |
| wseqiops        | int64  | 块设备最大顺序写iops | （0, $2^{63}$) |
| wrandiops       | int64  | 块设备最大随机写iops | （0, $2^{63}$) |

### psi

`psi`字段用于标识基于psi指标的干扰检测特性配置。目前，psi特性支持监测CPU、内存和I/O资源，用户可以按需配置该字段，单独或组合监测资源的PSI取值。

| 配置键\[=默认值] | 类型       | 描述                             | 可选值      |
| --------------- | ---------- | -------------------------------- | ----------- |
| interval=10 |int|psi指标监测间隔（单位：秒）| \[10,30]|
| resource=\[]     | string数组 | 资源类型，声明何种资源需要被访问 | cpu, memory, io |
| avg10Threshold=5.0     | float | psi some类型资源平均10s内的压制百分比阈值（单位：%），超过该阈值则驱逐离线业务 | \[5.0,100]|

### CPU驱逐水位线控制

`cpuevict`字段用于标识CPU驱逐水位线控制特性配置。该特性依照指定采样间隔采集节点CPU利用率，并统计指定窗口内的CPU平均利用率。若CPU平均利用率大于驱逐水位线，则驱逐离线Pod。一旦rubik驱逐离线Pod，则在冷却时间内不再驱逐Pod。

| 配置键\[=默认值] | 类型       | 描述                             | 可选值      |
| --------------- | ---------- | -------------------------------- | ----------- |
| threshold=60 | int | 窗口期内平均CPU利用率的阈值(%)，超过该阈值，则驱逐离线Pod | \[1,99]|
| interval=1   | int | 节点CPU利用率采集间隔(s) | \[1, 3600] |
| windows=2    | int | 节点平均CPU利用率的窗口时间（s）。窗口必须大于interval。若未设置windows，则windows设置为interval的两倍 | \[1, 3600]|
| cooldown=20  | int | 冷却时间（s），两次驱逐之间至少需要间隔冷却时间 | \[1, 9223372036854775806]|

### 内存驱逐水位线控制

`memoryevict`字段用于标识内存驱逐水位线控制特性配置。该特性依照指定采样间隔采集节点内存利用率。若节点内存利用率大于驱逐水位线，则驱逐离线Pod。一旦rubik驱逐离线Pod，则在冷却时间内不再驱逐Pod。

| 配置键\[=默认值] | 类型       | 描述                             | 可选值      |
| --------------- | ---------- | -------------------------------- | ----------- |
| threshold | int | 内存利用率的阈值(%)，超过该阈值，则驱逐离线Pod。若不指定该值，则无法使用本功能。 | \[1,99]|
| interval=1 | int | 节点CPU利用率采集间隔(s) | \[1, 3600] |
| cooldown=4 | int | 冷却时间（s），两次驱逐之间至少需要间隔冷却时间 | \[1, 9223372036854775806]|

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/running_the_test_pod.md
---
# Running the Test Pod

## Configuration File

```bash
$ cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
```

## Starting the Pod

Run the kubectl command to run Nginx.

```bash
$ kubectl apply -f nginx.yaml
deployment.apps/nginx-deployment created
$ kubectl get pods
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-66b6c48dd5-6rnwz   1/1     Running   0          33s
nginx-deployment-66b6c48dd5-9pq49   1/1     Running   0          33s
nginx-deployment-66b6c48dd5-lvmng   1/1     Running   0          34s
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/security/safeguard/safeguard_user_guide.md
---
# safeguard 用户指南

## 配置

safeguard 的配置文件是一个YAML格式的文件，包含了`key: value` 或者 `key: [value list]` 的键值对。

## 配置选项

| Config | Type | Description |
|:------:|:----|:-----------:|
| `network` | List  | Rule for network restrictions. |
| `files` | List  | Rule for file access restrictions. |
| `process` | List  | Rule for process restrictions. |
| `mount` | List  | Rule for mount restrictions. |
| `dns_proxy` | List  | DNS Proxy configurations |
| `log` | List containing the following sub-keys: `format: [json\|text]``output: <path>``max_size:`: Maximum size to rotate (MB). Default: 100MB`max_age`: Period for which logs are kept. Default: 365`labels`: Key / Value to be added to the log.| Log configuration. |

## network

| Config | Type | Description |
|:------:|:----|:-----------:|
| `enable` | Enum with the following possible values: `true`, `false` | Whether to enable restrictions or not. Default is `true`. |
| `mode` | Enum with the following possible values: `monitor`, `block` | If `monitor` is specified, events are only logged. If `block` is specified, network access is blocked. |
| `target` | Enum with the following possible values: `host`, `container` | Selecting `host` applies the restriction to the host-wide. Selecting `container` will apply the restriction only to containers. |
| `cidr` | List containing the following sub-keys:`allow: [cidr list]``deny: [cidr list]`| Allow or Deny CIDRs. |
| `domain` | List containing the following sub-keys:`allow: [domain list]``deny: [domain list]`| Allow or Deny Domains. |
| `command` | List containing the following sub-keys:`allow: [command list]``deny: [command list]`| Allow or Deny commands. |
| `uid` | List containing the following sub-keys:`allow: [uid list]``deny: [uid list]`| Allow or Deny uids. |
| `gid` | List containing the following sub-keys:`allow: [gid list]``deny: [gid list]`| Allow or Deny gids. |

### 示例

#### Allow all network connections

Allows all network communications and monitors their connections.

```yaml
network:
  mode: monitor
  target: host
  cidr:
    allow: ['0.0.0.0/0']
```

#### Block specify Private Networks

Block access to `192.168.1.1/24` and `10.0.1.1/24`.

```yaml
network:
  mode: block
  target: host
  cidr:
    allow: ['0.0.0.0/0']
    deny:
      - 192.168.1.1/24
      - 10.0.1.1/24
```

#### Block Metadata service API

Block access to the public cloud Metadata Service. This is a mitigation measure against SSRF, etc.

```yaml
network:
  mode: block
  target: host
  cidr:
    allow: ['0.0.0.0/0']
    deny:
      - 169.254.169.254/32
```

#### Block connections to the specified domain

Block connections to `example.com`. safeguard periodically looks up IP addresses, so it keeps up with IP address changes.

```yaml
network:
  mode: block
  target: host
  cidr:
    allow: ['0.0.0.0/0']
  domain:
    deny:
      - example.com
```

#### Block network connections of containers

Allow communication from the host, but block communication from the containers.

```yaml
network:
  mode: block
  target: container
  cidr:
    allow: ['0.0.0.0/0']
  domain:
    deny:
    - example.com
```

!!! example

```shell
vagrant@ubuntu-impish:~$ curl -I https://example.com
HTTP/2 200

vagrant@ubuntu-impish:~$ sudo docker run --rm -it curlimages/curl https://example.com
curl: (7) Couldn't connect to server
```

#### Block all connections from curl

```yaml
network:
  mode: monitor
  target: container
  cidr:
    allow: ['0.0.0.0/0']
  command:
    deny: ['curl']
```

!!! example

```shell
vagrant@ubuntu-impish:~$ curl -I https://example.com
curl: (6) Could not resolve host: example.com

vagrant@ubuntu-impish:~$ wget https://example.com -O /dev/null
--2022-03-09 14:45:11--  http://example.com/
Resolving example.com (example.com)... 93.184.216.34
Connecting to example.com (example.com)|93.184.216.34|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1256 (1.2K) [text/html]
Saving to: ‘/dev/null’

/dev/null               100%[============================>]   1.23K  --.-KB/s    in 0s

2022-03-09 14:45:12 (70.1 MB/s) - ‘/dev/null’ saved [1256/1256]
```

#### Block all connections by users with UID 1000

Setting that blocks all network access for UID 1000 user, but does not apply restrictions to UID 0 (root).

```yaml
network:
  mode: monitor
  target: container
  cidr:
    allow: ['0.0.0.0/0']
  uid:
    allow: [0]
    deny: [1000]
```

!!! example

```shell
vagrant@ubuntu-impish:~$ id
uid=1000(vagrant) gid=1000(vagrant) groups=1000(vagrant)

vagrant@ubuntu-impish:~$ curl -I https://example.com
curl: (6) Could not resolve host: example.com

vagrant@ubuntu-impish:~$ sudo curl -I https://example.com
HTTP/2 200
```

## files

Linux Kernel >= 5.13 is required to use this option.

| Config | Type | Description |
|:------:|:----|:-----------:|
| `enable` | Enum with the following possible values: `true`, `false` | Whether to enable restrictions or not. Default is `true`. |
| `mode` | Enum with the following possible values: `monitor`, `block` | If `monitor` is specified, events are only logged. If `block` is specified, network access is blocked. |
| `target` | Enum with the following possible values: `host`, `container` | Selecting `host` applies the restriction to the host-wide. Selecting `container` will apply the restriction only to containers. |
| `allow` | A list of allow file paths | |
| `deny` | A list of allow file paths | |

### 示例

#### Allow access to all files

```yaml
file:
  mode: monitor
  target: host
  allow:
    - /
```

#### Block access to `/etc/passwd`

```yaml
file:
  mode: block
  target: host
  allow:
    - /
  deny:
    - /etc/passwd
```

#### Block all access to the `/root/.ssh` directory

```yaml
file:
  mode: block
  target: host
  allow:
    - /
  deny:
    - /root/.ssh
```

#### Block access to the `/proc/sys` directory in the container

```yaml
file:
  mode: block
  target: container
  allow:
    - /
  deny:
    - /proc/sys
```

!!! example

```shell
root@ubuntu-impish:/# ls /proc/sys
abi  debug  dev  fs  kernel  net  user  vm

root@ubuntu-impish:/# docker run --privileged --rm -it ubuntu:latest bash
root@9cf961922b00:/# ls /proc/sys
ls: cannot open directory '/proc/sys': Operation not permitted
```

#### Block escapes from Privileged Container

```yaml
file:
  mode: block
  target: container
  allow:
    - /
  deny:
    - /proc/sysrq-trigger
    - /sys/kernel
    - /proc/sys/kernel
```

!!! example

```shell
root@ubuntu-impish:/# docker run --privileged --rm -it ubuntu:latest bash
root@e3b2ffe5b284:/# echo c > /proc/sysrq-trigger
bash: /proc/sysrq-trigger: Operation not permitted

root@e3b2ffe5b284:/# echo '/path/to/evil' > /sys/kernel/uevent_helper
bash: /sys/kernel/uevent_helper: Operation not permitted

root@e3b2ffe5b284:/# echo '|/path/to/evil' > /proc/sys/kernel/core_pattern
bash: /proc/sys/kernel/core_pattern: Operation not permitted
```

## process

| Config | Type | Description |
|:------:|:----|:-----------:|
| `enable` | Enum with the following possible values: `true`, `false` | Whether to enable restrictions or not. Default is `true`. |
| `mode` | Enum with the following possible values: `monitor` | If `monitor` is specified, events are only logged.  |
| `target` | Enum with the following possible values: `host`, `container` | Selecting `host` applies the restriction to the host-wide. Selecting `container` will apply the restriction only to containers. |

### 示例

```yaml
mount:
  mode: monitor
  target: host
```

## mount

| Config | Type | Description |
|:------:|:----|:-----------:|
| `enable` | Enum with the following possible values: `true`, `false` | Whether to enable restrictions or not. Default is `true`. |
| `mode` | Enum with the following possible values: `monitor`, `block` | If `monitor` is specified, events are only logged. If `block` is specified,  access is blocked. |
| `target` | Enum with the following possible values: `host`, `container` | Selecting `host` applies the restriction to the host-wide. Selecting `container` will apply the restriction only to containers. |
| `deny` | A list of allow mount paths | |

### 示例

#### Block mount `/var/run/docker.sock` to container

```yaml
mount:
  mode: block
  target: host
  deny:
    - /var/run/docker.sock
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/security/safeguard/about_safeguard.md
---
# safeguard简介

针对操作系统、内核安全，safeguard 是一个基于 eBPF 的 Linux 安全防护系统，可以实现安全操作的拦截及审计记录。项目采用 libbpfgo 库，使用go语言实现顶层控制。目前项目已在 openEuler sig-ebpf 社区开源，链接：<https://atomgit.com/openeuler/safeguard>。

## KRSI(eBPF+LSM)

eBPF 是扩展的伯克利包过滤器（extended Berkeley Packet Filter）的缩写，它是一种可以在内核空间运行沙箱化程序的技术。eBPF 程序可以在不修改或重新编译内核，也不需要加载内核模块的情况下，动态地增加内核的能力。通过 eBPF，可以实现网络、观测、跟踪和安全等多种用例。

LSM 是 Linux 安全模块（Linux Security Module）的缩写，它是一种提供可插拔的安全框架的机制，可以让不同的安全模块在内核中注册并实施自己的安全策略。LSM 提供了一系列的钩子（hooks），可以在系统调用或其他关键操作之前或之后执行安全检查。

eBPF 和 LSM 可以结合使用，形成一种基于 eBPF 的 LSM 扩展，叫做 KRSI(eBPF+LSM)。它允许用户在运行时使用 eBPF 程序实现和执行自定义的安全策略和审计规则。它的优点是不需要修改或重新编译内核，也不需要配置现有的 LSM 模块。它的工作原理是将 eBPF 程序加载到 LSM 钩子中，然后在调用路径中执行这些程序，对系统资源的访问进行检查和控制。

## 特性

* 审计：记录配置文件范围内的行为，并输出日志。
* 控制：针对文件，进程，网络的安全访问控制。
* 行为分析：收集信息，进行资源，热点，异常等分析。

## 应用场景

safeguard 是一种基于 KRSI(eBPF+LSM) 的 Linux 安全审计和管控解决方案，可以实现对系统的全面监控和保护。下面是一些可能的应用场景：

* 容器安全：safeguard 可以对容器内部的行为进行审计和控制，例如记录容器的进程、文件、网络活动，限制容器访问特定的资源或端口，检测容器的异常行为等。这样可以有效地防止容器被恶意攻击或滥用，提高容器的安全性和稳定性。
* 云服务安全：safeguard 可以对云服务提供商的客户机进行审计和控制，例如记录客户机的操作系统、应用程序、用户等信息，限制客户机执行特定的命令或系统调用，检测客户机的恶意行为或漏洞利用等。这样可以有效地保护云服务提供商的资源和信誉，防止客户机被黑客入侵或破坏。
* 安全合规： safeguard 可以对系统的安全合规性进行审计和控制，例如记录系统的配置、权限、日志等信息，限制系统修改特定的设置或文件，检测系统的违规行为或异常事件等。这样可以有效地满足各种安全标准和法规要求，提高系统的可信度和合法性。

## 项目功能

### 审计控制

文件：

* 追踪文件系统的活动，包括文件的打开、关闭、读写、删除等。
* 修改文件系统的行为，例如拦截某些文件操作，或者实现自定义的**安全策略**。

安全策略：

```
1. 拦截或重定向某些文件操作，使用 eBPF 来拦截对敏感文件的读写操作，或者重定向对某些文件的访问到其他位置。
2. 实现自定义的访问控制，使用 eBPF 来检查对文件的访问者的身份、权限、环境等信息，然后根据一些规则来允许或拒绝访问。
3. 实现自定义的审计和监控，使用 eBPF 来记录对某些文件的操作的详细信息，如操作者、时间、内容等，并将这些信息输出到日志。
```

进程：

* 追踪进程的生命周期，例如进程的创建、终止、调度、上下文切换等。
* 修改进程的行为，例如注入或修改某些系统调用，或者实现自定义的调度策略。

网络：

* 追踪网络的活动，例如网络包的发送、接收、转发、丢弃等。
* 修改网络的行为，例如过滤或重写某些网络包，或者实现自定义的路由策略。

## 特性列表

### 针对文件，网络，进程的审计控制

* 文件

  功能：
  配置允许访问的文件列表。
  配置禁止访问的文件列表并拦截相关操作。
  文件打开的日志记录，包括操作命令，主机名称，PID，UID 等信息。

  影响的操作：
  打开文件或目录，包括创建、读取、写入、执行等模式。
  修改文件或目录的权限或属性。
  映射文件或目录到内存。

* 网络

  功能：
  可配置 CIDR 允许以及拒绝的列表。
  可配置域名允许以及拒绝的列表。
  可配置操作命令允许以及拒绝的列表。
  可配置 UID 允许以及拒绝的列表。
  可配置 GID 允许以及拒绝的列表。
  网络连接的日志记录，包括 CGroupID，PID，主机名，操作命令等。

  影响的操作：
  网络连接相关操作，创建 socket。
  当一个 socket 调用 connect() 或 sendto() 函数时，且目标地址不是 NULL。
  当一个 socket 调用 accept()或recvfrom() 函数时，且源地址不是 NULL。
  当一个 socket 调用 getpeername() 函数时，且返回的地址不是 NULL。

* 进程

  功能：

  进程创建的日志记录，包括 PID，PPID，主机名，操作命令等。

  影响的操作包括：

  进程创建相关操作，包括 fork，vfork，clone 等。

### 日志

配置日志格式为 json 或 text。
配置日志文件输出路径。
配置日志文件轮转大小，超出后自动创建新的日志文件，默认大小为100M。
配置日志文件轮转时间限制，超出后自动创建新的日志文件，默认时间为365天。
配置日志标签。
配置日志级别，DEBUG（10）、INFO（20）、WARNING（30）、ERROR（40）、CRITICAL（50）。

### 配置

配置文件，进程，网络，日志模块的打开或关闭以及对应模块子配置。
配置模式，可以选择监控或拦截，只有在拦截模式下才会阻止相关操作（如拒绝访问的文件列表）。

---

---
url: /en/docs/22.03_LTS_SP4/server/security/sbom/sbom.md
---
# SBOM User Guide

## 1. Introduction to SBOM

A Software Bill of Materials (SBOM) serves as a formal, machine-readable inventory that uniquely identifies software components and their contents. Beyond basic identification, it tracks copyright and licensing details. Organizations use SBOM to enhance supply chain transparency, and it is rapidly becoming a mandatory deliverable in software distribution.

## 2. SBOM Core Requirements

The National Telecommunications and Information Administration (NTIA) has established baseline requirements for SBOM implementation. These essential data elements enable component tracking throughout the software supply chain and serve as the foundation for extended features such as license tracking and vulnerability monitoring.

| Core Field                      | Definition                                                   |
| ------------------------------- | ------------------------------------------------------------ |
| Supplier                        | Entity responsible for component creation and identification |
| Component                       | Official designation of the software unit                    |
| Version                         | Tracking identifier for component iterations                 |
| Other identifiers               | Supplementary reference keys                                 |
| Dependencies                    | Mapping of component relationships and inclusions            |
| SBOM author                     | Entity generating the SBOM documentation                     |
| Timestamp                       | SBOM generation date and time                                |
| **Recommended Optional Fields** |                                                              |
| Component hash                  | Digital fingerprint for security verification                |
| Lifecycle phase                 | Development stage at SBOM creation                           |

## 3. openEuler SBOM Implementation

openEuler's SBOM framework incorporates extensive metadata tracking through SPDX, including:

| Base Field          | SPDX Path                                                                                                                                                              |
| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Supplier            | document->packages->supplier                                                                                                                                           |
| Name                | document->packages->name                                                                                                                                               |
| Version             | document->packages->versionInfo (epoch:version-release in openEuler)                                                                                                   |
| Other identifiers   | document->packages->externalRefs->purl                                                                                                                                 |
| Dependencies        | document->packages->externalRefs->purl                                                                                                                                 |
| SBOM author         | document->creationInfo->creators                                                                                                                                       |
| Timestamp           | document->creationInfo->created                                                                                                                                        |
| Component hash      | document->packages->checksums                                                                                                                                          |
| Lifecycle phase     | Not supported                                                                                                                                                          |
| Other relationships | Internal subcomponents: document->packages->externalRefs(category:PROVIDE\_MANAGER)->purlRuntime dependencies: document->relationships(relationshipType:DEPENDS\_ON) |
| License info        | document->packages->licenseDeclared document->packages->licenseConcluded                                                                                               |
| Copyright info      | document->packages->copyrightText                                                                                                                                      |
| Upstream community  | document->packages->externalRefs(category:SOURCE\_MANAGER)->url                                                                                                         |
| Patch information   | Patch files: document->files(fileTypes:SOURCE)Patch relationships: document->relationships(relationshipType:PATCH\_APPLIED)                                         |
| Component source    | document->packages->downloadLocation                                                                                                                                   |
| Component details   | document->packages->description document->packages->summary                                                                                                            |
| Website/Blog        | document->packages->homepage                                                                                                                                           |

## 4. SBOM Structure

The system uses RPM packages as the fundamental unit for SBOM generation and analysis.

![](./figures/image.png)

---

---
url: /zh/docs/22.03_LTS_SP4/server/security/sbom/sbom.md
---
# SBOM用户指南

## 1. SBOM介绍

SBOM是一种正式标准化的、机器可读的元数据，它唯一地标识软件组件及其内容；也可能包括版权和许可证等成分数据。SBOM旨在跨组织共享，有助于提供软件供应链成分清单与透明度，并且未来趋势将作为软件交付件必要清单。

## 2. SBOM最小集定义

美国国家电信和信息管理局(National Telecommunications and Information Administration)发布SBOM最小集的定义： 数据字段是关于必须捕获和维护每个组件的基础数据，以便在整个软件供应链中跟踪组件，并基于此扩展License和漏洞库等其他数据字段。

| 数据字段       | 描述                                                         |
| :------------- | :----------------------------------------------------------- |
| 供应商名称     | 创建、定义和标识组件的实体的名称。                           |
| 组件名称       | 分配给原始供应商定义的软件单元的名称。                       |
| 组件的版本     | 组件版本号、供应商用来指定软件从先前标识的版本发生变化的标识符。 |
| 其它唯一标识符 | 用于标识组件或用作相关数据库的查找键的其他标识符。           |
| 依赖关系       | 软件依赖关系、表征上游组件 X 包含在软件 Y 中的关系           |
| SBOM数据的作者 | 为此组件创建SBOM数据的实体的名称。                           |
| 时间戳         | 记录SBOM数据组装的日期和时间。                               |
| **推荐的数据** |                                                              |
| 组件的哈希     | 组件的唯一哈希，以帮助允许列表或拒绝列表。                   |
| 生命周期阶段   | SDLC 中捕获 SBOM 数据的获取的阶段。                          |

## 3. openEuler发布的SBOM字段说明

| 最小集数据字段    | SPDX                                                         |
| ----------------- | ------------------------------------------------------------ |
| 组件供应商名称    | document->packages->supplier                                 |
| 组件名称          | document->packages->name                                     |
| 组件版本          | document->packages->versionInfo（openEuler使用了epoch:version-release格式） |
| 组件其他唯一标识  | document->packages->externalRefs(category:PACKAGE\_MANAGER)->purl |
| 组件依赖关系      | document->packages->externalRefs(category:EXTERNAL\_MANAGER)->purl |
| SBOM数据作者      | document->creationInfo->creators                             |
| SBOM时间戳        | document->creationInfo->created                              |
| 组件的哈希        | document->packages->checksums                                |
| 生命周期阶段      | 未支持                                                       |
| 其他组件关系      | 内部子组件：document->packages->externalRefs(category:PROVIDE\_MANAGER)->purl 运行时依赖：document->relationships(relationshipType:DEPENDS\_ON) |
| 组件License信息   | document->packages->licenseDeclared document->packages->licenseConcluded |
| 组件Copyright信息 | document->packages->copyrightText                            |
| 组件上游社区信息  | document->packages->externalRefs(category:SOURCE\_MANAGER)->url |
| 组件补丁信息      | 补丁文件：document->files(fileTypes:SOURCE) 补丁关系：document->relationships(relationshipType:PATCH\_APPLIED) |
| 组件来源          | document->packages->downloadLocation                         |
| 组件信息          | document->packages->description document->packages->summary  |
| 组件官网/博客     | document->packages->homepage                                 |

## 4. SBOM文件示例

解析最小颗粒度是RPM包

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secdetector/install_secdetector.md
---
# secDetector Installation

## Software and Hardware Requirements

### Hardware Requirements

* x86\_64 or AArch64 processors
* Drive: 1 GB or more
* Memory: 100 MB or more

### OS Requirements

openEuler 22.03 LTS SP4 or later

### Environment Setup

Install the openEuler OS. For details, see the [openEuler Installation Guide](./../../installation_upgrade/installation/installation_guide.md).

## secDetector Installation

1. Configure the openEuler yum repository. Since openEuler 22.03 LTS has been configured with the yum repository by default, no additional operation is required. In special cases, configure the online yum repository by referring to the openEuler official document or configure the local yum repository by mounting an ISO file.

2. Install secDetector.

   ```shell
   #Install secDetector.
   sudo yum install secDetector
   ```

> \[!NOTE]NOTE:
>
> After secDetector is installed, you can obtain the following files required for deploying secDetector:

```shell
#Core framework of the kerneldriver of secDetector
/lib/modules/%{kernel_version}/extra/secDetector/secDetector_core.ko

#Functional component of the kerneldriver of secDetector
/lib/modules/%{kernel_version}/extra/secDetector/secDetector_xxx.ko

#Daemon process file of secDetector
/usr/bin/secDetectord

#SDK library files of secDetector
/usr/lib64/secDetector/libsecDetectorsdk.so
/usr/include/secDetector/secDetector_sdk.h
/usr/include/secDetector/secDetector_topic.h
```

## secDetector Deployment

secDetectord, the main body of secDetector, is deployed as a system service. The foreground service system can communicate with secDetectord by integrating the SDK. Because some of the capabilities of secDetector must be built in the kernel, the full set of functions of secDetectord also depends on its background driver.

### Deploying the Kerneldriver

1. Insert **secDetector\_core.ko**, the basic framework of the kernel driver. It must be deployed prior to other kernel modules. Find the **secDetector\_core.ko** directory after the installation and insert it into the kernel. The command is as follows:

   ```shell
   sudo insmod secDetector_core.ko
   ```

   **secDetector\_core** supports a command line parameter **ringbuf\_size**. You can specify the value of this parameter to control the buffer size of the data channel between the kerneldriver and secDetectord in user space. This parameter can be set to an integer ranging from 4 to 1024, in MB. The default value is **4**. The value must be a power of 2. The command is as follows:

   ```shell
   sudo insmod secDetector_core.ko ringbuf_size=128
   ```

2. Insert the functional modules of the kerneldriver, which are deployed in modular mode. You can deploy required functional modules based on the framework or deploy all modules. The command is as follows:

   ```shell
   sudo insmod secDetector_kmodule_baseline.ko

   sudo insmod secDetector_memory_corruption.ko

   sudo insmod secDetector_program_action.ko

   sudo insmod secDetector_xxx.ko
   ```

   * **secDetector\_kmodule\_baseline.ko** detects the kernel module list and is a memory modification probe.
   * **secDetector\_memory\_corruption.ko** detects memory modifications and is a memory modification probe.
   * **secDetector\_program\_action.ko** detects program behavior and is a program behavior probe.

### Deploying the usrdriver and observer\_agent

The usrdriver and the observer\_agent service have been integrated into secDetectord. The following command is for reference:

```shell
sudo ./secDetectord &
```

The usrdriver provides file operation probes and process management probes.

secDetectord supports the following configuration options:

```shell
Syntax: secDetectord [Option]
By default, secDetectord runs in the background, obtains data from probes, and forwards the data to subscribers.
Options:
  -d         Enter the debug mode in the foreground, and print the probe data on the console.
  -s <size>  Size of the eBPF buffer, in Mb. The default value is 4. The value of size ranges from 4 to 1024 and must be a power of 2. There are two independent buffers.
  -t <topic> Events to be subscribed to. By default, all events are subscribed to. A topic is in bitmap format. For example, -t 0x60 subscribes to process creation and exit events at the same time. For details, see include/secDetector_topic.h.
```

### SDK Deployment

By default, the library files of the SDK are deployed in the system library directory. You only need to reference the header files of the SDK in your program.

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secdetector/using_secdetector.md
---
# secDetector Usage

secDetector provides an SDK, that is, an **.so** library. Users can integrate the dynamic link library (DLL) into their applications to use secDetector through APIs. This chapter describes how to use the SDK.

## How to Use

After secDetector is installed by referring to [secDetector Installation](./install_secdetector.md), **libsecDetectorsdk.so**, **secDetector\_sdk.h**, and **secDetector\_topic.h** are deployed in the default path of the system user library.

1. After ensuring that the **include** path is included in the application developed using C or C++, reference the two header files in the application.

   ```c
   #include <secDetector/secDetector_topic.h>
   #include <secDetector/secDetector_sdk.h>
   ```

2. Call APIs provided by the SDK to access secDetector by referring to [API Reference](./api_reference.md).

   1. Call the subscription API secSub to subscribe to the required topics.
   2. Call the message reading API secReadFrom in an independent thread to read the messages from the subscribed topics in blocking mode.
   3. If secDetector is not required, call the secUnsub API for unsubscription. Use the return value of subscription during unsubscription.

## Sample Code

See the sample code compiled in Python in the secDetector code repository.

1. View the sample code at the following link:

   [examples/python · openEuler/secDetector (atomgit.com)](https://atomgit.com/openeuler/secDetector/tree/master/examples/python)

2. Alternatively, download sample code.

```shell
git clone https://atomgit.com/openeuler/secDetector.git
```

## Specifications and Constraints

1. Some functions (such as the security switch in memory modification probes) depend on the hardware architecture. They perform differently on different instruction set architectures.
2. The buffer size for transferring data from the kernel to the user mode is shared by probes. If the buffer is full, newly collected event information is discarded. The buffer size ranges from 4 MB to 1,024 MB and must be a power of 2.
3. The service process secDetectord can be run by the **root** user and does not support multiple instances. The program that is not the first to run exits.
4. The maximum number of user subscription connections is 5.
5. After a user subscribes to specific topics, a buffer needs to be provided for the message reading API. Messages that exceed the buffer length will be truncated. It is recommended that the buffer length be greater than or equal to 4096.
6. The length of the description character strings such as the file name and node name is limited. If the length is too long, the description character strings may be truncated.
7. Parallel multi-connection secDetectord for receiving messages is not supported within a single process of an application. Once a subscription is successful, a single connection is used to receive messages. You can subscribe to different topics only after unsubscribing from the subscribed topics.
8. The secDetectord process can be closed and exited only after all applications are disconnected, that is, all topics are unsubscribed.
9. Some functions (such as the security switch in memory modification probes) are based on the CPU status. The basic detection function is to detect the status change of the current CPU. If the status change of other CPUs is not synchronized to the current CPU in time, the status change of other CPUs will not be detected.

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secgear/secgear_installation.md
---
# secGear Installation

## Arm Environment

### Environment Requirements

#### Hardware

| Item  | Model                                               |
| ------ | --------------------------------------------------- |
| Server| TaiShan 200 server (model 2280)                      |
| Mainboard  | Kunpeng board                                           |
| BMC    | 1711 board (model BC82SMMAB); firmware version: 3.01.12.49 or later|
| CPU    | Kunpeng 920 processor (model 7260, 5250, or 5220)              |
| Chassis  | No special requirements; an 8- or 12-drive chassis recommended                                |

> \[!NOTE]NOTE
>
> * Ensure that the TrustZone feature kit has been preconfigured on the server. That is, the TEE OS, TEE OS boot key, BMC, BIOS, and license have been preconfigured on the server.
> * For common servers, the TrustZone feature cannot be enabled only by upgrading the BMC, BIOS, and TEE OS firmware.
> * By default, the TrustZone feature is disabled on the server. For details about how to enable the TrustZone feature on the server, see BIOS settings.

#### OS

openEuler 20.03 LTS SP4 or later

openEuler 22.09

openEuler 22.03 LTS or later

### Environment Preparation

For details, see [Environment Requirements](https://www.hikunpeng.com/document/detail/en/kunpengcctrustzone/fg-tz/kunpengtrustzone_04_0006.html) and [Procedure](https://www.hikunpeng.com/document/detail/en/kunpengcctrustzone/fg-tz/kunpengtrustzone_04_0007.html) on the Kunpeng official website.

### Installation

1. Configure the openEuler Yum source. You can configure an online Yum source or configure a local Yum source by mounting an ISO file. The following uses openEuler 22.03 LTS as an example. For other versions, use the Yum source of the corresponding version.

   ```shell
   vi openEuler.repo 
   [osrepo]
   name=osrepo
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/aarch64/
   enabled=1
   gpgcheck=1
   gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/aarch64/RPM-GPG-KEY-openEuler
   ```

2. Install secGear.

   ```shell
   # Install the compiler.
   yum install cmake ocaml-dune

   # Install secGear.
   yum install secGear-devel

   # Check whether the installations are successful. If the command output is as follows, the installations are successful.
   $ rpm -qa | grep -E 'secGear|itrustee|ocaml-dune'
   itrustee_sdk-xxx
   itrustee_sdk-devel-xxx
   secGear-xxx
   secGear-devel-xxx
   ocaml-dune-xxx
   ```

## x86 Environment

### Environment Requirements

#### Hardware

Processor that supports the Intel SGX feature

#### OS

openEuler 22.03 LTS SP4 or later

openEuler 22.09

openEuler 22.03 LTS or later

### Environment Preparation

Purchase a device that supports the Intel SGX feature and enable the SGX feature by referring to the BIOS setting manual of the device.

### Installation

1. Configure the openEuler Yum source. You can configure an online Yum source or configure a local Yum source by mounting an ISO file. The following uses openEuler 22.03 LTS as an example. For other versions, use the Yum source of the corresponding version.

   ```shell
   $ vi openEuler.repo 
   [osrepo]
   name=osrepo
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
   enabled=1
   gpgcheck=1
   gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/RPM-GPG-KEY-openEuler
   ```

2. Install secGear.

   ```shell
   # Install the compiler.
   yum install cmake ocaml-dune

   # Install secGear.
   yum install secGear-devel

   # Check whether the installations are successful. If the command output is as follows, the installations are successful.
   $ rpm -qa | grep -E 'secGear|ocaml-dune|sgx'
   secGear-xxx
   secGear-devel-xxx
   ocaml-dune-xxx
   libsgx-epid-xxx
   libsgx-enclave-common-xxx
   libsgx-quote-ex-xxx
   libsgx-aesm-launch-plugin-xxx
   libsgx-uae-service-xxx
   libsgx-ae-le-xxx
   libsgx-urts-xxx
   sgxsdk-xxx
   sgx-aesm-service-xxx
   linux-sgx-driver-xxx
   libsgx-launch-xxx
   ```

---

---
url: /en/docs/22.03_LTS_SP4/server/security/cert_signature/secure_boot.md
---
# Secure Boot

## Overview

Secure Boot relies on public and private key pairs to sign and verify components in the boot process. A typical boot process  uses the previous component to verify the digital signature of the next component. If the verification is successful, the next  component runs; if the verification fails, the boot stops. Secure Boot ensures the integrity of each component during system  boot and prevents unverified components from being loaded and running, mitigating security threats to the system and  user data.\
In Secure Boot, the order of components to be verified are: BIOS, shim, GRUB, and vmlinuz (kernel image).\
Related EFI boot components are signed by the openEuler signature platform in signcode mode. The public key certificate is integrated into the signature database by the BIOS. During the boot, the BIOS verifies shim. The shim and grub components obtain the public key certificate from the signature database of the BIOS to verify the next-level components.

## Scenarios and Solutions

In previous openEuler versions, secure boot components are not signed. Therefore, the secure boot function cannot be directly used to ensure the integrity of system components.\
In openEuler 22.03 LTS SP4 and later versions, openEuler uses the community signature platform to sign OS components, including the grub and vmlinuz components, and integrates the community signature root certificate into the shim component.\
For the shim component, to facilitate end-to-end secure boot, the signature platform of the openEuler community is used for signature. After external CAs officially operate the secure boot component signature service, their signatures will be integrated into the shim module of openEuler.

## Usage

### Obtaining the openEuler Certificate

To obtain the openEuler root certificate, visit <https://www.openeuler.org/en/security/certificate-center>.\
Download **openEuler Shim Default CA** (**default-x509ca.cert**).

### Operations on the BIOS

Import the openEuler root certificate to the BIOS certificate database and enable secure boot in the BIOS to implement secure boot.\
For details about how to import the BIOS certificate and enable secure boot, see the documents provided by the BIOS vendor.

### Operations on the OS

Viewing database certificate information: `mokutil --db`

![](./figures/mokutil-db.png)

Note: The screenshot displays only some important information.

Viewing the secure boot status: `mokutil --sb`

* SecureBoot disabled

![](./figures/mokutil-sb-off.png)

* SecureBoot enabled

![](./figures/mokutil-sb-on.png)

* not supported

![](./figures/mokutil-sb-unsupport.png)

## Constraints

* Software: The OS must be booted in UEFI mode.
* Architecture: Arm or x86
* Hardware: The BIOS must support verification functions related to secure boot.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/container_form/secure_container/overview.md
---
# Secure Container

## Overview

The secure container technology is an organic combination of virtualization and container technologies. Compared with a common Linux container, a secure container has better isolation performance.

Common Linux containers use namespaces to isolate the running environment between processes and use cgroups to limit resources. Essentially, these common Linux containers share the same kernel. Therefore, if a single container affects the kernel intentionally or unintentionally, the containers on the same host will be affected.

Secure containers are isolated by the virtualization layers. Containers on the same host do not affect each other.

**Figure  1**  Secure container architecture

![](./figures/kata-arch.png)

Secure containers are closely related to the concept of pod in Kubernetes. Kubernetes is the open-source ecosystem standard for the container scheduling management platform. It defines a group of container runtime interfaces (CRIs).

In the CRI standards, a pod is a logical grouping of one or more containers, which are scheduled together and share interprocess communication (IPC) and network namespaces. As the smallest unit for scheduling, a pod must contain a pause container and one or more service containers. The lifecycle of a pause container is the same as that of the pod.

A lightweight virtual machine (VM) in a secure container is a pod. The first container started in the VM is the pause container, and the containers started later are service containers.

In a secure container, you can start a single container or start a pod.

[Figure 2](#fig17734185518269)  shows the relationship between the secure container and peripheral components.

**Figure  2**  Relationship between the secure container and peripheral components\
![](./figures/relationship-between-the-secure-container-and-peripheral-components.png)

> \[!NOTE]Note\
> Root privileges are necessary for installing and operating secure containers.

---

---
url: /en/docs/22.03_LTS_SP4/tools/security.md
---


---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/security_and_isolation.md
---
# Security and Isolation

## Many-to-Many User Namespaces

### Function Description

User namespaces are used to map user  **root**  of a container to a common user of the host and allow the processes and user in the container (that are unprivileged on the host) to have privilege. This can prevent the processes in the container from escaping to the host and performing unauthorized operations. In addition, after user namespaces are used, the container and host use different UIDs and GIDs. This ensures that user resources in the container such as file descriptors are isolated from those on the host.

In system containers, you can configure the  **--user-remap**  API parameter to map user namespaces of different containers to different user namespaces on the host, isolating the user namespaces of containers.

### Parameter Description

### Constraints

* If  **--user-remap**  is specified in a system container, the rootfs directory must be accessible to users specified by  *uid*  or  *gid*  in  **--user-remap**. Otherwise, user namespaces of containers cannot access rootfs. As a result, the containers fail to be started.
* All IDs in the container can be mapped to the host rootfs. Some directories or files may be mounted from the host to containers, for example, device files in the  **/dev/pts**  directory. If  *offset*  is too small, the mounting may fail.
* *uid*,  *gid*, and  *offset*  are controlled by the upper-layer scheduling platform. The container engine only checks the validity of them.
* **--user-remap**  is available only in system containers.
* **--user-remap**  and  **--privileged**  cannot be set simultaneously. Otherwise, an error is reported during container startup.
* If  *uid*  or  *gid*  is set to  **0**,  **--user-remap**  does not take effect.
* If **--user-map** is specified for a system container, ensure that the user corresponding to the specified UID or GID can access the isulad metadata directories (**/var/lib/isulad/**, **/var/lib/isulad/engines/**, and **/var/lib/isulad/engines/lcr**).
* **--user-remap** and **--userns** cannot be specified at the same time.

### Usage Guide

> \[!NOTE] **NOTE:**\
> Before specifying the  **--user-remap**  parameter, configure an offset value for UIDs and GIDs of all directories and files in rootfs. The offset value should be equal to that for  *uid*  and  *gid*  in  **--user-remap**.\
> For example, run the following command to offset UIDs and GIDs of all files in the  **dev**  directory with 100000:\
> chown 100000:100000 dev

Specify the  **--user-remap**  parameter when the system container is started.

```shell
[root@localhost ~]# chmod 751 /var/lib/isulad/
[root@localhost ~]# chmod 751 /var/lib/isulad/engines/
[root@localhost ~]# chmod 751 /var/lib/isulad/engines/lcr
[root@localhost ~]# isula run -tid --user-remap 100000:100000:65535 --system-container --external-rootfs /home/root-fs none /sbin/init
eb9605b3b56dfae9e0b696a729d5e1805af900af6ce24428fde63f3b0a443f4a
```

Check the /sbin/init process information on the host and in a container.

```shell
[root@localhost ~]# isula exec eb ps aux | grep /sbin/init
root         1  0.6  0.0  21624  9624 ?        Ss   15:47   0:00 /sbin/init
[root@localhost ~]# ps aux | grep /sbin/init
100000    4861  0.5  0.0  21624  9624 ?        Ss   15:47   0:00 /sbin/init
root      4948  0.0  0.0 213032   808 pts/0    S+   15:48   0:00 grep --color=auto /sbin/init
```

The owner of the /sbin/init process in the container is user  **root**, but the owner of the host is the user whose UID is  **100000**.

Create a file in a container and view the file owner on the host.

```shell
[root@localhost ~]# isula exec -it eb bash
[root@localhost /]# echo test123 >> /test123
[root@localhost /]# exit
exit
[root@localhost ~]# ll /home/root-fs/test123
-rw-------. 1 100000 100000 8 Aug  2 15:52 /home/root-fs/test123
```

The owner of the file that is generated in the container is user  **root**, but the file owner displayed on the host is the user whose ID is  **100000**.

## User Permission Control

### Function Description

A container engine supports TLS for user identity authentication, which is used to control user permissions. Currently, container engines can connect to the authz plug-in to implement permission control.

### API Description

You can configure the startup parameters of the iSulad container engine to specify the permission control plug-in. The default daemon configuration file is  **/etc/isulad/daemon.json**.

### Constraints

* User permission policies need to be configured for authz. The default policy file is  **/var/lib/authz-broker/policy.json**. This file can be dynamically modified and the modification will take effect immediately without restarting the plug-in service.
* A container engine can be started by user  **root**. If some commands used are enabled for by common users, common users may obtain excessive permissions. Therefore, exercise caution when performing such operations. Currently, running the  **container\_attach**,  **container\_create**, and  **container\_exec\_create**  commands may cause risks.
* Some compound operations, such as running  **isula exec**  and  **isula inspect**  or running and  **isula attach**  and  **isula inspect**, depend on the permission of  **isula inspect**. If a user does not have this permission, an error is reported.
* Using SSL/TLS encryption channels hardens security but also reduces performance. For example, the delay increases, more CPU resources are consumed, and encryption and decryption require higher throughput. Therefore, the number of concurrent executions decreases compared with non-TLS communication. According to the test result, when the ARM server (Cortex-A72 64-core) is almost unloaded, TLS is used to concurrently start a container. The maximum number of concurrent executions is 200 to 250.
* If  **--tlsverify**  is specified on the server, the default path where authentication files store is  **/etc/isulad**. The default file names are  **ca.pem**,  **cert.pem**, and  **key.pem**.

### Example

1. Ensure that the authz plug-in is installed on the host. If the authz plug-in is not installed, run the following command to install and start the authz plug-in service:

   ```shell
   [root@localhost ~]# yum install authz
   [root@localhost ~]# systemctl start authz
   ```

2. To enable this function, configure the container engine and TLS certificate. You can use OpenSSL to generate the required certificate.

   ```shell
   #SERVERSIDE
    
   # Generate CA key
   openssl genrsa -aes256 -passout "pass:$PASSWORD" -out "ca-key.pem" 4096
   # Generate CA
   openssl req -new -x509 -days $VALIDITY -key "ca-key.pem" -sha256 -out "ca.pem" -passin "pass:$PASSWORD" -subj "/C=$COUNTRY/ST=$STATE/L=$CITY/O=$ORGANIZATION/OU=$ORGANIZATIONAL_UNIT/CN=$COMMON_NAME/emailAddress=$EMAIL"
   # Generate Server key
   openssl genrsa -out "server-key.pem" 4096
    
   # Generate Server Certs.
   openssl req -subj "/CN=$COMMON_NAME" -sha256 -new -key "server-key.pem" -out server.csr
    
   echo "subjectAltName = DNS:localhost,IP:127.0.0.1" > extfile.cnf
   echo "extendedKeyUsage = serverAuth" >> extfile.cnf
    
   openssl x509 -req -days $VALIDITY -sha256 -in server.csr -passin "pass:$PASSWORD" -CA "ca.pem" -CAkey "ca-key.pem" -CAcreateserial -out "server-cert.pem" -extfile extfile.cnf
    
   #CLIENTSIDE
    
   openssl genrsa -out "key.pem" 4096
   openssl req -subj "/CN=$CLIENT_NAME" -new -key "key.pem" -out client.csr
   echo "extendedKeyUsage = clientAuth" > extfile.cnf
   openssl x509 -req -days $VALIDITY -sha256 -in client.csr -passin "pass:$PASSWORD" -CA "ca.pem" -CAkey "ca-key.pem" -CAcreateserial -out "cert.pem" -extfile extfile.cnf
   ```

   If you want to use the preceding content as the script, replace the variables with the configured values. If the parameter used for generating the CA is empty, set it to  **"**.  **PASSWORD**,  **COMMON\_NAME**,  **CLIENT\_NAME**, and  **VALIDITY**  are mandatory.

3. When starting the container engine, add parameters related to the TLS and authentication plug-in and ensure that the authentication plug-in is running properly. In addition, to use TLS authentication, the container engine must be started in TCP listening mode instead of the Unix socket mode. The configuration on the container daemon is as follows:

   ```json
   {
       "tls": true,
       "tls-verify": true,
       "tls-config": {
              "CAFile": "/root/.iSulad/ca.pem",
              "CertFile": "/root/.iSulad/server-cert.pem",
              "KeyFile":"/root/.iSulad/server-key.pem"
       },
       "authorization-plugin": "authz-broker"
   }
   ```

4. Configure policies. For the basic authorization process, all policies are stored in the  **/var/lib/authz-broker/policy.json**  configuration file. The configuration file can be dynamically modified without restarting the plug-in. Only the SIGHUP signal needs to be sent to the authz process. In the file, a line contains one JSON policy object. The following provides policy configuration examples:

   * All users can run all iSuald commands:  **{"name":"policy\_0","users":\[""],"actions":\[""]}**
   * Alice can run all iSulad commands:  **{"name":"policy\_1","users":\["alice"],"actions":\[""]}**
   * A blank user can run all iSulad commands:  **{"name":"policy\_2","users":\[""],"actions":\[""]}**
   * Alice and Bob can create new containers:  **{"name":"policy\_3","users":\["alice","bob"],"actions":\["container\_create"]}**
   * service\_account can read logs and run  **docker top**:  **{"name":"policy\_4","users":\["service\_account"],"actions":\["container\_logs","container\_top"]}**
   * Alice can perform any container operations:  **{"name":"policy\_5","users":\["alice"],"actions":\["container"]}**
   * Alice can perform any container operations, but the request type can only be  **get**:  **{"name":"policy\_5","users":\["alice"],"actions":\["container"], "readonly":true}**

   > \[!NOTE] **NOTE:**
   >
   > * **actions**  supports regular expressions.
   > * **users**  does not support regular expressions.
   > * A users cannot be repeatedly specified by **users**. That is, a user cannot match multiple rules.

5. After updating the configurations, configure TLS parameters on the client to connect to the container engine. That is, access the container engine with restricted permissions.

   ```shell
   [root@localhost ~]#  isula version --tlsverify --tlscacert=/root/.iSulad/ca.pem --tlscert=/root/.iSulad/cert.pem --tlskey=/root/.iSulad/key.pem  -H=tcp://127.0.0.1:2375
   ```

   If you want to use the TLS authentication for default client connection, move the configuration file to  **~/.iSulad**  and set the  **ISULAD\_HOST**  and  **ISULAD\_TLS\_VERIFY**  variables (rather than transferring  **-H=tcp://$HOST:2375**  and -**-tlsverify**  during each call).

   ```shell
   [root@localhost ~]# mkdir -pv ~/.iSulad
   [root@localhost ~]# cp -v {ca,cert,key}.pem ~/.iSulad 
   [root@localhost ~]# export ISULAD_HOST=localhost:2375 ISULAD_TLS_VERIFY=1
   [root@localhost ~]# isula version
   ```

## proc File System Isolation

### Application Scenario

Container virtualization is lightweight and efficient, and can be quickly deployed. However, containers are not strongly isolated, which causes great inconvenience to users. Containers have some defects in isolation because the namespace feature of the Linux kernel is not perfect. For example, you can view the proc information on the host (such as meminfo, cpuinfo, stat, and uptime) in the proc file system of a container. You can use the lxcfs tool to replace the /proc content of instances in the container with the content in the /proc file system of the host so that services in the container can obtain the correct resource value.

### API Description

A system container provides two tool packages: lxcfs and lxcfs-toolkit, which are used together. Lxcfs resides on the host as the daemon process. lxcfs-toolkit mounts the lxcfs file system of the host to containers through the hook mechanism.

The command line of lxcfs-toolkit is as follows:

```shell
lxcfs-toolkit [OPTIONS] COMMAND [COMMAND_OPTIONS]
```

### Constraints

* Currently, only the  **cpuinfo**,  **meminfo**,  **stat**,  **diskstats**,  **partitions**,  **swaps**, and  **uptime**  files in the proc file system are supported. Other files are not isolated from other kernel API file systems (such as sysfs).
* After an RPM package is installed, a sample JSON file is generated in  **/var/lib/lcrd/hooks/hookspec.json**. To add the log function, you need to add the  **--log**  configuration during customization.
* The  **diskstats**  file displays only information about disks that support CFQ scheduling, instead of partition information. Devices in containers are displayed as names in the  **/dev**  directory. If a device name does not exist, the information is left blank. In addition, the device where the container root directory is located is displayed as  **sda**.
* The  **slave**  parameter is required when lxcfs is mounted. If the  **shared**  parameter is used, the mount point in containers may be leaked to the host, affecting the host running.
* Lxcfs supports graceful service degradation. If the lxcfs service crashes or becomes unavailable, the  **cpuinfo**,  **meminfo**,  **stat**,  **diskstats**,  **partitions**,  **swaps**and  **uptime**  files in containers are about host information, and other service functions of containers are not affected.
* Bottom layer of lxcfs depends on the FUSE kernel module and libfuse library. Therefore, the kernel needs to support FUSE.
* Lxcfs supports only the running of 64-bit applications in containers. If a 32-bit application is running in a container, the CPU information (**cpuinfo**) read by the application may fail to meet expectations.
* Lxcfs simulates the resource view only of container control groups (cgroups). Therefore, system calls (such as sysconf) in containers can obtain only host information. Lxcfs cannot implement the kernel isolation.
* The CPU information (**cpuinfo**) displayed after lxcfs implements the isolation has the following features:
  * **processor**: The value increases from 0.
  * **physical id**: The value increases from 0.
  * **sibliing**: It has a fixed value of  **1**.
  * **core id**: It has a fixed value of  **0**.
  * **cpu cores**: It has a fixed value of  **1**.

### Example

1. Install the lxcfs and lxcfs-toolkit packages and start the lxcfs service.

   ```shell
   [root@localhost ~]# yum install lxcfs lxcfs-toolkit 
   [root@localhost ~]# systemctl start lxcfs
   ```

2. After a container is started, check whether the lxcfs mount point exists in the container.

   ```shell
   [root@localhost ~]# isula run -tid -v /var/lib/lxc:/var/lib/lxc --hook-spec /var/lib/isulad/hooks/hookspec.json --system-container --external-rootfs /home/root-fs none init
   a8acea9fea1337d9fd8270f41c1a3de5bceb77966e03751346576716eefa9782
   [root@localhost ~]# isula exec a8 mount | grep lxcfs
   lxcfs on /var/lib/lxc/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/cpuinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/diskstats type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/meminfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/partitions type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/stat type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/swaps type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/uptime type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   ```

3. Run the  **update**  command to update the CPU and memory resource configurations of the container and check the container resources. As shown in the following command output, the container resource view displays the actual container resource data instead of data of the host.

   ```shell
   [root@localhost ~]# isula update --cpuset-cpus 0-1 --memory 1G a8
   a8
   [root@localhost ~]# isula exec a8 cat /proc/cpuinfo
   processor       : 0
   BogoMIPS        : 100.00
   cpu MHz         : 2400.000
   Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
   CPU implementer : 0x41
   CPU architecture: 8
   CPU variant     : 0x0
   CPU part        : 0xd08
   CPU revision    : 2

   processor       : 1
   BogoMIPS        : 100.00
   cpu MHz         : 2400.000
   Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
   CPU implementer : 0x41
   CPU architecture: 8
   CPU variant     : 0x0
   CPU part        : 0xd08
   CPU revision    : 2

   [root@localhost ~]# isula exec a8 free -m
                 total        used        free      shared  buff/cache   available
   Mem:           1024          17         997           7           8        1006
   Swap:          4095           0        4095
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/security_features.md
---
# Security Features

## Seccomp Security Configuration

### Scenarios

Secure computing mode (seccomp) is a simple sandboxing mechanism introduced to the Linux kernel from version 2.6.23. In some specific scenarios, you may want to perform some privileged operations in a container without starting the privileged container. You can add  **--cap-add**  at runtime to obtain some small-scope permissions. For container instances with strict security requirements, th capability granularity may not meet the requirements. You can use some methods to control the permission scope in a refined manner.

* Example

  In a common container scenario, you can use the  **-v**  flag to map a directory (including a binary file that cannot be executed by common users) on the host to the container.

  In the container, you can add chmod 4777 (the modification permission of the binary file) to the S flag bit. In this way, on the host, common users who cannot run the binary file (or whose running permission is restricted) can obtain the permissions of the binary file (such as the root permission) when running the binary file after the action added to the S flag bit is performed, so as to escalate the permission or access other files.

  In this scenario, if strict security requirements are required, the chmod, fchmod, and fchmodat system calls need to be tailored by using seccomp.

### Usage Restrictions

* Do not disable the seccomp feature of iSulad.

  By default, iSulad has a seccomp configuration. An allowlist is used in the configuration. syscalls that are not in the allowlist will be disabled by seccomp. You can use the **--security-opt 'seccomp:unconfined'** API to disable the seccomp feature. If seccomp is disabled or the user-defined seccomp configuration is used but the allowlist is incomplete, the attack surface of the container to the kernel increases.

* The default seccomp configuration is an allowlist. For syscalls that are not in the allowlist, **SCMP\_ACT\_ERRNO** is returned by default. In addition, different syscalls are made available based on different capabilities. iSulad does not grant permissions that are not in the allowlist to containers by default.

### Usage Guide

Use  **--security-opt**  to transfer the configuration file to the container where system calls need to be filtered.

```bash
isula run -itd --security-opt seccomp=/path/to/seccomp/profile.json rnd-dockerhub.huawei.com/official/busybox
```

> \[!NOTE] **NOTE:**
>
> * When the configuration file is transferred to the container by using  **--security-opt**  during container creation, the default configuration file (**/etc/isulad/seccomp\_default.json**) is used.
> * When  **--security-opt**  is set to  **unconfined**  during container creation, system calls are not filtered for the container.
> * **/path/to/seccomp/profile.json**  must be an absolute path.
> * **--security-opt** can be separated by equal signs (=) instead of colons (:).

#### Obtaining the Default Seccomp Configuration of a Common Container

* Start a common container (or a container with  **--cap-add**) and check its default permission configuration.

  ```bash
  cat /etc/isulad/seccomp_default.json | python -m json.tool > profile.json
  ```

  The  **seccomp**  field contains many  **syscalls**  fields. Then extract only the  **syscalls**  fields and perform the customization by referring to the customization of the seccomp configuration file.

  ```conf
  "defaultAction": "SCMP_ACT_ERRNO",
  "syscalls": [
  {
  "action": "SCMP_ACT_ALLOW",
  "name": "accept"
  },
  {
  "action": "SCMP_ACT_ALLOW",
  "name": "accept4"
  },
  {
  "action": "SCMP_ACT_ALLOW",
  "name": "access"
  },
  {
  "action": "SCMP_ACT_ALLOW",
  "name": "alarm"
  },
  {
  "action": "SCMP_ACT_ALLOW",
  "name": "bind"
  },
  ]...
  ```

* Check the seccomp configuration that can be identified by the LXC.

  ```bash
  cat /var/lib/isulad/engines/lcr/74353e38021c29314188e29ba8c1830a4677ffe5c4decda77a1e0853ec8197cd/seccomp
  ```

  ```text
  ...
  waitpid allow
  write allow
  writev allow
  ptrace allow
  personality allow [0,0,SCMP_CMP_EQ,0]
  personality allow [0,8,SCMP_CMP_EQ,0]
  personality allow [0,131072,SCMP_CMP_EQ,0]
  personality allow [0,131080,SCMP_CMP_EQ,0]
  personality allow [0,4294967295,SCMP_CMP_EQ,0]
  ...
  ```

#### Customizing the Seccomp Configuration File

When starting a container, use  **--security-opt**  to introduce the seccomp configuration file. Container instances will restrict the running of system APIs based on the configuration file. Obtain the default seccomp configuration of common containers, obtain the complete template, and customize the configuration file by referring to this section to start the container.

```bash
isula run --rm -it --security-opt seccomp:/path/to/seccomp/profile.json rnd-dockerhub.huawei.com/official/busybox
```

The configuration file template is as follows:

```conf
{
"defaultAction": "SCMP_ACT_ALLOW",
"syscalls": [
{
"name": "syscall-name",
"action": "SCMP_ACT_ERRNO",
"args": null
}
]
}
```

> \[!TIP] **NOTICE:**
>
> * **defaultAction**  and  **syscalls**: The types of their corresponding actions are the same, but their values must be different. The purpose is to ensure that each syscall has a default action. Clear definitions in the syscall array shall prevail. As long as the values of  **defaultAction**  and  **action**  are different, no action conflicts will occur. The following actions are supported:\
>   **SCMP\_ACT\_ERRNO**: forbids calling syscalls and displays error information.\
>   **SCMP\_ACT\_ALLOW**: allows calling syscalls.
> * **syscalls**: array, which can contain one or more syscalls.  **args**  is optional.
> * **name**: syscalls to be filtered.
> * **args**: array. The definition of each object in the array is as follows:
>
> ```go
> type Arg struct {  
> Index    uint     `json:"index"`     // Parameter ID. Take open(fd, buf, len) as an example. The fd corresponds to 0 and buf corresponds to 1.  
> Value    uint64   `json:"value"`     // Value to be compared with the parameter.  
> ValueTwo uint64   `json:"value_two"` // It is valid only when Op is set to MaskEqualTo. After the bitwise AND operation is performed on the user-defined value and the value of Value, the result is compared with the value of ValueTwo. If they are the same, the action is executed.  
> Op       Operator `json:"op"`  
> }  
> ```
>
> The value of  **Op**  in  **args**  can be any of the following:\
> "SCMP\_CMP\_NE":  NotEqualTo\
> "SCMP\_CMP\_LT":  LessThan\
> "SCMP\_CMP\_LE":  LessThanOrEqualTo\
> "SCMP\_CMP\_EQ":  EqualTo\
> "SCMP\_CMP\_GE":  GreaterThanOrEqualTo\
> "SCMP\_CMP\_GT":  GreaterThan\
> "SCMP\_CMP\_MASKED\_EQ": MaskEqualTo

## capabilities Security Configuration

### Scenarios

The capability mechanism is a security feature introduced to Linux kernel after version 2.2. The super administrator permission is controlled at a smaller granularity to prevent the root permission from being used. The root permission is divided based on different domains so that the divided permissions can be enabled or disabled separately. For details about capabilities, see the  *Linux Programmer's Manual*  ([capabilities(7) - Linux man page](http://man7.org/linux/man-pages/man7/capabilities.7.html)).

```bash
man capabilities
```

### Usage Restrictions

* The default capability list (whitelist) of the iSulad service, which is carried by common container processes by default, are as follows:

  ```conf
  "CAP_CHOWN",
  "CAP_DAC_OVERRIDE",
  "CAP_FSETID",
  "CAP_FOWNER",
  "CAP_MKNOD",
  "CAP_NET_RAW",
  "CAP_SETGID",
  "CAP_SETUID",
  "CAP_SETFCAP",
  "CAP_SETPCAP",
  "CAP_NET_BIND_SERVICE",
  "CAP_SYS_CHROOT",
  "CAP_KILL",
  "CAP_AUDIT_WRITE"
  ```

* Default configurations of capabilities include  **CAP\_SETUID**  and  **CAP\_FSETID**. If the host and a container share a directory, the container can set permissions for the binary file in the shared directory. Common users on the host can use this feature to elevate privileges. The container can write  **CAP\_AUDIT\_WRITE**  to the host, which may cause risks. If the application scenario does not require this capability, you are advised to use  **--cap-drop**  to delete the capability when starting the container.

* Adding capabilities means that the container process has greater capabilities than before. In addition, more system call APIs are opened.

### Usage Guide

iSulad uses  **--cap-add**  or  **--cap-drop**  to add or delete specific permissions for a container. Do not add extra permissions to the container unless necessary. You are advised to remove the default but unnecessary permissions from the container.

```bash
isula run --rm -it --cap-add all --cap-drop SYS_ADMIN rnd-dockerhub.huawei.com/official/busybox
```

## SELinux Security Configuration

### Scenarios

Security-Enhanced Linux (SELinux) is a Linux kernel security module that provides a mechanism for supporting access control security policies. Through Multi-Category Security (MCS), iSulad labels processes in containers to control containers' access to resources, reducing privilege escalation risks and preventing further damage.

### Usage Restrictions

* Ensure that SELinux is enabled for the host and daemon (the  **selinux-enabled**  field in the  **/etc/isulad/daemon.json**  file is set to  **true**  or  **--selinux-enabled**  is added to command line parameters).
* Ensure that a proper SELinux policy has been configured on the host. container-selinux is recommended.
* The introduction of SELinux affects the performance. Therefore, evaluate the scenario before setting SELinux. Enable the SELinux function for the daemon and set the SELinux configuration in the container only when necessary.
* When you configure labels for a mounted volume, the source directory cannot be a subdirectory of  **/**,  **/usr**,  **/etc**,  **/tmp**,  **/home**,  **/run**,  **/var**,  **/root**, or  **/usr**.

> \[!NOTE] **NOTE:**
>
> * iSulad does not support labeling the container file system. To ensure that the container file system and configuration directory are labeled with the container access permission, run the  **chcon**  command to label them.
> * If SELinux access control is enabled for iSulad, you are advised to add a label to the  **/var/lib/isulad**  directory before starting daemon. Files and folders generated in the directory during container creation inherit the label by default. For example:
>
> ```bash
> chcon -R system_u:object_r:container_file_t:s0 /var/lib/isulad  
> ```

### Usage Guide

* Enable SELinux for daemon.

  ```bash
  isulad --selinux-enabled
  ```

* Configure SELinux security context labels during container startup.

  **--security-opt="label=user:USER"**: Set the label user for the container.

  **--security-opt="label=role:ROLE"**: Set the label role for the container.

  **--security-opt="label=type:TYPE"**: Set the label type for the container.

  **--security-opt="label=level:LEVEL"**: Set the label level for the container.

  **--security-opt="label=disable"**: Disable the SELinux configuration for the container.

  ```bash
  $ isula run -itd --security-opt label=type:container_t --security-opt label=level:s0:c1,c2 rnd-dockerhub.huawei.com/official/centos
  9be82878a67e36c826b67f5c7261c881ff926a352f92998b654bc8e1c6eec370
  ```

* Add the selinux label to a mounted volume (**z**  indicates the shared mode).

  ```bash
  $ isula run -itd -v /test:/test:z rnd-dockerhub.huawei.com/official/centos
  9be82878a67e36c826b67f5c7261c881ff926a352f92998b654bc8e1c6eec370

  $ls -Z /test
  system_u:object_r:container_file_t:s0 file
  ```

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secharden/secharden.md
---
# Security Hardening Guide

This document describes how to perform security hardening for openEuler.

This document is intended for administrators who need to perform security hardening for openEuler. You must be familiar with the OS security architecture and technologies.

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secharden/security_hardening_tool.md
---
# Security Hardening Tools

## Security Hardening Procedure

### Overview

You need to modify the  **usr-security.conf**  file so that the security hardening tool can set hardening policies based on the  **usr-security.conf**  file. This section describes rules for modifying the  **usr-security.conf**  file. For details about the configurable security hardening items, see  [Security Hardening Guide](./secharden.md).

### Precautions

* After modifying the items, restart the security hardening service for the modification to take effect. For details about how to restart the service, see  [Hardening Items Taking Effect](#hardening-items-taking-effect).
* When modifying security hardening items, you only need to modify the  **/etc/openEuler\_security/usr-security.conf**  file. You are not advised to modify the  **/etc/openEuler\_security/security.conf**  file. The  **security.conf**  file contains basic hardening items which are executed only once.
* After the security hardening service is restarted for the configuration to take effect, the previous configuration cannot be deleted by deleting the corresponding hardening items from the  **usr-security.conf**  file and restarting the security hardening service.
* Security hardening operations are recorded in the  **/var/log/openEuler-security.log**  file.

### Configuration Format

Each line in the  **usr-security.conf**  file indicates a configuration item. The configuration format varies according to the configuration content. The following describes the format of each configuration item.

> \[!NOTE] **NOTE:**
>
> * All configuration items start with an execution ID. The execution ID is a positive integer and can be customized.
> * Contents of a configuration item are separated by an at sign (@).
> * If the actual configuration content contains an at sign (@), use two at signs (@@) to distinguish the content from the separator. For example, if the actual content is  **xxx@yyy**, set this item to  **xxx@@yyy**. Currently, an at sign (@) cannot be placed at the beginning or end of the configuration content.

* **d**: comment

  Format:  *Execution ID***@d@***Object file***@***Match item*

  Function: Comment out lines starting with the match item (the line can start with a space) in an object file by adding a number sign (#) at the beginning of the line.

  Example: If the execution ID is  **401**, comment out lines starting with  **%wheel**  in the  **/etc/sudoers**  file.

  ```text
  401@d@/etc/sudoers@%wheel
  ```

* **m**: replacement

  Format:  *Execution ID***@m@***Object file***@***Match item***@***Target value*

  Function: Replace lines starting with the match item (the line can start with a space) in an object file with  *match item*  and  *target value*. If the match line starts with spaces, the spaces will be deleted after the replacement.

  Example: If the execution ID is  **101**, replace lines starting with  **Protocol**  in the  **/etc/ssh/sshd\_config**  file with  **Protocol 2**. The spaces after  **Protocol**  are matched and replaced.

  ```text
  101@m@/etc/ssh/sshd_config@Protocol @2
  ```

* **sm**: accurate modification

  Format:  *Execution ID***@sm@***Object file***@***Match item***@***Target value*

  Function: Replace lines starting with the match item (the line can start with a space) in an object file with  *match item*  and  *target value*. If the match line starts with spaces, the spaces are retained after the replacement. This is the difference between  **sm**  and  **m**.

  Example: If the execution ID is  **201**, replace lines starting with  **size**  in the  **/etc/audit/hzqtest**  file with  **size 2048**.

  ```text
  201@sm@/etc/audit/hzqtest@size@ 2048
  ```

* **M**: subitem modification

  Format:  *Execution ID***@M@***Object file***@***Match item***@***Match subitem*\_\[@Value of the match subitem]\_

  Function: Match lines starting with the match item (the line can start with a space) in an object file and replace the content starting with the match subitem in these lines with the  *match subitem*  and  *value of the match subitem*. The value of the match subitem is optional.

  Example: If the execution ID is  **101**, find lines starting with  **key**  in the file and replace the content starting with  **key2**  in these lines with  **key2value2**.

  ```text
  101@M@file@key@key2@value2
  ```

* **systemctl**: service management

  Format:  *Execution ID***@systemctl@***Object service***@***Operation*

  Function: Use  **systemctl**  to manage object services. The value of  **Operation**  can be  **start**,  **stop**,  **restart**, or  **disable**.

  Example: If the execution ID is  **218**, stop the  **cups.service**. This provides the same function as running the  **systemctl stop cups.service**  command.

  ```text
  218@systemctl@cups.service@stop
  ```

* Other commands

  Format:  *Execution ID***@***Command***@***Object file*

  Function: Run the corresponding command, that is, run the command line  *Command* *Object file*.

  Example 1: If the execution ID is  **402**, run the  **rm -f**  command to delete the  **/etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem**  file.

  ```text
  402@rm -f @/etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem
  ```

  Example 2: If the execution ID is  **215**, run the  **touch**  command to create the  **/etc/cron.allow**  file.

  ```text
  215@touch @/etc/cron.allow
  ```

  Example 3: If the execution ID is  **214**, run the  **chown**  command to change the owner of the  **/etc/at.allow**  file to  **root:root**.

  ```text
  214@chown root:root @/etc/at.allow
  ```

  Example 4: If the execution ID is  **214**, run the  **chmod**  command to remove the  **rwx**  permission of the group to which the owner of the **/etc/at.allow**  file belongs and other non-owner users.

  ```text
  214@chmod og-rwx @/etc/at.allow
  ```

## Hardening Items Taking Effect

After modifying the  **usr-security.conf**  file, run the following command for the new configuration items to take effect:

```shell
systemctl restart openEuler-security.service
```

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secharden/selinux_configuration.md
---
# SELinux Configuration

## Overview

Discretionary Access Control (DAC) is the most common access control method, where a subject who has the ownership of (or control on) an object can grant other subjects one or more access permissions to the object, and can revoke these permissions at any time. DAC is based on the permissions of the object owner, owner group, and other users. Whether a resource can be accessed depends on whether a user has the required permissions on the resource. As a result, DAC does not allow the system administrator to create comprehensive and fine-grained security policies. Security-Enhanced Linux (SELinux) is a module of the Linux kernel and a security subsystem of Linux. SELinux implements mandatory access control (MAC). Each process and system resource has a special security label. In addition to the principles specified by DAC, SELinux also determines whether each process type has the permission to access a resource type. In this way, the system administrator can create comprehensive and fine-grained security policies.

By default, openEuler uses SELinux to improve system security. SELinux has three modes:

* **permissive**: The SELinux outputs alarms but does not forcibly execute the security policies.
* **enforcing**: The SELinux security policies are forcibly executed.
* **disabled**: The SELinux security policies are not loaded.

## Configuration Description

* Query the SELinux status.

  ```sh
  $ getenforce
  Enforcing
  ```

* Use the enforcing mode when SELinux is enabled.

  ```sh
  $ setenforce 1
  $ getenforce
  Enforcing
  ```

* Use the permissive mode when SELinux is enabled.

  ```sh
  $ setenforce 0
  $ getenforce
  Permissive
  ```

* Disable SELinux when it is enabled. (The system needs to be rebooted.)
  1. Set **SELINUX=disabled** in the SELinux configuration file **/etc/selinux/config**.

     ```sh
     $ cat /etc/selinux/config | grep "SELINUX="
     SELINUX=disabled
     ```

  2. Reboot the system.

     ```sh
     reboot
     ```

  3. Check if the SELinux status is changed.

     ```sh
     $ getenforce
     Disabled
     ```

* Use the permissive mode when SELinux is disabled.
  1. Set **SELINUX=permissive** in the SELinux configuration file **/etc/selinux/config**.

     ```sh
     $ cat /etc/selinux/config | grep "SELINUX="
     SELINUX=permissive
     ```

  2. Create a **.autorelabel** file in the root directory.

     ```sh
     touch /.autorelabel
     ```

  3. Reboot the system. The system will reboot twice.

     ```sh
     reboot
     ```

  4. Check if the SELinux status is changed.

     ```sh
     $ getenforce
     Permissive
     ```

* Use the enforcing mode when SELinux is disabled.
  1. Use the permissive mode by referring to the previous step.

  2. Set **SELINUX=enforcing** in the SELinux configuration file **/etc/selinux/config**.

     ```sh
     $ cat /etc/selinux/config | grep "SELINUX="
     SELINUX=enforcing
     ```

  3. Reboot the system.

     ```sh
     reboot
     ```

  4. Check if the SELinux status is changed.

     ```sh
     $ getenforce
     Enforcing
     ```

## SELinux Commands

* Query the SELinux status.  **SELinux status**  indicates the SELinux status.  **enabled**  indicates that SELinux is enabled, and  **disabled**  indicates that SELinux is disabled.  **Current mode**  indicates the current mode of the SELinux.

  ```sh
  $ sestatus
  SELinux status:                 enabled
  SELinuxfs mount:                /sys/fs/selinux
  SELinux root directory:         /etc/selinux
  Loaded policy name:             targeted
  Current mode:                   enforcing
  Mode from config file:          enforcing
  Policy MLS status:              enabled
  Policy deny_unknown status:     allowed
  Memory protection checking:     actual (secure)
  Max kernel policy version:      33
  ```

## Precautions

* Before enabling SELinux, you are advised to upgrade selinux-policy to the latest version using DNF. Otherwise, applications may fail to run properly. For example:

```sh
dnf update selinux-policy -y
```

* If the system cannot be started due to improper SELinux configuration (for example, a policy is deleted by mistake or no proper rule or security context is configured), you can add **selinux=0** to the startup parameters to disable SELinux.

---

---
url: /zh/docs/22.03_LTS_SP4/server/security/secharden/selinux_configuration.md
---
# SELinux配置

## 概述

自主访问控制DAC（Discretionary Access Control）是一种最为普遍的访问控制手段，是指对某个客体具有拥有权（或控制权）的主体能够将对该客体的一种访问权或多种访问权自主地授予其它主体，并随时可以将这些权限回收，这种方式基于用户、组和其他权限，决定一个资源是否能被访问的因素是某个资源是否拥有对应用户的权限，这就导致它不能使系统管理员创建全面和细粒度的安全策略。SELinux（Security-Enhanced Linux）是Linux内核的一个模块，也是Linux的一个安全子系统。SELinux实现了强制访问控制MAC（Mandatory Access Control ），每个进程和系统资源都有一个特殊的安全标签，资源能否被访问除了DAC规定的原则外，还需要判断每一类进程是否拥有对某一类资源的访问权限。这种方式能够满足系统管理员创建全面和细粒度的安全策略的需求。

openEuler默认使用SELinux提升系统安全性。SELinux分为三种模式：

* permissive：SELinux仅打印告警而不强制执行。
* enforcing：SELinux安全策略被强制执行。
* disabled：不加载SELinux安全策略。

## 配置说明

* 获取当前SELinux运行状态。

  ```sh
  # getenforce
  Enforcing
  ```

* SELinux开启的前提下，设置运行状态为enforcing模式。

  ```sh
  # setenforce 1
  # getenforce
  Enforcing
  ```

* SELinux开启的前提下，设置运行状态为permissive模式。

  ```sh
  # setenforce 0
  # getenforce
  Permissive
  ```

* SELinux开启的前提下，设置当前SELinux运行状态为disabled（关闭SELinux，需要重启系统）。

  1. 修改SELinux配置文件/etc/selinux/config，设置“SELINUX=disabled”。

     ```sh
     # cat /etc/selinux/config | grep "SELINUX="
     SELINUX=disabled
     ```

  2. 重启系统。

     ```sh
     # reboot
     ```

  3. 查看切换状态。

     ```sh
     # getenforce
     Disabled
     ```

* SELinux关闭的前提下，设置SELinux运行状态为permissive。
  1. 修改SELinux配置文件/etc/selinux/config，设置“SELINUX=permissive”。

     ```sh
     # cat /etc/selinux/config | grep "SELINUX="
     SELINUX=permissive
     ```

  2. 在根目录下创建.autorelabel文件。

     ```sh
     # touch /.autorelabel
     ```

  3. 重启系统，此时系统会重启两次。

     ```sh
     # reboot
     ```

  4. 查看切换状态。

     ```sh
     # getenforce
     Permissive
     ```

* SELinux关闭的前提下，设置SELinux运行状态为enforcing。
  1. 按照上一步骤所述，设置SELinux运行状态为permissive。

  2. 修改SELinux配置文件/etc/selinux/config，设置“SELINUX=enforcing”。

     ```sh
     # cat /etc/selinux/config | grep "SELINUX="
     SELINUX=enforcing
     ```

  3. 重启系统。

     ```sh
     # reboot
     ```

  4. 查看切换状态。

     ```sh
     # getenforce
     Enforcing
     ```

## SELinux相关命令

* 查询运行SELinux的系统状态。SELinux status表示SELinux的状态，enabled表示启用SELinux，disabled表示关闭SELinux。Current mode表示SELinux当前的安全策略。

  ```sh
  # sestatus
  SELinux status:                 enabled
  SELinuxfs mount:                /sys/fs/selinux
  SELinux root directory:         /etc/selinux
  Loaded policy name:             targeted
  Current mode:                   enforcing
  Mode from config file:          enforcing
  Policy MLS status:              enabled
  Policy deny_unknown status:     allowed
  Memory protection checking:     actual (secure)
  Max kernel policy version:      33
  ```

## 注意事项

* 如用户需要使用SELinux功能，建议通过dnf升级方式将selinux-policy更新为最新版本，否则应用程序有可能无法正常运行。升级命令示例：

  ```sh
  dnf update selinux-policy -y
  ```

* 如果用户由于SELinux配置不当（如误删策略或未配置合理的规则或安全上下文）导致系统无法启动，可以在启动参数中添加selinux=0，关闭SELinux功能，系统即可正常启动。

---

---
url: /en/docs/22.03_LTS_SP4/server.md
---


---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/administrator/service_management.md
---
# Service Management

This topic describes how to manage your operating system and services using the systemd.

## Introduction to systemd

The systemd is a system and service manager for Linux operating systems. It is designed to be backward compatible with SysV and LSB init scripts, and provides a number of features such as Socket & D-Bus based activation of services, on-demand activation of daemons, system state snapshots, and mount & automount point management. With systemd, the service control logic and parallelization are refined.

### Systemd Units

In systemd, the targets of most actions are units, which are resources systemd know how to manage. Units are categorized by the type of resources they represent and defined in unit configuration files. For example, the avahi.service unit represents the Avahi daemon and is defined in the  **avahi.service**  file.  [Table 1](#en-us_topic_0151921012_t2dcb6d973cc249ed9ccd56729751ca6b)  lists available types of systemd units.

**Table  1**  Available types of systemd units

All available types of systemd units are located in one of the following directories listed in  [Table 2](#en-us_topic_0151921012_t2523a0a9a0c54f9b849e52d1efa0160c).

**Table  2**  Locations of available systemd units

## Features

### Fast Activation

The systemd provides more aggressive parallelization than UpStart. The use of Socket- and D-Bus based activation reduces the time required to boot the operating system.

To accelerate system boot, systemd seeks to:

* Activate only the necessary processes
* Activate as many processes as possible in parallel

### On-Demand Activation

During SysVinit initialization, it activates all the possible background service processes that might be used. Users can log in only after all these service processes are activated. The drawbacks in SysVinit are obvious: slow system boot and a waste of system resources.

Some services may rarely or even never be used during system runtime. For example, CUPS, printing services are rarely used on most servers. SSHD is rarely accessed on many servers. It is unnecessary to spend time on starting these services and system resources.

systemd can only be activated when a service is requested. If the service request is over, systemd stops.

### Service Lifecycle Management by Cgroups

An important role of an init system is to track and manage the lifecycle of services. It can start and stop a service. However, it is more difficult than you could ever imagine to encode an init system into stopping services.

Service processes often run in background as daemons and sometimes fork twice. In UpStart, the expect stanza in the configuration file must be correctly configured. Otherwise, UpStart is unable to learn a daemon's PID by counting the number of forks.

Things are made simpler with Cgroups, which have long been used to manage system resource quotas. The ease of use comes largely from its file-system-like user interface. When a parent service creates a child service, the latter inherits all attributes of the Cgroup to which the parent service belongs. This means that all relevant services are put into the same Cgroup. The systemd can find the PIDs of all relevant services simply by traversing their control group and then stop them one by one.

### Mount and Automount Point Management

In traditional Linux systems, users can use the  **/etc/fstab**  file to maintain fixed file system mount points. These mount points are automatically mounted during system startup. Once the startup is complete, these mount points are available. These mount points are file systems critical to system running, such as the  **HOME**  directory. Like SysVinit, systemd manages these mount points so that they can be automatically mounted at system startup. systemd is also compatible with the  **/etc/fstab**  file. You can continue to use this file to manage mount points.

There are times when you need to mount or unmount on demand. For example, a temporary mounting point is required for you to access the DVD content, and the mounting point is canceled (using the  **umount**  command) if you no longer need to access the content, thereby saving resources. This is traditionally achieved using the autofs service.

The systemd allows automatic mount without a need to install autofs.

### Transactional Dependency Management

System boot involves a host of separate jobs, some of which may be dependent on each other. For example, a network file system (NFS) can be mounted only after network connectivity is activated. The systemd can run a large number of dependent jobs in parallel, but not all of them. Looking back to the NFS example, it is impossible to mount NFS and activate network at the same time. Before running a job, systemd calculates its dependencies, creates a temporary transaction, and verifies that this transaction is consistent (all relevant services can be activated without any dependency on each other).

### Compatibility with SysVinit Scripts

Like UpStart, systemd introduces new configuration methods and has new requirements for application development. If you want to replace the currently running initialization system with systemd, systemd must be compatible with the existing program. It is difficult to modify all the service code in any Linux distribution in a short time for the purpose of using systemd.

The systemd provides features compatible with SysVinit and LSB initscripts. You do not need to modify the existing services and processes in the system. This reduces the cost of migrating the system to systemd, making it possible for users to replace the existing initialization system with systemd.

### System State Snapshots and System Restoration

The systemd can be started on demand. Therefore, the running status of the system changes dynamically, and you cannot know the specific services that are running in the system. systemd snapshots enable the current system running status to be saved and restored.

For example, if services A and B are running in the system, you can run the  **systemd**  command to create a snapshot for the current system running status. Then stop process A or make any other change to the system, for example, starting process C. After these changes, run the snapshot restoration command of systemd to restore the system to the point at which the snapshot was taken. That is, only services A and B are running. A possible application scenario is debugging. For example, when an exception occurs on the server, a user saves the current status as a snapshot for debugging, and then perform any operation, for example, stopping the service. After the debugging is complete, restore the snapshot.

## Managing System Services

The systemd provides the systemctl command to start, stop, restart, view, enable, and disable system services.

### Comparison Between SysVinit and systemd Commands

The  **systemctl**  command from the  **systemd**  command has the functions similar to the  **SysVinit**  command. Note that the  **service**  and  **chkconfig**  commands are supported in this version. For details, see  [Table 3](#en-us_topic_0151920917_ta7039963b0c74b909b72c22cbc9f2e28). You are advised to manage system services by running the  **systemctl**  command.

**Table  3**  Comparison between SysVinit and systemd commands

### Listing Services

To list all currently loaded services, run the following command:

```shell
systemctl list-units --type service
```

To list all services regardless of whether they are loaded, run the following command (with the all option):

```shell
systemctl list-units --type service --all
```

Example list of all currently loaded services:

```shell
$ systemctl list-units --type service
UNIT                        LOAD   ACTIVE     SUB           DESCRIPTION  
atd.service                 loaded active     running       Deferred execution scheduler  
auditd.service              loaded active     running       Security Auditing Service  
avahi-daemon.service        loaded active     running       Avahi mDNS/DNS-SD Stack  
chronyd.service             loaded active     running       NTP client/server  
crond.service               loaded active     running       Command Scheduler  
dbus.service                loaded active     running       D-Bus System Message Bus  
dracut-shutdown.service     loaded active     exited        Restore /run/initramfs on shutdown  
firewalld.service           loaded active     running       firewalld - dynamic firewall daemon  
getty@tty1.service          loaded active     running       Getty on tty1  
gssproxy.service            loaded active     running       GSSAPI Proxy Daemon  
......
```

### Displaying Service Status

To display the status of a service, run the following command:

```shell
systemctl status name.service
```

[Table 4](#en-us_topic_0151920917_t36cd267d69244ed39ae06bb117ed8e62)  describes the parameters in the command output.

**Table  4**  Output parameters

To verify whether a particular service is running, run the following command:

```shell
systemctl is-active name.service
```

The output of the  **is-active**  command is as follows:

**Table  5**  Output of the is-active command

Similarly, to determine whether a particular service is enabled, run the following command:

```shell
systemctl is-enabled name.service
```

The output of the  **is-enabled**  command is as follows:

**Table  6**  Output of the is-enabled command

For example, to display the status of gdm.service, run the  **systemctl status gdm.service**  command.

```shell
# systemctl status gdm.service
gdm.service - GNOME Display Manager
   Loaded: loaded (/usr/lib/systemd/system/gdm.service; enabled)
   Active: active (running) since Thu 2013-10-17 17:31:23 CEST; 5min ago
 Main PID: 1029 (gdm)
   CGroup: /system.slice/gdm.service
           ├─1029 /usr/sbin/gdm
           ├─1037 /usr/libexec/gdm-simple-slave --display-id /org/gno...           
           └─1047 /usr/bin/Xorg :0 -background none -verbose -auth /r...Oct 17 17:31:23 localhost systemd[1]: Started GNOME Display Manager.
```

### Starting a Service

To start a service, run the following command as the user  **root**:

```shell
systemctl start name.service
```

For example, to start the httpd service, run the following command:

```shell
# systemctl start httpd.service
```

### Stopping a Service

To stop a service, run the following command as the user  **root**:

```shell
systemctl stop name.service
```

For example, to stop the Bluetooth service, run the following command:

```shell
# systemctl stop bluetooth.service
```

### Restarting a Service

To restart a service, run the following command as the user  **root**:

```shell
systemctl restart name.service
```

This command stops the selected service in the current session and immediately starts it again. If the selected service is not running, this command starts it too.

For example, to restart the Bluetooth service, run the following command:

```shell
# systemctl restart bluetooth.service
```

### Enabling a Service

To configure a service to start automatically at system boot time, run the following command as the user  **root**:

```shell
systemctl enable name.service
```

For example, to configure the httpd service to start automatically at system boot time, run the following command:

```shell
# systemctl enable httpd.service
ln -s '/usr/lib/systemd/system/httpd.service' '/etc/systemd/system/multi-user.target.wants/httpd.service'
```

### Disabling a Service

To prevent a service from starting automatically at system boot time, run the following command as the user  **root**:

```shell
systemctl disable name.service
```

For example, to prevent the Bluetooth service from starting automatically at system boot time, run the following command:

```shell
# systemctl disable bluetooth.service
Removed /etc/systemd/system/bluetooth.target.wants/bluetooth.service.
Removed /etc/systemd/system/dbus-org.bluez.service.
```

## Changing a Runlevel

### Targets and Runlevels

In systemd, the concept of runlevels has been replaced with systemd targets to improve flexibility. For example, you can inherit an existing target and turn it into your own target by adding other services.  [Table 7](#en-us_topic_0151920939_t9af92c282ad240ea9a79fb08d26e8181)  provides a complete list of runlevels and their corresponding systemd targets.

**Table  7**  Mapping between runlevels and targets

### Viewing the Default Startup Target

Run the following command to view the default startup target of the system:

```shell
systemctl get-default
```

### Viewing All Startup Targets

Run the following command to view all startup targets of the system:

```shell
systemctl list-units --type=target
```

### Changing the Default Target

To change the default target, run the following command as the user  **root**:

```shell
systemctl set-default name.target
```

### Changing the Current Target

To change the current target, run the following command as the user  **root**:

```shell
systemctl isolate name.target
```

### Changing to Rescue Mode

To change the operating system to rescue mode, run the following command as the user  **root**:

```shell
systemctl rescue
```

This command is similar to the  **systemctl isolate rescue.target**  command. After the command is executed, the following information is displayed on the serial port:

```console
You are in rescue mode. After logging in, type "journalctl -xb" to viewsystem logs, "systemctl reboot" to reboot, "systemctl default" or "exit"to boot into default mode.
Give root password for maintenance
(or press Control-D to continue):
```

> \[!NOTE] **NOTE:**
> You need to restart the system to enter the normal mode from the rescue mode.

### Changing to Emergency Mode

To change the operating system to emergency mode, run the following command as the user  **root**:

```shell
systemctl emergency
```

This command is similar to the  **systemctl isolate emergency.target**  command. After the command is executed, the following information is displayed on the serial port:

```console
You are in emergency mode. After logging in, type "journalctl -xb" to viewsystem logs, "systemctl reboot" to reboot, "systemctl default" or "exit"to boot into default mode.
Give root password for maintenance
(or press Control-D to continue):
```

> \[!NOTE] **NOTE:**
> You need to restart the system to enter the normal mode from the emergency mode.

## Shutting Down, Suspending, and Hibernating the Operating System

### systemctl Command

The systemd uses the systemctl command instead of old Linux system management commands to shut down, restart, suspend, and hibernate the operating system. Although previous Linux system management commands are still available in systemd for compatibility reasons, you are advised to use  **systemctl**  when possible. The mapping relationship is shown in  [Table 8](#en-us_topic_0151920964_t3daaaba6a03b4c36be9668efcdb61f3b).

**Table  8**  Mapping between old Linux system management commands and systemctl

### Shutting Down the Operating System

To shut down the system and power off the operating system, run the following command as the user  **root**:

```shell
systemctl poweroff
```

To shut down the operating system without powering it off, run the following command as the user  **root**:

```shell
systemctl halt
```

By default, running either of these commands causes systemd to send an informative message to all login users. To prevent systemd from sending this message, run this command with the  **--no-wall**  option. The command is as follows:

```shell
systemctl --no-wall poweroff
```

### Restarting the Operating System

To restart the operating system, run the following command as the user  **root**:

```shell
systemctl reboot
```

By default, running either of these commands causes systemd to send an informative message to all login users. To prevent systemd from sending this message, run this command with the  **--no-wall**  option. The command is as follows:

```shell
systemctl --no-wall reboot
```

### Suspending the Operating System

To suspend the operating system, run the following command as the user  **root**:

```shell
systemctl suspend
```

### Hibernating the Operating System

To hibernate the operating system, run the following command as the user  **root**:

```shell
systemctl hibernate
```

To suspend and hibernate the operating system, run the following command as the user  **root**:

```shell
systemctl hybrid-sleep
```

---

---
url: /en/docs/22.03_LTS_SP4/server/administration/sysmaster/service_management.md
---
# Service Management

Many background programs and processes in Linux, such as web servers, database servers, and mail servers, are started and stopped during system startup and running. sysmaster provides efficient service management commands and configurations to ensure the normal running of the system.

This document describes the installation and deployment of sysmaster, as well as its features and usage.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/nestos/nestos/usage.md
---
# Setting Up Kubernetes and iSulad

**Unless otherwise specified, perform the following steps on both the master and node.** This tutorial uses the master as an example.

## Before You Start

Prepare **NestOS-22.03-date.x86\_64.iso** and two hosts act as the master and node respectively.

## Downloading the Components

Open the repo source file to add the Alibaba Cloud source of Kubernetes.

```shell
vi /etc/yum.repos.d/openEuler.repo
```

Add the following content:

```text
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
```

Downloads the Kubernetes components and the components for synchronizing the system time.

```shell
rpm-ostree install kubelet kubeadm kubectl ntp ntpdate wget
```

Restart the system to use the components.

```shell
systemctl reboot
```

Select the latest version branch and enter the system.

## Configuring the Environment

### Change the Host Name of the Master

```shell
hostnamectl set-hostname k8s-master
sudo -i
```

Open the **/etc/hosts** file.

```shell
vi /etc/hosts
```

Add the IP addresses of the hosts.

```text
192.168.237.133 k8s-master
192.168.237.135 k8s-node01
```

### Synchronizing the System Time

```shell
ntpdate time.windows.com
systemctl enable ntpd
```

### Disabling the swap Partition, Firewall, and SELinux

By default, the NestOS does not have the swap partition and the firewall is disabled.
Run the following command to disable SELinux:

```shell
vi /etc/sysconfig/selinux
# Change the value of SELINUX to disabled.
```

### Enabling Forwarding Mechanisms

Create a configuration file.

```shell
vi /etc/sysctl.d/k8s.conf
```

Add the following content:

```text
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
```

Make the configuration take effect.

```shell
modprobe br_netfilter
sysctl -p /etc/sysctl.d/k8s.conf
```

## Configuring iSula

Check the OS image required by Kubernetes. Pay attention to the version number of the pause container.

```shell
kubeadm config images list
```

Modify the **daemon.json** configuration file.

```shell
vi /etc/isulad/daemon.json
```

```text
## Description of the added items ##
Set registry-mirrors to "docker.io".
Set insecure-registries to "rnd-dockerhub.huawei.com".
Set pod-sandbox-image to "registry.aliyuncs.com/google_containers/pause:3.5". (The Alibaba Cloud source is used. The pause version is obtained in the previous step.)
Set network-plugin to "cni".
Set cni-bin-dir to "/opt/cni/bin".
Set cni-conf-dir to "/etc/cni/net.d".
```

The modified file is as follows:

```json
{"group": "isula",
"default-runtime": "runc",
"graph": "/var/lib/isulad",
"state": "/var/run/isulad",
"engine": "lcr",
"log-level": "ERROR",
"pidfile": "/var/run/isulad.pid",
"log-opts": {
"log-file-mode": "0600",
"log-path": "/var/lib/isulad",
"max-file": "1",
"max-size": "30KB"
},
"log-driver": "stdout",
"container-log": {
"driver": "json-file"
},
"hook-spec": "/etc/default/isulad/hooks/default.json",
"start-timeout": "2m",
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"registry-mirrors": [
"docker.io"
],
"insecure-registries": [
"rnd-dockerhub.huawei.com"
],
"pod-sandbox-image": "registry.aliyuncs.com/google_containers/pause:3.5",
"native.umask": "secure",
"network-plugin": "cni",
"cni-bin-dir": "/opt/cni/bin",
"cni-conf-dir": "/etc/cni/net.d",
"image-layer-check": false,
"use-decrypted-key": true,
"insecure-skip-verify-enforce": false
}
```

Start the services.

```shell
systemctl restart isulad
systemctl enable isulad
systemctl enable kubelet
```

**Perform the preceding steps on both the master and node.**

## Initializing the Master

**Perform this step only on the master.**
Run the following command and wait for the host to pull the image. You can also manually pull the image before performing this step.

```shell
kubeadm init --kubernetes-version=1.22.2 --apiserver-advertise-
address=192.168.237.133 --cri-socket=/var/run/isulad.sock --image-repository
registry.aliyuncs.com/google_containers --service-cidr=10.10.0.0/16 --pod-
network-cidr=10.122.0.0/16
```

```text
## Description of initialization parameters ##
kubernetes-version indicates the version to be installed.
apiserver-advertise-address indicates the IP address of the master.
cri-socket specifies the iSulad engine.
image-repository specifies that the image source is Alibaba Cloud. You do not need to modify the tag.
service-cidr specifies the IP address range allocated to the service.
pod-network-cidr specifies the IP address range allocated to the Pod network.
```

After the initialization is successful, copy the `kubeadm join` command that is output by `kubeadm init` for subsequent node joining.

```text
kubeadm join 192.168.237.133:6443 --token j7kufw.yl1gte0v9qgxjzjw --discovery-
token-ca-cert-hash
sha256:73d337f5edd79dd4db997d98d329bd98020b712f8d7833c33a85d8fe44d0a4f5 --cri-
socket=/var/run/isulad.sock
```

**Note**: `--cri-socket=/var/run/isulad.sock` specifies that iSulad is used as the container engine.\
View the downloaded image.

```shell
isula images
```

Configure the cluster based on the output of the initialization command.

```shell
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
source /etc/profile
```

Check the health status.

```shell
kubectl get cs
```

The status of **controller-manager** and **scheduler** may be **unhealthy**. To rectify the fault, perform the following steps:\
Edit the configuration file.

```shell
vi /etc/kubernetes/manifests/kube-controller-manager.yaml
```

Comment out the following content:
**--port=0**
Modify hostpath:
Change all **/usr/libexec/kubernetes/kubelet-plugins/volume/exec** to **/opt/libexec/...**

```shell
vi /etc/kubernetes/manifests/kube-scheduler.yaml
```

Comment out the following content:
**--port=0**

After the modification is complete, check the health status again.

## Configuring the Network Plugin

Configure the network plugin only on the master. However, you need to pull images on all hosts in advance. The commands for pulling images are as follows:

```shell
isula pull calico/node:v3.19.3
isula pull calico/cni:v3.19.3
isula pull calico/kube-controllers:v3.19.3
isula pull calico/pod2daemon-flexvol:v3.19.3
```

**Perform the following steps only on the master.**\
Obtain the configuration file.

```shell
wget https://docs.projectcalico.org/v3.19/manifests/calico.yaml
```

Edit **calico.yaml** and change all **/usr/libexec/...** to **/opt/libexec/...**.
Run the following command to install Calico:

```shell
kubectl apply -f calico.yaml
```

Run the `kubectl get pod -n kube-system` command to check whether Calico is successfully installed.
Run the `kubectl get pod -n kube-system` command to check whether all Pods are in the**running** status.

## Joining the Node to the Cluster

Run the following command on the node to join the node to the cluster:

```text
kubeadm join 192.168.237.133:6443 --token j7kufw.yl1gte0v9qgxjzjw --discovery-
token-ca-cert-hash
sha256:73d337f5edd79dd4db997d98d329bd98020b712f8d7833c33a85d8fe44d0a4f5 --cri-
socket=/var/run/isulad.sock
```

Run the `kubectl get node` command to check whether the master and node statuses are **ready**.

If yes, Kubernetes is successfully deployed.

# Using rpm-ostree

## Installing Software Packages Using rpm-ostree

Install wget.

```shell
rpm-ostree install wget
```

Restart the system. During the startup, use the up and down arrow keys on the keyboard to enter system before or after the RPM package installation. **ostree:0** indicates the version after the installation.

```shell
systemctl reboot
```

Check whether wget is successfully installed.

```shell
rpm -qa | grep wget
```

## Manually Upgrading NestOS Using rpm-ostree

Run the following command in NestOS to view the current rpm-ostree status and version:

```shell
rpm-ostree status
```

Run the check command to check whether a new version is available.

```shell
rpm-ostree upgrade --check
```

Preview the differences between the versions.

```shell
rpm-ostree upgrade --preview
```

In the latest version, the nano package is imported.
Run the following command to download the latest ostree and RPM data without performing the deployment.

```shell
rpm-ostree upgrade --download-only
```

Restart NestOS. After the restart, the old and new versions of the system are available. Enter the latest version.

```shell
rpm-ostree upgrade --reboot
```

## Comparing NestOS Versions

Check the status. Ensure that two versions of ostree exist: **LTS.20210927.dev.0** and **LTS.20210928.dev.0**.

```shell
rpm-ostree status
```

Compare the ostree versions based on commit IDs.

```shell
rpm-ostree db diff 55eed9bfc5ec fe2408e34148
```

## Rolling Back the System

When a system upgrade is complete, the previous NestOS deployment is still stored on the disk. If the upgrade causes system problems, you can roll back to the previous deployment.

### Temporary Rollback

To temporarily roll back to the previous OS deployment, hold down **Shift** during system startup. When the boot load menu is displayed, select the corresponding branch from the menu.

### Permanent Rollback

To permanently roll back to the previous OS deployment, log in to the target node and run the `rpm-ostree rollback` command. This operation sets the previous OS deployment as the default deployment to boot into.
Run the following command to roll back to the system before the upgrade:

```shell
rpm-ostree rollback
```

## Switching Versions

NestOS is rolled back to an older version. You can run the following command to switch the rpm-ostree version used by NestOS to a newer version.

```shell
rpm-ostree deploy -r 22.03.20220325.dev.0
```

After the restart, check whether NestOS uses the latest ostree version.

# Using Zincati for Automatic Update

Zincati automatically updates NestOS. Zincati uses the Cincinnati backend to check whether a new version is available. If a new version is available, Zincati downloads it using rpm-ostree.

Currently, the Zincati automatic update service is disabled by default. You can modify the configuration file to set the automatic startup upon system startup for Zincati.

```shell
vi /etc/zincati/config.d/95-disable-on-dev.toml
```

Set **updates.enabled** to true.
Create a configuration file to specify the address of the Cincinnati backend.

```shell
vi /etc/zincati/config.d/update-cincinnati.toml
```

Add the following content:

```text
[cincinnati]
base_url="http://nestos.org.cn:8080"
```

Restart the Zincati service.

```shell
systemctl restart zincati.service
```

When a new version is available, Zincati automatically detects the new version. Check the rpm-ostree status. If the status is **busy**, the system is being upgraded.

After a period of time, NestOS automatically restarts. Log in to NestOS again and check the rpm-ostree status. If the status changes to **idle** and the current version is **20220325**, rpm-ostree has been upgraded.

View the zincati service logs to check the upgrade process and system restart logs. In addition, the information "auto-updates logic enabled" in the logs indicates that the update is automatic.

# Customizing NestOS

You can use the nestos-installer tool to customize the original NestOS ISO file and package the Ignition file to generate a customized NestOS ISO file. The customized NestOS ISO file can be used to automatically install NestOS after the system is started for easy installation.

Before customizing NestOS, make the following preparations:

* Downloading the NestOS ISO.
* Preparing a **config.ign** File.

## Generating a Customized NestOS ISO File

### Setting Parameter Variables

```shell
export COREOS_ISO_ORIGIN_FILE=nestos-22.03.20220324.x86_64.iso
export COREOS_ISO_CUSTOMIZED_FILE=my-nestos.iso
export IGN_FILE=config.ign
```

### Checking the ISO File

Ensure that the original NestOS ISO file does not contain the Ignition configuration.

```shell
$ nestos-installer iso ignition show $COREOS_ISO_ORIGIN_FILE 

Error: No embedded Ignition config.
```

### Generating a Customized NestOS ISO File

Package the Ignition file into the original NestOS ISO file to generate a customized NestOS ISO file.

```shell
nestos-installer iso ignition embed $COREOS_ISO_ORIGIN_FILE --ignition-file $IGN_FILE $COREOS_ISO_ORIGIN_FILE --output $COREOS_ISO_CUSTOMIZED_FILE
```

### Checking the ISO File

Ensure that the customized NestOS ISO file contains the Ignition configuration.

```shell
nestos-installer iso ignition show $COREOS_ISO_CUSTOMIZED_FILE
```

The previous command displays the Ignition configuration.

## Installing the Customized NestOS ISO File

The customized NestOS ISO file can be used to directly boot the installation. NestOS is automatically installed based on the Ignition configuration. After the installation is complete, you can use **nest/password** to log in to NestOS on the VM console.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/administrator/setting_up_the_database_server.md
---
# Setting Up the Database Server

## PostgreSQL Server

### Software Description

[Figure 1](#fig26022387391)  shows the PostgreSQL architecture and  [Table 1](#table62020913417)  describes the main processes.

**Figure  1**  PostgreSQL architecture\
![](./figures/postgresql-architecture.png)

**Table  1**  Main processes in PostgreSQL

### Configuring the Environment

> \[!NOTE] **NOTE:**
> The following environment configuration is for reference only. Configure the environment based on the site requirements.

#### Disabling the Firewall and Automatic Startup

> \[!NOTE] **NOTE:**
> It is recommended that firewall be disabled in the test environment to prevent network impact. Configure the firewall based on actual requirements.

1. Stop the firewall service as the **root** user.

   ```shell
   systemctl stop firewalld
   ```

2. Disable the firewall service as the **root** user.

   ```shell
   systemctl disable firewalld
   ```

   > \[!NOTE] **NOTE:**
   > The automatic startup is automatically disabled as the firewall is disabled.

#### Disabling SELinux

1. Modify the configuration file as the **root** user.

   ```shell
   sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
   ```

#### Creating a User Group and a User

> \[!NOTE] **NOTE:**
> In the server environment, independent users are assigned to each process to implement permission isolation for security purposes. The user group and user are created for the OS, not for the database.

1. Create a PostgreSQL user or user group as the **root** user.

   ```shell
   groupadd  postgres
   ```

   ```shell
   useradd  -g postgres postgres
   ```

2. Set the postgres user password as the **root** user. (Enter the password twice for confirmation.)

   ```shell
   passwd postgres
   ```

#### Creating Data Drives

> \[!NOTE] **NOTE:**
>
> * When testing the ultimate performance, you are advised to attach NVMe SSDs with better I/O performance to create PostgreSQL test instances to avoid the impact of disk I/O on the performance test result. This section uses NVMe SSDs as an example. For details, see Step 1 to Step 4.
> * In a non-performance test, run the following command as the **root** user to create a data directory. Then skip this section.\
>   \# mkdir /data

1. Create a file system (xfs is used as an example as the **root** user. Create the file system based on the site requirements.). If a file system has been created for a disk, an error will be reported when you run this command. You can use the  **-f**  parameter to forcibly create a file system.

   ```shell
   mkfs.xfs /dev/nvme0n1
   ```

2. Create a data directory.

   ```shell
   mkdir /data
   ```

3. Mount disks.

   ```shell
   mount -o noatime,nobarrier /dev/nvme0n1 /data
   ```

#### Data Directory Authorization

1. Modify the directory permission as the **root** user.

   ```shell
   chown -R postgres:postgres /data/
   ```

### Installing, Running, and Uninstalling PostgreSQL

#### Installing PostgreSQL

1. Configure the local yum repository. For details, see  [Configuring the Repo Server](./configuring_the_repo_server.md).

2. Clear the cache.

   ```shell
   dnf clean all
   ```

3. Create a cache.

   ```shell
   dnf makecache
   ```

4. Install the PostgreSQL server as the **root** user.

   ```shell
   dnf install postgresql-server
   ```

5. Check the installed RPM package.

   ```shell
   rpm -qa | grep postgresql
   ```

#### Running PostgreSQL

##### Initializing the Database

> \[!TIP] **NOTICE:**
> Perform this step as the postgres user.

1. Switch to the created PostgreSQL user.

   ```shell
   su - postgres
   ```

2. Initialize the database. In the command,  **/usr/bin**  is the directory where the  **initdb**  command is located.

   ```shell
   /usr/bin/initdb -D /data/
   ```

##### Starting the Database

1. Enable the PostgreSQL database.

   ```shell
   /usr/bin/pg_ctl -D /data/ -l /data/logfile start
   ```

2. Check whether the PostgreSQL database process is started properly.

   ```shell
   ps -ef | grep postgres
   ```

   If the following information is displayed, the PostgreSQL processes have been started.

   ![](./figures/postgres.png)

##### Logging In to the Database

1. Log in to the database.

   ```shell
   /usr/bin/psql -U postgres
   ```

   ```text
   psql (13.3)
   Type "help" for help.

   postgres=#
   ```

   > \[!NOTE] **NOTE:**
   > You do not need to enter a password when logging in to the database for the first time.

##### Configuring the Database Accounts and Passwords

1. After login, set the postgres user password.

   ```shell
   postgres=#alter user postgres with password '123456';
   ```

   ![](./figures/en-us_image_0230050789.png)

##### Exiting the Database

1. Run  **\q**  to exit from the database.

   ```shell
   postgres=# \q
   ```

##### Stopping the Database

1. Stop the PostgreSQL database.

   ```shell
   /usr/bin/pg_ctl -D /data/ -l /data/logfile stop
   ```

#### Uninstalling PostgreSQL

1. Stop the database as the postgres user.

   ```shell
   /usr/bin/pg_ctl -D /data/ -l /data/logfile stop
   ```

2. Run the  **dnf remove postgresql-server**  command as the user  **root**  to uninstall the PostgreSQL database.

   ```shell
   dnf remove postgresql-server
   ```

### Managing Database Roles

#### Creating a Role

You can use the  **CREATE ROLE**  statement or  **createuser**  command to create a role. The  **createuser**  command encapsulates the  **CREATE ROLE**  statement and needs to be executed on the shell GUI instead of the database GUI.

```pgsql
CREATE ROLE rolename [ [ WITH ] option [ ... ] ];
```

```shell
createuser rolename
```

In the preceding information:

* **rolename**: indicates a role name.
* Parameters of the  *option*  are as follows:
  * **SUPERUSER | NOSUPERUSER**: determines whether a new role is a superuser. If this parameter is not specified, the default value **NOSUPERUSER**  is used, indicating that the role is not a superuser.
  * **CREATEDB | NOCREATEDB**: specifies whether a role can create a database. If this parameter is not specified, the default value  **NOCREATEDB**  is used, indicating that the role cannot create a database.
  * **CREATEROLE | NOCREATEROLE**: determines whether a role can create roles. If this parameter is not specified, the default value  **NOCREATEROLE**  is used, indicating that the role cannot create roles.
  * **INHERIT | NOINHERIT**: determines whether a role inherits the other roles' permissions in the group to which the role belongs. A role with the INHERIT attribute can automatically use any permissions that have been assigned to its direct or indirect group. If this parameter is not specified, the default value  **INHERIT**  is used.
  * **LOGIN | NOLOGIN**: determines whether a role can log in. A role with the LOGIN attribute can be considered as a user. A role without this attribute can be used to manage database permissions but is not a user. If this attribute is not specified, the default value  **NOLOGIN**  is used. However, if  **CREATE USER**  instead of  **CREATE ROLE**  is used to create a role, the LOGIN attribute is used by default.
  * **\[ENCRYPTED | UNENCRYPTED] PASSWORD'password'**: password of a role. The password is valid only for roles with the LOGIN attribute.  **ENCRYPTED | UNENCRYPTED**: determines whether to encrypt the password. If this parameter is not specified, the value  **ENCRYPTED**  is used, that is, the password is encrypted.
  * **VALID UNTIL'timestamp'**: specifies the timestamp when the password of a role expires. If this parameter is not specified, the password is permanently valid.
  * **IN ROLE rolename1**: lists one or more existing roles. The new role  *rolename*  will be added to and become a member of  **rolename1**.
  * **ROLE rolename2**: lists one or more existing roles. These roles will be automatically added as members of the new role  *rolename*. That is, the new role is a user group.

To run this command, you must have the CREATEROLE permission or is the database superuser.

##### Example

\# Create a role  **roleexample1**  who can log in.

```shell
postgres=# CREATE ROLE roleexample1 LOGIN;
```

\# Create a role  **roleexample2**  with the password  **123456**.

```shell
postgres=# CREATE ROLE roleexample2 WITH LOGIN PASSWORD '123456';
```

\# Create a role named  **roleexample3**.

```console
[postgres@localhost ~]$ createuser roleexample3
```

#### Viewing Roles

You can run the  **SELECT**  statement or the PostgreSQL meta-command  **\du**  to view the role.

```pgsql
SELECT rolename FROM pg_roles;
```

```pgsql
\du
```

In the preceding command,  *rolename*  indicates the role name.

##### Example

\# View the  **roleexample1**  role.

```shell
postgres=# SELECT roleexample1 from pg_roles;
```

\# View the existing roles.

```shell
postgres=# \du
```

#### Modifying a Role

##### Modifying a Username

Use the  **ALTER ROLE**  statement to modify an existing role name.

```pgsql
ALTER ROLE oldrolername RENAME TO newrolename;
```

In the preceding information:

* *oldrolername*: original role name.
* *newrolename*: new role name.

##### Example of Modifying a User

\# Change the role name  **roleexample1**  to  **roleexapme2**.

```shell
postgres=# ALTER ROLE roleexample1 RENAME TO roleexample2;
```

##### Modifying a User Password

Use the  **ALTER ROLE**  statement to modify the login password of a role.

```pgsql
ALTER ROLE rolename PASSWORD 'password'
```

In the preceding information:

* *rolename*: indicates a role name.
* *password*: password.

##### Example of Modifying the Password of a Role

\# Modify the password of  **roleexample1**  to  **456789**.

```shell
postgres=# ALTER ROLE roleexample1 WITH PASSWORD '456789';
```

#### Deleting a Role

You can use the  **DROP ROLE**  statement or  **dropuser**  command to delete a role. The  **dropuser**  command encapsulates the  **DROP ROLE**  statement and needs to be executed on the shell GUI instead of the database GUI.

```pgsql
DROP ROLE rolename;
```

```shell
dropuser rolename
```

In the preceding command,  *rolename*  indicates the role name.

##### Example

\# Delete the  **userexample1**  role.

```shell
postgres=# DROP ROLE userexample1;
```

\# Delete the  **userexample2**  role.

```console
[postgres@localhost ~]$ dropuser userexample2
```

#### Role Permissions

You can use the  **GRANT**  statement to grant permissions to a role.

Grant the table operation permission to a role.

```pgsql
GRANT { { SELECT | INSERT | UPDATE | DELETE | REFERENCES | TRIGGER } [,...] | ALL [ PRIVILEGES ] } ON [ TABLE ] tablename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ]
```

Grant the sequence operation permission to a role.

```pgsql
GRANT { { USAGE | SELECT | UPDATE } [,...] | ALL [ PRIVILEGES ] } ON SEQUENCE sequencename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] 
```

Grant the database operation permission to a role.

```pgsql
GRANT { { CREATE | CONNECT | TEMPORARY | TEMP } [,...] | ALL [ PRIVILEGES ] } ON DATABASE databasename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ]  
```

Grant the function operation permission to a role.

```pgsql
GRANT { EXECUTE | ALL [ PRIVILEGES ] } ON FUNCTION funcname ( [ [ argmode ] [ argname ] argtype [, ...] ] ) [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ]  
```

Grant the operation permission of the procedural language to a role.

```pgsql
GRANT { USAGE | ALL [ PRIVILEGES ] } ON LANGUAGE langname [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ]  
```

Grant the schema operation permission to a role.

```pgsql
GRANT { { CREATE | USAGE } [,...] | ALL [ PRIVILEGES ] } ON SCHEMA schemaname [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ]  
```

Grant the tablespace operation permission to a role.

```pgsql
GRANT { CREATE | ALL [ PRIVILEGES ] } ON TABLESPACE tablespacename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] 
```

Assign the member relationship of rolename1 to rolename2.

```pgsql
GRANT rolename1 [, ...] TO rolename2 [, ...] [ WITH ADMIN OPTION ]
```

In the preceding information:

* **SELECT**,  **INSERT**,  **UPDATE**,  **DELETE**,  **REFERENCES**,  **TRIGGER**,  **USAGE**,  **CREATE**,  **CONNECT**,  **TEMPORARY**,  **TEMP**,  **EXECUTE**, and  **ALL \[***PRIVILEGES***]**  indicate user operation permissions.  **ALL \[***PRIVILEGES***]**  indicates all permissions, the  *PRIVILEGES*  keyword is optional in PostgreSQL, but it is required in strict SQL statements.
* **ON**  clause: specifies the object on which the permission is granted.
* **tablename**: table name.
* **TO**  clause: specifies the role to which the permission is granted.
* **rolename**,  **rolename1**, and  **rolename2**: role names.
* **groupname**: name of a role group.
* **PUBLIC**: indicates that the permission is granted to all roles, including users who may be created later.
* **WITH GRANT OPTION**: indicates that the recipient of a permission can grant the permission to others. This option cannot be assigned to PUBLIC.
* **sequencename**: sequence name.
* **databasename**: database name.
* **funcname (\[\[argmode] \[argname] argtype \[, ...]])**: function name and its parameters.
* **langname**: procedural language name.
* **schemaname**: schema name.
* **tablespacename**: tablespace name.
* **WITH ADMIN OPTION**: A member can assign the member relationship of a role to other roles and cancel the member relationship of other roles.

##### Example

\# Grant the CREATE permission on database1 to userexample.

```shell
postgres=# GRANT CREATE ON DATABASE database1 TO userexample; 
```

\# Grant all permissions on table1 to all users.

```shell
postgres=# GRANT ALL PRIVILEGES ON TABLE table1 TO PUBLIC; 
```

#### Deleting User Permissions

You can use the  **REVOKE**  statement to revoke the permissions previously granted to one or more roles.

Revoke the table operation permission from a role.

```pgsql
REVOKE [ GRANT OPTION FOR ] { { SELECT | INSERT | UPDATE | DELETE | REFERENCES | TRIGGER } [,...] | ALL [ PRIVILEGES ] } ON [ TABLE ] tablename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...]
```

Revoke the sequence operation permission from a role.

```pgsql
REVOKE [ GRANT OPTION FOR ] { { USAGE | SELECT | UPDATE } [,...] | ALL [ PRIVILEGES ] } ON SEQUENCE sequencename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ]  
```

Revoke the database operation permission from a role.

```pgsql
REVOKE [ GRANT OPTION FOR ] { { CREATE | CONNECT | TEMPORARY | TEMP } [,...] | ALL [ PRIVILEGES ] } ON DATABASE databasename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] 
```

Revoke the function operation permission from a role.

```pgsql
REVOKE [ GRANT OPTION FOR ] { EXECUTE | ALL [ PRIVILEGES ] } ON FUNCTION funcname ( [ [ argmode ] [ argname ] argtype [, ...] ] ) [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] 
```

Revoke the procedural language operation permission from a role.

```pgsql
REVOKE [ GRANT OPTION FOR ] { USAGE | ALL [ PRIVILEGES ] } ON LANGUAGE langname [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] 
```

Revoke the schema operation permission from a role.

```pgsql
REVOKE [ GRANT OPTION FOR ] { { CREATE | USAGE } [,...] | ALL [ PRIVILEGES ] } ON SCHEMA schemaname [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ]  
```

Revoke the tablespace operation permission from a role.

```pgsql
REVOKE [ GRANT OPTION FOR ] { CREATE | ALL [ PRIVILEGES ] } ON TABLESPACE tablespacename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] 
```

Revoke the member relationship of rolename1 from rolename2.

```pgsql
REVOKE [ ADMIN OPTION FOR ] rolename1 [, ...] FROM rolename2 [, ...] [ CASCADE | RESTRICT ]
```

In the preceding information:

* **GRANT OPTION FOR**: The permission cannot be granted to others, but permission itself is not revoked.
* **SELECT**,  **INSERT**,  **UPDATE**,  **DELETE**,  **REFERENCES**,  **TRIGGER**,  **USAGE**,  **CREATE**,  **CONNECT**,  **TEMPORARY**,  **TEMP**,  **EXECUTE**, and  **ALL \[***PRIVILEGES***]**  indicate user operation permissions.  **ALL \[***PRIVILEGES***]**  indicates all permissions, the  *PRIVILEGES*  keyword is optional in PostgreSQL, but it is required in strict SQL statements.
* **ON**  clause: specifies the object on which the permission is revoked.
* *tablename*: table name.
* **FROM**  clause: specifies the role whose permission is revoked.
* *rolename*,  *rolename1*, and  *rolename2*: role names.
* *groupname*: name of a role group.
* **PUBLIC**: revokes the implicitly defined groups that have all roles. However, this does not mean that all roles lose the permissions. The permissions directly obtained and the permissions obtained through a group are still valid.
* *sequencename*: sequence name.
* **CASCADE**: revokes all dependent permissions.
* **RESTRICT**: does not revoke all dependent permissions.
* *databasename*: database name.
* **funcname (***\[\[argmode] \[argname] argtype \[, ...]]***)**: function name and its parameters.
* *langname*: procedural language name.
* *schemaname*: schema name.
* *tablespacename*: tablespace name.
* **ADMIN OPTION FOR**: The transferred authorization is not automatically revoked.

##### Example

\# Grant the CREATE permission on database1 to userexample.

```shell
postgres=# GRANT CREATE ON DATABASE database1 TO userexample; 
```

\# Grant all permissions on table1 to all users.

```shell
postgres=# GRANT ALL PRIVILEGES ON TABLE table1 TO PUBLIC; 
```

### Managing Databases

#### Creating a Database

You can use the  **CREATE DATABASE**  statement or the  **createdb**  command to create a database. The  **createdb**  command encapsulates the  **CREATE DATABASE**  statement and needs to be executed on the shell GUI instead of the database GUI.

```pgsql
CREATE DATABASE databasename;
```

```shell
createdb databasename
```

In the preceding command,  **databasename**  indicates the database name.

To use this command, you must have the CREATEDB permission.

##### Example

\# Create a database named  **database1**.

```shell
postgres=# CREATE DATABASE database1;
```

#### Selecting a Database

Use the  **\c**  statement to select a database.

```pgsql
\c databasename;
```

In the preceding command,  **databasename**  indicates the database name.

##### Example

\# Select the  **databaseexample**  database.

```shell
postgres=# \c databaseexample;
```

#### Viewing a Database

Use the  **\l**  statement to view the database.

```pgsql
\l;
```

##### Example

\# View all databases.

```shell
postgres=# \l;
```

#### Deleting a Database

You can run the  **DROP DATABASE**  statement or  **dropdb**  command to delete a database. The  **dropdb**  command encapsulates the  **DROP DATABASE**  statement and needs to be executed on the shell GUI instead of the database GUI.

> \[!CAUTION]CAUTION:
> Exercise caution when deleting a database. Once a database is deleted, all tables and data in the database will be deleted.

```pgsql
DROP DATABASE databasename;
```

```shell
dropdb databasename
```

In the preceding command,  **databasename**  indicates the database name.

The  **DROP DATABASE**  statement deletes the system directory items of the database and the file directories that contain data.

**DROP DATABASE**  can be executed only by the super administrator or database owner.

##### Example

\# Delete the  **databaseexample**  database.

```shell
postgres=# DROP DATABASE databaseexample;
```

#### Backing Up a Database

Run the  **pg\_dump**  command to back up the database and dump the database to a script file or another archive file.

```shell
pg_dump [option]... [databasename] > outfile
```

In the preceding information:

* *databasename*: database name. If this parameter is not specified, the environment variable  **PGDATABASE**  is used. If that environment variable is not specified, use the username that initiates the connection.
* *outfile*: database backup file.
* *option*: parameter option of the  **pg\_dump**  command. Multiple parameters can be separated by spaces. The common parameters of the  **pg\_dump**  command are as follows:
  * **-f, --file**= *filename*: specified output file. If this parameter is ignored, the standard output is used.
  * **-d, --dbname**= *databasename*: database to be dumped.
  * **-h, --host**= *hostname*: specifies the hostname.
  * **-p, --port**= *portnumber*: port number.
  * **-U, --username**= *username*: username of the connection.
  * **-W, --password**: forces PostgreSQL to prompt for a password before connecting to a database.

##### Example

\# Back up the database1 database of user  **postgres**  on port  **3306**  of the host whose IP address is  **192.168.202.144**  to the  **db1.sql**  file.

```shell
[postgres@localhost ~]$ pg_dump -h 192.168.202.144 -p 3306 -U postgres -W database1 > db1.sql
```

#### Restoring a Database

Run the  **psql**  command to restore the database.

```shell
psql [option]... [databasename [username]] < infile
```

In the preceding information:

* *databasename*: database name. If this parameter is not specified, the environment variable  **PGDATABASE**  is used. If that environment variable is not specified, use the username that initiates the connection.
* *username*: name of a user.
* *infile*:  **outfile**  parameter in the  **pg\_dump**  command.
* *option*: parameter option of the  **psql**  command. Multiple parameters can be separated by spaces. The common parameters of the  **psql**  command are as follows:
  * **-f, --file**= *filename*: specified output file. If this parameter is ignored, the standard output is used.
  * **-d, --dbname**= *databasename*: database to be dumped.
  * **-h, --host**= *hostname*: specifies the hostname.
  * **-p, --port**= *portnumber*: port number.
  * **-U, --username**= *username*: username of the connection.
  * **-W, --password**: forces PostgreSQL to prompt for a password before connecting to a database.

The  **psql**  command cannot be used to automatically create the  **databasename**  database. Therefore, you need to create the  **databasename**  database before running the  **psql**  command to restore the database.

##### Example

\# Import the  **db1.sql**  script file to the newdb database of the postgres user on the host  **192.168.202.144**  through port  **3306**.

```shell
[postgres@localhost ~]$ createdb newdb
[postgres@localhost ~]$ psql -h 192.168.202.144 -p 3306 -U postgres -W -d newdb < db1.sql
```

## MariaDB Server

### Software Description

The MariaDB database management system is a branch of MySQL and is maintained by the open-source community. The MariaDB database management system uses the General Public License (GPL). MariaDB is designed to be fully compatible with MySQL, including APIs and command lines, so that it can easily replace MySQL. MariaDB also provides many new features.

[Figure 2](#fig13492418164520)  shows the MariaDB architecture.

**Figure  2**  MariaDB logical architecture\
![](./figures/mariadb-logical-architecture.png)

When MariaDB receives a SQL statement, the execution process is as follows:

1. When a client connects to MariaDB, the hostname, username, and password of the client are authenticated. The authentication function can be implemented as a plug-in.
2. If the login is successful, the client sends SQL commands to the server. The parser parses the SQL statements.
3. The server checks whether the client has the permission to obtain the required resources.
4. If the query has been stored in the query cache, the result is returned immediately.
5. The optimizer will find the fastest execution policy or plan. That is, the optimizer can determine which tables will be read, which indexes will be accessed, and which temporary tables will be used. A good policy can reduce a large number of disk access and sorting operations.
6. Storage engines read and write data and index files. Caches are used to accelerate these operations. Other features such as transactions and foreign keys are processed at the storage engine layer.

Storage engines manage and control data at the physical layer. They manage data files, data, indexes, and caches, making data management and reading more efficient. Each table has a .frm file that contains table definitions.

Each storage engine manages and stores data in different ways, and supports different features and performance. For example:

* MyISAM: suitable for environments with more reads and fewer writes. It does not support transactions and supports full-text indexes.
* noDB: supports transactions, row locks, and foreign keys.
* MEMORY: stores data in the memory.
* CSV: stores data in CSV format.

### Configuring the Environment

> \[!NOTE] **NOTE:**
> The following environment configuration is for reference only. Configure the environment based on the site requirements.

#### Disabling the Firewall and Automatic Startup

> \[!NOTE] **NOTE:**
> It is recommended that firewall be disabled in the test environment to prevent network impact. Configure the firewall based on actual requirements.

1. Stop the firewall service as the **root** user.

   ```shell
   systemctl stop firewalld
   ```

2. Disable the firewall service as the **root** user.

   ```shell
   systemctl disable firewalld
   ```

   > \[!NOTE] **NOTE:**
   > The automatic startup is automatically disabled as the firewall is disabled.

#### Disabling SELinux

1. Modify the configuration file as the **root** user.

   ```shell
   sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
   ```

#### Creating a User Group and a User

> \[!NOTE] **NOTE:**
> In the server environment, independent users are assigned to each process to implement permission isolation for security purposes. The user group and user are created for the OS, not for the database.

1. Create a MySQL user or user group as the **root** user.

   ```shell
   groupadd mysql
   ```

   ```shell
   useradd -g mysql mysql
   ```

2. Set the user password as the **root** user.

   ```shell
   passwd mysql
   ```

   Enter the password twice for confirmation.

#### Creating Data Drives

> \[!NOTE] **NOTE:**
>
> * If a performance test needs to be performed, an independent drive is required for the data directory. You need to format and mount the drive. For details, see Method 1 or Method 2.
> * In a non-performance test, run the following command as the **root** user to create a data directory. Then skip this section.\
>   \# mkdir /data

##### Method 1: Using fdisk for Drive Management as the **root** user

1. Create a partition, for example,  **/dev/sdb**.

   ```shell
   fdisk /dev/sdb
   ```

2. Enter  **n**  and press  **Enter**.

3. Enter  **p**  and press  **Enter**.

4. Enter  **1**  and press  **Enter**.

5. Retain the default settings and press  **Enter**.

6. Retain the default settings and press  **Enter**.

7. Enter  **w**  and press  **Enter**.

8. Create a file system, for example,  **xfs**.

   ```shell
   mkfs.xfs /dev/sdb1
   ```

9. Mount the partition to  **/data**  for the OS.

   ```shell
   mkdir /data
   ```

   ```shell
   mount /dev/sdb1 /data
   ```

10. Run the  **vi /etc/fstab**  command and edit the  **/etc/fstab**  file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure.

    In the last line,  **/dev/nvme0n1p1**  is only an example.

    ![](./figures/creat_datadisk1.png)

##### Method 2: Using LVM for Drive Management as the **root** user

> \[!NOTE] **NOTE:**
> Install the LVM2 package in the image as follows:
>
> 1. Configure the local yum repository. For details, see  [Configuring the Repo Server](./configuring_the_repo_server.md). If the repository has been configured, skip this step.
> 2. Install LVM2.\
>    **# yum install lvm2**

1. Create a physical volume, for example,  **sdb**.

   ```shell
   pvcreate /dev/sdb
   ```

2. Create a physical volume group, for example,  **datavg**.

   ```shell
   vgcreate datavg  /dev/sdb
   ```

3. Create a logical volume, for example,  **datalv**  of 600 GB.

   ```shell
   lvcreate -L 600G -n datalv datavg
   ```

4. Create a file system.

   ```shell
   mkfs.xfs /dev/datavg/datalv
   ```

5. Create a data directory and mount it.

   ```shell
   mkdir /data
   ```

   ```shell
   mount /dev/datavg/datalv /data
   ```

6. Run the  **vi /etc/fstab**  command and edit the  **/etc/fstab**  file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure.

   In the last line,  **/dev/datavg/datalv**  is only an example.

   ![](./figures/D1376B2A-D036-41C4-B852-E8368F363B5E.png)

#### Creating a Database Directory and Granting Permissions

1. In the created data directory  **/data**, create directories for processes and grant permissions to the MySQL group or user created as the **root** user.

   ```shell
   mkdir -p /data/mariadb
   cd /data/mariadb
   mkdir data tmp run log
   chown -R mysql:mysql /data
   ```

### Installing, Running, and Uninstalling MariaDB Server

#### Installing MariaDB

1. Configure the local yum repository. For details, see  [Configuring the Repo Server](./configuring_the_repo_server.md).

2. Clear the cache.

   ```shell
   dnf clean all
   ```

3. Create a cache.

   ```shell
   dnf makecache
   ```

4. Install the MariaDB server.

   ```shell
   dnf install mariadb-server
   ```

5. Check the installed RPM package.

   ```shell
   rpm -qa | grep mariadb
   ```

#### Running MariaDB Server

1. Start the MariaDB server as the **root** user.

   ```shell
   systemctl start mariadb
   ```

2. Initialize the database as the **root** user.

   ```shell
   /usr/bin/mysql_secure_installation
   ```

   During the command execution, you need to enter the password of the database user  **root**. If no password is set, press  **Enter**. Then, set the password as prompted.

3. Log in to the database.

   ```shell
   mysql -u root -p
   ```

   After the command is executed, the system prompts you to enter the password. The password is the one set in  [2](#li197143190587).

   > \[!NOTE] **NOTE:**
   > Run the  **\q**  or  **exit**  command to exit the database.

#### Uninstalling MariaDB

1. Stop the database process as the **root** user.

   ```shell
   $ ps -ef | grep mysql
   # kill -9 PID
   ```

2. Run the  **dnf remove mariadb-server**  command as the **root** user to uninstall MariaDB.

   ```shell
   dnf remove mariadb-server
   ```

### Managing Database Users

#### Creating Users

Run the  **CREATE USER**  statement to create one or more users and set corresponding passwords.

```pgsql
CREATE USER 'username'@'hostname' IDENTIFIED BY 'password';
```

In the preceding information:

* *username*: name of a user.
* *host*: hostname, that is, the name of the host where the user connects to the database. As a local user, you can set the parameter to  **localhost**. If the host name is not specified during user creation, the host name is  **%**  by default, indicating a group of hosts.
* *password*: password for logging in to the server. The password can be null. If the password is null, the user can log in to the server without entering the password. This method, however, is not recommended because it provides low security.

To use the  **CREATE USER**  statement, you must have the INSERT permission on the database or the global CREATE USER permission.

After a user account is created using the  **CREATE USER**  statement, a record is added to the user table in the database. If the account to be created exists, an error will occur during statement execution.

A new user has few permissions and can perform only operations that do not require permissions. For example, a user can run the  **SHOW**  statement to query the list of all storage engines and character sets.

##### Example

\# Create a local user whose password is 123456 and username is userexample1.

```pgsql
> CREATE USER 'userexample1'@'localhost' IDENTIFIED BY '123456';
```

\# Create a user whose password is 123456, username is userexample2, and hostname is 192.168.1.100.

```pgsql
> CREATE USER 'userexample2'@'192.168.1.100' IDENTIFIED BY '123456';
```

#### Viewing Users

Run the  **SHOW GRANTS**  or  **SELECT**  statement to view one or more users.

View a specific user:

```pgsql
SHOW GRANTS [FOR 'username'@'hostname'];
```

```pgsql
SELECT USER,HOST,PASSWORD FROM mysql.user WHERE USER='username';
```

View all users:

```pgsql
SELECT USER,HOST,PASSWORD FROM mysql.user;
```

In the preceding information:

* *username*: name of a user.
* *hostname*: host name.

##### Example

\# View the user userexample1.

```pgsql
> SHOW GRANTS FOR 'userexample1'@'localhost';
```

\# View all users in the MySQL database.

```pgsql
> SELECT USER,HOST,PASSWORD FROM mysql.user;
```

#### Modifying Users

##### Modifying a Username

Run the  **RENAME USER**  statement to change one or more existing usernames.

```pgsql
RENAME USER 'oldusername'@'hostname' TO 'newusername'@'hostname';
```

In the preceding information:

* *oldusername*: original username.
* *newusername*: new username.
* *hostname*: host name.

The  **RENAME USER**  statement is used to rename an existing account. If the original account does not exist in the system or the new account exists, an error will occur when the statement is executed.

To use the  **RENAME USER**  statement, you must have the UPDATE permission on the database or the global CREATE USER permission.

##### Example of Modifying a User

\# Change the username  **userexample1**  to  **userexample2**  and change the hostname to  **localhost**.

```pgsql
> RENAME USER 'userexample1'@'localhost' TO 'userexample2'@'localhost';
```

##### Modifying a User Password

Use the  **SET PASSWORD**  statement to modify the login password of a user.

```pgsql
SET PASSWORD FOR 'username'@'hostname' = PASSWORD('newpassword');
```

In the preceding information:

* **FOR 'username'@'hostname'**: specifies the username and hostname whose password is to be changed. This parameter is optional.
* **PASSWORD('newpassword')**: indicates that the  **PASSWORD()**  function is used to set a new password. That is, the new password must be transferred to the  **PASSWORD()**  function for encryption.

> \[!CAUTION]CAUTION:
> The  **PASSWORD()**  function is a unidirectional encryption function. Once encrypted, the original plaintext cannot be decrypted.

If the  **FOR**  clause is not added to the  **SET PASSWORD**  statement, the password of the current user is changed.

The  **FOR**  clause must be given in the format of  **'***username***'@'***hostname***'**, where  *username*  indicates the username of the account and  *hostname*  indicates the hostname of the account.

The account whose password is to be changed must exist in the system. Otherwise, an error occurs when the statement is executed.

##### Example of Changing a User Password

\# Change the password of user  **userexample**  whose hostname is  **localhost**  to  **0123456**.

```pgsql
> SET PASSWORD FOR 'userexample'@'localhost' = PASSWORD('0123456') ;
```

#### Deleting Users

Use the  **DROP USER**  statement to delete one or more user accounts and related permissions.

```pgsql
DROP USER 'username1'@'hostname1' [,'username2'@'hostname2']...;
```

> \[!CAUTION]CAUTION:
> The deletion of users does not affect the tables, indexes, or other database objects that they have created, because the database does not record the accounts that have created these objects.

The  **DROP USER**  statement can be used to delete one or more database accounts and their original permissions.

To use the  **DROP USER**  statement, you must have the DELETE permission on the database or the global CREATE USER permission.

In the  **DROP USER**  statement, if the hostname of an account is not specified, the hostname is  **%**  by default.

##### Example

\# Delete the local user  **userexample**.

```pgsql
> DROP USER 'userexample'@'localhost';
```

#### Granting Permissions to a User

Run the  **GRANT**  statement to grant permissions to a new user.

```pgsql
GRANT privileges ON databasename.tablename TO 'username'@'hostname';
```

In the preceding information:

* **ON**  clause: specifies the object and its level on which the permission is granted.
* **privileges**: indicates the operation permissions of a user, such as  **SELECT**, INSERT, and  **UPDATE**. To grant all permissions to a user, use  **ALL**.
* *databasename*: database name.
* *tablename*: table name.
* **TO**  clause: sets the user password and specifies the user to whom the permission is granted.
* *username*: name of a user.
* *hostname*: host name.

To grant the user the permission to operate all databases and tables, use asterisks (\*), for example,  **\*.\***.

If you specify a password for an existing user in the  **TO**  clause, the new password will overwrite the original password.

If the permission is granted to a non-existent user, a  **CREATE USER**  statement is automatically executed to create the user, but the password must be specified for the user.

##### Example

\# Grant the SELECT and INSERT permissions to local user userexample.

```pgsql
> GRANT SELECT,INSERT ON *.* TO 'userexample'@'localhost'; 
```

#### Deleting User Permissions

Run the  **REVOKE**  statement to delete the permissions of a user, but the user will not be deleted.

```pgsql
REVOKE privilege ON databasename.tablename FROM 'username'@'hostname';
```

The parameters in the  **REVOKE**  statement are the same as those in the  **GRANT**  statement.

To use the  **REVOKE**  statement, you must have the global CREATE USER or UPDATE permission for the database.

##### Example

\# Delete the INSERT permission of local user userexample.

```pgsql
> REVOKE INSERT ON *.* FROM 'userexample'@'localhost';
```

### Managing Databases

#### Creating a Database

Run the  **CREATE DATABASE**  statement to create a database.

```pgsql
CREATE DATABASE databasename;
```

In the preceding command,  *databasename*  can be replaced with the database name, which is case insensitive.

##### Example

\# Create a database named  **databaseexample**.

```pgsql
> CREATE DATABASE databaseexample;
```

#### Viewing a Database

Run the  **SHOW DATABASES**  statement to view a database.

```pgsql
SHOW DATABASES;
```

##### Example

\# View all databases.

```pgsql
> SHOW DATABASES;
```

#### Selecting a Database

Generally, you need to select a target database before creating or querying a table. Use the  **USE**  statement to select a database.

```pgsql
USE databasename;
```

In the preceding command,  **databasename**  indicates the database name.

##### Example

\# Select the  **databaseexample**  database.

```pgsql
> USE databaseexample;
```

#### Deleting a Database

You can run the  **DROP DATABASE**  statement to delete a database.

> \[!CAUTION]CAUTION:
> Exercise caution when deleting a database. Once a database is deleted, all tables and data in the database will be deleted.

```pgsql
DROP DATABASE databasename;
```

In the preceding command,  **databasename**  indicates the database name.

The  **DROP DATABASE**  command is used to delete an existing database. After this command is executed, all tables in the database are deleted, but the user permissions of the database are not automatically deleted.

To use  **DROP DATABASE**, you need the  **DROP**  permission on the database.

**DROP SCHEMA**  is a synonym of  **DROP DATABASE**.

##### Example

\# Delete the  **databaseexample**  database.

```pgsql
> DROP DATABASE databaseexample;
```

#### Backing Up a Database

Run the  **mysqldump**  command as the **root** user to back up the database.

Back up one or more tables.

```shell
mysqldump [options] databasename [tablename ...] > outfile
```

Back up one or more databases:

```shell
mysqldump [options] -databases databasename ... > outfile
```

Back up all databases:

```shell
mysqldump [options] -all-databases > outputfile
```

In the preceding information:

* *databasename*: database name.
* *tablename*: name of a data table.
* *outfile*: database backup file.
* *options*: parameter option of the  **mysqldump**  command. Multiple parameters can be separated by spaces. The common parameters of the  **mysqldump**  command are as follows:
  * **-u, --user**= *username*: specifies the username.
  * **-p, --password**\[= *password*]: specifies the password.
  * **-P, --port**= *portnumber*: specifies the port number.
  * **-h, --host**= *hostname*: specifies the hostname.
  * **-r, --result-file**= *filename*: saves the export result to a specified file, which is equivalent to  **>**.
  * **-t**: backs up data only.
  * **-d**: backs up the table structure only.

##### Example

\# Back up all the databases of the user  **root**  on the host  **192.168.202.144**  through port  **3306**  to the  **alldb.sql**  file.

```shell
mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --all-databases > alldb.sql
```

\# Back up the db1 database of the user  **root**  on the host  **192.168.202.144**  through port  **3306**  to the  **db1.sql**  file.

```shell
mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --databases db1 > db1.sql
```

\# Back up the tb1 table of the db1 database of the user  **root**  on the host  **192.168.202.144**  through port  **3306** to the  **db1tb1.sql**  file.

```shell
mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 db1 tb1 > db1tb1.sql
```

\# Back up only the table structure of the db1 database of user  **root**  on port  **3306**  of the host whose IP address is  **192.168.202.144**  to the  **db1.sql**  file.

```shell
mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -d db1 > db1.sql
```

\# Back up only the data of the db1 database of the user  **root**  on the host  **192.168.202.144**  through port  **3306** to the  **db1.sql**  file.

```shell
mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 > db1.sql
```

#### Restoring a Database

Run the  **mysql**  command as the **root** user to restore the database.

Restore one or more tables:

```shell
mysql -h hostname -P portnumber -u username -ppassword databasename < infile
```

In the preceding information:

* *hostname*: host name.
* *portnumber*: port number.
* *username*: name of a user.
* *password*: password.
* *databasename*: database name.
* *infile*:  **outfile**  parameter in the  **mysqldump** command.

##### Example

\# Restore a database.

```shell
mysql -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 < db1.sql
```

## MySQL Server

### Software Description

MySQL is a relational database management system (RDBMS) developed by the Swedish company MySQL AB, which was bought by Sun Microsystems (now Oracle). It is one of the most popular Relational Database Management Systems (RDBMSs) in the industry, especially for web applications.

A relational database stores data in different tables instead of in a large data warehouse to improve efficiency and flexibility.

The Structured Query Language (SQL) used by MySQL is the most common standard language for accessing databases. MySQL uses dual-licensing distribution and is available in two editions: Community Edition and Commercial Edition. MySQL is optimal for small or medium-sized websites because of its small size, fast speed, low cost, and especially the open source code.

### Configuring the Environment

> \[!NOTE] **NOTE:**
> The following environment configuration is for reference only. Configure the environment based on the site requirements.

#### Disabling the Firewall and Automatic Startup

> \[!NOTE] **NOTE:**
> It is recommended that firewall be disabled in the test environment to prevent network impact. Configure the firewall based on actual requirements.

1. Stop the firewall service as the **root** user.

   ```shell
   systemctl stop firewalld
   ```

2. Disable the firewall service as the **root** user.

   ```shell
   systemctl disable firewalld
   ```

   > \[!NOTE] **NOTE:**
   > The automatic startup is automatically disabled as the firewall is disabled.

#### Disabling SELinux

1. Modify the configuration file as the **root** user.

   ```shell
   sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
   ```

#### Creating a User Group and a User

> \[!NOTE] **NOTE:**
> In the server environment, independent users are assigned to each process to implement permission isolation for security purposes. The user group and user are created for the OS, not for the database.

1. Create a MySQL user or user group as the **root** user.

   ```shell
   groupadd mysql
   ```

   ```shell
   useradd -g mysql mysql
   ```

2. Set the user password as the **root** user.

   ```shell
   passwd mysql
   ```

   Enter the password twice for confirmation.

#### Creating Data Drives

> \[!NOTE] **NOTE:**
>
> * If a performance test needs to be performed, an independent drive is required for the data directory. You need to format and mount the drive. For details, see Method 1 or Method 2.
> * In a non-performance test, run the following command as the **root** user to create a data directory. Then skip this section.\
>   \# mkdir /data

##### Method 1: Using fdisk for Drive Management as the **root** user

1. Create a partition, for example,  **/dev/sdb**.

   ```shell
   fdisk /dev/sdb
   ```

2. Enter  **n**  and press  **Enter**.

3. Enter  **p**  and press  **Enter**.

4. Enter  **1**  and press  **Enter**.

5. Retain the default settings and press  **Enter**.

6. Retain the default settings and press  **Enter**.

7. Enter  **w**  and press  **Enter**.

8. Create a file system, for example,  **xfs**.

   ```shell
   mkfs.xfs /dev/sdb1
   ```

9. Mount the partition to  **/data**  for the OS.

   ```shell
   mkdir /data
   ```

   ```shell
   mount /dev/sdb1 /data
   ```

10. Run the  **vi /etc/fstab**  command and edit the  **/etc/fstab**  file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure.

    In the last line,  **/dev/nvme0n1p1**  is only an example.

    ![](./figures/creat_datadisk.png)

##### Method 2: Using LVM for Drive Management as the **root** user

> \[!NOTE] **NOTE:**
> Install the LVM2 package in the image as follows:
>
> 1. Configure the local yum repository. For details, see  [Configuring the Repo Server](./configuring_the_repo_server.md). If the repository has been configured, skip this step.
> 2. Install LVM2.\
>    **# yum install lvm2**

1. Create a PV, for example,  **sdb**.

   ```shell
   pvcreate /dev/sdb
   ```

2. Create a physical VG, for example,  **datavg**.

   ```shell
   vgcreate  datavg  /dev/sdb
   ```

3. Create an LV, for example,  **datalv**  of 600 GB.

   ```shell
   lvcreate -L 600G -n datalv datavg
   ```

4. Create a file system.

   ```shell
   mkfs.xfs /dev/datavg/datalv
   ```

5. Create a data directory and mount it.

   ```shell
   mkdir /data
   ```

   ```shell
   mount /dev/datavg/datalv /data
   ```

6. Run the  **vi /etc/fstab**  command and edit the  **/etc/fstab**  file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure.

   In the last line,  **/dev/datavg/datalv**  is only an example.

   ![](./figures/D1376B2A-D036-41C4-B852-E8368F363B5E-1.png)

#### Creating a Database Directory and Granting Permissions

1. In the created data directory  **/data**, create directories for processes and grant permissions to the MySQL group or user created as the **root** user.

   ```shell
   mkdir -p /data/mysql
   cd /data/mysql
   mkdir data tmp run log
   chown -R mysql:mysql /data
   ```

### Installing, Running, and Uninstalling MySQL

#### Installing MySQL

1. Configure the local yum repository. For details, see  [Configuring the Repo Server](./configuring_the_repo_server.md).

2. Clear the cache.

   ```shell
   dnf clean all
   ```

3. Create a cache.

   ```shell
   dnf makecache
   ```

4. Install the MySQL server as the **root** user.

   ```shell
   dnf install mysql-server
   ```

5. Check the installed RPM package.

   ```shell
   rpm -qa | grep mysql-server
   ```

#### Running MySQL

1. Modify the configuration file.
   1. Create the  **my.cnf**  file as the **root** user and change the file paths (including the software installation path  **basedir**  and data path  **datadir**) based on the actual situation.

      ```shell
      vi /etc/my.cnf
      ```

      Edit the  **my.cnf**  file as follows:

      ```shell
      [mysqld_safe]
      log-error=/data/mysql/log/mysql.log
      pid-file=/data/mysql/run/mysqld.pid
      [mysqldump]
      quick
      [mysql]
      no-auto-rehash
      [client]
      default-character-set=utf8
      [mysqld]
      basedir=/usr/local/mysql
      socket=/data/mysql/run/mysql.sock
      tmpdir=/data/mysql/tmp
      datadir=/data/mysql/data
      default_authentication_plugin=mysql_native_password
      port=3306
      user=mysql
      ```

   2. Ensure that the  **my.cnf**  file is correctly modified.

      ```shell
      cat /etc/my.cnf
      ```

      ![](./figures/en-us_image_0231563132.png)

      > \[!CAUTION]CAUTION:
      > In the configuration file,  **basedir**  specifies the software installation path. Change it based on actual situation.

   3. Change the group and user of the  **/etc/my.cnf**  file to  **mysql:mysql** as the **root** user.

      ```shell
      chown mysql:mysql /etc/my.cnf
      ```

2. Configure environment variables.
   1. Add the path of the MySQL binary files to the  **PATH**  parameter as the **root** user.

      ```shell
      echo export  PATH=$PATH:/usr/local/mysql/bin  >> /etc/profile
      ```

      > \[!CAUTION]CAUTION:
      > In the command,  **/usr/local/mysql/bin**  is the absolute path of the  **bin**  files in the MySQL software installation directory. Change it based on actual situation.

   2. Run the following command as the **root** user to make the environment variables take effect:

      ```shell
      source /etc/profile
      ```

3. Initialize the database as the **root** user.

   > \[!NOTE] **NOTE:**
   > The second line from the bottom contains the initial password, which will be used when you log in to the database.

   ```shell
   $ mysqld --defaults-file=/etc/my.cnf --initialize
   2020-03-18T03:27:13.702385Z 0 [System] [MY-013169] [Server] /usr/local/mysql/bin/mysqld (mysqld 8.0.17) initializing of server in progress as process 34014
   2020-03-18T03:27:24.112453Z 5 [Note] [MY-010454] [Server] A temporary password is generated for root@localhost: iNat=)#V2tZu
   2020-03-18T03:27:28.576003Z 0 [System] [MY-013170] [Server] /usr/local/mysql/bin/mysqld (mysqld 8.0.17) initializing of server has completed
   ```

   If the command output contains "initializing of server has completed", the database has been initialized. In the command output, "iNat=)# V2tZu" in "A temporary password is generated for root@localhost: iNat=)# V2tZu" is the initial password.

4. Start the database.

   > \[!CAUTION]CAUTION:
   > Start MySQL as user  **mysql**  if it is the first time to start the database service. If you start MySQL as user  **root**, a message will be displayed indicating that the  s**mysql.log**  file is missing. If you tart MySQL as user  **mysql**, the  **mysql.log**  file will be generated in the  **/data/mysql/log**  directory. No error will be displayed if you start the database as user  **root**  again.

   1. Modify the file permission as the **root** user.

      ```shell
      chmod 777 /usr/local/mysql/support-files/mysql.server
      ```

   2. Start MySQL as the **root** user.

      ```shell
      cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysql
      chkconfig mysql on
      ```

      Start MySQL as user  **mysql**.

      ```shell
      su - mysql
      service mysql start
      ```

5. Log in to the database.

   > \[!NOTE] **NOTE:**
   >
   > * Enter the initial password generated during database initialization ([3](#li15634560582)).
   > * If MySQL is installed by using an RPM package obtained from the official website, the  **mysqld**  file is located in the  **/usr/sbin**  directory. Ensure that the directory specified in the command is correct.

   ```shell
   /usr/local/mysql/bin/mysql -uroot -p  -S /data/mysql/run/mysql.sock
   ```

   ![](./figures/en-us_image_0231563134.png)

6. Configure the database accounts and passwords.
   1. After logging in to the database, change the password of user  **root**  for logging in to the database.

      ```shell
      mysql>alter user 'root'@'localhost' identified by "123456";
      ```

   2. Create a user  **root**  for all the other hosts in the domain.

      ```shell
      mysql>create user 'root'@'%' identified by '123456';
      ```

   3. Grant permissions to the user  **root**.

      ```shell
      mysql>grant all privileges on *.* to 'root'@'%';
      mysql>flush privileges;
      ```

      ![](./figures/en-us_image_0231563135.png)

7. Exit the database.

   Run the  **\q**  or  **exit**  command to exit the database.

   ```shell
   mysql>exit
   ```

   ![](./figures/en-us_image_0231563136.png)

#### Uninstalling MySQL

1. Stop the database process as the **root** user.

   ```shell
   $ ps -ef | grep mysql
   # kill -9 PID
   ```

2. Run the  **dnf remove mysql**  command as the **root** user to uninstall MySQL.

   ```shell
   dnf remove mysql
   ```

### Managing Database Users

#### Creating Users

Run the  **CREATE USER**  statement to create one or more users and set corresponding passwords.

```pgsql
CREATE USER 'username'@'hostname' IDENTIFIED BY 'password';
```

In the preceding information:

* *username*: name of a user.
* *host*: hostname, that is, the name of the host where the user connects to the database. As a local user, you can set the parameter to  **localhost**. If the host name is not specified during user creation, the host name is  **%**  by default, indicating a group of hosts.
* *password*: password for logging in to the server. The password can be null. If the password is null, the user can log in to the server without entering the password. This method, however, is not recommended because it provides low security.

To use the  **CREATE USER**  statement, you must have the  **INSERT**  permission on the database or the global  **CREATE USER**  permission.

After a user account is created using the  **CREATE USER**  statement, a record is added to the user table in the database. If the account to be created exists, an error will occur during statement execution.

A new user has few permissions and can perform only operations that do not require permissions. For example, a user can run the  **SHOW**  statement to query the list of all storage engines and character sets.

##### Example

\# Create a local user whose password is  **123456**  and username is  **userexample1**.

```pgsql
> CREATE USER 'userexample1'@'localhost' IDENTIFIED BY '123456';
```

\# Create a user whose password is  **123456**, username is  **userexample2**, and hostname is  **192.168.1.100**.

```pgsql
> CREATE USER 'userexample2'@'192.168.1.100' IDENTIFIED BY '123456';
```

#### Viewing Users

Run the  **SHOW GRANTS**  or  **SELECT**  statement to view one or more users.

View a specific user:

```pgsql
SHOW GRANTS [FOR 'username'@'hostname'];
```

```pgsql
SELECT USER,HOST,PASSWORD FROM mysql.user WHERE USER='username';
```

View all users:

```pgsql
SELECT USER,HOST FROM mysql.user;
```

In the preceding information:

* *username*: name of a user.
* *hostname*: host name.

##### Example

\# View the user  **userexample1**.

```pgsql
> SHOW GRANTS FOR 'userexample1'@'localhost';
```

\# View all users in the MySQL database.

```pgsql
> SELECT USER,HOST FROM mysql.user;
```

#### Modifying Users

##### Modifying a Username

Run the  **RENAME USER**  statement to change one or more existing usernames.

```pgsql
RENAME USER 'oldusername'@'hostname' TO 'newusername'@'hostname';
```

In the preceding information:

* *oldusername*: original username.
* *newusername*: new username.
* *hostname*: host name.

The  **RENAME USER**  statement is used to rename an existing account. If the original account does not exist in the system or the new account exists, an error will occur when the statement is executed.

To use the  **RENAME USER**  statement, you must have the  **UPDATE**  permission on the database or the global  **CREATE USER**  permission.

##### Example of Modifying a User

\# Change the username  **userexample1**  to  **userexample2**  and change the hostname to  **localhost**.

```pgsql
> RENAME USER 'userexample1'@'localhost' TO 'userexample2'@'localhost';
```

##### Modifying a User Password

Use the  **SET PASSWORD**  statement to modify the login password of a user.

```pgsql
SET PASSWORD FOR 'username'@'hostname' = 'newpassword';
```

In the preceding information:

* **FOR'***username***'@'***hostname***'**: specifies the username and hostname whose password is to be changed. This parameter is optional.
* *newpassword*: new password.

If the  **FOR**  clause is not added to the  **SET PASSWORD**  statement, the password of the current user is changed.

The  **FOR**  clause must be given in the format of  **'***username***'@'***hostname***'**, where  *username*  indicates the username of the account and  *hostname*  indicates the hostname of the account.

The account whose password is to be changed must exist in the system. Otherwise, an error occurs when the statement is executed.

##### Example of Changing a User Password

\# Change the password of user  **userexample**  whose hostname is  **localhost**  to  **0123456**.

```pgsql
> SET PASSWORD FOR 'userexample'@'localhost' = '0123456';
```

#### Deleting Users

Use the  **DROP USER**  statement to delete one or more user accounts and related permissions.

```pgsql
DROP USER 'username1'@'hostname1' [,'username2'@'hostname2']...;
```

> \[!CAUTION]CAUTION:
> The deletion of users does not affect the tables, indexes, or other database objects that they have created, because the database does not record the accounts that have created these objects.

The  **DROP USER**  statement can be used to delete one or more database accounts and their original permissions.

To use the  **DROP USER**  statement, you must have the  **DELETE**  permission on the database or the global  **CREATE USER**  permission.

In the  **DROP USER**  statement, if the hostname of an account is not specified, the hostname is  **%**  by default.

##### Example

\# Delete the local user  **userexample**.

```pgsql
> DROP USER 'userexample'@'localhost';
```

#### Granting Permissions to a User

Run the  **GRANT**  statement to grant permissions to a new user.

```pgsql
GRANT privileges ON databasename.tablename TO 'username'@'hostname';
```

In the preceding information:

* **ON**  clause: specifies the object and level on which the permission is granted.
* *privileges*: indicates the operation permissions of a user, such as  **SELECT**, INSERT, and  **UPDATE**. To grant all permissions to a user, use  **ALL**.
* *databasename*: database name.
* *tablename*: table name.
* **TO**  clause: sets the user password and specifies the user to whom the permission is granted.
* *username*: name of a user.
* *hostname*: host name.

To grant the user the permission to operate all databases and tables, use asterisks (\*), for example,  **\*.\***.

If you specify a password for an existing user in the  **TO**  clause, the new password will overwrite the original password.

If the permission is granted to a non-existent user, a  **CREATE USER**  statement is automatically executed to create the user, but the password must be specified for the user.

##### Example

\# Grant the  **SELECT**  and  **INSERT**  permissions to local user  **userexample**.

```pgsql
> GRANT SELECT,INSERT ON *.* TO 'userexample'@'localhost'; 
```

#### Deleting User Permissions

Run the  **REVOKE**  statement to delete the permissions of a user, but the user will not be deleted.

```pgsql
REVOKE privilege ON databasename.tablename FROM 'username'@'hostname';
```

The parameters in the  **REVOKE**  statement are the same as those in the  **GRANT**  statement.

To use the  **REVOKE**  statement, you must have the global  **CREATE USER**  or  **UPDATE**  permission for the database.

##### Example

\# Delete the  **INSERT**  permission of local user  **userexample**.

```pgsql
> REVOKE INSERT ON *.* FROM 'userexample'@'localhost';
```

### Managing Databases

#### Creating a Database

Run the  **CREATE DATABASE**  statement to create a database.

```pgsql
CREATE DATABASE databasename;
```

In the preceding command,  *databasename*  can be replaced with the database name, which is case insensitive.

##### Example

\# Create a database named  **databaseexample**.

```pgsql
> CREATE DATABASE databaseexample;
```

#### Viewing a Database

Run the  **SHOW DATABASES**  statement to view a database.

```pgsql
SHOW DATABASES;
```

##### Example

\# View all databases.

```pgsql
> SHOW DATABASES;
```

#### Selecting a Database

Generally, you need to select a target database before creating or querying a table. Use the  **USE**  statement to select a database.

```pgsql
USE databasename;
```

In the preceding command,  *databasename*  indicates the database name.

##### Example

\# Select the  **databaseexample**  database.

```pgsql
> USE databaseexample;
```

#### Deleting a Database

Run the  **DROP DATABASE**  statement to delete a database.

> \[!CAUTION]CAUTION:
> Exercise caution when deleting a database. Once a database is deleted, all tables and data in the database will be deleted.

```pgsql
DROP DATABASE databasename;
```

In the preceding command,  *databasename*  indicates the database name.

The  **DROP DATABASE**  command is used to delete an existing database. After this command is executed, all tables in the database are deleted, but the user permissions of the database are not automatically deleted.

To use  **DROP DATABASE**, you need the  **DROP**  permission on the database.

**DROP SCHEMA**  is a synonym of  **DROP DATABASE**.

##### Example

\# Delete the  **databaseexample**  database.

```pgsql
> DROP DATABASE databaseexample;
```

#### Backing Up a Database

Run the  **mysqldump**  command as the **root** user to back up the database.

Back up one or more tables:

```shell
mysqldump [options] databasename [tablename ...] > outfile
```

Back up one or more databases:

```shell
mysqldump [options] -databases databasename ... > outfile
```

Back up all databases:

```shell
mysqldump [options] -all-databases > outputfile
```

In the preceding information:

* *databasename*: database name.
* *tablename*: name of a data table.
* *outfile*: database backup file.
* *options*: parameter option of the  **mysqldump**  command. Multiple parameters can be separated by spaces. The common parameters of the  **mysqldump**  command are as follows:
  * **-u, --user**= *username*: specifies the username.
  * **-p, --password**\[= *password*]: specifies the password.
  * **-P, --port**= *portnumber*: specifies the port number.
  * **-h, --host**= *hostname*: specifies the hostname.
  * **-r, --result-file**= *filename*: saves the export result to a specified file, which is equivalent to  **>**.
  * **-t**: backs up data only.
  * **-d**: backs up the table structure only.

##### Example

\# Back up all the databases of user  **root**  on port  **3306**  of the host whose IP address is  **192.168.202.144**  to the  **alldb.sql**  file.

```shell
mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --all-databases > alldb.sql
```

\# Back up the db1 database of user  **root**  on port  **3306**  of the host whose IP address is  **192.168.202.144**  to the  **db1.sql**  file.

```shell
mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --databases db1 > db1.sql
```

\# Back up the tb1 table of the db1 database of user  **root**  on port  **3306**  of the host whose IP address is  **192.168.202.144**  to the  **db1tb1.sql**  file.

```shell
mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 db1 tb1 > db1tb1.sql
```

\# Back up only the table structure of the db1 database of user  **root**  on port  **3306**  of the host whose IP address is  **192.168.202.144**  to the  **db1.sql**  file.

```shell
mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -d db1 > db1.sql
```

\# Back up only the table structure of the db1 database of user  **root**  on port  **3306**  of the host whose IP address is  **192.168.202.144**  to the  **db1.sql**  file.

```shell
mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 > db1.sql
```

#### Restoring a Database

Run the  **mysql**  command as the **root** user to restore the database.

Restore one or more tables:

```shell
mysql -h hostname -P portnumber -u username -ppassword databasename < infile
```

In the preceding information:

* *hostname*: host name.
* *portnumber*: port number.
* *username*: name of a user.
* *password*: password.
* *databasename*: database name.
* *infile*:  **outfile**  parameter in the  **mysqldump** command.

##### Example

\# Restore a database.

```shell
mysql -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 < db1.sql
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/shared_memory_channels.md
---
# Shared Memory Channels

## Function Description

System containers enable the communication between container and host processes through shared memory. You can set the  **--host-channel**  parameter when creating a container to allow the host to share the same tmpfs with the container so that they can communicate with each other.

## Parameter Description

## Constraints

* The lifecycle of tmpfs mounted on the host starts from the container startup to the container deletion. After a container is deleted and its occupied space is released, the space is removed.
* When a container is deleted, the path to which tmpfs is mounted on the host is deleted. Therefore, an existing directory on the host cannot be used as the mount path.
* To ensure that processes running by non-root users on the host can communicate with containers, the permission for tmpfs mounted on the host is 1777.

## Example

Specify the  **--host-channel**  parameter when creating a container.

```sh
[root@localhost ~]# isula run --rm -it --host-channel /testdir:/testdir:rw:32M --system-container --external-rootfs /root/myrootfs none init 
root@3b947668eb54:/# dd if=/dev/zero of=/testdir/test.file bs=1024 count=64K 
dd: error writing '/testdir/test.file': No space left on device 
32769+0 records in 
32768+0 records out 
33554432 bytes (34 MB, 32 MiB) copied, 0.0766899 s, 438 MB/s
```

> \[!NOTE] **NOTE:**
>
> * If  **--host-channel**  is used for size limit, the file size is constrained by the memory limit in the container. (The OOM error may occur when the memory usage reaches the upper limit.)
> * If a user creates a shared file on the host, the file size is not constrained by the memory limit in the container.
> * If you need to create a shared file in the container and the service is memory-intensive, you can add the value of  **--host-channel**  to the original value of the container memory limit, eliminating the impact.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/skylark.md
---
# Skylark

## Skylark Introduction

### Scenario

With the rapid growth of the cloud computing market, cloud vendors are increasing their investment in cloud infrastructure. However, the industry still faces the problem of low resource utilization. Improving resource utilization has become an important technical subject. This document describes openEuler Skylark, as well as how to install and use it.

### Overview

Hybrid deployment of services of different priorities is a typical and effective method to improve resource utilization. Services can be classified into high-priority and low-priority services based on latency sensitivity. When high-priority services compete with low-priority services for resources, resources are preferentially provided for high-priority services. Therefore, the core technology of service hybrid deployment is resource isolation control, which involves kernel-mode basic resource isolation and user-mode QoS control.

This document describes the user-mode QoS control technology provided by Skylark of openEuler 22.09. In Skylark, the priority granularity is VMs. That is, a priority attribute is added to each VM. Resources are isolated and controlled based on VMs. Skylark is a QoS-aware resource scheduler in hybrid deployment scenarios. It improves physical machine resource utilization while ensuring the QoS of high-priority VMs.

For details about how to better use the priority feature of Skylark in actual application scenarios, see [Best Practices](#best-practices).

## Architecture and Features

### Overall Architecture

The core class of Skylark is `QoSManager`. Class members include data collection class instances, QoS analysis class instances, QoS control class instances, and task scheduling class instances.

* `DataCollector`: data collection class. It has the `HostInfo` and `GuestInfo` members, which collect host information and VM information, respectively.
* `PowerAnalyzer`: power consumption analysis class, which analyzes power consumption interference and low-priority VMs to be restricted.
* `CpuController`: CPU bandwidth control class, which limits the CPU bandwidth of low-priority VMs.
* `CacheMBWController`: last-level cache (LLC) and memory bandwidth control class, which limits the LLC and memory bandwidth of low-priority VMs.
* `BackgroundScheduler`: task scheduling class, which periodically drives the preceding modules to continuously manage QoS.

After checking the host environment, Skylark creates a daemon process. The daemon has a main scheduling thread and one or more job threads.

* The main scheduling thread is unique. It connects to libvirt, creates and initializes the `QosManager` class instance, and then starts to drive the Job threads.
* Each Job thread periodically executes a QoS management task.

### Power Consumption Interference Control

Compared with non-hybrid deployment, host resource utilization is higher in hybrid deployment scenarios. High utilization means high power consumption. When the power consumption exceeds the thermal design power (TDP) of the server, CPU frequency reduction is triggered. When the power consumption exceeds the preset TDP (that is, a TDP hotspot occurs), Skylark limits the CPU bandwidth of low-priority VMs to reduce the power consumption of the entire system and ensure the QoS of high-priority VMs.

During initialization, Skylark sets the power consumption interference control attributes based on the related configuration values in [Skylark Configuration](#skylark-configuration). In each control period, host information and control attributes are comprehensively analyzed to determine whether TDP hotspots occur. If a hotspot occurs, Skylark analyzes the low-priority VMs whose CPU bandwidth needs to be limited based on the VM information.

### LLC/MB Interference Control

Skylark can limit the LLC and memory bandwidth of low-priority VMs. Currently, only static allocation is supported. Skylark uses the **/sys/fs/resctrl** interface provided by the OS to implement the limitation.

1. Skylark creates the **low\_prio\_machine** folder in the **/sys/fs/resctrl** directory and writes the PID of the low-priority VM to the **/sys/fs/resctrl/low\_prio\_machine/tasks** file.
2. Skylark allocates LLC ways and memory bandwidth for low-priority VMs based on the LLC/MB configuration items in [Skylark Configuration](#skylark-configuration). The configuration items are written into the **/sys/fs/resctrl/low\_prio\_machine/schemata** file.

### CPU Interference Control

In hybrid deployment scenarios, low-priority VMs generate CPU time slice interference and hardware hyper-threading (SMT) interference on high-priority VMs.

* When threads of high- and low-priority VMs are running on the same minimum CPU topology unit (core or SMT execution unit), they compete for CPU time slices.
* When threads of high- and low-priority VMs are running on different SMT execution units of the same CPU core at the same time, they compete for resources in the core shared by the SMT execution units.

CPU interference control includes CPU time slice interference control and SMT interference control, which are implemented based on the **QOS\_SCHED** and **SMT\_EXPELLER** features provided by the kernel, respectively.

* The **QOS\_SCHED** feature enables high-priority VM threads on a single CPU core or SMT execution unit to suppress low-priority VM threads, eliminating CPU time slice interference.
* The **SMT\_EXPELLER** feature enables high-priority VM threads to suppress low-priority VM threads on different SMT execution units of the same CPU core, eliminating SMT interference.

During initialization, Skylark sets the **cpu.qos\_level** field of the slice level corresponding to the low-priority VM under the cgroup CPU subcontroller to -1 to enable the preceding kernel features. By doing this, the kernel controls CPU-related interference without the intervention of Skylark.

## Skylark Installation

### Hardware Requirements

Processor architecture: AArch64 or x86\_64

* For Intel processors, the RDT function must be supported.
* For the AArch64 architecture, only Kunpeng 920 processor is supported, and the BIOS must be upgraded to 1.79 or later to support the MPAM function.

### Software Requirements

* python3, python3-APScheduler, and python3-libvirt
* systemd 249-32 or later
* libvirt 1.0.5 or later
* openEuler kernel 5.10.0 or later.

### Installation Procedure

You are advised to install the Skylark component using Yum for automatic processing of the software dependencies:

```shell
# yum install -y skylark
```

Check whether the Skylark is successfully installed. If the installation is successful, the skylarkd background service status is displayed:

```shell
# systemctl status skylarkd
```

(Optional) Enable the Skylark service to automatically start upon system startup:

```shell
# systemctl enable skylarkd
```

## Skylark Configuration

After the Skylark component is installed, you can modify the configuration file if the default configuration does not meet your requirements. The Skylark configuration file is stored in **/etc/sysconfig/skylarkd**. The following describes the configuration items in the configuration file.

### Logs

* The **LOG\_LEVEL** parameter is a character string used to set the minimum log level. The supported log levels are **critical > error > warning > info > debug**. Logs whose levels are lower than **LOG\_LEVEL** are not recorded in the log file **/var/log/skylark.log**. Skylark backs up logs every seven days for a maximum of four times. (When the number of backup times reaches the limit, the oldest logs are deleted.) The backup log is saved as **/var/log/skylark.log. %Y- %m- %d**.

### Power Consumption Interference Control

* **POWER\_QOS\_MANAGEMENT** is a boolean value used to control whether to enable power consumption QoS management. Only x86 processors support this function. This function is useful if the CPU usage of VMs on the host can be properly limited.

* **TDP\_THRESHOLD** is a floating point number used to control the maximum power consumption of a VM. When the power consumption of the host exceeds **TDP \* TDP\_THRESHOLD**, a TDP hotspot occurs, and a power consumption control operation is triggered. The value ranges from 0.8 to 1, with the default value being 0.98.

* **FREQ\_THRESHOLD** is a floating point number used to control the minimum CPU frequency when a TDP hotspot occurs on the host. The value ranges from 0.8 to 1, with the default value being 0.98.
  1. When the frequency of some CPUs is lower than **max\_freq \* FREQ\_THRESHOLD**, Skylark limits the CPU bandwidth of low-priority VMs running on these CPUs.
  2. If such a CPU does not exist, Skylark limits the CPU bandwidth of some low-priority VMs based on the CPU usage of low-priority VMs.

* **QUOTA\_THRESHOLD** is a floating point number used to control the CPU bandwidth that a restricted low-priority VM can obtain (CPU bandwidth before restriction x **QUOTA\_THRESHOLD**). The value ranges from 0.8 to 1, with the default value being 0.9.

* **ABNORMAL\_THRESHOLD** is an integer used to control the number of low-priority VM restriction periods. The value ranges from 1 to 5, with the default value being 3.
  1. In each power consumption control period, if a low-priority VM is restricted, its number of remaining restriction periods is updated to **ABNORMAL\_THRESHOLD**. Otherwise, its number of remaining restriction periods decreases by 1.
  2. When the number of remaining restriction periods of the VM is 0, the CPU bandwidth of the VM is restored to the value before the restriction.

### LLC/MB Interference Control

Skylark's interference control on LLC/MB depends on the RDT/MPAM function provided by hardware. For Intel x86\_64 processors, **rdt=cmt,mbmtotal,mbmlocal,l3cat,mba** needs to be added to kernel command line parameters. For Kunpeng920 processors, **mpam=acpi** needs to be added to kernel command line parameters.

* **MIN\_LLC\_WAYS\_LOW\_VMS** is an integer used to control the number of LLC ways that can be accessed by low-priority VMs. The value ranges from 1 to 3, with the default value being 2. During initialization, Skylark limits the numfer of accessible LLC ways for low-priority VMs to this value.

* **MIN\_MBW\_LOW\_VMS** is a floating point number used to control the memory bandwidth ratio available to low-priority VMs. The value ranges from 0.1 to 0.2, with the default value being 0.1. Skylark limits the memory bandwidth of low-priority VMs based on this value during initialization.

## Skylark Usage

### Starting the Service

Start Skylark for the first time:

```shell
# systemctl start skylarkd
```

Restart Skylark (a service restart is required after modifying the configuration file):

```shell
# systemctl restart skylarkd
```

### Creating VMs

Skylark uses the **partition** tag in the XML configuration file of a VM to identify the VM priority.

To create a low-priority VM, configure the XML file as follows:

```xml
<domain>
  ...
  <resource>
    <partition>/low_prio_machine</partition>
  </resource>
  ...
</domain>
```

To create a high-priority VM, configure the XML file as follows:

```xml
<domain>
  ...
  <resource>
    <partition>/high_prio_machine</partition>
  </resource>
  ...
</domain>
```

The subsequent VM creation process is the same as the normal process.

### Running VMs

Skylark detects VM creation events, manages VMs of different priorities, and performs automatic QoS management based on CPU, power consumption, and LLC/MB resources.

## Best Practices

### VM Service Recommendation

* High-priority VMs are suitable for latency-sensitive services, such as web services, high-performance databases, real-time rendering, and AI inference.
* Low-priority VMs are suitable for non-latency-sensitive services, such as video encoding, big data processing, offline rendering, and AI training.

### CPU Binding Configuration

To ensure optimal performance of high-priority VMs, you are advised to bind each vCPU of high-priority VMs to a physical CPU. To enable low-priority VMs to make full use of idle physical resources, you are advised to bind vCPUs of low-priority VMs to CPUs that are bound to high-priority VMs.

To ensure that low-priority VMs are scheduled when high-priority VMs occupy CPU resources for a long time, you are advised to reserve a small number of for low-priority VMs.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/skylark.md
---
# Skylark

## Skylark概述

### 问题背景

随着云计算市场规模的快速增长，各云厂商基础设施投入也不断增加。资源利用率低是行业普遍存在的问题，在上述背景下，提升资源利用率已经成为了一个重要的技术课题。本文档介绍 openEuler Skylark 组件，并给出安装方法及使用指导。

### 总体介绍

将业务区分优先级混合部署（下文简称混部）是典型有效的资源利用率提升手段。业务可根据时延敏感性分为高优先级业务和低优先级业务。当高优先级业务和低优先级业务发生资源竞争时，需优先保障高优先级业务的资源供给。因此，业务混部的核心技术是资源隔离控制，主要涉及内核态基础资源隔离技术及用户态 QoS 控制技术。

本文描述的对象为用户态 QoS 控制技术，由 openEuler Skylark 组件承载，首发于 openEuler 22.09 版本。在 Skylark 视角下，优先级粒度为虚拟机级别，即给虚拟机新增高低优先级属性，以虚拟机为粒度进行资源的隔离和控制。Skylark 是一种混部场景下的 QoS 感知的资源调度器，在保障高优先级虚拟机 QoS 前提下提升物理机资源利用率。

在实际应用场景中如何更好地利用 Skylark 的高低优先级特性，请参考[最佳实践](#最佳实践)章节。

## 架构及特性

### 总体实现框架

Skylark 核心类为`QoSManager`，类成员包括数据收集类实例、QoS 分析类实例、QoS 控制类实例、以及任务调度类实例：

* `DataCollector`：数据收集类，有`HostInfo`和`GuestInfo`两个成员，分别用于收集主机信息和虚拟机信息。
* `PowerAnalyzer`：功耗分析类，用于分析功耗干扰以及需要限制的低优先级虚拟机。
* `CpuController`：CPU 带宽控制类，用于限制低优先级虚拟机的 CPU 带宽。
* `CacheMBWController`：LLC 及内存带宽控制类，用于限制低优先级虚拟机的 LLC 和内存带宽。
* `BackgroundScheduler`：任务调度类，用于周期性驱动以上模块，持续进行 QoS 管理。

Skylark 检查主机环境后，创建守护进程。守护进程有两种线程：主调度线程和 Job 线程：

* 主调度线程是唯一的，首先连接 Libvirt，然后创建并初始化`QosManager`类实例，最后开始驱动 Job 线程。
* Job 线程可能不止一个，每个 Job 线程负责周期性执行某个 QoS 管理任务。

### 功耗干扰控制

相比非混部情况，混部后主机利用率更高，高利用率意味着高功耗，服务器功耗在超过 TDP 时会触发 CPU 降频。Skylark 支持当功耗超过预设的 TDP 阈值（即出现 TDP 热点）时，通过对低优先级虚拟机的 CPU 带宽进行限制，以此达到降低整机功耗的同时保障高优先级虚拟机 QoS。

Skylark 初始化时，根据[配置Skylark](#配置skylark)中相关配置值，设置功耗干扰控制属性。在每个控制周期，综合分析主机信息和控制属性，判断是否出现 TDP 热点。如果出现热点，进一步根据虚拟机信息分析出需要对哪些低优先级虚拟机进行 CPU 带宽的限制。

### LLC/MB干扰控制

Skylark 支持对低优先级虚拟机的 LLC 和内存带宽进行限制，当前仅支持静态分配。Skylark 通过操作系统提供的`/sys/fs/resctrl`接口来限制低优先级虚拟机的 LLC 和内存带宽。

1. Skylark 在`/sys/fs/resctrl`目录下建立`low_prio_machine`文件夹，并将低优先级虚拟机的 pid 写入`/sys/fs/resctrl/low_prio_machine/tasks`文件中。
2. Skylark 根据[配置Skylark](#配置skylark)章节中 LLC/MB 相关配置项对低优先级虚拟机的 LLC ways 和内存带宽进行分配，配置项写入`/sys/fs/resctrl/low_prio_machine/schemata`文件中。

### CPU干扰控制

混部场景下，低优先级虚拟机会对高优先级虚拟机产生 CPU 时间片干扰和 SMT（硬件超线程）干扰。

* 当高低优先级虚拟机相关线程在同一个最小 CPU 拓扑单元（core 或 SMT）上同时处于可运行状态时，会竞争 CPU 时间片。
* 当高低优先级虚拟机相关线程在同一个 CPU core 的不同 SMT 上同时处于可运行状态时，会竞争 SMT 共享的 core 内资源。

CPU 干扰控制分为 CPU 时间片干扰控制及 SMT 干扰控制，分别基于内核提供的 `QOS_SCHED` 及 `SMT_EXPELLER` 特性实现。

* `QOS_SCHED` 特性实现了单个 CPU core 或 SMT 上高优先级虚拟机对低优先级虚拟机的绝对压制，解决了 CPU 时间片干扰问题。
* `SMT_EXPELLER` 特性实现了同一个 CPU core 的不同 SMT 上高优先级虚拟机对低优先级虚拟机的绝对压制，解决了 SMT 干扰问题。

Skylark 初始化时，会把 Cgroup CPU 子控制器下低优先级虚拟机对应 slice 层级的`cpu.qos_level`字段设置为 -1，以使能上述内核特性，后续就由内核实现对 CPU 相关干扰的控制，Skylark 无需介入。

## 安装Skylark

### 硬件要求

处理器架构：仅支持 AArch64 和 Intel x86\_64 处理器架构。

* Intel 处理器需支持 RDT 功能。
* AArch64 当前仅支持 Kunpeng920，且需将 BIOS 升级到 1.79 及以上以支持 MPAM 功能。

### 软件要求

* 依赖 python3、python3-APScheduler、python3-libvirt 等 python 组件。
* 依赖 systemd 组件，版本 >= 249-32。
* 依赖 libvirt 组件，版本 >= 1.0.5。
* 依赖 openEuler 内核，版本 >= 5.10.0。

### 安装方法

推荐使用 yum 安装 Skylark 组件，因为 yum 会自动处理上述软件依赖：

```shell
# yum install -y skylark
```

检查 Skylark 是否安装成功，若安装成功则会显示 skylarkd 后台服务状态：

```shell
# systemctl status skylarkd
```

设置 Skylark 服务开机自启动（可选）：

```shell
# systemctl enable skylarkd
```

## 配置Skylark

安装好 Skylark 组件后，若默认配置不满足需求，可修改配置文件。Skylark 的配置文件路径为`/etc/sysconfig/skylarkd`，下面对该配置文件包含的配置项作详细说明。

### 日志

* `LOG_LEVEL`用于设置最小日志级别，类型为字符串。所有可设置的日志级别及其关系为`critical > error > warning > info > debug`。级别小于`LOG_LEVEL`的日志将不会输出到日志文件。日志文件路径为`/var/log/skylark.log`。Skylark 会每 7 天备份一次日志，最多备份 4 次（当次数超限时，会删除最旧的日志）。备份的日志路径为`/var/log/skylark.log.%Y-%m-%d`。

### 功耗干扰控制

* `POWER_QOS_MANAGEMENT`用于控制是否打开功耗 QoS 管理功能，类型为布尔。当前仅 x86 支持该功能。如果主机上虚拟机的 CPU 利用率能被很好地限制，该功能可选。

* `TDP_THRESHOLD`用于控制虚拟机可达到的最大功耗。当主机功耗超过`TDP * TDP_THRESHOLD`时，将判断为出现 TDP 热点，触发功耗控制操作。类型为 float，可接受的输入范围为 0.8-1，默认值为 0.98。

* `FREQ_THRESHOLD`用于控制当主机出现 TDP 热点时，CPU 运行的最低频率。类型为 float，可接受的输入范围为 0.9-1，默认值为 0.98。
  1. 当存在某些 CPU 的频率低于`max_freq * FREQ_THRESHOLD`时，Skylark 会限制在这些 CPU 上运行的低优先级虚拟机的 CPU 带宽。
  2. 当找不到这样的 CPU，则 Skylark 也会根据低优先级虚拟机的 CPU 利用率情况，选择性限制某些低优先级虚拟机的 CPU 带宽。

* `QUOTA_THRESHOLD`用于控制低优先级虚拟机被限制后所能获得的 CPU 带宽（限制前的 CPU 带宽 \* `QUOTA_THRESHOLD`）。类型为 float，可接受的输入范围为 0.8-1，默认值为 0.9。

* `ABNORMAL_THRESHOLD`用于控制低优先级虚拟机被限制的周期。类型为 int，可接受的输入范围为 1-5，默认值为 3。
  1. 在每个功耗控制周期内，如果某个低优先级虚拟机被限制，其剩余被限制周期刷新为`ABNORMAL_THRESHOLD`，否则其剩余被限制周期减 1。
  2. 当虚拟机的剩余被限制周期等于 0 时，其 CPU 带宽恢复为被限制前的值。

### LLC/MB干扰控制

Skylark 对 LLC/MB 的干扰控制依赖于硬件使能 RDT/MPAM 功能，Intel x86\_64 架构处理器需在内核 cmdline 配置`rdt=cmt,mbmtotal,mbmlocal,l3cat,mba`，Kunpeng920 处理器需在内核 cmdline 配置`mpam=acpi`。

* `MIN_LLC_WAYS_LOW_VMS`用于控制低优先级虚拟机可访问的 LLC ways。类型为 int，可接受的输入范围为 1-3，默认值为 2。Skylark 会在初始化时，限制低优先级虚拟机的 LLC ways 为该值。

* `MIN_MBW_LOW_VMS`用于控制低优先级虚拟机可访问的内存带宽比例。类型为 float，可接受的输入范围为 0.1～0.2，默认值为 0.1。Skylark 会在初始化时，限制低优先级虚拟机的内存带宽为该值。

## 使用Skylark

### 启动服务

初次启动：

```shell
# systemctl start skylarkd
```

重新启动（修改配置文件后需重启）：

```shell
# systemctl restart skylarkd
```

### 创建虚拟机

Skylark 借助虚拟机 XML 配置文件的`partition`标签标识虚拟机优先级属性。

创建低优先级虚拟机，其 XML 需做如下配置：

```xml
<domain>
  ...
  <resource>
    <partition>/low_prio_machine</partition>
  </resource>
  ...
</domain>
```

创建高优先级虚拟机，其 XML 需做如下配置：

```xml
<domain>
  ...
  <resource>
    <partition>/high_prio_machine</partition>
  </resource>
  ...
</domain>
```

后续创建虚拟机流程和一般流程无异。

### 虚拟机运行

Skylark 能感知到虚拟机创建事件，纳管所有高、低优先级虚拟机，并围绕 CPU、功耗、LLC/MB 等资源做自动化 QoS 管理。

## 最佳实践

### 虚拟机业务推荐

* 高优先级虚拟机业务推荐：时延敏感类业务，如 web 服务、高性能数据库、实时渲染、机器学习推理等。
* 低优先级虚拟机业务推荐：非时延敏感类业务，如视频编码、大数据处理、离线渲染、机器学习训练等。

### 虚拟机绑核配置

为了让高优先级虚拟机达到最佳性能，推荐高优先级虚拟机 vCPU 与物理 CPU 一对一绑核。为了让低优先级虚拟机充分利用空闲物理资源，推荐低优先级虚拟机 vCPU 范围绑核，且绑核范围覆盖高优先级虚拟机绑核范围。

同时为了防止出现因高优先级虚拟机长时间占满 CPU 导致低优先级虚拟机无法被调度的情况，需要预留少量低优先级虚拟机专用的 CPU，该部分 CPU 不可让高优先级虚拟机绑定，且要求让低优先级虚拟机绑定。

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/source_code.md
---
# Source Code

openEuler contains two code repositories:

* Code repository:  <https://atomgit.com/openeuler>
* Software package repository:  <https://atomgit.com/src-openeuler>

The openEuler release packages also provide the source ISO files. For details, see  [Installing the OS](./os_installation.md).

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/specifying_rootfs_to_create_a_container.md
---
# Specifying Rootfs to Create a Container

## Function Description

Different from a common container that needs to be started by specifying a container image, a system container is started by specifying a local root file system (rootfs) using the **--external-rootfs** parameter. The rootfs contains the operating system environment on which the container depends during running.

## Parameter Description

## Constraints

* The rootfs directory specified using the **--external-rootfs** parameter must be an absolute path.
* The rootfs directory specified using the **--external-rootfs** parameter must be a complete OS environment including **systemd** package. Otherwise, the container fails to be started.
* When a container is deleted, the rootfs directory specified using **--external-rootfs** is not deleted.
* Containers based on an ARM rootfs cannot run in the x86 environment. Containers based on an x86 rootfs cannot run in the ARM environment.
* You are advised not to start multiple container instances in the same rootfs. That is, one rootfs is used by only one container instance that is in the lifecycle.

## Example

Assuming the local rootfs path is **/root/myrootfs**, run the following command to start a system container:

```sh
# isula run  -tid --system-container --external-rootfs /root/myrootfs none init
```

> \[!NOTE] **NOTE:**\
> The rootfs is a user-defined file system. Prepare it by yourself. For example, a rootfs is generated after the TAR package of a container image is decompressed.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/statistics.md
---
# Statistics

## events

Syntax:  **docker events \[***options***]**

Function: Obtains real-time events from the docker daemon.

Parameter description:

**--since=""**: Displays events generated after the specified timestamp.

**--until=""**: Displays events generated before the specified timestamp.

Example:

After the  **docker events**  command is executed, a container is created and started by running the  **docker run**  command. create and start events are output.

```sh
$ sudo docker events
2019-08-28T16:23:09.338838795+08:00 container create 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (image=busybox:latest, name=eager_wu)
2019-08-28T16:23:09.339909205+08:00 container attach 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (image=busybox:latest, name=eager_wu)
2019-08-28T16:23:09.397717518+08:00 network connect e2e20f52662f1ee2b01545da3b02e5ec7ff9c85adf688dce89a9eb73661dedaa (container=53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e, name=bridge, type=bridge)
2019-08-28T16:23:09.922224724+08:00 container start 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (image=busybox:latest, name=eager_wu)
2019-08-28T16:23:09.924121158+08:00 container resize 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (height=48, image=busybox:latest, name=eager_wu, width=210)
```

## info

Syntax:  **docker info**

Function: Displays the Docker system information, including the number of containers, number of images, image storage driver, container execution driver, kernel version, and host OS version.

Parameter description: none.

Example:

```sh
$ sudo docker info
Containers: 4
 Running: 3
 Paused: 0
 Stopped: 1
Images: 45
Server Version: 18.09.0
Storage Driver: overlay2
 Pool Name: docker-thinpool
 Pool Blocksize: 524.3kB
 Base Device Size: 10.74GB
 Backing Filesystem: ext4
 Udev Sync Supported: true
 Data Space Used: 11GB
 Data Space Total: 51GB
 Data Space Available: 39.99GB
 Metadata Space Used: 5.083MB
 Metadata Space Total: 532.7MB
 Metadata Space Available: 527.6MB
 Thin Pool Minimum Free Space: 5.1GB
 Deferred Removal Enabled: true
 Deferred Deletion Enabled: true
 Deferred Deleted Device Count: 0
......
```

## version

Syntax:  **docker version**

Function: Displays the Docker version information, including the client version, server version, Go version, and OS and Arch information.

Parameter description: none.

Example:

```sh
$ sudo docker version
Client:
 Version:           18.09.0
 EulerVersion:      18.09.0.325
 API version:       1.39
 Go version:        go1.17.3
 Git commit:        ce4ae23
 Built:             Mon Jun  26 00:00:00 2023
 OS/Arch:           linux/arm64
 Experimental:      false

Server:
 Engine:
  Version:          18.09.0
  EulerVersion:     18.09.0.325
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.17.3
  Git commit:       ce4ae23
  Built:            Mon Jun  26 00:00:00 2023
  OS/Arch:          linux/arm64
  Experimental:     false
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/stratovirt_vfio_instructions.md
---
# StratoVirt VFIO Instructions

With device passthrough, a virtualization platform can enable VMs to directly use hardware devices, improving VM performance. This chapter describes the device passthrough feature supported by StratoVirt.

## Prerequisites

To use device passthrough, a host must meet the following requirements:

1. Enable the Input/Output Memory Management Unit (IOMMU).

   The IOMMU enables PCI/PCIe resources to be directly allocated to VMs.

   Run the following command on the host to check whether the IOMMU is enabled:

   ```shell
   # dmesg | grep iommu
   ```

   If it is enabled, the following information is displayed:

   ```shell
   iommu: Default domain type: Translated
   hibmc-drm 0000:0a:00.0: Adding to iommu group 0
   ehci-pci 0000:7a:01.0: Adding to iommu group 1
   ehci-pci 0000:ba:01.0: Adding to iommu group 2
   ohci-pci 0000:7a:00.0: Adding to iommu group 3
   ohci-pci 0000:ba:00.0: Adding to iommu group 4
   xhci_hcd 0000:7a:02.0: Adding to iommu group 5
   ......
   ```

   If it is not enabled, no command output is displayed or only the following information is displayed:

   ```shell
   iommu: Default domain type: Translated
   ```

   Enable IOMMU:

   1.Add boot parameters for Linux kernel: `intel_iommu=on iommu=pt`;

   ```shell
   vim /boot/grub2/grub.cfg
   linux /vmlinuz-5.15.0+ root=/dev/mapper/openeuler-root ro resume=/dev/mapper/openeuler-swap rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap crashkernel=512M intel_iommu=on iommu=pt
   ```

   2.Reboot Host OS;

2. Load the vfio-pci kernel module.

   ```shell
   # modprobe vfio-pci
   # lsmod | grep vfio_pci
   ```

   If the vfio-pci module is successfully loaded, the following information is displayed:

   ```shell
   vfio_pci              327680  0
   vfio_virqfd           327680  1 vfio_pci
   vfio                  327680  2 vfio_iommu_type1,vfio_pci
   ```

3. Unbind the PCI device from the host and bind it to the vfio-pci driver again.

   If Hi1822 NICs are directly connected through the Virtual Function I/O (VFIO), check the information about the PCI devices that correspond to the NICs first.

   ```shell
   # lspci -v | grep "Eth" | grep 1822
   03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
   04:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
   05:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
   06:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
   ```

   Select the PCI device whose bus ID is 03, slot ID is 00, and function ID is 0, that is, **03:00.0**. Unbind it from the host.

   ```shell
   # echo 0000:03:00.0 > /sys/bus/pci/devices/0000:03:00.0/driver/unbind
   ```

   Finally bind the PCI device to the vfio-pci driver.

   ```shell
   lspci -ns 0000:03:00.0 |awk -F':| ' '{print 5" "6}' > /sys/bus/pci/drivers/vfio-pci/new_id
   ```

   After the NIC is bound to the vfio-pci driver, the NIC information cannot be queried on the host. Only the PCI device information can be queried.

## VFIO Device Passthrough

### Introduction

The VFIO is a user-mode device driver solution provided by the kernel. The VFIO driver can securely present capabilities such as device I/O, interrupt, and DMA to user space. After StratoVirt uses the VFIO device passthrough solution, the I/O performance of VMs is greatly improved.

### Using VFIO Passthrough

StratoVirt interconnects with libvirt to enable you to manage and configure VMs by modifying corresponding XML files. The following describes how to enable VFIO passthrough by modifying the XML file of a VM.

**Step 1** Modify the XML file.

(1) Run the following command on the host to query the CPU architecture information:

```shell
# uname -m
```

(2) For the AArch64 and x86\_64 architectures, [download](https://atomgit.com/openeuler/stratovirt/tree/master/docs) the StratoVirt XML file **stratovirt\_aarch64.xml** or **stratovirtvirt\_x86.xml** and save it to any directory, for example, **/home**.

```shell
# cp stratovirt/docs/stratovirt_$arch.xml /home
```

(3) Modify the VFIO configuration in the XML file based on the site requirements. **bus**, **slot**, and **function** specify the PCI device bound to the vfio-pci driver. The related configurations are as follows:

```shell
<!-- vfio -->
<hostdev mode='subsystem' type='pci' managed='yes'>
<driver name='vfio'/>
<source>
    <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</source>
</hostdev>
```

In the preceding example, the device type is PCI, and **managed='yes'** indicates that libvirt unbinds the PCI device from the host and rebinds it to the vfio-pci driver. In the**source** item, the **domain**, **bus**, **slot**, and **function** of the VFIO passthrough device are configured.

**Step 2** Create and log in to a VM using the libvirt command line.

```shell
# virsh create stratovirt_$arch.xml
# virsh list --all
Id  Name   State
--------------------
1  StratoVirt  running
# virsh console 1
```

**Step 3** View and use the VFIO passthrough NIC on the VM.

(1) Check the NIC information before configuration.

```shell
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 inet 127.0.0.1/8 scope host lo
     valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
 link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff
```

(2) Dynamically configure the IP address of the NIC.

```shell
# dhclient
```

(3) Check whether the IP address is configured successfully.

```shell
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 inet 127.0.0.1/8 scope host lo
  valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
 link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff
 inet 192.168.1.3/16 brd 192.168.255.255 scope global dynamic enp1s0
  valid_lft 86453sec preferred_lft 86453sec
```

The preceding command output indicates that the IP address 192.168.1.3 is successfully assigned and the VM can directly use the configured NIC.

Note: If the passthrough NIC is not connected to a physical network, network information cannot be obtained.

### Unbinding the VFIO Driver

To unbind a passthrough NIC from a VM, log in to the host and run the following command to bind the NIC to the host again.**hinic** indicates the NIC driver type.

```shell
# echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/unbind
# echo 0000:03:00.0 > /sys/bus/pci/drivers/hinic/bind
```

Note: Before binding a VFIO driver, you can run the **ethtool -i enp0** command on the host to obtain the NIC driver type.**enp0** indicates the name of the corresponding NIC.

## SR-IOV Passthrough

### Introduction

When VFIO passthrough is enabled, VMs can directly access hardware, but each device can be exclusively used by only one VM. The SR-IOV passthrough technology can virtualize a physical function (PF) into multiple virtual functions (VFs) and directly pass the VFs to different VMs. This technology increases the number of available devices.

### Procedure

**Step 1** Create multiple VFs.

The **sriov\_numvfs** file is used to describe the count of VFs provided by SR-IOV and is stored in **/sys/bus/pci/devices/domain:bus:slot.function/**. For example, for the device whose bus ID is 03, slot ID is 00, and function ID is 0 in the preceding example, you can run the following command to create four VFs:

```shell
# echo 4 > /sys/bus/pci/devices/0000\:03\:00.0/sriov_numvfs
```

**Step 2** Verify that the VFs are successfully created.

```shell
# lspci -v | grep "Eth" | grep 1822
```

If the following information is displayed, four VFs 03:00.1, 03:00.2, 03:00.3, and 03:00.4 are successfully created:

```shell
03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
03:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
03:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
03:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
03:00.4 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
```

**Step 3** All the created VFs can be passed to VMs. The method for using an SR-IOV device is the same as that for using a common PCI device.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/stratovirt_vfio_instructions.md
---
# StratoVirt-VFIO使用

## 管理设备直通

虚拟化平台使用设备直通，可以使虚拟机直接使用相关硬件设备，提升虚拟机性能。本章介绍 StratoVirt 支持的设备直通特性。

### 前提条件

使用设备直通，主机需要满足如下条件：

1. 开启 IOMMU 功能

   IOMMU 全称是 Input/Output Memory Management Unit，该技术可以让 PCI/PCIe 设备的资源直接分配给虚拟机。

   在主机上执行如下命令，查看 IOMMU 是否已经开启。

   ```shell
   # dmesg | grep iommu
   ```

   若已开启，回显如下：

   ```shell
   iommu: Default domain type: Translated
   hibmc-drm 0000:0a:00.0: Adding to iommu group 0
   ehci-pci 0000:7a:01.0: Adding to iommu group 1
   ehci-pci 0000:ba:01.0: Adding to iommu group 2
   ohci-pci 0000:7a:00.0: Adding to iommu group 3
   ohci-pci 0000:ba:00.0: Adding to iommu group 4
   xhci_hcd 0000:7a:02.0: Adding to iommu group 5
   ......
   ```

   若未开启，则没有回显或只显示如下信息：

   ```shell
   iommu: Default domain type: Translated
   ```

   开启IOMMU：

   1.为Linux内核增加启动参数: `intel_iommu=on iommu=pt`;

   ```shell
   vim /boot/grub2/grub.cfg
   linux /vmlinuz-5.15.0+ root=/dev/mapper/openeuler-root ro resume=/dev/mapper/openeuler-swap rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap crashkernel=512M intel_iommu=on iommu=pt
   ```

   2.重启Host OS;

2. 加载 vfio-pci 内核模块

   ```shell
   # modprobe vfio-pci
   # lsmod | grep vfio_pci
   ```

   成功加载 vfio-pci 模块，则回显如下：

   ```shell
   vfio_pci              327680  0
   vfio_virqfd           327680  1 vfio_pci
   vfio                  327680  2 vfio_iommu_type1,vfio_pci
   ```

3. 将 PCI 设备从主机解绑，重新绑定到 vfio-pci 驱动

   假设使用 VFIO 直通 Hi1822 网卡设备，首先查看网卡设备对应的 PCI 设备信息：

   ```shell
   # lspci -v | grep "Eth" | grep 1822
   03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
   04:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
   05:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
   06:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
   ```

   选择其中 bus 号 03，slot 号 00，function 号 0 的设备，即上述的 03:00.0。然后将该 PCI 设备从主机上解绑。

   ```shell
   # echo 0000:03:00.0 > /sys/bus/pci/devices/0000:03:00.0/driver/unbind
   ```

   最后将该 PCI 设备重新绑定到 vfio-pci 驱动。

   ```shell
   lspci -ns 0000:03:00.0 |awk -F':| ' '{print 5" "6}' > /sys/bus/pci/drivers/vfio-pci/new_id
   ```

   将网卡绑定到 vfio-pci 驱动后，在主机上无法查询到对应网卡信息，只能查询到对应的 PCI 设备信息。

### VFIO 设备直通

#### 简介

VFIO(Virtual Function I/O) 是内核提供的一种用户态设备驱动方案。VFIO 驱动可以安全地把设备 I/O，中断，DMA 等能力呈现给用户空间。StratoVirt 虚拟化平台使用 VFIO 设备直通方案后，在虚拟机可以极大限度地提升 I/O 性能。

#### 使用 VFIO 直通

StratoVirt 支持 libvirt 管理，可以使用 XML 文件配置虚拟机。以下内容介绍通过修改虚拟机 XML 文件的方式，使用 VFIO 设备直通功能。

一、修改 XML 文件

1. 在主机上执行如下命令，查询 CPU 架构信息

   ```shell
   # uname -m
   ```

2. aarch64 和 x86\_64 架构分别[下载](https://atomgit.com/openeuler/stratovirt/tree/master/docs) StratoVirt 自带的 XML 文件 stratovirt\_aarch64.xml 或 stratovirtvirt\_x86.xml，并存放到任一目录，例如 /home：

   ```shell
   # cp stratovirt/docs/stratovirt_$arch.xml /home
   ```

3. 根据实际需求，修改XML文件中的VFIO配置。 bus，slot，function 为上述绑定到 vfio-pci 驱动的 PCI 设备。相关配置如下：

   ```shell
   <!-- vfio -->
   <hostdev mode='subsystem' type='pci' managed='yes'>
   <driver name='vfio'/>
   <source>
       <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
   </source>
   </hostdev>
   ```

   上例中，设备类型为 PCI 设备，managed='yes' 表示 libvirt 将把 PCI 设备从主机解绑，并重新绑定到 vfio-pci 驱动。source 项配置了需要作为 VFIO 直通设备的 domain，bus，slot，function 信息。

二、使用 libvirt 命令行创建并登录虚拟机

```shell
# virsh create stratovirt_$arch.xml
# virsh list --all
Id  Name   State
--------------------
1  StratoVirt  running
# virsh console 1
```

三、在虚拟机内查看并使用 VFIO 直通网卡

1. 配置前查看网卡信息

   ```shell
   # ip a
   1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
        valid_lft forever preferred_lft forever
   2: enp1s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff
   ```

2. 动态配置网卡的 IP 地址

   ```shell
   # dhclient
   ```

3. 查询 IP 是否配置成功

   ```shell
   # ip a
   1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
     valid_lft forever preferred_lft forever
   2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.3/16 brd 192.168.255.255 scope global dynamic enp1s0
     valid_lft 86453sec preferred_lft 86453sec
   ```

   如上回显可知，成功分配了 IP 地址 192.168.1.3，虚拟机可以直接使用配置的网卡

   说明：使用的直通网卡如果没有连接物理网络，将获取不到网络信息。

#### 解绑 VFIO 驱动

如果需要将直通给虚拟机使用的网卡解除绑定，可以登录主机，执行如下命令，将网卡设备重新绑定到主机上。其中，hinic是对应网卡设备驱动的类型：

```shell
# echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/unbind
# echo 0000:03:00.0 > /sys/bus/pci/drivers/hinic/bind
```

说明：绑定 VFIO 驱动前，可以再主机上执行 ethtool -i enp0 命令，获取网卡设备驱动类型。enp0 为对应网卡名称。

### SR-IOV 直通

#### 简介

使用 VFIO 设备直通时，虚拟机能直接访问硬件，但每个设备只能被一个虚拟机独占。SR-IOV 直通技术支持将一个 PF(Physical Function) 虚拟出多个 VF (Virtual Function)，并直通给不同虚拟机，解决了设备直通的独占问题，增加可用的设备。

#### 操作步骤

1. 创建多个 VF：

   sriov\_numvfs 文件用于描述 SR-IOV 提供的 VF 个数，存放在 `/sys/bus/pci/devices/domain\:bus\:slot.function/` 路径下，例如上述例子中的 bus 号 03，slot 号 00，function 号 0 的设备，可以使用如下命令创建4个 VF：

   ```shell
   # echo 4 > /sys/bus/pci/devices/0000\:03\:00.0/sriov_numvfs
   ```

2. 确认 VF 设备创建成功

   ```shell
   # lspci -v | grep "Eth" | grep 1822
   ```

   回显如下，说明成功创建了4个 VF 03:00.1、03:00.2、03:00.3、03:00.4：

   ```shell
   03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45)
   03:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
   03:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
   03:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
   03:00.4 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45)
   ```

3. 上述创建的 VF 设备均可以直通给虚拟机，使用 SR-IOV 设备的方法与普通 PCI 设备的直通方法相同。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/stratovirt_introduction.md
---
# StratoVirt介绍

## 概述

StratoVirt是计算产业中面向云数据中心的企业级虚拟化平台，实现了一套架构统一支持虚拟机、容器、Serverless三种场景。StratoVirt在轻量低噪、软硬协同、Rust语言级安全等方面具备核心竞争力，在架构设计上预留了组件化拼装的能力和接口，可以按需灵活组装高级特性直至演化到支持标准虚拟化，在特性需求、应用场景和轻快灵巧之间达到平衡。

## 架构说明

StratoVirt核心架构自顶向下分为三层：

* 外部接口：兼容QMP（QEMU Monitor Protocol）协议，具有完备的OCI兼容能力，同时支持对接libvirt。
* BootLoader：轻量化场景下抛弃传统BIOS+GRUB的启动模式实现快速启动，同时标准虚拟化场景下支持UEFI启动。
* 模拟主板：
  * microvm：充分利用软硬协同能力，精简化设备模型，低时延资源伸缩能力。
  * 标准机型：提供ACPI表实现UEFI启动，支持添加virtio-pci以及VFIO直通设备等，极大提高虚拟机的I/O性能。

整体架构视图如**图1**所示。

**图1** StratoVirt整体架构图

![](./figures/StratoVirt_architecture.jpg)

## 特性

* 基于硬件的高隔离能力；
* 快速冷启动：得益于极简设计，StratoVirt可以在50ms内启动microvm机型；
* 低内存开销： StratoVirt的内存占用小于4MB ；
* IO增强： StratoVirt提供普通IO能力与极简IO设备仿真；
* OCI兼容性：StratoVirt与isula和kata容器配合使用，可以完美融入Kubernetes生态系统；
* 多平台支持：全面支持Intel和ARM平台；
* 可扩展性：StratoVirt保留接口和设计，用于导入更多特性，甚至扩展到标准虚拟化支持；
* 安全性：运行时系统调用数小于46。

## 实现

### 运行架构

* StratoVirt虚拟机是Linux中一个独立的进程。进程有三种线程：主线程、VCPU线程、I/O线程：
  * 主线程是异步收集和处理来自外部模块（如VCPU线程）的事件的循环；
  * 每个VCPU都有一个线程处理本VCPU的trap事件；
  * 可以为I/O设备配置iothread提升I/O性能。

#### 约束

* 仅支持Linux操作系统，推荐内核版本为4.19, 5.10；
* 虚拟机操作系统仅支持Linux，内核版本建议为4.19, 5.10；
* 最大支持254个CPU。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/supporting_oci_hooks.md
---
# Supporting OCI hooks

## Description

The running of standard OCI hooks within the lifecycle of a container is supported. There are three types of standard hooks:

* prestart hook: executed after the  **isula start**  command is executed and before the init process of the container is started.
* poststart hook: executed after the init process is started and before the  **isula start**  command is returned.
* poststop hook: executed after the container is stopped and before the stop command is returned.

The configuration format specifications of OCI hooks are as follows:

* **path**: (Mandatory) The value must be a character string and must be an absolute path. The specified file must have the execute permission.
* **args**: (Optional) The value must be a character string array. The syntax is the same as that of  **args**  in  **execv**.
* **env**: (Optional) The value must be a character string array. The syntax is the same as that of environment variables. The content is a key-value pair, for example,  **PATH=/usr/bin**.
* **timeout**: (Optional) The value must be an integer that is greater than 0. It indicates the timeout interval for hook execution. If the running time of the hook process exceeds the configured time, the hook process is killed.

The hook configuration is in JSON format and usually stored in a file ended with  **json**. An example is as follows:

```json
{
        "prestart": [
            {
                "path": "/usr/bin/echo",
                "args": ["arg1", "arg2"],
                "env":  [ "key1=value1"],
                "timeout": 30
            },
            {
                "path": "/usr/bin/ls",
                "args": ["/tmp"]
            }
        ],
        "poststart": [
            {
                "path": "/usr/bin/ls",
                "args": ["/tmp"],
                "timeout": 5
            }
        ],
        "poststop": [
            {
                "path": "/tmp/cleanup.sh",
                "args": ["cleanup.sh", "-f"]
            }
        ]
}
```

## APIs

Both iSulad and iSula provide the hook APIs. The default hook configurations provided by iSulad apply to all containers. The hook APIs provided by iSula apply only to the currently created container.

The default OCI hook configurations provided by iSulad are as follows:

* Set the configuration item  **hook-spec**  in the  **/etc/isulad/daemon.json**  configuration file to specify the path of the hook configuration file. Example:  **"hook-spec": "/etc/default/isulad/hooks/default.json"**
* Use the  **isulad --hook-spec**  parameter to set the path of the hook configuration file.

The OCI hook configurations provided by iSula are as follows:

* **isula create --hook-spec**: specifies the path of the hook configuration file in JSON format.
* **isula run --hook-spec**: specifies the path of the hook configuration file in JSON format.

The configuration for  **run**  takes effect in the creation phase.

## Usage Restrictions

* The path specified by  **hook-spec**  must be an absolute path.
* The file specified by  **hook-spec**  must exist.
* The path specified by  **hook-spec**  must contain a common text file in JSON format.
* The file specified by  **hook-spec**  cannot exceed 10 MB.
* **path**  configured for hooks must be an absolute path.
* The file that is designated by  **path**  configured for hooks must exist.
* The file that is designated by  **path**  configured for hooks must have the execute permission.
* The owner of the file that is designated by  **path**  configured for hooks must be user  **root**.
* Only user  **root**  has the write permission on the file that is designated by  **path**  configured for hooks.
* The value of  **timeout**  configured for hooks must be greater than  **0**.

---

---
url: /en/docs/22.03_LTS_SP4/server/administration/sysmaster/sysmaster_usage.md
---
# sysmaster Usage Instructions

This section provides examples on how to use sysmaster, including:

* service unit configuration file creation
* unit service management operations, such as starting, stopping, and viewing services

## Unit Configuration File Creation

You can create unit configuration files in the **/usr/lib/sysmaster/system/** directory.

### Types of Unit Configuration Files

Currently, sysmaster supports unit configuration files of the **target**, **socket**, and **service** types.

* **target**: Encapsulated startup target managed by sysmaster, which is used for grouping units as a synchronization point. sysmaster provides targets for different states. For example, **multi-user.target** indicates that the system has been started. You can use this target to configure services to run in this state.
* **socket**: Encapsulated socket for inter-process communication to support socket-based startup. For example, you can configure a service unit to depend on a socket. When data is written to the socket, sysmaster starts the corresponding service unit.
* **service**: Encapsulated process monitored and controlled by sysmaster.

### Composition of Unit Configuration Files

A unit configuration file consists of three sections:

* **Unit**: common configuration description of the unit, such as the service name, description, and dependencies
* **Install**: description of how the service is installed and started
* **Service** and **Socket**: configurations of different unit types

### Creating a service Unit

The **sshd** service is used to remotely log in to the server and run commands and perform operations on the remote terminal.
The following configuration items are used to create an **sshd.service** service unit:

```bash
[Unit]
Description="OpenSSH server daemon"
Documentation="man:sshd(8) man:sshd_config(5)"
After="sshd-keygen.target"
Wants="sshd-keygen.target"

[Service]
Type="notify"
EnvironmentFile="-/etc/sysconfig/sshd"
ExecStart="/usr/sbin/sshd -D $OPTIONS"
ExecReload="/bin/kill -HUP $MAINPID"
KillMode="process"
Restart="on-failure"
RestartSec=42

[Install]
WantedBy="multi-user.target"
```

The configuration items in the example are described as follows:

* **Description**: Main functions of the unit.
* **Documentation**: Document link of the unit.
* **After**: Unit startup sequence. In the example, **sshd.service** is started after **sshd-keygen.target**.
* **Wants**: Dependency on another unit. In the example, **sshd-keygen.target** is automatically started with **sshd.service**.
* **Type**: How sysmaster starts the service. **notify** indicates that a notification will be sent after the main process is started.
* **EnvironmentFile**: Path of file that stores environment variables to be loaded.
* **ExecStart**: Command executed when the service is started. In the example, `sshd` is executed when **sshd.service** is started.
* **ExecReload**: Command executed to reload the **sshd.service** configurations.
* **KillMode**: How the process is killed when the service process needs to be stopped. **process** indicates that only the main process is killed.
* **Restart**: Whether to restart the service when the service exits or stops in different situations. **on-failure** indicates that the service is restarted when the service exits abnormally.
* **RestartSec**: Amount of time to wait before the service is restarted after the service exits.
* **WantedBy**: Units that depend on **sshd.service**.

## Unit Service Management

`sctl` is a CLI tool of sysmaster. It is used to check and control the behavior of the sysmaster server and the status of each service. It can start, stop, restart, and check system services.

### Starting a Service

Run the following command to start the **sshd** service and run the commands specified by **ExecStart**:

```bash
# sctl start sshd.service
```

### Stopping a Service

Run the following command to stop the **sshd** service and kill the process started by **ExecStart**:

```bash
# sctl stop sshd.service
```

### Restarting a Service

Run the following command to restart the **sshd** service. After the command is executed, the **sshd** service is stopped and then started.

```bash
# sctl restart sshd.service
```

### Checking Service Status

Run the following command to check the status of the **sshd** service. You can check whether the service is running properly by viewing the service status.

```bash
# sctl status sshd.service
```

---

---
url: /en/docs/22.03_LTS_SP4/server/administration/sysmaster/overview.md
---
# sysMaster User Guide

## Overview

sysMaster is a collection of ultra-lightweight and highly reliable service management programs. It provides an innovative implementation of PID 1 to replace the conventional init process. Written in Rust, sysMaster is equipped with fault monitoring, second-level self-recovery, and quick startup capabilities, which help improve OS reliability and service availability.

sysMaster manages processes, containers, and VMs centrally, and is ideal for server, cloud computing, and embedded scenarios.

sysMaster divides the functions of traditional PID 1 into a 1+1+N architecture based on application scenarios.

As shown in the figure, sysMaster consists of three components:\
• sysmaster-init, a new implementation of PID 1, is applicable to embedded systems with functions such as system initialization, zombie process recycling, and keep-alive monitoring.\
• sysmaster-core undertakes the core service management functions and incorporates the reliability framework to enable live updates and quick self-recovery in the event of crashes, ensuring 24/7 service availability.\
• sysmaster-exts offers a collection of components (such as devMaster for device management and busMaster for bus communication) that deliver key system functions. You can choose the components to use as required.

**Figure 1** sysMaster architecture\
![sysMaster](./figures/sysMaster.png)

Currently, sysMaster consists of the sysmaster and devmaster services, which manages services and devices, respectively. Their functions will be described in the following sections.

## Intended Audience

This document is intended for openEuler users who need to manage services and devices. Users must:

* Know basic Linux operations.
* Be familiar to configuration of services and devices.

---

---
url: /zh/docs/22.03_LTS_SP4/server/administration/sysmaster/sysmaster_usage.md
---
# sysmaster使用说明

本章主要通过一些实例来带领用户初步使用 `sysmaster`，例如:

* 如何创建 `service`服务单元配置文件。
* 如何管理单元服务，例如启动、停止、查看服务。

## 创建单元配置文件

用户可以在 `/usr/lib/sysmaster/system/`目录下创建单元配置文件。

### 单元配置文件的类型

当前 `sysmaster`支持 `target`、`socket`、`service`类型的单元配置文件。

* `target`：封装了一个由 `sysmaster`管理的启动目标，用于将多个单元集中到一个同步点。`sysmaster`提供不同阶段的 `target`单元，例如   `multi-user.target`代表系统已完成启动，用户可以依赖此目标，启动自己的服务。
* `socket`：封装了一个用于进程间通信的套接字 `socket`， 以支持基于套接字的启动。例如用户可以配置 `service`单元依赖此 `socket`，当此 `socket`有数据写入时，`sysmaster`会拉起对应的 `service`单元。
* `service`：封装了一个被 `sysmaster`监视与控制的进程。

### 单元配置文件的构成

单元配置文件通常由3块组成：

* `Unit`：单元的公共配置说明，如服务名称、描述、依赖关系等。
* `Install`：描述如何安装和启动服务。
* `Service`、`Socket`：各个单元类型的配置。

### 创建service单元配置

`sshd`服务被用来远程登录到服务器，并在远程终端上执行命令和操作。
使用如下配置项来创建一个 `sshd.service`服务单元配置。

```bash
[Unit]
Description="OpenSSH server daemon"
Documentation="man:sshd(8) man:sshd_config(5)"
After="sshd-keygen.target"
Wants="sshd-keygen.target"

[Service]
Type="notify"
EnvironmentFile="-/etc/sysconfig/sshd"
ExecStart="/usr/sbin/sshd -D $OPTIONS"
ExecReload="/bin/kill -HUP $MAINPID"
KillMode="process"
Restart="on-failure"
RestartSec=42

[Install]
WantedBy="multi-user.target"
```

以下是对单元配置文件中选项配置的说明。

* `Description`：说明该 `unit`的主要功能。
* `Documentation`：说明该 `unit`的文档链接。
* `After`：配置同时启动的单元的先后顺序，`sshd.service`服务将在 `sshd-keygen.target`之后启动。
* `Wants`：配置一个单元对另一个单元的依赖，启动 `sshd.service`服务，将会自动启动 `sshd-keygen.target`。
* `Type`：配置 `sysmaster` 如何启动此服务，`notify`表明需要主进程启动完成后发送通知消息。
* `EnvironmentFile`：设置环境变量的文件读取路径。
* `ExecStart`：配置服务启动时执行的命令，启动 `sshd.service`服务会执行 `sshd`命令。
* `ExecReload`：配置重新加载 `sshd.service`的配置时执行的命令。
* `KillMode`：配置当需要停止服务进程时，杀死服务进程的方法，`process`表示只杀死主进程。
* `Restart`：配置服务不同情况下退出或终止，是否重新启动服务，`on-failure`表示当服务非正常退出时重新启动服务。
* `RestartSec`：配置当服务退出时，重新拉起服务的间隔时间。
* `WantedBy`：配置依赖当前 `sshd.service`服务的单元。

## 管理单元服务

`sctl`是 `sysmaster`的命令行工具，用于检查和控制 `sysmaster`服务端行为和各个服务的状态，它可以启动、停止、重启、检查系统服务。

### 启动服务

使用以下命令可以启动 `sshd`服务和运行 `ExecStart`所配置的命令。

```bash
# sctl start sshd.service
```

### 停止服务

使用以下命令可以停止 `sshd`服务，杀死 `ExecStart`所运行的进程。

```bash
# sctl stop sshd.service
```

### 重启服务

使用以下命令可以重启 `sshd`服务，该命令会先停止后启动服务。

```bash
# sctl restart sshd.service
```

### 查看服务状态

使用以下命令可以查看服务 `sshd`运行状态，用户可以查看服务的状态来获取服务是否正常运行。

```bash
# sctl status sshd.service
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/administration/sysmaster/overview.md
---
# sysMaster用户指南

## 概述

`sysMaster` 是一套超轻量、高可靠的服务管理程序集合，是对 `1` 号进程的全新实现，旨在改进传统的 `init` 守护进程。它使用 `Rust` 编写，具有故障监测、秒级自愈和快速启动等能力，从而提升操作系统可靠性和业务可用度。

`sysMaster` 支持进程、容器和虚拟机的统一管理，其适用于服务器、云计算和嵌入式等多个场景。

`sysMaster` 实现思路是将传统 `1` 号进程的功能解耦分层，结合使用场景，拆分出 `1+1+N` 的架构。

如下面 `sysMaster` 系统架构图所示，主要包含三个方面:\
• `sysmaster-init`：新的 `1` 号进程提供系统初始化、僵尸进程回收、监控保活等功能，可单独应用于嵌入式场景。\
• `sysmaster-core`：承担原有服务管理的核心功能，引入可靠性框架，使其具备崩溃快速自愈、热升级等能力，保障业务全天在线。\
• `sysmaster-exts`：使原本耦合的各组件功能独立，提供系统关键功能的组件集合（如设备管理 `devMaster`，总线通信 `busMaster` 等），各组件可单独使用，可根据不同场景灵活选用。

**图1** sysMaster整体架构图
![sysMaster](./figures/sysMaster.png)

`sysMaster` 目前主要由 `sysmaster` 和 `devmaster` 两部分功能组成，其中 `sysmaster` 负责服务的管理，`devmaster` 负责设备的管理，下面将对这两部分功能进行说明。

## 读者对象

本文档主要适用于使用 `openEuler` 并需要对服务和设备进行管理的用户。用户需要具备以下经验和技能：

* 熟悉 `Linux` 基本操作
* 对服务配置和设备有一定了解

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/sysmonitor/sysmonitor_user_guide.md
---
# sysmonitor

## Introduction

The system monitor (sysmonitor) daemon monitors exceptions that occur during OS running and records the exceptions in the system log file **/var/log/sysmonitor.log**. sysmonitor runs as a service. You can run the `systemctl start|stop|restart|reload sysmonitor` command to start, stop, restart, and reload the service. You are advised to deploy sysmonitor to locate system exceptions.

![](./figures/sysmonitor_functions.png)

### Precautions

* sysmonitor cannot run concurrently.
* Ensure that all configuration files are valid. Otherwise, the monitoring service may be abnormal.
* The root privilege is required for sysmonitor service operations, configuration file modification, and log query. The **root** user has the highest permission in the system. When performing operations as the **root** user, follow the operation guide to avoid system management and security risks caused by improper operations.

### Configuration Overview

Configuration file **/etc/sysconfig/sysmonitor** of sysmonitor defines the monitoring period of each monitoring item and specifies whether to enable monitoring. Spaces are not allowed between the configuration item, equal sign (=), and configuration value, for example, **PROCESS\_MONITOR="on"**.

Configuration description

| Item                   | Description                                                  | Mandatory| Default Value                                |
| ------------------------- | ------------------------------------------------------------ | -------- | -------------------------------------- |
| PROCESS\_MONITOR           | Whether to enable key process monitoring. The value can be **on** or **off**.               | No      | on                                   |
| PROCESS\_MONITOR\_PERIOD    | Monitoring period on key processes, in seconds.                              | No      | 3                                     |
| PROCESS\_RECALL\_PERIOD     | Interval for attempting to restart a key process after the process fails to be recovered, in minutes. The value can be an integer ranging from 1 to 1440.| No      | 1                                   |
| PROCESS\_RESTART\_TIMEOUT   | Timeout interval for recovering a key process service from an exception, in seconds. The value can be an integer ranging from 30 to 300.| No      | 90                                    |
| PROCESS\_ALARM\_SUPPRESS\_NUM | Number of alarm suppression times when the key process monitoring configuration uses the alarm command to report alarms. The value is a positive integer.| No      | 5                                      |
| FILESYSTEM\_MONITOR        | Whether to enable ext3 and ext4 file system monitoring. The value can be **on** or **off**.  | No      | on                                   |
| DISK\_MONITOR              | Whether to enable drive partition monitoring. The value can be **on** or **off**.              | No      | on                                   |
| DISK\_MONITOR\_PERIOD       | Drive monitoring period, in seconds.                                    | No      | 60                                    |
| INODE\_MONITOR             | Whether to enable drive inode monitoring. The value can be **on** or **off**.         | No      | on                                   |
| INODE\_MONITOR\_PERIOD      | Drive inode monitoring period, in seconds.                             | No      | 60                                   |
| NETCARD\_MONITOR           | Whether to enable NIC monitoring. The value can be **on** or **off**.                | No      | on                                   |
| FILE\_MONITOR              | Whether to enable file monitoring. The value can be **on** or **off**.                 | No      | on                                   |
| CPU\_MONITOR               | Whether to enable CPU monitoring. The value can be **on** or **off**.                   | No      | on                                   |
| MEM\_MONITOR               | Whether to enable memory monitoring. The value can be **on** or **off**.                    | No      | on                                   |
| PSCNT\_MONITOR             | Whether to enable process count monitoring. The value can be **on** or **off**.                    | No      | on                                   |
| FDCNT\_MONITOR             | Whether to enable file descriptor (FD) count monitoring. The value can be **on** or **off**.                 | No      | on                                   |
| CUSTOM\_DAEMON\_MONITOR     | Whether to enable custom daemon item monitoring. The value can be **on** or **off**.        | No      | on                                   |
| CUSTOM\_PERIODIC\_MONITOR   | Whether to enable custom periodic item monitoring. The value can be **on** or **off**.   | No      | on                                   |
| IO\_DELAY\_MONITOR          | Whether to enable local drive I/O latency monitoring. The value can be **on** or **off**.             | No      | off                                   |
| PROCESS\_FD\_NUM\_MONITOR    | Whether to enable process FD count monitoring. The value can be **on** or **off**.        | No      | on                                   |
| PROCESS\_MONITOR\_DELAY     | Whether to wait until all monitoring items are normal when sysmonitor is started. The value can be **on** (wait) or **off** (do not wait).| No      | on                                   |
| NET\_RATE\_LIMIT\_BURST      | NIC route information printing rate, that is, the number of logs printed per second.        | No      | 5 Valid range: 0 to 100     |
| FD\_MONITOR\_LOG\_PATH       | FD monitoring log file                                        | No      | /var/log/sysmonitor.log|
| ZOMBIE\_MONITOR            | Whether to monitor zombie processes                                            | No      | off                                   |
| CHECK\_THREAD\_MONITOR      | Whether to enable internal thread self-healing. The value can be **on** or **off**.                       | No      | on       |
| CHECK\_THREAD\_FAILURE\_NUM  | Number of internal thread self-healing checks in a period.                                  | No      | 3Valid range: 2 to 10             |

* After modifying the **/etc/sysconfig/sysmonitor** configuration file, restart the sysmonitor service for the configurations to take effect.
* If an item is not configured in the configuration file, it is enabled by default.
* After the internal thread self-healing function is enabled, if a sub-thread of the monitoring item is suspended and the number of checks in a period exceeds the configured value, the sysmonitor service is restarted for restoration. The configuration is reloaded. The configured key process monitoring and customized monitoring are restarted. If this function affects user experience, you can disable it.

### Command Reference

* Start sysmonitor.

```shell
systemctl start sysmonitor
```

* Stop sysmonitor.

```shell
systemctl stop sysmonitor
```

* Restart sysmonitor.

```shell
systemctl restart sysmonitor
```

* Reload sysmonitor for the modified configurations to take effect.

```shell
systemctl reload sysmonitor
```

### Monitoring Logs

By default, logs is split and dumped to prevent the **sysmonitor.log** file from getting to large. Logs are dumped to a drive directory. In this way, a certain number of logs can be retained.

The configuration file is **/etc/rsyslog.d/sysmonitor.conf**. Because this rsyslog configuration file is added, after sysmonitor is installed for the first time, you need to restart the rsyslog service to make the sysmonitor log configuration take effect.

```sh
$template sysmonitorformat,"%TIMESTAMP:::date-rfc3339%|%syslogseverity-text%|%msg%\n"

$outchannel sysmonitor, /var/log/sysmonitor.log, 2097152, /usr/libexec/sysmonitor/sysmonitor_log_dump.sh
if ($programname == 'sysmonitor' and $syslogseverity <= 6) then {
:omfile:$sysmonitor;sysmonitorformat
stop
}

if ($msg contains 'Time has been changed') then {
:omfile:$sysmonitor;sysmonitorformat
stop
}

if ($programname == 'sysmonitor' and $syslogseverity > 6) then {
/dev/null
stop
}
```

## ext3/ext4 Filesystem Monitoring

### Introduction

A fault in the filesystem may trigger I/O operation errors, which further cause OS faults. File system fault detection can detect the faults in real time so that system administrators or users can rectify them in a timely manner.

### Configuration File Description

None

### Exception Logs

For a file system to which the errors=remount-ro mounting option is added, if the ext3 or ext4 file system is faulty, the following exception information is recorded in the **sysmonitor.log** file:

```sh
info|sysmonitor[127]: loop0 filesystem error. Remount filesystem read-only.
```

In other exception scenarios, if the ext3 or ext4 file system is faulty, the following exception information is recorded in the **sysmonitor.log** file:

```sh
info|sysmonitor[127]: fs_monitor_ext3_4: loop0 filesystem error. flag is 1879113728.
```

## Key Processing Monitoring

### Introduction

Key processes in the system are periodically monitored. When a key process exits abnormally, sysmonitor automatically attempts to recover the key process. If the recovery fails, alarms can be reported. The system administrator can be promptly notified of the abnormal process exit event and whether the process is restarted. Fault locating personnel can locate the time when the process exits abnormally from logs.

### Configuration File Description

The configuration file directory is **/etc/sysmonitor/process**. Each process or module corresponds to a configuration file.

```sh
USER=root
NAME=irqbalance
RECOVER_COMMAND=systemctl restart irqbalance
MONITOR_COMMAND=systemctl status irqbalance
STOP_COMMAND=systemctl stop irqbalance
```

The configuration items are as follows:

| Item                | Description                                                  | Mandatory| Default Value                                             |
| ---------------------- | ------------------------------------------------------------ | -------- | --------------------------------------------------- |
| NAME                   | Process or module name                                                | Yes      | None                                                 |
| RECOVER\_COMMAND        | Recovery command                                                    | No      | None                                                 |
| MONITOR\_COMMAND        | Monitoring command If the command output is 0, the process is normal. If the command output is greater than 0, the process is abnormal.| No      | pgrep -f $(which xxx)*xxx* is the process name configured in the **NAME** field.|
| STOP\_COMMAND           | Stopping command                                                    | No      | None                                                 |
| USER                   | User nameUser for executing the monitoring, recovery, and stopping commands or scripts | No      | If this item is left blank, the **root** user is used by default.                    |
| CHECK\_AS\_PARAM         | Parameter passing switchIf this item is on, the return value of **MONITOR\_COMMAND** is transferred to the **RECOVER\_COMMAND** command or script as an input parameter. If this item is set to off or other values, the function is disabled.| No      | None                                                 |
| MONITOR\_MODE           | Monitoring mode- **parallel** or **serial** | No      | serial                                              |
| MONITOR\_PERIOD         | Monitoring period- Parallel monitoring period- This item does not take effect when the monitoring mode is **serial**.| No      | 3                                                   |
| USE\_CMD\_ALARM          | Alarm modeIf this parameter is set to **on** or **ON**, alarms are reported using the alarm reporting command.       | No      | None                                                 |
| ALARM\_COMMAND          | Alarm reporting command                                                | No      | None                                                 |
| ALARM\_RECOVER\_COMMAND | Alarm recovery command                                                | No      | No                                                 |

* After modifying the configuration file for monitoring key processes, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period.
* The recovery command and monitoring command must not block. Otherwise, the monitoring thread of the key process becomes abnormal.
* When the recovery command is executed for more than 90 seconds, the stopping command is executed to stop the process.
* If the recovery command is empty or not configured, the monitoring command does not attempt to recover the key process when detecting that the key process is abnormal.
* If a key process is abnormal and fails to be started for three consecutive times, the process is started based on the period specified by **PROCESS\_RECALL\_PERIOD** in the global configuration file.
* If the monitored process is not a daemon process, **MONITOR\_COMMAND** is mandatory.
* If the configured key service does not exist in the current system, the monitoring does not take effect and the corresponding information is printed in the log. If a fatal error occurs in other configuration items, the default configuration is used and no error is reported.
* The permission on the configuration file is 600. You are advised to set the monitoring item to the **service** type of systemd (for example, **MONITOR\_COMMAND=systemctl status irqbalance**). If a process is monitored, ensure that the **NAME** field is an absolute path.
* The restart, reload, and stop of sysmonitor do not affect the monitored processes or services.
* If **USE\_CMD\_ALARM** is set to **on**, you must ensure the validiy of **ALARM\_COMMAND** and **ALARM\_RECOVER\_COMMAND**. If **ALARM\_COMMAND** or **ALARM\_RECOVER\_COMMAND** is empty or not configured, no alarm is reported.
* The security of user-defined commands, such as the monitoring, recovery, stopping, alarm reporting, and alarm recovery commands, is ensured by users. Commands are executed by the user **root**. You are advised to set the script command permission to be used only by the user **root** to prevent privilege escalation for common users.
* If the length of the monitoring command cannot be greater than 200 characters. Otherwise, the process monitoring fails to be added.
* When the recovery command is set to a systemd service restart command (for example, **RECOVER\_COMMAND=systemctl restart irqbalance**), check whether the recovery command conflicts with the open source systemd service recovery mechanism. Otherwise, the behavior of key processes may be affected after exceptions occur.
* The processes started by the sysmonitor service are in the same cgroup as the sysmonitor service, and resources cannot be restricted separately. Therefore, you are advised to use the open source systemd mechanism to recover the processes.

### Exception Logs

* **RECOVER\_COMMAND** configured

  If a process or module exception is detected, the following exception information is recorded in the **/var/log/sysmonitor.log** file:

  ```sh
  info|sysmonitor[127]: irqbalance is abnormal, check cmd return 1, use "systemctl restart irqbalance" to recover
  ```

  If the process or module recovers, the following information is recorded in the **/var/log/sysmonitor.log** file:

  ```sh
  info|sysmonitor[127]: irqbalance is recovered
  ```

* **RECOVER\_COMMAND** not configured

  If a process or module exception is detected, the following exception information is recorded in the **/var/log/sysmonitor.log** file:

  ```sh
  info|sysmonitor[127]: irqbalance is abnormal, check cmd return 1, recover cmd is null, will not recover
  ```

  If the process or module recovers, the following information is recorded in the **/var/log/sysmonitor.log** file:

  ```sh
  info|sysmonitor[127]: irqbalance is recovered
  ```

## File Monitoring

### Introduction

If key system files are deleted accidentally, the system may run abnormally or even break down. Through file monitoring, you can learn about the deletion of key files or the addition of malicious files in the system in a timely manner, so that administrators and users can learn and rectify faults in a timely manner.

### Configuration File Description

The configuration file is **/etc/sysmonitor/file**. Each monitoring configuration item occupies a line. A monitoring configuration item contains the file (directory) and event to be monitored. The file (directory) to be monitored is an absolute path. The file (directory) to be monitored and the event to be monitored are separated by one or more spaces.

The file monitoring configuration items can be added to the **/etc/sysmonitor/file.d** directory. The configuration method is the same as that of the **/etc/sysmonitor/file** directory.

* Due to the log length limit, it is recommended that the absolute path of a file or directory be less than 223 characters. Otherwise, the printed logs may be incomplete.

* Ensure that the path of the monitored file is correct. If the configured file does not exist or the path is incorrect, the file cannot be monitored.

* Due to the path length limit of the system, the absolute path of the monitored file or directory must be less than 4096 characters.

* Directories and regular files can be monitored. **/proc**, **/proc/\***, **/dev**, **/dev/\***, **/sys**, **/sys/\***, pipe files, or socket files cannot be monitored.

* Only deletion events can be monitored in **/var/log** and **/var/log/\***.

* If multiple identical paths exist in the configuration file, the first valid configuration takes effect. In the log file, you can see messages indicating that the identical paths are ignored.

* Soft links cannot be monitored. When a hard link file deletion event is configured, the event is printed only after the file and all its hard links are deleted.

* When a monitored event occurs after the file monitoring is successfully added, the monitoring log records the absolute path of the configured file.

* Currently, directories cannot be monitored recursively. The configured directory is monitored but not its subdirectories.

* The events to be monitored are configured using bitmaps as follows.

```sh
  -------------------------------
  | 11~32   | 10   | 9   |  1~8 | 
  -------------------------------
```

Each bit in the event bitmap represents an event. If bit *n* is set to 1, the event corresponding to bit *n* is monitored. The hexadecimal number corresponding to the monitoring bitmap is the event monitoring item written to the configuration file.

| Item| Description        | Mandatory|
| ------ | ------------------ | -------- |
| 1~8    | Reserved              | No      |
| 9      | File or directory addition event| Yes      |
| 10     | File or directory deletion event| Yes      |
| 11~32  | Reserved              | No      |

* After modifying the file monitoring configuration file, run `systemctl reload sysmonitor`. The new configuration takes effect within 60 seconds.
* Strictly follow the preceding rules to configure events to be monitored. If the configuration is incorrect, the events cannot be monitored. If an event to be monitored in the configuration item is empty, only the deletion event is monitored by default, that is, **0x200**.
* After a file or directory is deleted, the deletion event is reported only when all processes that open the file stop.
* If a monitored a is modified by `vi` or `sed`, "File XXX may have been changed" is recorded in the monitoring log.
* Currently, file addition and deletion events can be monitored, that is, the ninth and tenth bits take effect. Other bits are reserved and do not take effect. If a reserved bit is configured, the monitoring log displays a message indicating that the event monitoring is incorrectly configured.

**Example**

Monitor the subdirectory addition and deletion events in **/home**. The lower 12-bit bitmap is 001100000000. The configuration is as follows:

```sh
/home 0x300
```

Monitor the file deletion events of **/etc/ssh/sshd\_config**. The lower 12-bit bitmap is 001000000000. The configuration is as follows:

```sh
/etc/sshd/sshd_config 0x200
```

### Exception Logs

If a configured event occurs to the monitored file, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]: 1 events queued
info|sysmonitor[127]: 1th events handled
info|sysmonitor[127]: Subfile "111" under "/home" was added.
```

## Drive Partition Monitoring

### Introduction

The system periodically monitors the drive partitions mounted to the system. When the drive partition usage is greater than or equal to the configured alarm threshold, the system records a drive space alarm. When the drive partition usage falls below the configured alarm recovery threshold, a drive space recovery alarm is recorded.

### Configuration File Description

The configuration file is **/etc/sysmonitor/disk**.

```sh
DISK="/var/log"  ALARM="90" RESUME="80"
DISK="/" ALARM="95" RESUME="85"
```

| Item| Description            | Mandatory| Default Value|
| ------ | ---------------------- | -------- | ------ |
| DISK   | Mount directory        | Yes      | None    |
| ALARM  | Integer indicating the drive space alarm threshold| No      | 90     |
| RESUME | Integer indicating the drive space alarm recovery threshold| No      | 80     |

* After modifying the configuration file for drive space monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period.
* If a mount directory is configured repeatedly, the last configuration item takes effect.
* The value of **ALARM** must be greater than that of **RESUME**.
* Only the mount point or the drive partition of the mount point can be monitored.
* When the CPU usage and I/O usage are high, the `df` command execution may time out. As a result, the drive usage cannot be obtained.
* If a drive partition is mounted to multiple mount points, an alarm is reported for each mount point.

### Exception Logs

If a drive space alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
warning|sysmonitor[127]: report disk alarm, /var/log used:90% alarm:90%
info|sysmonitor[127]: report disk recovered, /var/log used:4% resume:10%
```

## NIC Status Monitoring

### Introduction

During system running, the NIC status or IP address may change due to human factors or exceptions. You can monitor the NIC status and IP address changes to detect exceptions in a timely manner and locate exception causes.

### Configuration File Description

The configuration file is **/etc/sysmonitor/network**.

```sh
#dev event
eth1 UP
```

The following table describes the configuration items.

| Item| Description                                                  | Mandatory| Default Value                                           |
| ------ | ------------------------------------------------------------ | -------- | ------------------------------------------------- |
| dev    | NIC name                                                      | Yes      | None                                               |
| event  | Event to be monitored. The value can be **UP**, **DOWN**, **NEWADDR**, or **DELADDR**.- UP: The NIC is up.- DOWN: The NIC is down.- NEWADDR: An IP address is added.- DELADDR: An IP address is deleted.| No      | If this item is empty, **UP**, **DOWN**, **NEWADDR**, and **DELADDR** are monitored.|

* After modifying the configuration file for NIC monitoring, run `systemctl reload sysmonitor` for the new configuration to take effect.
* The **UP** and **DOWN** status of virtual NICs cannot be monitored.
* Ensure that each line in the NIC monitoring configuration file contains less than 4096 characters. Otherwise, a configuration error message will be recorded in the monitoring log.
* By default, all events of all NICs are monitored. That is, if no NIC monitoring is configured, the **UP**, **DOWN**, **NEWADDR**, and **DELADDR** events of all NICs are monitored.
* If a NIC is configured but no event is configured, all events of the NIC are monitored by default.
* The events of route addition can be recorded five times per second. You can change the number of times by setting **NET\_RATE\_LIMIT\_BURST** in **/etc/sysconfig/sysmonitor**.

### Exception Logs

If a NIC event is detected, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]: lo: ip[::1] prefixlen[128] is added, comm: (ostnamed)[1046], parent comm: syst    emd[1]
info|sysmonitor[127]: lo: device is up, comm: (ostnamed)[1046], parent comm: systemd[1]
```

If a route event is detected, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[881]: Fib4 replace table=255 192.168.122.255/32, comm: daemon-init[1724], parent com    m: systemd[1]
info|sysmonitor[881]: Fib4 replace table=254 192.168.122.0/24, comm: daemon-init[1724], parent comm:     systemd[1]
info|sysmonitor[881]: Fib4 replace table=255 192.168.122.0/32, comm: daemon-init[1724], parent comm:     systemd[1]
info|sysmonitor[881]: Fib6 replace fe80::5054:ff:fef6:b73e/128, comm: kworker/1:3[209], parent comm:     kthreadd[2]
```

## CPU Monitoring

### Introduction

The system monitors the global CPU usage or the CPU usage in a specified domain. When the CPU usage exceeds the configured alarm threshold, the system runs the configured log collection command.

### Configuration File Description

The configuration file is **/etc/sysmonitor/cpu**.

When the global CPU usage of the system is monitored, an example of the configuration file is as follows:

```sh
# cpu usage alarm percent
ALARM="90"

# cpu usage alarm resume percent
RESUME="80"

# monitor period (second)
MONITOR_PERIOD="60"

# stat period (second)
STAT_PERIOD="300"

# command executed when cpu usage exceeds alarm percent
REPORT_COMMAND=""
```

When the CPU usage of a specific domain is monitored, an example of the configuration file is as follows:

```sh
# monitor period (second)
MONITOR_PERIOD="60"

# stat period (second)
STAT_PERIOD="300"

DOMAIN="0,1"  ALARM="90" RESUME="80"
DOMAIN="2,3"  ALARM="50" RESUME="40"

# command executed when cpu usage exceeds alarm percent
REPORT_COMMAND=""
```

| Item        | Description                                                  | Mandatory| Default Value|
| -------------- | ------------------------------------------------------------ | -------- | ------ |
| ALARM          | Number greater than 0, indicating the CPU usage alarm threshold                                   | No      | 90     |
| RESUME         | Number greater than or equal to 0, indicating the CPU usage alarm recovery threshold                               | No      | 80     |
| MONITOR\_PERIOD | Monitoring period, in seconds. The value is greater than 0.                                   | No      | 60     |
| STAT\_PERIOD    | Statistical period, in seconds. The value is greater than 0.                                   | No      | 300    |
| DOMAIN         | CPU IDs in the domain, represented by decimal numbers- CPU IDs can be enumerated and separated by commas, for example, **1,2,3**. CPU IDs can be specified as a range in the formate of *X*-*Y*, for example, **0-2**. The two representations can be used together, for example, **0, 1, 2-3** or **0-1, 2-3**. Spaces or other characters are not allowed.- Each monitoring domain has an independent configuration item. Each configuration item supports a maximum of 256 CPUs. A CPU ID must be unique in a domain and across domains.| No      | None    |
| REPORT\_COMMAND | Command for collecting logs after the CPU usage exceeds the alarm threshold                      | No      | None    |

* After modifying the configuration file for CPU monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period.
* The value of **ALARM** must be greater than that of **RESUME**.
* After the CPU domain monitoring is configured, the global average CPU usage of the system is not monitored, and the separately configured **ALARM** and **RESUME** values do not take effect.
* If the configuration of a monitoring domain is invalid, CPU monitoring is not performed at all.
* All CPUs configured in **DOMAIN** must be online. Otherwise, the domain cannot be monitored.
* The command of **REPORT\_COMMAND** cannot contain insecure characters such as **&**, **;**, and **>**, and the total length cannot exceed 159 characters. Otherwise, the command cannot be executed.
* Ensure the security and validity of **REPORT\_COMMAND**. sysmonitor is responsible only for running the command as the **root** user.
* **REPORT\_COMMAND** must not block. When the execution time of the command exceeds 60s, the sysmonitor forcibly stops the execution.
* Even if the CPU usage of multiple domains exceeds the threshold in a monitoring period, **REPORT\_COMMAND** is executed only once.

### Exception Logs

If a global CPU usage alarm is detected or cleared and the log collection command is configured, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]: CPU usage alarm: 91.3%
info|sysmonitor[127]: cpu monitor: execute REPORT_COMMAND[sysmoniotrcpu] successfully
info|sysmonitor[127]: CPU usage resume 70.1%
```

If a domain average CPU usage alarm is detected or cleared and the log collection command is configured, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]: CPU 1,2,3 usage alarm: 91.3%
info|sysmonitor[127]: cpu monitor: execute REPORT_COMMAND[sysmoniotrcpu] successfully
info|sysmonitor[127]: CPU 1,2,3 usage resume 70.1%
```

## Memory Monitoring

### Introduction

Monitors the system memory usage and records logs when the memory usage exceeds or falls below the threshold.

### Configuration File Description

The configuration file is **/etc/sysmonitor/memory**.

```sh
# memory usage alarm percent
ALARM="90"

# memory usage alarm resume percent
RESUME="80"

# monitor period(second)
PERIOD="60"
```

### Configuration Item Description

| Item| Description                   | Mandatory| Default Value|
| ------ | ----------------------------- | -------- | ------ |
| ALARM  | Number greater than 0, indicating the memory usage alarm threshold    | No      | 90     |
| RESUME | Number greater than or equal to 0, indicating the memory usage alarm recovery threshold| No      | 80     |
| PERIOD | Monitoring period, in seconds. The value is greater than 0.   | No      | 60     |

* After modifying the configuration file for memory monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period.
* The value of **ALARM** must be greater than that of **RESUME**.
* The average memory usage in three monitoring periods is used to determine whether an alarm is reported or cleared.

### Exception Logs

If a memory alarm is detected, sysmonitor obtains the **/proc/meminfo** information and prints the information in the **/var/log/sysmonitor.log** file. The information is as follows:

```sh
info|sysmonitor[127]: memory usage alarm: 90%
info|sysmonitor[127]:---------------show /proc/meminfo: ---------------
info|sysmonitor[127]:MemTotal: 3496388 kB
info|sysmonitor[127]:MemFree: 2738100 kB
info|sysmonitor[127]:MemAvailable: 2901888 kB
info|sysmonitor[127]:Buffers: 165064 kB
info|sysmonitor[127]:Cached: 282360 kB
info|sysmonitor[127]:SwapCached: 4492 kB
......
info|sysmonitor[127]:---------------show_memory_info end. ---------------
```

If the following information is printed, sysmonitor runs `echo m > /proc/sysrq-trigger` to export memory allocation information. You can view the information in **/var/log/messages**.

```sh
info|sysmonitor[127]: sysrq show memory ifno in message.
```

When the alarm is recovered, the following information is displayed:

```sh
info|sysmonitor[127]: memory usage resume: 4.6%
```

## Process and Thread Monitoring

### Introduction

Monitors the number of processes and threads. When the total number of processes or threads exceeds or falls below the threshold, a log is recorded or an alarm is reported.

### Configuration File Description

The configuration file is **/etc/sysmonitor/pscnt**.

```sh
# number of processes(include threads) when alarm occur
ALARM="1600"

# number of processes(include threads) when alarm resume
RESUME="1500"

# monitor period(second)
PERIOD="60"

# process count usage alarm percent
ALARM_RATIO="90"

# process count usage resume percent
RESUME_RATIO="80"

# print top process info with largest num of threads when threads alarm
# (range: 0-1024, default: 10, monitor for thread off:0)
SHOW_TOP_PROC_NUM="10"
```

| Item           | Description                                                  | Mandatory| Default Value|
| ----------------- | ------------------------------------------------------------ | -------- | ------ |
| ALARM             | Integer greater than 0, indicating the process count alarm threshold                             | No      | 1600   |
| RESUME            | Integer greater than or equal to 0, indicating the process count alarm recovery threshold                           | No      | 1500   |
| PERIOD            | Monitoring period, in seconds. The value is greater than 0.                                   | No      | 60     |
| ALARM\_RATIO       | Number greater than 0 and less than or equal to 100. Process count alarm threshold.        | No      | 90     |
| RESUME\_RATIO      | Number greater than 0 and less than or equal to 100. Process count alarm recovery threshold, which must be less than **ALARM\_RATIO**.| No      | 80     |
| SHOW\_TOP\_PROC\_NUM | Whether to use the latest `top` information about threads                             | No      | 10     |

* After modifying the configuration file for process count monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period.
* The value of **ALARM** must be greater than that of **RESUME**.
* The process count alarm threshold is the larger between **ALARM** and **ALARM\_RATIO** in **/proc/sys/kernel/pid\_max**. The alarm recovery threshold is the larger of **RESUME** and **RESUME\_RATIO** in **/proc/sys/kernel/pid\_max**.
* The thread count alarm threshold is the larger between **ALARM** and **ALARM\_RATIO** in **/proc/sys/kernel/threads-max**. The alarm recovery threshold is the larger of **RESUME** and **RESUME\_RATIO** in **/proc/sys/kernel/threads-max**.
* The value of **SHOW\_TOP\_PROC\_NUM** ranges from 0 to 1024. 0 indicates that thread monitoring is disabled. A larger value, for example, 1024, indicates that thread alarms will be generated in the environment. If the alarm threshold is high, the performance is affected. You are advised to set this parameter to the default value 10 or a smaller value. If the impact is huge, you are advised to set this parameter to 0 to disable thread monitoring.
* The value of **PSCNT\_MONITOR** in **/etc/sysconfig/sysmonitor** and the value of **SHOW\_TOP\_PROC\_NUM** in **/etc/sysmonitor/pscnt** determine whether thread monitoring is enabled.
  * If **PSCNT\_MONITOR** is on and **SHOW\_TOP\_PROC\_NUM** is set to a valid value, thread monitoring is enabled.
  * If **PSCNT\_MONITOR** is on and **SHOW\_TOP\_PROC\_NUM** is 0, thread monitoring is disabled.
  * If **PSCNT\_MONITOR** is off, thread monitoring is disabled.
* When a process count alarm is generated, the system FD usage information and memory information (**/proc/meminfo**) are printed.
* When a thread count alarm is generated, the total number of threads, `top` process information, number of processes in the current environment, number of system FDs, and memory information (**/proc/meminfo**) are printed.
* If system resources are insufficient before a monitoring period ends, for example, the thread count exceeds the maximum number allowed, the monitoring cannot run properly due to resource limitation. As a result, the alarm cannot be generated.

### Exception Logs

If a process count alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]:---------------process count alarm start: ---------------
info|sysmonitor[127]: process count alarm:1657
info|sysmonitor[127]: process count alarm, show sys fd count: 2592
info|sysmonitor[127]: process count alarm, show mem info
info|sysmonitor[127]:---------------show /proc/meminfo: ---------------
info|sysmonitor[127]:MemTotal: 3496388 kB
info|sysmonitor[127]:MemFree: 2738100 kB
info|sysmonitor[127]:MemAvailable: 2901888 kB
info|sysmonitor[127]:Buffers: 165064 kB
info|sysmonitor[127]:Cached: 282360 kB
info|sysmonitor[127]:SwapCached: 4492 kB
......
info|sysmonitor[127]:---------------show_memory_info end. ---------------
info|sysmonitor[127]:---------------process count alarm end: ---------------
```

If a process count recovery alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]: process count resume: 1200
```

If a thread count alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]:---------------threads count alarm start: ---------------
info|sysmonitor[127]:threads count alarm: 273
info|sysmonitor[127]:open threads most 10 processes is [top1:pid=1756900,openthreadsnum=13,cmd=/usr/bin/sysmonitor --daemon]
info|sysmonitor[127]:open threads most 10 processes is [top2:pid=3130,openthreadsnum=13,cmd=/usr/lib/gassproxy -D]
.....
info|sysmonitor[127]:---------------threads count alarm end. ---------------
```

## System FD Count Monitoring

### Introduction

Monitors the number of system FDs. When the total number of system FDs exceeds or is less than the threshold, a log is recorded.

### Configuration File Description

The configuration file is **/etc/sysmonitor/sys\_fd\_conf**.

```sh
# system fd usage alarm percent
SYS_FD_ALARM="80"
# system fd usage alarm resume percent
SYS_FD_RESUME="70"
# monitor period (second)
SYS_FD_PERIOD="600"
```

Configuration items:

| Item       | Description                                               | Mandatory| Default Value|
| ------------- | --------------------------------------------------------- | -------- | ------ |
| SYS\_FD\_ALARM  | Integer greater than 0 and less than 100, indicating the alarm threshold of the percentage of the total number of FDs and the maximum number of FDs allowed.| No      | 80    |
| SYS\_FD\_RESUME | Integer greater than 0 and less than 100, indicating the alarm recovery threshold of the percentage of the total number of FDs and the maximum number of FDs allowed.| No      | 70    |
| SYS\_FD\_PERIOD | Integer between 100 and 86400, indicating the monitor period in seconds               | No      | 600    |

* After modifying the configuration file for FD count monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period.
* The value of **SYS\_FD\_ALARM** must be greater than that of **SYS\_FD\_RESUME**. If the value is invalid, the default value is used and a log is recorded.

### Exception Logs

An FD count alarm is recorded in the monitoring logs when detected. The following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]: sys fd count alarm: 259296
```

When a system FD usage alarm is generated, the top three processes that use the most FDs are printed.

```sh
info|sysmonitor[127]:open fd most three processes is:[top1:pid=23233,openfdnum=5000,cmd=/home/openfile]
info|sysmonitor[127]:open fd most three processes is:[top2:pid=23267,openfdnum=5000,cmd=/home/openfile]
info|sysmonitor[127]:open fd most three processes is:[top3:pid=30144,openfdnum=5000,cmd=/home/openfile]
```

## Drive Inode Monitoring

### Introduction

Periodically monitors the inodes of mounted drive partitions. When the drive partition inode usage is greater than or equal to the configured alarm threshold, the system records a drive inode alarm. When the drive inode usage falls below the configured alarm recovery threshold, a drive inode recovery alarm is recorded.

### Configuration File Description

The configuration file is **/etc/sysmonitor/inode**.

```sh
DISK="/"
DISK="/var/log"
```

| Item| Description               | Mandatory| Default Value|
| ------ | ------------------------- | -------- | ------ |
| DISK   | Mount directory           | Yes      | None    |
| ALARM  | Integer indicating the drive inode alarm threshold| No      | 90     |
| RESUME | Integer indicating the drive inode alarm recovery threshold| No      | 80     |

* After modifying the configuration file for drive inode monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period.
* If a mount directory is configured repeatedly, the last configuration item takes effect.
* The value of **ALARM** must be greater than that of **RESUME**.
* Only the mount point or the drive partition of the mount point can be monitored.
* When the CPU usage and I/O usage are high, the `df` command execution may time out. As a result, the drive inode usage cannot be obtained.
* If a drive partition is mounted to multiple mount points, an alarm is reported for each mount point.

### Exception Logs

If a drive inode alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[4570]:report disk inode alarm, /var/log used:90% alarm:90%
info|sysmonitor[4570]:report disk inode recovered, /var/log used:79% alarm:80%
```

## Local Drive I/O Latency Monitoring

### Introduction

Reads the local drive I/O latency data every 5 seconds and collects statistics on 60 groups of data every 5 minutes. If more than 30 groups of data are greater than the configured maximum I/O latency, the system records a log indicating excessive drive I/O latency.

### Configuration File Description

The configuration file is **/etc/sysmonitor/iodelay**.

```sh
DELAY_VALUE="500"
```

| Item     | Description          | Mandatory| Default Value|
| ----------- | -------------------- | -------- | ------ |
| DELAY\_VALUE | Maximum drive I/O latency| Yes      | 500    |

### Exception Logs

If a drive I/O latency alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]:local disk sda IO delay is too large, I/O delay threshold is 70.
info|sysmonitor[127]:disk is sda, io delay data: 71 72 75 87 99 29 78 ......
```

If a drive I/O latency recovery alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]:local disk sda IO delay is normal, I/O delay threshold is 70.
info|sysmonitor[127]:disk is sda, io delay data: 11 22 35 8 9 29 38 ......
```

## Zombie Process Monitoring

### Introduction

Monitors the number of zombie processes in the system. If the number is greater than the alarm threshold, an alarm log is recorded. When the number drops lower than the recovery threshold, a recovery alarm is reported.

### Configuration File Description

The configuration file is **/etc/sysmonitor/zombie**.

```sh
# Ceiling zombie process counts of alarm
ALARM="500"

# Floor zombie process counts of resume
RESUME="400"

# Periodic (second)
PERIOD="600"
```

| Item| Description                     | Mandatory| Default Value|
| ------ | ------------------------------- | -------- | ------ |
| ALARM  | Number greater than 0, indicating the zombie process count alarm threshold    | No      | 500    |
| RESUME | Number greater than or equal to 0, indicating the zombie process count recovery threshold| No      | 400    |
| PERIOD | Monitoring period, in seconds. The value is greater than 0.      | No      | 60     |

### Exception Logs

If a zombie process count alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file:

```sh
info|sysmonitor[127]: zombie process count alarm: 600
info|sysmonitor[127]: zombie process count resume: 100
```

## Custom Monitoring

### Introduction

You can customize monitoring items. The monitoring framework reads the content of the configuration file, parses the monitoring attributes, and calls the monitoring actions to be performed. The monitoring module provides only the monitoring framework. It is not aware of what users are monitoring or how to monitor, and does not report alarms.

### Configuration File Description

The configuration files are stored in **/etc/sysmonitor.d/**. Each process or module corresponds to a configuration file.

```sh
MONITOR_SWITCH="on"
TYPE="periodic"
EXECSTART="/usr/sbin/iomonitor_daemon"
PERIOD="1800"
```

| Item        | Description                                                  | Mandatory             | Default Value|
| -------------- | ------------------------------------------------------------ | --------------------- | ------ |
| MONITOR\_SWITCH | Monitoring switch                                                    | No                   | off    |
| TYPE           | Custom monitoring item type**daemon**: background execution**periodic**: periodic execution| Yes                   | None    |
| EXECSTART      | Monitoring command                                                | Yes                   | None    |
| ENVIROMENTFILE | Environment variable file                                            | No                   | None    |
| PERIOD         | If the type is **periodic**, this parameter is mandatory and sets the monitoring period. The value is an integer greater than 0.| Yes when the type is **periodic**| None    |

* The absolute path of the configuration file or environment variable file cannot contain more than 127 characters. The environment variable file path cannot be a soft link path.
* The length of the **EXECSTART** command cannot exceed 159 characters. No space is allowed in the key field.
* The execution of the periodic monitoring command cannot time out. Otherwise, the custom monitoring framework will be affected.
* Currently, a maximum of 256 environment variables can be configured.
* The custom monitoring of the daemon type checks whether the `reload` command is delivered or whether the daemon process exits abnormally every 10 seconds. If the `reload` command is delivered, the new configuration is loaded 10 seconds later. If a daemon process exits abnormally, the daemon process is restarted 10 seconds later.
* If the content of the **ENVIROMENTFILE** file changes, for example, an environment variable is added or the environment variable value changes, you need to restart the sysmonitor service for the new environment variable to take effect.
* You are advised to set the permission on the configuration files in the **/etc/sysmonitor.d/** directory to 600. If **EXECSTART** is only an executable file, you are advised to set the permission on the executable file to 550.
* After a daemon process exits abnormally, sysmonitor reloads the configuration file of the daemon process.

### Exception Logs

If a monitoring item of the daemon type exits abnormally, the **/var/log/sysmonitor.log** file records the following information:

```sh
info|sysmonitor[127]: custom daemon monitor: child process[11609] name unetwork_alarm exit code[127],[1] times.
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/sysmonitor/sysmonitor_user_guide.md
---
# sysmonitor

## 介绍

System Monitor Daemon

sysmonitor 负责监控 OS 系统运行过程中的异常，将监控到的异常记录到系统日志（`/var/log/sysmonitor.log`）中。sysmonitor 以服务的形式提供，可以通过 `systemctl start|stop|restart|reload sysmonitor` 启动、关闭、重启、重载服务。建议产品部署 sysmonitor 调测软件，便于定位系统异常问题。

![](./figures/sysmonitor功能列表.png)

### 注意事项

* sysmonitor 不支持并发执行。
* 各配置文件须合法配置，否则可能造成监控框架异常。
* sysmonitor 服务操作和配置文件修改，日志查询需要 root 权限。root 用户具有系统最高权限，在使用 root 用户进行操作时，请严格按照操作指导进行操作，避免不规范操作造成系统管理及安全风险。

### 配置总览

sysmonitor 有一个主配置文件(`/etc/sysconfig/sysmonitor`)，用于配置各监控项的监控周期、是否需要监控。配置项的=和"之间不能有空格,如`PROCESS_MONITOR="on"`。

配置说明

| 配置项                    | 配置项说明                                                   | 是否必配 | 默认值                                 |
| ------------------------- | ------------------------------------------------------------ | -------- | -------------------------------------- |
| PROCESS\_MONITOR           | 设定是否开启关键进程监控，on为开启，off为关闭                | 否       | on                                   |
| PROCESS\_MONITOR\_PERIOD    | 设置关键进程监控的周期，单位秒                               | 否       | 3s                                     |
| PROCESS\_RECALL\_PERIOD     | 关键进程恢复失败后再次尝试拉起周期，单位分，取值范围为1到1440之间的整数 | 否       | 1min                                   |
| PROCESS\_RESTART\_TIMEOUT   | 关键进程服务异常恢复过程中超时时间，单位秒，取值范围为30至300之间的整数 | 否       | 90s                                    |
| PROCESS\_ALARM\_SUPRESS\_NUM | 设置关键进程监控配置使用告警命令上报告警时的告警抑制次数，取值范围为正整数 | 否       | 5                                      |
| FILESYSTEM\_MONITOR        | 设定是否开启 ext3/ext4 文件系统监控，on 为开启，off 为关闭   | 否       | on                                   |
| DISK\_MONITOR              | 设定是否开启磁盘分区监控，on为开启，off 为关闭               | 否       | on                                   |
| DISK\_MONITOR\_PERIOD       | 设定磁盘监控周期，单位秒                                     | 否       | 60s                                    |
| INODE\_MONITOR             | 设定是否开启磁盘 inode 监控，on 为开启， off 为关闭          | 否       | on                                   |
| INODE\_MONITOR\_PERIOD      | 设定磁盘 inode 监控周期，单位秒                              | 否       | 开启                                   |
| NETCARD\_MONITOR           | 设定是否开启网卡监控，on 为开启， off 为关闭                 | 否       | on                                   |
| FILE\_MONITOR              | 设定是否开启文件监控，on为开启， off 为关闭                  | 否       | on                                   |
| CPU\_MONITOR               | 设定是否开启 cpu 监控，on 为开启， off 为关闭                    | 否       | on                                   |
| MEM\_MONITOR               | 设定是否开启内存监控，on 为开启， off 为关闭                     | 否       | on                                   |
| PSCNT\_MONITOR             | 设定是否开启进程数监控，on为开启，off 为关闭                     | 否       | on                                   |
| FDCNT\_MONITOR             | 设定是否开启 fd 总数监控，on 为开启，off 为关闭                  | 否       | on                                   |
| CUSTOM\_DAEMON\_MONITOR     | 用户自定义的 daemon类型的监控项，on为开启，off为关闭         | 否       | on                                   |
| CUSTOM\_PERIODIC\_MONITOR   | 用户自定义的 periodic 类型的监控项，on为开启， off 为关闭    | 否       | on                                   |
| IO\_DELAY\_MONITOR          | 本地磁盘 IO 延时监控开关，on 为开启，off 为关闭              | 否       | off                                   |
| PROCESS\_FD\_NUM\_MONITOR    | 设定是否开启单个进程句柄数监控，on为开启，off 为关闭         | 否       | on                                   |
| PROCESS\_MONITOR\_DELAY     | sysmonitor 启动时，是否等待所有的监控项都正常，on为等待，off为不等待 | 否       | on                                   |
| NET\_RATE\_LIMIT\_BURST      | 网卡监控路由信息打印抑制频率，即一秒内打印多少条日志         | 否       | 5 有效范围是 0-100，默认为5      |
| FD\_MONITOR\_LOG\_PATH       | 文件句柄监控日志文件                                         | 否       | 默认配置路径为 /var/log/sysmonitor.log |
| ZOMBIE\_MONITOR            | 僵尸进程监控开关                                             | 否       | off                                   |
| CHECK\_THREAD\_MONITOR      | 内部线程自愈开关，on为开启，off为关闭                        | 否       | on若不配置，默认值为开启       |
| CHECK\_THREAD\_FAILURE\_NUM  | 内部线程自愈的周期检查次数                                   | 否       | 默认值为3，范围为【2,10】              |

* 修改 `/etc/sysconfig/sysmonitor` 配置文件后，需要重启 sysmonitor 服务生效。
* 配置文件中，如果某一项没有配置，默认为监控项开启。
* 内部线程自愈开启后，当监控项子线程卡住，且超过配置的周期检查次数，会重启 sysmonitor 服务，进行恢复，会重新加载配置，对于配置的关键进程监控和自定义监控，会重新拉起执行。如果对于用户使用有影响，可以选择关闭该功能。

### 命令参考

* 启动监控服务

```shell
systemctl start sysmonitor
```

* 关闭监控服务

```shell
systemctl stop sysmonitor
```

* 重启监控服务

```shell
systemctl restart sysmonitor
```

* 修改监控项的配置文件后，重载监控服务可使修改后的配置动态生效

```shell
systemctl reload sysmonitor
```

### 监控日志

在默认情况下，为了防止 sysmonitor.log 文件过大，提供了切分转储日志的机制。日志将被转储到磁盘目录下，这样就能够保持一定量的日志。

配置文件为`/etc/rsyslog.d/sysmonitor.conf`，因为增加了 rsyslog 配置文件，第一次安装 sysmonitor 后，需要重启 rsyslog 服务生效 sysmonitor 日志配置。

```sh
$template sysmonitorformat,"%TIMESTAMP:::date-rfc3339%|%syslogseverity-text%|%msg%\n"

$outchannel sysmonitor, /var/log/sysmonitor.log, 2097152, /usr/libexec/sysmonitor/sysmonitor_log_dump.sh
if ($programname == 'sysmonitor' and $syslogseverity <= 6) then {
:omfile:$sysmonitor;sysmonitorformat
stop
}

if ($msg contains 'Time has been changed') then {
:omfile:$sysmonitor;sysmonitorformat
stop
}

if ($programname == 'sysmonitor' and $syslogseverity > 6) then {
/dev/null
stop
}
```

## ext3/ext4  文件系统监控

### 简介

当文件系统出现故障时会导致 IO 操作异常从而引发操作系统一系列问题。通过文件系统故障检测及时发现，以便于系统管理员或用户及时处理故障，修复问题。

### 配置文件说明

无

### 异常日志

对于增加了 errors=remount-ro 挂载选项的文件系统，如果监控到 ext3/ext4文件系统故障，sysmonitor.log 中打印异常信息示例如下：

```sh
info|sysmonitor[127]: loop0 filesystem error. Remount filesystem read-only.
```

其他异常场景下，如果监控到 ext3/ext4 文件系统故障，sysmonitor.log 中打印异常信息示例如下：

```sh
info|sysmonitor[127]: fs_monitor_ext3_4: loop0 filesystem error. flag is 1879113728.
```

## 关键进程监控

### 简介

定期监控系统中关键进程，当系统内关键进程异常退出时，自动尝试恢复关键进程。如果恢复失败并需要告警，可上报告警。系统管理员能被及时告知进程异常退出事件，以及进程是否被恢复拉起。问题定位人员能从日志中定位进程异常退出的时间。

### 配置文件说明

配置目录为`/etc/sysmonitor/process`, 每个进程或模块一个配置文件。

```sh
USER=root
NAME=irqbalance
RECOVER_COMMAND=systemctl restart irqbalance
MONITOR_COMMAND=systemctl status irqbalance
STOP_COMMAND=systemctl stop irqbalance
```

各配置项如下：

| 配置项                 | 配置项说明                                                   | 是否必配 | 默认值                                              |
| ---------------------- | ------------------------------------------------------------ | -------- | --------------------------------------------------- |
| NAME                   | 进程或模块名                                                 | 是       | 无                                                  |
| RECOVER\_COMMAND        | 恢复命令                                                     | 否       | 无                                                  |
| MONITOR\_COMMAND        | 监控命令 命令返回值为0视为进程正常，命令返回大于 0视为进程异常 | 否       | pgrep -f $(which xxx) "xxx"为NAME字段中配置的进程名 |
| STOP\_COMMAND           | 停止命令                                                     | 否       | 无                                                  |
| USER                   | 用户名 使用指定的用户执行、监控、恢复、停止命令或脚本  | 否       | 如果配置项为空，则默认使用 root                     |
| CHECK\_AS\_PARAM         | 参数传递开关 开关设置为 on 时，在执行 RECOVER\_COMMAND 命令时，会将 MONITOR\_COMMAND 的返回值作为入参，传给 RECOVER\_COMMAND 命令或脚本。 开关为 off 或其他时，功能关闭 | 否       | 无                                                  |
| MONITOR\_MODE           | 监控模式- 配置为 parallel，并行监控- 配置为 serial，串行监控 | 否       | serial                                              |
| MONITOR\_PERIOD         | 监控周期- 并行监控监控周期- 监控模块配置为 serial，该配置项不生效 | 否       | 3                                                   |
| USE\_CMD\_ALARM          | 告警模式配置为 on 或 ON，则使用告警命令上报告警        | 否       | 无                                                  |
| ALARM\_COMMAND          | 上报告警命令                                                 | 否       | 无                                                  |
| ALARM\_RECOVER\_COMMAND | 恢复告警命令                                                 | 否       | 否                                                  |

* 修改关键进程监控的配置文件后，须执行 `systemctl reload sysmonitor`， 新的配置在一个监控周期后生效。
* 恢复命令和监控命令不阻塞，否则会造成关键进程监控线程异常。
* 当恢复命令执行超过 90 s时，会调用停止命令终止进程。
* 当恢复命令配置为空或不配置时，监控命令检查到关键进程异常时，不会尝试进行拉起。
* 当关键进程异常时，并且尝试拉起三次都不成功，最终会按照全局配置文件中配置的 PROCESS\_RECALL\_PERIOD 周期进行拉起。
* 当监控的进程不是 daemon 进程，MONITOR\_COMMAND 必配。
* 若配置的关键服务在当前系统上不存在，则该监控不会生效，日志中会有相应提示；其他配置项，出现致命性错误，将使用默认配置，不报错。
* 配置文件权限为 600，监控项建议为 systemd 中的 service类型（如 MONITOR\_COMMAND=systemctl status irqbalance）, 若监控的为进程，请确保 NAME 字段为绝对路径。
* sysmonitor 重启（restart）、重载（reload）、退出（stop）都不会影响所监控的进程或服务。
* 若 USE\_CMD\_ALARM 的配置为 on，ALARM\_COMMAND、ALARM\_RECOVER\_COMMAND 的配置由用户保障。ALARM\_COMMAND、ALARM\_RECOVER\_COMMAND 为空或没有配置，则不上报告警。
* 对于用户自行配置的命令，如监控命令，恢复命令，停止命令，上报告警命令，恢复告警命令等，命令的安全性由用户保证。命令由 root 权限执行，建议脚本命令权限设置为仅供 root 使用，避免普通用户提权风险。
* 配置监控命令的长度不大于200，大于 200，添加进程监控失败。
* 当恢复命令配置为 systemd 的重启服务命令时（如`RECOVER_COMMAND=systemctl restart irqbalance`），需注意是否与开源 systemd 恢复服务的机制冲突，否则可能会影响关键进程异常后的行为模式。
* 由 sysmonitor 恢复拉起的进程将和 sysmonitor 服务在同一个 Cgroup 当中，无法单独进行资源限制，因此建议优先使用开源 systemd 机制进行恢复。

### 异常日志

* 配置 RECOVER\_COMMAND

  如果监控到进程或模块异常，/var/log/sysmonitor.log 中打印异常信息示例如下：

  ```sh
  info|sysmonitor[127]: irqbalance is abnormal, check cmd return 1, use "systemctl restart irqbalance" to recover
  ```

  如果监控到进程或模块恢复正常，/var/log/sysmonitor.log 中打印日志示例如下：

  ```h
  info|sysmonitor[127]: irqbalance is recovered
  ```

* 不配置 RECOVER\_COMMAND

  如果监控到进程或模块异常，/var/log/sysmonitor.log 中打印异常信息示例如下：

  ```h
  info|sysmonitor[127]: irqbalance is abnormal, check cmd return 1, recover cmd is null, will not recover
  ```

  如果监控到进程或模块恢复正常，/var/log/sysmonitor.log 中打印日志示例如下：

  ```h
  info|sysmonitor[127]: irqbalance is recovered
  ```

## 文件监控

### 简介

系统关键文件被意外删除后，会导致系统运行异常甚至崩溃。通过文件监控可以及时获知系统中关键文件被删除或者有恶意文件被添加，以便管理员和用户及时获知并处理故障。

### 配置文件说明

配置文件为 `/etc/sysmonitor/file`。每个监控配置项为一行，监控配置项包含两个内容：监控文件（目录）和监控事件。监控文件（目录）是绝对路径，监控文件（目录）和监控事件中间由一个或多个空格隔开。

配置文件支持在`/etc/sysmonitor/file.d` 目录下增加文件监控项配置，配置方法与 `/etc/sysmoitor/file` 相同。

* 由于日志长度限制，建议配置的文件和目录绝对路径长度小于 223。如果配置的监控对象绝对路径长度超过223，可能会有日志打印不完整的现象出现。

* 请用户自行确保监控文件路径正确，如果配置文件不存在或路径错误则无法监控到该文件。

* 由于系统路径长度限制，监控的文件或目录绝对路径长度必须小于 4096。

* 支持监控目录和常规文件，`/proc` 和 `/proc/*` `/dev` 和 `/dev/*` `/sys` 和 `/sys/*` 管道文件 socket 文件等均不支持监控。

* /var/log 和 /var/log/\* 均只支持删除事件。

* 当配置文件中存在多个相同路径的时候，以第一条合法配置为准，其他相同配置均不生效。在日志文件中可以查看到其他相同配置被忽略的提示。

* 不支持对软链接配置监控；当配置硬链接文件的删除事件时，需删除该文件和它的全部硬链接才会打印文件删除事件。

* 当文件添加监控成功及监控的事件发生时，监控日志打印的是配置文件中路径的绝对路径。

* 目前暂不支持目录递归监控，只能监控配置文件中的目录，子目录不会监控。

* 监控文件（目录）采用了位图的方式配置要监控的事件，对文件或目录进行监控的事件位图如下所示：

```sh
  -------------------------------
  | 11~32   | 10   | 9   |  1~8 | 
  -------------------------------
```

事件位图每一位代表一个事件，第N位如果置1，则表示监控第n位对应的事件；如果第 n 位置 0，则表示不监控第 n 位对应的事件。监控位图对应的 16 进制数，即是写到配置文件中的监控事件项。

| 配置项 | 配置项说明         | 是否必配 |
| ------ | ------------------ | -------- |
| 1~8    | 保留               | 否       |
| 9      | 文件、目录添加事件 | 是       |
| 10     | 文件、目录删除事件 | 是       |
| 11~32  | 保留               | 否       |

* 修改文件监控的配置文件后，须执行`systemctl reload sysmonitor`，新的配置在最多 60 秒后生效。
* 监控事件需要严格遵守上述规则，如果配置有误，则无法监控；如果配置项中监控事件为空，则默认只监控删除事件，即 0x200。
* 文件或目录删除后，只有当所有打开该文件的进程都停止后才会上报删除事件。
* 监控的文件通过 vi、sed 等操作修改后会在监控日志中打印 File "XXX" may have been changed。
* 文件监控目前实现了对添加和删除事件的监控，即第9位和第10位有效，其他位为保留位，暂不生效。如果配置了保留位，监控日志会提示监控事件配置错误。

**示例**

配置对 /home 下子目录的增加和删除事件监控，低12 位位图为：001100000000，则可以配置如下：

```sh
/home 0x300
```

配置对 /etc/ssh/sshd\_config 文件的删除事件监控，低12位位图为：001000000000，则可以配置如下：

```sh
/etc/sshd/sshd_config 0x200
```

### 异常日志

如果监控文件有配置的事件发生，/var/log/sysmonitor.log 中打印日志示例如下：

```sh
info|sysmonitor[127]: 1 events queued
info|sysmonitor[127]: 1th events handled
info|sysmonitor[127]: Subfile "111" under "/home" was added.
```

## 磁盘分区监控

### 简介

定期监控系统中挂载的磁盘分区空间，当磁盘分区使用率大于或等于用户设置的告警阈值时，记录磁盘空间告警。当磁盘分区使用率小于用户设置的告警恢复阈值时，记录磁盘空间恢复告警。

### 配置文件说明

配置文件为 `/etc/sysmonitor/disk`。

```sh
DISK="/var/log"  ALARM="90" RESUME="80"
DISK="/" ALARM="95" RESUME="85"
```

| 配置项 | 配置项说明             | 是否必配 | 默认值 |
| ------ | ---------------------- | -------- | ------ |
| DISK   | 磁盘挂载目录名         | 是       | 无     |
| ALARM  | 整数，磁盘空间告警阈值 | 否       | 90     |
| RESUME | 整数，磁盘空间恢复阈值 | 否       | 80     |

* 修改磁盘空间监控的配置文件后，须执行 systemctl reload sysmonitor，新的配置在一个监控周期后生效。
* 重复配置的挂载目录，最后一个配置项生效。
* ALARM 值应该大于 RESUME 值。
* 只能针对挂载点或被挂载点的磁盘分区做监控。
* 在 CPU 和 IO 高压场景下，df 命令执行超时，会导致磁盘利用率获取不到。
* 当多个挂载点对应同一个磁盘分区时，以挂载点为准来上报告警。

### 异常日志

如果监控到磁盘空间告警，`/var/log/sysmonitor.log`中打印信息示例如下：

```sh
warning|sysmonitor[127]: report disk alarm, /var/log used:90% alarm:90%
info|sysmonitor[127]: report disk recovered, /var/log used:4% resume:10%
```

## 网卡状态监控

### 简介

系统运行过程中可能出现人为原因或异常而导致网卡状态或 IP 发生改变，对网卡状态和 IP 变化进行监控，以便及时感知到异常并方便定位异常原因。

### 配置文件说明

配置文件为 `/etc/sysmonitor/network`。

```sh
#dev event
eth1 UP
```

各配置项说明如下表

| 配置项 | 配置项说明                                                   | 是否必配 | 默认值                                            |
| ------ | ------------------------------------------------------------ | -------- | ------------------------------------------------- |
| dev    | 网卡名                                                       | 是       | 无                                                |
| event  | 侦听事件，可取 UP, DOWN,NEWADDR, DELADDR.- UP： 网卡 UP- DOWN： 网卡  DOWN- NEWADDR： 增加 ip 地址- DELADDR： 删除 ip 地址 | 否       | 若侦听事件为空则 UP，DOWN，NEWADDR，DELADDR都监控 |

* 修改网卡监控的配置文件后，执行 `systemctl reload sysmonitor`，新的配置生效。
* 不支持虚拟网卡 UP 和 DOWN 状态监控。
* 请确保网卡监控的配置文件每行少于 4096 个字符，若超过4096个字符会在监控日志中打印配置错误的提示信息。
* 默认监控所有网卡的所有事件信息，即不配置任何网卡，默认监控所有网卡的 UP，DOWN，NEWADDR，DELADDR 事件。
* 如果配置网卡，不配置事件，则默认监控改网卡的所有事件。
* 增加路由信息，默认一秒五条，可通过/etc/sysconfig/sysmonitor 的 NET\_RATE\_LIMIT\_BURST 配置选项配置一秒钟打印路由信息数量。

### 异常日志

如果监控到配置的网卡事件，`/var/log/sysmonitor.log` 中打印信息示例如下：

```sh
info|sysmonitor[127]: lo: ip[::1] prefixlen[128] is added, comm: (ostnamed)[1046], parent comm: syst    emd[1]
info|sysmonitor[127]: lo: device is up, comm: (ostnamed)[1046], parent comm: systemd[1]

```

如果监控到路由事件, `/var/log/sysmonitor.log` 中打印信息示例如下：

```sh

info|sysmonitor[881]: Fib4 replace table=255 192.168.122.255/32, comm: daemon-init[1724], parent com    m: systemd[1]
info|sysmonitor[881]: Fib4 replace table=254 192.168.122.0/24, comm: daemon-init[1724], parent comm:     systemd[1]
info|sysmonitor[881]: Fib4 replace table=255 192.168.122.0/32, comm: daemon-init[1724], parent comm:     systemd[1]
info|sysmonitor[881]: Fib6 replace fe80::5054:ff:fef6:b73e/128, comm: kworker/1:3[209], parent comm:     kthreadd[2]

```

## cpu 监控

### 简介

监控系统全局或指定域内 cpu 的占用情况，当 cpu 使用率超出用户设置的告警阈值时，执行用户配置的日志收集命令。

### 配置文件说明

配置文件为`/etc/sysmonitor/cpu`。

当监控系统全局 cpu 时，配置文件示例如下：

```sh
# cpu usage alarm percent
ALARM="90"

# cpu usage alarm resume percent
RESUME="80"

# monitor period (second)
MONITOR_PERIOD="60"

# stat period (second)
STAT_PERIOD="300"

# command executed when cpu usage exceeds alarm percent
REPORT_COMMAND=""
```

当监控系统指定域 cpu 时，配置文件示例如下：

```sh
# monitor period (second)
MONITOR_PERIOD="60"

# stat period (second)
STAT_PERIOD="300"

DOMAIN="0,1"  ALARM="90" RESUME="80"
DOMAIN="2,3"  ALARM="50" RESUME="40"

# command executed when cpu usage exceeds alarm percent
REPORT_COMMAND=""
```

| 配置项         | 配置项说明                                                   | 是否必配 | 默认值 |
| -------------- | ------------------------------------------------------------ | -------- | ------ |
| ALARM          | 大于0，cpu 使用率告警阈值                                    | 否       | 90     |
| RESUME         | 大于等于0，cpu 使用率恢复阈值                                | 否       | 80     |
| MONITOR\_PERIOD | 监控周期（秒），取值大于0                                    | 否       | 60     |
| STAT\_PERIOD    | 统计周期（秒），取值大于0                                    | 否       | 300    |
| DOMAIN         | 域内的 cpu 信号，cpu 号均以十进制数字表示- 可以通过列举方式指定，cpu 号之间通过逗号分隔，例如：1,2,3。也可以通过范围方式指定，格式 X-Y（X\<Y）,例如：0-2。两种方式可以搭配使用，例如0,1,2-3 或 0-1,2-3。不要有空格等其他额外字符。- 每个监控域单独一个配置项，每个项支持最多配置 256 个 cpu，域内以及域之间  cpu 号均不能重复 | 否       | 无     |
| REPORT\_COMMAND | cpu 使用率超过告警阈值后的日志收集命令                       | 否       | 无     |

* 修改 cpu 监控的配置文件后，须执行 systemctl reload sysmonitor， 新的配置在一个监控周期后生效。
* ALARM 值应该大于 RESUME 值。
* 当配置监控 cpu 域后，不再对系统全局 cpu 平均使用率进行监控，单独配置的 ALARM、RESUME 值不生效。
* 如果某个监控域的配置存在非法，则整个 cpu 监控不执行。
* DOMAIN 内配置的 cpu 必须全部处于在线工作状态，否则对该域的监控无法正常进行。
* REPORT\_COMMAND 项的命令不能包含 &、；、> 等不安全字符且总长度不能超过 159个字符，否则命令无法生效。
* REPORT\_COMMAND 项的命令安全性、有效性由用户自己保证，sysmonitor 只负责以 root 用户执行该命令。
* REPORT\_COMMAND 项的命令不能阻塞，当该命令执行时间超过 60s后，sysmonitor 会强行终止执行。
* 每轮监控即使有多个域 cpu 使用率超过阈值，REPORT\_COMMAND 也仅会执行一次。

### 异常日志

如果监控到全局 cpu 使用率告警或恢复且配置了日志收集命令，`/var/log/sysmonitor.log` 中打印信息示例如下：

```sh
info|sysmonitor[127]: CPU usage alarm: 91.3%
info|sysmonitor[127]: cpu monitor: execute REPORT_COMMAND[sysmoniotrcpu] successfully
info|sysmonitor[127]: CPU usage resume 70.1%
```

如果监控到某个域的 cpu 平均使用率告警或恢复且配置了日志收集命令，`/var/log/sysmonitor.log` 中打印信息示例如下：

```sh
info|sysmonitor[127]: CPU 1,2,3 usage alarm: 91.3%
info|sysmonitor[127]: cpu monitor: execute REPORT_COMMAND[sysmoniotrcpu] successfully
info|sysmonitor[127]: CPU 1,2,3 usage resume 70.1%
```

## 内存监控

### 简介

监控系统内存占用情况，当内存使用率超出或低于阈值时，记录日志。

### 配置文件说明

配置文件为 `/etc/sysmonitor/memory`。

```sh
# memory usage alarm percent
ALARM="90"

# memory usage alarm resume percent
RESUME="80"

# monitor period(second)
PERIOD="60"
```

### 配置项说明

| 配置项 | 配置项说明                    | 是否必配 | 默认值 |
| ------ | ----------------------------- | -------- | ------ |
| ALARM  | 大于0，内存占用率告警阈值     | 否       | 90     |
| RESUME | 大于等于0，内存占用率恢复阈值 | 否       | 80     |
| PERIOD | 监控周期（秒），取值大于 0    | 否       | 60     |

* 修改内存监控的配置文件后，须执行 `systemctl reload sysmonitor`，新的配置在一个监控周期后生效。
* ALARM 值应该大于 RESUME值。
* 取三个监控周期的内存占用的平均值，来作为是否上报发生告警或恢复告警的依据。

### 异常日志

如果监控到内存告警，sysmonitor 获取 `/proc/meminfo`信息，打印到`/var/log/sysmonitor.log` 中，信息如下：

```sh
info|sysmonitor[127]: memory usage alarm: 90%
info|sysmonitor[127]:---------------show /proc/meminfo: ---------------
info|sysmonitor[127]:MemTotal: 3496388 kB
info|sysmonitor[127]:MemFree: 2738100 kB
info|sysmonitor[127]:MemAvailable: 2901888 kB
info|sysmonitor[127]:Buffers: 165064 kB
info|sysmonitor[127]:Cached: 282360 kB
info|sysmonitor[127]:SwapCached: 4492 kB
......
info|sysmonitor[127]:---------------show_memory_info end. ---------------
```

sysmonitor 有如下打印信息时，表示 sysmonitor 会调用 "echo m > /proc/sysrq-trigger" 命令导出内存分配的信息（可以在 /var/log/messages 中进行查看)。

```sh
info|sysmonitor[127]: sysrq show memory ifno in message。
```

告警恢复时，打印信息如下：

```sh
info|sysmonitor[127]: memory usage resume: 4.6%
```

## 进程数/线程数监控

### 简介

监控系统进程数目和线程数目，当进程总数或线程总数超出或低于阈值时，记录日志或上报告警。

### 配置文件说明

配置文件为 `/etc/sysmonitor/pscnt`。

```sh
# number of processes(include threads) when alarm occur
ALARM="1600"

# number of processes(include threads) when alarm resume
RESUME="1500"

# monitor period(second)
PERIOD="60"

# process count usage alarm percent
ALARM_RATIO="90"

# process count usage resume percent
RESUME_RATIO="80"

# print top process info with largest num of threads when threads alarm
# (range: 0-1024, default: 10, monitor for thread off:0)
SHOW_TOP_PROC_NUM="10"
```

| 配置项            | 配置项说明                                                   | 是否必配 | 默认值 |
| ----------------- | ------------------------------------------------------------ | -------- | ------ |
| ALARM             | 大于 0 的整数，进程总数告警阈值                              | 否       | 1600   |
| RESUME            | 大于等于0的整数，进程总数恢复阈值                            | 否       | 1500   |
| PERIOD            | 监控周期（秒），取值大于0                                    | 否       | 60     |
| ALARM\_RATIO       | 大于0小于等于100的值，可以为小数。进程使用率告警阈值         | 否       | 90     |
| RESUME\_RATIO      | 大于等于0小于100的值，可以为小数。进程使用率恢复阈值，必须必告警阈值小。 | 否       | 80     |
| SHOW\_TOP\_PROC\_NUM | 使用线程数量最新 TOP 的进程信息                              | 否       | 10     |

* 修改进程数监控的配置文件后，须执行`systemctl reload sysmonitor`，新的配置在一个监控周期后生效。
* ALARM 值应该大于 RESUME 值。
* 进程数告警产生阈值取 ALARM 值与 `/proc/sys/kernel/pid_max` 的 ALARM\_RATIO 中的最大值，告警恢复阈值取 RESUME 值与 `/proc/sys/kernel/pid_max` 的 RESUME\_RATIO 中的最大值。
* 线程数告警产生阈值取 ALARM 值与 `/proc/sys/kernel/threads-max` 的 ALARM\_RATIO 中的最大值，告警恢复阈值取 RESUME 值与 `/proc/sys/kernel/threads-max` 的 RESUME\_RATIO 中的最大值。
* SHOW\_TOP\_PROC\_NUM 的取值范围为0-1024，为0时，表示不启用线程监控；当设置值较大时，如 1024，在环境中产生线程告警，且告警阈值较高时，会有性能影响，建议设置为默认值 10 及更小值，若影响较大，建议设置为 0，不启动线程监控。
* 线程监控启动时，由 `/etc/sysconfig/sysmonitor` 中 `PSCNT_MONITOR` 项和 `/etc/sysmonitor/pscnt` 中 `SHOW_TOP_PROC_NUM` 项设置。
  * `PSCNT_MONITOR` 为 on，且 `SHOW_TOP_PROC_NUM` 设置为合法值时，为启动。
  * `PSCNT_MONITOR` 为 on， `SHOW_TOP_PROC_NUM` 为 0时，为关闭。
  * `PSCNT_MONITOR`为 off，为关闭。
* 进程数量告警时，增加打印系统句柄使用信息和内存信息（/proc/meminfo）。
* 线程数量告警时，会记录线程总数信息，TOP 进程信息，当前环境进程数量信息，系统句柄数信息，内存信息(/proc/meminfo)。
* 监控项监控周期到达前，若系统出现资源不足（如线程数超过系统最大线程数），则监控告警本身将由于资源受限无法正常运行，进而无法进行告警。

### 异常日志

如果监控到进程数告警，`/var/log/sysmonitor.log` 中打印信息示例如下：

```sh
info|sysmonitor[127]:---------------process count alarm start: ---------------
info|sysmonitor[127]: process count alarm:1657
info|sysmonitor[127]: process count alarm, show sys fd count: 2592
info|sysmonitor[127]: process count alarm, show mem info
info|sysmonitor[127]:---------------show /proc/meminfo: ---------------
info|sysmonitor[127]:MemTotal: 3496388 kB
info|sysmonitor[127]:MemFree: 2738100 kB
info|sysmonitor[127]:MemAvailable: 2901888 kB
info|sysmonitor[127]:Buffers: 165064 kB
info|sysmonitor[127]:Cached: 282360 kB
info|sysmonitor[127]:SwapCached: 4492 kB
......
info|sysmonitor[127]:---------------show_memory_info end. ---------------
info|sysmonitor[127]:---------------process count alarm end: ---------------

```

如果监控到进程数恢复告警，`/var/log/sysmonitor.log` 中打印信息示例如下：

```sh
info|sysmonitor[127]: process count resume: 1200
```

如果监控到线程数告警，`/var/log/sysmonitor.log` 中打印信息示例如下：

```sh
info|sysmonitor[127]:---------------threads count alarm start: ---------------
info|sysmonitor[127]:threads count alarm: 273
info|sysmonitor[127]:open threads most 10 processes is [top1:pid=1756900,openthreadsnum=13,cmd=/usr/bin/sysmonitor --daemon]
info|sysmonitor[127]:open threads most 10 processes is [top2:pid=3130,openthreadsnum=13,cmd=/usr/lib/gassproxy -D]
.....
info|sysmonitor[127]:---------------threads count alarm end. ---------------
```

## 系统句柄总数监控

### 简介

监控系统文件句柄（fd）数目，当系统文件句柄总数超过或低于阈值时，记录日志。

### 配置文件说明

配置文件为 `/etc/sysmonitor/sys_fd_conf`。

```sh
# system fd usage alarm percent
SYS_FD_ALARM="80"
# system fd usage alarm resume percent
SYS_FD_RESUME="70"
# monitor period (second)
SYS_FD_PERIOD="600"
```

配置项说明：

| 配置项        | 配置项说明                                                | 是否必配 | 默认值 |
| ------------- | --------------------------------------------------------- | -------- | ------ |
| SYS\_FD\_ALARM  | 大于0小于100的整数，fd总数与系统最大 fd数百分比的告警阈值 | 否       | 80%    |
| SYS\_FD\_RESUME | 大于0小于100的整数，fd 总数与系统最大fd数百分比的恢复阈值 | 否       | 70%    |
| SYS\_FD\_PERIOD | 监控周期（秒），取值为100~86400 之间的整数                | 否       | 600    |

* 修改fd 总数监控的配置文件后，须执行 `systemctl reload sysmonitor`，新的配置在一个监控周期后生效。
* `SYS_FD_ALARM` 值应该大于 `SYS_FD_RESUME` 值，当配置非法时，会使用默认值，并打印日志。

### 异常日志

如果监控到 fd 总数告警，在监控日志中打印告警。`/var/log/sysmonitor.log` 中打印信息示例如下：

```sh
info|sysmonitor[127]: sys fd count alarm: 259296
```

系统句柄使用告警时，会打印前三个使用句柄数最多的进程：

```sh
info|sysmonitor[127]:open fd most three processes is:[top1:pid=23233,openfdnum=5000,cmd=/home/openfile]
info|sysmonitor[127]:open fd most three processes is:[top2:pid=23267,openfdnum=5000,cmd=/home/openfile]
info|sysmonitor[127]:open fd most three processes is:[top3:pid=30144,openfdnum=5000,cmd=/home/openfile]
```

## 磁盘 inode 监控

### 简介

定期监控系统中挂载的磁盘分区 inode，当磁盘分区 inode 使用率大于或等于用户设置的告警阈值，记录磁盘 inode 告警。发生告警后，当磁盘分区 inode 使用率小于用户设置的告警恢复阈值，记录磁盘 inode 恢复告警。

### 配置文件说明

配置文件为 `/etc/sysmonitor/inode`。

```sh
DISK="/"
DISK="/var/log"
```

| 配置项 | 配置项说明                | 是否必配 | 默认值 |
| ------ | ------------------------- | -------- | ------ |
| DISK   | 磁盘挂载目录名            | 是       | 无     |
| ALARM  | 整数，磁盘 inode 告警阈值 | 否       | 90     |
| RESUME | 整数，磁盘 inode 恢复阈值 | 否       | 80     |

* 修改磁盘 inode 监控的配置文件后，须执行 `systemctl reload sysmonitor`，新的配置在一个监控周期后生效。
* 重复配置的挂载目录，最后一个配置项生效。
* ALARM 值应该大于 RESUME 值。
* 只能针对挂载点或被挂载的磁盘分区做监控。
* 在 CPU 和 IO 高压场景下，df 执行命令超时，会导致磁盘 inode 利用率获取不到。
* 当多个挂载点对应同一个磁盘分区，以挂载点为准来上报告警。

### 异常日志

如果监控到磁盘 inode 告警，`/var/log/sysmonitor.log`中打印信息示例如下：

```sh
info|sysmonitor[4570]:report disk inode alarm, /var/log used:90% alarm:90%
info|sysmonitor[4570]:report disk inode recovered, /var/log used:79% alarm:80%
```

## 本地磁盘 io 延时监控

### 简介

每5秒读取一次本地磁盘 io 延时数据，每五分钟对在该五分钟内60组数据进行统计，如果有多于30次（一半）的数据大于配置的最大 IO 延时数据，则记录该磁盘的 IO 延时过大日志。

### 配置文件说明

配置文件为 `/etc/sysmonitor/iodelay`。

```sh
DELAY_VALUE="500"
```

| 配置项      | 配置项说明           | 是否必配 | 默认值 |
| ----------- | -------------------- | -------- | ------ |
| DELAY\_VALUE | 磁盘 IO 延时的最大值 | 是       | 500    |

### 异常日志

如果监控到本地磁盘 IO 延时过大告警，`/var/log/sysmonitor.log` 中打印信息示例如下：

```sh
info|sysmonitor[127]:local disk sda IO delay is too large, I/O delay threshold is 70.
info|sysmonitor[127]:disk is sda, io delay data: 71 72 75 87 99 29 78 ......
```

如果监控到本地磁盘 IO 延时告警恢复，`/var/log/sysmonitor.log` 中打印信息示例如下：

```sh
info|sysmonitor[127]:local disk sda IO delay is normal, I/O delay threshold is 70.
info|sysmonitor[127]:disk is sda, io delay data: 11 22 35 8 9 29 38 ......
```

## 僵尸进程监控

### 简介

监控系统僵尸进程数量，大于告警阈值，记录告警日志。当系统僵尸进程数小于恢复阈值时，告警恢复。

### 配置文件说明

配置文件为`/etc/sysmonitor/zombie`。

```sh
# Ceiling zombie process counts of alarm
ALARM="500"

# Floor zombie process counts of resume
RESUME="400"

# Periodic (second)
PERIOD="600"
```

| 配置项 | 配置项说明                      | 是否必配 | 默认值 |
| ------ | ------------------------------- | -------- | ------ |
| ALARM  | 大于0，僵尸进程个数告警阈值     | 否       | 500    |
| RESUME | 大于等于0，僵尸进程个数恢复阈值 | 否       | 400    |
| PERIOD | 监控周期（秒），取值大于0       | 否       | 60     |

### 异常日志

如果监控到僵尸进程个数告警，`/var/log/sysmonitor.log`中打印信息如下:

```sh
info|sysmonitor[127]: zombie process count alarm: 600
info|sysmonitor[127]: zombie process count resume: 100
```

## 自定义监控

### 简介

用户可以自定义监控项，监控框架读取配置文件内容，解析配置文件各监控属性，在监控框架里调用用户要执行的监控动作。监控模块仅提供监控框架，不感知用户在监控的内容以及如何监控，不负责上报告警。

### 配置文件说明

配置文件位于`/etc/sysmonitor.d/`路径下,每个进程或模块对应一个配置文件。

```sh
MONITOR_SWITCH="on"
TYPE="periodic"
EXECSTART="/usr/sbin/iomonitor_daemon"
PERIOD="1800"
```

| 配置项         | 配置项说明                                                   | 是否必配              | 默认值 |
| -------------- | ------------------------------------------------------------ | --------------------- | ------ |
| MONITOR\_SWITCH | 监控开关                                                     | 否                    | off    |
| TYPE           | 自定义监控项的类型daemon：后台运行periodic：周期运行 | 是                    | 无     |
| EXECSTART      | 执行监控命令                                                 | 是                    | 无     |
| ENVIROMENTFILE | 环境变量存放文件                                             | 否                    | 无     |
| PERIOD         | 若 type 为 periodic 类型，此为必配项，为自定义监控的周期，取值为大于0的整数 | periodic 类型为必配项 | 无     |

* 配置文件名称，环境变量文件名称，加上绝对路径总长度不能超过127个字符。环境变量文件必须为绝对路径和实际路径，不能是软链接路径。
* EXECSTART项的命令总长度不能超过159个字符，关键字段配置不能有空格。
* 周期性监控的执行命令不能超时，否则对自定义监控框架产生影响。
* 目前支持配置的环境变量最多为256个。
* daemon 类型的自定义监控每间隔10s会统一查询是否有 reload 命令下发，或者是否有 daemon 进程异常退出；如果有reload 命令下发，需要等待 10s 后才会重新加载新的配置，如果有 daemon 进程异常退出，需要等待 10s才会重新拉起。
* ENVIROMENTFLE 对应的文件中的内容发生变化，如新增环境变量，或环境变量的值发生变化，需要重启 sysmonitor 服务，新的环境变量才能生效。
* `/etc/sysmonitor.d/`目录下的配置文件权限建议为 600， EXECSTART 项中若只配置了执行文件，则执行文件的权限建议为 550。
* daemon 进程异常退出后，sysmonitor 会重新加载该 daemon进程的配置文件。

### 异常日志

如果 daemon 类型监控项异常退出，/var/log/sysmonitor.log 中会有如下记录：

```sh
info|sysmonitor[127]: custom daemon monitor: child process[11609] name unetwork_alarm exit code[127],[1] times.
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/syssentry/syssentry_introduction.md
---
# sysSentry简介

sysSentry主要提供故障巡检框架，该框架通过提供统一的北向故障上报接口以及南向提供支持不同巡检/诊断能力的插件，支持对系统中CPU、内存、磁盘、NPU等硬件故障进行巡检和诊断。

![输入图片说明](figures/sysSentry.png)

sysSentry功能设计如下：

1. 统一告警/事件通知服务：通过提供一个统一的告警服务，接收各个插件上报的故障信息，并由该通知服务进行统一转发，各个业务订阅服务可以根据需要进行不同故障的消息订阅。
2. 统一日志服务：通过提供统一的日志服务，支持各个插件的故障信息进行汇总记录，提升问题定位效率。
3. 故障诊断/巡检框架：该框架支持以插件化的方式进行各项巡检任务以及诊断任务的开发和配置，不同插件支持独立启动、停止、状态查询、结果查询以及启动方式设置，并且支持C/C++、Python、Shell等不同编程语言的插件。
4. 轻量级数据采集服务：该服务支持通过内核接口、BIOS、BMC等接口，查询硬件的各个状态信息，供各个插件进行分析和使用，并且支持适配底层不同的架构、版本以及数据采集服务。

---

---
url: /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/overview.md
---
# System Container

System containers are used for heavyweight applications and cloud-based services in scenarios with re-computing, high performance, and high concurrency. Compared with the VM technology, system containers can directly inherit physical machine features and has better performance and less overhead. In addition, system containers can be allocated more computing units of limited resources, reducing costs. Therefore, system containers can be used to build differentiated product competitiveness and provide computing unit instances with higher computing density, lower price, and better performance.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/system_resource_management.md
---
# System Resource Management

The  **libvirt**  command manages VM system resources, such as vCPU and virtual memory resources.

Before you start:

* Ensure that the libvirtd daemon is running on the host.
* Run the  **virsh list --all**  command to check that the VM has been defined.

## Managing vCPU

### CPU Shares

#### Overview

In a virtualization environment, multiple VMs on the same host compete for physical CPUs. To prevent some VMs from occupying too many physical CPU resources and affecting the performance of other VMs on the same host, you need to balance the vCPU scheduling of VMs to prevent excessive competition for physical CPUs.

The CPU share indicates the total capability of a VM to compete for physical CPU computing resources. You can set  **cpu\_shares**  to specify the VM capacity to preempt physical CPU resources. The value of  **cpu\_shares**  is a relative value without a unit. The CPU computing resources obtained by a VM are the available computing resources of physical CPUs (excluding reserved CPUs) allocated to VMs based on the CPU shares. Adjust the CPU shares to ensure the service quality of VM CPU computing resources.

#### Procedure

Change the value of  **cpu\_shares**  allocated to the VM to balance the scheduling between vCPUs.

* Check the current CPU share of the VM.

  ```shell
  # virsh schedinfo <VMInstance>
  Scheduler      : posix
  cpu_shares     : 1024
  vcpu_period    : 100000
  vcpu_quota     : -1
  emulator_period: 100000
  emulator_quota : -1
  global_period  : 100000
  global_quota   : -1
  iothread_period: 100000
  iothread_quota : -1
  ```

* Online modification: Run the  **virsh schedinfo**  command with the  **--live**  parameter to modify the CPU share of a running VM.

  ```shell
  # virsh schedinfo <VMInstance> --live cpu_shares=<number>
  ```

  For example, to change the CPU share of the running  *openEulerVM*  from  **1024**  to  **2048**, run the following commands:

  ```shell
  # virsh schedinfo openEulerVM --live cpu_shares=2048
  Scheduler      : posix
  cpu_shares     : 2048
  vcpu_period    : 100000
  vcpu_quota     : -1
  emulator_period: 100000
  emulator_quota : -1
  global_period  : 100000
  global_quota   : -1
  iothread_period: 100000
  iothread_quota : -1
  ```

  The modification of the  **cpu\_shares**  value takes effect immediately. The running time of the  *openEulerVM*  is twice the original running time. However, the modification will become invalid after the VM is shut down and restarted.

* Permanent modification: Run the  **virsh schedinfo**  command with the  **--config**  parameter to change the CPU share of the VM in the libvirt internal configuration.

  ```shell
  # virsh schedinfo <VMInstance> --config cpu_shares=<number>
  ```

  For example, run the following command to change the CPU share of  *openEulerVM*  from  **1024**  to  **2048**:

  ```shell
  # virsh schedinfo openEulerVM --config cpu_shares=2048
  Scheduler      : posix
  cpu_shares     : 2048
  vcpu_period    : 0
  vcpu_quota     : 0
  emulator_period: 0
  emulator_quota : 0
  global_period  : 0
  global_quota   : 0
  iothread_period: 0
  iothread_quota : 0
  ```

  The modification on  **cpu\_shares**  does not take effect immediately. Instead, the modification takes effect after the  *openEulerVM*  is started next time and takes effect permanently. The running time of the  *openEulerVM*  is twice that of the original VM.

### Binding the QEMU Process to a Physical CPU

#### Overview

You can bind the QEMU main process to a specific physical CPU range, ensuring that VMs running different services do not interfere with adjacent VMs. For example, in a typical cloud computing scenario, multiple VMs run on one physical machine, and they carry diversified services, causing different degrees of resource occupation. To avoid interference of a VM with dense-storage I/O to an adjacent VM, storage processes that process I/O of different VMs need to be completely isolated. The QEMU main process handles frontend and backend services. Therefore, isolation needs to be implemented.

#### Procedure

Run the  **virsh emulatorpin**  command to bind the QEMU main process to a physical CPU.

* Check the range of the physical CPU bound to the QEMU process:

  ```shell
  # virsh emulatorpin openEulerVM
  emulator: CPU Affinity
  ----------------------------------
         *: 0-63
  ```

  This indicates that the QEMU main process corresponding to VM  **openEulerVM**  can be scheduled on all physical CPUs of the host.

* Online binding: Run the  **vcpu emulatorpin**  command with the  **--live**  parameter to modify the binding relationship between the QEMU process and the running VM.

  ```shell
  # virsh emulatorpin openEulerVM --live 2-3

  # virsh emulatorpin openEulerVM
  emulator: CPU Affinity
  ----------------------------------
         *: 2-3
  ```

  The preceding commands bind the QEMU process corresponding to VM  **openEulerVM**  to physical CPUs  **2**  and  **3**. That is, the QEMU process is scheduled only on the two physical CPUs. The binding relationship takes effect immediately but becomes invalid after the VM is shut down and restarted.

* Permanent binding: Run the  **virsh emulatorpin**  command with the  **--config**  parameter to modify the binding relationship between the VM and the QEMU process in the libvirt internal configuration.

  ```shell
  # virsh emulatorpin openEulerVM --config 0-3,^1

  # virsh emulatorpin euler
  emulator: CPU Affinity
  ----------------------------------
         *: 0,2-3
  ```

  The preceding commands bind the QEMU process corresponding to VM  **openEulerVM**  to physical CPUs  **0**,  **2**  and  **3**. That is, the QEMU process is scheduled only on the three physical CPUs. The modification of the binding relationship does not take effect immediately. Instead, the modification takes effect after the next startup of the VM and takes effect permanently.

### Adjusting the vCPU Binding Relationship

#### Overview

The vCPU of a VM is bound to a physical CPU. That is, the vCPU is scheduled only on the bound physical CPU to improve VM performance in specific scenarios. For example, in a NUMA system, vCPUs are bound to the same NUMA node to prevent cross-node memory access and VM performance deterioration. If the vCPU is not bound, by default, the vCPU can be scheduled on any physical CPU. The specific binding policy is determined by the user.

#### Procedure

Run the  **virsh vcpupin**  command to adjust the binding relationship between vCPUs and physical CPUs.

* View the vCPU binding information of the VM.

  ```shell
   # virsh vcpupin openEulerVM
   VCPU   CPU Affinity
  ----------------------
   0      0-63
   1      0-63
   2      0-63
   3      0-63
  ```

  This indicates that all vCPUs of VM  **openEulerVM**  can be scheduled on all physical CPUs of the host.

* Online adjustment: Run the  **vcpu vcpupin**  command with the  **--live**  parameter to modify the vCPU binding relationship of a running VM.

  ```shell
   # virsh vcpupin openEulerVM  --live 0 2-3

   # virsh vcpupin euler
   VCPU   CPU Affinity
  ----------------------
   0      2-3
   1      0-63
   2      0-63
   3      0-63
  ```

  The preceding commands bind vCPU  **0**  of VM  **openEulerVM**  to pCPU  **2**  and pCPU  **3**. That is, vCPU  **0**  is scheduled only on the two physical CPUs. The binding relationship takes effect immediately but becomes invalid after the VM is shut down and restarted.

* Permanent adjustment: Run the  **virsh vcpupin**  command with the  **--config**  parameter to modify the vCPU binding relationship of the VM in the libvirt internal configuration.

  ```shell
   # virsh vcpupin openEulerVM --config 0 0-3,^1

   # virsh vcpupin openEulerVM
   VCPU   CPU Affinity
  ----------------------
   0      0,2-3
   1      0-63
   2      0-63
   3      0-63
  ```

  The preceding commands bind vCPU  **0**  of VM  **openEulerVM**  to physical CPUs  **0**,  **2**, and  **3**. That is, vCPU  **0**  is scheduled only on the three physical CPUs. The modification of the binding relationship does not take effect immediately. Instead, the modification takes effect after the next startup of the VM and takes effect permanently.

### CPU Hot Add

#### Overview

This feature allows users to hot add CPUs to a running VM without affecting its normal running. When the internal service pressure of a VM keeps increasing, all CPUs will be overloaded. To improve the computing capability of the VM, you can use the CPU hot add function to increase the number of CPUs on the VM without stopping it.

#### Constraints

* For processors using the AArch64 architecture, the specified VM chipset type (machine) needs to be virt-4.1 or a later version when a VM is created. For processors using the x86\_64 architecture, the specified VM chipset type (machine) needs to be pc-i440fx-1.5 or a later version when a VM is created.
* When configuring Guest NUMA, you need to configure the vCPUs that belong to the same socket in the same vNode. Otherwise, the VM may be soft locked up after the CPU is hot added, which may cause the VM panic.
* VMs do not support CPU hot add during migration, hibernation, wake-up, or snapshot.
* Whether the hot added CPU can automatically go online depends on the VM OS logic rather than the virtualization layer.
* CPU hot add is restricted by the maximum number of CPUs supported by the Hypervisor and GuestOS.
* When a VM is being started, stopped, or restarted, the hot added CPU may become invalid. However, the hot added CPU takes effect after the VM is restarted.
* During VM CPU hot add, if the number of added CPUs is not an integer multiple of the number of cores in the VM CPU topology configuration item, the CPU topology displayed in the VM may be disordered. You are advised to add CPUs whose number is an integer multiple of the number of cores each time.
* If the hot added CPU needs to take effect online and is still valid after the VM is restarted, the --config and --live options need to be transferred to the virsh setvcpus API to persist the hot added CPU.

#### Procedure

**VM XML Configuration**

1. To use the CPU hot add function, configure the number of CPUs, the maximum number of CPUs supported by the VM, and the VM chipset type when creating the VM. (For the AArch64 architecture, the virt-4.1 or a later version is required. For the x86\_64 architecture, the pc-i440fx-1.5 or later version is required. The AArch64 VM is used as an example. The configuration template is as follows:

   ```xml
   <domain type='kvm'>
       ...
       <vcpu placement='static' current='m'>n</vcpu>
       <os>
           <type arch='aarch64' machine='virt-4.1'>hvm</type>
       </os>
       ...
   <domain>
   ```

   > \[!NOTE] **Note**
   >
   > * The value of placement must be static.
   > * m indicates the current number of CPUs on the VM, that is, the default number of CPUs after the VM is started. n indicates the maximum number of CPUs that can be hot added to a VM. The value cannot exceed the maximum CPU specifications supported by the Hypervisor or GuestOS. n is greater than or equal to m.

   For example, if the current number of CPUs of a VM is 4 and the maximum number of hot added CPUs is 64, the XML configuration is as follows:

   ```xml
   <domain type='kvm'>
   ...
       <vcpu placement='static' current='4'>64</vcpu>
       <os>
           <type arch='aarch64' machine='virt-4.1'>hvm</type>
       </os>
   ...
   ```

**Hot Adding and Bringing CPUs Online**

1. If the hot added CPU needs to be automatically brought online, create the udev rules file **/etc/udev/rules.d/99-hotplug-cpu.rules** in the VM as user root and define the udev rules in the file. The following is an example:

   ```text
   ### automatically online hot-plugged cpu
   ACTION=="add", SUBSYSTEM=="cpu", ATTR{online}="1"
   ```

   > \[!NOTE] **Note**\
   > If you do not use the udev rules, you can use the root permission to manually bring the hot added CPU online by running the following commands:
   >
   > ```shell
   > for i in `grep -l 0 /sys/devices/system/cpu/cpu*/online`
   > do
   >    echo 1 > $i
   > done
   > ```

2. Use the virsh tool to hot add CPUs to the VM. For example, to set the number of CPUs after hot adding to 6 on the VM named openEulerVM and make the hot add take effect online, run the following command:

   ```shell
   virsh setvcpus openEulerVM 6 --live
   ```

   > \[!NOTE] **Note**\
   > The format for running the virsh setvcpus command to hot add a VM CPU is as follows:
   >
   > ```shell
   > virsh setvcpus <domain> <count> [--config] [--live]
   > ```
   >
   > * domain: Parameter, which is mandatory. Specifies the name of a VM.
   > * count: Parameter, which is mandatory. Specifies the number of target CPUs, that is, the number of CPUs after hot adding.
   > * \--config: Option, which is optional. This parameter is still valid when the VM is restarted.
   > * \--live: Option, which is optional. The configuration takes effect online.

## Managing Virtual Memory

### Introduction to NUMA

Traditional multi-core computing uses the symmetric multi-processor (SMP) mode. Multiple processors are connected to a centralized memory and I/O bus. All processors can access only the same physical memory. Therefore, the SMP system is also referred to as a uniform memory access (UMA) system. Uniformity means that a processor can only maintain or share a unique value for each data record in memory at any time. Obviously, the disadvantage of SMP is its limited scalability, because when the memory and the I/O interface are saturated, adding a processor cannot obtain higher performance.

The non-uniform memory access architecture (NUMA) is a distributed memory access mode. In this mode, a processor can access different memory addresses at the same time, which greatly improves concurrency. With this feature, a processor is divided into multiple nodes, each of which is allocated a piece of local memory space. The processors of all nodes can access all physical memories, but the time required for accessing the memory on the local node is much shorter than that on a remote node.

### Configuring Host NUMA

To improve VM performance, you can specify NUMA nodes for a VM using the VM XML configuration file before the VM is started so that the VM memory is allocated to the specified NUMA nodes. This feature is usually used together with the vCPU to prevent the vCPU from remotely accessing the memory.

#### Procedure

* Check the NUMA topology of the host.

  ```shell
   # numactl -H
  available: 4 nodes (0-3)
  node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
  node 0 size: 31571 MB
  node 0 free: 17095 MB
  node 1 cpus: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
  node 1 size: 32190 MB
  node 1 free: 28057 MB
  node 2 cpus: 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
  node 2 size: 32190 MB
  node 2 free: 10562 MB
  node 3 cpus: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
  node 3 size: 32188 MB
  node 3 free: 272 MB
  node distances:
  node   0   1   2   3
    0:  10  15  20  20
    1:  15  10  20  20
    2:  20  20  10  15
    3:  20  20  15  10
  ```

* Add the  **numatune**  field to the VM XML configuration file to create and start the VM. For example, to allocate NUMA node 0 on the host to the VM, configure parameters as follows:

  ```xml
    <numatune>
      <memory mode="strict" nodeset="0"/>
    </numatune>
  ```

  If the vCPU of the VM is bound to the physical CPU of  **node 0**, the performance deterioration caused by the vCPU accessing the remote memory can be avoided.

  > \[!NOTE] **NOTE:**
  >
  > * The sum of memory allocated to the VM cannot exceed the remaining available memory of the NUMA node. Otherwise, the VM may fail to start.
  > * You are advised to bind the VM memory and vCPU to the same NUMA node to avoid the performance deterioration caused by vCPU access to the remote memory. For example, bind the vCPU to NUMA node 0 as well.

### Configuring Guest NUMA

Many service software running on VMs is optimized for the NUMA architecture, especially for large-scale VMs. openEuler provides the Guest NUMA feature to display the NUMA topology in VMs. You can identify the structure to optimize the performance of service software and ensure better service running.

When configuring guest NUMA, you can specify the location of vNode memory on the host to implement memory block binding and vCPU binding so that the vCPU and memory on the vNode are on the same physical NUMA node.

#### Procedure

After Guest NUMA is configured in the VM XML configuration file, you can view the NUMA topology on the VM.  **\<numa>**  is mandatory for Guest NUMA.

```xml
  <cputune>
    <vcpupin vcpu='0' cpuset='0-3'/>
    <vcpupin vcpu='1' cpuset='0-3'/>
    <vcpupin vcpu='2' cpuset='16-19'/>
    <vcpupin vcpu='3' cpuset='16-19'/>
  </cputune>
  <numatune>
    <memnode cellid="0" mode="strict" nodeset="0"/>
    <memnode cellid="1" mode="strict" nodeset="1"/>
  </numatune>
  [...]
  <cpu>
    <numa>
      <cell id='0' cpus='0-1' memory='2097152'/>
      <cell id='1' cpus='2-3' memory='2097152'/>
    </numa>
  </cpu>
```

> \[!NOTE] **NOTE:**
>
> * **\<numa>**  provides the NUMA topology function for VMs.  **cell id**  indicates the vNode ID,  **cpus**  indicates the vCPU ID, and  **memory**  indicates the memory size on the vNode.
> * If you want to use Guest NUMA to provide better performance, configure <**numatune>**  and  **\<cputune>**  so that the vCPU and memory are distributed on the same physical NUMA node.
> * **cellid**  in  **\<numatune>**  corresponds to  **cell id**  in  **\<numa>**.  **mode**  can be set to  **strict**  (apply for memory from a specified node strictly. If the memory is insufficient, the application fails.),  **preferred**  (apply for memory from a node first. If the memory is insufficient, apply for memory from another node), or  **interleave**  (apply for memory from a specified node in cross mode).;  **nodeset**  indicates the specified physical NUMA node.
> * In  **\<cputune>**, you need to bind the vCPU in the same  **cell id**  to the physical NUMA node that is the same as the  **memnode**.

### Memory Hot Add

#### Overview

In virtualization scenarios, the memory, CPU, and external devices of VMs are simulated by software. Therefore, the memory can be adjusted online for VMs at the virtualization bottom layer. In the current openEuler version, memory can be added to a VM online. If the physical memory of a VM is insufficient and the VM cannot be shut down, you can use this feature to add physical memory resources to the VM.

#### Constraints

* For processors using the AArch64 architecture, the specified VM chipset type (machine) needs to be virt-4.1 or a later version when a VM is created.For processors using the x86 architecture, the specified VM chipset type (machine) needs to be a later version than pc-i440fx-1.5 when a VM is created.
* Guest NUMA on which the memory hot add feature depends needs to be configured on the VM. Otherwise, the memory hot add process cannot be completed.
* When hot adding memory, you need to specify the ID of Guest NUMA node to which the new memory belongs. Otherwise, the memory hot add fails.
* The VM kernel should support memory hot add. Otherwise, the VM cannot identify the newly added memory or the memory cannot be brought online.
* For a VM that uses hugepages, the capacity of the hot added memory should be an integral multiple of hugepagesz. Otherwise, the hot add fails.
* The hot added memory size should be an integral multiple of the Guest physical memory block size (block\_size\_bytes). Otherwise, the VM cannot go online. The value of block\_size\_bytes can be obtained using the lsmem command in Guest.
* After n pieces of virtio-net NICs are configured, the maximum number of hot add times is set to min{max\_slot, 64 - n} to reserve slots for NICs.
* The vhost-user device and the memory hot add feature are mutually exclusive. A VM configured with the vhost-user device does not support memory hot add. After the memory is hot added to a VM, the vhost-user device cannot be hot added.
* If the VM OS is Linux, ensure that the initial memory is greater than or equal to 4 GB.
* If the VM OS is Windows, the first hot added memory needs to be specified to Guest NUMA node0. Otherwise, the hot added memory cannot be identified by the VM.
* In passthrough scenarios, memory needs to be allocated in advance. Therefore, it is normal that the startup and hot add of memory are slower than those of common VMs (especially large-specification VMs).
* It is recommended that the ratio of the available memory to the hot added memory be at least 1:32. That is, at least 1 GB available memory is required for the VM with 32 GB hot added memory. If the ratio is less than 1:32, the VM may be suspended.
* Whether the hot added memory can automatically go online depends on the VM OS logic. You can manually bring the memory online or configure the udev rules to automatically bring the memory online.

#### Procedure

**VM XML Configuration**

1. To use the memory hot add function, configure the maximum hot add memory size and reserved slot number, and configure the Guest NUMA topology when creating a VM.

   For example, run the following command to configure 32 GB initial memory for a VM, reserve 256 slots, set the memory upper limit to 1 TB, and configure two NUMA nodes:

   ```xml
   <domain type='kvm'>
       <memory unit='GiB'>32</memory>
       <maxMemory slots='256' unit='GiB'>1024</maxMemory>
       <cpu mode='host-passthrough' check='none'>
           <topology sockets='2' cores='2' threads='1'/>
           <numa>
             <cell id='0' cpus='0-1' memory='16' unit='GiB'/>
             <cell id='1' cpus='2-3' memory='16' unit='GiB'/>
           </numa>
        </cpu>
       ....
   ```

> \[!NOTE] **Note**\
> In the preceding information,
> the value of slots in the maxMemory field indicates the reserved memory slots. The maximum value is 256.
> maxMemory indicates the maximum physical memory supported by the VM.
> For details about how to configure Guest NUMA, see "Configuring Guest NUMA."

**Hot Adding and Bringing Memory Online**

1. If the hot added memory needs to be automatically brought online, create the udev rules file /etc/udev/rules.d/99-hotplug-memory.rules in the VM as user root and define the udev rules in the file. The following is an example:

   ```text
   ### automatically online hot-plugged memory
   ACTION=="add", SUBSYSTEM=="memory", ATTR{state}="online"
   ```

2. Create a memory description XML file based on the size of the memory to be hot added and the Guest NUMA node of the VM.

   For example, to hot add 1 GB memory to NUMA node0, run the following command:

   ```xml
   <memory model='dimm'>
     <target>
     <size unit='MiB'>1024</size>
     <node>0</node>
     </target>
   </memory>
   ```

3. Run the virsh attach-device command to hot add memory to the VM. In the command, openEulerVM indicates the VM name, memory.xml indicates the description file of the hot added memory, and --live indicates that the hot added memory takes effect online. You can also run the --config command to persist the hot added memory to the VM XML file.

   ```shell
   ### virsh attach-device openEulerVM memory.xml --live
   ```

   > \[!NOTE] **Note**\
   > If you do not use the udev rules, you can use the root permission to manually bring the hot added memory online by running the following command:
   >
   > ```shell
   > for i in `grep -l offline /sys/devices/system/memory/memory*/state`
   > do
   >    echo online > $i
   > done
   > ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/performance/system_resource/system_resources_and_performance.md
---
# System Resources and Performance

## CPU

### Basic Concepts

A central processing unit (CPU) is one of main devices of a computer, and a function of the CPU is to interpret computer instructions and process data in computer software.

1. Physical core: an actual CPU core that can be seen. It has independent circuit components and L1 and L2 caches and can independently execute instructions. A CPU can have multiple physical cores.
2. Logical core: a core that exists at the logical layer in the same physical core. Generally, a physical core corresponds to a thread. However, if hyper-threading is enabled and the number of hyper-threads is *n*, a physical core can be divided into *n* logical cores.

You can run the **lscpu** command to check the number of CPUs on the server, the number of physical cores in each CPU, and the number of logical cores in each CPU.

### Demarcation and Locating

### Common CPU Performance Analysis Tools

1. **uptime**: prints the average system load. The last three numbers indicate the average load within the last one, five, and fifteen minutes.

   If the average load is greater than the number of CPUs, the CPUs are insufficient to serve threads and some threads are waiting. If the average load is less than the number of CPUs, there are remaining CPUs.

   ![en-us\_image\_0000001384808269](./images/en-us_image_0000001384808269.png)

2. **vmstat**: dynamically monitors the usage of system resources and checks which phase occupies the most system resources.

   You can run the **vmstat -h** command to view command parameters.
   Example:

   ```shell
   # Monitor the status and update the status every second.
   vmstat 1
   ```

   ![](./images/en-us_image_0000001385585749.png)
   The fields in the command output are described as follows:

   |Field|Description|
   |--|--|
   |procs|Process information.|
   |memory|Memory information.|
   |swap|Swap partition information.|
   |io|Drive read/write information.|
   |system|System information.|
   |cpu|CPU information.  **-us**: percentage of the CPU computing time consumed by non-kernel processes.  **-sy**: percentage of the CPU computing time consumed by kernel processes.  **-id**: idle.  **-wa**: percentage of CPU resources consumed by waiting for I/Os.  **-st**: percentage of CPUs stolen by VMs.|

3. **sar**: analyzes system performance, observes current activities and configurations, and archives and reports historical statistics.
   Example:

   ```shell
   # Check the overall CPU load of the system. Collect the statistics every 3 seconds for five times.
   sar -u 3 5
   ```

   ```text
   [root@openEuler ~]# sar -u 3 5
   Linux 5.10.0-153.12.0.92.oe2203SP3.aarch64 (openEuler) 05/20/2023      _aarch64_      (4 CPU)

   04:38:27 PM     CPU      %user     %nice     %system     %iowait     %steal     %idle
   04:38:30 PM     all       0.00      0.00        0.00        0.00       0.00    100.00
   04:38:33 PM     all       0.00      0.00        0.33        0.00       0.00     99.67
   04:38:36 PM     all       0.00      0.00        0.00        0.00       0.00    100.00
   04:38:39 PM     all       0.08      0.00        0.00        0.00       0.00     99.92
   04:38:42 PM     all       0.00      0.00        0.08        0.00       0.00     99.92
   Average:        all       0.02      0.00        0.08        0.00       0.00     99.90
   ```

   The fields in the command output are described as follows:

   |Field|Description|
   |--|--|
   |%user|Percentage of the CPU time consumed in user mode.|
   |%nice|Percentage of the CPU time consumed by a process whose scheduling priority is changed through **nice** in user mode.|
   |%system|Percentage of the CPU time consumed in system mode.|
   |%iowait|Percentage of the time consumed by the CPU to wait for drive I/Os in idle state.|
   |%steal|Percentage of the time used for waiting for other virtual CPU computing by using virtualization technologies of the OS.|
   |%idle|Percentage of CPU idle time.|

4. **ps**: displays running processes.

   ```shell
   # View all processes in the system, and view the PIDs and priorities of the their parent processes.
   ps -le
   ```

   ![en-us\_image\_0000001337039920](./images/en-us_image_0000001337039920.png)

   ```shell
   # View the processes generated by the current shell.
   ps -l
   ```

   ![en-us\_image\_0000001385611905](./images/en-us_image_0000001385611905.png)

5. **top**: dynamically and continuously monitors the running status of processes and displays the processes that consume the most CPU resources.

   ```shell
   top
   ```

   ![en-us\_image\_0000001335457246](./images/en-us_image_0000001335457246.png)

## Memory

### Basic Concepts

The memory is an important component of a computer, and is used to temporarily store operation data in the CPU and data exchanged with an external memory such as hardware. In particular, a non-uniform memory access architecture (NUMA) is a memory architecture designed for a multiprocessor computer. The memory access time depends on the location of the memory relative to the processor. In NUMA mode, a processor accesses the local memory faster than the non-local memory (the memory is located in another processor or shared between processors).

### Demarcation and Locating

### Common Memory Analysis Tools and Methods

1. **free**: displays the system memory status.
   Example:

   ```shell
   # Display the system memory status in MB.
   free -m
   ```

   ![en-us\_image\_0000001386699925](./images/en-us_image_0000001386699925.png)

   The fields in the command output are described as follows:

   |Field|Description|
   |--|--|
   |total|Total memory size.|
   |used|Used memory.|
   |free|Free memory.|
   |shared|Total memory shared by multiple processes.|
   |buff/cache|Total number of buffers and caches.|
   |available|Estimated available memory to start a new application without swapping.|

2. **vmstat**: dynamically monitors the system memory and views the system memory usage.

   Example:

   ```shell
   # Monitor the system memory and display active and inactive memory.
   vmstat -a
   ```

   ![en-us\_image\_0000001388972645](./images/en-us_image_0000001388972645.png)

   In the command output, the field related to the memory is described as follows:

   |Field|Description|
   |--|--|
   |memory|Memory information.  **-swpd**: usage of the virtual memory, in KB.  **-free**: free memory capacity, in KB.  **-inact**: inactive memory capacity, in KB.  **-active**: active memory capacity, in KB.|

3. **sar**: monitors the memory usage of the system.

   Example:

   ```shell
   # Monitor the memory usage in the sampling period in the system. Collect the statistics every two seconds for three times.
   sar -r 2 3
   ```

   ```text
   [root@openEuler ~]# sar -r 2 3
   Linux 5.10.0-153.12.0.92.oe2203SP3.aarch64 (openEuler) 05/20/2023      _aarch64_      (4 CPU)

   04:56:08 PM     kbmemfree     kbavail     kbmemused     %memused     kbbuffers     kbcached     kbcommit     %commint     kbactive     kbinact     kbdirty
   04:56:10 PM        324264     2250588        188320         6.98        143160      1772412       787944        11.52       474668     1588704           0
   04:56:12 PM        324264     2250588        188320         6.98        143160      1772412       787944        11.52       474668     1588704           0
   04:56:14 PM        324296     2250620        188388         6.98        143160      1772412       787944        11.52       474668     1588772           0
   Average:           324275     2250599        188309         6.98        143160      1772412       787944        11.52       474668     1588727           0
   ```

   The fields in the command output are described as follows:

   |Field|Description|
   |--|--|
   |kbmemfree|Unused memory space.|
   |kbmemused|Used memory space.|
   |%memused|Percentage of the used space.|
   |kbbuffers|Amount of data stored in the buffer.|
   |kbcached|Data access volume in all domains of the system.|

4. **numactl**: displays the NUMA node configuration and status.

   Example:

   ```shell
   # Check the current NUMA configuration.
   numactl -H
   ```

   ![en-us\_image\_0000001337000118](./images/en-us_image_0000001337000118.png)

   The server contains one NUMA node. The NUMA node that contains four cores and 6 GB memory.
   The command also displays the distance between NUMA nodes. The further the distance, the higher the latency of cross-node memory accesses, which should be avoided as much as possible.

   **numastat**: displays NUMA node status.

   ```shell
   # Check the NUMA node status.
   numastat
   ```

   ![en-us\_image\_0000001337172594](./images/en-us_image_0000001337172594.png)

   The fields in the **numastat** command output are described as follows:

   |Field|Description|
   |--|--|
   |numa\_hit|Number of times that the CPU core accesses the local memory on a node.|
   |numa\_miss|Number of times that the core of a node accesses the memory of other nodes.|
   |numa\_foreign|Number of pages that were allocated to the local node but moved to other nodes. Each numa\_foreign corresponds to a numa\_miss event.|
   |interleave\_hit|Number of pages of the interleave policy that are allocated to this node.|
   |local\_node|Size of memory that was allocated to this node by processes on this node.|
   |other\_node|Size of memory that was allocated to other nodes by processes on this node.|

## I/O

### Basic Concepts

I/O indicates input/output. Input refers to the operation of receiving signals or data by the system, and output refers to the operation of sending signals or data from the system. For a combination of CPU and main memory, any information incoming to or outgoing from the CPU/memory combination is considered as I/Os.

### Demarcation and Locating

### Common I/O Performance Analysis Tools

1. **iostat**: reports statistics about all online drives.

   Example:

   ```shell
   # Display the drive information in KB. Collect the statistics every 100 seconds until you press Ctrl+C.
   iostat -d -k -x 100
   # Display the drive information in KB. Collect the statistics every second and for 100 seconds.
   iostat -d -k -x 1 100
   ```

   ```text
   [root@openEuler ~]# iostat -d -k -x 1 100
   Linux 5.10.0-153.12.0.92.oe2203SP3.aarch64 (openEuler) 05/20/2023      _aarch64_      (4 CPU)

   Device  r/s  rkB/s  rrqm/s  %rrqm  r_await  rareq-sz  w/s  wKB/s  wrqm/s  %wrqm  w_await  wareq-sz  d/s  dKB/s  drqm/s  %drqm  d_await  dareq-sz  f/s  f_await  aqu-sz  %util
   dm-0   0.00  0.00     0.00   0.00     0.00     0.00  0.00   0.00    0.00   0.00     0.00     0.00  0.00   0.00    0.00   0.00     0.00    0.00   0.00     0.00    0.00   0.00
   dm-1   0.00  0.00     0.00   0.00     0.00     0.00  0.00   0.00    0.00   0.00     0.00     0.00  0.00   0.00    0.00   0.00     0.00    0.00   0.00     0.00    0.00   0.00
   vda    0.00  0.00     0.00   0.00     0.00     0.00  0.00   0.00    0.00   0.00     0.00     0.00  0.00   0.00    0.00   0.00     0.00    0.00   0.00     0.00    0.00   0.00
   ```

   The fields in the command output are described as follows:

   |Field|Description|
   |--|--|
   |Device|Name of the monitoring device.|
   |r/s|Number of read requests completed by the device per second (after combination).|
   |rKB/s|Number of KBs read from the drive per second.|
   |rrqm/s|Number of read operations merged into the request queue per second.|
   |%rrqm|Percentage of read requests merged before they are sent to the device.|
   |r\_await|Average time consumed by each read request.|
   |rareq-sz|Average size of read requests sent to the device, in KB.|
   |w/s|Number of write requests completed by the device per second (after combination).|
   |wKB/s|Number of KBs written to the drive per second.|
   |wrqm/s|Number of write operations merged into the request queue per second.|
   |%wrqm|Percentage of write requests merged before they are sent to the device.|
   |w\_await|Average time consumed by each write request.|
   |wareq-sz|Average size of write requests sent to the device, in KB.|
   |d/s|Number of discard requests processed by the device per second.|
   |dKB/s|Number of sectors (KB) discarded by the device per second.|
   |drqm/s|Number of discard requests merged into the device queue per second.|
   |%drqm|Percentage of discard requests merged before they are sent to the device.|
   |d\_await|Average time for sending discard requests to the device to be served.|
   |dareq-sz|Average size of discard requests sent to the device, in KB.|
   |f/s|Number of refresh requests completed by the device per second (after combination).|
   |f\_await|Average time for sending refresh requests to the device to be served.|
   |aqu-sz|Average queue length of requests sent to the device.|
   |%util|Percentage of the I/O operation time, that is, the usage.|

2. **sar**: displays the read and write performance of the system drive.

   Example:

   ```shell
   # Display the usage status of all hard drives in the system in the sampling period. Collect the statistics every 3 seconds for five times.
   sar -d 3 5
   ```

   ```text
   [root@openEuler ~]# sar -d 3 5
   Linux 5.10.0-153.12.0.92.oe2203SP3.aarch64 (openEuler) 05/20/2023      _aarch64_      (4 CPU)

   04:38:27 PM     DVE      tps     rkB/s     wkB/s     dkB/s     areq-sz     aqu-sz     await    %util
   04:38:30 PM     vda     0.00      0.00      0.00      0.00        0.00       0.00      0.00     0.00
   04:38:33 PM    dm-0     0.00      0.00      0.00      0.00        0.00       0.00      0.00     0.00
   04:38:36 PM    dm-1     0.00      0.00      0.00      0.00        0.00       0.00      0.00     0.00
   ```

   The fields in the command output are described as follows:

   |Field|Description|
   |--|--|
   |tps|Total number of transfers sent to the physical device per second.|
   |rKB/s|Number of KBs read from the device per second.|
   |wKB/s|Number of KBs written to the device per second.|
   |dKB/s|Number of KBs discarded by the device per second.|
   |areq-sz|Average size (KB) of I/O requests sent to the device.|
   |aqu-sz|Average queue length of requests sent to the device.|
   |await|Average time for sending I/O requests to the device to be served.|
   |%util|Percentage of the time used to send I/O requests to the device (bandwidth usage of the device).|

3. vmstat

   ```shell
   # Run the vmstat command to monitor and report drive statistics.
   vmstat -d
   ```

   ![en-us\_image\_0000001389098425](./images/en-us_image_0000001389098425.png)

   The fields in the command output are described as follows:

   |Field|Description|
   |--|--|
   |reads|**-total**: total number of reads that have been successfully completed.  **-merged**: number of merged reads (resulting in one I/O).  **-sectors**: sectors from which data is successfully read.  **-ms**: number of milliseconds spent on reading data.|
   |writes|**-total**: total number of writes that have been successfully completed.  **-merged**: merged writes (resulting in one I/O).  **-sectors**: sectors to which data is successfully written.  **-ms**: number of milliseconds spent on writing data.|
   |IO|Drive read/write information.  **-bi**: total amount of data read from the block device, in blocks.  **-bo**: total amount of data written to the block device, in blocks.|

---

---
url: /en/docs/22.03_LTS_SP4/server/security/secharden/system_services.md
---
# System Services

## Hardening the SSH Service

### Description

The Secure Shell (SSH) is a reliable security protocol for remote logins and other network services. SSH prevents information disclosure during remote management. SSH encrypts transferred data to prevent domain name server (DNS) spoofing and IP spoofing. OpenSSH was created as an open source alternative to the proprietary SSH protocol.

Hardening the SSH service is to modify configurations of the SSH service to set the algorithm and authentication parameters when the system uses the OpenSSH protocol, improving the system security.  [Table 1](#en-us_topic_0152100390_ta2fdb8e4931b4c1a8f502b3c7d887b95)  describes the hardening items, recommended hardening values, and default policies.

### Implementation

To harden a server, perform the following steps:

1. Open the configuration file  **/etc/ssh/sshd\_config**  of the SSH service on the server, and modify or add hardening items and values in the file.
2. Save the  **/etc/ssh/sshd\_config**  file.
3. Run the following command to restart the SSH service:

   ```shell
   systemctl restart sshd
   ```

To harden a client, perform the following steps:

1. Open the configuration file  **/etc/ssh/ssh\_config**  of the SSH service on the client, and modify or add hardening items and values in the file.
2. Save the  **/etc/ssh/ssh\_config**  file.

### Hardening Items

* Server hardening policies

  All SSH service hardening items are stored in the  **/etc/ssh/sshd\_config**  configuration file. For details about the server hardening items, hardening suggestions, and whether the hardening items are configured as suggested, see  [Table 1](#en-us_topic_0152100390_ta2fdb8e4931b4c1a8f502b3c7d887b95).

  **Table  1**  SSH hardening items on a server

  > \[!NOTE] **NOTE:**
  > By default, the messages displayed before and after SSH login are saved in the  **/etc/issue.net**  file. The default information in the  **/etc/issue.net**  file is  **Authorized users only.** **All activities may be monitored and reported.**

* Client hardening policies

  All SSH service hardening items are stored in the  **/etc/ssh/ssh\_config**  configuration file. For details about the client hardening items, hardening suggestions, and whether the hardening items are configured as suggested, see  [Table 2](#en-us_topic_0152100390_tb289c5a6f1c7420ab4339187f9018ea4).

  **Table  2**  SSH hardening items on a client

  > \[!NOTE] **NOTE:**
  > Third-party clients and servers that use the Diffie-Hellman algorithm are required to allow at least 2048-bit connection.

### Other Security Suggestions

* The SSH service only listens on specified IP addresses.

  For security purposes, you are advised to only listen on required IP addresses rather than listen on 0.0.0.0 when using the SSH service. You can specify the IP addresses that SSH needs to listen on in the ListenAddress configuration item in the  **/etc/ssh/sshd\_config**  file.

  1. Open and modify the  **/etc/ssh/sshd\_config**  file.

     ```shell
     vi /etc/ssh/sshd_config
     ```

     The following information indicates that the bound listening IP address is  **192.168.1.100**. You can change the listening IP address based on the site requirements.

     ```shell
     ...
     ListenAddress 192.168.1.100
     ...
     ```

  2. Restart the SSH service.

     ```shell
     systemctl restart sshd.service
     ```

* SFTP users are restricted from access to upper-level directories.

  SFTP is a secure FTP designed to provide secure file transfer over SSH. Users can only use dedicated accounts to access SFTP for file upload and download, instead of SSH login. In addition, directories that can be accessed over SFTP are limited to prevent directory traversal attacks. The configuration process is as follows:

  > \[!NOTE] **NOTE:**
  > In the following configurations,  **sftpgroup**  is an example user group name, and  **sftpuser**  is an example username.

  1. Create an SFTP user group.

     ```shell
     groupadd sftpgroup
     ```

  2. Create an SFTP root directory.

     ```shell
     mkdir /sftp
     ```

  3. Modify the ownership of and permission on the SFTP root directory.

     ```shell
     chown root:root /sftp
     chmod 755 /sftp
     ```

  4. Create an SFTP user.

     ```shell
     useradd -g sftpgroup -s /sbin/nologin sftpuser
     ```

  5. Set the password of the SFTP user.

     ```shell
     passwd sftpuser
     ```

  6. Create an SFTP user directory.

     ```shell
     mkdir /sftp/sftpuser
     ```

  7. Modify the ownership of and permission on the SFTP user directory.

     ```shell
     chown root:root /sftp/sftpuser
     chmod 777 /sftp/sftpuser
     ```

  8. Create a directory used to store files uploaded by the SFTP user.

     ```shell
     mkdir /sftp/sftpuser/sftpupload
     ```

  9. Modify the ownership of the upload directory of the SFTP user.

     ```shell
     chown sftpuser:sftpgroup /sftp/sftpuser/sftpupload
     ```

  10. Modify the  **/etc/ssh/sshd\_config**  file.

      ```shell
      vi /etc/ssh/sshd_config
      ```

      Modify the following information:

      ```text
      #Subsystem sftp /usr/libexec/openssh/sftp-server -l INFO -f AUTH
      Subsystem sftp internal-sftp -l INFO -f AUTH
      ...

      Match Group sftpgroup
          ChrootDirectory /sftp/%u
          ForceCommand internal-sftp
      ```

      > \[!NOTE] **NOTE:**
      >
      > * **%u**  is a wildcard character. Enter  **%u**  to represent the username of the current SFTP user.
      > * The following content must be added to the end of the  **/etc/ssh/sshd\_config**  file:
      >
      >   ```text
      >     Match Group sftpgroup
      >       ChrootDirectory /sftp/%u
      >       ForceCommand internal-sftp
      >   ```

  11. Restart the SSH service.

      ```shell
      systemctl restart sshd.service
      ```

* Remotely execute commands using SSH.

  When a command is executed remotely through OpenSSH, TTY is disabled by default. If a password is required during command execution, the password is displayed in plain text. To ensure password input security, you are advised to add the  **-t**  option to the command. Example:

  ```shell
  ssh -t testuser@192.168.1.100 su
  ```

  > \[!NOTE] **NOTE:**
  > **192.168.1.100**  is an example IP address, and  **testuser**  is an example username.

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/aops/systrace_user_guide.md
---
# sysTrace用户指南

## 简介

sysTrace是一款运用于在AI训练任务中的软件，在AI训练中，常常出现训练任务故障导致训练成本浪费，业务痛点如下：

* AI训练性能故障缺乏常态化监控、检测能力
* Host bound引发的AI任务慢，卡故障缺乏全栈跟踪能力

sysTrace工具支持如下功能：

* 采集torch\_npu层的python函数的调用栈
* 采集cann层的内存持有情况，判断是否发生HBM OOM故障
* 采集mspti的通信算子下发/执行，判断是否发生算子慢的情况，从而定位到慢卡

## 安装

安装sysTrace工具需要操作系统为 openEuler 22.03 SP4，在配置了 openEuler yum 源的机器直接使用 yum 命令安装，此处介绍如何安装sysTrace工具。

### 环境要求

* 操作系统：openEuler 22.03 SP4
* Ascend CANN 版本不低于8.0 RC3
* Libunwind 版本不低于 1.7

### 安装步骤

配置openEuler的yum源，直接使用yum命令安装

```shell
yum install sysTrace
```

## 使用方法

### 采集数据

安装完sysTrace软件包后，使用LD\_PRELAOD的方式将动态库加载到AI训练任务中（注：sysTrace 开销受AI训练任务影响，建议实际测试任务的训练开销波动在0.5%以下，波动较大场景测试开销可能影响较大，放大sysTrace的开销占用（实测2%左右））

```shell
LD_PRELOAD=/usr/local/lib/libunwind.so.8.2.0:/usr/local/lib/libunwind-aarch64.so.8.2.0:/home/ascend-toolkit-bak/ascend-toolkit/8.0.RC3.10/tools/mspti/lib64/libmspti.so:/usr/lib64/libsysTrace.so python ...
```

### 转换数据

```python
## 转化火焰图（注意：只能转换0卡的数据）
python /usr/bin/sysTrace/convert_mem_to_flamegraph.py <mem_trace_xxx_rank0.pb> <output_file>

python /usr/bin/sysTrace/convert_pytorch_to_timeline.py --output <output_file>
```

### 展示

将最终的数据上传到<https://ui.perfetto.dev/>并展示

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/terms_of_use.md
---
# Terms of Use

## Copyright © 2023 openEuler Community

Your replication, use, modification, and distribution of this document are governed by the Creative Commons License Attribution-ShareAlike 4.0 International Public License (CC BY-SA 4.0). You can visit  <https://creativecommons.org/licenses/by-sa/4.0/>  to view a human-readable summary of (and not a substitute for) CC BY-SA 4.0. For the complete CC BY-SA 4.0, visit  <https://creativecommons.org/licenses/by-sa/4.0/legalcode>.

## Trademarks and Permissions

All trademarks and registered trademarks mentioned in the documents are the property of their respective holders. The use of the openEuler trademark must comply with the [Use Specifications of the openEuler Trademark](https://www.openeuler.org/en/other/brand/).

## Disclaimer

This document is used only as a guide. Unless otherwise specified by applicable laws or agreed by both parties in written form, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, including but not limited to non-infringement, timeliness, and specific purposes.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/tool_guide.md
---
# Tool Guide

To help users better use virtualization, openEuler provides a set of tools, including vmtop and LibcarePlus. This section describes how to install and use these tools.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/vmtop.md
---
# Tool Guide

## vmtop

### Overview

vmtop is a user-mode tool running on the host machine. You can use the vmtop tool to dynamically view the usage of VM resources in real time, such as CPU usage, memory usage, and the number of vCPU traps. Therefore, the vmtop tool can be used to locate virtualization problems and optimize performance.

#### Multi-Architecture Support

Currently, the vmtop supports the AArch64 and x86\_64 processor architectures.

#### Display Item Description

The vmtop display items vary according to the processor architecture. This document describes the meaning of each display item and whether it is displayed in the corresponding architecture.

> \[!NOTE] **Note:**
> The following sampling difference refers to the difference between two times of data obtained in a specified interval.

##### Display Items of the AArch64 and x86\_64 Architectures

* **VM/task-name**: VM/Process name
* **DID**: VM ID
* **PID**: PID of the qemu process of the VM
* **%CPU**: CPU usage of a process
* **EXTsum**: Total number of KVM exits (sampling difference)
* **S**: Process status
* **P**: ID of the physical CPU occupied by a process
* **%ST**: Ratio of the preemption time to the CPU running time
* **%GUE**: Ratio of the VM internal occupation time to the CPU running time
* **%HYP**: Virtualization overhead ratio

##### Display Items Only for the Aarch64 Architecture

* **EXThvc**: Number of hvc-exits (sampling difference)
* **EXTwfe**: Number of wfe-exits (sampling difference)
* **EXTwfi**: Number of wfi-exits (sampling difference)
* **EXTmmioU**: Number of mmioU-exits (sampling difference)
* **EXTmmioK**: Number of mmioK-exits (sampling difference)
* **EXTfp**: Number of fp-exits (sampling difference)
* **EXTirq**: Number of irq-exits (sampling difference)
* **EXTsys64**: Number of sys64 exits (sampling difference)
* **EXTmabt**: Number of mem abort exits (sampling difference)

##### Display Items Only for the x86\_64 Architecture

* **PFfix**: Number of page faults (sampling difference)
* **PFgu**: Number of times that page faults are injected to the guest OS (sampling difference)
* **INvlpg**: Number of times that a TLB item is flushed (one of the TLB items, which is not fixed)
* **EXTio**: Number of io VM-exit times (sampling difference)
* **EXTmmio**: Number of mmio VM-exit times (sampling difference)
* **EXThalt**: Number of halt VM-exit times (sampling difference)
* **EXTsig**: Number of VM-exits caused by signal processing (sampling difference)
* **EXTirq**: Number of VM-exits caused by interrupts (sampling difference)
* **EXTnmiW**: Number of VM-exit times caused by non-maskable interrupts (sampling difference)
* **EXTirqW**: Interruptwindow mechanism. When the interrupt function is enabled, exit is used to inject interrupts (sampling difference)
* **IrqIn**: Number of times that IRQ interrupts are injected (sampling difference)
* **NmiIn**: Number of times that NMI interrupts are injected (sampling difference)
* **TLBfl**: Number of times that the entire TLB is flushed (sampling difference)
* **HostReL**: Number of times that the host status is overloaded (sampling difference)
* **Hyperv**: Number of times that the guest OS is simulated to call hypercall in virtualization-assistant mode (sampling difference)
* **EXTcr**: Number of times that the access to the CR register exits (sampling difference)
* **EXTrmsr**: Number of times that the read MSR exits (sampling difference)
* **EXTwmsr**: Number of times that the write MSR exits (sampling difference)
* **EXTapic**: Number of APIC write times (sampling difference)
* **EXTeptv**: Ept page fault exit times (sampling difference)
* **EXTeptm**: Number of Ept error exits (sampling difference)
* **EXTpau**: Number of times that the VCPU pauses and exits (sampling difference)

### Usage

vmtop is a command line tool. You can directly run the vmtop in command line mode.
In addition, the vmtop tool provides different options for querying different information.

#### Syntax

```sh
vmtop [option]
```

#### Option Description

* `-d`: sets the refresh interval, in seconds.
* `-H`: displays the VM thread information.
* `-n`: sets the number of refresh times and exits after the refresh is complete.
* `-b`: displays Batch mode, which can be used to redirect to a file.
* `-h`: displays help information.
* `-v`: displays versions.
* `-p`: monitors the VM with a specified ID.

#### Keyboard Shortcut

Shortcut key used when the vmtop is running.

* **H**: displays or stops the VM thread information. The information is displayed by default.
* Up/Down: moves the VM list upwards or downwards.
* Left/Right: moves the cursor leftwards or rightwards to display the columns that are hidden due to the screen width.
* **f**: enters the editing mode of a monitoring item and selects the monitoring item to be enabled.
* **q**: exits the vmtop process.

### Example

Run the vmtop command on the host.

```sh
vmtop
```

The command output is as follows:

```sh
vmtop - 2020-09-14 09:54:48 - 1.0
Domains: 1 running

  DID  VM/task-name     PID  %CPU    EXThvc    EXTwfe    EXTwfi  EXTmmioU  EXTmmioK     EXTfp    EXTirq  EXTsys64   EXTmabt    EXTsum    S    P   %ST  %GUE  %HYP
    2       example 4054916  13.0         0         0      1206        10         0       144        62       174         0      1452    S  106   0.0  99.7  16.0
```

As shown in the output, there is only one VM named **example** on the host. The ID is 2. The CPU usage is 13.0%. The total number of traps within one second is 1452. The physical CPU occupied by the VM process is CPU 106. The ratio of the VM internal occupation time to the CPU running time is 99.7%.

1. Display VM thread information.
   Press **H** to display the thread information.

   ```sh
   vmtop - 2020-09-14 10:11:27 - 1.0
   Domains: 1 running

     DID  VM/task-name     PID  %CPU    EXThvc    EXTwfe    EXTwfi  EXTmmioU  EXTmmioK     EXTfp    EXTirq  EXTsys64   EXTmabt    EXTsum    S    P   %ST  %GUE  %HYP
       2       example 4054916  13.0         0         0    1191        17         4       120        76       147         0      1435    S  119   0.0 123.7   4.0
     |_      qemu-kvm 4054916   0.0         0         0         0         0         0         0         0         0         0         0    S  119   0.0   0.0   0.0
     |_      qemu-kvm 4054928   0.0         0         0         0         0         0         0         0         0         0         0    S  119   0.0   0.0   0.0
     |_  signalfd_com 4054929   0.0         0         0         0         0         0         0         0         0         0         0    S  120   0.0   0.0   0.0
     |_  IO mon_iothr 4054932   0.0         0         0         0         0         0         0         0         0         0         0    S  117   0.0   0.0   0.0
     |_     CPU 0/KVM 4054933   3.0         0         0       280         6         4        28        19        41         0       350    S  105   0.0  27.9   0.0
     |_     CPU 1/KVM 4054934   3.0         0         0       260         0         0        16        12        36         0       308    S   31   0.0  20.0   0.0
     |_     CPU 2/KVM 4054935   3.0         0         0       341         0         0        44        20        26         0       387    R  108   0.0  27.9   4.0
     |_     CPU 3/KVM 4054936   5.0         0         0       310        11         0        32        25        44         0       390    S  103   0.0  47.9   0.0
     |_   memory_lock 4054940   0.0         0         0         0         0         0         0         0         0         0         0    S  126   0.0   0.0   0.0
     |_    vnc_worker 4054944   0.0         0         0         0         0         0         0         0         0         0         0    S  118   0.0   0.0   0.0
     |_        worker 4143738   0.0         0         0         0         0         0         0         0         0         0         0    S  120   0.0   0.0   0.0
   ```

   The example VM has 11 threads, including the vCPU thread, vnc\_worker, and IO mon\_iotreads. Each thread also displays detailed CPU usage and trap information.

2. Select the monitoring item.
   Enter f to edit the monitoring item.

```sh
field filter - select which field to be showed
Use up/down to navigate, use space to set whether chosen filed to be showed
'q' to quit to normal display

 * DID
 * VM/task-name
 * PID
 * %CPU
 * EXThvc
 * EXTwfe
 * EXTwfi
 * EXTmmioU
 * EXTmmioK
 * EXTfp
 * EXTirq
 * EXTsys64
 * EXTmabt
 * EXTsum
 * S
 * P
 * %ST
 * %GUE
 * %HYP
```

All monitoring items are displayed by default. You can press the up or down key to select a monitoring item, press the space key to set whether to display or hide the monitoring item, and press the q key to exit.
After %**ST**, **%GUE**, and **%HYP** are hidden, the following information is displayed:

```sh
vmtop - 2020-09-14 10:23:25 - 1.0
Domains: 1 running

  DID  VM/task-name     PID  %CPU    EXThvc    EXTwfe    EXTwfi  EXTmmioU  EXTmmioK     EXTfp    EXTirq  EXTsys64   EXTmabt    EXTsum    S    P
    2       example 4054916  12.0         0         0      1213        14         1       144        68       168         0      1464    S  125
   |_    qemu-kvm 4054916   0.0         0         0         0         0         0         0         0         0         0         0    S  125
   |_    qemu-kvm 4054928   0.0         0         0         0         0         0         0         0         0         0         0    S  119
   |_  signalfd_com 4054929   0.0         0         0         0         0         0         0         0         0         0         0    S  120
   |_  IO mon_iothr 4054932   0.0         0         0         0         0         0         0         0         0         0         0    S  117
   |_     CPU 0/KVM 4054933   2.0         0         0       303         6         0        29        10        35         0       354    S   98
   |_     CPU 1/KVM 4054934   4.0         0         0       279         0         0        39        17        49         0       345    S    1
   |_     CPU 2/KVM 4054935   3.0         0         0       283         0         0        33        20        40         0       343    S  122
   |_     CPU 3/KVM 4054936   3.0         0         0       348         8         1        43        21        44         0       422    S  110
   |_   memory_lock 4054940   0.0         0         0         0         0         0         0         0         0         0         0    S  126
   |_    vnc_worker 4054944   0.0         0         0         0         0         0         0         0         0         0         0    S  118
   |_        worker    1794   0.0         0         0         0         0         0         0         0         0         0         0    S  126
```

**%ST**, **%GUE**, and **%HYP** will not be displayed on the screen.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/eggo_tool_introduction.md
---
# Tool Introduction

This chapter describes the information related to the automatic deployment tool. You are advised to read this chapter before deployment.

## Deployment Modes

The automatic Kubernetes cluster deployment tool provided by openEuler supports one-click deployment using the CLI. The tool provides the following deployment modes:

* Offline deployment: Prepare all required RPM packages, binary files, plugins, and container images on the local host, pack the packages into a tar.gz file in a specified format, and compile the corresponding YAML configuration file. Then, you can run commands to deploy the cluster in one-click. This deployment mode can be used when the VM cannot access the external network.
* Online deployment: Compile the YAML configuration file. The required RPM packages, binary files, plugins, and container images are automatically downloaded from the Internet during installation and deployment. In this mode, the VM must be able to access the software sources and the image repository on which the cluster depends, for example, Docker Hub.

## Configurations

When you use the automatic Kubernetes cluster deployment tool, use the YAML configuration file to describe the cluster deployment information. This section describes the configuration items and provides configuration examples.

### Configuration Items

* cluster-id: Cluster name, which must comply with the naming rules for the DNS names. Example: k8s-cluster

* username: User name used to log in to the hosts using SSH where the Kubernetes cluster is to be deployed. The user name must be identical on all hosts.

* private-key-path:The path of the key for password-free SSH login. You only need to configure either private-key-path or password. If both are configured, private-key-path is used preferentially.

* masters: The master node list. It is recommended that each master node is also set as a worker node. Each master node contains the following sub-items. Each master node must be configured with a group of sub-items:
  * name: The name of the master node, which is the node name displayed to the Kubernetes cluster.
  * ip: The IP address of the master node.
  * port: The port for SSH login of the node. The default value is 22.
  * arch: CPU architecture of the master node. For example, the value for x86\_64 CPUs is amd64.

* workers: The list of the worker nodes. Each worker node contains the following sub-items. Each worker node must be configured with a group of sub-items:
  * name: The name of the worker node, which is the node name displayed to the Kubernetes cluster.
  * ip: The IP address of the master node.
  * port: The port for SSH login of the node. The default value is 22.
  * arch: CPU architecture of the worker node. For example, the value for x86\_64 CPUs is amd64.

* etcds: The list of etcd nodes. If this parameter is left empty, one etcd node is deployed for each master node. Otherwise, only the configured etcd node is deployed. Each etcd node contains the following sub-items. Each etcd node must be configured with a group of sub-items:
  * name: The name of the etcd node, which is the node name displayed to the Kubernetes cluster.
  * ip: The IP address of the etcd node.
  * port: The port for SSH login.
  * arch: CPU architecture of the etcd node. For example, the value for x86\_64 CPUs is amd64.

* loadbalance: The loadbalance node list. Each loadbalance node contains the following sub-items. Each loadbalance node must be configured with a group of sub-items:
  * name: The name of the loadbalance node, which is the node name displayed to the Kubernetes cluster.
  * ip: The IP address of the loadbalance node.
  * port: The port for SSH login.
  * arch: CPU architecture of the loadbalance node. For example, the value for x86\_64 CPUs is amd64.
  * bind-port: The listening port of the load balancing service.

* external-ca: Whether to use an external CA certificate. If yes, set this parameter to true. Otherwise, set this parameter to false.

* external-ca-path: The path of the external CA certificate file. This parameter takes affect only when external-ca is set to true.

* service: service information created by Kubernetes. The service configuration item contains the following sub-items:
  * cidr: The IP address segment of the service created by Kubernetes.
  * dnsaddr: DNS address of the service created by Kubernetes
  * gateway: The gateway address of the service created by Kubernetes.
  * dns: The configuration item of the CoreDNS created by Kubernetes. The dns configuration item contains the following sub-items:
    * corednstype: The deployment type of the CoreDNS created by Kubernetes. The value can be pod or binary.
    * imageversion: The CoreDNS image version of the pod deployment type.
    * replicas: The number of CoreDNS replicas of the pod deployment type.

* network: The network configuration of the Kubernetes cluster. The network configuration item contains the following sub-items:
  * podcidr: IP address segment of the Kubernetes cluster network.
  * plugin: The network plugin deployed in the Kubernetes cluster
  * plugin-args: The configuration file path of the network plugin of the Kubernetes cluster network. Example: {"NetworkYamlPath": "/etc/kubernetes/addons/calico.yaml"}

* apiserver-endpoint: The IP address or domain name of the APIServer service that can be accessed by external systems. If loadbalance is configured, set this parameter to the IP address of the loadbalance node. Otherwise, set this parameter to the IP address of the first master node.

* apiserver-cert-sans: The IP addresses and domain names that need to be configured in the APIServer certificate. This configuration item contains the following sub-items:
  * dnsnames: The array list of the domain names that need to be configured in the APIServer certificate.
  * ips: The array list of IP addresses that need to be configured in the APIServer certificate.

* apiserver-timeout: APIServer response timeout interval.

* etcd-token: The etcd cluster name.

* dns-vip: The virtual IP address of the DNS.

* dns-domain: The DNS domain name suffix.

* pause-image: The complete image name of the pause container.

* network-plugin: The type of the network plugin. This parameter can only be set to cni. If this item is not configured, the default Kubernetes network is used.

* cni-bin-dir: network plugin address. Use commas (,) to separate multiple addresses. For example: /usr/libexec/cni,/opt/cni/bin.

* runtime: The type of the container runtime. Currently, docker and iSulad are supported.

* runtime-endpoint: The endpoint of the container runtime. This parameter is optional when runtime is set to docker.

* registry-mirrors: The mirror site address of the image repository used for downloading container images.

* insecure-registries: The address of the image repository used for downloading container images through HTTP.

* config-extra-args: The extra parameters for starting services of each component (such as kube-apiserver and etcd). This configuration item contains the following sub-items:
  * name: The component name. The value can be etcd, kube-apiserver, kube-controller-manager, kube-scheduler, kube-proxy or kubelet.

  * extra-args: The extended parameters of the component. The format is key: value. Note that the component parameter corresponding to key must be prefixed with a hyphen (-) or two hyphens (--).

  * open-ports: Configure the ports that need to be enabled additionally. The ports required by Kubernetes do not need to be configured. Other plugin ports need to be configured additionally.
    * worker | master | etcd | loadbalance: The type of the node where the ports are enabled. Each configuration item contains one or more port and protocol sub-items.
      * port: The port address.
      * protocol: The port type. The value can be tcp or udp.

  * install: Configure the detailed information about the installation packages or binary files to be installed on each type of nodes. Note that the corresponding files must be packaged in a tar.gz installation package. The following describes the full configuration. Select the configuration items as needed.

    * package-source: The detailed information about the installation package.
      * type: The compression type of the installation package. Currently, only tar.gz installation packages are supported.
      * dstpath: The path where the installation package is to be decompressed on the peer host. The path must be valid absolute path.
      * srcpath: The path for storing the installation packages of different architectures. The architecture must correspond to the host architecture. The path must be a valid absolute path.
        * arm64: The path of the installation package of the ARM64 architecture. This parameter is required if any ARM64 node is included in the configuration.
        * amd64: The path of the installation package of the AMD64 architecture. This parameter is required if any x86\_64 node is included in the configuration.

    > \[!NOTE]**NOTE**:
    >
    > * In the install configuration item, the sub-items of etcd, kubernetes-master, kubernetes-worker, network, loadbalance, container, image, and dns are the same, that is, name, type, dst, schedule, and TimeOut. dst, schedule, and TimeOut are optional. You can determine whether to configure them based on the files to be installed. The following uses the etcd and kubernetes-master nodes as an example.

    * etcd: The list of packages or binary files to be installed on etcd nodes.
      * name: The names of the software packages or binary files to be installed. If the software package is an installation package, enter only the name and do not specify the version. During the installation, `$name*` is used for identification. Example: etcd. If there are multiple software packages, use commas (,) to separate them.
      * type: The type of the configuration item. The value can be pkg, repo, bin, file, dir, image, yaml, or shell. If type is set to repo, configure the repo source on the corresponding node.
      * dst: The path of the destination folder. This parameter is required when type is set to bin, file, or dir. It indicates the directory where a file or folder is stored. To prevent users from incorrectly configuring a path and deleting important files during cleanup, this parameter must be set to a path in the whitelist. For details, see "Whitelist Description."
    * kubernetes-master: The list of packages or binary files to be installed on the Kubernetes master nodes.
    * kubernetes-worker: The list of packages or binary files to be installed on the Kubernetes worker nodes.
    * network: The list of packages or binary files to be installed for the network.
    * loadbalance: The list of packages or binary files to be installed on the loadbalance nodes.
    * container: The list of packages or binary files to be installed for the containers.
    * image: The tar package of the container image.
    * dns: Kubernetes CoreDNS installation package. If corednstype is set to pod, this parameter is not required.
    * addition: The list of additional installation packages or binary files.
      * master: The following configurations will be installed on all master nodes.
        * name: The name of the software package or binary file to be installed.
        * type: The type of the configuration item. The value can be pkg, repo, bin, file, dir, image, yaml, or shell. If type is set to repo, configure the repo source on the corresponding node.
        * schedule: Valid only when type is set to shell. This parameter indicates when the user wants to execute the script. The value can be prejoin (before the node is added), postjoin (after the node is added), precleanup (before the node is removed), or postcleanup (after the node is removed).
        * TimeOut: The script execution timeout interval. If the execution times out, the process is forcibly stopped. The default value is 30s.
      * worker: The configurations will be installed on all worker nodes. The configuration format is the same as that of master under addition.

### Whitelist Description

The value of dst under install must match the whitelist rules. Set it to a path in the whitelist or a subdirectory of the path. The current whitelist is as follows:

* /usr/bin
* /usr/local/bin
* /opt/cni/bin
* /usr/libexec/cni
* /etc/kubernetes
* /usr/lib/systemd/system
* /etc/systemd/system
* /tmp

### Configuration Example

The following is an example of the YAML file configuration. As shown in the example, nodes of different types can be deployed on a same host, but the configurations of these nodes must be the same. For example, a master node and a worker node are deployed on test0.

```yaml
cluster-id: k8s-cluster
username: root
private-key-path: /root/.ssh/private.key
masters:
- name: test0
  ip: 192.168.0.1
  port: 22
  arch: arm64
workers:
- name: test0
  ip: 192.168.0.1
  port: 22
  arch: arm64
- name: test1
  ip: 192.168.0.3
  port: 22
  arch: arm64
etcds:
- name: etcd-0
  ip: 192.168.0.4
  port: 22
  arch: amd64
loadbalance:
  name: k8s-loadbalance
  ip: 192.168.0.5
  port: 22
  arch: amd64
  bind-port: 8443
external-ca: false
external-ca-path: /opt/externalca 
service:
  cidr: 10.32.0.0/16
  dnsaddr: 10.32.0.10
  gateway: 10.32.0.1
  dns:  
    corednstype: pod
    imageversion: 1.8.4
    replicas: 2
network:
  podcidr: 10.244.0.0/16
  plugin: calico
  plugin-args: {"NetworkYamlPath": "/etc/kubernetes/addons/calico.yaml"}  
apiserver-endpoint: 192.168.122.222:6443      
apiserver-cert-sans:
  dnsnames: []
  ips: []
apiserver-timeout: 120s
etcd-external: false
etcd-token: etcd-cluster
dns-vip: 10.32.0.10 
dns-domain: cluster.local
pause-image: k8s.gcr.io/pause:3.2
network-plugin: cni 
cni-bin-dir: /usr/libexec/cni,/opt/cni/bin
runtime: docker
runtime-endpoint: unix:///var/run/docker.sock 
registry-mirrors: []
insecure-registries: []
config-extra-args:
  - name: kubelet
    extra-args:
      "--cgroup-driver": systemd
open-ports:
  worker:
  - port: 111
    protocol: tcp
  - port: 179
    protocol: tcp
install:
  package-source:
    type: tar.gz
    dstpath: ""
    srcpath: 
      arm64: /root/rpms/packages-arm64.tar.gz 
      amd64: /root/rpms/packages-x86.tar.gz
  etcd:
  - name: etcd
    type: pkg
    dst: ""
  kubernetes-master:
  - name: kubernetes-client,kubernetes-master
    type: pkg
  kubernetes-worker:
  - name: docker-engine,kubernetes-client,kubernetes-node,kubernetes-kubelet
    type: pkg
    dst: ""
  - name: conntrack-tools,socat
    type: pkg
    dst: ""
  network:
  - name: containernetworking-plugins
    type: pkg
    dst: ""
  loadbalance:
  - name: gd,gperftools-libs,libunwind,libwebp,libxslt
    type: pkg
    dst: ""
  - name: nginx,nginx-all-modules,nginx-filesystem,nginx-mod-http-image-filter,nginx-mod-http-perl,nginx-mod-http-xslt-filter,nginx-mod-mail,nginx-mod-stream
    type: pkg
    dst: ""
  container:
  - name: emacs-filesystem,gflags,gpm-libs,re2,rsync,vim-filesystem,vim-common,vim-enhanced,zlib-devel
    type: pkg
    dst: ""
  - name: libwebsockets,protobuf,protobuf-devel,grpc,libcgroup
    type: pkg
    dst: ""
  - name: yajl,lxc,lxc-libs,lcr,clibcni,iSulad
    type: pkg
    dst: ""
  image:  
  - name: pause.tar
    type: image
    dst: ""
  dns:    
  - name: coredns
    type: pkg
    dst: ""
  addition:
    master:
    - name: prejoin.sh
      type: shell
      schedule: "prejoin"
      TimeOut:  "30s"
    - name: calico.yaml
      type: yaml
      dst: ""
    worker:
    - name: docker.service
      type: file
      dst: /usr/lib/systemd/system/
    - name: postjoin.sh
      type: shell
      schedule: "postjoin"
```

### Installation Package Structure

For offline deployment, you need to prepare the Kubernetes software package and the related offline installation packages, and store the offline installation packages in a specific directory structure. The directory structure is as follows:

```shell
package
├── bin
├── dir
├── file
├── image
├── pkg
└── packages_notes.md
```

The preceding directories are described as follows:

* The directory structure of the offline deployment package corresponds to the package types in the cluster configuration file config. The package types include pkg, repo, bin, file, dir, image, yaml and shell.

* The bin directory stores binary files, corresponding to the bin package type.

* The dir directory stores the directory that needs to be copied to the target host. You need to configure the dst destination path, corresponding to the dir package type.

* The file directory stores three types of files: file, yaml, and shell. The file type indicates the files to be copied to the target host, and requires the dst destination path to be configured. The yaml type indicates the user-defined YAML files, which will be applied after the cluster is deployed. The shell type indicates the scripts to be executed, and requires the schedule execution time to be configured. The execution time includes prejoin (before the node is added), postjoin (after the node is added), precleanup (before the node is removed), and postcleanup (after the node is removed).

* The image directory stores the container images to be imported. The container images must be in a tar package format that is compatible with Docker (for example, images exported by Docker or isula-build).

* The pkg directory stores the rpm/deb packages to be installed, corresponding to the pkg package type. You are advised to use binary files to facilitate cross-release deployment.

### Command Reference

To utilize the cluster deployment tool provided by openEuler, use the eggo command to deploy the cluster.

#### Deploying the Kubernetes Cluster

Run the following command to deploy a Kubernetes cluster using the specified YAML configuration:

**eggo deploy** \[ **-d** ]  **-f** *deploy.yaml*

| Parameter| Mandatory (Yes/No)| Description                           |
| ------------- | -------- | --------------------------------- |
| --debug | -d | No| Displays the debugging information.|
| --file | -f  | Yes| Specifies the path of the YAML file for the Kubernetes cluster deployment.|

#### Adding a Single Node

Run the following command to add a specified single node to the Kubernetes cluster:

**eggo** **join** \[ **-d** ]  **--id** *k8s-cluster*  \[ **--type** *master,worker* ] **--arch** *arm64* **--port** *22* \[ **--name** *master1*] *IP*

| Parameter| Mandatory (Yes/No) | Description|
| ------------- | -------- | ------------------------------------------------------------ |
| --debug | -d | No| Displays the debugging information.|
| --id          | Yes| Specifies the name of the Kubernetes cluster where the node is to be added.|
| --type | -t  | No| Specifies the type of the node to be added. The value can be master or worker. Use commas (,) to separate multiple types. The default value is worker.|
| --arch | -a  | Yes| Specifies the CPU architecture of the node to be added.|
| --port | -p  | Yes| Specifies the port number for SSH login of the node to be added.|
| --name | -n  | No| Specifies the name of the node to be added.|
| *IP*          | Yes| Actual IP address of the node to be added.|

#### Adding Multiple Nodes

Run the following command to add specified multiple nodes to the Kubernetes cluster:

**eggo** **join**  \[ **-d** ]  **--id** *k8s-cluster*  **-f** *nodes.yaml*

| Parameter| Mandatory (Yes/No) | Description                           |
| ------------- | -------- | -------------------------------- |
| --debug | -d | No| Displays the debugging information.|
| --id          | Yes| Specifies the name of the Kubernetes cluster where the nodes are to be added.|
| --file | -f  | Yes| Specifies the path of the YAML configuration file for adding the nodes.|

#### Deleting Nodes

Run the following command to delete one or more nodes from the Kubernetes cluster:

**eggo delete**  \[ **-d** ]  **--id** *k8s-cluster* *node* \[*node...*]

| Parameter| Mandatory (Yes/No) | Description                               |
| ------------- | -------- | -------------------------------------------- |
| --debug | -d | No| Displays the debugging information.|
| --id          | Yes| Specifies the name of the cluster where the one or more nodes to be deleted are located.|
| *node*        | Yes| Specifies the IP addresses or names of the one or more nodes to be deleted.|

#### Deleting the Cluster

Run the following command to delete the entire Kubernetes cluster:

**eggo cleanup**  \[ **-d** ]  **--id** *k8s-cluster* \[ **-f** *deploy.yaml* ]

| Parameter| Mandatory (Yes/No) | Description|
| ------------- | -------- | ------------------------------------------------------------ |
| --debug | -d | No| Displays the debugging information.|
| --id          | Yes| Specifies the name of the Kubernetes cluster to be deleted.|
| --file | -f  | No| Specifies the path of the YAML file for the Kubernetes cluster deletion. If this parameter is not specified, the cluster configuration cached during cluster deployment is used by default. In normal cases, you are advised not to set this parameter. Set this parameter only when an exception occurs.|

> \[!NOTE]**NOTE**:
>
> * The cluster configuration cached during cluster deployment is recommended when you delete the cluster. That is, you are advised not to set the --file | -f parameter in normal cases. Set this parameter only when the cache configuration is damaged or lost due to an exception.

#### Querying the Cluster

Run the following command to query all Kubernetes clusters deployed using eggo:

**eggo list** \[ **-d** ]

| Parameter| Mandatory (Yes/No) | Description |
| ------------- | -------- | ------------ |
| --debug | -d | No| Displays the debugging information.|

#### Generating the Cluster Configuration File

Run the following command to quickly generate the required YAML configuration file for the Kubernetes cluster deployment.

**eggo template**  **-d**  **-f** *template.yaml* **-n** *k8s-cluster* **-u** *username* **-p** *password* **--etcd** \[*192.168.0.1,192.168.0.2*]  **--masters** \[*192.168.0.1,192.168.0.2*] **--workers** *192.168.0.3* **--loadbalance** *192.168.0.4*

| Parameter| Mandatory (Yes/No) | Description                           |
| ------------------- | -------- | ------------------------------- |
| --debug | -d       | No| Displays the debugging information.|
| --file | -f        | No| Specifies the path of the generated YAML file.|
| --name | -n        | No| Specifies the name of the Kubernetes cluster.|
| --username | -u    | No| Specifies the user name for SSH login of the configured node.|
| --password | -p    | No| Specifies the password for SSH login of the configured node.|
| --etcd              | No| Specifies the IP address list of the etcd nodes.|
| --masters           | No| Specifies the IP address list of the master nodes.|
| --workers           | No| Specifies the IP address list of the worker nodes.|
| --loadbalance | -l | No| Specifies the IP address of the loadbalance node.|

#### Querying the Help Information

Run the following command to query the help information of the eggo command:

**eggo help**

#### Querying the Help Information of Subcommands

Run the following command to query the help information of the eggo subcommands:

**eggo deploy | join | delete | cleanup | list | template -h**

| Parameter| Mandatory (Yes/No) | Description |
| ----------- | -------- | ------------ |
| --help| -h | Yes| Displays the help information.|

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/troubleshooting/troubleshooting.md
---
# Troubleshooting

## Triggering kdump Restart

```shell
# Write 1 to the sysrq file to enable the SysRq function. After this function is enabled, the kernel will respond to any operation.
echo 1 > /proc/sys/kernel/sysrq

# Make the system crash.
echo c > /proc/sysrq-trigger
```

## Performing Forcible Restart

You can use either of the following methods to forcibly restart the OS:

* Manually restart the OS.

```shell
reboot -f
```

* Forcibly power on and off the OS through iBMC.

![en-us\_image\_0000001372249333](./images/en-us_image_0000001372249333.png)

## Restarting the Network

openEuler uses NetworkManager to manage the network. Run the following command to restart the network:

```shell
systemctl restart NetworkManager
```

## Repairing the File System

After the OS is forcibly powered off and then powered on, the file system may be damaged. When the OS is started, it automatically checks and repairs the file system. If the file system fails to be repaired, you need to run the **fsck** command to scan for and repair the file system.

```shell
# In this case, the system enters the rescue mode. Check which file system is damaged in the log.
journalctl -xb
# Check whether the partition has been mounted before the repair.
cat /proc/mounts
# Uninstall the directory.
umount xx
# If the directory cannot be uninstalled, kill the process that occupies the directory.
lsof | grep xxx
kill xxx
# Run the fsck command to rectify the fault. Enter yes or no when prompted.
fsck -y /dev/xxx
```

## Manually Dropping Cache

```shell
# Different values of N can achieve different clearance purposes. According to the Linux kernel document, run the sync command before clearing data. (The drop operation does not release any dirty objects. The sync command writes all unwritten system buffers to drives, including modified inodes, delayed block I/Os, and read/write mapping files. In this way, dirty objects can be reduced so that more objects can be released.)
echo N > /proc/sys/vm/drop_caches

# Release the page caches.
echo 1 > /proc/sys/vm/drop_caches

# Release dentries and inodes.
echo 2 > /proc/sys/vm/drop_caches

# Release the page caches, dentries, and inodes.
echo 3 > /proc/sys/vm/drop_caches
```

## Rescue Mode and Single-User Mode

* Rescue mode

  Mount the openEuler 22.03 LTS SP4 ISO image and enter the rescue mode.

  1. Select **Troubleshooting**.
  2. Select **Rescue a openEuler system**.
  3. Proceed as prompted.

  ```text
  1)Continue

  2)Read-only mount

  3)Skip to shell

  4)Quit(Reboot)
  ```

* Single-user mode

  On the login page, enter **e** to go to the grub page, add **init=/bin/sh** to the **linux** line, and press **Ctrl**+**X**.

  1. Run the `mount -o remount,rw /` command.
  2. Perform operations such as changing the password.
  3. Enter **exit** to exit.

---

---
url: /en/docs/22.03_LTS_SP4/server/security/trusted_computing/trusted_computing.md
---
# Trusted Computing

## Trusted Computing Basics

### What Is Trusted Computing

The definition of being trusted varies with international organizations.

1. Trusted Computing Group (TCG):

   An entity that is trusted always achieves the desired goal in an expected way.

2. International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC) (1999):

   The components, operations, or processes involved in computing are predictable under any conditions and are resistant to viruses and a certain degree of physical interference.

3. IEEE Computer Society Technical Committee on Dependable Computing:

   Being trusted means that the services provided by the computer system can be proved to be reliable, and mainly refers to the reliability and availability of the system.

In short, being trusted means that the system operates according to a pre-determined design and policy.

A trusted computing system consists of a root of trust, a trusted hardware platform, operating system (OS), and application. The basic idea of the system is to create a trusted computing base (TCB) first, and then establish a trust chain that covers the hardware platform, OS, and application. In the trust chain, authentication is performed from the root to the next level, extending trust level by level and building a secure and trusted computing environment.

![](./figures/trusted_chain.png)

Unlike the traditional security mechanism that eliminates viruses without solving the root of the problem, trusted computing adopts the whitelist mechanism to allow only authorized kernels, kernel modules, and applications to run on the system. The system will reject the execution of a program that is unknown or has been changed.

## Kernel Integrity Measurement Architecture (IMA)

### Overview

#### IMA

The integrity measurement architecture (IMA) is a subsystem in the kernel. The IMA can measure files accessed through **execve()**, **mmap()**, and **open()** systems based on user-defined policies. The measurement result can be used for **local or remote attestation**, or can be compared with an existing reference value to **control the access to files**.

According to the Wiki definition, the function of the kernel integrity subsystem includes three parts:

* Measure: Detects accidental or malicious modifications to files, either remotely or locally.
* Appraise: Measures a file and compares it with a reference value stored in the extended attribute to control the integrity of the local file.
* Audit: Writes the measurement result into system logs for auditing.

Figuratively, IMA measurement is an observer that only records modification without interfering in it, and IMA appraisal is more like a strict security guard that rejects any unauthorized access to programs.

#### EVM

The extended verification module (EVM) is used to calculate a hash value based on the security extended attributes of a file in the system, including **security.ima** and **security.selinux**. Then this value is signed by the key stored in the TPM or other trusted environments. The signature value is stored in **security.evm** and cannot be tampered with. If the value is tampered with, the signature verification fails when the file is accessed again.

In summary, the EVM is used to provide offline protection for security extended attributes by calculating the digest of the attributes and signing and storing them in **security.evm**.

#### IMA Digest Lists

IMA Digest Lists are an enhancement of the original kernel integrity protection mechanism provided by openEuler. It replaces the original IMA mechanism to protect file integrity.

Digest lists are binary data files in a special format. Each digest list corresponds to an RPM package and records the hash values of protected files (executable files and dynamic library files) in the RPM package.

After the startup parameters are correctly configured, the kernel maintains a hash table (invisible to the user space) and provides interfaces (**digest\_list\_data** and **digest\_list\_data\_del**) that update the hash table using **securityfs**. The digest lists are signed by the private key when they are built. When uploaded to the kernel through the interface, the digest lists need to be verified by the public key in the kernel.

![](./figures/ima_digest_list_update.png)

When IMA appraisal is enabled, each time an executable file or dynamic library file is accessed, the hook in the kernel is invoked to calculate the hash values of the file content and extended attributes and search in the kernel hash table. If the calculated hash values match the one in the table, the file is allowed to be executed. Otherwise, the access is denied.

![1599719649188](./figures/ima_verification.png)

The IMA Digest Lists extension provided by the openEuler kernel provides higher security, performance, and usability than the native IMA mechanism of the kernel community, facilitating the implementation of the integrity protection mechanism in the production environment.

* **A complete trust chain for high security**

  The native IMA mechanism requires that the file extended attribute be generated and marked in advance on the live network. When the file is accessed, the file extended attribute is used as a reference value, resulting in an incomplete trust chain.

  The IMA Digest Lists extension saves the reference digest value of the file in the kernel space. During the construction, the reference digest value of the file is carried in the released RPM package in the form of a digest list. When the RPM package is installed, the digest list is imported and the signature is verified, ensuring that the reference value comes from the software publisher and implementing a complete trust chain.

* **Superior performance**

  The trusted platform module (TPM) chip is a low-speed chip, making the PCR extension operation a performance bottleneck in the IMA measurement scenario. To shatter this bottleneck, the Digest Lists extension reduces unnecessary PCR extension operations while ensuring security, providing 65% higher performance than the native IMA mechanism.

  In the IMA appraisal scenario, the Digest Lists extension performs signature verification in the startup phase to prevent signature verification from being performed each time the file is accessed. This helps deliver a 20% higher file access performance in the operation phase than that in the native IMA appraisal scenario.

* **Fast deployment and smooth upgrade**

  When the native IMA mechanism is deployed for the first time or the software package is updated, you need to switch to the fix mode, manually mark the extended attributes of the file, and then restart the system to enter the enforcing mode. In this way, the installed program can be accessed normally.

  The Digest Lists extension can be used immediately after the installation is completed. In addition, the RPM package can be directly installed or upgraded in the enforcing mode without restarting the system or manually marking the extended attributes of the file. This minimizes user perception during the operation, allowing for quick deployment and smooth upgrade on the live network.

Note: The IMA Digest Lists extension advances the signature verification of the native IMA to the startup phase. This causes the assumption that the memory in the kernel space cannot be tampered with. As a result, the IMA depends on other security mechanisms (secure startup of kernel module and dynamic memory measurement) to protect the integrity of the kernel memory.

However, either the native IMA mechanism of the community or the IMA Digest Lists extension is only a link in the trust chain of trusted computing, and cannot ensure the system security alone. Security construction is always a systematic project that builds in-depth defense.

### Constraints

1. The current IMA appraisal mode can only protect immutable files in the system, including executable files and dynamic library files.
2. The IMA provides integrity measurement at the application layer. The security of the IMA depends on the reliability of the previous links.
3. Currently, the IMA does not support the import of the third-party application digest lists.
4. The startup log may contain `Unable to open file: /etc/keys/x509_ima.der`. This error is reported from the open source community and does not affect the use of the IMA digest lists feature.
5. In the ARM version, audit errors may occur when the log mode is enabled for the IMA. This occurs because the modprobe loads the kernel module before the digest lists are imported, but does not affect the normal functions.

### Application Scenario

#### IMA Measurement

The purpose of IMA measurement is to detect unexpected or malicious modifications to system files. The measurement result can be used for local or remote attestation.

If a TPM chip exists in the system, the measurement result is extended to a specified PCR register of the TPM chip. Due to the unidirectional PCR extension and the hardware security of the TPM chip, a user cannot modify the extended measurement result, thereby ensuring authenticity of the measurement result.

The file scope and triggering conditions of IMA measurement can be configured by the user using the IMA policy.

By default, IMA is disabled. However, the system searches for the **ima-policy** policy file in the `/etc/ima/` path. If the file is found, the system measures the files in the system based on the policy during startup. If you do not want to manually compile the policy file, you can configure the `ima_policy=tcb` in the startup parameters using the default policy. For details about more policy parameters, see the section *IMA Startup Parameters* in *Appendix*.

You can check the currently loaded IMA policy in the `/sys/kernel/security/ima/policy` file. The IMA measurement log is located in the `/sys/kernel/security/ima/ascii_runtime_measurements` file, as shown in the following figure:

```shell
$ head /sys/kernel/security/ima/ascii_runtime_measurements
10 ddee6004dc3bd4ee300406cd93181c5a2187b59b ima-ng sha1:9797edf8d0eed36b1cf92547816051c8af4e45ee boot_aggregate
10 180ecafba6fadbece09b057bcd0d55d39f1a8a52 ima-ng sha1:db82919bf7d1849ae9aba01e28e9be012823cf3a /init
10 ac792e08a7cf8de7656003125c7276968d84ea65 ima-ng sha1:f778e2082b08d21bbc59898f4775a75e8f2af4db /bin/bash
10 0a0d9258c151356204aea2498bbca4be34d6bb05 ima-ng sha1:b0ab2e7ebd22c4d17d975de0d881f52dc14359a7 /lib64/ld-2.27.so
10 0d6b1d90350778d58f1302d00e59493e11bc0011 ima-ng sha1:ce8204c948b9fe3ae67b94625ad620420c1dc838 /etc/ld.so.cache
10 d69ac2c1d60d28b2da07c7f0cbd49e31e9cca277 ima-ng sha1:8526466068709356630490ff5196c95a186092b8 /lib64/libreadline.so.7.0
10 ef3212c12d1fbb94de9534b0bbd9f0c8ea50a77b ima-ng sha1:f80ba92b8a6e390a80a7a3deef8eae921fc8ca4e /lib64/libc-2.27.so
10 f805861177a99c61eabebe21003b3c831ccf288b ima-ng sha1:261a3cd5863de3f2421662ba5b455df09d941168 /lib64/libncurses.so.6.1
10 52f680881893b28e6f0ce2b132d723a885333500 ima-ng sha1:b953a3fa385e64dfe9927de94c33318d3de56260 /lib64/libnss_files-2.27.so
10 4da8ce3c51a7814d4e38be55a2a990a5ceec8b27 ima-ng sha1:99a9c095c7928ecca8c3a4bc44b06246fc5f49de /etc/passwd
```

From left to right, the content of each record indicates:

1. PCR: PCR register for extending measurement results (The default value is 10. This register is valid only when the TPM chip is installed in the system.)
2. Template hash value: hash value that is finally used for extension, combining the file content hash and the length and value of the file path
3. Template: template of the extended measurement value, for example, **ima-ng**
4. File content hash value: hash value of the measured file content
5. File path: path of the measured file
6. The ko compression feature is enabled in this version. When loading a compressed ko file, if the **appraise func=MODULE\_CHECK** policy needs to be enabled for IMA, set **module.sig\_enforce=1** in the boot parameters.

#### IMA Appraisal

The purpose of IMA appraisal is to control access to local files by comparing the reference value with the standard reference value.

IMA uses the security extension attributes **security.ima** and **security.evm** to store the reference values of file integrity measurement.

* **security.ima**: stores the hash value of the file content
* **security.evm**: stores the hash value signature of a file extended attribute

When a protected file is accessed, the hook in the kernel is triggered to verify the integrity of the extended attributes and content of the file.

1. Use the public key in the kernel keyring to verify the signature value in the extended attribute of the **security.evm** file, and compare this signature value with the hash value of the extended attribute of the current file. If they match, the extended attribute of the file is complete (including **security.ima**).
2. When the extended attribute of the file is complete, the system compares the extended attribute of the file **security.ima** with the digest value of the current file content. If they match, the system allows for the access to the file.

Likewise, the file scope and trigger conditions for IMA appraisal can be configured by users using IMA policies.

#### IMA Digest Lists

Currently, the IMA Digest Lists extension supports the following three combinations of startup parameters:

* IMA measurement mode:

  ```shell
  ima_policy=exec_tcb ima_digest_list_pcr=11
  ```

* IMA appraisal log mode + IMA measurement mode:

  ```shell
  ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=log evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest
  ```

* IMA appraisal enforcing mode + IMA measurement mode:

  ```shell
  ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=enforce-evm evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest
  ```

### Procedure

#### Initial Deployment in the Native IMA Scenario

When the system is started for the first time, you need to configure the following startup parameters:

```shell
ima_appraise=fix ima_policy=appraise_tcb
```

In the `fix` mode, the system can be started when no reference value is available. `appraise_tcb` corresponds to an IMA policy. For details, see *IMA Startup Parameters* in the *Appendix*.

Next, you need to access all the files that need to be verified to add IMA extended attributes to them:

```shell
time find / -fstype ext4 -type f -uid 0 -exec dd if='{}' of=/dev/null count=0 status=none \;
```

This process takes some time. After the command is executed, you can see the marked reference value in the extended attributes of the protected file.

```shell
$ getfattr -m - -d /sbin/init
# file: sbin/init
security.ima=0sAXr7Qmun5mkGDS286oZxCpdGEuKT
security.selinux="system_u:object_r:init_exec_t"
```

Configure the following startup parameters and restart the system:

```shell
ima_appraise=enforce ima_policy=appraise_tcb
```

#### Initial Deployment in the Digest Lists Scenario

1. Set kernel parameters to enter the log mode.

   Add the following parameters to the `/boot/efi/EFI/openEuler/grub.cfg` file:

   ```shell
   ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=log evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest
   ```

   Run the `reboot` command to restart the system and enter the log mode. In this mode, integrity check has been enabled, but the system can be started even if the check fails.

2. Install the dependency package.

   Run the **yum** command to install **digest-list-tools** and **ima-evm-utils**. Ensure that the versions are not earlier than the following:

   ```shell
   $ yum install digest-list-tools ima-evm-utils
   $ rpm -qa | grep digest-list-tools
   digest-list-tools-0.3.93-1.oe1.x86_64
   $ rpm -qa | grep ima-evm-utils
   ima-evm-utils-1.2.1-9.oe1.x86_64
   ```

3. If the **plymouth** package is installed, you need to add `-a` to the end of the **cp** command in line 147 in the `/usr/libexec/plymouth/plymouth-populate-initrd` script file:

   ```shell
       ...
       ddebug "Installing $_src"
    cp -a --sparse=always -pfL "$PLYMOUTH_SYSROOT$_src" "${initdir}/$target"
   }
   ```

4. Run `dracut` to generate **initrd** again:

   ```shell
   dracut -f -e xattr
   ```

   Edit the `/boot/efi/EFI/openEuler/grub.cfg` file and change **ima\_appraise=log** to **ima\_appraise=enforce-evm**.

   ```shell
   ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=enforce-evm evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest
   ```

   Run the **reboot** command to complete the initial deployment.

#### Building Digest Lists on OBS

Open Build Service (OBS) is a compilation system that was first used for building software packages in openSUSE and supports distributed compilation of multiple architectures.

Before building a digest list, ensure that your project contains the following RPM packages from openEuler:

* digest-list-tools
* pesign-obs-integration
* selinux-policy
* rpm
* openEuler-rpm-config

Add **Project Config** in the deliverable project:

```shell
Preinstall: pesign-obs-integration digest-list-tools selinux-policy-targeted
Macros:
%__brp_digest_list /usr/lib/rpm/openEuler/brp-digest-list %{buildroot}
:Macros
```

* The following content is added to **Preinstall**: **digest-list-tools** for generating the digest list; **pesign-obs-integration** for generating the digest list signature; **selinux-policy-targeted**, ensuring that the SELinux label in the environment is correct when the digest list is generated.
* Define the macro **%\_\_brp\_digest\_list** in Macros. The RPM runs this macro to generate a digest list for the compiled binary file in the build phase. This macro can be used as a switch to control whether the digest list is generated in the project.

After the configuration is completed, OBS automatically performs full build. In normal cases, the following two files are added to the software package:

* **/etc/ima/digest\_lists/0-metadata\_list-compact-\[package name]-\[version number]**
* **/etc/ima/digest\_lists.tlv/0-metadata\_list-compact\_tlv-\[package name]-\[version number]**

#### Building Digest Lists on Koji

Koji is a compilation system of the Fedora community. The openEuler community will support Koji in the future.

### FAQs

1. Why does the system fail to be started, or commands fail to be executed, or services are abnormal after the system is started in enforcing mode?

   In enforcing mode, IMA controls file access. If the content or extended attributes of a file to be accessed are incomplete, the access will be denied. If key commands that affect system startup cannot be executed, the system cannot be started.

   Check whether the following problems exist:

   * **Check whether the digest list is added to initrd.**

     Check whether the **dracut** command is executed to add the digest list to the kernel during the initial deployment. If the digest list is not added to **initrd**, the digest list cannot be imported during startup. As a result, the startup fails.

   * **Check whether the official RPM package is used.**

     If a non-official openEuler RPM package is used, the RPM package may not carry the digest list, or the private key for signing the digest list does not match the public key for signature verification in the kernel. As a result, the digest list is not imported to the kernel.

   If the cause is not clear, enter the log mode and find the cause from the error log:

   ```shell
   dmesg | grep appraise
   ```

2. Why access control is not performed on system files in enforcing mode?

   When the system does not perform access control on the file as expected, check whether the IMA policy in the startup parameters is correctly configured:

   ```shell
   $ cat /proc/cmdline
   ...ima_policy=exec_tcb|appraise_exec_tcb|appraise_exec_immutable...
   ```

   Run the following command to check whether the IMA policy in the current kernel has taken effect:

   ```shell
   cat /sys/kernel/security/ima/policy
   ```

   If the policy file is empty, it indicates that the policy fails to be set. In this case, the system does not perform access control.

3. After the initial deployment is completed, do I need to manually run the **dracut** command to generate **initrd** after installing, upgrading, or uninstalling the software package?

   No. The **digest\_list.so** plug-in provided by the RPM package can automatically update the digest list at the RPM package granularity, allowing users to be unaware of the digest list.

### Appendixes

#### Description of the IMA securityfs Interface

The native IMA provides the following **securityfs** interfaces:

> Note: The following interface paths are in the `/sys/kernel/security/` directory.

| Path                           | Permission | Description                                                  |
| ------------------------------ | ---------- | ------------------------------------------------------------ |
| ima/policy                     | 600        | IMA policy interface                                         |
| ima/ascii\_runtime\_measurement  | 440        | IMA measurement result in ASCII code format                  |
| ima/binary\_runtime\_measurement | 440        | IMA measurement result in binary format                      |
| ima/runtime\_measurement\_count  | 440        | Measurement result statistics                                |
| ima/violations                 | 440        | Number of IMA measurement result conflicts                   |
| evm                            | 660        | EVM mode, that is, the mode for verifying the integrity of extended attributes of files |

The values of **/sys/kernel/security/evm** are as follows:

* 0: EVM uninitialized.

* 1: Uses HMAC (symmetric encryption) to verify the integrity of extended attributes.

* 2: Uses the public key signature (asymmetric encryption) to verify the integrity of extended attributes.

* 6: Disables the integrity check of extended attributes (This mode is used for openEuler).

The additional **securityfs** interfaces provided by the IMA Digest Lists extension are as follows:

| Path                     | Permission | Description                                                |
| ------------------------ | ---------- | ---------------------------------------------------------- |
| ima/digests\_count        | 440        | Total number of digests (IMA+EVM) in the system hash table |
| ima/digest\_list\_data     | 200        | New interfaces in the digest list                          |
| ima/digest\_list\_data\_del | 200        | Interfaces deleted from the digest list                    |

#### IMA Policy Syntax

Each IMA policy statement must start with an **action** represented by the keyword action and be followed by a **filtering condition**:

* **action**: indicates the action of a policy. Only one **action** can be selected for a policy.

  > Note: You can **ignore the word action** and directly write **dont\_measure** instead of **action=dont\_measure**.

* **func**: indicates the type of the file to be measured or authenticated. It is often used together with **mask**. Only one **func** can be selected for a policy.

  * **FILE\_CHECK** can be used only with **MAY\_EXEC**, **MAY\_WRITE**, and **MAY\_READ**.
  * **MODULE\_CHECK**, **MMAP\_CHECK**, and **BPRM\_CHECK** can be used only with **MAY\_EXEC**.
  * A combination without the preceding matching relationships does not take effect.

* **mask**: indicates the operation upon which files will be measured or appraised. Only one **mask** can be selected for a policy.

* **fsmagic**: indicates the hexadecimal magic number of the file system type, which is defined in the `/usr/include/linux/magic.h` file.

  > Note: By default, all file systems are measured unless you use the **dont\_measure/dont\_appraise** to mark a file system not to be measured.

* **fsuid**: indicates the UUID of a system device. The value is a hexadecimal string of 16 characters.

* **objtype**: indicates the file type. Only one file type can be selected for a policy.

  > Note: **objtype** has a finer granularity than **func**. For example, **obj\_type=nova\_log\_t** indicates the nova log file.

* **uid**: indicates the user (represented by the user ID) who performs operations on the file. Only one **uid** can be selected for a policy.

* **fowner**: indicates the owner (represented by the user ID) of the file. Only one **fowner** can be selected for a policy.

The values and description of the keywords are as follows:

| Keyword       | Value              | Description                                                  |
| ------------- | ------------------ | ------------------------------------------------------------ |
| action        | measure            | Enables IMA measurement                                      |
|               | dont\_measure       | Disables IMA measurement                                     |
|               | appraise           | Enables IMA appraisal                                        |
|               | dont\_appraise      | Disables IMA appraisal                                       |
|               | audit              | Enables audit                                                |
| func          | FILE\_CHECK         | File to be opened                                            |
|               | MODULE\_CHECK       | Kernel module file to be loaded                              |
|               | MMAP\_CHECK         | Dynamic library file to be mapped to the memory space of the process |
|               | BRPM\_CHECK         | File to be executed (excluding script files opened by programs such as `/bin/bash`) |
|               | POLICY\_CHECK       | File to be loaded as a supplement to the IMA policy          |
|               | FIRMWARE\_CHECK     | Firmware to be loaded into memory                            |
|               | DIGEST\_LIST\_CHECK  | Digest list file to be loaded into the kernel                |
|               | KEXEC\_KERNEL\_CHECK | kexec kernel to be switched to                               |
| mask          | MAY\_EXEC           | Executes a file                                              |
|               | MAY\_WRITE          | Writes data to a file This operation is not recommended because it is restricted by open source mechanisms such as echo and vim (the essence of modification is to create a temporary file and then rename it). The IMA measurement of **MAY\_WRITE** is not triggered each time the file is modified. |
|               | MAY\_READ           | Reads a file                                                 |
|               | MAY\_APPEND         | Extends file attributes                                      |
| fsmagic       | fsmagic=xxx        | Hexadecimal magic number of the file system type             |
| fsuuid        | fsuuid=xxx         | UUID of a system device. The value is a hexadecimal string of 16 characters. |
| fowner        | fowner=xxx         | User ID of the file owner                                    |
| uid           | uid=xxx            | ID of the user who operates the file                         |
| obj\_type      | obj\_type=xxx\_t     | File type (based on the SELinux tag)                         |
| pcr           | pcr=\<num>          | Selects the PCR used to extend the measurement values in the TPM. The default value is 10. |
| appraise\_type | imasig             | Signature-based IMA appraisal                                |
|               | meta\_immutable     | Evaluates the extended attributes of the file based on signatures (supporting the digest list). |

> Note: **PATH\_CHECK** is equivalent to **FILE\_CHECK**, and **FILE\_MMAP** is equivalent to **MMAP\_CHECK**. They are not mentioned in this table.

#### IMA Native Startup Parameters

The following table lists the kernel startup parameters of the native IMA.

| Parameter        | Value        | Description                                                  |
| ---------------- | ------------ | ------------------------------------------------------------ |
| ima\_appraise     | off          | Disables the IMA appraisal mode. The integrity check is not performed when the file is accessed and no new reference value is generated for the file. |
|                  | enforce      | Enables the IMA appraisal enforcing mode to perform the integrity check when the file is accessed. That is, the file digest value is calculated and compared with the reference value. If the comparison fails, the file access is rejected. In this case, the IMA generates a new reference value for the new file. |
|                  | fix          | Enables the IMA repair mode. In this mode, the reference value of a protected file can be updated. |
|                  | log          | Enables the IMA appraisal log mode to perform the integrity check when the file is accessed. However, commands can be executed even if the check fails, and only logs are recorded. |
| ima\_policy       | tcb          | Measures all file execution, dynamic library mapping, kernel module import, and device driver loading. The file read behavior of the root user is also measured. |
|                  | appraise\_tcb | Evaluates all files whose owner is the root user.            |
|                  | secure\_boot  | Evaluates the kernel module import, hardware driver loading, kexec kernel switchover, and IMA policies. The prerequisite is that these files have IMA signatures. |
| ima\_tcb          | None         | Equivalent to **ima\_policy=tcb**.                           |
| ima\_appraise\_tcb | None         | Equivalent to **ima\_policy=appraise\_tcb**.                 |
| ima\_hash         | sha1/md5/... | IMA digest algorithm. The default value is sha1.             |
| ima\_template     | ima          | IMA measurement extension template                           |
|                  | ima-ng       | IMA measurement extension template                           |
|                  | ima-sig      | IMA measurement extension template                           |
| integrity\_audit  | 0            | Basic integrity audit information (default)                  |
|                  | 1            | Additional integrity audit information                       |

> Note: The **ima\_policy** parameter can specify multiple values at the same time, for example, **ima\_policy=tcb|appraise\_tcb**. After the system is started, the IMA policy of the system is the sum of the policies for the two parameters.

The IMA policy for the `ima_policy=tcb` startup parameter is as follows:

```text
# PROC_SUPER_MAGIC = 0x9fa0
dont_measure fsmagic=0x9fa0
# SYSFS_MAGIC = 0x62656572
dont_measure fsmagic=0x62656572
# DEBUGFS_MAGIC = 0x64626720
dont_measure fsmagic=0x64626720
# TMPFS_MAGIC = 0x01021994
dont_measure fsmagic=0x1021994
# DEVPTS_SUPER_MAGIC=0x1cd1
dont_measure fsmagic=0x1cd1
# BINFMTFS_MAGIC=0x42494e4d
dont_measure fsmagic=0x42494e4d
# SECURITYFS_MAGIC=0x73636673
dont_measure fsmagic=0x73636673
# SELINUX_MAGIC=0xf97cff8c
dont_measure fsmagic=0xf97cff8c
# SMACK_MAGIC=0x43415d53
dont_measure fsmagic=0x43415d53
# CGROUP_SUPER_MAGIC=0x27e0eb
dont_measure fsmagic=0x27e0eb
# CGROUP2_SUPER_MAGIC=0x63677270
dont_measure fsmagic=0x63677270
# NSFS_MAGIC=0x6e736673
dont_measure fsmagic=0x6e736673
measure func=MMAP_CHECK mask=MAY_EXEC
measure func=BPRM_CHECK mask=MAY_EXEC
measure func=FILE_CHECK mask=MAY_READ uid=0
measure func=MODULE_CHECK
measure func=FIRMWARE_CHECK
```

The IMA policy for the `ima_policy=tcb_appraise` startup parameter is as follows:

```text
# PROC_SUPER_MAGIC = 0x9fa0
dont_appraise fsmagic=0x9fa0
# SYSFS_MAGIC = 0x62656572
dont_appraise fsmagic=0x62656572
# DEBUGFS_MAGIC = 0x64626720
dont_appraise fsmagic=0x64626720
# TMPFS_MAGIC = 0x01021994
dont_appraise fsmagic=0x1021994
# RAMFS_MAGIC
dont_appraise fsmagic=0x858458f6
# DEVPTS_SUPER_MAGIC=0x1cd1
dont_appraise fsmagic=0x1cd1
# BINFMTFS_MAGIC=0x42494e4d
dont_appraise fsmagic=0x42494e4d
# SECURITYFS_MAGIC=0x73636673
dont_appraise fsmagic=0x73636673
# SELINUX_MAGIC=0xf97cff8c
dont_appraise fsmagic=0xf97cff8c
# SMACK_MAGIC=0x43415d53
dont_appraise fsmagic=0x43415d53
# NSFS_MAGIC=0x6e736673
dont_appraise fsmagic=0x6e736673
# CGROUP_SUPER_MAGIC=0x27e0eb
dont_appraise fsmagic=0x27e0eb
# CGROUP2_SUPER_MAGIC=0x63677270
dont_appraise fsmagic=0x63677270
appraise fowner=0
```

The IMA policy for the `ima_policy=secure_boot` startup parameter is as follows:

```text
appraise func=MODULE_CHECK appraise_type=imasig
appraise func=FIRMWARE_CHECK appraise_type=imasig
appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig
appraise func=POLICY_CHECK appraise_type=imasig
```

#### IMA Digest List Startup Parameters

The kernel startup parameters added to the IMA digest list feature are as follows:

| Parameter                | Value                   | Description                                                  |
| ------------------------ | ----------------------- | ------------------------------------------------------------ |
| integrity                | 0                       | Disables the IMA feature (by default)                        |
|                          | 1                       | Enables the IMA feature                                      |
| ima\_appraise             | off                     | Disables the IMA appraisal mode                              |
|                          | enforce-evm             | Enables the IMA appraisal forced mode to perform the integrity check when the file is accessed and control the access. |
| ima\_appraise\_digest\_list | digest                  | When the EVM is disabled, the abstract list is used for IMA appraise. The abstract list protects both the content and extended attributes of the file. |
|                          | digest-nometadata       | If the EVM digest value does not exist, the integrity check is performed only based on the IMA digest value (the file extended attribute is not protected). |
| evm                      | fix                     | Allows for any modification to the extended attribute (even if the modification causes the failure to verify the integrity of the extended attribute). |
|                          | ignore                  | Allowed to modify the extended attribute only when it does not exist or is incorrect. |
| ima\_policy               | exec\_tcb                | IMA measurement policy. For details, see the following policy description. |
|                          | appraise\_exec\_tcb       | IMA appraisal policy. For details, see the following policy description. |
|                          | appraise\_exec\_immutable | IMA appraisal policy. For details, see the following policy description. |
| ima\_digest\_list\_pcr      | 11                      | Uses PCR 11 instead of PCR 10, and uses only the digest list for measurement. |
|                          | +11                     | The PCR 10 measurement is reserved. When the TPM chip is available, the measurement result is written to the TPM chip. |
| initramtmpfs             | None                    | Adds the support for **tmpfs**.                              |

The IMA policy for the `ima_policy=exec_tcb` startup parameter is as follows:

```text
dont_measure fsmagic=0x9fa0
dont_measure fsmagic=0x62656572
dont_measure fsmagic=0x64626720
dont_measure fsmagic=0x1cd1
dont_measure fsmagic=0x42494e4d
dont_measure fsmagic=0x73636673
dont_measure fsmagic=0xf97cff8c
dont_measure fsmagic=0x43415d53
dont_measure fsmagic=0x27e0eb
dont_measure fsmagic=0x63677270
dont_measure fsmagic=0x6e736673
measure func=MMAP_CHECK mask=MAY_EXEC
measure func=BPRM_CHECK mask=MAY_EXEC
measure func=MODULE_CHECK
measure func=FIRMWARE_CHECK
measure func=POLICY_CHECK
measure func=DIGEST_LIST_CHECK
measure parser
```

The IMA policy for the `ima_policy=appraise_exec_tcb` startup parameter is as follows:

```text
appraise func=MODULE_CHECK appraise_type=imasig
appraise func=FIRMWARE_CHECK appraise_type=imasig
appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig
appraise func=POLICY_CHECK appraise_type=imasig
appraise func=DIGEST_LIST_CHECK appraise_type=imasig
dont_appraise fsmagic=0x9fa0
dont_appraise fsmagic=0x62656572
dont_appraise fsmagic=0x64626720
dont_appraise fsmagic=0x858458f6
dont_appraise fsmagic=0x1cd1
dont_appraise fsmagic=0x42494e4d
dont_appraise fsmagic=0x73636673
dont_appraise fsmagic=0xf97cff8c
dont_appraise fsmagic=0x43415d53
dont_appraise fsmagic=0x6e736673
dont_appraise fsmagic=0x27e0eb
dont_appraise fsmagic=0x63677270
```

The IMA policy for the `ima_policy=appraise_exec_immutable` startup parameter is as follows:

```text
appraise func=BPRM_CHECK appraise_type=imasig appraise_type=meta_immutable
appraise func=MMAP_CHECK
appraise parser appraise_type=imasig
```

#### IMA Kernel Compilation Options

The native IMA provides the following compilation options:

| Compilation Option               | Description                                             |
| -------------------------------- | ------------------------------------------------------- |
| CONFIG\_INTEGRITY                 | IMA/EVM compilation switch                              |
| CONFIG\_INTEGRITY\_SIGNATURE       | Enables IMA signature verification                      |
| CONFIG\_INTEGRITY\_ASYMMETRIC\_KEYS | Enables IMA asymmetric signature verification           |
| CONFIG\_INTEGRITY\_TRUSTED\_KEYRING | Enables IMA/EVM key ring                                |
| CONFIG\_INTEGRITY\_AUDIT           | Compiles the IMA audit module                           |
| CONFIG\_IMA                       | IMA compilation switch                                  |
| CONFIG\_IMA\_WRITE\_POLICY          | Allows updating the IMA policy in the running phase     |
| CONFIG\_IMA\_MEASURE\_PCR\_IDX       | Allows specifying the PCR number of the IMA measurement |
| CONFIG\_IMA\_LSM\_RULES             | Allows configuring LSM rules                            |
| CONFIG\_IMA\_APPRAISE              | IMA appraisal compilation switch                        |
| IMA\_APPRAISE\_BOOTPARAM           | Enables IMA appraisal startup parameters                |
| CONFIG\_EVM                       | EVM compilation switch                                  |

The additional compilation options provided by the IMA Digest Lists extension are as follows:

| Compilation Option | Description                         |
| ------------------ | ----------------------------------- |
| CONFIG\_DIGEST\_LIST | Enables the IMA Digest List feature |

#### IMA Performance Reference Data

The following figure compares the performance when IMA is disabled, native IMA is enabled, and IMA digest list is enabled.

![img](./figures/ima_performance.png)

#### IMA Root Certificate Configuration

Currently, openEuler uses the RPM key to sign the IMA digest list. To ensure that the IMA function is available out of the box, openEuler imports the RPM root certificate (PGP certificate) to the kernel by default during kernel compilation. Currently, there are two PGP certificates, namely, the OBS certificate used in the earlier version and the openEuler certificate used in the switchover of openEuler 22.03 LTS SP4:

```text
# cat /proc/keys | grep PGP
1909b4ad I------     1 perm 1f030000     0     0 asymmetri private OBS b25e7f66: PGP.rsa b25e7f66 []
2f10cd36 I------     1 perm 1f030000     0     0 asymmetri openeuler <openeuler@compass-ci.com> fb37bc6f: PGP.rsa fb37bc6f []
```

The current kernel does not support the import of the PGP sub-public key, and the switched openEuler certificate uses the sub-key signature. Therefore, the openEuler kernel preprocesses the certificate before compilation, extracts the sub-public key, and imports it to the kernel. For details, see the process\_pgp\_certs.sh script file in the code repository of the kernel software package: <https://atomgit.com/src-openeuler/kernel/blob/openEuler-22.03-LTS-SP4/process_pgp_certs.sh>.

If the user does not use the IMA digest list function or uses other keys to implement signature/verification, you can remove the related code and configure the kernel root certificate by yourself.

## Remote Attestation (Kunpeng Security Library)

### Introduction

This project develops basic security software components running on Kunpeng processors. In the early stage, the project focuses on trusted computing fields such as remote attestation to empower security developers in the community.

### Software Architecture

On the platform without TEE enabled, this project can provide the platform remote attestation feature, and its software architecture is shown in the following figure:

![img](./figures/RA-arch-1.png)

On the platform that has enabled TEE, this project can provide TEE remote attestation feature, and its software architecture is shown in the following figure:

![img](./figures/RA-arch-2.png)

### Installation and Configuration

1. Run the following command to use the RPM package of the Yum installation program:

   ```shell
   yum install kunpengsecl-ras kunpengsecl-rac kunpengsecl-rahub kunpengsecl-qcaserver kunpengsecl-attester kunpengsecl-tas kunpengsecl-devel
   ```

2. Prepare the database environment. Go to the **/usr/share/attestation/ras** directory and run the **prepare-database-env.sh** script to automatically configure the database environment.

3. The configuration files required for program running are stored in three paths: current path **./config.yaml**, home path **${HOME}/.config/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)/config.yaml**, and system path **/etc/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)/config.yaml**.

4. (Optional) To create a home directory configuration file, run the **prepare-ras(rac)(hub)(qca)(attester)(tas)conf-env.sh** script in **/usr/share/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)** after installing the RPM package.

### Options

#### RAS Boot Options

Run the `ras` command to start the RAS program. Note that you need to provide the ECDSA public key in the current directory and name it **ecdsakey.pub**. Options are as follows:

```console
  -H  --https         HTTP/HTTPS mode switch. The default value is https(true), false=http.
  -h  --hport         RESTful API port listened by RAS in HTTPS mode.
  -p, --port string   Client API port listened by RAS.
  -r, --rest string   RESTful API port listened by RAS in HTTP mode.
  -T, --token         Generates a verification code for test and exits.
  -t, --test          Starts in test mode.
  -v, --verbose       Prints more detailed RAS runtime log information.
  -V, --version       Prints the RAS version and exits.
```

**Note:**

> 1.To use TEE remote attestation feature, you must pre-install the **libqca.so** and **libteec.so** library provided by the TEE team.\
> 2.To not use TEE remote attestation feature, you must copy the **libqca.so** and **libteec.so** library in **${DESTDIR}/usr/share/attestation/qcaserver** path to **/usr/lib** or **/usr/lib64** path.

#### RAC Boot Options

Run the `sudo raagent` command to start the RAC program. Note that the sudo permission is required to enable the physical TPM module. Options are as follows:

```console
  -s, --server string   Specifies the RAS service port to be connected.
  -t, --test            Starts in test mode.
  -v, --verbose         Prints more detailed RAC runtime log information.
  -V, --version         Prints the RAC version and exits.
  -i, --imalog          Specifies the path of the IMA file.
  -b, --bioslog         Specifies the path of the BIOS file.
  -T, --tatest          Starts in TA test mode.
```

**Note:**

> 1.To use TEE remote attestation feature, you must start RAC not in TA test mode. And place the uuid, whether to use TCB, mem\_hash and img\_hash of the TA to be attestated sequentially in the **talist** file under the RAC execution path. The format of the **talist** file is as follows:
>
> ```text
> e08f7eca-e875-440e-9ab0-5f381136c600 false ccd5160c6461e19214c0d8787281a1e3c4048850352abe45ce86e12dd3df9fde 46d5019b0a7ffbb87ad71ea629ebd6f568140c95d7b452011acfa2f9daf61c7a
> ```
>
> 2.To not use TEE remote attestation feature, you must start RAC in TA test mode.\
> 3.If the physical TPM module cannot be enabled, RAC needs to be started in test mode. We have provided a set of platform benchmark files for RAC to read in test mode. Before starting RAC, you must copy the files in **$(DESTDIR)/etc/attestation/default\_test** directory to directory in which you run RAC.

#### QCA Boot Options

Run the `${DESTDIR}/usr/bin/qcaserver` command to start the QCA program. Note that to start QTA normally, the full path of qcaserver must be used, and the CA path parameter in QTA needs to be kept the same as the path. Options are as follows:

```console
  -C, --scenario int    Sets the application scenario of the program, The default value is sce_no_as(0), 1=sce_as_no_daa, 2=sce_as_with_daa.
  -S, --server string   Specifies the open server address/port.
```

#### ATTESTER Boot Options

Run the `attester` command to start the ATTESTER program. Options are as follows:

```console
  -B, --basevalue string   Sets the base value file read path
  -M, --mspolicy int       Sets the measurement strategy, which defaults to -1 and needs to be specified manually. 1=compare only img-hash values, 2=compare only hash values, and 3=compare both img-hash and hash values at the same time.
  -S, --server string      Specifies the address of the server to connect to.
  -U, --uuid int           Specifies the trusted apps to verify.
  -V, --version            Prints the program version and exit.
  -T, --test               Reads fixed nonce values to match currently hard-coded trusted reports.
```

#### TAS Boot Options

Run the `tas` command to start the TAS program. Options are as follows:

```console
  -T, --token         Generates a verification code for test and exits.
```

**Note:**

> 1.To enable the TAS, you must configure the private key for TAS. Run the following command to modify the configuration file in the home directory:
>
> ```shell
> $ cd ${HOME}/.config/attestation/tas
> $ vim config.yaml
> # The values of the following DAA_GRP_KEY_SK_X and DAA_GRP_KEY_SK_Y are for testing purposes only.
> # Be sure to update their contents to ensure safety before normal use.
> tasconfig:
>  port: 127.0.0.1:40008
>  rest: 127.0.0.1:40009
>  akskeycertfile: ./ascert.crt
>  aksprivkeyfile: ./aspriv.key
>  huaweiitcafile: ./Huawei IT Product CA.pem
>  DAA_GRP_KEY_SK_X: 65a9bf91ac8832379ff04dd2c6def16d48a56be244f6e19274e97881a776543c65a9bf91ac8832379ff04dd2c6def16d48a56be244f6e19274e97881a776543c
>  DAA_GRP_KEY_SK_Y: 126f74258bb0ceca2ae7522c51825f980549ec1ef24f81d189d17e38f1773b56126f74258bb0ceca2ae7522c51825f980549ec1ef24f81d189d17e38f1773b56
> ```
>
> Then enter `tas` to start TAS program.
>
> 2.In an environment with TAS, in order to improve the efficiency of QCA's certificate configuration process, not every boot needs to access the TAS to generate the certificate, but through the localized storage of the certificate. That is, read the certification path configured in `config.yaml` on QCA side, check if a TAS-issued certificate has been saved locally through the `func hasAKCert(s int) bool` function. If the certificate is successfully read, there is no need to access TAS. If the certificate cannot be read, you need to access TAS and save the certificate returned by TAS locally.

### API Definition

#### RAS APIs

To facilitate the administrator to manage the target server, RAS and the user TA in the TEE deployed on the target server, the following APIs are designed for calling:

| API                                               | Method                      |
| ---------------------------------                 | --------------------------- |
| /                                                 | GET                         |
| /{id}                                             | GET, POST, DELETE           |
| /{from}/{to}                                      | GET                         |
| /{id}/reports                                     | GET                         |
| /{id}/reports/{reportid}                          | GET, DELETE                 |
| /{id}/basevalues                                  | GET                         |
| /{id}/newbasevalue                                | POST                        |
| /{id}/basevalues/{basevalueid}                    | GET, POST, DELETE           |
| /{id}/ta/{tauuid}/status                          | GET                         |
| /{id}/ta/{tauuid}/tabasevalues                    | GET                         |
| /{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}    | GET, POST, DELETE           |
| /{id}/ta/{tauuid}/newtabasevalue                  | POST                        |
| /{id}/ta/{tauuid}/tareports                       | GET                         |
| /{id}/ta/{tauuid}/tareports/{tareportid}          | GET, POST, DELETE           |
| /{id}/basevalues/{basevalueid}                    | GET, DELETE                 |
| /version                                          | GET                         |
| /config                                           | GET, POST                   |
| /{id}/container/status                            | GET                         |
| /{id}/device/status                               | GET                         |

The usage of the preceding APIs is described as follows:

To query information about all servers, use `/`.

```shell
curl -X GET -H "Content-Type: application/json" http://localhost:40002/
```

***

To query detailed information about a target server, use the GET method of `/{id}`. **{id}** is the unique ID allocated by RAS to the target server.

```shell
curl -X GET -H "Content-Type: application/json" http://localhost:40002/1
```

***

To modify information about the target server, use the POST method of `/{id}`. `$AUTHTOKEN` is the identity verification code automatically generated by running the `ras -T` command.

```go
type clientInfo struct {
  Registered   *bool `json:"registered"`  // Registration status of the target server
  IsAutoUpdate *bool `json:"isautoupdate"`// Target server base value update policy
}
```

```shell
curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1 -d '{"registered":false, "isautoupdate":false}'
```

***

To delete a target server, use the DELETE method of `/{id}`.

> **Note:**
> This method does not delete all information about the target server. Instead, it sets the registration status of the target server to `false`.

```shell
curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1
```

***

To query information about all servers in a specified range, use the GET method of `/{from}/{to}`.

```shell
curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/9
```

***

To query all trust reports of the target server, use the GET method of `/{id}/reports`.

```shell
curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/reports
```

***

To query details about a specified trust report of the target server, use the GET method of `/{id}/reports/{reportid}`. **{reportid}** indicates the unique ID assigned by RAS to the trust report of the target server.

```shell
curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/reports/1
```

***

To delete a specified trust report of the target server, use the DELETE method of `/{id}/reports/{reportid}`.

**Note:**

> This method will delete all information about the specified trusted report, and the report cannot be queried through the API.

```shell
curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/reports/1
```

***

To query all base values of the target server, use the GET method of `/{id}/reports/{reportid}`.

```shell
curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/basevalues
```

***

To add a base value to the target server, use the POST method of `/{id}/newbasevalue`.

```go
type baseValueJson struct {
  BaseType   string `json:"basetype"`   // Base value type
  Uuid       string `json:"uuid"`       // ID of a container or device
  Name       string `json:"name"`       // Base value name
  Enabled    bool   `json:"enabled"`    // Whether the base value is available
  Pcr        string `json:"pcr"`        // PCR value
  Bios       string `json:"bios"`       // BIOS value
  Ima        string `json:"ima"`        // IMA value
  IsNewGroup bool   `json:"isnewgroup"` // Whether this is a group of new reference values
}
```

```shell
curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/newbasevalue -d '{"name":"test", "basetype":"host", "enabled":true, "pcr":"testpcr", "bios":"testbios", "ima":"testima", "isnewgroup":true}'
```

***

To query details about a specified base value of a target server, use the get method of `/{id}/basevalues/{basevalueid}`. **{basevalueid}** indicates the unique ID allocated by RAS to the specified base value of the target server.

```shell
curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/basevalues/1
```

***

To change the availability status of a specified base value of the target server, use the POST method of `/{id}/basevalues/{basevalueid}`.

```shell
curl -X POST -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40002/1/basevalues/1 -d '{"enabled":true}'
```

***

To delete a specified base value of the target server, use the DELETE method of `/{id}/basevalues/{basevalueid}`.

**Note:**

> This method will delete all the information about the specified base value, and the base value cannot be queried through the API.

```shell
curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/basevalues/1
```

To query the trusted status of a specific user TA on the target server, use the GET method of the `"/{id}/ta/{tauuid}/status"` interface. Where {id} is the unique identification number assigned by RAS to the target server, and {tauuid} is the identification number of the specific user TA.

```shell
curl -X GET -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40002/1/ta/test/status
```

***

To query all the baseline value information of a specific user TA on the target server, use the GET method of the `"/{id}/ta/{tauuid}/tabasevalues"` interface.

```shell
curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tabasevalues
```

***

To query the details of a specified base value for a specific user TA on the target server, use the GET method of the `"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"` interface. where {tabasevalueid} is the unique identification number assigned by RAS to the specified base value of a specific user TA on the target server.

```shell
curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tabasevalues/1
```

***

To modify the available status of a specified base value for a specific user TA on the target server, use the `POST` method of the `"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"` interface.

```shell
curl -X POST -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN"  http://localhost:40002/1/ta/test/tabasevalues/1 --data '{"enabled":true}'
```

***

To delete the specified base value of a specific user TA on the target server, use the `DELETE` method of the `"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"` interface.

**Note:**

> This method will delete all information about the specified base value, and the base value cannot be queried through the API.

```shell
curl -X DELETE -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" -k http://localhost:40002/1/ta/test/tabasevalues/1
```

***

To add a baseline value to a specific user TA on the target server, use the `POST` method of the `"/{id}/ta/{tauuid}/newtabasevalue"` interface.

```go
type tabaseValueJson struct {
  Uuid      string `json:"uuid"`       // the identification number of the user TA
  Name      string `json:"name"`       // base value name
  Enabled   bool   `json:"enabled"`    // whether a baseline value is available
  Valueinfo string `json:"valueinfo"`  // mirror hash value and memory hash value
}
```

```shell
curl -X POST -H "Content-Type: application/json" -H "Authorization: $AUTHTOKEN" -k http://localhost:40002/1/ta/test/newtabasevalue -d '{"uuid":"test", "name":"testname", "enabled":true, "valueinfo":"test info"}'
```

***

To query the target server for all trusted reports for a specific user TA, use the `GET` method of the `"/{id}/ta/{tauuid}/tareports"` interface.

```shell
curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports
```

***

To query the details of a specified trusted report for a specific user TA on the target server, use the `GET` method of the `"/{id}/ta/{tauuid}/tareports/{tareportid}"` interface. Where {tareportid} is the unique identification number assigned by RAS to the specified trusted report of a specific user TA on the target server.

```shell
curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports/2
```

***

To delete the specified trusted report of a specific user TA on the target server, use the `DELETE` method of the `"/{id}/ta/{tauuid}/tareports/{tareportid}"` interface.

**Note:**

> This method will delete all information of the specified trusted report, and the report cannot be queried through the API.

```shell
curl -X DELETE -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports/2
```

***

To obtain the version information of the program, use the GET method of `/version`.

```shell
curl -X GET -H "Content-Type: application/json" http://localhost:40002/version
```

***

To query the configuration information about the target server, RAS, or database, use the GET method of `/config`.

```shell
curl -X GET -H "Content-Type: application/json" http://localhost:40002/config
```

***

To modify the configuration information about the target server, RAS, or database, use the POST method of /config.

```go
type cfgRecord struct {
  // Target server configuration
    HBDuration      string `json:"hbduration" form:"hbduration"`
    TrustDuration   string `json:"trustduration" form:"trustduration"`
    DigestAlgorithm string `json:"digestalgorithm" form:"digestalgorithm"`
  // RAS configuration
    MgrStrategy     string `json:"mgrstrategy" form:"mgrstrategy"`
    ExtractRules    string `json:"extractrules" form:"extractrules"`
    IsAllupdate     *bool  `json:"isallupdate" form:"isallupdate"`
    LogTestMode     *bool  `json:"logtestmode" form:"logtestmode"`
}
```

```shell
curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/config -d '{"hbduration":"5s","trustduration":"20s","DigestAlgorithm":"sha256"}'
```

#### TAS APIs

To facilitate the administrator's management of TAS for remote control, the following API is designed for calling:

| API                 | Method            |
| --------------------| ------------------|
| /config             | GET, POST         |

To query the configuration information, use the GET method of the `/config` interface.

```shell
curl -X GET -H "Content-Type: application/json" http://localhost:40009/config
```

***

To modify the configuration information, use the POST method of the `/config` interface.

```shell
curl -X POST -H "Content-Type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40009/config -d '{"basevalue":"testvalue"}'
```

**Note:**

> Currently, only the base value in the configuration information of TAS is supported for querying and modifying.

### FAQs

1. Why cannot RAS be started after it is installed?

   > In the current RAS design logic, after the program is started, it needs to search for the `ecdsakey.pub` file in the current directory and read the file as the identity verification code for accessing the program. If the file does not exist in the current directory, an error is reported during RAS boot.
   >
   > > Solution 1: Run the `ras -T` command to generate a test token. The `ecdsakey.pub` file is generated.\
   > > Solution 2: After deploying the oauth2 authentication service, save the verification public key of the JWT token generator as `ecdsakey.pub`.

2. Why cannot RAS be accessed through REST APIs after it is started?

   > RAS is started in HTTPS mode by default. Therefore, you need to provide a valid certificate for RAS to access it. However, RAS started in HTTP mode does not require a certificate.

3. Why does the issue of 'WARNING: failed to verify x509 cert' appear after RAS/ATTESTER is started?

   > Because the CA certificate is missing.
   >
   > > Solution: Copy the Huawei IT Product certificate named `Huawei IT Product CA.pem` under `$(DESTDIR)/usr/bin` directory to the running directory of RAS/ATTESTER.

## Trusted Platform Control Module

### Background

Trusted computing has undergone continuous development and improvement in the past 40 years and has become an important branch of information security. Trusted computing technologies have developed rapidly in recent years and have solved the challenges in Trusted Computing 2.0—integration of trusted systems and existing systems, trusted management, and simplification of trusted application development. These technical breakthroughs form Trusted Computing 3.0, that is, trusted computing based on an active immune system. Compared with the passive plug-in architecture of the previous generation, Trusted Computing 3.0 proposes a new trusted system framework based on self-controlled cryptography algorithms, control chips, trusted software, trusted connections, policy management, and secure and trusted protection applications, implementing trust across the networks.

The trusted platform control module (TPCM) is a base and core module that can be integrated into a trusted computing platform to establish and ensure a trust source. As one of the innovations in Trusted Computing 3.0 and the core of active immunity, TPCM implements active control over the entire platform.

The TPCM-based Trusted Computing 3.0 architecture consists of the protection module and the computing module. On the one hand, based on the Trusted Cryptography Module (TPM), the TPCM main control firmware measures the reliability of the protection and computing modules, as well as their firmware. On the other hand, the Trusted Software Base (TSB) measures the reliability of system software and application software. In addition, the TPCM management platform verifies the reliability measurement and synchronizes and manages the trust policies.

### Feature Description

The overall system design consists of the protection module, computing module, and trusted management center software, as shown in the following figure.

![](./figures/TPCM.png)

* Trusted management center: This centralized management platform, provided by a third-party vendor, formulates, delivers, maintains, and stores protection policies and reference values for trusted computing nodes.

* Protection module: This module operates independently of the computing module and provides trusted computing protection functions that feature active measurement and active control to implement security protection during computing. The protection module consists of the TPCM main control firmware, TCB, and TCM. As a key module for implementing trust protection in a trusted computing node, the TPCM can be implemented in multiple forms, such as cards, chips, and IP cores. It contains a CPU and memory, firmware, and software such as an OS and trusted function components. The TPCM operates alongside the computing module and works according to the built-in protection policy to monitor the trust of protected resources, such as hardware, firmware, and software of the computing module. The TPCM is the Root of Trust in a trusted computing node.

* Computing module: This module includes hardware, an OS, and application layer software. The running of the OS can be divided into the boot phase and the running phase. In the boot phase, GRUB2 and shim of openEuler support the reliability measurement capability, which protects boot files such as shim, GRUB2, kernel, and initramfs. In the running phase, openEuler supports the deployment of the trusted verification agent (provided by third-party vendor HTTC). The agent sends data to the TPCM for trusted measurement and protection in the running phase.

The TPCM interacts with other components as follows:

1. The TPCM hardware, firmware, and software provide an operating environment for the TSB. The trusted function components of the TPCM provide support for the TSB to implement measurement, control, support, and decision-making based on the policy library interpretation requirements.
2. The TPCM accesses the TCM for trusted cryptography functions to complete computing tasks such as trusted verification, measurement, and confidential storage, and provides services for TCM access.
3. The TPCM connects to the trusted management center through the management interface to implement protection policy management and trusted report processing.
4. The TPCM uses the built-in controller and I/O port to interact with the controller of the computing module through the bus to actively monitor the computing module.
5. The built-in protection agent in the OS of the computing module obtains the code and data related to the preset protection object and provides them to the TPCM. The TPCM forwards the monitoring information to the TSB, and the TSB analyzes and processes the information according to the policy library.

### Constraints

Supported server: TaiShan 200 Server (Model 2280) VF
Supported BMC card: BC83SMMC

### Application Scenarios

The TPCM enables a complete trust chain to ensure that the OS boots into a trusted computing environment.

---

---
url: /en/docs/22.03_LTS_SP4/tools/community_tools/uadk/uadk_quick_start.md
---
# UADK Quick Start Guide

## Overview

This chapter describes how to quickly start using UADK and the UADK engine.

### UADK

UADK is a general-purpose user space accelerator framework that uses shared virtual addressing (SVA) to provide a unified programming interface for hardware acceleration of cryptographic and compression algorithms. UADK includes Unified/User-space-access-intended Accelerator Framework (UACCE), which enables hardware accelerators from different vendors that support SVA to adapt to UADK.

UADK consists of UACCE, vendors' drivers, and an algorithm layer. UADK requires the hardware accelerator to support SVA, and the operating system to support IOMMU and SVA. Hardware accelerators from different vendors are registered as different character devices with UACCE by using kernel-mode drivers of the vendors. A user can access the hardware accelerators by performing user-mode operations on the character devices. UADK provides an algorithm layer for invoking the cryptographic and compression algorithms in a unified manner. Currently, UADK supports the following algorithms:

* AES, SM4, DES, SM3, SHA*x*, MD5, AEAD and HMAC
* RSA and DH
* gzip and zlib

Currently, Kunpeng hardware accelerators have been registered with UACCE. Through the UADK framework, users can run cryptographic and compression algorithms using hardware accelerators instead of CPUs, freeing up CPU computing power and improving computing performance.

### UADK Engine

The UADK engine is an upper-layer application of UADK developed based on the OpenSSL engine mechanism. The UADK engine provides the function of using hardware accelerators through the OpenSSL command line tools and OpenSSL standard interface to quickly migrate existing services.

The UADK engine consists of five sub-modules: RSA engine, DH engine, ECC engine, Cipher engine, and Digest engine. After hardware accelerators from different vendors are registered with UADK as devices, you can use the OpenSSL command line tools or OpenSSL standard interface through the UADK engine to obtain the hardware acceleration functions of the devices. The engine ID is **uadk\_engine**. The sub-modules and functions of the UADK engine are as follows:

* RSA engine: supports key generation, asymmetric encryption and decryption, and digital signature.
* DH engine: supports key negotiation.
* ECC engine: generates data verification codes.
* Cipher engine: supports symmetric encryption and decryption.
* Digest engine: generates message digests.

After a Kunpeng hardware accelerator is registered with UADK, you can use the OpenSSL command line tools or OpenSSL standard interface to use the functions of the Kunpeng hardware accelerator through the UADK engine.

### Application Scenarios

Big data, data confidentiality, intelligent security, web services, and distributed storage.

***

## Usage Requirements

This section uses the Kunpeng hardware accelerator as an example to describe the usage requirements of UADK and the UADK engine. The usage requirements of other vendors' hardware accelerators are similar.

### Hardware

A CPU of the Kunpeng 9*xx* series that has been registered with UADK.

### Software

#### Operating System

openEuler 22.03 LTS or later.
The OS kernel must support the IOMMU and SVA features.

#### Other Software Packages

OpenSSL 1.1.1a or later.

### Toolchain

Compiler used to build UADK and the UADK engine:

GCC 10.2.0

***

## Installation and Deployment

This section uses the Kunpeng hardware accelerator as an example to describe how to install, upgrade, and uninstall UADK and the UADK engine. The installation, upgrade, and uninstallation of hardware accelerators from other vendors are similar. The kernel-mode driver of the Kunpeng hardware accelerator and the user-mode driver of UADK need to be used together. Perform the operations in sequence.

### Installing and Deploying UADK

The UADK algorithm library can be installed using the Yum source or RPM package, or built from source. You can select an installation method as required.

#### Installing Using the Yum Source

On openEuler 22.03 LTS SP4 or later, run the following command to install UADK from the Yum source:

```shell
yum install libwd
```

#### Installing Using the RPM Package

Obtain the [UADK RPM package](https://atomgit.com/src-openeuler/libwd) from the openEuler community.

The installation commands are as follows:

```shell
cd /usr/src/
git clone https://atomgit.com/src-openeuler/libwd.git
mkdir -p /root/rpmbuild
cd /root/rpmbuild
mkdir BUILD BUILDROOT RPMS SOURCES SPECS SRPMS
cp /usr/src/libwd/libwd*.tar.gz /usr/src/libwd/*patch /root/rpmbuild/SOURCES/
cp /usr/src/libwd/warpdrive.spec /root/rpmbuild/SPECS/
rpmbuild --bb SPECS/warpdrive.spec
rpm -ivh /root/rpmbuild/RPMS/aarch64/libwd*.rpm
```

#### Building from Source

Obtain the [UADK source code](https://github.com/Linaro/uadk) from the Linaro community.

For details about how to build, install, and configure UADK, visit
<https://github.com/Linaro/uadk/blame/master/INSTALL>.

### Loading the UACCE Driver

Before loading the hardware accelerator driver of the vendor, you need to load **uacce.ko**.

Run `modprobe uacce` or `insmod /lib/modules/$(uname -r)/uacce.ko` to load **uacce.ko**.

### Loading the Accelerator Driver of the Vendor

The following uses the Kunpeng hardware accelerator as an example to describe how to load a driver.

| Accelerator Module | Module Loading Sequence            |
| ------------------ | ---------------------------------- |
| HPRE               | uacce.ko, hisi\_qm.ko, hisi\_hpre.ko |
| ZIP                | uacce.ko, hisi\_qm.ko, hisi\_zip.ko  |
| SEC                | uacce.ko, hisi\_qm.ko, hisi\_sec2.ko |

> \[!NOTE] **Note:**\
> When loading **hisi\_hpre.ko**, **hisi\_zip.ko**, or **hisi\_sec2.ko**, you can specify **uacce\_mode**.  `uacce_mode=1` indicates the SVA mode. `uacce_mode=2` indicates the no-SVA mode.
> The user-mode driver of the Kunpeng hardware accelerator depends on the UACCE framework (while the kernel-mode driver does not). Therefore, you need to load **uacce.ko** first. The ZIP, HPRE, and SEC modules of the Kunpeng hardware accelerator depend on the QM module for queue management. Therefore, after loading **uacce.ko**, you need to load **hisi\_qm.ko**, and then load the drivers of the ZIP, HPRE, and SEC modules.

You can use the insmod or modprobe tool to load the drivers.

To load the drivers using the modprobe tool, perform the following steps:

* Load the user-mode driver of the HPRE module in SVA mode.

  ```shell
  modprobe hisi_hpre uacce_mode=1
  ```

* Load the user-mode driver of the SEC module in SVA mode.

  ```shell
  modprobe hisi_sec2 uacce_mode=1
  ```

* Load the user-mode driver of the ZIP module in SVA mode.

  ```shell
  modprobe hisi_zip uacce_mode=1
  ```

To load the drivers using the insmod tool, perform the following steps:

* Load the user-mode driver of the HPRE module in SVA mode.

  ```shell
  insmod /lib/modules/$(uname -r)/uacce.ko
  insmod /lib/modules/$(uname -r)/hisi_qm.ko
  insmod /lib/modules/$(uname -r)/hisi_hpre.ko uacce_mode=1
  ```

* Load the user-mode driver of the SEC module in SVA mode.

  ```shell
  insmod /lib/modules/$(uname -r)/uacce.ko
  insmod /lib/modules/$(uname -r)/hisi_qm.ko
  insmod /lib/modules/$(uname -r)/hisi_sec2.ko uacce_mode=1
  ```

* Load the user-mode driver of the ZIP module in SVA mode.

  ```shell
  insmod /lib/modules/$(uname -r)/uacce.ko
  insmod /lib/modules/$(uname -r)/hisi_qm.ko
  insmod /lib/modules/$(uname -r)/hisi_zip.ko uacce_mode=1
  ```

  Module parameter configuration:

When loading the drivers, you can set the module parameters in any sequence. After configuring the module parameters and loading the driver, you can query the module parameters by using the `cat /sys/bus/pci/drivers/<driver>/module/parameters/` command. The module parameters cannot be updated after the driver is loaded. To modify the module parameters, you need to unload the driver, set the new module parameters, and reload the driver.

* The formats of the module parameter configuration commands are as follows:

  ```shell
  insmod <driver> [uacce_mode] [pf_q_num] [vfs_num] [sgl_sge_nr] [ctx_q_num]
  ```

  ```shell
  modprobe <driver_name> [uacce_mode] [pf_q_num] [vfs_num] [sgl_sge_nr] [ctx_q_num]
  ```

  * The parameters in square brackets (\[]) are optional and have default values. The parameters can be in any sequence.
  * The default value of **uacce\_mode** for all modules is **0**, indicating that the user mode is not supported. Therefore, you need to set **uacce\_mode=1** for users in user mode.
  * The default value of **pf\_q\_num** for the SEC module is **256**. The default value of **pf\_q\_num** for the HPRE or ZIP module is **64**.
  * The default value of **vfs\_num** for all modules is **0**.
  * The default value of **sgl\_sge\_nr** for all modules is **10**.
  * The default value of **ctx\_q\_num** for all modules is **2**.

  For example, if you choose not to use the default values when loading the ZIP driver, run the following command to manually configure the parameters:

  ```shell
  insmod /lib/modules/$(uname -r)/hisi_zip.ko uacce_mode=1 pf_q_num =16 vfs_num=1 sgl_sge_nr=16
  ```

  * If only the SVA feature is required for the first time, set **uacce\_mode=1**.

### Unloading the Accelerator Driver of the Vendor

The following uses the Kunpeng hardware accelerator as an example to describe how to unload a driver. To unload a driver, run the following command:

```shell
modprobe -r hisi_hpre
```

or

```shell
rmmod hisi_hpre
```

### Installing and Deploying the UADK Engine

The UADK engine can be installed using the Yum source or RPM package, or built from source. You can select an installation method as required.

#### Installing Using the Yum Source

On openEuler 22.03 LTS SP4, run the following command to install the UADK engine using the Yum source:

```shell
yum install uadk_engine
```

#### Installing Using the RPM Package

Obtain the [UADK engine RPM package](https://atomgit.com/src-openeuler/uadk_engine) from the openEuler community.

The installation commands are as follows:

```shell
cd /usr/src/
git clone https://atomgit.com/src-openeuler/uadk_engine.git
mkdir -p /root/rpmbuild
cd /root/rpmbuild
mkdir BUILD BUILDROOT RPMS SOURCES SPECS SRPMS
cp /usr/src/uadk_engine/uadk_engine*.tar.gz /usr/src/uadk_engine/*patch /root/rpmbuild/SOURCES/
cp /usr/src/uadk_engine/uadk_engine.spec /root/rpmbuild/SPECS/
rpmbuild --bb SPECS/uadk_engine.spec
rpm -ivh /root/rpmbuild/RPMS/aarch64/uadk_engine*.rpm --prefix=/usr/local/openssl/lib/engines-1.1
```

#### Building from Source

Obtain the [UADK engine source code](https://github.com/Linaro/uadk_engine) from the Linaro community.

For details about how to build and install the UADK engine, visit
<https://github.com/Linaro/uadk_engine/blob/master/README>.

***

## Getting Started

### Using UADK

UADK provides a performance test tool. After UADK is built and installed, a tool named uadk\_tool is generated. You can view the usage and parameter description of the performance test tool using the `uadk_tool benchmark --help` command.

#### Enabling Environment Variables

Run the `export` commands to set the numbers of queues.

```shell
export WD_RSA_CTX_NUM="sync:2@0,async:4@0"
export WD_DH_CTX_NUM="sync:2@0,async:4@0"
export WD_CIPHER_CTX_NUM="sync:2@2,async:4@2"
export WD_DIGEST_CTX_NUM="sync:2@2,async:4@2"
```

The input parameter format of the environment variables is **ctx\_mode:ctx\_num@node**, indicating that a number of *ctx\_num* queues in *ctx\_mode* are set on the NUMA node whose index is *node*.
For example, **"sync:2@0,async:4@0"** indicates that two queues in sync mode and four queues in async mode are set on the NUMA 0 node.

#### Performing Performance Tests

* MD5 performance test

  Test the digest calculation performance of MD5 in SVA mode.

  ```shell
  uadk_tool benchmark --alg md5 --mode sva --opt 0 --sync --seconds 5 --thread 2 --multi 1 --ctxnum 6
  ```

* SM3 performance test

  Test the digest calculation performance of SM3 in SVA mode.

  ```shell
  uadk_tool benchmark --alg sm3 --mode sva --opt 0 --sync --seconds 5 --thread 2 --multi 1 --ctxnum 6
  ```

* SHA performance test

  Test the digest calculation performance of SHA-512 in SVA mode.

  ```shell
  uadk_tool benchmark --alg sha-512 --mode sva --opt 0 --sync --seconds 5 --thread 2 --multi 1 --ctxnum 6
  ```

* AES performance test

  Test the performance of AES-128-CBC encryption in SVA mode.

  ```shell
  uadk_tool benchmark --alg aes-128-cbc --mode sva --opt 0 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

  Test the performance of AES-128-CBC decryption in SVA mode.

  ```shell
  uadk_tool benchmark --alg aes-128-cbc --mode sva --opt 1 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

* SM4 performance test

  Test the performance of SM4-128-ECB encryption in SVA mode.

  ```shell
  uadk_tool benchmark --alg sm4-128-ecb --mode sva --opt 0 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

  Test the performance of SM4-128-ECB decryption in SVA mode.

  ```shell
  uadk_tool benchmark --alg sm4-128-ecb --mode sva --opt 1 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

* DES performance test

  Test the performance of 3DES-128-ECB encryption in SVA mode.

  ```shell
  uadk_tool benchmark --alg 3des-128-ecb --mode sva --opt 0 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

  Test the performance of 3DES-128-ECB decryption in SVA mode.

  ```shell
  uadk_tool benchmark --alg 3des-128-ecb --mode sva --opt 1 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

  For other test scenarios, use the `uadk_tool benchmark --help` command to view the parameter and configuration description.

### Using the UADK Engine

You can use the OpenSSL command line tools to directly invoke the UADK engine. Use the help menu of each OpenSSL tool to learn about how to use the tool.

#### Enabling Environment Variables

The UADK engine supports environment variables. You can set the numbers of queues for executing tasks as required.

1. Add the following content to the beginning of the **openssl.cnf** file (usually in **/usr/local/ssl/**):

   ```text
   openssl_cnf=openssl_def
   [openssl_def]
   engines=engine_section
   [engine_section]
   uadk_engine=uadk_section
   [uadk_section]
   UADK_CMD_ENABLE_RSA_ENV=1
   UADK_CMD_ENABLE_DH_ENV=1
   UADK_CMD_ENABLE_CIPHER_ENV=1
   UADK_CMD_ENABLE_DIGEST_ENV=1
   ```

2. Run the `export` commands to set the numbers of queues.

   ```shell
   export WD_RSA_CTX_NUM="sync:2@0,async:4@0"  
   export WD_DH_CTX_NUM="sync:2@0,async:4@0"  
   export WD_CIPHER_CTX_NUM="sync:2@2,async:4@2"  
   export WD_DIGEST_CTX_NUM="sync:2@2,async:4@2"
   ```

   The input parameter format of the environment variables is **ctx\_mode:ctx\_num@node**, indicating that a number of *ctx\_num* queues in *ctx\_mode* are set on the NUMA node whose index is *node*.
   For example, **"sync:2@0,async:4@0"** indicates that two queues in sync mode and four queues in async mode are set on the NUMA 0 node.

#### Performing Function Tests

* RSA function test

  Generate a private key.

  ```shell
  openssl genrsa -out prikey.pem -engine uadk_engine 1024
  ```

  Obtain a public key.

  ```shell
  openssl rsa -in prikey.pem -pubout -out pubkey.pem -engine uadk_engine
  ```

  Assume that the file to be encrypted is **plain.txt**.

  ```shell
  echo "Content to be encrypted" > plain.txt
  ```

  Encrypt the file.

  ```shell
  openssl rsautl -encrypt -in plain.txt -inkey pubkey.pem -pubin -out enc.txt -engine uadk_engine
  ```

  Decrypt the file.

  ```shell
  openssl rsautl -decrypt -in enc.txt -inkey prikey.pem -out dec.txt -engine uadk_engine
  ```

  Assume that the file to be signed is **msg.txt**.

  ```shell
  echo "Content to be signed" > msg.txt
  ```

  Sign the file.

  ```shell
  openssl rsautl -sign -in msg.txt -inkey prikey.pem -out signed.txt -engine uadk_engine
  ```

  Verify the signature.

  ```shell
  openssl rsautl -verify -in signed.txt -inkey pubkey.pem -pubin -out verified.txt -engine uadk_engine
  ```

  Use the openssl speed tool to perform the test.

  ```shell
  openssl speed -elapsed -engine uadk_engine rsa1024
  openssl speed -elapsed -engine uadk_engine -async_jobs 10 rsa1024
  openssl speed -elapsed -engine uadk_engine -async_jobs 36 rsa1024
  ```

* DH function test

  Generate a global public key parameter.

  ```shell
  openssl dhparam -out dhparam.pem 768
  ```

  Generate Alice's private key.

  ```shell
  openssl genpkey -paramfile dhparam.pem -out alice_prikey.pem -engine uadk_engine
  ```

  Obtain Alice's public key.

  ```shell
  openssl pkey -in alice_prikey.pem -pubout -out alice_pubkey.pem
  ```

  Generate Bob's private key.

  ```shell
  openssl genpkey -paramfile dhparam.pem -out bob_prikey.pem -engine uadk_engine
  ```

  Obtain Bob's public key.

  ```shell
  openssl pkey -in bob_prikey.pem -pubout -out bob_pubkey.pem
  ```

  Exchange public keys and generate negotiated keys.

  ```shell
  openssl pkeyutl -derive -inkey alice_prikey.pem -peerkey bob_pubkey.pem -out secret1.bin -engine uadk_engine
  openssl pkeyutl -derive -inkey bob_prikey.pem -peerkey alice_pubkey.pem -out secret2.bin -engine uadk_engine
  ```

  Compare the negotiated shared keys.

  ```shell
  cmp secret1.bin secret2.bin
  xxd secret1.bin
  xxd secret2.bin
  ```

* MD5 function test

  Assume that the digest file to be calculated is **data.txt**.

  ```shell
  echo "Content to be hashed" > data.txt
  ```

  Calculate the digest.

  ```shell
  openssl md5 -engine uadk_engine data.txt
  ```

  Use the openssl speed tool to perform the test.

  ```shell
  openssl speed -engine uadk_engine -async_jobs 1 -evp md5
  ```

* SM3 function test

  Assume that the digest file to be calculated is **data.txt**.

  ```shell
  echo "Content to be hashed" > data.txt
  ```

  Calculate the digest.

  ```shell
  openssl sm3 -engine uadk_engine data.txt
  ```

* SHA function test

  Assume that the digest file to be calculated is **data.txt**.

  ```shell
  echo "Content to be hashed" > data.txt
  ```

  Calculate the digest.

  ```shell
  openssl sha1 -engine uadk_engine data.txt
  openssl sha256 -engine uadk_engine data.txt
  openssl sha512 -engine uadk_engine data.txt
  ```

* AES function test

  Assume that the file to be encrypted is **data.txt**.

  ```shell
  echo "Content to be encrypted" > data
  ```

  Use AES-128-CBC to encrypt the file.

  ```shell
  openssl enc -aes-128-cbc -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use AES-128-CBC to decrypt the file.

  ```shell
  openssl enc -aes-128-cbc -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use AES-192-CBC to encrypt the file.

  ```shell
  openssl enc -aes-192-cbc -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use AES-192-CBC to decrypt the file.

  ```shell
  openssl enc -aes-192-cbc -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use AES-256-CBC to encrypt the file.

  ```shell
  openssl enc -aes-256-cbc -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use AES-256-CBC to decrypt the file.

  ```shell
  openssl enc -aes-256-cbc -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use AES-128-ECB to encrypt the file.

  ```shell
  openssl enc -aes-128-ecb -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-128-ECB to decrypt the file.

  ```shell
  openssl enc -aes-128-ecb -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-192-ECB to encrypt the file.

  ```shell
  openssl enc -aes-192-ecb -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-192-ECB to decrypt the file.

  ```shell
  openssl enc -aes-192-ecb -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-256-ECB to encrypt the file.

  ```shell
  openssl enc -aes-256-ecb -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-256-ECB to decrypt the file.

  ```shell
  openssl enc -aes-256-ecb -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-128-CTR to encrypt the file.

  ```shell
  openssl enc -aes-128-ctr -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-128-CTR to decrypt the file.

  ```shell
  openssl enc -aes-128-ctr -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-192-CTR to encrypt the file.

  ```shell
  openssl enc -aes-192-ctr -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-192-CTR to decrypt the file.

  ```shell
  openssl enc -aes-192-ctr -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-256-CTR to encrypt the file.

  ```shell
  openssl enc -aes-256-ctr -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  Use AES-256-CTR to decrypt the file.

  ```shell
  openssl enc -aes-256-ctr -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

* SM4 function test

  Use SM4-CBC to encrypt the file.

  ```shell
  openssl enc -sm4-cbc -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use SM4-CBC to decrypt the file.

  ```shell
  openssl enc -sm4-cbc -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use SM4-ECB to encrypt the file.

  ```shell
  openssl enc -sm4-ecb -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use SM4-ECB to decrypt the file.

  ```shell
  openssl enc -sm4-ecb -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

* DES function test

  Use DES-EDE3-CBC to encrypt the file.

  ```shell
  openssl enc -des-ede3-cbc -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use DES-EDE3-CBC to decrypt the file.

  ```shell
  openssl enc -des-ede3-cbc -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use DES-EDE3-ECB to encrypt the file.

  ```shell
  openssl enc -des-ede3-ecb -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  Use DES-EDE3-ECB to decrypt the file.

  ```shell
  openssl enc -des-ede3-ecb -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

---

---
url: /zh/docs/22.03_LTS_SP4/tools/community_tools/uadk/uadk_quick_start.md
---
# UADK 快速入门

## 概述

欢迎使用UADK 和 UADK engine。
本文档编写的目的是帮助用户快速开始使用UADK 和UADK engine。

### UADK

UADK（User space Accelerator Development Kit，用户态加速器开发包）是采用SVA（Shared Virtual Address）技术的通用型用户态加速器框架，为用户提供了硬件加速计算密码学、压缩等算法的统一编程接口。UADK中包含UACCE（Unified/User-space-access-intended Accelerator Framework），能使不同厂商支持SVA技术的硬件加速器均可适配到UADK框架。

UADK框架包含UACCE、厂商驱动、抽象算法层。需要硬件加速器设备支持SVA，操作系统支持IOMMU和SVA。不同厂商的硬件加速器设备通过厂商自身的内核态驱动，在UACCE上注册成为不同的字符设备，使得用户能够在用户态通过字符设备操作来访问不同厂商的硬件加速器设备。UADK针对密码学、压缩等算法抽象出了算法层，该抽象算法层实现了通用的调用接口。目前主要支持以下算法：

* AES、SM4、DES、SM3、SHAx、MD5、AEAD、HMAC算法；
* RSA、DH算法；
* gzip、zlib算法。

当前已有鲲鹏（Kunpeng）硬件加速器设备注册到UACCE，并通过UADK框架，为用户提供了卸载CPU、使用硬件加速器计算密码学、压缩等算法的功能，达到了释放CPU算力、提升计算性能的目的。

### UADK engine

UADK engine是UADK的一种上层应用，基于OpenSSL的engine机制开发。UADK engine提供通过OpenSSL命令行工具以及OpenSSL标准接口使用硬件加速器设备的功能，能够实现快速迁移现有业务。

UADK engine由RSA engine、DH engine、ECC engine、Cipher engine、Digest engine这5个子模块组成。不同厂商的硬件加速器设备，在注册到UADK框架之后，用户均能够通过UADK engine使用OpenSSL命令行工具或OpenSSL标准接口获得不同厂商的硬件加速计算功能。engine id统一为uadk\_engine。UADK engine的组成模块和支持功能如下：

* RSA engine子模块，支持密钥生成、非对称加解密、数字签名。
* DH engine子模块，支持密钥协商。
* ECC engine子模块，支持产生数据校验码。
* Cipher engine子模块，支持对称加解密。
* Digest engine子模块，支持生成消息摘要。

当前鲲鹏（Kunpeng）硬件加速器设备注册到UADK框架后，用户能使用OpenSSL命令行工具和OpenSSL标准接口通过UADK engine获得鲲鹏（Kunpeng）硬件加速器设备的相关功能。

### 使用场景

大数据、数据机密、智能安防、 Web服务、分布式存储等。

***

## 使用要求

本章节以鲲鹏（Kunpeng）硬件加速器设备为例，介绍UADK以及UADK engine的使用要求，其他厂商的硬件加速器设备使用要求类似。

### 硬件

已注册到UADK框架的鲲鹏（Kunpeng）9xx系列CPU。

### 软件

#### 操作系统要求

openEuler 22.03及以上版本。

操作系统要求内核支持IOMMU & SVA特性。

#### 其他软件包要求

OpenSSL 1.1.1a及以上版本。

### 工具链

编译UADK 和UADK engine 依赖的编译器：

gcc version 10.2.0 (GCC)

***

## 安装部署

本章节以鲲鹏（Kunpeng）硬件加速器设备为例，介绍UADK以及UADK engine的安装、升级和卸载，其他厂商的硬件加速器设备类似。
需要鲲鹏（Kunpeng）加速器内核态驱动、UADK用户驱动态配合使用。请按照顺序进行安装。

### UADK 安装部署

UADK抽象算法库安装部署方式有三种：yum源安装、RPM包安装以及源码编译安装。可根据实际情况，选择一种安装方式。

#### yum源安装

对于openEuler 22.03 LTS SP4及以上版本的用户，可以直接用yum源安装，安装命令：

```shell
yum install libwd
```

#### RPM包安装

从openEuler社区获取UADK rpm包：<https://atomgit.com/src-openeuler/libwd>。

安装命令如下：

```shell
cd /usr/src/

git clone https://atomgit.com/src-openeuler/libwd.git

mkdir -p /root/rpmbuild

cd /root/rpmbuild

mkdir BUILD BUILDROOT RPMS SOURCES SPECS SRPMS

cp /usr/src/libwd/libwd*.tar.gz /usr/src/libwd/*patch /root/rpmbuild/SOURCES/

cp /usr/src/libwd/warpdrive.spec /root/rpmbuild/SPECS/

rpmbuild --bb SPECS/warpdrive.spec

rpm -ivh /root/rpmbuild/RPMS/aarch64/libwd*.rpm
```

#### 源码编译安装

从Linaro社区上获取到UADK源码：<https://github.com/Linaro/uadk>。

详细源码编译、安装、配置步骤参见以下链接：
<https://github.com/Linaro/uadk/blame/master/INSTALL>。

### 加载UACCE驱动

在加载厂商的硬件加速器驱动前，需要先加载uacce.ko。

加载命令：`modprobe uacce` 或`insmod /lib/modules/$(uname -r)/uacce.ko`

### 加载厂商加速器驱动

以鲲鹏（Kunpeng）硬件加速器的驱动加载为例。

| 加速器模块 | 模块加载顺序 |
|---|---|
HPRE|uacce.ko, hisi\_qm.ko, hisi\_hpre.ko|
ZIP|uacce.ko, hisi\_qm.ko, hisi\_zip.ko|
SEC|uacce.ko, hisi\_qm.ko, hisi\_sec2.ko|

> \[!NOTE]说明
> 加载 hisi\_hpre.ko/hisi\_zip.ko/hisi\_sec2.ko时可以指定uacce\_mode，uacce\_mode=1为SVA模式，uacce\_mode=2为no-SVA模式。
> 鲲鹏（Kunpeng）加速器用户态驱动依赖uacce框架（内核态不依赖），因此需要先加载uacce.ko。鲲鹏（Kunpeng）硬件加速器的ZIP、HPRE、SEC模块依赖QM队列管理，因此加载uacce.ko之后需要加载hisi\_qm.ko，然后加载ZIP、HPRE、SEC模块的驱动。

可以使用insmod或modprobe工具加载驱动。

modprobe工具加载方式：

* 加载HPRE模块SVA模式用户态驱动的命令：

  ```shell
  modprobe hisi_hpre uacce_mode=1
  ```

* 加载SEC模块SVA模式用户态驱动的命令：

  ```shell
  modprobe hisi_sec2 uacce_mode=1
  ```

* 加载ZIP模块SVA模式用户态驱动的命令：

  ```shell
  modprobe hisi_zip uacce_mode=1
  ```

insmod工具加载方式：

* 加载HPRE模块SVA模式用户态驱动的命令：

  ```shell
  insmod /lib/modules/$(uname -r)/uacce.ko
  insmod /lib/modules/$(uname -r)/hisi_qm.ko
  insmod /lib/modules/$(uname -r)/hisi_zip.ko uacce_mode=1
  ```

* 加载SEC模块SVA模式用户态驱动的命令：

  ```shell
  insmod /lib/modules/$(uname -r)/uacce.ko
  insmod /lib/modules/$(uname -r)/hisi_qm.ko
  insmod /lib/modules/$(uname -r)/hisi_sec2.ko uacce_mode=1
  ```

* 加载ZIP模块SVA模式用户态驱动的命令：

  ```shell
  insmod /lib/modules/$(uname -r)/uacce.ko
  insmod /lib/modules/$(uname -r)/hisi_qm.ko
  insmod /lib/modules/$(uname -r)/hisi_hpre.ko uacce_mode=1
  ```

模块参数配置说明：

在加载驱动时，可以进行模块参数配置，参数没有先后顺序要求。在配置模块参数加载驱动后，可以通过`cat /sys/bus/pci/drivers/<driver>/module/parameters/` 来查询模块参数。模块参数在加载驱动后不支持更新；如果需要更改模块参数，需要先卸载驱动，在重新加载驱动时设置新的模块参数。

* 模块参数配置命令格式：

  ```shell
  insmod <driver> [uacce_mode] [pf_q_num] [vfs_num] [sgl_sge_nr] [ctx_q_num]
  ```

  ```shell
  modprobe <driver_name> [uacce_mode] [pf_q_num] [vfs_num] [sgl_sge_nr] [ctx_q_num]
  ```

  * \[]中的参数为可选项，均有默认值，无顺序要求。

  * 所有模块uacce\_mode默认值为0，表示不支持用户态，因此对于用户态的用户来说，需要配置uacce\_mode=1。

  * SEC模块pf\_q\_num的默认值为256；HPRE/ZIP模块pf\_q\_num的默认值为64。

  * 所有模块vfs\_num的默认值为0。

  * 所有模块sgl\_sge\_nr的默认值为10。

  * 所有模块ctx\_q\_num的默认值为2。

    例如，加载ZIP驱动时，如果不使用默认值，手动参数配置命令如下：

    ```shell
    insmod /lib/modules/$(uname -r)/hisi_zip.ko uacce_mode=1 pf_q_num =16 vfs_num=1 sgl_sge_nr=16
    ```

  * 初次仅需要使用SVA特性时，配置uacce\_mode=1即可。

### 卸载厂商加速器驱动

以鲲鹏（Kunpeng）硬件加速器驱动卸载为例。卸载命令：

```shell
modprobe -r hisi_hpre
```

或者

```shell
rmmod hisi_hpre
```

### UADK engine 安装部署

UADK engine安装部署方式有三种：yum源安装、RPM包安装以及源码编译安装。可根据实际情况，选择一种安装方式。

#### yum源安装

对于openEuler 22.03 LTS SP4用户，可以直接用yum源安装，安装命令：

```shell
yum install uadk_engine
```

#### RPM包安装

从openEuler社区获取uadk\_engine rpm包：<https://atomgit.com/src-openeuler/uadk_engine>。

安装命令如下：

```shell
cd /usr/src/

git clone https://atomgit.com/src-openeuler/uadk_engine.git

mkdir -p /root/rpmbuild

cd /root/rpmbuild

mkdir BUILD BUILDROOT RPMS SOURCES SPECS SRPMS

cp /usr/src/uadk_engine/uadk_engine*.tar.gz /usr/src/uadk_engine/*patch /root/rpmbuild/SOURCES/

cp /usr/src/uadk_engine/uadk_engine.spec /root/rpmbuild/SPECS/

rpmbuild --bb SPECS/uadk_engine.spec

rpm -ivh /root/rpmbuild/RPMS/aarch64/uadk_engine*.rpm --prefix=/usr/local/openssl/lib/engines-1.1

```

#### 源码编译安装

可以从Linaro社区获取到UADK engine源码：<https://github.com/Linaro/uadk_engine>。

详细源码编译安装步骤参见以下链接：
<https://github.com/Linaro/uadk_engine/blob/master/README>。

***

## 开始使用

### 使用UADK

UADK提供了性能测试工具，编译安装UADK后会生成一个名为uadk\_tool的工具，可以通过`uadk_tool benchmark --help`查看性能测试工具使用方法及参数说明。

#### 开启环境变量

使用export命令设置队列数量:

```shell
export WD_RSA_CTX_NUM="sync:2@0,async:4@0"
export WD_DH_CTX_NUM="sync:2@0,async:4@0"
export WD_CIPHER_CTX_NUM="sync:2@2,async:4@2"
export WD_DIGEST_CTX_NUM="sync:2@2,async:4@2"
```

环境变量的入参格式为`ctx_mode:ctx_num@node`，表示在索引号为node的numa节点上设置ctx\_num个ctx\_mode模式的队列。
例如，`"sync:2@0,async:4@0"`表示，在numa 0节点上设置2个sync模式的队列和4个async模式的队列。

#### 性能测试

* MD5性能测试

  测试md5在SVA模式计算摘要的性能：

  ```shell
  uadk_tool benchmark --alg md5 --mode sva --opt 0 --sync --seconds 5 --thread 2 --multi 1 --ctxnum 6
  ```

* SM3性能测试

  测试sm3在SVA模式计算摘要的性能：

  ```shell
  uadk_tool benchmark --alg sm3 --mode sva --opt 0 --sync --seconds 5 --thread 2 --multi 1 --ctxnum 6
  ```

* SHA性能测试

  测试sha-512在SVA模式计算摘要的性能：

  ```shell
  uadk_tool benchmark --alg sha-512 --mode sva --opt 0 --sync --seconds 5 --thread 2 --multi 1 --ctxnum 6
  ```

* AES性能

  测试aes-128-cbc在SVA模式进行加密的性能：

  ```shell
  uadk_tool benchmark --alg aes-128-cbc --mode sva --opt 0 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

  测试aes-128-cbc在SVA模式进行解密的性能：

  ```shell
  uadk_tool benchmark --alg aes-128-cbc --mode sva --opt 1 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

* SM4性能测试

  测试sm4-128-ecb在SVA模式进行加密的性能：

  ```shell
  uadk_tool benchmark --alg sm4-128-ecb --mode sva --opt 0 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

  测试sm4-128-ecb在SVA模式进行解密的性能：

  ```shell
  uadk_tool benchmark --alg sm4-128-ecb --mode sva --opt 1 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

* DES性能测试

  测试3des-128-ecb在SVA模式进行加密的性能：

  ```shell
  uadk_tool benchmark --alg 3des-128-ecb --mode sva --opt 0 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

  测试3des-128-ecb在SVA模式进行解密的性能：

  ```shell
  uadk_tool benchmark --alg 3des-128-ecb --mode sva --opt 1 --sync --pktlen 1024 --seconds 5 --multi 1 --thread 2 --ctxnum 6
  ```

其他测试场景，可以使用`uadk_tool benchmark --help`查看参数和配置说明。

### 使用UADK engine

可以通过openssl命令行工具直接调用uadk engine。可以通过各个openssl工具的帮助菜单查看使用方式。

#### 4.2.1 开启环境变量

UADK engine支持环境变量配置功能，能够根据需要设置执行任务的队列数量，需要进行以下配置：

1. 修改openssl.cnf文件（一般在/usr/local/ssl/路径下），将以下内容添加到配置文件的开头：

   ```shell
   openssl_cnf = openssl_def
   [openssl_def]
   engines = engine_section
   [engine_section]
   uadk_engine = uadk_section
   [uadk_section]
   UADK_CMD_ENABLE_RSA_ENV = 1
   UADK_CMD_ENABLE_DH_ENV = 1
   UADK_CMD_ENABLE_CIPHER_ENV = 1
   UADK_CMD_ENABLE_DIGEST_ENV = 1
   ```

2. 使用export命令设置队列数量，命令如下：

   ```shell
   export WD_RSA_CTX_NUM="sync:2@0,async:4@0"
   export WD_DH_CTX_NUM="sync:2@0,async:4@0"
   export WD_CIPHER_CTX_NUM="sync:2@2,async:4@2"
   export WD_DIGEST_CTX_NUM="sync:2@2,async:4@2"
   ```

   环境变量的入参格式为`ctx_mode:ctx_num@node`，表示在索引号为node的numa节点上设置ctx\_num个ctx\_mode模式的队列。
   例如，`"sync:2@0,async:4@0"`表示，在numa 0节点上设置2个sync模式的队列和4个async模式的队列。

#### 功能测试

* RSA功能测试

  生成私钥：

  ```shell
  openssl genrsa -out prikey.pem -engine uadk_engine 1024
  ```

  获取公钥：

  ```shell
  openssl rsa -in prikey.pem -pubout -out pubkey.pem -engine uadk_engine
  ```

  假设需要加密的文件为plain.txt:

  ```shell
  echo "Content to be encrypted" > plain.txt
  ```

  加密：

  ```shell
  openssl rsautl -encrypt -in plain.txt -inkey pubkey.pem -pubin -out enc.txt -engine uadk_engine
  ```

  解密：

  ```shell
  openssl rsautl -decrypt -in enc.txt -inkey prikey.pem -out dec.txt -engine uadk_engine
  ```

  假设需要签名的文件为msg.txt:

  ```shell
  echo "Content to be signed" > msg.txt
  ```

  签名：

  ```shell
  openssl rsautl -sign -in msg.txt -inkey prikey.pem -out signed.txt -engine uadk_engine
  ```

  验签：

  ```shell
  openssl rsautl -verify -in signed.txt -inkey pubkey.pem -pubin -out verified.txt -engine uadk_engine
  ```

  使用openssl speed工具测试：

  ```shell
  openssl speed -elapsed -engine uadk_engine rsa1024
  openssl speed -elapsed -engine uadk_engine -async_jobs 10 rsa1024
  openssl speed -elapsed -engine uadk_engine -async_jobs 36 rsa1024
  ```

* DH功能测试

  生成全局公钥参数：

  ```shell
  openssl dhparam -out dhparam.pem 768
  ```

  生成Alice私钥：

  ```shell
  openssl genpkey -paramfile dhparam.pem -out alice_prikey.pem -engine uadk_engine
  ```

  获取Alice公钥：

  ```shell
  openssl pkey -in alice_prikey.pem -pubout -out alice_pubkey.pem
  ```

  生成Bob私钥：

  ```shell
  openssl genpkey -paramfile dhparam.pem -out bob_prikey.pem -engine uadk_engine
  ```

  获取Bob公钥：

  ```shell
  openssl pkey -in bob_prikey.pem -pubout -out bob_pubkey.pem
  ```

  交换公钥，各自生成协商出的密钥：

  ```shell
  openssl pkeyutl -derive -inkey alice_prikey.pem -peerkey bob_pubkey.pem -out secret1.bin -engine uadk_engine
  openssl pkeyutl -derive -inkey bob_prikey.pem -peerkey alice_pubkey.pem -out secret2.bin -engine uadk_engine
  ```

  对比协商出的共享密钥：

  ```shell
  cmp secret1.bin secret2.bin
  xxd secret1.bin
  xxd secret2.bin
  ```

* MD5功能测试

  假设需要计算摘要的文件为data.txt：

  ```shell
  echo "Content to be hashed" > data.txt
  ```

  计算摘要：

  ```shell
  openssl md5 -engine uadk_engine data.txt
  ```

  openssl speed工具测试：

  ```shell
  openssl speed -engine uadk_engine -async_jobs 1 -evp md5
  ```

* SM3功能测试

  假设需要计算摘要的文件为data.txt：

  ```shell
  echo "Content to be hashed" > data.txt
  ```

  计算摘要：

  ```shell
  openssl sm3 -engine uadk_engine data.txt
  ```

* SHA功能测试

  假设需要计算摘要的文件为data.txt：

  ```shell
  echo "Content to be hashed" > data.txt
  ```

  计算摘要：

  ```shell
  openssl sha1 -engine uadk_engine data.txt
  openssl sha256 -engine uadk_engine data.txt
  openssl sha512 -engine uadk_engine data.txt
  ```

* AES功能测试

  假设需要加密的文件为data.txt：

  ```shell
  echo "Content to be encrypted" > data
  ```

  aes-128-cbc模式加密：

  ```shell
  openssl enc -aes-128-cbc -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  aes-128-cbc模式解密：

  ```shell
  openssl enc -aes-128-cbc -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  aes-192-cbc模式加密：

  ```shell
  openssl enc -aes-192-cbc -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  aes-192-cbc模式解密：

  ```shell
  openssl enc -aes-192-cbc -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  aes-256-cbc模式加密：

  ```shell
  openssl enc -aes-256-cbc -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  aes-256-cbc模式解密：

  ```shell
  openssl enc -aes-256-cbc -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  aes-128-ecb模式加密：

  ```shell
  openssl enc -aes-128-ecb -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-128-ecb模式解密：

  ```shell
  openssl enc -aes-128-ecb -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-192-ecb模式加密：

  ```shell
  openssl enc -aes-192-ecb -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-192-ecb模式解密：

  ```shell
  openssl enc -aes-192-ecb -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-256-ecb模式加密：

  ```shell
  openssl enc -aes-256-ecb -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-256-ecb模式解密：

  ```shell
  openssl enc -aes-256-ecb -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-128-ctr模式加密：

  ```shell
  openssl enc -aes-128-ctr -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-128-ctr模式解密：

  ```shell
  openssl enc -aes-128-ctr -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-192-ctr模式加密：

  ```shell
  openssl enc -aes-192-ctr -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-192-ctr模式解密：

  ```shell
  openssl enc -aes-192-ctr -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-256-ctr模式加密：

  ```shell
  openssl enc -aes-256-ctr -a -in data -out data.en -pass pass:123456 -K abc -engine uadk_engine -p
  ```

  aes-256-ctr模式解密：

  ```shell
  openssl enc -aes-256-ctr -a -d -in data.en -out data.de -pass pass:123456 -K abc -engine uadk_engine -p
  ```

* SM4功能测试

  sm4-cbc模式加密：

  ```shell
  openssl enc -sm4-cbc -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  sm4-cbc模式解密：

  ```shell
  openssl enc -sm4-cbc -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  sm4-ecb模式加密：

  ```shell
  openssl enc -sm4-ecb -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  sm4-ecb模式解密：

  ```shell
  openssl enc -sm4-ecb -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

* DES功能测试

  des-ede3-cbc模式加密：

  ```shell
  openssl enc -des-ede3-cbc -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  des-ede3-cbc模式解密：

  ```shell
  openssl enc -des-ede3-cbc -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  des-ede3-ecb模式加密：

  ```shell
  openssl enc -des-ede3-ecb -a -in data -out data.en -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

  des-ede3-ecb模式解密：

  ```shell
  openssl enc -des-ede3-ecb -a -d -in data.en -out data.de -pass pass:123456 -K abc -iv abc -engine uadk_engine -p
  ```

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/ukui/ukui_user_guide.md
---
# UKUI Desktop Environment

## Overview

The desktop environment is the basis for GUI-based operations. UKUI provides multiple functions including taskbar and start menu. The home screen is shown in figure below.

![Fig. 1 Home screen](./figures/1.png)

## Desktop

### Desktop Icons

The Computer, Recycle Bin and Home folder icons are displayed on the desktop by default. Double click an icon to open the page. The functions are shown in table below.

| Icon                     | Description                                                        |
| :----------------------- | :----------------------------------------------------------------- |
| ![](./figures/icon1.png) | Computer: shows the drives and hardwares connected to the machine. |
| ![](./figures/icon2.png) | Recycle Bin: shows documents that have been deleted.               |
| ![](./figures/icon3.png) | Personal: shows the user's home directory.                         |

Right-clicking "Computer" and selecting "Properties" shows the current system version, kernel version, and other related information.

![Fig. 2 "Computer" - "Properties"-big](./figures/2.png)

### Context Menu

Right-click on the desktop blank area to display the context menu, as shown in figure below.

![Fig. 3 Right-click Menu](./figures/3.png)

Some of the options are described in table below.

| Option    | Description                                               |
| :-------- | :-------------------------------------------------------- |
| New       | Creates folders, documents, and WPS files.                |
| View type | Displays small, medium, large, or super large icons.      |
| Sort by   | Sorts files by name, type, size, or date of modification. |

## Taskbar

### Basic Function

The taskbar is located at the bottom and includes the start menu, multi view switch, file browser, Firefox, WPS, and tray.

![Fig. 4 Taskbar](./figures/4.png)

| Component                | Description                                                                                                                                 |
| :----------------------- | :------------------------------------------------------------------------------------------------------------------------------------------ |
| ![](./figures/icon4.png) | Start menu: Open the start menu to find applications and files.                                                                             |
| ![](./figures/icon5.png) | Multi view switch: Operate in multiple workspaces.                                                                                          |
| ![](./figures/icon6.png) | File browser: Browse and manage documents in the system.                                                                                    |
| ![](./figures/icon7.png) | Firefox: Access the Internet conveniently and safely.                                                                                       |
| ![](./figures/icon8.png) | WPS: Perform the most commonly used office operations to process text, tables, and presentations.                                           |
| Window Display Area      | The blank area in the middle of the task bar displays the running programs or opened documents, and allows you to close and top the window. |
| ![](./figures/icon9.png) | Tray: Change settings for sound, Kylin Weather, internet connection, input method, notification center, date, and night mode.               |
| Show Desktop             | The button on the far right is used to minimize all windows on the desktop or restore the windows.                                          |

#### Multi View Switch

Click the icon "![](./figures/icon10-o.png)" on the taskbar to enter the interface shown in figure below, and select the operation area that users need to work on at the moment in multiple work areas.

![Fig. 5 Multi View Switch-big](./figures/5.png)

#### Preview Window

Users move the mouse over the app icon in the taskbar, and then a small preview window will be shown if this app has already been opened.

Hover over the specified window as shown below for hover state, the window will be slightly fuzzy glass effect (left), the rest of the window as default Status (right).

![Fig. 6 Taskbar - Preview Window](./figures/6.png)

Users can close the application by right-clicking on the app icon in the taskbar.

![Fig. 7 Taskbar - Right-click Preview](./figures/7.png)

#### Sidebar

The sidebar is located at the right of the entire desktop. Click the icon "![](./figures/icon11-o.png)" in the taskbar tray menu to open the storage menu, and click the icon "![](./figures/icon12-o.png)" in Sidebar to pop up the sidebar as shown in figure below.

The sidebar consists of two parts: Notification Center, Clipboard and Widget.

![Fig. 8 Sidebar without message status-big](./figures/8.png)

##### Notification Center

Notification center will display a list of recent important and newest information.

Select "Clear" in the upper right corner to clear the list of information; Select "Setting" in the upper right corner to go to the notification settings in the control center, and users can set which applications can show information and the quantity of information.

![Fig. 9 Notification Center-big](./figures/9.png)

Workspace at right side can be set to fold by applications.

![Fig. 10 Fold messages by applications-big](./figures/10.png)

Icon "![](./figures/icon13-o.png)" at the top right corner of the sidebar can store unimportant information. When the messages are more than 999+, it will be shown as the form of ![](./figures/icon14-o.png) which means limitless.

![Fig. 11 Message Organizer](./figures/11.png)

##### Clipboard

Clipboard can save the contents those were recently selected to copy or cut, and users can operate them by using the icons in Table.

![Fig. 12 Clipboard](./figures/12.png)

Clicking "![](./figures/icon15-o.png)", users can edit the the contents of the clipboard.

![Fig. 13 edit the content](./figures/13.png)

| Icon                      | Description        | Icon                      | Description      |
| :------------------------ | :----------------- | :------------------------ | :--------------- |
| ![](./figures/icon16.png) | Copy the content   | ![](./figures/icon18.png) | Edit the content |
| ![](./figures/icon17.png) | Delete the content |                           |                  |

The second label of the clipboard is the small plug-in that contains alarm clock, sticky notes, and user feedback.

![Fig. 14 Plug-in](./figures/14.png)

#### Tray Menu

##### Storage Menu

Click "![](./figures/icon19-o.png)" at the tray menu to open the storage menu.

It contains Kylin Weather, Input Method, Bluetooth, USB, etc.

![Fig. 15 Storage Menu](./figures/15.png)

##### Input Method

The taskbar input method defaults to Sogou input method. Use the shortcut key "Ctrl+Space" to switch it out, and the "Shift" key to switch between Chinese and English modes.

![Fig. 16 Input Method](./figures/16.png)

##### USB

When a USB flash drive is inserted, the dat inside is automatically read.

Click "![](./figures/icon26-o.png)" to open the window as shown in figure below.

When users need to umount the USB, please click the icon "![](./figures/icon27-o.png)".

![Fig. 17 The status of USB](./figures/17.png)

##### Power Supply

Click the icon "![](./figures/icon28-o.png)":

When no power supply is detected.

![Fig. 18 No Power Supply](./figures/18.png)

When power supply is detected.

![Fig. 19 Have Power Supply](./figures/19.png)

Users right-click the icon "![](./figures/icon30-o.png)" of power manager to open the power setting menu.

It provides two setting options: adjust screen brightness, and set power and sleep.

![Fig. 20 Power Manager](./figures/20.png)

If the power manager pops up a"low battery" window, users can click to turn on the power save mode, and the power manager will set the machine to run in this mode immediately.

![Fig. 21 Power Saving Mode](./figures/21.png)

##### Network

Users can choose wired or wireless network connections by clicking the icon "![](./figures/icon31-o.png)" of network manager.

| Icon                      | Description        | Icon                      | Description             |
| :------------------------ | :----------------- | :------------------------ | :---------------------- |
| ![](./figures/icon32.png) | Connected          | ![](./figures/icon37.png) | Unconnected             |
| ![](./figures/icon33.png) | Connection limited | ![](./figures/icon38.png) | Locked                  |
| ![](./figures/icon34.png) | Connecting         | ![](./figures/icon39.png) | Wifi connected          |
| ![](./figures/icon35.png) | Wifi unconnected   | ![](./figures/icon40.png) | Wifi connection limited |
| ![](./figures/icon36.png) | Wifi locked        | ![](./figures/icon41.png) | Wifi connecting         |

![Fig. 22 Network Connection](./figures/22.png)

* Wired Network
  In the wired network connection interface, click on the wired network plan to expand. Details of the network.

  ![Fig. 23 Wired Network](./figures/23.png)

* Wireless Network
  Click the switch button in the upper right corner to turn on the wireless network connection, and select the WiFi from the list of available wireless networks. Enter the password to access the Internet.

  ![Fig. 24 Wireless Network](./figures/24.png)

* Network Setting
  Right-click the icon "![](./figures/icon42-o.png)" of network manager to pop up the setting menu.

  ![Fig. 25 Wired Network Setting](./figures/25.png)

  Click network setting to go to the setting window immediately.

  ![Fig. 26 Network Setting](./figures/26.png)

##### Volume

Click the icon "![](./figures/icon43-o.png)" to open the volume window, and there provides three modes.

* Mini Mode
  It only displays the volume of the speaker.

  ![Fig. 27 Mini Mode](./figures/27.png)

* According to Equipment
  It contains input equipment and output equipment.

  ![Fig. 28 According to Equipment List](./figures/28.png)

* According to Application
  It contains system volume and other applications' volume.

  ![Fig. 29 According to Application List](./figures/29.png)

##### Calendar

Click the date\&time on the taskbar to open the calendar window.

Users can view the day's information by filtering the year, month, day. The date will be displayed in large letters, with the time, the week, the festival,and the lunar calendar. Taboos can be seen by checking.

![Fig. 30 Calendar-big](./figures/30.png)

##### Night Mode

Click the icon "![](./figures/icon44-o.png)" on the Taskbar and then the system changes to the night mode.

#### Advanced Setting

Right-click the Taskbar to open the menu.

![Fig. 31 Right-Clicking Menu](./figures/31.png)

Users can set the layout of taskbar in "Taskbar Settings".

## Window

### Window Manager

The functions provided as shown in Table.

| Function                | Description                                                                                                  |
| :---------------------- | :----------------------------------------------------------------------------------------------------------- |
| Title Bar               | Show the title name of current window                                                                        |
| Minimize/Maximize/Close | The three icon buttons at the right of the title bar correspond to minimize, maximize and close respectively |
| Side Sliding            | Users can scroll up and down to view the page by the slider at the right of the window                       |
| Stack                   | Allow overlap among windows                                                                                  |
| Drag and Drop           | Long press the left mouse button at the title bar to move the window to any position                         |
| Resize                  | Move the mouse to the corner of the window and long press the left button to resize the window               |

### Window Switch

There are three ways to switch windows:

* Click the window title on the Taskbar.

* Click the different window at the desktop.

* Use shortcut keys < Alt > + < Tab >.

## Start Menu

### Basic Function

Click the button to open the "Start Menu".

It provides sliding bar.

![Fig. 32 Start Menu](./figures/32.png)

#### Category Menu at right side

When the mouse is over the right side of the start menu, it will appear a pre-expanded cue bar. Clicking to expand, and then three categories are showing at the right side by default: "Common Software", "Alphabetical Category", and "Functional category".

* All Software: List all software, recently used software will be displayed on the top of this page.

* Alphabetical Category: List all software by first letter.

* Functional category: List all software by their functions.

Users can click the button at top right corner to view full-screen menu mode.

![Fig. 33 Full-screen Menu-big](./figures/33.png)

#### Function Button at right side

It provides User Avatar, Computer, Control Center and Shutdown four options.

##### User Avatar

Click "![](./figures/icon45-o.png)" to view user's information.

##### Computer

Click "![](./figures/icon46-o.png)" to open personal home folder

##### Control Center

Click "![](./figures/icon47-o.png)" to go to the control center.

##### Shutdown

###### Lock Screen

When users do not need to use the computer temporarily, the lock screen can be selected (without affecting the current running state of the system) to prevent misoperations. And input the password to re-enter the system.

The system will automatically lock the screen after a period of idle time by default.

![Fig. 34 Lock Screen-big](./figures/34.png)

###### Switch Users & Log Out

When users want to select another user to log in using the computer, users can select "Log out" or "Switch user".

At this point, the system will close all running applications; Therefore, please save the current jobs before performing this action.

###### Shutdown & Reboot

There are two ways:

1\)"Start Menu" > "Power" > "Shutdown"

It will pop up a dialog box, and users can choose shutdown or reboot as needed.

![Fig. 35 Shutdown Dialog Box-big](./figures/35.png)

2\)"Start Menu" > right side menu of the "Shutdown" button > "Shutdown"/"Reboot"

The system will shutdown or reboot immediately without popping up the dialog box.

### Advanced Setting

Right-clicking Start Menu, it provides lock screen, switch user, log out, reboot, and shutdown five shortcut options.

### Applications

Users can search apps in the search box by key words. As shown in figure below, the result will show up automatically with the input.

![Fig. 36 Search Apps](./figures/36.png)

Right-clicking one app in the Start Menu, the right-click menu popping up.

![Fig. 37 Right-click Menu](./figures/37.png)

The options are described in table below.

| Option                   | Description                                                      |
| :----------------------- | :--------------------------------------------------------------- |
| Attach to "All Software" | Add the selected software to the top of the list of All Software |
| Attach to Taskbar        | Generate icon for the application on the Taskbar                 |
| Add to Desktop Shortcut  | Generate shortcut icon for the application on the desktop        |
| Uninstall                | Remove the application                                           |

## FAQ

### I Cannot Login to the System After Locking the Screen

* Switch to character terminal by pressing **Ctrl + Alt + F2**.

* Input the user-name and passwd to login to the system.

* Do "sudo rm -rf ~/.Xauthority".

* Switch to graphical interface by pressing **Ctrl + Alt + F1**, and input the password.

## Appendix

### Shortcut Key

| Shortcut Key        | Function            |
| :------------------ | :------------------ |
| F5                  | Refresh the desktop |
| F1                  | Open the user-guide |
| Alt + Tab           | Switch the window   |
| win                 | Open the Start Menu |
| Ctrl + Alt + L      | Lock Screen         |
| Ctrl + Alt + Delete | Log out             |

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/ukui/ukui_installation.md
---
# UKUI Installation

UKUI is a Linux desktop built by the KylinSoft software team over the years, primarily based on GTK and QT. Compared to other UI interfaces, UKUI is easy to use. The components of UKUI are small and low coupling, can run alone without relying on other suites. It can provide user a friendly and efficient experience.

UKUI supports both x86\_64 and aarch64 architectures.

You are advised to create an administrator user before installing UKUI.

1. Download openEuler 22.03 LTS SP4 and update the software source.

   ```shell
   sudo dnf update
   ```

2. Install UKUI.

   ```shell
   sudo dnf install ukui
   ```

3. If you want to set the system to start with the graphical interface after confirming the installation, run the following command and reboot the system (`reboot`).

   ```shell
   systemctl set-default graphical.target
   ```

UKUI is constantly updated. Please check the latest installation method:
openEuler UKUI Issues

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/ukui/ukui_user_guide.md
---
# UKUI 用户指南

## 概述

桌面是用户进行图形界面操作的基础，UKUI(UbuntuKylin UI)提供了多个功能部件，包括任务栏、开始菜单等，本文主要描述 UKUI 的使用。

主界面如下图所示。

![图 1 桌面主界面-big](./figures/1.png)

## 桌面

### 桌面图标

系统默认放置了计算机、回收站、主文件夹三个图标，鼠标左键双击即可打开页面，功能如下表。

| 图标 |  说明 |
| :------------ | :------------ |
| ![](./figures/icon1.png) | 计算机：显示连接到本机的驱动器和硬件|
| ![](./figures/icon2.png) | 回收站：显示移除的文件|
| ![](./figures/icon3.png) | 主文件夹：显示个人主目录|

另外，右键单击“计算机”，选择“属性”，可显示当前系统版本、内核版本等相关信息。

![图 2 “计算机”-“属性”-big](./figures/2.png)

### 右键菜单

在桌面空白处单击鼠标右键，出现的菜单如下图所示，为用户提供了一些快捷功能。

![图 3 右键菜单](./figures/3.png)

部分选项说明如下表。

| 选项 | 说明|
| :------------ | :------------ |
| 新建 | 可新建文件夹、文本文档、WPS文件 |
| 视图类型 | 提供四种视图类型：小图标、中图标、大图标、超大图标 |
| 排序方式 | 提供根据文件名称、文件类型、文件大小、修改日期排列的四种方式|

## 任务栏

### 基本功能

任务栏位于底部，包括开始菜单、多视图切换、文件浏览器、Firefox网络浏览器、WPS、托盘菜单。

![图 4 任务栏](./figures/4.png)

| 组件| 说明 |
| :------------ | :------------ |
|![](./figures/icon4.png)| 开始菜单，用于弹出系统菜单，可查找应用和文件 |
|![](./figures/icon5.png)| 多视图切换，可在多个工作区互不干扰进行操作|
|![](./figures/icon6.png)| 文件浏览器，可浏览和管理系统中的文件|
|![](./figures/icon7.png)| Firefox网页浏览器，提供便捷安全的上网方式|
|![](./figures/icon8.png)| WPS办公套件，可以实现办公软件最常用的文字、表格、演示等多种功能|
|窗口显示区 |横条中间空白部分；显示正在运行的程序或打开的文档，可进行关闭窗口、窗口置顶操作。|
|![](./figures/icon9.png)| 托盘菜单，包含了对声音、麒麟天气、网络连接、输入法、通知中心、日期、夜间模式的设置|
|显示桌面| 按钮位于最右侧；最小化桌面的所有窗口，返回桌面；再次单击将恢复窗口|

#### 多视图切换

点击任务栏“![](./figures/icon10-o.png)”图标，即可进入如下图所示界面，在多个工作区内选择当下需要工作的操作区。

![图 5 多视图切换-big](./figures/5.png)

#### 预览窗口

用户将鼠标移动到任务栏的应用图标上，会对该应用打开的窗口进行小窗口预览，悬停在指定窗口如下图所示为悬停状态，该窗口会微微呈现毛玻璃效果（左），其余窗口为默认状态（右）。

![图 6 任务栏预览窗口](./figures/6.png)

用户通过鼠标右键点击任务栏的应用图标，可关闭该应用。

![图 7 任务栏右键预览](./figures/7.png)

#### 侧边栏

侧边栏位于整个桌面的右侧，点击任务栏托盘菜单中的“![](./figures/icon11-o.png)”图标打开收纳菜单，点击侧边栏“![](./figures/icon12-o.png)”图标，弹出侧边栏如下图所示。

侧边栏由两部分构成：通知中心、剪切板和小插件。

![图 8 侧边栏无消息状态-big](./figures/8.png)

##### 通知中心

通知中心将会显示近期最新的重要信息列表，选择右上角“清空”可将信息列表清空；用户可通过选择右上角“设置”跳转进入控制面板的通知设置界面，能设置显示信息的应用，以及信息的数量。

![图 9 通知中心-big](./figures/9.png)

右侧工作区可设置为按应用折叠的模式。

![图 10 按应用折叠通知消息-big](./figures/10.png)

侧边栏右上角“![](./figures/icon13-o.png)”图标可收纳不重要信息，可以打开不重要的和已被设置为收纳的应用软件信息，消息超过999+后显示成![](./figures/icon14-o.png)的形式表示无穷大。

![图 11 消息收纳箱](./figures/11.png)

##### 剪切板

剪切板可保存近期选择复制或剪切的内容，同时可通过表上说明的图标进行相应操作。

其中点击“![](./figures/icon15-o.png)”图标，可对剪切板的内容进行编辑。

|图标| 说明| 图标 |说明 |
| :------------ | :------------ | :------------ | :------------ |
|![](./figures/icon16.png)| 复制剪切板上的该内容 |![](./figures/icon18.png)| 编辑剪切板上的该内容 |
|![](./figures/icon17.png)| 删除剪切板上的该内容 |  |  |

![图 12 剪切板](./figures/12.png)

![图 13 编辑选中实的剪切板内容](./figures/13.png)

剪切板的第二个标签为小插件，插件包含：闹钟、麒麟便签本、用户反馈，可供用户快捷选择。

![图 14 小插件](./figures/14.png)

#### 托盘菜单

##### 收纳菜单

点击任务栏托盘菜单中的“![](./figures/icon19-o.png)”图标打开收纳菜单，收纳菜单中可收纳麒麟天气、输入法、蓝牙、u盘等小工具。

![图 15 收纳菜单](./figures/15.png)

##### 输入法

任务栏输入法默认为搜狗输入法，使用快捷键“Ctrl+Space”可切换出来，“Shift”按键切换中英文模式。

![图 16 输入法](./figures/16.png)

##### U盘

U盘插入主机后，自动读取U盘数据，点击任务栏中U盘“![](./figures/icon26-o.png)”图标弹窗如下图所示。

需要卸载U盘时仅需点击弹出“![](./figures/icon27-o.png)”图标即可。

![图 17 U盘状态窗口](./figures/17.png)

##### 电源

没有检测到电源设备时，用户通过点击鼠标左键任务栏中电源“![](./figures/icon28-o.png)”图标。

![图 18 无电源设备](./figures/18.png)

若检测到接入的电源设备，用户通过点击鼠标左键任务栏中电源“![](./figures/icon29-o.png)”图标。

![图 19 电源管理器窗口](./figures/19.png)

用户通过点击鼠标右键任务栏中电源“![](./figures/icon30-o.png)”图标，弹出电源管理器设置菜单，设置调整屏幕亮度、设置电源和休眠两项。

![图 20 电源管理器设置](./figures/20.png)

若电源管理器弹出“电池电量不足”的弹窗后，用户可点击开启节能模式，电源管理器则即刻将本机设为节能模式运行。

![图 21 电池电量不足开启节能模式](./figures/21.png)

##### 网络

用户通过鼠标左键点击任务栏上的网络“![](./figures/icon31-o.png)”图标，可根据需要选择有线和无线两种网络连接方式。

|图标 |说明| 图标 |说明 |
| :------------ | :------------ | :------------ | :------------ |
|![](./figures/icon32.png)| 网络已连接 |![](./figures/icon37.png)| 网络未连接 |
|![](./figures/icon33.png)| 网络连接受限 |![](./figures/icon38.png)| 网络已上锁 |
|![](./figures/icon34.png)| 网络正在连接 |![](./figures/icon39.png)| Wifi已连接 |
|![](./figures/icon35.png)| Wifi未连接 |![](./figures/icon40.png)| Wifi连接受限 |
|![](./figures/icon36.png)| Wifi已上锁 |![](./figures/icon41.png)| Wifi正在连接 |

![图 22 网络连接界面](./figures/22.png)

* 有线网络

  在有线网络连接界面，点击有线网络方案即可展开，查看网络的详细信息。

  ![图 23 有线网络连接](./figures/23.png)

* 无线网络

  无线网络连接，点击右上角开关按钮打开无线网络连接，并在可用无线网络列表中选择需要连接的WiFi，并键入密码即可通过WiFi上网。

  ![图 24 无线网络连接](./figures/24.png)

* 网络设置窗口

  用户通过鼠标右键点击任务栏上的网络“![](./figures/icon42-o.png)”图标，弹出网络设置菜单。

  ![图 25 有线网络设置](./figures/25.png)

  点击设置网络，即刻进入网络设置窗口。

  ![图 26 网络设置窗口](./figures/26.png)

##### 音量

用户通过鼠标左键点击任务栏上的音量“![](./figures/icon43-o.png)”图标，打开声音界面。

* mini模式

  音量mini模式，仅显示扬声器的音量。

  ![图 27 音量mini模式](./figures/27.png)

* 按设备

  音量按设备标签包括输出设备、输入设备。

  ![图 28 按设备音量列表](./figures/28.png)

* 按应用

  音量按应用标签包括系统音量、其他应用音量。

  ![图 29 按应用音量列表](./figures/29.png)

##### 日历

用户通过鼠标左键点击任务栏上的时间日期弹出日历窗口，查看日历、月历、年历窗口。

用户可通过筛选年 > 月 > 日查看一日信息，会以大字显示当日日期，并有当日的时间、星期、节气、农历，点击下方宜忌勾选可查看。

![图 30 日历查看-big](./figures/30.png)

##### 夜间模式

用户通过鼠标左键点击任务栏上的夜间模式“![](./figures/icon44-o.png)”图标，可设置为夜间模式。

#### 高级设置

右键单击任务栏，出现的菜单。

![图 31 任务栏右键菜单](./figures/31.png)

用户可对任务栏的布局进行设定，在“设置任务栏”中可进行相关设置。

## 窗口

### 窗口管理器

窗口管理器为用户提供了如表所示的功能。

|功能|说明 |
| :--------| :----------|
|窗口标题栏| 显示当前窗口的标题名称 |
|最小化/最大化/关闭 |标题栏右侧的三个图标按钮，分别对应最小化窗口、最大化窗口、关闭窗口的功能 |
|侧边滑动 |在窗口右侧提供滑动条，可上下滚动查看页面 |
|窗口堆叠| 允许窗口之间产生重叠 |
|窗口拖拽 |在窗口标题栏长按鼠标左键，可移动窗口到任意位置 |
|窗口大小调整 |将鼠标移至窗口四角，长按左键，可任意调整窗口大小 |

### 窗口切换

用户有三种方式可以切换：

* 在任务栏上点击窗口标题；

* 在桌面上点击不同窗口；

* 使用快捷键**Alt** + **Tab**；

## 开始菜单

### 基本功能

单击“开始菜单”按钮，菜单具备滑动条功能。

![图 32 开始菜单主界面](./figures/32.png)

#### 右侧分类菜单

用户将鼠标停留在开始菜单右侧，会出现一个右侧预展开的提示栏，点击展开，即在右侧默认显示三个分类：“所有软件”、“字母分类”、“功能分类”，其中：

* 所有软件：列出所有软件，近期使用过的软件将会在此页面置顶显示。

* 字母分类：列出系统根据首字母进行分类显示所有软件。

* 功能分类：列出系统根据功能进行分类显示所有软件。

用户可通过点击右上角开始菜单的全屏图标，查看全屏菜单。

![图 33 全屏开始菜单-big](./figures/33.png)

#### 右侧功能键

右下侧显示用户头像、计算机、设置和电源四个选项。

##### 用户头像

点击“![](./figures/icon45-o.png)”图标，进入控制面板查看用户信息。

##### 计算机

点击“![](./figures/icon46-o.png)”图标进入计算机：个人主文件夹。

##### 设置

点击“![](./figures/icon47-o.png)”图标进入控制面板。

##### 电源

###### 锁定屏幕

当用户暂时不需要使用计算机时，可以选择锁屏（不会影响系统当前的运行状态），防止误操作；用户返回后，输入密码即可重新进入系统。

在默认设置下，系统在一段空闲时间后，将自动锁定屏幕。

锁屏界面如下图所示。

![图 34 锁屏界面-big](./figures/34.png)

###### 切换用户和注销

当要选择其他用户登录使用计算机时，可选择“注销”或“切换用户”。

此时，系统会关闭所有正在运行的应用；所以，在执行此操作前，请先保存当前工作。

###### 关机与重启

有两种操作方式：

1）“开始菜单” > “电源” > “关机”

会弹出对话框，用户可根据需要选择重启或关机。

![图 35 关闭系统对话框-big](./figures/35.png)

2）“开始菜单” > “关机” 按钮右边菜单 > “关机”/“重启”

系统将直接关机/重启，不再弹出对话框。

### 高级设置

右键单击开始菜单图标，提供锁屏、切换用户、注销、重启、关闭五个快捷选项。

### 应用

用户可以在搜索框中，通过关键字搜索应用。如下图所示，可输入中文，如：搜索用户手册，查询结果会随着输入自动显示出来。

![图 36 搜索应用](./figures/36.png)

通过右键点击开始菜单中的某个应用，弹出右键菜单，可将选中应用固定到“所有软件”、任务栏，可添加该应用到桌面方式，可快捷卸载该应用。

![图 37 应用的右键菜单](./figures/37.png)

各个选项说明如下表。

| 选项               | 说明                           |
| :----------------- | :----------------------------- |
| 固定到所有用软件   | 将选中软件在所有软件列表中置顶 |
| 固定到任务栏       | 在任务栏上生成应用的图标       |
| 添加到桌面快捷方式 | 在桌面生成应用的快捷方式图标   |
| 卸载               | 卸载软件                       |

## 常见问题

### 锁屏后无法登录系统

* 通过“Ctrl + Alt + F2”切换到字符终端。

* 输入用户名和密码后登录。

* 执行命令“sudo rm -rf ~/.Xauthority”。

* 通过“Ctrl + Alt + F1”切回图形界面，输入用户密码登录。

## 附录

### 快捷键

|快捷键|功能 |
| :------ | :-----  |
|F5| 刷新桌面 |
|F1 |打开用户手册 |
|Alt + Tab |切换窗口 |
|win |打开开始菜单 |
|Ctrl + Alt + L| 锁屏 |
|Ctrl + Alt + Delete| 注销 |

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/uninstallation.md
---
# Uninstallation

To uninstall iSulad, perform the following operations:

1. Uninstall iSulad and its dependent software packages.
   * If the  **yum**  command is used to install iSulad, run the following command to uninstall iSulad:

     ```sh
     # yum remove iSulad
     ```

   * If the  **rpm**  command is used to install iSulad, uninstall iSulad and its dependent software packages. Run the following command to uninstall an RPM package.

     ```sh
     # rpm -e iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm
     ```

2. Images, containers, volumes, and related configuration files are not automatically deleted. The reference command is as follows:

   ```sh
   # rm -rf /var/lib/iSulad
   ```

---

---
url: /en/docs/22.03_LTS_SP4/embedded/uniproton/uniproton_apis.md
---
# UniProton APIs

This document is currently not available in English.

---

---
url: /en/docs/22.03_LTS_SP4/embedded/uniproton/uniproton_functions.md
---
# UniProton Feature Design

## Task Management

UniProton is a single-process multi-thread operating system (OS). In UniProton, a task represents a thread. Tasks in UniProton are scheduled in preemption mode instead of time slice rotation scheduling. High-priority tasks can interrupt low-priority tasks. Low-priority tasks can be scheduled only after high-priority tasks are suspended or blocked.

A total of 32 priorities are defined, with priority 0 being the highest and 31 being the lowest. Multiple tasks can be created in a priority.

The task management module of UniProton provides the following functions:  Creates, deletes, suspends, resumes, and delays tasks; Locks and unlocks task scheduling; Obtains the current task ID; Obtains and sets task private data; Query the pending semaphore ID of a specified task; Query the status, context, and general information of a specified task; Obtains and sets task priorities; Adjusts the task scheduling order of a specified priority; Register and unregister hooks for task creation, deletion, and switching. During initialization, UniProton creates an idle task with the lowest priority by default. When no task is in the running status, the idle task is executed.

## Event Management

The event mechanism enables communication between threads. Event communication can only be event notifications and no data is transmitted.

As an extension of tasks, events allow tasks to communicate with each other. Each task supports 32 event types, each represented by a bit of a 32-bit value.

UniProton can read current task events and write specified task events. Multiple event types can be read or written at one time.

## Queue Management

A queue, also called message queue, is a method commonly used for inter-thread communication to store and transfer data. Data can be written to the head or tail of a queue based on the priority, but can be read only from the head of a queue.

When creating a queue, UniProton allocates memory space for the queue based on the queue length and message unit size input by the user. The queue control block contains **Head** and **Tail** pointers, which indicate the storage status of data in a queue. **Head** indicates the start position of occupied message nodes in the queue. **Tail** indicates the end position of the occupied message nodes in the queue.

## Hard Interrupt Management

A hardware interrupt is a level signal that is triggered by hardware and affects system running. A hardware interrupt is used to notify the CPU of a hardware event. Hardware interrupts include maskable interrupts and non-maskable interrupts (NMIs).

Hardware interrupts have different internal priorities, but they all have a higher priority than other tasks. When multiple hardware interrupts are triggered at the same time, the hardware interrupt with the highest priority is always responded first. Whether a high-priority hardware interrupt can interrupt a low-priority hardware interrupt that is being executed (that is, nested interrupts) depends on the chip platform.

The OS creates a tick hardware interrupt during initialization for task delay and software timer purposes. The tick is essentially a hardware timer.

## Memory Management

Memory management is to dynamically divide and manage large memory areas allocated by users. When a section of a program needs to use the memory, the program calls the memory application function of the OS to obtain the memory block of a specified size. After using the memory, the program calls the memory release function to release the occupied memory.

UniProton provides the FSC memory algorithm. The following table lists the advantages, disadvantages, and application scenarios of FSC.

| Algorithm                                                    | Advantages                                                        | Disadvantages                          | Application Scenarios                            |
| :----------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------ | ------------------------------------ |
| Private FSC algorithm| The memory control block information occupies a small amount of memory. The minimum 4-byte-aligned memory block size can be applied for. Adjacent memory blocks can be quickly split and merged without creating memory fragmentation.| The efficiency of memory application and release is low.| It can flexibly adapt to various product scenarios.|

The FSC memory algorithm is described as follows:

### FSC Memory Algorithm

#### Core Idea

The size of the requested memory is **uwSize**. If the size is in binary, it is expressed as **0b{0}1xxx**. **{0}** indicates that there may be one or more zeros before **1**. Regardless of the content of following **1** (**xxx**), if **1** is changed to **10** and **xxx** is changed to **0**, **10yyy** is always greater than **1xxx** (**yyy** indicates that the corresponding bits of **xxx** are changed to **0**).

The subscript of the leftmost 1 can be directly obtained. The subscript values are 0 to 31 from the most significant bit to the least significant bit (BitMap), or 0 to 31 from the least significant bit to the most significant bit (uwSize). If the subscripts of the bits of the 32-bit register are 0 to 31 from the most significant bit to the least significant bit, the subscript of the leftmost 1 of 0x80004000 is 0. Therefore, we can maintain an idle linked list header array (the number of elements does not exceed 31). The subscript of the leftmost 1 of the memory block size is used as the index of the linked list header array. That is, all memory blocks with the same subscript of the leftmost 1 are mounted to the same idle linked list.

For example, the sizes of idle blocks that can be mounted to the linked list whose index is 2 are 4, 5, 6, and 7, and the sizes of idle blocks that can be mounted to the linked list whose index is N are 2^N to 2^(N+1)-1.

![](./figures/FCS.png)

#### Memory Application

When applying for the memory of uwSize, use assembly instructions to obtain the subscript of the leftmost 1 first. Assume that the subscript is **n**. To ensure that the first idle memory block in the idle linked list meets the uwSize requirement, the search starts from the index n+1. If the idle linked list of index n+1 is not empty, the first idle block in the linked list is used. If the linked list of n+1 is empty, the linked list of n+2 is checked, and so on, until a non-empty linked list is found or the index reaches 31.

A 32-bit BitMap global variable is defined to prevent the for loop from checking whether the idle linked list is empty recursively. If the idle linked list of n is not empty, the value whose subscript is n of BitMap is set to 1. Otherwise, the value is set to 0. The bit whose subscript is 31 of the BitMap is directly set to 1 during initialization. Therefore, the first non-idle linked list is searched from linked list of n+1. Bits 0 to n of the BitMap copy can be cleared first, and then a subscript of the leftmost 1 of the copy is obtained. If the subscript is not equal to 31, the subscript is the array index of the first non-empty idle linked list.

All idle blocks are connected in series in the form of a bidirectional idle linked list. If the first idle block obtained from the linked list is large, that is, after a usSize memory block is split, the remaining space can be allocated at least once, The remaining idle blocks are added to the corresponding idle linked list.

![](./figures/MemoryApplication.png)

The memory control header records the size of the idle memory block (including the control header itself). The memory control header contains a reused member at the beginning. When a memory block is idle, it is used as a pointer to the next idle memory block. When a memory block is occupied, it stores a magic number, indicating that the memory block is not idle. To prevent the magic number from conflicting with the pointer (same as the address value), the upper and lower four bits of the magic number are 0xf. The start addresses of the allocated memory blocks are 4-byte-aligned. Therefore, no conflict occurs.

#### Memory Release

When the memory is released, adjacent idle blocks are combined. First, the validity of the address parameter (**pAddr**) is determined by checking the magic number in the control header. The start address of the control header of the next memory block is obtained by adding the start address to the offset value. If the next memory block is idle, the next memory block is deleted from the idle linked list to which it belongs, and the size of the current memory block is adjusted.

To quickly find the control header of the previous memory block and determine whether the previous memory block is idle during memory release, a member is added to the memory control header to mark whether the previous memory block is idle. When the memory is applied for, the flag of the next memory block can be set to the occupied state (if the idle memory block is divided into two, and the previous memory block is idle, the flag of the current memory block is set to the idle state). When the memory is released, the flag of the next memory block is set to the idle state. When the current memory is released, if the previous memory block is marked as occupied, the previous memory block does not need to be merged; if the previous memory block is marked as idle, the previous memory block needs to be merged. If a memory block is idle, the flag of the next control block is set to the distance to the current control block.

![](./figures/MemoryRelease.png)

## Timer Management

UniProton provides the software timer function to meet the requirements of timing services.

Software timers are based on the tick interrupts. Therefore, the period of a timer must be an integral multiple of the tick. The timeout scanning of the software timer is performed in the tick handler function.

Currently, the software timer interface can be used to create, start, stop, restart, and delete timers.

## Semaphore Management

A semaphore is typically used to coordinate a group of competing tasks to access to critical resources. When a mutex is required, the semaphore is used as a critical resource counter. Semaphores include intra-core semaphores and inter-core semaphores.

The semaphore object has an internal counter that supports the following operations:

* Pend: The Pend operation waits for the specified semaphore. If the counter value is greater than 0, it is decreased by 1 and a success message is returned. If the counter value of the semaphore is 0, the requesting task is blocked until another task releases the semaphore. The amount of time the task will wait for the semaphore is user configurable.

* Post: The Post operation releases the specified semaphore. If no task is waiting for the semaphore, the counter is incremented by 1 and returned. Otherwise, the first task (the earliest blocked task) in the list of tasks pending for this semaphore is woken up.

The counter value of a semaphore corresponds to the number of available resources. It means mutually exclusive resources remained that could be occupied. The counter value can be:

* 0, indicating that there is no accumulated post operation, and there may be a task blocked on the semaphore.

* A positive value, indicating that there are one or more post release operations.

## Exception Management

Exception takeover of UniProton is a maintenance and test feature that records as much information as possible when an exception occurs to facilitate subsequent fault locating. In addition, the exception hook function is provided so that users can perform special handling when an exception occurs. The exception takeover feature handles internal exceptions and external hardware exceptions.

## CPU Usage Statistics

The system CPU usage (CPU percentage, CPUP) in UniProton refers to the CPU usage of the system within a period of time. It reflects the CPU load and the system running status (idle or busy) in the given period of time. The valid range of the system CPUP is 0 to 10000, in basis points. 10000 indicates that the system is fully loaded.

The thread CPUP refers to the CPU usage of a single thread. It reflects the thread status, busy or idle, in a period of time. The valid range of the thread CPUP is 0 to 10000, in basis points. 10000  indicates that the process is being executed for a period of time. The total CPUPs of all threads (including interrupts and idle tasks) in a single-core system is 10000.

The system-level CPUP statistics of UniProton depends on the tick module, which is implemented by tick sampling idle tasks or idle software interrupt counter.

## STM32F407ZGT6 Development Board Support

The kernel peripheral startup process and board driver of UniProton supports the STM32F407ZGT6 development board. The directory structure is as follows:

├─apps         # Demo based on the real-time OS of UniProton
│ └─hello\_world   # hello\_world example program
├─bsp          # Board-level driver to interconnect with the OS
├─build          # Build script to build the final image
├─config         # Configuration items to adjust running parameters
├─include         # APIs provided by the real-time OS of UniProton
└─libs          # Static libraries of the real-time OS of UniProton. The makefile example in the build directory has prepared the reference of the header file and static libraries.

## OpenAMP Hybrid Deployment

OpenAMP is an open source software framework designed to standardize the interaction between environments in heterogeneous embedded systems through open source solutions based on asymmetric multi-processing. OpenAMP consists of the following components:

1. Remoteproc manages the life cycle of the slave core, shared memory, and resources such as buffer and vring used for communication, and initializes RPMsg and virtio.
2. RPMsg enables multi-core communication based on virtio.
3. Virtio, which is a paravirtualization technology, uses a set of virtual I/Os to implement driver communication between the master and slave cores.
4. libmetal shields OS implementation details, provides common user APIs to access devices, and handles device interrupts and memory requests.

## POSIX Standard APIs

[UniProton supports POSIX standard APIs](./uniproton_apis.md).

## Device Drivers

UniProton's driver architecture follows a Linux-like approach, treating devices as files through its Virtual File System (VFS). Drivers register with the file system via registration interfaces, enabling applications to access hardware through standard system calls. Adapted from Nuttx's open-source RTOS driver module, the framework maintains Nuttx-compatible interfaces. The file\_operations structure (defined in fs.h) stores device operation methods, while register\_driver associates devices with their inode structures that describe node locations and data. System calls reference these inodes to locate corresponding driver functions. For interface specifications, see [UniProton APIs](./uniproton_apis.md).

## Shell Commands

UniProton features a shell interface for command-line interaction with OS services, parsing user input and processing system outputs. Adapted from LiteOS's shell module, it supports custom command creation (requiring recompilation). Current implementation includes only the help command, with more commands planned for future releases.

| Interface | Description |
| :---: | :--: |
| SHELLCMD\_ENTRY | Static command registration |
| osCmdReg | Dynamic command registration |

Static registration (5 parameters) typically handles system commands, while dynamic registration (4 parameters) manages user commands. Both share four common parameters after the static method's unique first parameter. Details in [UniProton APIs](./uniproton_apis.md).

---

---
url: /en/docs/22.03_LTS_SP4/embedded/uniproton/overview.md
---
# UniProton User Guide

## Introduction

UniProton is an operating system (OS) for embedded scenarios provided by the openEuler community. It aims to build a high-quality OS platform that shields underlying hardware differences for upper-layer service software and provides powerful debugging functions. UniProton allows service software to be quickly ported to different hardware platforms, facilitates chip selection, and reduces costs for hardware procurement and software maintenance.

This document describes the basic functions and APIs of UniProton.

## Compilation

For details about compilation, see <https://atomgit.com/openeuler/UniProton/blob/master/doc>.

---

---
url: /zh/docs/22.03_LTS_SP4/embedded/uniproton/uniproton_functions.md
---
# UniProton 功能设计

## 支持任务管理

UniProton 是一个单进程支持多线程的操作系统。在 UniProton 中，一个任务表示一个线程。UniProton 中的任务为抢占式调度机制，而非时间片轮转调度方式。高优先级的任务可打断低优先级任务，低优先级任务必须在高优先级任务挂起或阻塞后才能得到调度。

UniProton 的任务一共有32个优先级(0-31)，最高优先级为0，最低优先级为31。每个优先级可以创建多个任务。

UniProton 任务管理模块提供任务创建、任务删除、任务挂起、任务恢复、任务延时、锁任务调度、解锁任务调度、当前任务ID获取、任务私有数据获取与设置、查询指定任务正在 Pending 的信号量 ID、查询指定任务状态、上下文信息、任务通用信息、任务优先级设定与获取、调整指定优先级的任务调度顺序、注册及取消任务创建钩子、任务删除钩子、任务切换钩子等功能。UniProton 在初始化阶段，默认会创建一个最低优先级的 IDLE 任务，用户在没有处于运行态的任务时，IDLE 任务被运行。

## 支持事件管理

事件机制可以实现线程之间的通讯。事件通讯只能是事件类型的通讯，无数据传输。

UniProton 事件作为任务的扩展，实现任务之间的通讯。每个任务支持32种类型事件（32个 bit 位，每 bit 代表一种事件类型）。

UniProton 提供读取本任务事件和写指定任务事件的功能。读事件时可以同时读取多种事件，也可以只读取一种事件，写事件时也可以同时写一种或多种类型事件。

## 支持队列管理

队列（Queue），又称消息队列，是线程间实现通信的一种方式，实现了数据的存储和传递功能。根据优先级可以将数据写入到队列头或队列尾，但只能从队列的头处读取数据。

UniProton 创建队列时，根据用户传入队列长度和消息单元大小来开辟相应的内存空间以供该队列使用。在队列控制块中维护一个头指针 Head 和一个尾指针 Tail 来表示当前队列中数据存储情况。头指针 Head 表示队列中被占用消息的起始地址，尾指针 Tail 表示队列中空闲消息的起始地址。

## 支持硬中断管理

硬中断是由硬件触发的会改变系统运行轨迹的一个电平信号，硬中断用于通知 CPU 某个硬件事件的发生。硬中断一般分为可屏蔽中断和不可屏蔽中断（NMI）两种。

硬中断的优先级高于所有任务，其内部也有不同的优先级，当同时有多个硬中断被触发时，最高优先级的硬中断总是优先得到响应。高优先级硬中断是否能打断正在执行的低优先级硬中断（即中断嵌套），视不同芯片平台而异。

出于任务延时、软件定时器等需要，OS 会在初始化阶段，创建1个 Tick 硬中断，其实质是一个周期性的硬件定时器。

## 支持内存管理

内存管理主要工作是动态的划分并管理用户分配好的大片内存区间。当程序某一部分需要使用内存，可以通过操作系统的内存申请函数索取指定大小内存块，一旦使用完毕，通过内存释放函数归还所占用内存，使之可以重复使用。

目前 UniProton 提供了 FSC 内存算法，该算法优缺点及应用场景如下表所示：

| 内存算法                                                     | 优点                                                         | 缺点                           | 应用场景                             |
| :----------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------ | ------------------------------------ |
| 类型私有 FSC 算法 | 内存控制块信息占用内存较少，支持最小4字节对齐的内存块大小申请；支持相邻内存块的快速分割合并，无内存碎片。 | 内存申请和内存释放的效率较低。 | 能够灵活适应各种产品的场景。 |

如下简要描述一下FSC内存算法：

### FSC内存算法

#### 核心思想

对于申请的内存大小为 uwSize，如果用二进制，则表示为 0b{0}1xxx，{0}表示1前面可能有0个或多个零。无论1后面xxx为何内容，如果将1变成10，xxx 则全部变成0，则总会出现 10yyy > 1xxx（此处yyy表示xxx的对应位全部变成0）。

我们可以直接得到最左边的1的下标。下标值或者从高位到低位依次为0-31（BitMap），或者从低位到高位依次为0-31（uwSize）。如果32位寄存器从高位到低位的bit位的下标依次为0-31，则 0x80004000 的最左边1的下标为0。于是我们可以维护一个空闲链表头数组（元素数不超过31），以内存块大小最左边的1的下标做为链表头的数组索引，即将所有最左边的1的下标相同的内存块挂接在同一个空闲链表中。

如：索引为2的链表可挂接的空闲块大小为4、5、6、7；索引为N的链表可挂接的空闲块大小为2^N到2^(N+1)-1。

![](./figures/FCS.png)

#### 内存申请

当申请 uwSize 大小的内存时，首先利用汇编指令得到最左边的1的下标，假定为n。为确保空闲链表中的第一个空闲内存块满足 uwSize，从索引为 n+1 开始搜索。若 n+1 所属空闲链表不为空，则取该链表中的第一个空闲块。若 n+1 链表为空，则判断 n+2 链表，依次类推，直到找到非空链表或索引达到31。

为避免 for 循环逐级判断空闲链表是否为空，定义一个32位的 BitMap 全局变量。若索引n的空闲链表非空，则 BitMap 的下标为n的位置1，否则清0。BitMap 的下标为31的位在初始化时直接置1。于是查找从 n+1 开始的第一个非空闲链表，可以首先将 BitMap 复本的0到n位清零，然后获取复本的最左边的1的下标，若不等于31，即为第一个空闲链表非空的数组索引。

所有的空闲块都以双向链表形式，串接在空闲链表中。若从链表中获取的第一个空闲块比较大，即分割出一个 usSiz e的内存块后，剩下的空间至少可做一次最小分配。则将剩余的空闲块调整到对应的空闲链表中。

![](./figures/MemoryApplication.png)

内存控制头中记录有空闲内存块的大小（包括控制头本身）。内存控制头中有一个复用成员，位于最首部时。当内存块空闲时，作为指向后一个空闲内存块的指针；当内存块占用时，存放魔术字，表示该内存块非空闲。为避免魔术字与指针冲突（与地址值相同），高低4位均为0xf。因为已分配的内存块起始地址需按4字节对齐，所以不存在冲突。

#### 内存释放

当释放内存时，需要将前后相邻的空闲块进行合并。首先，通过判断控制头中的魔术字，确认地址参数（pAddr）的合法性。通过首地址加偏移值的方式，得到后邻的内存块控制头的起始地址。若后邻内存块是空闲的，则将后邻内存块从所属空闲链表中删除，调整当前内存块的大小。

为了使内存释放时能迅速找到前邻的内存块控制头，及判断前邻的内存块是否空闲。内存控制头中增加一个成员，标记前邻的内存块是否空闲。可在内存申请的时，将后邻的该标记设置为占用态（若空闲内存块被分割成两块，前一块为空闲，将当前内存块的该标记设置为空闲态）；在内存释放时，将后邻的该标记设置为空闲态。释放当前内存时，若前邻的内存块标记为使用，则不需要合并前邻的内存块；若前邻的内存块标记为空闲，则需要进行合并。若某个内存块为空闲时，则将其后邻控制块的标记设为到本控制块的距离值。

![](./figures/MemoryRelease.png)

## 支持定时器管理

定时器管理是为满足产品定时业务需要，UniProton 提供了软件定时器功能。

对于软件定时器，是基于 Tick 实现，所以定时周期必须为 Tick 的整数倍，在 Tick 处理函数中进行软件定时器的超时扫描。

目前提供的软件定时器接口，可以完成定时器创建，启动，停止，重启，删除操作。

## 支持信号量管理

信号量（Semaphore）常用于协助一组互相竞争的任务来访问临界资源，在需要互斥的场合作为临界资源计数使用，根据临界资源使用场景分为核内信号量和核间信号量。

信号量对象有一个内部计数器，它支持如下两种操作：

* 申请（Pend）：Pend 操作等待指定的信号量，若其计数器值大于0，则直接减1返回成功。否则任务阻塞，等待其他线程发布该信号量，等待的容忍时间可设定。

* 释放（Post）：Post 操作发布指定的信号量，若无任务等待该信号量，则直接将计数器加1返回。否则唤醒为此信号量挂起的任务列表中的第一个任务（最早阻塞的）。

通常一个信号量的计数值用于对应有效的资源数，表示剩余可被占用的互斥资源数。其值的含义如下有两种情况：

* 为0值：表示没有积累下来的Post操作，且有可能有在此信号量上阻塞的任务。

* 为正值：表示有一个或多个Post下来的发布操作。

## 支持异常管理

UniProton 中的异常接管属于维测特性，其主要目的是在系统出现异常后，记录尽可能多的异常现场信息，便于后续问题定位。同时提供异常时的钩子函数，便于用户能够在异常发生时做一些用户化的特殊处理。其主要功能是接管内部异常处理或者外部硬件异常。

## 支持 CPU 占用率统计

UniProton 中的系统 CPU 占用率（CPU Percent）是指周期时间内系统的 CPU 占用率，用于表示系统一段时间内的闲忙程度，也表示 CPU 的负载情况。系统CPU占用率的有效表示范围为 0～10000，其精度为万分比。10000 表示系统满负荷运转。

UniProton 中的线程 CPU 占用率指单个线程的 CPU 占用率，用于表示单个线程在一段时间内的闲忙程度。线程 CPU 占用率的有效表示范围为 0～10000，其精度为万分比。10000 表示在一段时间内系统一直在运行该线程。单核系统所有线程（包括中断和空闲任务）的 CPU 之和为 10000。

UniProton 的系统级 CPU 占用率依赖于 Tick 模块，通过 Tick 采样 IDLE 任务或 IDLE 软中断计数来实现。

## 支持 STM32F407ZGT6 开发板

支持开发板主要涉及 OS 内核外围的启动流程和单板驱动，目录结构如下：

├─apps         # 基于 UniProton 实时 OS 编程的 demo 程序。

│ └─hello\_world   # hello\_world 示例程序。

├─bsp          # 提供的板级驱动与 OS 对接。

├─build          # 提供编译脚本编译出最终镜像。

├─config         # 配置选项，供用户调整运行时参数。

├─include         # UniProton 实时部分提供的编程接口 API。

└─libs          # UniProton 实时部分的静态库，build 目录中的 makefile 示例已经将头文件和静态库的引用准备好，应用可直接使用。

## 支持 OpenAMP 混合部署

OpenAMP 是一个开源软件框架，旨在通过非对称多处理器的开源解决方案，来标准化异构嵌入式系统中操作环境之间的交互。OpenAMP 包括如下四大组件：

1. remoteproc：管理从核的生命周期，管理共享内存、通信使用的buffer、vring等资源，初始化rpmsg和virtio。

2. rpmsg：实现多核通信的通道，基于virtio实现。

3. virtio：通过一套虚拟IO实现主从核的驱动程序通信，是一种半虚拟化技术。

4. libmetal：屏蔽操作系统实现细节，提供通用用户API访问设备，处理设备中断、内存请求。

## 支持POSIX标准接口

[UniProton支持posix标准接口](./uniproton_apis.md)

## 支持设备驱动

UniProton 的驱动结构、风格与 linux 类似，将驱动设备文件化，即 VFS 系统，通过驱动注册接口，将驱动注册到文件系统中，应用层只需要通过标准系统调用，即可调用底层驱动。整个驱动框架代码适配自开源 RTOS 系统 Nuttx 的驱动模块，因此接口调用也与 Nuttx 基本一致。struct file\_operations 结构体保存设备文件操作的方法，定义在 fs.h 头文件中，通过 register\_driver 接口将驱动设备挂到对应的 struct inode 节点中，struct inode 描述了每个设备节点的位置和数据。当系统调用操作设备文件时，根据对应文件的 inode 就能索引到对应的函数。接口详细信息可以查看[UniProton接口说明](./uniproton_apis.md)。

## 支持 Shell 命令行

UniProton 提供 shell 命令行，它能够以命令行交互的方式访问操作系统的功能或服务：它接受并解析用户输入的命令，并处理操作系统的输出结果。UniProton 的 shell 模块代码适配自开源 ROTS 系统 LiteOS 的 shell 模块。因此与 LiteOS 一致，用户可以新增定制的命令，新增命令需重新编译烧录后才能执行。当前 UniProton 只支持了 help 命令，其他命令将在后续的版本中进行完善。Shell 模块为用户提供下面几个接口。

|      接口名      |    描述     |
|      :---:      |    :--:    |
| SHELLCMD\_ENTRY  | 静态注册命令 |
| osCmdReg        | 动态注册命令 |

通常静态注册命令方式一般用于注册系统常用命令，动态注册命令方式一般用于注册用户命令。静态注册命令有5个入参，动态注册命令有4个入参。下面除去第一个入参是静态注册独有的，剩余的四个入参两个注册命令是一致的。接口详细信息可以查看[UniProton接口说明](./uniproton_apis.md)。

---

---
url: /zh/docs/22.03_LTS_SP4/embedded/uniproton/uniproton_apis.md
---
# UniProton接口说明

## 任务

### 创建并激活任务

在OS启动之前(比如在uniAppInit)中创建的任务，只是简单地加入就绪队列。 OS启动后创建的任务，如果优先级高于当前任务且未锁任务，则立即发生任务调度并被运行，否则加入就绪队列，等待执行。

**输入**： 任务创建参数，包括任务名、任务栈大小、任务优先级、任务处理函数等。

**处理**：

1. 申请任务栈空间，初始化任务栈，置栈顶魔术字。
2. 初始化任务上下文。
3. 初始化任务控制块。
4. 激活任务，任务是否马上能得到执行，取决于OS是否已经启动、优先级是否高于当前任务且没有锁任务调度、当前线程是否为硬中断。

**输出**  ：

* 成功：任务ID，若任务具备执行条件，则直接运行任务，否则将任务挂入就绪队列。

* 失败：提示错误码。

### 删除任务

删除任务并释放任务资源。

**输入**：任务ID。

**处理**：

1. 检查任务是否具备删除条件，如锁任务调度情况下不允许删除任务。
2. 如果任务处于阻塞状态，从对应的阻塞队列中摘除。
3. 释放任务控制块。
4. 释放任务栈空间。
5. 从就绪队列中载入最高优先级的任务，若具备调度条件，则执行。

**输出**：

* 成功：若具备调度条件，则执行就绪队列中的最高任务；

* 失败：返回错误码。

### 挂起任务

挂起任务。

**输入**：任务ID。

**处理**：将指定任务从就绪队列中摘除，若指定任务处于Running态，则会触发任务切换。

**输出**：

* 成功：挂起指定任务。

* 失败：返回错误码。

### 恢复任务

恢复挂起的任务。

**输入**：任务ID。

**处理**：恢复挂起的任务，若任务仍处于延时、阻塞态，则只是取消挂起态，并不加入就绪队列。

**输出**：

* 成功：取消任务挂起状态。
* 失败：返回错误码。

### 任务延时

将当前任务延时指定时间。

**输入**：延时时间。

**处理**：

1. 延时时间转换成OS的Tick数。
2. 将当前任务从就绪队列中摘除，置成延时态。
3. 从就绪队列中载入最高优先级的任务，并执行。
4. Tick中断处理函数中判断任务的延时时间是否已经足够，如果足够，将任务加入就绪队列。

**输出**：

* 成功：当前任务切出，就绪队列中的最高优先级任务切入。
* 失败：返回错误码。

### 锁任务调度

禁止任务之间的切换。

**输入**：锁任务调度请求。

**处理**：

1. 若有任务切换请求，将其清除。
2. 锁任务调度次数依次递增。

**输出**： 任务之间无法切换。

### 恢复任务调度的锁/解锁状态

与锁任务调度配对使用，是否解锁任务调度，取决于最近一次锁任务调度前，是否允许任务调度。

**输入**：恢复任务调度的锁/解锁状态请求。

**处理**：

1. 锁任务调度次数依次递减。
2. 若锁任务调度次数等于0，则发起任务调度。

**输出**：若最近一次锁任务调度前，允许任务调度，则从就绪队列中载入最高优先级任务，并执行。否则，维持原状，不能发生任务切换。

### 任务PID合法性检查

检查指定任务PID是否合法。

**输入**：任务PID。

**处理**：判断输入的任务PID是否超过最大任务PID号或是否已创建。

**输出**：

* TRUE ：任务PID有效。
* FALSE：任务PID无效。

### 任务私有数据获取

获取当前任务的私有数据。

**输入**：无。

**处理**：将任务TCB中记录的任务私有数据返回。

**输出**：任务私有数据。

### 查询本核指定任务正在PEND的信号量

查询指定任务正在PEND的信号量ID。

**输入**：任务PID。

**处理**： 根据任务状态和任务控制块，判断任务是否PEND在某个信号量上，以及PEND的信号量ID。

**输出**：

* 成功：返回信号量ID。
* 失败：返回错误码。

### 查询任务状态

获取指定任务的状态。

**输入**：任务PID。

**处理**：将指定任务的TCB中记录的任务状态字段返回。

**输出**：

* 成功：返回任务状态信息。
* 失败：返回错误码。

### 查询任务上下文信息

获取指定任务的上下文信息。

**输入**：任务PID。

**处理**：  将指定任务的TCB中记录的任务上下文信息返回。

**输出**：

* 成功：返回任务上下文信息。
* 失败：返回错误码。

### 查询任务基本信息

获取任务基本信息，包括任务切换时的PC，SP、任务状态、优先级、任务栈大小、栈顶值，任务名等。

**输入**：任务PID，用于存放任务基本信息查询结果的缓冲区

**处理**： 将指定任务的TCB中记录的任务基本信息返回。

**输出**：

* 成功：返回任务基本信息。
* 失败：返回错误码。

### 任务优先级获取

获取指定任务的优先级。

**输入**：任务PID

**处理**：将指定任务的TCB中记录的优先级字段返回。

**输出**：

* 成功：返回任务优先级信息。
* 失败：返回错误码。

### 任务优先级设定

设置指定任务的优先级。

**输入**：任务PID、优先级值

**处理**：将输入的任务优先级信息存入指定任务TCB中优先级字段

**输出**：

* 成功：指定任务的优先级被修改。
* 失败：返回错误码。

### 调整指定优先级的任务调度顺序

设置指定任务的优先级以及调整调度顺序。

**输入**：指定的优先级、指定需要调整调度顺序的任务，用于保存被调整到队首的任务ID的缓冲。

**处理**：若指定要调整调度顺序的任务为TASK\_NULL\_ID，则优先级队列中的第一个就绪任务调整至队尾；否则，将指定要调整调度顺序的任务调整至优先级队列的队首。

**输出**：

* 成功：指定优先级的任务调度顺序被修改。
* 失败：返回错误码。

## 事件

### 写事件

写事件操作实现对指定任务写入指定类型的事件，可以一次同时写多个事件。

**输入**：任务ID、事件号。

**处理**：

1. 对指定任务事件类型写上输入事件。
2. 判断目的任务是否正在接收等待事件，且其等待的事件是否已经符合唤醒条件（唤醒条件即读取的事件已经发生）。
3. 如果符合唤醒条件，则需清除任务读事件状态。
4. 如果符合唤醒条件，则需清除任务读事件状态。
5. 清除任务超时状态。
6. 在任务没有被挂起的情况下，需要将任务加入就绪队列并尝试任务调度。

**输出**：

* 成功：事件写入成功。
* 失败：错误码。

### 读事件

读事件操作可以根据入参事件掩码类型读取单个或者多个事件。

**输入**：要读取的事件掩码、读取事件所采取的策略、超时时间、接收事件的指针。

**处理**：

1. 根据入参事件掩码类型对自身任务输入读取事件类型。
2. 判断事件读取模式，是读取输入的所有事件还是其中的任意一种事件。
3. 根据读取模式，判断期望的事件是否满足读取情况。
4. 判断事件等待模式：如果为等待事件模式则根据模式来设置相应的超时时间；如果为非等待模式则事件读取失败。
5. 如果需要等待阻塞读取，则需要将自己的任务从就绪列表中删除，并进行任务调度。
6. 读取成功后，清除读取的事件类型，并且把事件类型返回。

**输出**：

* 成功：读事件成功，事件指针赋值。
* 失败：错误码

## 队列

### 创建队列

创建一个队列，创建时可以设定队列长度和队列结点大小。

**输入**： 队列节点个数、每个队列节点大小、队列ID指针。

**处理**：

1. 申请一个空闲的队列资源。
2. 申请队列所需内存。
3. 初始化队列配置。

**输出**：

* 成功：队列ID。
* 失败：错误码。

### 读队列

读指定队列的数据。

**输入**：队列ID、缓冲区指针、长度指针、超时时间。

**处理**：

1. 获取指定队列控制块。
2. 读队列PEND标志，根据缓冲区大小填入队列数据。
3. 修改队列头指针。

**输出**：

* 成功：缓冲区内填入队列数据。
* 失败：错误码。

### 写队列

写数据到指定队列。

**输入**： 队列ID、缓冲区指针、缓冲区长度、超时时间、优先级。

**处理**：

1. 获取指定队列控制块。
2. 读队列PEND标志，选取消息节点，初始化消息节点并拷贝数据。
3. 队列读资源计数器加一。

**输出**：

* 成功：写入队列数据成功。
* 失败：错误码。

### 删除队列

删除一个消息队列，删除后队列资源被回收。

**输入**：队列ID。

**处理**：

1. 获取指定队列控制块，确保队列未在使用中。
2. 释放队列内存。

**输出**：

* 成功：删除队列成功。
* 失败：错误码

### 查询队列的历史最大使用长度

获取从队列创建到删除前的历史最大使用长度。

**输入**：队列ID、队列节点使用峰值指针。

**处理**：

1. 获取指定队列控制块。
2. 将队列节点使用峰值赋值到指针参数。

**输出**：

* 成功：获取峰值成功。
* 失败：错误码

### 查询指定源PID的待处理消息个数

从指定队列中，获取指定源PID的待处理消息个数。

**输入**：队列ID、线程PID、待处理的消息个数指针。

**处理**：

1. 获取指定队列控制块。
2. 遍历队列查询待处理的消息个数，赋值到指针变量。

**输出**：

* 成功：获取待处理的消息个数成功。
* 失败：错误码。

## 中断

### 创建硬中断

硬中断在使用前，必须先创建。

**输入**：硬中断的创建参数，如：硬中断号（与芯片相关）、硬中断优先级、硬中断处理函数等。

**处理**：根据硬中断号设置硬中断优先级、处理函数。

**输出**：

* 成功：硬中断触发后，CPU能够响应该硬中断，并回调硬中断处理函数。
* 失败：返回错误码。

### 硬中断属性设置

在创建硬中断前，需要设置硬中断的模式，包括独立型（#OS\_HWI\_MODE\_ENGROSS）和组合型（#OS\_HWI\_MODE\_COMBINE）两种配置模式。

**输入**：硬中断号、硬中断模式。

**处理**：根据硬中断号设置硬中断的模式；

**输出**：

* 成功：指定的硬中断号被设置好硬中断模式。
* 失败：返回错误码。

### 删除硬中断

删除相应硬中断或事件，取消硬中断处理函数的注册。

**输入**：硬中断号。

**处理**：取消指定硬中断的处理函数与中断号的绑定关系。

**输出**：硬中断被删除，当硬中断信号触发后，CPU不会响应该中断。

### 使能硬中断

使能指定的硬中断。

**输入**：硬中断号。

**处理**：将指定硬中断的使能位置位。

**输出**：指定的硬中断被使能，当硬中断信号触发后，CPU会响应该中断。

### 屏蔽硬中断

屏蔽指定的硬中断。

**输入**：硬中断号。

**处理**：清除指定硬中断的使能位。

**输出**：指定的硬中断被屏蔽，当硬中断信号触发后，CPU不会响应该中断。

### 恢复指定硬中断

恢复指定的硬中断。

**输入**：硬中断号、中断使能寄存器的保存值。

**处理**：还原指定硬中断的使能位。

**输出**：指定中断的使能位恢复为指定状态。

### 禁止硬中断

禁止响应所有可屏蔽硬中断。

**输入**：禁止硬中断请求。

**处理**：

1. 记录系统状态，用于后续返回。
2. 禁止响应所有可屏蔽硬中断。

**输出**：

* 所有可屏蔽硬中断都不能响应。
* 禁止硬中断响应前的系统状态。

### 恢复硬中断

恢复硬中断的禁止或允许响应状态，与禁止硬中断配对使用。是否允许响应硬中断，取决于最近一次禁止硬中断前，系统是否允许响应硬中断。

**输入**：最近一次禁止硬中断前的系统状态。

**处理**：将系统状态恢复到最近一次禁止硬中断前。

**输出**：系统状态恢复到最近一次禁止硬中断前。

### 响应硬中断

硬中断触发后，CPU会响应硬中断。

**输入**：硬件触发的硬中断信号，且系统没有禁止硬中断。

**处理**：

1. 保存当前上下文。
2. 调用硬中断处理函数。
3. 若任务被打断，则恢复最高优先级任务的上下文，该任务不一定是被中断打断的任务。
4. 若低优先级中断被打断，则直接恢复低先级中断的上下文。

**输出**：硬中断被响应。

### 触发硬中断

触发指定核号的指定硬中断。

**输入**：核号、硬中断号。

**处理**：

1. 目前只支持触发本核的硬中断，若指定的核号不为本核，则做报错处理。
2. 目前只支持触发软件可触发的硬中断，若指定的中断无法进行软件触发，则做报错处理。
3. 当以前两个条件都满足，则设置对应的中断触发寄存器，软件触发中断。

**输出**：

* 成功：响应的硬中断被触发。
* 失败：返回错误码。

### 清除中断位

清除所有的中断请求位或指定的中断请求位。

**输入**：硬中断号。

**处理**：清除所有的中断请求位或指定的中断请求位。

**输出**：所有的中断请求位或指定的中断请求位被清除

## 定时器

### 定时器创建

根据定时器类型，触发模式，定时时长，处理函数等创建一个定时器。

**输入**：

1. 创建参数结构体（包括定时器类型，触发模式，定时时长，处理函数等）。
2. 用于保存输出的定时器句柄的指针。

**处理**：根据入参找到空闲控制块，将入参内容填入控制块中相应的字段中。

**输出**：

* 成功：定时器创建成功，后续可根据得到的定时器句柄做启动、删除等操作。
* 失败：返回错误码。

### 定时器删除

删除指定的定时器。

**输入**： 定时器句柄

**处理**：根据传入的定时器句柄，找到定时器控制块，将其内容清空并将控制块挂接到相应的空闲链表中。

**输出**：

* 成功：定时器被删除。
* 失败：返回错误码。

### 定时器启动

指定的定时器开始计时。

**输入**： 定时器句柄

**处理**：对于软件定时器，根据当前Tick计数以及定时器周期，计算结束时间，将此定时器控制块挂入定时器SortLink。

**输出**：

* 成功：定时器开始计时。
* 失败：返回错误码。

### 定时器停止

指定的定时器停止计时。

**输入**：定时器句柄。

**处理**：对于软件定时器，计算剩余时间并将其保存后，将此定时器控制块从定时器SortLink中摘除。

**输出**：

* 成功：指定任务的信号量计数值被修改。
* 失败：返回错误码。

### 定时器重启

指定的定时器重新开始计时。

**输入**：定时器句柄

**处理**：对于软件定时器，根据当前Tick计数以及定时器周期，计算结束时间，将此定时器控制块挂入定时器SortLink。

**输出**：

* 成功：指定任务的信号量计数值被修改。
* 失败：返回错误码。

### 软件定时器组创建

创建一个软件定时器组，后续的软件定时器创建时需要以此为基础。

**输入**：

1. 软件定时器组创建参数（主要关注时钟源类型及最大支持的定时器个数）。
2. 用于保存输出的定时器组号的地址。

**处理**：根据传入的最大支持的定时器个数申请定时器控制块内存，并完成其初始化操作。

**输出**：

* 成功：基于Tick的软件定时器组被成功创建。
* 失败：返回错误码。

## 信号量

### 信号量创建

创建一个信号量，并设置其初始计数器数值。

**输入**：信号量初始计数值、用于保存创建得到句柄的地址。

**处理**：找到一个空闲信号量控制块，将输入的初始计数值填入后将信号量ID当做句柄返回。

**输出**：

* 成功：信号量被创建。
* 失败：返回错误码。

### 信号量删除

删除指定信号量，若有任务阻塞于该信号量，则删除失败。

**输入**：信号量句柄

**处理**：对于核内信号量，根据输入的信号量句柄找到信号量控制块，通过查看控制块中任务阻塞链表是否为空来判断是否有任务阻塞于该信号量，若有则删除失败返回，否则释放该信号量控制块。

**输出**：

* 成功：信号量被删除。
* 失败：返回错误码。

### Pend信号量

申请指定的信号量，若其计数值大于0，则直接将计数值减1返回，否则发生任务阻塞，等待时间可通过入参设定。

**输入**：信号量句柄、等待时间

**处理**：

![](./figures/pend_semaphore.png)

**输出**：

* 成功：返回0。
* 失败：返回错误码。

### Post信号量

发布信号量，将该信号量计数值+1，若有任务阻塞于该信号量，则将其唤醒。

**输入**：信号量句柄。

**处理**：

![](./figures/post_semaphore.png)

**输出**：

* 成功：信号量发布成功。
* 失败：返回错误码。

### 信号量计数值重置

设置指定信号量计数值，如果有任务阻塞于该信号量，则设置失败。

**输入**：信号量句柄、信号量计数值。

**处理**：根据输入的信号量句柄，找到相应的信号量控制块，查看控制块中任务阻塞链表，若其不为空，则返回错误，否则将控制块中信号量计数值设为输入的计数值。

**输出**：

* 成功：指定信号量的计数值被修改；
* 失败：返回错误码。

### 信号量计数值获取

获取指定信号量计数值。

**输入**： 信号量句柄

**处理**：根据输入的信号量句柄，找到相应的信号量控制块，将控制块中记录的信号量计数值返回。

**输出**：

* 成功：返回信号量计数值。
* 失败：返回错误计数值标记。

### 信号量阻塞任务PID获取

获取阻塞在指定信号量上的任务个数及任务PID列表。

**输入**：

1. 信号量句柄。
2. 用于存放输出的阻塞任务个数的地址。
3. 用于存放输出的阻塞任务PID的缓冲区首地址。
4. 用于存放输出的阻塞任务PID的缓冲区长度。

**处理**：若有任务阻塞于指定信号量，则输出阻塞任务的个数及任务PID清单；否则，将阻塞任务个数置为0。

**输出**：

* 成功：输出阻塞于该信号量的任务个数及任务PID清单。
* 失败：返回错误码。

## 异常

### 用户注册异常处理钩子

用户注册异常处理函数类型定义的异常处理函数钩子，记录异常信息。

**输入**：类型为ExcProcFunc的钩子函数。

**处理**：将用户注册的钩子函数注册到OS框架里，发生异常时调用。

**输出**：

* 成功：注册成功。
* 失败：返回错误码。

## CPU占用率

### 获取当前cpu占用率

通过本接口获取当前cpu占用率。

**输入**：无。

**处理**：采用基于IDLE计数的统计算法，统计结果会有一定误差，误差不超过百分之五。

**输出**：

* 成功：返回当前的cpu占用率\[0,10000]。
* 失败：返回错误码。

### 获取指定个数的线程的CPU占用率

根据用户输入的线程个数，获取指定个数的线程CPU占用率。

**输入**： 线程个数、缓冲区指针、实际线程个数指针。

**处理**：

1. 采用基于 IDLE 计数的统计算法，统计结果会有一定误差，误差不超过百分之五。
2. 当配置项中的采样周期值等于0时，线程级CPUP采样周期为两次调用该接口或者PRT\_CpupNow之间的间隔。否则，线程级CPUP采样周期为配置项中的OS\_CPUP\_SAMPLE\_INTERVAL大小。
3. 输出的实际线程个数不大于系统中实际的线程个数（任务个数和一个中断线程）。
4. 若在一个采样周期内有任务被删除，则统计的任务线程和中断线程CPUP总和小于10000。

**输出**：

* 成功：在缓冲区写入cpu占用率。
* 失败：返回错误码。

### 设置CPU占用率告警阈值

根据用户配置的 CPU 占用率告警阈值 warn 和告警恢复阈值 resume，设置告警和恢复阈值。

**输入**：告警阈值、恢复阈值。

**处理**：设置 CPUP 告警阈值和恢复阈值

**输出**：

* 成功：设置成功。
* 失败：返回错误码。

### 查询CPUP告警阈值和告警恢复阈值

根据用户配置的告警阈值指针 warn 和告警恢复阈值指针 resume，查询告警阈值和告警恢复阈值。

**输入**：告警阈值指针、恢复阈值指针。

**处理**：获取 CPUP 告警阈值和恢复阈值，赋值指针变量。

**输出**：

* 成功：获取成功。
* 失败：返回错误码。

### 注册CPUP告警回调函数

根据用户配置的回调函数 hook，注册 CPUP 告警回调函数。

**输入**：类型为 CpupHookFunc 的 CPU 告警回调函数。

**处理**：将用户的钩子函数注册到 OS 框架。

**输出**：

* 成功：注册成功。
* 失败：错误码

## OS启动

### main函数入口

二进制执行文件函数入口。

**输入**：无

**输出**：

* 成功：返回OK。
* 失败：错误码

### 用户业务入口

PRT\_AppInit 用户业务函数入口，在 main 函数后调用，在此函数中添加业务功能代码。

**输入**：无

**输出**：

* 成功：返回OK。
* 失败：错误码

### 硬件驱动初始化入口

PRT\_HardDrvInit 硬件驱动初始化函数入口，在 main 函数后调用，在此函数中添加板级驱动初始化功能代码。

**输入**：无

**输出**：

* 成功：返回OK。
* 失败：错误码

### 硬件启动流程入口

PRT\_HardBootInit 在 OS 启动时调用，在main函数前被调用，可以用于 BSS 初始化、随机值设置等。

**输入**：无

**输出**：

* 成功：返回OK。
* 失败：错误码。

## openamp

### 初始化openamp资源函数

初始化保留内存，初始化 remoteproc、virtio、rpmsg，建立 Uniproton 与 Linux 两端配对的 endpoint，供消息收发使用。

**输入**：无。

**输出**：

* 成功：初始化成功。
* 失败：错误码。

### 消息接收函数

接收消息，并触发SGI中断

**输入**：

1. 类型为 unsigned char \* 的存储消息的缓冲区。
2. 类型为 int 的消息预期长度。
3. 类型为 int \*，用于获取消息实际长度。

**输出**：

* 成功：消息接收成功。
* 失败：错误码。

### 消息发送函数

发送消息和SGI中断

**输入**：类型为 unsigned char \* 的存储消息的缓冲区、类型为 int 的消息长度。

**输出**：

* 成功：消息发送成功。
* 失败：错误码。

### 释放openamp资源

用于释放openamp资源。

**输入**：无

**输出**：

* 成功：资源释放成功。
* 失败：错误码。

## POSIX接口

|       接口名         |       适配情况        |
|       :---:         |       :-----:       |
|  [pthread\_atfork](#pthread_atfork)  |  不支持  |
|  [pthread\_attr\_destroy](#pthread_attr_destroy)  |  支持  |
|  [pthread\_attr\_getdetachstate](#pthread_attr_getdetachstate)  |  支持  |
|  [pthread\_attr\_getguardsize](#pthread_attr_getguardsize)  |  不支持  |
|  [pthread\_attr\_getinheritsched](#pthread_attr_getinheritsched)  |  支持  |
|  [pthread\_attr\_getschedparam](#pthread_attr_getschedparam)  |  支持  |
|  [pthread\_attr\_getschedpolicy](#pthread_attr_getschedpolicy)  |  支持  |
|  [pthread\_attr\_getscope](#pthread_attr_getscope)  |  支持  |
|  [pthread\_attr\_getstack](#pthread_attr_getstack)  |  支持  |
|  [pthread\_attr\_getstackaddr](#pthread_attr_getstackaddr)  |  支持  |
|  [pthread\_attr\_getstacksize](#pthread_attr_getstacksize)  |  支持  |
|  [pthread\_attr\_init](#pthread_attr_init)  |  支持  |
|  [pthread\_attr\_setdetachstate](#pthread_attr_setdetachstate)  |  支持  |
|  [pthread\_attr\_setguardsize](#pthread_attr_setguardsize)  |  不支持  |
|  [pthread\_attr\_setinheritsched](#pthread_attr_setinheritsched)  |  支持  |
|  [pthread\_attr\_setschedparam](#pthread_attr_setschedparam)  |  部分支持  |
|  [pthread\_attr\_setschedpolicy](#pthread_attr_setschedpolicy)  |  部分支持  |
|  [pthread\_attr\_setscope](#pthread_attr_setscope)  |  部分支持  |
|  [pthread\_attr\_setstack](#pthread_attr_setstack)  |  支持  |
|  [pthread\_attr\_setstackaddr](#pthread_attr_setstackaddr)  |  支持  |
|  [pthread\_attr\_setstacksize](#pthread_attr_setstacksize)  |  支持  |
|  [pthread\_barrier\_destroy](#pthread_barrier_destroy)  |  支持  |
|  [pthread\_barrier\_init](#pthread_barrier_init)  |  部分支持  |
|  [pthread\_barrier\_wait](#pthread_barrier_wait)  |  支持  |
|  [pthread\_barrierattr\_getpshared](#pthread_barrierattr_getpshared)  |  支持  |
|  [pthread\_barrierattr\_setpshared](#pthread_barrierattr_setpshared)  |  部分支持  |
|  [pthread\_cancel](#pthread_cancel)  |  支持  |
|  [pthread\_cond\_broadcast](#pthread_cond_broadcast)  |  支持  |
|  [pthread\_cond\_destroy](#pthread_cond_destroy)  |  支持  |
|  [pthread\_cond\_init](#pthread_cond_init)  |  支持  |
|  [pthread\_cond\_signal](#pthread_cond_signal)  |  支持  |
|  [pthread\_cond\_timedwait](#pthread_cond_timedwait)  |  支持  |
|  [pthread\_cond\_wait](#pthread_cond_wait)  |  支持  |
|  [pthread\_condattr\_destroy](#pthread_condattr_destroy)  |  支持  |
|  [pthread\_condattr\_getclock](#pthread_condattr_getclock)  |  支持  |
|  [pthread\_condattr\_getpshared](#pthread_condattr_getpshared)  |  支持  |
|  [pthread\_condattr\_init](#pthread_condattr_init)  |  支持  |
|  [pthread\_condattr\_setclock](#pthread_condattr_setclock)  |  部分支持  |
|  [pthread\_condattr\_setpshared](#pthread_condattr_setpshared)  |  部分支持  |
|  [pthread\_create](#pthread_create)  |  支持  |
|  [pthread\_detach](#pthread_detach)  |  支持  |
|  [pthread\_equal](#pthread_equal)  |  支持  |
|  [pthread\_exit](#pthread_exit)  |  支持  |
|  [pthread\_getcpuclockid](#pthread_getcpuclockid)  |  不支持  |
|  [pthread\_getschedparam](#pthread_getschedparam)  |  支持  |
|  [pthread\_getspecific](#pthread_getspecific)  |  支持  |
|  [pthread\_join](#pthread_join)  |  支持  |
|  [pthread\_key\_create](#pthread_key_create)  |  支持  |
|  [pthread\_key\_delete](#pthread_key_delete)  |  支持  |
|  [pthread\_kill](#pthread_kill)  |  不支持  |
|  [pthread\_mutex\_consistent](#pthread_mutex_consistent)  |  不支持  |
|  [pthread\_mutex\_destroy](#pthread_mutex_destroy)  |  支持  |
|  [pthread\_mutex\_getprioceiling](#pthread_mutex_getprioceiling)  |  不支持  |
|  [pthread\_mutex\_init](#pthread_mutex_init)  |  支持  |
|  [pthread\_mutex\_lock](#pthread_mutex_lock)  |  支持  |
|  [pthread\_mutex\_setprioceiling](#pthread_mutex_setprioceiling)  |  不支持  |
|  [pthread\_mutex\_timedlock](#pthread_mutex_timedlock)  |  支持  |
|  [pthread\_mutex\_trylock](#pthread_mutex_trylock)  |  支持  |
|  [pthread\_mutex\_unlock](#pthread_mutex_unlock)  |  支持  |
|  [pthread\_mutexattr\_destroy](#pthread_mutexattr_destroy)  |  支持  |
|  [pthread\_mutexattr\_getprioceiling](#pthread_mutexattr_getprioceiling)  |  不支持  |
|  [pthread\_mutexattr\_getprotocol](#pthread_mutexattr_getprotocol)  |  支持  |
|  [pthread\_mutexattr\_getpshared](#pthread_mutexattr_getpshared)  |  部分支持  |
|  [pthread\_mutexattr\_getrobust](#pthread_mutexattr_getrobust)  |  部分支持  |
|  [pthread\_mutexattr\_gettype](#pthread_mutexattr_gettype)  |  支持  |
|  [pthread\_mutexattr\_init](#pthread_mutexattr_init)  |  支持  |
|  [pthread\_mutexattr\_setprioceiling](#pthread_mutexattr_setprioceiling)  |  不支持  |
|  [pthread\_mutexattr\_setprotocol](#pthread_mutexattr_setprotocol)  |  部分支持  |
|  [pthread\_mutexattr\_setpshared](#pthread_mutexattr_setpshared)  |  不支持  |
|  [pthread\_mutexattr\_setrobust](#pthread_mutexattr_setrobust)  |  部分支持  |
|  [pthread\_mutexattr\_settype](#pthread_mutexattr_settype)  |  支持  |
|  [pthread\_once](#pthread_once)  |  部分支持  |
|  [pthread\_rwlock\_destroy](#pthread_rwlock_destroy)  |  支持  |
|  [pthread\_rwlock\_init](#pthread_rwlock_init)  |  支持  |
|  [pthread\_rwlock\_rdlock](#pthread_rwlock_rdlock)  |  支持  |
|  [pthread\_rwlock\_timedrdlock](#pthread_rwlock_timedrdlock)  |  支持  |
|  [pthread\_rwlock\_timedwrlock](#pthread_rwlock_timedwrlock)  |  支持  |
|  [pthread\_rwlock\_tryrdlock](#pthread_rwlock_tryrdlock)  |  支持  |
|  [pthread\_rwlock\_trywrlock](#pthread_rwlock_trywrlock)  |  支持  |
|  [pthread\_rwlock\_unlock](#pthread_rwlock_unlock)  |  支持  |
|  [pthread\_rwlock\_wrlock](#pthread_rwlock_wrlock)  |  支持  |
|  [pthread\_rwlockattr\_destroy](#pthread_rwlockattr_destroy)  |  不支持  |
|  [pthread\_rwlockattr\_getpshared](#pthread_rwlockattr_getpshared)  |  部分支持  |
|  [pthread\_rwlockattr\_init](#pthread_rwlockattr_init)  |  不支持  |
|  [pthread\_rwlockattr\_setpshared](#pthread_rwlockattr_setpshared)  |  部分支持  |
|  [pthread\_self](#pthread_self)  |  支持  |
|  [pthread\_setcancelstate](#pthread_setcancelstate)  |  支持  |
|  [pthread\_setcanceltype](#pthread_setcanceltype)  |  支持  |
|  [pthread\_setschedparam](#pthread_setschedparam)  |  部分支持  |
|  [pthread\_setschedprio](#pthread_setschedprio)  |  支持  |
|  [pthread\_setspecific](#pthread_setspecific)  |  支持  |
|  [pthread\_sigmask](#pthread_sigmask)  |  不支持  |
|  [pthread\_spin\_init](#pthread_spin_init)  |  不支持  |
|  [pthread\_spin\_destory](#pthread_spin_destory)  |  不支持  |
|  [pthread\_spin\_lock](#pthread_spin_lock)  |  不支持  |
|  [pthread\_spin\_trylock](#pthread_spin_trylock)  |  不支持  |
|  [pthread\_spin\_unlock](#pthread_spin_unlock)  |  不支持  |
|  [pthread\_testcancel](#pthread_testcancel)  |  支持  |
|  [sem\_close](#sem_close)  |  支持  |
|  [sem\_destroy](#sem_destroy)  |  支持  |
|  [sem\_getvalue](#sem_getvalue)  |  支持  |
|  [sem\_init](#sem_init)  |  支持  |
|  [sem\_open](#sem_open)  |  支持  |
|  [sem\_post](#sem_post)  |  支持  |
|  [sem\_timedwait](#sem_timedwait)  |  支持  |
|  [sem\_trywait](#sem_trywait)  |  支持  |
|  [sem\_unlink](#sem_unlink)  |  部分支持  |
|  [sem\_wait](#sem_wait)  |  支持  |
|  [sched\_yield](#sched_yield)  |  支持  |
|  [sched\_get\_priority\_max](#sched_get_priority_max)  |  支持  |
|  [sched\_get\_priority\_min](#sched_get_priority_min)  |  支持  |
|  [asctime](#asctime)  |  支持  |
|  [asctime\_r](#asctime_r)  |  支持  |
|  [clock](#clock)  |  支持  |
|  [clock\_getcpuclockid](#clock_getcpuclockid)  |  部分支持  |
|  [clock\_getres](#clock_getres)  |  部分支持  |
|  [clock\_gettime](#clock_gettime)  |  支持  |
|  [clock\_nanosleep](#clock_nanosleep)  |  部分支持  |
|  [clock\_settime](#clock_settime)  |  支持  |
|  [ctime](#ctime)  |  支持  |
|  [ctime\_r](#ctime_r)  |  支持  |
|  [difftime](#difftime)  |  支持  |
|  [getdate](#getdate)  |  不支持  |
|  [gettimeofday](#gettimeofday)  |  支持  |
|  [gmtime](#gmtime)  |  支持  |
|  [gmtime\_r](#gmtime_r)  |  支持  |
|  [localtime](#localtime)  |  支持  |
|  [localtime\_r](#localtime_r)  |  支持  |
|  [mktime](#mktime)  |  支持  |
|  [nanosleep](#nanosleep)  |  支持  |
|  [strftime](#strftime)  |  不支持  |
|  [strftime\_l](#strftime_l)  |  不支持  |
|  [strptime](#strptime)  |  支持  |
|  [time](#time)  |  支持  |
|  [timer\_create](#timer_create)  |  支持  |
|  [timer\_delete](#timer_delete)  |  支持  |
|  [timer\_getoverrun](#timer_getoverrun)  |  支持  |
|  [timer\_gettime](#timer_gettime)  |  支持  |
|  [timer\_settime](#timer_settime)  |  支持  |
|  [times](#times)  |  支持  |
|  [timespec\_get](#timespec_get)  |  支持  |
|  [utime](#utime)  |  不支持  |
|  [wcsftime](#wcsftime)  |  不支持  |
|  [wcsftime\_l](#wcsftime_l)  |  不支持  |
|  [malloc](#malloc)  |  支持  |
|  [free](#free)  |  支持  |
|  [memalign](#memalign)  |  支持  |
|  [realloc](#realloc)  |  支持  |
|  [malloc\_usable\_size](#malloc_usable_size)  |  支持  |
|  [aligned\_alloc](#aligned_alloc)  |  支持  |
|  [reallocarray](#reallocarray)  |  支持  |
|  [calloc](#calloc)  |  支持  |
|  [posix\_memalign](#posix_memalign)  |  支持  |
|  [abort](#abort)  |  支持  |
|  [\_Exit](#_exit)  |  支持  |
|  [atexit](#atexit)  |  支持  |
|  [quick\_exit](#quick_exit)  |  支持  |
|  [at\_quick\_exit](#at_quick_exit)  |  支持  |
|  [assert](#assert)  |  支持  |
|  [div](#div)  |  支持  |
|  [ldiv](#ldiv)  |  支持  |
|  [lldiv](#lldiv)  |  支持  |
|  [imaxdiv](#imaxdiv)  |  支持  |
|  [wcstol](#wcstol)  |  支持  |
|  [wcstod](#wcstod)  |  支持  |
|  [fcvt](#fcvt)  |  支持  |
|  [ecvt](#ecvt)  |  支持  |
|  [gcvt](#gcvt)  |  支持  |
|  [qsort](#qsort)  |  支持  |
|  [abs](#abs)  |  支持  |
|  [labs](#labs)  |  支持  |
|  [llabs](#llabs)  |  支持  |
|  [imaxabs](#imaxabs)  |  支持  |
|  [strtol](#strtol)  |  支持  |
|  [strtod](#strtod)  |  支持  |
|  [atoi](#atoi)  |  支持  |
|  [atol](#atol)  |  支持  |
|  [atoll](#atoll)  |  支持  |
|  [atof](#atof)  |  支持  |
|  [bsearch](#bsearch)  |  支持  |
|  [semget](#semget)  |  支持  |
|  [semctl](#semctl)  |  部分支持  |
|  [semop](#semop)  |  部分支持  |
|  [semtimedop](#semtimedop)  |  部分支持  |
|  [msgget](#msgget)  |  支持  |
|  [msgctl](#msgctl)  |  部分支持  |
|  [msgsnd](#msgsnd)  |  部分支持  |
|  [msgrcv](#msgrcv)  |  部分支持  |
|  [shmget](#shmget)  |  不支持  |
|  [shmctl](#shmctl)  |  不支持  |
|  [shmat](#shmat)  |  不支持  |
|  [shmdt](#shmdt)  |  不支持  |
|  [ftok](#ftok)  |  不支持  |

### 任务管理

#### pthread\_attr\_init

pthread\_attr\_init() 函数初始化一个线程对象的属性,需要用 pthread\_attr\_destroy() 函数对其去除初始化。

**参数**：指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr，结构中的元素分别对应着新线程的运行属性。

**输出**：

* 0：初始化成功。
* ENOMEM：内存不足，无法初始化线程属性对象。
* EBUSY：attr是以前初始化但未销毁的线程属性。

#### pthread\_attr\_destroy

pthread\_attr\_destroy()函数应销毁线程属性对象。被销毁的attr属性对象可以使用pthread\_attr\_init()重新初始化；在对象被销毁后引用该对象的结果是未定义的。

**参数**：指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。

**输出**：

* 0：函数销毁对象成功。
* EINVAL：attr指向的是未初始化的线程属性对象。

#### pthread\_attr\_setstackaddr

pthread\_attr\_setstackaddr()函数设置attr对象中的线程创建堆栈addr属性。堆栈addr属性指定用于创建线程堆栈的存储位置。

**输入**：指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr、 栈地址stackaddr。

**输出**：

* 0：设置成功。
* EINVAL：attr指向的是未初始化的线程属性对象。

#### pthread\_attr\_getstackaddr

pthread\_attr\_getstackaddr()如果成功，函数将堆栈地址属性值存储在堆栈地址中。

**参数**：指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr、栈地址stackaddr.

**输出**：

* 0：获取成功。
* EINVAL：attr指向的是未初始化的线程属性对象。

#### pthread\_attr\_getstacksize

pthread\_attr\_getstacksize()和pthread\_attr\_setstacksize()函数分别应获取和设置 attr 对象中的线程创建堆栈大小属性（以字节为单位）。

**参数**：

1. 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr.
2. 栈大小指针stacksize，指向设置或获取的堆栈大小。

**输出**：

* 0：获取成功。
* EINVAL：attr指向的是未初始化的线程属性对象。

#### pthread\_attr\_setstacksize

设置attr对象中的线程创建堆栈大小属性。

**参数**：

1. 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
2. 栈大小指针stacksize，指向设置或获取的堆栈大小。

**输出**：

* 0：设置成功。
* EINVAL：堆栈size小于最小值或超过限制。

#### pthread\_attr\_getinheritsched

获取线程的继承属性。

**参数**：

* 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
* 线程的继承性指针inheritsched。

**输出**：

* 0：获取成功。
* EINVAL：attr指向的是未初始化的线程属性对象。

#### pthread\_attr\_setinheritsched

设置线程的继承属性。可设置如下参数：

* PTHREAD\_INHERIT\_SCHED：指定线程调度属性应继承自创建线程，并且应忽略此attr参数中的调度属性。
* PTHREAD\_EXPLICIT\_SCHED：指定线程调度属性应设置为此属性对象中的相应值。

**参数**：

* 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
* 线程的继承性inheritsched。

**输出**：

* 0：设置成功。
* EINVAL：继承的值无效，或attr指向的是未初始化的线程属性对象。
* ENOTSUP：试图将属性设置为不支持的值。

#### pthread\_attr\_getschedpolicy

获取调度策略属性，策略支持SCHED\_FIFO。当使用调度策略SCHED\_FIFO执行的线程正在等待互斥体时，互斥体解锁，它们应按优先级顺序获取互斥体。

**参数**：

1. 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
2. 线程的调度策略指针policy。

**输出**：

* 0：获取成功。
* EINVAL：attr指向的是未初始化的线程属性对象。

#### pthread\_attr\_setschedpolicy

设置调度策略属性，策略支持SCHED\_FIFO。当使用调度策略SCHED\_FIFO执行的线程正在等待互斥体时，互斥体解锁时，它们应按优先级顺序获取互斥体。

**参数**：

1. 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
2. 线程的调度策略policy。

**输出**：

* 0：设置成功。
* EINVAL：policy的值无效，或者attr指向没有初始化的线程对象。
* ENOTSUP：试图将属性设置为不支持的值。

#### pthread\_attr\_getdetachstate

获取线程分离属性，分离状态应设置为PTHREAD\_CREATE\_DETAED或PTHREAD\_CREATE\_JOI无BLE。

**参数**：

1. 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
2. 分离属性指针detachstate。

**输出**：

* 0：获取成功。
* EINVAL：attr指向没有初始化的线程对象。

#### pthread\_attr\_setdetachstate

设置线程分离属性。分离状态应设置为PTHREAD\_CREATE\_DETAED或PTHREAD\_CREATE\_JOINABLE。

**参数**：

1. 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
2. 分离属性detachstate。

**输出**：

* 0：设置成功。
* EINVAL：attr指向没有初始化的线程对象或分离状态的值无效。

#### pthread\_attr\_setschedparam

pthread\_attr\_setschedparam() 可用来设置线程属性对象的优先级属性。

**参数**：

1. 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
2. 调度属性指针schedparam。

**输出**：

* 0：操作成功。
* EINVAL：参数不合法或attr未初始化。
* ENOTSUP：schedparam的优先级属性不支持。

#### pthread\_attr\_getschedparam

pthread\_attr\_getschedparam() 可用来获取线程属性对象的优先级属性。

**参数**：

1. 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
2. 调度属性指针schedparam。

**输出**：

* 0：操作成功。
* EINVAL：参数不合法或attr未初始化。

#### pthread\_attr\_getscope

pthread\_attr\_getscope() 可用来获取线程属性对象的作用域属性。

**参数**：

* 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
* 线程的作用域属性指针scope。

**输出**：

* 0：获取成功。
* EINVAL：指针未初始化。

#### pthread\_attr\_setscope

设置线程的作用域，支持PTHREAD\_SCOPE\_SYSTEM，控制线程在系统级竞争资源。

**参数**：

1. 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
2. 作用域scope。

**输出**：

* 0：设置成功。
* EINVAL：scope的值无效，或者attr指向没有初始化的线程对象。
* ENOTSUP：试图将属性设置为不支持的值。

#### pthread\_attr\_getstack

pthread\_attr\_getstack() 可用来获取线程属性对象的栈信息。

**参数**：

* 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
* 线程的栈地址指针stackAddr。
* 线程的栈大小指针stackSize。

**输出**：

* 0：获取成功。
* EINVAL：指针未初始化。

#### pthread\_attr\_setstack

pthread\_attr\_setstack() 可用来设置线程属性对象的栈地址和栈大小。

**参数**：

* 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
* 线程的栈地址stackAddr。
* 线程的栈大小stackSize。

**输出**：

* 0：获取成功。
* EINVAL：指针未初始化或值无效。

#### pthread\_attr\_getguardsize

暂不支持。

#### pthread\_attr\_setguardsize

暂不支持。

#### pthread\_atfork

暂不支持。

#### pthread\_create

pthread\_create()函数创建一个新线程，其属性由 attr 指定。如果 attr 为 NULL，则使用默认属性。创建成功后，pthread\_create()应将创建的线程的ID存储在参数 thread 的位置。

**参数**：

1. 指向线程[标识符](https://baike.baidu.com/item/标识符?fromModule=lemma_inlink)的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)thread。
2. 指向一个线程属性结构的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)attr。
3. 线程处理函数的起始地址 start\_routine。
4. 运行函数的参数 arg。

**输出**：

* 0：创建成功。
* EINVAL：attr指定的属性无效。
* EAGAIN：系统缺少创建新线程所需的资源，或者将超过系统对线程总数施加的限制。
* EPERM：调用者没有权限。

#### pthread\_cancel

取消线程的执行。pthread\_cancel()函数应请求取消线程。目标线程的可取消状态和类型决定取消何时生效。当取消被操作时，应调用线程的取消处理程序。

**参数**：线程的ID thread。

**输出**：

* 0：取消成功。
* ESRCH：找不到与给定线程ID相对应的线程。

#### pthread\_testcancel

设置可取消状态。pthread\_testcancel()函数应在调用线程中创建一个取消点。如果禁用了可取消性,pthread\_testcancel()函数将无效。

**参数**：无

**输出**：无

#### pthread\_setcancelstate

pthread\_setcancelstate() 将调用线程的可取消性状态设置为 state 中给出的值。线程以前的可取消性状态返回到oldstate所指向的缓冲区中。state状态的合法值为PTHREAD\_CANCEL\_E无BLE和PTHREAD\_CANCEL\_DISABLE。

**参数**：

* 线程的可取消性状态 state。
* 之前的可取消状态 oldstate。

**输出**：

* 0：设置成功。
* EINVAL：指定的状态不是 PTHREAD\_CANCEL\_E无BLE 或 PTHREAD\_CANCEL\_DISABLE。

#### pthread\_setcanceltype

pthread\_setcanceltype()函数应原子地将调用线程的可取消类型设置为指定的类型，并在oldtype引用的位置返回上一个可取消类型。类型的合法值为PTHREAD\_CANCEL\_DEFERRED和PTHREAD\_CANCEL\_ASYNCHRONOUS。

**输入**：

* 线程的可取消类型type。
* 之前的可取消类型oldtype。

**输出**：

* 0：设置成功。
* EINVAL：指定的类型不是PTHREAD\_CANCEL\_DEFERRED或PTHREAD\_CANCEL\_ASYNCHRONOUS。

#### pthread\_exit

线程的终止可以是调用 pthread\_exit 或者该线程的例程结束。由此可看出，一个线程可以隐式退出，也可以显式调用 pthread\_exit 函数来退出。pthread\_exit 函数唯一的参数 value\_ptr 是函数的返回代码，只要 pthread\_join 中的第二个参数 value\_ptr 不是NULL，这个值将被传递给 value\_ptr。

**参数**：线程退出状态value\_ptr，通常传NULL。

**输出**：无

#### pthread\_cleanup\_push

pthread\_cleanup\_push() 函数应将指定的取消处理程序推送到调用线程的取消堆栈上。pthread\_cleanup\_push必须和pthread\_cleanup\_pop同时使用。当push后，在线程退出前使用pop，便会调用清理函数。

**参数**：

1. 取消处理程序入口地址 routine。
2. 传递给处理函数的参数 arg。

**输出**：无

#### pthread\_cleanup\_pop

pthread\_cleanup\_pop()应删除调用线程取消处理程序，并可选择调用它（如果execute非零）。

**参数**：执行参数execute。

**输出**：无

#### pthread\_setschedprio

pthread\_setschedprio()函数应将线程 ID 指定的调度优先级设置为 prio 给出的值。如果 pthread\_setschedprio()函数失败，则目标线程的调度优先级不应更改。

**参数**：

1. 线程ID：thread。
2. 优先级：prio。

**输出**：

* 0，设置成功。
* EINVAL：prio对指定线程的调度策略无效。
* ENOTSUP：试图将优先级设置为不支持的值。
* EPERM：调用者没有设置指定线程的调度策略的权限。
* EPERM：不允许将优先级修改为指定的值。
* ESRCH：thread指定的线程不存在。

#### pthread\_self

pthread\_self()函数应返回调用线程的线程ID。

**参数**：无

**输出**：返回调用线程的线程ID。

#### pthread\_equal

此函数应比较线程ID t1和t2。

**参数**：

1. 线程ID t1。
2. 线程ID t2。

**输出**：

* 如果t1和t2相等，pthread\_equal()函数应返回非零值。
* 如果t1和t2不相等，应返回零。
* 如果t1或t2不是有效的线程ID，则行为未定义。

#### sched\_yield

sched\_yield()函数应强制正在运行的线程放弃处理器，并触发线程调度。

**参数**：无

**输出**：输出0时，成功完成；否则应返回值-1。

#### sched\_get\_priority\_max

sched\_get\_priority\_max()和 sched\_get\_priority\_min()函数应分别返回指定调度策略的优先级最大值或最小值。

**参数**：调度策略policy。

**输出**：

返回值：

* -1：失败。
* 返回优先级最大值。

errno：

* EINVAL：调度策略非法。

#### sched\_get\_priority\_min

返回指定调度策略的优先级最小值

**参数**：调度策略policy。

**输出**：

返回值：

* -1：失败。
* 返回优先级最小值。

errno：

* EINVAL：调度策略非法。

#### pthread\_join

pthread\_join() 函数，以阻塞的方式等待 thread 指定的线程结束。当函数返回时，被等待线程的资源被收回。如果线程已经结束，那么该函数会立即返回。并且 thread 指定的线程必须是 joi无ble 的。当 pthread\_join()成功返回时，目标线程已终止。对指定同一目标线程的pthread\_join()的多个同时调用的结果未定义。如果调用pthread\_join()的线程被取消，则目标线程不应被分离

**参数**：

1. 线程ID：thread。
2. 退出线程：返回值value\_ptr。

**输出**：

* 0：成功完成。
* ESRCH：找不到与给定ID相对应的线程。
* EDEADLK：检测到死锁或thread的值指定调用线程。
* EINVAL：thread指定的线程不是joinable的。

#### pthread\_detach

实现线程分离，即主线程与子线程分离，子线程结束后，资源自动回收。

**参数**：线程ID：thread。

**输出**：

* 0：成功完成。
* EINVAL：thread是分离线程。
* ESRCH：给定线程ID指定的线程不存在。

#### pthread\_key\_create

分配用于标识线程特定数据的键。pthread\_key\_create 第一个参数为指向一个键值的[指针](https://baike.baidu.com/item/指针/2878304?fromModule=lemma_inlink)，第二个参数指明了一个 destructor 函数，如果这个参数不为空，那么当每个线程结束时，系统将调用这个函数来释放绑定在这个键上的内存块。

**参数**：

1. 键值的[指针](https://baike.baidu.com/item/指针/2878304?fromModule=lemma_inlink)key。
2. destructor 函数入口 destructor。

**输出**：

* 0：创建成功。
* EAGAIN：系统缺乏创建另一个特定于线程的数据密钥所需的资源，或者已超过系统对每个进程的密钥总数施加的限制。
* ENOMEM：内存不足，无法创建密钥。

#### pthread\_setspecific

pthread\_setspecific() 函数应将线程特定的 value 与通过先前调用 pthread\_key\_create()获得的 key 关联起来。不同的线程可能会将不同的值绑定到相同的键上。这些值通常是指向已保留供调用线程使用的动态分配内存块的指针。

**参数**：

1. 键值key。
2. 指针value

**输出**：

* 0：设置成功。
* ENOMEM：内存不足，无法将非NULL值与键关联。
* EINVAL：key的值不合法。

#### pthread\_getspecific

将与key关联的数据读出来，返回数据类型为 void \*，可以指向任何类型的数据。需要注意的是，在使用此返回的指针时，需满足是 void 类型，虽指向关联的数据地址处，但并不知道指向的数据类型，所以在具体使用时，要对其进行强制类型转换。

**参数**：键值key。

**输出**：

* 返回与给定 key 关联的线程特定数据值。
* NULL：没有线程特定的数据值与键关联。

#### pthread\_key\_delete

销毁线程特定数据键。

**参数**：需要删除的键key。

**输出**：

* 0：删除成功。
* EINVAL：key值无效。

#### pthread\_getcpuclockid

暂不支持。

#### pthread\_getschedparam

获取线程调度策略和优先级属性。

**参数**：

1. 线程对象指针thread。
2. 调度策略指针policy。
3. 调度属性对象指针param。

**输出**：

* 0：删除成功。
* EINVAL：指针未初始化。

#### pthread\_setschedparam

设置线程调度策略和优先级属性。调度策略仅支持SCHED\_FIFO。

**参数**：

1. 线程对象指针thread。
2. 调度策略指针policy。
3. 调度属性对象指针param。

**输出**：

* 0：删除成功。
* EINVAL：指针未初始化。
* ENOTSUP：设置不支持的值。

#### pthread\_kill

暂不支持。

#### pthread\_once

pthread\_once() 函数使用指定once\_contol变量会保证init\_routine函数只执行一次。当前init\_routine函数不支持被取消。

**参数**：

1. 控制变量control。
2. 执行函数init\_routine。

**输出**：

* 0：删除成功。
* EINVAL：指针未初始化。

#### pthread\_sigmask

暂不支持。

#### pthread\_spin\_init

暂不支持。

#### pthread\_spin\_destory

暂不支持。

#### pthread\_spin\_lock

暂不支持。

#### pthread\_spin\_trylock

暂不支持。

#### pthread\_spin\_unlock

暂不支持。

### 信号量管理

#### sem\_init

sem\_init()函数应初始化 sem 引用的匿名信号量。初始化信号量的值应为 value。在成功调用 sem\_init()后，信号量可用于后续调用 sem\_wait()、sem\_timedwait()、sem\_trywait()、sem\_post()和sem\_destroy()。此信号量应保持可用，直到信号量被销毁。

**参数**：

1. 指向信号量指针sem。
2. 指明信号量的类型pshared。
3. 信号量值的大小value。

**输出**：

* 0：初始化成功。
* EINVAL：值参数超过{SEM\_VALUE\_MAX}。
* ENOSPC：初始化信号量所需的资源已耗尽，或已达到信号量的限制。
* EPERM：缺乏初始化信号量的权限。

#### sem\_destroy

sem\_destroy()函数销毁 sem 指示的匿名信号量。只有使用 sem\_init()创建的信号量才能使用 sem\_destroy()销毁；使用命名信号量调用 sem\_destroy()的效果未定义。在 sem 被另一个对 sem\_init()的调用重新初始化之前，后续使用信号量 sem 的效果是未定义的。

**参数**：指向信号量指针sem。

**输出**：

* 0：销毁成功。
* EINVAL：sem不是有效的信号量。
* EBUSY：信号量上当前有线程被阻止。

#### sem\_open

创建并初始化有名信号量。此信号量可用于后续对 sem\_wait()、sem\_timedwait()、sem\_trywait()、sem\_post()和sem\_close() 的调用。

**参数**：

1. 信号量名无me指针。

2. oflag参数控制信号量是创建还是仅通过调用sem\_open()访问。以下标志位可以在oflag中设置：

   * O\_CREAT：如果信号量不存在，则此标志用于创建信号量。

   * O\_EXCL：如果设置了O\_EXCL和O\_CREAT，且信号量名称存在，sem\_open()将失败。如果设置了O\_EXCL而未设置O\_CREAT，则效果未定义。

3. 如果在oflag参数中指定了O\_CREAT和O\_EXCL以外的标志，则效果未指定。

**输出**：

* 创建并初始化成功，返回信号量地址。
* EACCES：创建命名信号量的权限被拒绝。
* EEXIST：已设置O\_CREAT和O\_EXCL，且命名信号量已存在。
* EINTR：sem\_open()操作被信号中断。
* EINVAL：给定名称不支持sem\_open()，或在oflag中指定了O\_CREAT，并且值大于最大值。
* EMFILE：当前使用的信号量描述符或文件描述符太多。
* ENAMETOOLONG：name参数的长度超过{PATH\_MAX}，或者路径名组件的长度超过{NAME\_MAX}。
* ENFILE：系统中当前打开的信号量太多。
* ENOENT：未设置O\_CREAT且命名信号量不存在。
* ENOSPC：没有足够的空间来创建新的命名信号量。

#### sem\_close

关闭一个命名信号量。未命名的信号量（由sem\_init() 创建的信号量）调用 sem\_close() 的效果未定义。sem\_close() 函数应解除系统分配给此信号量的任何系统资源。此过程后续使用sem指示的信号量的影响未定义。

**参数**：信号量指针sem。

**输出**：

* 0： 销毁成功。
* EINVAL：sem参数不是有效的信号量描述符。

#### sem\_wait

sem\_wait()函数通过对 sem 引用的信号量执行信号量锁定操作来锁定该信号量。如果信号量值当前为零，则调用线程在锁定信号量或调用被信号中断之前，不会从对 sem\_wait()的调用返回。

**参数**：信号量指针sem。

**输出**：

* 0：操作成功。
* EAGAIN：信号量已被锁定，无法立即被 sem\_trywait()操作。
* EDEADLK：检测到死锁条件。
* EINTR：信号中断了此功能。
* EINVAL：sem参数未引用有效的信号量。

#### sem\_trywait

只有当信号量当前未锁定时，即信号量值当前为正值，sem\_trywait()函数才应锁定 sem 引用的信号量。否则它不应锁定信号量。

**参数**：信号量指针sem。

**输出**：

* 0：操作成功。
* EAGAIN：信号量已被锁定，无法立即被sem\_trywait()操作。
* EDEADLK：检测到死锁条件。
* EINTR：信号中断了此功能。
* EINVAL：sem参数未引用有效的信号量。

#### sem\_timedwait

sem\_timedwait()函数应锁定 sem 引用的信号量，就像 sem\_wait()函数一样。如果在不等待另一个线程执行sem\_post()解锁信号量的情况下无法锁定信号量，则在指定的超时到期时，此等待将终止。

**参数**：

1. 信号量指针sem。
2. 阻塞时间指针abs\_timeout。

**输出**：

* 0：操作成功。
* EINVAL：线程可能会阻塞，abs\_timeout 指定的纳秒值小于0或大于等于1000 million。
* ETIMEDOUT：在指定的超时到期之前，无法锁定信号量。
* EDEADLK：检测到死锁条件。
* EINTR：信号中断了此功能。
* EINVAL：sem参数未引用有效的信号量。

#### sem\_post

sem\_post()函数应通过对 sem 引用的信号量执行信号量解锁操作，当有线程阻塞在这个信号量上时，调用这个函数会使其中一个线程不在阻塞，选择机制是由线程的调度策略决定的。

**参数**：信号量指针sem。

**输出**：

* 0：操作成功。
* EINVAL：sem参数未引用有效的信号量。

#### sem\_getvalue

sem\_getvalue()函数获取 sem 引用的信号量的值，而不影响信号量的状态。获取的 sval 值表示在调用期间某个未指定时间发生的实际信号量值。

**参数**：

1. 信号量指针sem。
2. 信号量计数值指针sval。

**输出**：

* 0：操作成功。
* EINVAL：sem参数未引用有效的信号量。

#### sem\_unlink

sem\_unlink() 函数将删除由字符串名称命名的信号量。如果信号量当前被其他进程引用，那么sem\_unlink() 将不会影响信号量的状态。如果在调用sem\_unlink() 时一个或多个进程打开了信号量，则信号量的销毁将被推迟，直到信号量的所有引用都被销毁了。

**参数**：信号量名称name。

**输出**：

* 0：操作成功。
* -1：name参数未引用有效的信号量。

### 互斥量管理

#### pthread\_mutexattr\_init

pthread\_mutexattr\_init()函数初始化互斥锁。如果调用 pthread\_mutexattr\_init()指定已初始化的attr属性对象行为未定义。

**参数**：互斥锁属性对象指针attr。

**输出**：

* 0：操作成功。
* ENOMEM：内存不足，无法初始化互斥属性对象。

#### pthread\_mutexattr\_destroy

注销一个互斥锁。销毁一个互斥锁即意味着释放它所占用的资源，且要求锁当前处于开放状态。

**参数**：互斥锁属性对象指针attr。

**输出**：

* 0：操作成功。
* EINVAL：attr指定的值无效。

#### pthread\_mutexattr\_settype

pthread\_mutexattr\_settype()函数设置互斥 type 属性。默认值为 PTHREAD\_MUTEX\_DEFAULT。有效的互斥类型包括：

PTHREAD\_MUTEX\_NORMAL：此类型的互斥锁不会检测死锁。

* 如果线程在不解除互斥锁的情况下尝试重新锁定该互斥锁，则会产生死锁。
* 如果尝试解除由其他线程锁定的互斥锁，会产生不确定的行为。
* 如果尝试解除锁定的互斥锁未锁定，则会产生不确定的行为。

PTHREAD\_MUTEX\_ERRORCHECK：此类型的互斥锁可提供错误检查。

* 如果线程在不解除锁定互斥锁的情况下尝试重新锁定该互斥锁，则会返回错误。
* 如果线程尝试解除锁定的互斥锁已经由其他线程锁定，则会返回错误。
* 如果线程尝试解除锁定的互斥锁未锁定，则会返回错误。

PTHREAD\_MUTEX\_RECURSIVE：

* 如果线程在不解除锁定互斥锁的情况下尝试重新锁定该互斥锁，则可成功锁定该互斥锁。 与 PTHREAD\_MUTEX\_NORMAL 类型的互斥锁不同，对此类型互斥锁进行重新锁定时不会产生死锁情况。多次锁定互斥锁需要进行相同次数的解除锁定才可以释放该锁，然后其他线程才能获取该互斥锁。
* 如果线程尝试解除锁定的互斥锁已经由其他线程锁定，则会返回错误。
* 如果线程尝试解除锁定的互斥锁未锁定，则会返回错误。

PTHREAD\_MUTEX\_DEFAULT：

* 如果尝试以[递归](https://baike.baidu.com/item/递归?fromModule=lemma_inlink)方式锁定此类型的互斥锁，则会产生不确定的行为。
* 对于不是由调用线程锁定的此类型互斥锁，如果尝试对它解除锁定，则会产生不确定的行为。
* 对于尚未锁定的此类型互斥锁，如果尝试对它解除锁定，也会产生不确定的行为。

**参数**：

1. 互斥锁属性对象指针attr。
2. 互斥锁类型type。

**输出**：

* 0：操作成功。
* EINVAL：attr指定的值无效，或type无效。

#### pthread\_mutexattr\_gettype

pthread\_mutexattr\_gettype() 可用来获取由 pthread\_mutexattr\_settype() 设置的互斥锁的 type 属性。

**参数**：

1. 互斥锁属性对象指针attr。
2. 互斥锁类型指针type。

**输出**：

* 0：操作成功。
* EINVAL：attr指定的值无效。

#### pthread\_mutexattr\_setprotocol

pthread\_mutexattr\_setprotocol() 可用来设置互斥锁属性对象的协议属性。定义的 protocol 可以为以下值之一：

* PTHREAD\_PRIO\_NONE
* PTHREAD\_PRIO\_INHERIT
* PTHREAD\_PRIO\_PROTECT(当前版本暂不支持)

**参数**：

1. 互斥锁属性对象指针 attr。
2. 互斥锁属性对象的协议 protocol。

**输出**：

* 0：操作成功。
* ENOTSUP：协议指定的值不支持。
* EINVAL：attr指定的值无效。
* EPERM：调用者没有权限。

#### pthread\_mutexattr\_getprotocol

pthread\_mutexattr\_getprotocol() 获取互斥锁属性对象的协议属性。

**参数**：

1. 互斥锁属性对象指针attr。
2. 互斥锁属性对象的协议指针protocol。

**输出**：

* 0：操作成功。
* EINVAL：attr指定的值无效。
* EPERM：调用者没有权限。

#### pthread\_mutexattr\_getprioceiling

暂不支持。

#### pthread\_mutexattr\_setprioceiling

暂不支持。

#### pthread\_mutexattr\_getpshared

获取互斥锁属性对象的共享属性。当前支持PTHREAD\_PROCESS\_PRIVATE，互斥锁为进程内私有。

**参数**：

1. 互斥锁属性对象指针attr。
2. 共享属性指针pshared。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_mutexattr\_setpshared

暂不支持。

#### pthread\_mutexattr\_getrobust

获取互斥锁属性对象的健壮属性。当前支持PTHREAD\_MUTEX\_STALLED，如果互斥锁的所有者在持有互斥锁时终止，则不会执行特殊操作。

**参数**：

1. 互斥锁属性对象指针attr。
2. 健壮属性指针robust。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_mutexattr\_setrobust

设置互斥锁属性对象的健壮属性。当前支持PTHREAD\_MUTEX\_STALLED。

**参数**：

1. 互斥锁属性对象指针attr。
2. 健壮属性robust。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。
* ENOTSUP：设置不支持的值。

#### pthread\_mutex\_init

pthread\_mutex\_init()函数初始化互斥锁，属性由 attr 指定。如果 attr 为NULL，则使用默认互斥属性。

**参数**：

1. 互斥锁指针mutex。
2. 互斥锁属性对象指针attr。

**输出**：

* 0：操作成功。
* EAGAIN：缺少初始化互斥锁所需的资源（内存除外）。
* ENOMEM：内存不足，无法初始化互斥体。
* EPERM：没有执行操作的权限。
* EBUSY：互斥锁已经初始化但尚未销毁。
* EINVAL：attr指定的值无效。

#### pthread\_mutex\_destroy

pthread\_mutex\_destroy() 用于注销一个互斥锁。销毁一个互斥锁即意味着释放它所占用的资源，且要求锁当前处于开放状态。

**参数**：互斥锁指针mutex。

**输出**：

* 0：操作成功。
* EBUSY：锁当前未处于开放状态。
* EINVAL：mutex指定的值无效。

#### pthread\_mutex\_lock

当pthread\_mutex\_lock() 返回时，该[互斥锁](https://baike.baidu.com/item/互斥锁/841823?fromModule=lemma_inlink)已被锁定。[线程](https://baike.baidu.com/item/线程/103101?fromModule=lemma_inlink)调用该函数让互斥锁上锁，如果该互斥锁已被另一个线程锁定和拥有，则调用该线程将阻塞，直到该互斥锁变为可用为止。

**参数**：互斥锁指针mutex。

**输出**：

* 0：操作成功。
* EINVAL：mutex指定的值未初始化。
* EAGAIN：无法获取互斥锁。
* EDEADLK：当前线程已经拥有互斥锁。

#### pthread\_mutex\_trylock

pthread\_mutex\_trylock() 语义与 pthread\_mutex\_lock() 类似，不同点在于锁已经被占据时返回 EBUSY， 而非挂起等待。

**参数**：互斥锁指针mutex。

**输出**：

* 0，操作成功。
* EBUSY：mutex指定的锁已经被占据。
* EINVAL：mutex指定的值未初始化。
* EAGAIN：无法获取互斥锁。
* EDEADLK：当前线程已经拥有互斥锁。

#### pthread\_mutex\_timedlock

pthread\_mutex\_timedlock() 语义与pthread\_mutex\_lock() 类似，不同点在于锁已经被占据时增加一个超时时间，等待超时返回错误码。

**参数**：

1. 互斥锁指针mutex。
2. 超时时间指针abs\_timeout。

**输出**：

* 0：操作成功。
* EINVAL：mutex指定的值未初始化，abs\_timeout指定的纳秒值小于0或大于等于1000 million。
* ETIMEDOUT：等待超时。
* EAGAIN：无法获取互斥锁。
* EDEADLK：当前线程已经拥有互斥锁。

#### pthread\_mutex\_unlock

释放互斥锁。

**参数**：互斥锁指针mutex。

**输出**：

* 0：操作成功。
* EINVAL：mutex指定的值未初始化。
* EPERM：当前线程不拥有互斥锁。

#### pthread\_mutex\_consistent

暂不支持。

#### pthread\_mutex\_getprioceiling

暂不支持。

#### pthread\_mutex\_setprioceiling

暂不支持。

### 读写锁编程

#### pthread\_rwlock\_init

pthread\_rwlock\_init()初始化读写锁。如果 attr 为 NULL，则使用默认的读写锁属性。一旦初始化，锁可以使用任何次数，而无需重新初始化。调用 pthread\_rwlock\_init()指定已初始化的读写锁行为未定义。如果在没有初始化的情况下使用读写锁，则结果是未定义的。

**参数**：

1. 读写锁指针rwlock。
2. 读写锁属性指针attr。

**输出**：

* 0：操作成功。
* EAGAIN：系统缺少初始化读写锁所需的资源（内存除外）。
* ENOMEM：内存不足，无法初始化读写锁。
* EPERM：没有执行操作的权限。
* EBUSY：rwlock是以已初始化但尚未销毁的读写锁。
* EINVAL：attr指定的值无效。

#### pthread\_rwlock\_destroy

pthread\_rwlock\_destroy()函数应销毁 rwlock 引用的读写锁，并释放锁使用的资源。在再次调用pthread\_rwlock\_init()重新初始化锁之前，后续使用锁的行为未定义。如果在任何线程持有 rwlock 时调用pthread\_rwlock\_destroy()行为未定义。尝试销毁未初始化的读写锁行为未定义。

**参数**：读写锁指针rwlock。

**输出**：

* 0：操作成功。
* EBUSY： rwlock引用的对象被锁定时销毁该对象。
* EINVAL：attr指定的值无效。

#### pthread\_rwlock\_rdlock

pthread\_rwlock\_rdlock()函数应将读锁应用于rwlock引用的读写锁。

**参数**：读写锁指针rwlock。

**输出**：

* 0：操作成功。
* EINVAL：rwlock是未初始化的读写锁。
* EAGAIN：无法获取读锁，因为已超过rwlock的最大读锁数。
* EDEADLK：检测到死锁条件或当前线程已拥有写锁。

#### pthread\_rwlock\_tryrdlock

pthread\_rwlock\_tryrdlock()函数语义与pthread\_rwlock\_rdlock()类似。在任何情况下，pthread\_rwlock\_tryrdlock()函数都不会阻塞；它会一直获取锁，或者失败并立即返回。

**参数**：读写锁指针rwlock。

**输出**：

* 0：操作成功。
* EINVAL：rwlock是未初始化的读写锁。
* EAGAIN：无法获取读锁，因为已超过rwlock的最大读锁数。
* EBUSY：无法获取读写锁以进行读取，因为写入程序持有该锁。

#### pthread\_rwlock\_timedrdlock

pthread\_rwlock\_timedrdlock()语义与pthread\_rwlock\_rdlock()类似，不同的是在锁已经被占据时增加一个超时时间，等待超时返回错误码。

**参数**：

1. 读写锁指针rwlock。
2. 超时时间指针abs\_timeout。

**输出**：

* 0：操作成功。
* ETIMEDOUT：在指定的超时到期之前，无法获取锁。
* EAGAIN：无法获取读锁，超过锁的最大读锁数量。
* EDEADLK：检测到死锁条件或调用线程已在rwlock上持有写锁。
* EINVAL：rwlock指定的锁未初始化，或者abs\_timeout纳秒值小于0或大于等于1000 million。

#### pthread\_rwlock\_wrlock

pthread\_rwlock\_wrlock()函数将写锁应用于 rwlock 引用的读写锁。如果没有其他线程持有读写锁 rwlock，调用线程将获得写锁。否则，线程应阻塞，直到它能够获得锁。如果调用线程在调用时持有读写锁（无论是读锁还是写锁），则调用线程可能会死锁。

**参数**：读写锁指针rwlock。

**输出**：

* 0：操作成功。
* EINVAL：rwlock指定的值未初始化。
* EDEADLK：检测到死锁情况，或者当前线程已经拥有用于写入或读取的读写锁。

#### pthread\_rwlock\_trywrlock

pthread\_rwlock\_trywrlock()函数类似 pthread\_rwlock\_wrlock()，但如果任何线程当前持有rwlock（用于读取或写入，该函数将失败）。

**参数**：读写锁指针rwlock。

**输出**：

* 0：操作成功。
* EBUSY：无法获取读写锁以进行写入，因为它已被锁定以进行读取或写入。
* EINVAL：rwlock指定的值未初始化。

#### pthread\_rwlock\_timedwrlock

pthread\_rwlock\_timedwrlock()语义与pthread\_rwlock\_wrlock()类似，不同的是在锁已经被占据时增加一个超时时间，等待超时返回错误码。

**参数**：

1. 读写锁指针rwlock。
2. 超时时间指针abs\_timeout。

**输出**：

* 0：操作成功。
* ETIMEDOUT：在指定的超时到期之前，无法获取锁。
* EAGAIN：无法获取读锁，超过锁的最大读锁数量。
* EDEADLK：检测到死锁条件或调用线程已在rwlock上持有写锁。
* EINVAL；rwlock指定的锁未初始化，或者abs\_timeout纳秒值小于0或大于等于1000 million。

#### pthread\_rwlock\_unlock

pthread\_rwlock\_unlock()函数释放rwlock引用的读写锁上持有的锁。如果读写锁rwlock未被调用线程持有，则结果未定义。

**参数**：读写锁指针rwlock。

**输出**：

* 0：操作成功。
* EINVAL：rwlock指定的锁未初始化。
* EPERM：当前线程不持有读写锁。

#### pthread\_rwlockattr\_init

暂不支持

#### pthread\_rwlockattr\_destroy

暂不支持

#### pthread\_rwlockattr\_getpshared

pthread\_rwlockattr\_getpshared() 函数从attr引用的读写锁属性对象中获取进程共享属性的值。

**参数**：

1. 读写锁属性指针attr。
2. 共享属性指针pshared。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

### pthread\_rwlockattr\_setpshared

设置读写锁属性对象中进程共享属性的值。当前支持PTHREAD\_PROCESS\_PRIVATE，读写锁为进程私有。

**参数**：

1. 读写锁属性指针attr。
2. 共享属性指针pshared。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。
* ENOTSUP：设置不支持的值。

### 线程屏障管理

#### pthread\_barrier\_destroy

销毁线程屏障变量，并释放该屏障使用的任何资源。

**参数**：屏障变量指针b。

**输出**：

* 0：操作成功。
* EBUSY：另一个线程在使用该变量。

#### pthread\_barrier\_init

分配线程屏障变量所需的资源，并使用attr的属性初始化屏障。如果attr为NULL，则使用默认的屏障属性。

**参数**：

1. 屏障变量指针b。
2. 屏障属性指针attr。
3. 等待线程个数count。

**输出**：

* 0：操作成功。
* EINVAL：count为0。
* ENOTSUP：attr指定的屏障属性不支持。
* EAGAIN：系统缺乏初始化一个屏障所需的资源。

#### pthread\_barrier\_wait

pthread\_barrier\_wait() 阻塞调用线程，直到等待的线程达到了预定的数量。

**参数**：屏障变量指针b。

**输出**：

* 0：操作成功。
* -1：第一个线程成功返回。

#### pthread\_barrierattr\_getpshared

获取屏障属性的共享属性值。

**参数**：

1. 屏障属性指针a。
2. 共享属性值指针pshared。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_barrierattr\_setpshared

设置屏障属性的共享属性值。支持PTHREAD\_PROCESS\_PRIVATE，该屏障为进程私有的，不允许不同进程的线程访问该屏障。

**参数**：

1. 屏障属性指针a。
2. 共享属性值pshared。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。
* ENOTSUP：试图将属性设置为不支持的值。

### 条件变量管理

#### pthread\_cond\_init

使用attr引用的属性初始化cond引用的条件变量。如果attr为NULL，则使用默认条件变量属性。

**参数**

1. 条件变量指针cond。
2. 条件变量属性指针attr。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。
* EAGAIN：系统缺乏初始化一个条件变量所需的资源。

#### pthread\_cond\_destroy

销毁指定条件变量，使得该条件变量未初始化，可以使用pthread\_cond\_init() 重新初始化。

**参数**：条件变量指针cond。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。
* EBUSY：另一个线程在使用该变量。

#### pthread\_cond\_broadcast

pthread\_cond\_broadcast（）函数取消阻塞指定条件变量cond上当前阻塞的所有线程。

**参数**：条件变量指针cond。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_cond\_signal

pthread\_cond\_signal() 函数取消阻塞在指定的条件变量cond上阻塞的线程中的至少一个（如果有任何线程在cond上被阻塞）。

**参数**：条件变量指针cond。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_cond\_timedwait

pthread\_cond\_timedwait() 函数阻塞当前线程等待cond指定的条件变量，并释放互斥体指定的互斥体。只有在另一个线程使用相同的条件变量调用pthread\_cond\_signal() 或pthread\_cond\_broadcast() 后，或者如果系统时间达到指定的时间，并且当前线程重新获得互斥锁时，等待线程才会解锁。

**参数**：

1. 条件变量指针cond。
2. 互斥锁指针m。
3. 超时时间指针ts。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。
* ETIMEDOUT：阻塞超时

#### pthread\_cond\_wait

pthread\_cond\_wait() 函数与pthread\_cond\_timedwait() 类似，阻塞当前线程等待cond指定的条件变量，并释放互斥体指定的互斥体。只有在另一个线程使用相同的条件变量调用pthread\_cond\_signal() 或pthread\_cond\_broadcast() 后，并且当前线程重新获得互斥锁时，等待线程才会解锁。

**参数**：

1. 条件变量指针cond。
2. 互斥锁指针m。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_condattr\_init

使用属性的默认值初始化条件变量属性对象attr。

**参数**： 条件变量属性对象指针attr。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_condattr\_destroy

pthread\_condattr\_destroy）函数销毁条件变量属性对象，使对象变得未初始化，可以使用pthread\_condattr\_init() 重新初始化。

**参数**：条件变量属性对象指针attr。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_condattr\_getclock

从attr引用的属性对象中获取时钟属性的值。

**参数**：

1. 条件变量属性对象指针attr。
2. 时钟属性指针clk。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_condattr\_setclock

设置attr引用的属性对象中时钟属性的值。当前支持CLOCK\_REALTIME，采用系统时间。

**参数**：

1. 条件变量属性对象指针attr。
2. 时钟属性clock。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。
* ENOTSUP：设置不支持的值。

#### pthread\_condattr\_getpshared

从attr引用的属性对象中获取共享属性的值。

**参数**：

1. 条件变量属性对象指针attr。
2. 共享属性指针pshared。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_condattr\_setpshared

设置attr引用的属性对象中共享属性属性的值。当前支持PTHREAD\_PROCESS\_PRIVATE，该条件变量为进程私有的，不允许不同进程的线程访问。

**参数**：

1. 条件变量属性对象指针attr。
2. 共享属性pshared。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。
* ENOTSUP：设置不支持的值。

### 时钟管理

#### asctime

asctime() 函数将timeptr指向的tm结构体对象转换为的字符串。

**参数**： tm结构体指针timeptr。

**输出**：

* 成功则返回字符串指针。
* 失败返回NULL。

#### asctime\_r

与asctime() 函数类似，将timeptr指向的tm结构体对象转换为的字符串。不同的是该字符串放置在用户提供的缓冲区buf（至少包含26字节）中，然后返回buf。

**参数**：

1. tm结构体指针timeptr。
2. 字符串缓冲区buf。

**输出**：

* 成功则返回字符串指针。
* 失败返回NULL。

#### clock

返回该进程使用等处理器时间的最佳近似值。

**参数**：无

**输出**：

* 成功则返回时间。
* -1：失败。

#### clock\_gettime

clock\_gettime()函数应返回指定时钟的当前值tp。

**参数**：

1. 时钟类型clock\_id。
2. timespec结构体指针tp。

**输出**：

返回值：

* 0：操作成功。
* -1：操作失败。

errno：

* EINVAL：clock\_id不合法。
* ENOTSUP：clock\_id不支持。

#### clock\_settime

clock\_settime()函数应将指定的clock\_id设置为tp指定的值。

**参数**：

1. 时钟类型clock\_id。
2. timespec结构体指针tp。

**输出**：

返回值：

* 0：操作成功。
* -1：操作失败。

errno：

* EINVAL：clock\_id不合法，或tp参数指定的纳秒值小于0或大于等于1000 million。
* ENOTSUP：clock\_id不支持。

#### clock\_getres

clock\_getres()返回时钟的分辨率。如果参数res不为NULL，则指定时钟的分辨率应存储在res指向的位置。如果res为NULL，则不返回时钟分辨率。如果clock\_settime()的时间参数不是res的倍数，则该值将被截断为res的倍数。

**参数**：

1. 时钟类型clock\_id。
2. timespec结构体指针res。

**输出**：

返回值：

* 0：操作成功。
* -1：操作失败。

errno：

* EINVAL：clock\_id不合法。
* ENOTSUP：clock\_id不支持。

#### clock\_getcpuclockid

clock\_getcpuclockid函数获取CPU时间时钟的ID，当前进程只有一个，因此无论传入的pid是什么，都返回CLOCK\_PROCESS\_CPUTIME\_ID。

**参数**：

1. 进程ID：pid。
2. 时钟指针：clk。

**输出**：

* 0：操作成功。

#### clock\_nanosleep

与nanosleep类似，clock\_nanosleep() 允许调用线程在以纳秒精度指定的时间间隔内休眠，并可以将睡眠间隔指定为绝对值或相对值。当前支持CLOCK\_REALTIME。

**参数**：

1. 时钟ID：clk。
2. 是否为绝对值：flag。
3. 指定的时间间隔值eq。
4. 剩余时间值：rem。

**输出**：

* 0：操作成功。
* -1：操作失败。
* EINVAL：时钟ID错误。
* ENOTSUP：不支持的时钟ID。

#### nanosleep

nanosleep()函数应导致当前线程暂停执行，直到rqtp参数指定的时间间隔过去或信号传递到调用线程。挂起时间可能比请求的长，因为参数值被四舍五入到睡眠分辨率的整数倍，或者因为系统调度了其他活动。但是，除被信号中断外，暂停时间不得小于rqtp规定的时间。

如果rmtp参数是非NULL，则更新其为剩余的时间量（请求的时间减去实际睡眠时间）。如果rmtp参数为NULL，则不返回剩余时间。

**参数**：

1. timespec结构体指针rqtp。
2. timespec结构体指针rmtp。

**输出**：

返回值：

* 0：操作成功。
* -1：操作失败。

errno：

* EINVAL：rqtp参数指定的纳秒值小于0或大于等于1000 million。
* EINTR：信号中断。

#### sleep

sleep()函数应导致调用线程暂停执行，直到参数seconds指定的实时秒数过去或信号被传递到调用线程。由于系统安排了其他活动，暂停时间可能比请求的要长。

**参数**： 秒数seconds。

**输出**：

* 0：操作成功。
* 如果由于信号的传递而返回，则返回值应为“未睡眠”量，以秒为单位。

#### timer\_create

timer\_create()函数使用指定的时钟clock\_id作为时序基创建计时器，在timerid引用的位置返回计时器ID，用于标识计时器。在删除计时器之前，此计时器ID在调用过程中应是唯一的。

**参数**：

1. 时钟类型clock\_id。
2. sigevent结构体指针evp。(仅支持SIGEV\_THREAD)
3. 定时器ID指针timerid。

**输出**：

* 0：操作成功。
* EINVAL：clock\_id不合法。
* EAGAIN：系统缺少足够的资源来满足请求。
* EINVAL：指定的时钟ID未定义。
* ENOTSUP：不支持创建附加到clock\_id时钟上的计时器。

#### timer\_delete

删除定时器。

**参数**：定时器ID指针timerid。

**输出**：

* 0：操作成功。
* EINVAL：timerid不合法。

#### timer\_settime

如果value的it\_value成员非0，timer\_settime()函数从value参数的it\_value成员设置timerid指定的计时器的到期时间。如果在调用timer\_settime()时指定的计时器已启用，则此调用应将下次到期的时间重置为指定的值。如果value的it\_value成员为0，则应解除计时器。

**参数**：

1. 定时器ID timerid。
2. 计时器的特征flag。
3. itimerspec结构体指针value。
4. itimerspec结构体指针ovalue。返回上一次计时器设置超时时间。

**输出**：

* 0：操作成功。
* EINVAL：timerid不合法。

#### timer\_gettime

timer\_gettime() 函数存储定时器 timerid 的剩余时间以及间隔。value 的 it\_value 成员包含计时器到期前的时间量，如果计时器已解除，则为零。value 的 it\_interval 成员将包含 timer\_settime() 上次设置的间隔时间。

**参数**：

1. 定时器ID timerid。
2. itimerspec结构体指针value。

**输出**：

* 0：操作成功。
* EINVAL：timerid不合法。

#### timer\_getoverrun

根据指定的定时器ID，获取定时器的超时次数。

**参数**：

1. 定时器ID timerid。
2. itimerspec结构体指针value。

**输出**：

* 非负数：超时次数。
* -1：操作失败。

errno：

* EINVAL：无效ID或定时器未初始化。

#### times

获取进程的执行时间。由于UniProton无用户模式/内核模式且无子进程概念，出参和返回值均为进程执行总时间。

**参数**：

1. tms结构体指针ts。

**输出**：

* 非负数：进程的执行时间。

#### ctime

ctime() 函数将tp指向的time\_t结构体对象转换为的字符串。效果等同于asctime(localtime(t))。

**参数**： time\_t结构体指针tp。

**输出**：

* 成功则返回字符串指针。
* 失败返回NULL。

#### ctime\_r

ctime\_r() 函数将tp指向的time\_t结构体对象转换为的字符串，并将字符串放入buf指向的数组中（其大小应至少为26字节）并返回buf。

**参数**：

1. tm结构体指针timeptr。
2. 字符串缓冲区buf。

**输出**：

* 成功则返回字符串指针。
* 失败返回NULL。

#### difftime

计算两个日历时间之间的差值（由第一个参数减去第二个参数）。

**参数**：

1. 第一个时间值t1。
2. 第二个时间值t0。

**输出**：

* 返回时间差值。

#### getdate

暂不支持

#### gettimeofday

gettimeofday() 函数应获取当前时间，并将其存储在tp指向的timeval结构中。如果时区结果tz不是空指针，则行为未指定。

**参数**：

1. timeval结构体指针tp。
2. 时区指针tz。

**输出**：

* 返回0。

#### gmtime

将time\_t结构表示的日历时间转换为tm结构表示的时间，无时区转换。

**参数**：time\_t结构体指针。

**输出**：

返回值：

* tm结构体指针。

errno：

* EOVERFLOW：转换溢出。

#### gmtime\_r

与gmtime函数类似，不同的是gmtime\_r会将结果放入在用户提供的tm结构体中。

**参数**：

1. time\_t结构体指针。
2. tm结构体指针。

**输出**：

返回值：

* tm结构体指针。

errno：

* EOVERFLOW：转换溢出。

#### localtime

将time\_t结构表示的日历时间转换为tm结构表示的本地时间，受时区的影响。

**参数**：time\_t结构体指针。

**输出**：

返回值：

* tm结构体指针。

errno：

* EOVERFLOW：转换溢出。

#### localtime\_r

与localtime函数类似，不同的是localtime\_r会将结果放入在用户提供的tm结构体中。

**参数**：

1. time\_t结构体指针。
2. tm结构体指针。

**输出**：

返回值：

* tm结构体指针。

errno：

* EOVERFLOW：转换溢出。

#### mktime

将已经根据时区信息计算好的tm结构表示的时间转换为time\_t结构表示的时间戳，受时区的影响。

**参数**：tm结构体指针。

**输出**：

返回值：

* time\_t结构体指针。

errno：

* EOVERFLOW：转换溢出。

#### strftime

暂不支持

#### strftime\_l

暂不支持

#### strptime

使用format指定的格式，将buf指向的字符串解析转换为tm结构体的时间值。

**参数**：

1. 时间字符串buf。
2. 格式字符串format。
3. tm结构体指针tp。

**输出**：

* 成功则返回指针，指向解析的最后一个字符后面的字符。
* 失败返回NULL。

#### time

获取当前的日历时间，即从一个标准时间点到此时的时间经过的秒数。

**参数**：time\_t结构体指针t。

**输出**：time\_t结构体指针t。

#### timespec\_get

返回基于给定时基base的时间，由timespec结构体保存。时基通常为TIME\_UTC。

**参数**：

1. timespec结构体指针ts。
2. 时基base

**输出**：

* 成功则返回时基的值。
* 失败则返回0。

#### utime

暂不支持。

#### wcsftime

暂不支持

#### wcsftime\_l

暂不支持

### 内存管理

#### malloc

malloc()分配大小（以字节为单位）size 的未使用的空间。

**参数**：大小size。

**输出**：分配成功时，返回指向分配空间的指针。

* 如果size 为0，则返回空指针或可以成功传递给 free()的唯一指针。
* 否则它将返回一个空指针，并设置 errno 来指示错误：ENOMEM 存储空间不足。

#### free

Free()函数释放ptr指向的空间，即可供进一步分配。如果ptr是空指针，则不发生任何操作。如果空间已被对free()或realloc()的调用释放，则行为未定义。

**参数**：指针ptr。

**输出**：无

#### memalign

memalign()函数将分配按align大小字节对齐，大小为len的内存空间指针。

**参数**：align是对齐字节数，len指定分配内存的字节大小。

**输出**：成功完成后，空间大小为len的指针。

#### realloc

realloc()函数将释放ptr所指向的旧对象，并返回一个指向新对象的指针，该对象的大小由size指定。并拷贝旧指针指向的内容到新指针，然后释放旧指针指向的空间。如果ptr是空指针，则realloc()对于指定的大小应等同于malloc()。

**参数**：旧指针地址；新指针的目标分配空间大小。

**输出**：在成功完成后，realloc()将返回一个指向分配空间的指针。如果size为0，则行为不可预测。

#### malloc\_usable\_size

malloc\_usable\_size()函数返回ptr所指向的块中的可用字节数。

**参数**：待计算内存块大小的指针。

**输出**：返回ptr指向的已分配内存块中的可用字节数。如果ptr为NULL，则返回0。

#### aligned\_alloc

aligned\_alloc()函数分配size字节未初始化的存储空间，按照alignment指定对齐。

**参数**：alignment指定对齐；size是分配的字节数。

**输出**：返回指向新分配内存的指针。

#### reallocarray

reallocarray()函数将释放ptr所指向的旧对象，并返回一个指向新对象的指针，该对象的大小由size由入参m和n决定。等同于realloc(ptr, m \* n);

**参数**：ptr待释放的指针内容，m和n代表数组的长度和单个元素的字节数。

**输出**：在成功完成后返回一个指向分配空间的指针。如果size为0，则行为不可预测。

#### calloc

calloc()函数将为一个数组分配未使用的空间，并将该空间应初始化为所有位0。

**参数**：m和n分别代表数组的元素个数或单个元素的大小。

**输出**：分配成功时，返回指向分配空间的指针。失败时则行为不可预测。

#### posix\_memalign

posix\_memalign()函数将分配按align指定的边界对齐的大小字节，并返回指向在memptr中分配的内存的指针。对齐的值应该是sizeof(void \*)的2倍幂。

**参数**：res分配好的内存空间的首地址，align是对齐字节数，len指定分配内存的字节大小。

**输出**：成功完成后，posix\_memalign()将返回零;否则，将返回一个错误号来表示错误，并且不修改memptr的内容，或者将其设置为空指针。

### 退出管理

#### abort

abort()函数触发程序的异常终止，除了信号SIGABRT没有被捕获或者返回。

**参数**：无

**输出**：无

#### \_Exit

\_Exit()函数终止程序。

**参数**：入参是0,EXIT\_SUCCESS, EXIT\_FAILURE或任何其他值。wait()和waitpid()只能获得最低有效的8位(即status & 0377);完整的值应该可以从waitid()和siginfo\_t中获得，SIGCHLD传递给信号处理程序。

**输出**：无

#### atexit

atexit()注册一个在程终止时运行的函数。在正常的程序终止时，所有由atexit()函数注册的函数都应该按照其注册的相反顺序被调用，除非一个函数在之前注册的函数之后被调用，而这些函数在注册时已经被调用了。正常的终止发生在调用exit()或从main()返回时。

**参数**：函数指针，该入参函数不带参数。

**输出**：成功返回0；失败返回非0。

#### quick\_exit

quick\_exit()函数触发快速程序终止，并以后进先出(LIFO)的顺序调用由at\_quick\_exit注册的函数。

**参数**：程序退出的状态码。

**输出**：无

#### at\_quick\_exit

at\_quick\_exit()函数注册由func指向的函数，在快速程序终止时（通过quick\_exit）调用。最多能注册32个函数。

**参数**：指向快速程序退出时要调用的函数的指针。

**输出**：注册成功返回0，否则为非零值。

#### assert

assert()宏将在程序中插入断言,它将扩展为一个void表达式。当它被执行时，如果判断条件失败。assert()将写失败特定的调用信息，并将调用abort()退出程序。

**参数**：判断表达式。

**输出**：无

### stdlib接口

#### div

div()函数计算int型除法的商和余数。如果余数或商不能表示，结果是未知的。

**参数**：int numer(分子), int denom(分母)。

**输出**：结构体div\_t，int型的商和余数。

#### ldiv

ldiv()函数将计算long型除法的商和余数。如果余数或商不能表示，结果是未知的。

**参数**：long numer(分子), long denom(分母)。

**输出**：结构体ldiv\_t，long型的商和余数。

#### lldiv

lldiv()函数将计算long long型除法的商和余数。如果余数或商不能表示，结果是未知的。

**参数**：long long numer(分子), long long denom(分母)。

**输出**：结构体lldiv\_t，long long型的商和余数。

#### imaxdiv

imaxdiv()函数将计算intmax\_t型除法的商和余数。如果余数或商不能表示，结果是未知的。

**参数**：intmax\_t numer(分子), intmax\_t denom(分母)。

**输出**：结构体imaxdiv\_t，intmax\_t型的商和余数。

#### wcstol

wcstol()将宽字符串转换为long型正数。输入字符串分解为三部分。

1. 初始的(可能为空的)空白宽字符代码序列(由iswspace()指定)。
2. long型整数，进制的类型由base入参决定。
3. 由一个或多个无法识别的宽字符代码组成的最终宽字符串。

**参数**：指向要解释的以空字符结尾的宽字符串的指针；指向宽字符的指针；解释的整数值的基数。

**输出**：转换后的long型数值。如果无法进行转换，则返回0，并设置errno表示错误。如果正确值在可表示的值范围之外，则返回LONG\_MIN，LONG\_MAX，LLONG\_MIN或LLONG\_MAX，并将errno设置为ERANGE。

#### wcstod

wcstod()将宽字符串转换为double型浮点数。输入字符串分解为三部分。

1. 初始的(可能为空的)空白宽字符代码序列(由iswspace()指定)。
2. double型浮点数、无穷大或者NaN。
3. 由一个或多个无法识别的宽字符代码组成的最终宽字符串。

**参数**：指向要解释的以空字符结尾的宽字符串的指针；指向宽字符的指针；

**输出**：转换后的double型浮点数。如果越界，则可能返回±HUGE\_VAL, ±HUGE\_VALF或±HUGE\_VALL，并将errno设置为ERANGE。

#### fcvt

fcvt()将浮点数转换为要求长度的字符串，没有小数点，如果超过value的数字长度将补零。

**参数**：待转换的浮点数、转换后字符串的长度、小数点所在位指针、符号位指针。

**输出**：转换后字符串指针。

#### ecvt

ecvt()函数将浮点数转换为要求长度的字符串，没有小数点，如果超过value的数字长度不补零(与fcvt的区别)。

**参数**：待转换的浮点数、转换后字符串的长度、小数点所在位指针、符号位指针。

**输出**：转换后字符串指针。

#### gcvt

gcvt()函数将double类型的值转换为要求长度的字符串，包含小数点。

**参数**：待转换的浮点数，转换后字符串的长度、转换后字符串指针。

**输出**：转换后字符串指针(等于函数成功调用后第三个入参的指针)。

#### qsort

qsort()函数对数据表进行排序。

**参数**：qsort()函数将对nel对象数组进行排序，该数组的初始元素由base指向。每个对象的大小(以字节为单位)由width参数指定。如果nel参数的值为0，则不会调用comp所指向的比较函数，也不会进行重排。应用程序应确保compar所指向的比较函数不会改变数组的内容。实现可以在调用比较函数之间对数组元素重新排序，但不能改变任何单个元素的内容。

**输出**：无

#### abs

abs()函数计算并返回int型数值的绝对值。

**参数**：int整型数值。

**输出**：int整型数值的绝对值。

#### labs

labs()函数计算并返回long型数值的绝对值。

**参数**：long型数值。

**输出**：long型数值的绝对值。

#### llabs

llabs()函数计算并返回long long型数值的绝对值。

**参数**：long long型数值。

**输出**：long long型数值的绝对值。

#### imaxabs

imaxabs()函数计算并返回intmax\_t型的绝对值。

**参数**：intmax\_t型数值。

**输出**：intmax\_t型数值的绝对值。

#### strtol

strtol()函数转换字符串到long型数值。这将nptr所指向的字符串的初始部分转换为long类型的表示形式。首先，它们将输入字符串分解为三部分。

1. 一个初始的、可能为空的空白字符序列(由isspace()函数判断)。
2. long型整数，进制的类型由base入参决定。
3. 由一个或多个不可识别字符组成的最后字符串，包括输入字符串的终止NUL字符。

**参数**：待转换的字符串的指针；指向字符的指针；解释的整数值的基数。

**输出**：转换后的long型。如果无法进行转换，则返回0，并设置errno表示错误。如果正确值在可表示的值范围之外，则返回LONG\_MIN， LONG\_MAX， LLONG\_MIN或LLONG\_MAX，并将errno设置为EINVAL。

#### strtod

strtod()函数将字符串转换为double型。输入字符串分解为三部分。

1. 初始的(可能为空的)空白字符代码序列(由isspace()指定)。
2. double型浮点数、无穷大或者NaN。
3. 由一个或多个无法识别的字符代码组成的最终字符串。

**参数**：待转换的字符串的指针；指向字符的指针；

**输出**：转换后的double型浮点数。如果越界，则可能返回±HUGE\_VAL, ±HUGE\_VALF或±HUGE\_VALL，并将errno设置为EINVAL。

#### atoi

atoi()函数将字符串转换为int型整数。

**参数**：待转换的字符串的指针。

**输出**：转换后的int型数值。如果是无法显示数值，返回值不可预测。

#### atol

atol()函数将字符串转换为long型整数。

**参数**：待转换的字符串的指针。

**输出**：转换后的long型数值。如果是无法显示数值，返回值不可预测。

#### atoll

atoll()函数将字符串转换为long long型整数。

**参数**：待转换的字符串的指针。

**输出**：转换后的long long型数值。如果是无法显示数值，返回值不可预测。

#### atof

atof()函数将字符串转换为double型浮点数。

**参数**：待转换的字符串的指针。

**输出**：转换后的double型数值。如果是无法显示数值，返回值不可预测。

#### bsearch

bsearch()函数二分查找一个已排序表.将搜索一个nel对象数组，该数组的初始元素由base指向，以查找与key指向的对象匹配的元素。数组中每个元素的大小由width指定。如果nel参数的值为0，则不会调用compar所指向的比较函数，也不会找到匹配项。

**参数**：依次为目标查找的元素，待查找的数组的指针，数组的元素个数，数组每个元素的size大小，两个元素的比较函数。

**输出**：指向数组中匹配成员的指针，如果没找到则返回空指针。

### SystemV IPC

#### semget

semget()函数返回与参数key相关联的SystemV信号量集的标识符。它可用于获得先前创建的信号量集合的标识符（当flag为0且key不为IPC\_PRIVATE时）或来创建一个新的集合。最多可以支持创建SEMSET\_MAX\_SYS\_LIMIT个信号量集合，每个集合最多支持SEMSET\_MAX\_SEM\_NUM个信号量。

**参数**：

1. 键值key。
2. 信号量的个数nsems。
3. 信号量的创建方式和权限flag。

**输出**：

* 非负数：信号量集的标识符。
* -1: 操作失败。

errno：

* EINVAL：参数错误。
* ENOENT：信号量集不存在。
* ENOSPC：超出最大信号量集合的限制。
* EEXIST：flag包含了IPC\_CREAT和IPC\_EXCL但标识符已存在。

#### semctl

semctl()函数在由semid标识的SystemV信号量集合中的第semnum个信号量上执行由cmd指定的控制操作。集合中的信号量从0开始编号。当前支持的cmd包括IPC\_STAT（支持获取信号量集合中的个数）、GETALL（获取信号量集合中所有信号量的值）、GETVAL（获取单个信号量的值）和IPC\_RMID（根据标识符删除信号量集合）。

**参数**：

1. 信号量集合标识符semid。
2. 信号量中的编号semnum。
3. 要执行的操作命令cmd。
4. 可选参数union semun结构体arg。

**输出**：

* 0：操作成功。
* -1: 操作失败。

errno：

* EINVAL：参数错误。
* EIDRM：信号量集合已删除。
* EFAULT：arg中的buf或array指针为空。

#### semop

semop()函数对semid关联的信号量集合中选定的信号量进行操作，也就是使用资源或者释放资源。具体操作由struct sembuf结构体来决定。结构体包括数组索引semnum，信号量操作（支持+1或-1，表示释放资源和使用资源）op，操作方式flag（支持IPC\_NOWAIT，不阻塞操作）。当前只支持单个信号量的操作。

**参数**：

1. 信号量集合标识符semid。
2. 指向struct sembuf结构体的数组sops。
3. 数组个数nsops。

**输出**：

* 0：操作成功。
* -1: 操作失败。

errno：

* EINVAL：参数错误。
* ENOTSUP：操作不支持。
* EFAULT：数组指针sops为空。
* E2BIG：数组个数nsops超过限制。
* EIDRM；信号量集合已删除。
* EFBIG：某个信号量索引超过限制。
* EAGAIN：操作无法立即进行，如flag包含了IPC\_NOWAIT或超时。

#### semtimedop

semtimedop()的行为与semop()相同，不同点在于增加一个超时时间，等待超时返回错误码。

**参数**：

1. 信号量集合标识符semid。
2. 指向struct sembuf结构体的数组sops。
3. 数组个数nsops。
4. timespec结构体指针timeout。

**输出**：

* 0：操作成功。
* -1: 操作失败。

errno：

* EINVAL：参数错误。
* ENOTSUP：操作不支持。
* EFAULT：数组指针sops为空。
* E2BIG：数组个数nsops超过限制。
* EIDRM；信号量集合已删除。
* EFBIG：某个信号量索引超过限制。
* EAGAIN：操作无法立即进行，如flag包含了IPC\_NOWAIT或超时。

#### msgget

msgget()返回与参数key相关联的SystemV消息队列的标识符。它可用于获得先前创建的消息队列的标识符（当flag为0且key不为IPC\_PRIVATE时）或来创建一个新的消息队列。最多支持创建MSGQUE\_MAX\_SYS\_LIMIT个消息队列，消息队列默认大小为MSGQUE\_MAX\_MSG\_NUM，消息大小默认为MSGQUE\_MAX\_MSG\_SIZE。

**参数**：

1. 键值key。
2. 消息队列的创建方式和权限flag。

**输出**：

* 非负数：消息队列的标识符。
* -1: 操作失败。

errno：

* EINVAL：参数错误。
* ENOENT：消息队列不存在。
* ENOSPC：超出最大消息队列的限制。
* EEXIST：flag包含了IPC\_CREAT和IPC\_EXCL但标识符已存在。
* ENOMEM：内存不足。

#### msgctl

msgctl()在标识为msgqid的SystemV消息队列上执行cmd指定的控制操作。当前支持IPC\_STAT（支持获取消息队列中的消息个数和大小）、IPC\_RMID（删除消息队列）。

**参数**：

1. 消息队列标识符msgqid。
2. 消息队列控制命令cmd。
3. 消息队列信息msqid\_ds结构体buf。

**输出**：

* 0：操作成功。
* -1: 操作失败。

errno：

* EINVAL：参数错误。
* EFAULT：msqid\_ds结构体指针为空。
* EIDRM：消息队列已删除。
* ENOTSUP：不支持的命令。

#### msgsnd

msgsnd()将msgp指向的消息追加到msgqid指定的SystemV消息队列中，如果队列有足够空间，msgsnd立即执行。消息大小不超过MSGQUE\_MAX\_MSG\_SIZE。当前flag支持IPC\_NOWAIT，表示操作不等待。

**参数**：

1. 消息队列标识符msgqid。
2. 需要发送的消息msgp。
3. 发送消息的大小msgsz。
4. 发送方式flag。

**输出**：

* 0：操作成功。
* -1: 操作失败。

errno：

* EINVAL：参数错误。
* EFAULT：msgp指针为空。
* EIDRM：消息队列已删除。
* ENOTSUP：不支持的命令。

#### msgrcv

msgrcv()函数将消息从msgqid指定的消息队列中移除，并放入msgp指向的缓冲区中。参数msgsz指定了缓冲区buf的大小。当前msgtype支持的值为0，flag支持IPC\_NOWAIT，表示操作不等待。

**参数**：

1. 消息队列标识符msgqid。
2. 需要接受消息的缓冲区msgp。
3. 接受消息的大小msgsz。
4. 接受消息的类型msgtype。
5. 发送方式flag。

**输出**：

* 0：操作成功。
* -1: 操作失败。

errno：

* EINVAL：参数错误。
* EFAULT：msgp指针为空。
* EIDRM：消息队列已删除。
* ENOTSUP：不支持的命令。
* ENOMSG：消息队列中没有请求类型的消息。

#### shmget

暂不支持

#### shmctl

暂不支持

#### shmat

暂不支持

#### shmdt

暂不支持

#### ftok

暂不支持

## C11接口

|       接口名         |       适配情况        |
|       :---:         |       :-----:       |
|  [cnd\_broadcast](#cnd_broadcast)  |  支持  |
|  [cnd\_destroy](#cnd_destroy)  |  支持  |
|  [cnd\_init](#cnd_init)  |  支持  |
|  [cnd\_signal](#cnd_signal)  |  支持  |
|  [cnd\_timedwait](#cnd_timedwait)  |  支持  |
|  [cnd\_wait](#cnd_wait)  |  支持  |
|  [mtx\_destroy](#mtx_destroy)  |  支持  |
|  [mtx\_init](#mtx_init)  |  支持  |
|  [mtx\_lock](#mtx_lock)  |  支持  |
|  [mtx\_timedlock](#mtx_timedlock)  |  支持  |
|  [mtx\_trylock](#mtx_trylock)  |  支持  |
|  [thrd\_create](#thrd_create)  |  支持  |
|  [thrd\_current](#thrd_current)  |  支持  |
|  [thrd\_detach](#thrd_detach)  |  支持  |
|  [thrd\_equal](#thrd_equal)  |  支持  |
|  [thrd\_exit](#thrd_exit)  |  支持  |
|  [thrd\_join](#thrd_join)  |  支持  |
|  [thrd\_sleep](#thrd_sleep)  |  支持  |
|  [thrd\_yield](#thrd_yield)  |  支持  |
|  [tss\_create](#tss_create)  |  支持  |
|  [tss\_delete](#tss_delete)  |  支持  |
|  [tss\_get](#tss_get)  |  支持  |
|  [tss\_set](#tss_set)  |  支持  |

### 条件变量管理

#### cnd\_init

初始化条件变量cond。同使用条件变量属性为NULL的pthread\_cond\_init()。

**参数**

1. 条件变量指针cond。
2. 条件变量属性指针attr。

**输出**：

* thrd\_success：操作成功。
* thrd\_error：操作失败。

#### cnd\_destroy

销毁指定条件变量，使得该条件变量未初始化，可以使用cnd\_init() 重新初始化。同pthread\_cond\_destory()。

**参数**：条件变量指针cond。

**输出**：无。

#### cnd\_broadcast

取消阻止当前等待cond所指向的条件变量的所有线程。如果没有线程被阻塞，则不执行任何操作并返回thrd\_success。

**参数**：条件变量指针cond。

**输出**：

* thrd\_success：操作成功。
* thrd\_error：操作失败。

#### cnd\_signal

取消阻塞在指定的条件变量cond上阻塞的线程中的至少一个（如果有任何线程在cond上被阻塞）。

**参数**：条件变量指针cond。

**输出**：

* thrd\_success：操作成功。
* thrd\_error：操作失败。

#### cnd\_timedwait

阻塞当前线程等待cond指定的条件变量，并释放互斥体指定的互斥体。只有在另一个线程使用相同的条件变量调用cnd\_signal() 或cnd\_broadcast() 后，或者如果系统时间达到指定的时间，并且当前线程重新获得互斥锁时，等待线程才会解锁。

**参数**：

1. 条件变量指针cond。
2. 互斥锁指针m。
3. 超时时间指针ts。

**输出**：

* thrd\_success：操作成功。
* thrd\_error：操作失败。
* thrd\_timedout：阻塞超时

#### cnd\_wait

cnd\_wait() 函数与cnd\_timedwait() 类似，阻塞当前线程等待cond指定的条件变量，并释放互斥体指定的互斥体。只有在另一个线程使用相同的条件变量调用cnd\_signal() 或cnd\_broadcast() 后，并且当前线程重新获得互斥锁时，等待线程才会解锁。

**参数**：

1. 条件变量指针cond。
2. 互斥锁指针m。

**输出**：

* thrd\_success：操作成功。
* thrd\_error：操作失败。

### 互斥锁管理

#### mtx\_init

mtx\_init()函数根据属性type初始化互斥锁。

**参数**：

1. 互斥锁指针mutex。
2. 互斥锁属性type。

**输出**：

* thrd\_success：操作成功。
* thrd\_error：操作失败。

#### mtx\_destroy

mtx\_destroy() 用于注销一个互斥锁。销毁一个互斥锁即意味着释放它所占用的资源，且要求锁当前处于开放状态。

**参数**：互斥锁指针mutex。

**输出**：无。

#### mtx\_lock

当pthread\_mutex\_lock() 返回时，该[互斥锁](https://baike.baidu.com/item/互斥锁/841823?fromModule=lemma_inlink)已被锁定。[线程](https://baike.baidu.com/item/线程/103101?fromModule=lemma_inlink)调用该函数让互斥锁上锁，如果该互斥锁已被另一个线程锁定和拥有，则调用该线程将阻塞，直到该互斥锁变为可用为止。

**参数**：互斥锁指针mutex。

**输出**：

* thrd\_success：操作成功。
* thrd\_error：操作失败。

#### mtx\_timedlock

mtx\_timedlock() 语义与mtx\_lock() 类似，不同点在于锁已经被占据时增加一个超时时间，等待超时返回错误码。

**参数**：

1. 互斥锁指针mutex。
2. 超时时间指针ts。

**输出**：

* thrd\_success：操作成功。
* thrd\_error：操作失败。
* thrd\_timedout：等待超时。

#### mtx\_trylock

mtx\_trylock() 语义与 mtx\_lock() 类似，不同点在于锁已经被占据时返回 thrd\_busy， 而非挂起等待。

**参数**：互斥锁指针mutex。

**输出**：

* thrd\_success：操作成功。
* thrd\_busy：mutex指定的锁已经被占据。
* thrd\_error：操作失败。

### 任务管理

#### thrd\_create

thrd\_create()函数创建一个执行函数为func的新线程，创建成功后，将创建的线程的ID存储在参数 thread 的位置。

**参数**：

1. 指向线程[标识符](https://baike.baidu.com/item/标识符?fromModule=lemma_inlink)的[指针](https://baike.baidu.com/item/指针?fromModule=lemma_inlink)thread。
2. 线程处理函数的起始地址 func。
3. 运行函数的参数 arg。

**输出**：

* thrd\_success：创建成功。
* thrd\_error：attr指定的属性无效。
* thrd\_nomem：系统缺少创建新线程所需的资源。

#### thrd\_current

返回调用线程的线程ID。

**参数**：无

**输出**：返回调用线程的线程ID。

#### thrd\_detach

实现线程分离，即主线程与子线程分离，子线程结束后，资源自动回收。

**参数**：线程ID：thread。

**输出**：

* 0：成功完成。
* EINVAL：thread是分离线程。
* ESRCH：给定线程ID指定的线程不存在。

#### thrd\_equal

此函数应比较线程ID t1和t2。

**参数**：

1. 线程ID t1。
2. 线程ID t2。

**输出**：

* 如果t1和t2相等，pthread\_equal()函数应返回非零值。
* 如果t1和t2不相等，应返回零。
* 如果t1或t2不是有效的线程ID，则行为未定义。

#### thrd\_exit

线程的终止可以是调用 thrd\_exit 或者该线程的例程结束。由此可看出，一个线程可以隐式退出，也可以显式调用 thrd\_exit 函数来退出。thrd\_exit 函数唯一的参数 value\_ptr 是函数的返回代码，只要 thrd\_join 中的第二个参数 value\_ptr 不是NULL，这个值将被传递给 value\_ptr。

**参数**：线程退出状态value\_ptr，通常传NULL。

**输出**：无

#### thrd\_join

thrd\_join() 函数，以阻塞的方式等待 thread 指定的线程结束。当函数返回时，被等待线程的资源被收回。如果线程已经结束，那么该函数会立即返回。并且 thread 指定的线程必须是 joinable 的。当 thrd\_join()成功返回时，目标线程已终止。对指定同一目标线程的thrd\_join()的多个同时调用的结果未定义。如果调用thrd\_join()的线程被取消，则目标线程不应被分离。

**参数**：

1. 线程ID：thread。
2. 退出线程：返回值value\_ptr。

**输出**：

* thrd\_success：操作成功。

#### thrd\_sleep

至少在达到time\_point指向的基于TIME\_UTC的时间点之前，阻塞当前线程的执行。如果收到未被忽略的信号，睡眠可能会恢复。

**参数**：

1. 应等待时间：req。
2. 实际等待时间：rem。

**输出**：

* 0：操作成功。
* -2: 操作失败。

#### thrd\_yield

thrd\_yield()函数应强制正在运行的线程放弃处理器，并触发线程调度。

**参数**：无

**输出**：输出0时，成功完成；否则应返回值-1。

#### tss\_create

分配用于标识线程特定数据的键。tss\_create 第一个参数为指向一个键值的[指针](https://baike.baidu.com/item/指针/2878304?fromModule=lemma_inlink)，第二个参数指明了一个 destructor 函数，如果这个参数不为空，那么当每个线程结束时，系统将调用这个函数来释放绑定在这个键上的内存块。

**参数**：

1. 键值的[指针](https://baike.baidu.com/item/指针/2878304?fromModule=lemma_inlink)tss。
2. destructor 函数入口 destructor。

**输出**：

* thrd\_success：操作成功。
* thrd\_error：操作失败。

#### tss\_delete

销毁线程特定数据键。

**参数**：需要删除的键key。

**输出**：无

#### tss\_get

将与key关联的数据读出来，返回数据类型为 void \*，可以指向任何类型的数据。需要注意的是，在使用此返回的指针时，需满足是 void 类型，虽指向关联的数据地址处，但并不知道指向的数据类型，所以在具体使用时，要对其进行强制类型转换。

**参数**：键值key。

**输出**：

* 返回与给定 key 关联的线程特定数据值。
* NULL：没有线程特定的数据值与键关联。

#### tss\_set

tss\_set() 函数应将线程特定的 value 与通过先前调用 tss\_create()获得的 key 关联起来。不同的线程可能会将不同的值绑定到相同的键上。这些值通常是指向已保留供调用线程使用的动态分配内存块的指针。

**参数**：

1. 键值key。
2. 指针value

**输出**：

\_ 0：设置成功。

## 其他接口

|       接口名         |       适配情况        |
|       :---:         |       :-----:       |
|  [pthread\_getattr\_default\_np](#pthread_getattr_default_np)  |  支持  |
|  [pthread\_getattr\_np](#pthread_getattr_np)  |  支持  |
|  [pthread\_getname\_np](#pthread_getattr_np)  |  支持  |
|  [pthread\_setattr\_default\_np](#pthread_setattr_default_np)  |  支持  |
|  [pthread\_setname\_np](#pthread_setname_np)  |  支持  |
|  [pthread\_timedjoin\_np](#pthread_timedjoin_np)  |  支持  |
|  [pthread\_tryjoin\_np](#pthread_tryjoin_np)  |  支持  |
|  [ftime](#ftime)  |  支持  |
|  [timegm](#timegm)  |  支持  |

### pthread\_getattr\_default\_np

pthread\_getattr\_default\_np() 函数初始化attr引用的线程属性对象，使其包含用于创建线程的默认属性。

**参数**：线程属性对象attr。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

### pthread\_setattr\_default\_np

pthread\_setattr\_default\_np() 函数用于设置创建新线程的默认属性，即当使用NULL的第二个参数调用pthread\_create时使用的属性。

**参数**：线程属性对象attr。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

### pthread\_getattr\_np

pthread\_getattr\_np() 函数初始化attr引用的线程属性对象，使其包含描述正在运行的线程线程的实际属性值。

**参数**：

1. 线程ID值thread。
2. 线程属性对象attr。

**输出**：

* 0：操作成功。
* 非0值：操作失败。

### pthread\_getname\_np

pthread\_getname\_np() 函数可用于检索线程的名称。thread参数指定要检索其名称的线程。

**参数**：

1. 线程ID值thread。
2. 线程名字符串name。
3. 字符串大小len。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

### pthread\_setname\_np

pthread\_setname\_np() 函数可用于设置线程的名称。

**参数**：

1. 线程ID值thread。
2. 线程名字符串name。
3. 字符串大小len。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。

#### pthread\_timedjoin\_np

类似pthread\_join，如果线程尚未终止，则调用将阻塞直到abstime中指定的最大时间。如果超时在线程终止之前到期，则调用将返回错误。

**参数**：

1. 线程ID值thread。
2. 线程退出状态status。
3. 阻塞时间指针ts。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。
* ETIMEDOUT：阻塞超时。

### pthread\_tryjoin\_np

类似pthread\_join，但如果线程尚未终止，将立即返回EBUSY。

**参数**：

1. 线程ID值thread。
2. 线程退出状态status。

**输出**：

* 0：操作成功。
* EINVAL：指针未初始化。
* EBUSY：调用时线程尚未终止。

### ftime

取得当前的时间和日期，由一个timeb结构体返回。

**参数**：

1. timeb结构体指针tp。

**输出**：无

### timegm

将tm结构体表示的时间转换为自一个标准时间点以来的时间，不受本地时区的影响。

**参数**：

1. tm结构体指针tp。

**输出**：

返回值：

* time\_t结构体表示的时间值。
* -1: 转换失败。

errno：

* EOVERFLOW：转换溢出。

## math数学库

|       接口名         |       描述        |       输入参数        |       适配情况        |
|       :---:         |       :-----:    |       :-----:     |       :-----:       |
|  acos  | 计算参数x的反余弦值，参数x的取值范围\[-1, +1]，返回类型double | double类型的浮点数x |  支持  |
|  acosf  | 计算参数x的反余弦值，参数x的取值范围\[-1, +1]，返回类型float | float类型的浮点数x |  支持  |
|  acosl  | 计算参数x的反余弦值，参数x的取值范围\[-1, +1]，返回类型long double | long double类型的浮点数x |  支持  |
|  acosh  | 计算参数x的反双曲余弦值，返回类型double | double类型的浮点数x |  支持  |
|  acoshf  | 计算参数x的反双曲余弦值，返回类型float | float类型的浮点数x |  支持  |
|  acoshl  | 计算参数x的反双曲余弦值，返回类型long double | long double类型的浮点数x |  支持  |
|  asin  |  计算参数x的反正弦值，参数x的取值范围为\[-1, +1] |  duoble类型的浮点数x  |  支持  |
|  asinf  |  计算参数x的反正弦值，参数x的取值范围为\[-1, +1] |  float类型的浮点数x  |  支持  |
|  asinl  |  计算参数x的反正弦值，参数x的取值范围为\[-1, +1] |  long double类型的浮点数x  |  支持  |
|  asinh  |  计算参数x的反双曲正弦值，返回类型double |  double类型的浮点数x  |  支持  |
|  asinhf  |  计算参数x的反双曲正弦值，返回类型float |  float类型的浮点数x  |  支持  |
|  asinhl  |  计算参数x的反双曲正弦值，返回类型long double |  long double类型的浮点数x  |  支持  |
|  atan  | 计算参数x的反正切值，返回类型double |  double类型的浮点数x  |  支持  |
|  atanf  |  计算参数x的反正切值，返回类型float | float类型的浮点数x  |  支持  |
|  atanl  |  计算参数x的反正切值，返回类型long double | long double类型的浮点数x  |  支持  |
|  atan2  |  计算参数y除以x的反正切值，使用两个参数的符号确定返回值的象限 | double类型的浮点数ydouble类型的浮点数x  |  支持  |
|  atan2f  | 计算参数y除以x的反正切值，使用两个参数的符号确定返回值的象限 | float类型的浮点数yfloat类型的浮点数x  |  支持  |
|  atan2l  | 计算参数y除以x的反正切值，使用两个参数的符号确定返回值的象限 | long double类型的浮点数ylong double类型的浮点数x  |  支持  |
|  atanh  |  计算参数x的反双曲正切值，返回类型double | double类型的浮点数x  |  支持  |
|  atanhf  | 计算参数x的反双曲正切值，返回类型float | float类型的浮点数x  |  支持  |
|  atanhl  |  计算参数x的反双曲正切值，返回类型long double | long double类型的浮点数x  |  支持  |
|  cbrt  |  计算参数x的立方根，返回类型double | double类型的浮点数x  |  支持  |
|  cbrtf  |  计算参数x的立方根，返回类型float | float类型的浮点数x  |  支持  |
|  cbrtl  |  计算参数x的立方根，返回类型long double | long double类型的浮点数x  |  支持  |
|  ceil  |  计算不小于参数x的最小整数值，返回类型double  | duoble类型的浮点数x  |  支持  |
|  ceilf  | 计算不小于参数x的最小整数值，返回类型float  | float类型的浮点数x  |  支持  |
|  ceill  |  计算不小于参数x的最小整数值，返回类型long double  | long duoble类型的浮点数x  |  支持  |
|  copysign  |  生成一个值，该值具有参数x的大小和参数y的符号  | duoble类型的浮点数xdouble类型的浮点数y  |  支持  |
|  copysignf  |  生成一个值，该值具有参数x的大小和参数y的符号  | float类型的浮点数xfloat类型的浮点数y  |  支持  |
|  copysignl  |  生成一个值，该值具有参数x的大小和参数y的符号  | long duoble类型的浮点数xlong double类型的浮点数y  |  支持  |
|  cos  |  计算参数x的余弦值，参数应为弧度值，返回类型double | duoble类型的浮点数x  |  支持  |
|  cosf  | 计算参数x的余弦值，参数应为弧度值，返回类型float | float类型的浮点数x  |  支持  |
|  cosl  | 计算参数x的余弦值，参数应为弧度值，返回类型long double | long double类型的浮点数x  |  支持  |
|  cosh  |  计算参数x的双曲余弦值，返回类型double | double类型的浮点数x  |  支持  |
|  coshf  | 计算参数x的双曲余弦值，返回类型float | float类型的浮点数x  |  支持  |
|  coshl  | 计算参数x的双曲余弦值，返回类型long double | long double类型的浮点数x  |  支持  |
|  erf  | 计算参数x的高斯误差函数的值 | double类型的浮点数x  |  支持  |
|  erff  | 计算参数x的高斯误差函数的值 | float类型的浮点数x  |  支持  |
|  erfl  | 计算参数x的高斯误差函数的值 | long double类型的浮点数x  |  支持  |
|  erfc  | 计算参数x的高斯误差函数的值 | double类型的浮点数x  |  支持  |
|  erfcf  | 计算参数x的互补误差函数的值 | float类型的浮点数x  |  支持  |
|  erfcl  | 计算参数x的互补误差函数的值 | long double类型的浮点数x  |  支持  |
|  exp  |  以e为基数的指数，即$e^x$的值，返回类型double  | double类型的浮点数x  |  支持  |
|  expf  |  以e为基数的指数，即$e^x$的值，返回类型float  | float类型的浮点数x  |  支持  |
|  expl  |  以e为基数的指数，即$e^x$的值，返回类型long double  | long double类型的浮点数x  |  支持  |
|  exp10  | 以10为基数的指数，即$10^x$的值，返回类型double | double类型的浮点数x  |  支持  |
|  exp10f  | 以10为基数的指数，即$10^x$的值，返回类型float | float类型的浮点数x  |  支持  |
|  exp10l  |  以10为基数的指数，即$10^x$的值，返回类型long double | long double类型的浮点数x  |  支持  |
|  exp2  |  以2为基数的指数函数，返回类型double  | double类型的浮点数x  |  支持  |
|  exp2f  |  以2为基数的指数函数，返回类型float  | float类型的浮点数x  |  支持  |
|  exp2l  |  以2为基数的指数函数，返回类型long double  | long double类型的浮点数x  |  支持  |
|  expm1  | 计算$e^x - 1$的值。如果参数x是个小值，expm1(x)函数的值比表达式$e^x - 1$更准确 | double类型的浮点数x |  支持  |
|  expm1f  | 计算$e^x - 1$的值。如果参数x是个小值，expm1(x)函数的值比表达式$e^x - 1$更准确 | float类型的浮点数x |  支持  |
|  expm1l  | 计算$e^x - 1$的值。如果参数x是个小值，expm1(x)函数的值比表达式$e^x - 1$更准确 | long double类型的浮点数x |  支持  |
|  fabs  |  计算参数x的绝对值，返回类型double  |  double类型的浮点数x  |  支持  |
|  fabsf  |  计算参数x的绝对值，返回类型float  |  float类型的浮点数x  |  支持  |
|  fabsl  |  计算参数x的绝对值，返回类型long double  | long double类型的浮点数x  |  支持  |
|  fdim  |  计算参数x和参数y之间的正差值  | double类型的浮点数xdouble类型的浮点数y |  支持  |
|  fdimf  |  计算参数x和参数y之间的正差值  | float类型的浮点数xfloat类型的浮点数y |  支持  |
|  fdiml  |  计算参数x和参数y之间的正差值  | long double类型的浮点数xlong double类型的浮点数y |  支持  |
|  finite  |  如果参数x既不是无限值也不是NaN，则返回一个非零值，否则返回0  | double类型的浮点数x |  支持  |
|  finitef  |  如果参数x既不是无限值也不是NaN，则返回一个非零值，否则返回0  | float类型的浮点数x|  支持  |
|  floor  |  计算不大于参数x到最大整数值，返回类型double  | double类型的浮点数x  |  支持  |
|  floorf  |  计算不大于参数x到最大整数值，返回类型float  | float类型的浮点数x  |  支持  |
|  floorl  |  计算不大于参数x到最大整数值，返回类型long double  | long double类型的浮点数x  |  支持  |
|  fma  | 计算表达式$(x \* y) + z$的值，返回double类型 | double类型的浮点数xdouble类型的浮点数ydouble类型的浮点数z  |  支持  |
|  fmaf  | 计算表达式$(x \* y) + z$的值，返回float类型 | float类型的浮点数xfloat类型的浮点数yfloat类型的浮点数z  |  支持  |
|  fmal  | 计算表达式$(x \* y) + z$的值，返回long double类型 | long double类型的浮点数xlong double类型的浮点数ylong double类型的浮点数z |  支持  |
|  fmax  | 确定其参数的最大数值。如果一个参数是非数值(NaN)，另一个参数是数值，fmax函数将选择数值 | double类型的浮点数xdouble类型的浮点数y |  支持  |
|  fmaxf  | 确定其参数的最大数值。如果一个参数是非数值(NaN)，另一个参数是数值，fmax函数将选择数值 | float类型的浮点数xfloat类型的浮点数y |  支持  |
|  fmaxl  | 确定其参数的最大数值。如果一个参数是非数值(NaN)，另一个参数是数值，fmax函数将选择数值 | long double类型的浮点数xlong double类型的浮点数y |  支持  |
|  fmin  | 返回其参数的最小数值。非数值NaN参数视为缺失数据。如果一个参数是非数值，另一个参数是数值，fmin函数将选择数值 | double类型的浮点数xdouble类型的浮点数y |  支持  |
|  fminf  | 返回其参数的最小数值。非数值NaN参数视为缺失数据。如果一个参数是非数值，另一个参数是数值，fmin函数将选择数值 | float类型的浮点数xfloat类型的浮点数y |  支持  |
|  fminl  | 返回其参数的最小数值。非数值NaN参数视为缺失数据。如果一个参数是非数值，另一个参数是数值，fmin函数将选择数值 | long double类型的浮点数xlong double类型的浮点数y |  支持  |
|  fmod  |  计算表达式x/y的浮点余数，返回double类型 |  double类型的浮点数xdouble类型的浮点数y |  支持  |
|  fmodf  | 计算表达式x/y的浮点余数，返回float类型 |  float类型的浮点数xfloat类型的浮点数y |  支持  |
|  fmodl  | 计算表达式x/y的浮点余数，返回long double类型 | long double类型的浮点数xlong double类型的浮点数y |  支持  |
|  frexp  | 将浮点数分解为规格化小数和2的整数幂，并将整数存入参数exp指向的对象中 | double类型的浮点数xint *类型的浮点数y |  支持  |
|  frexpf  | 将浮点数分解为规格化小数和2的整数幂，并将整数存入参数exp指向的对象中 | float类型的浮点数xint *类型的浮点数y |  支持  |
|  frexpl  | 将浮点数分解为规格化小数和2的整数幂，并将整数存入参数exp指向的对象中 | long double类型的浮点数xint *类型的浮点数y |  支持  |
|  hypot  | 计算表达式$(x^2 + y^2)^{1/2}$的值 | double类型的浮点数xdouble类型的浮点数y |  支持  |
|  hypotf  | 计算表达式$(x^2 + y^2)^{1/2}$的值 | float类型的浮点数xfloat类型的浮点数y |  支持  |
|  hypotl  | 计算表达式$(x^2 + y^2)^{1/2}$的值 | long double类型的浮点数xlong double类型的浮点数y |  支持  |
|  ilogb  |  以FLT\_RADIX作为对数的底数，返回double类型x的对数的整数部分  |  double类型的浮点数x  |  支持  |
|  ilogbf  |  以FLT\_RADIX作为对数的底数，返回float类型x的对数的整数部分   |  float类型的浮点数x  |  支持  |
|  ilogbl  |  以FLT\_RADIX作为对数的底数，返回long double类型x的对数的整数部分   |  long double类型的浮点数x  |  支持  |
|  j0  | 计算参数x的第一类0阶贝塞尔函数 | double类型浮点数x |  支持  |
|  j0f  | 计算参数x的第一类0阶贝塞尔函数 | float类型浮点数x |  支持  |
|  j1  | 计算参数x的第一类1阶贝塞尔函数 | double类型浮点数x |  支持  |
|  j1f  | 计算参数x的第一类1阶贝塞尔函数 | float类型浮点数x |  支持  |
|  jn  | 计算参数x的第一类n阶贝塞尔函数 | int类型阶数double类型浮点数x |  支持  |
|  jnf  | 计算参数x的第一类n阶贝塞尔函数 | int类型阶数float类型浮点数x |  支持  |
|  ldexp  | 计算参数x与2的exp次幂的乘积，即返回$x \* 2^{exp}$的double类型值。 | double类型的浮点数xint类型的指数exp |  支持  |
|  ldexpf  | 计算参数x与2的exp次幂的乘积，即返回$x \* 2^{exp}$的float类型值。 | float类型的浮点数xint类型的指数exp |  支持  |
|  ldexpl  | 计算参数x与2的exp次幂的乘积，即返回$x \* 2^{exp}$的long double类型值。 | long double类型的浮点数xint类型的指数exp |  支持  |
|  lgamma  |  计算参数x伽玛绝对值的自然对数，返回double类型  |  double类型的浮点数x  |  支持  |
|  lgammaf  |  计算参数x伽玛绝对值的自然对数，返回float类型  |  float类型的浮点数x  |  支持  |
|  lgammal  |  计算参数x伽玛绝对值的自然对数，返回long double类型  | long double类型的浮点数x  |  支持  |
|  lgamma\_r  |  计算参数x伽玛绝对值的自然对数，与lgamma不同在于是线程安全的  | double类型的浮点数xint *类型符号参数  |  支持  |
|  lgamma\_r  |  计算参数x伽玛绝对值的自然对数，与lgamma不同在于是线程安全的  | float类型的浮点数xint *类型符号参数 |  支持  |
|  llrint  | 根据当前舍入模式，将参数舍入为long long int类型的最接近整数值  | double类型的浮点数x   |  支持  |
|  llrintf  | 根据当前舍入模式，将参数舍入为long long int类型的最接近整数值  | float类型的浮点数x   |  支持  |
|  llrintl  | 根据当前舍入模式，将参数舍入为long long int类型的最接近整数值  | long double类型的浮点数x   |  支持  |
|  llround  |  将double类型x舍入为浮点形式表示的long long int型最近整数值。如果x位于两个整数中心，将向远离0的方向舍入。 | double类型的浮点数x   |  支持  |
|  llroundf  |  将float类型x舍入为浮点形式表示的long long int型最近整数值。如果x位于两个整数中心，将向远离0的方向舍入。 | float类型的浮点数x   |  支持  |
|  llroundl  |  将long double类型x舍入为浮点形式表示的long long int型最近整数值。如果x位于两个整数中心，将向远离0的方向舍入。 | long double类型的浮点数x   |  支持  |
|  log  |  double类型x的自然对数函数  |  double类型的浮点数x  |  支持  |
|  logf  |  float类型x的自然对数函数  |  float类型的浮点数x  |  支持  |
|  logl  |  long double类型x的自然对数函数  |  long double类型的浮点数x  |  支持  |
|  log10  | double类型x以10为底数的对数函数 |  double类型的浮点数x  |  支持  |
|  log10f  | float类型x以10为底数的对数函数 |  float类型的浮点数x  |  支持  |
|  log10l  | long double类型x以10为底数的对数函数 |  long double类型的浮点数x  |  支持  |
|  log1p  | 以e为底数的对数函数，计算$log\_e(1 + x)$的值。如果参数x是个小值，表达式log1p(x)比表达式log(1 + x)更准确 | double类型的浮点数x  |  支持  |
|  log1pf  | 以e为底数的对数函数，计算$log\_e(1 + x)$的值。如果参数x是个极小的值，表达式log1p(x)比表达式log(1 + x)更准确 | float类型的浮点数x  |  支持  |
|  log1pl  | 以e为底数的对数函数，计算$log\_e(1 + x)$的值。如果参数x是个极小的值，表达式log1p(x)比表达式log(1 + x)更准确 | long double类型的浮点数x  |  支持  |
|  log2  |  double类型x以2为底数的对数函数 |  double类型的浮点数x  |  支持  |
|  log2f  |  float类型x以2为底数的对数函数 |  flaot类型的浮点数x  |  支持  |
|  log2l  |  long double类型x以2为底数的对数函数 |  long double类型的浮点数x  |  支持  |
|  logb  |  double类型x以FLT\_RADIX为的底数到对数函数  | double类型的浮点数x   |  支持  |
|  logbf  |  float类型x以FLT\_RADIX为的底数到对数函数  | float类型的浮点数x   |  支持  |
|  logbl  |  double类型x以FLT\_RADIX为的底数到对数函数  | double类型的浮点数x   |  支持  |
|  lrint  |  根据当前舍入模式，将参数舍入为long int类型的最接近整数值 | double类型的浮点数x   |  支持  |
|  lrintf  |  根据当前舍入模式，将参数舍入为long int类型的最接近整数值 | float类型的浮点数x   |  支持  |
|  lrintl  |  根据当前舍入模式，将参数舍入为long int类型的最接近整数值 | long double类型的浮点数x   |  支持  |
|  lround  |  将double类型x舍入为浮点形式表示的long int型最近整数值。如果x位于两个整数中心，将向远离0的方向舍入。 | double类型的浮点数x   |  支持  |
|  lroundf  |  将float类型x舍入为浮点形式表示的long int型最近整数值。如果x位于两个整数中心，将向远离0的方向舍入。 | float类型的浮点数x   |  支持  |
|  lroundl  |  将long double类型x舍入为浮点形式表示的long int型最近整数值。如果x位于两个整数中心，将向远离0的方向舍入。 | long double类型的浮点数x   |  支持  |
|  modf | 将double类型的参数value分成整数部分和小数部分，两部分与参数value具有相同的类型和符号。整数部分以浮点形式存入参数iptr指向的对象中 | double类型的浮点数valuedouble *类型的指数iptr |  支持  |
|  modff  | 将float类型的参数value分成整数部分和小数部分，两部分与参数value具有相同的类型和符号。整数部分以浮点形式存入参数iptr指向的对象中 | float类型的浮点数valuefloat *类型的指数iptr |  支持  |
|  modfl  | 将long double类型的参数value分成整数部分和小数部分，两部分与参数value具有相同的类型和符号。整数部分以浮点形式存入参数iptr指向的对象中 | long double类型的浮点数valuelong double *类型的指数iptr |  支持  |
|  nan  | 返回一个double类型的非数值NaN，内容由参数tagp确定 | const char*类型tagp |  支持  |
|  nanf  | 返回一个float类型的非数值NaN，内容由参数tagp确定 | const char*类型tagp |  支持  |
|  nanl  | 返回一个long double类型的非数值NaN，内容由参数tagp确定 | const char*类型tagp |  支持  |
|  nearbyint  | 根据当前舍入模式，将double型参数x舍入为浮点格式的double型整数值 | double类型x |  支持  |
|  nearbyintf  | 根据当前舍入模式，将float型参数x舍入为浮点格式的float型整数值 | float类型x |  支持  |
|  nearbyintl  | 根据当前舍入模式，将long double型参数x舍入为浮点格式的double型整数值 | long double类型x |  支持  |
|  nextafter  | 返回double类型参数x沿参数y方向的下一个可表示值 | double类型xdouble类型y |  支持  |
|  nextafterf  | 返回double类型参数x沿参数y方向的下一个可表示值 | float类型xflaot类型y |  支持  |
|  nextafterl  | 返回double类型参数x沿参数y方向的下一个可表示值 | long double类型xlong double类型y |  支持  |
|  nexttoward  | 返回double类型参数x沿参数y方向的下一个可表示值，等价于nextafter，区别在于参数y为long double | double类型浮点数xlong double类型浮点数y |  支持  |
|  nexttowardf  | 返回double类型参数x沿参数y方向的下一个可表示值，等价于nextafter，区别在于参数y为long double | float类型浮点数xlong double类型浮点数y |  支持  |
|  nexttowardl  | 返回double类型参数x沿参数y方向的下一个可表示值，等价于nextafter，区别在于参数y为long double | long double类型浮点数xlong double类型浮点数y |  支持  |
|  pow  | 计算表达式$x^y$的值 | double类型浮点数xdouble类型浮点数y |  支持  |
|  powf  | 计算表达式$x^y$的值 | float类型浮点数xfloat类型浮点数y |  支持  |
|  powl  | 计算表达式$x^y$的值 | long double类型浮点数xlong double类型浮点数y |  支持  |
|  pow10  | 计算表达式$10^x$的值 | double类型浮点数x |  支持  |
|  pow10f  | 计算表达式$10^x$的值 | float类型浮点数x|  支持  |
|  pow10l  | 计算表达式$10^x$的值 | long double类型浮点数x |  支持  |
|  remainder  | 计算参数x除以y的余数，等同于drem | double类型浮点数xdouble类型浮点数y |  支持  |
|  remainderf  | 计算参数x除以y的余数，等同于dremf | float类型浮点数xfloat类型浮点数y |  支持  |
|  remainderl  | 计算参数x除以y的余数 | long double类型浮点数xlong double类型浮点数y |  支持  |
|  remquo  | 计算参数x和参数y的浮点余数，并将商保存在传递的参数指针quo中 | double类型浮点数xdouble类型浮点数yint *类型商que |  支持  |
|  remquof  | 计算参数x和参数y的浮点余数，并将商保存在传递的参数指针quo中 | float类型浮点数xfloat类型浮点数yint *类型商que |  支持  |
|  remquol  | 计算参数x和参数y的浮点余数，并将商保存在传递的参数指针quo中 | long double类型浮点数xlong double类型浮点数yint *类型商que |  支持  |
|  rint  | 根据当前舍入模式，将参数x舍入为浮点个数的整数值 | double类型的浮点数x |  支持  |
|  rintf  | 根据当前舍入模式，将参数x舍入为浮点个数的整数值 | float类型的浮点数x |  支持  |
|  rintl  | 根据当前舍入模式，将参数x舍入为浮点个数的整数值 | long double类型的浮点数x |  极速  |
|  round  |  将double类型x舍入为浮点形式表示的double型最近整数值。如果x位于两个整数中心，将向远离0的方向舍入。 | double类型的浮点数x   |  支持  |
|  roundf  |  将float类型x舍入为浮点形式表示的float型最近整数值。如果x位于两个整数中心，将向远离0的方向舍入。 | float类型的浮点数x   |  支持  |
|  roundl  |  将long double类型x舍入为浮点形式表示的long double型最近整数值。如果x位于两个整数中心，将向远离0的方向舍入。 | long double类型的浮点数x   |  支持  |
|  scalb  |  计算$x \* FLT\_RADIX^{exp}$的double类型值 | double类型的浮点数xdouble类型的指数exp  |  支持  |
|  scalbf  | 计算$x \* FLT\_RADIX^{exp}$的float类型值 | float类型的浮点数xfloat类型的指数exp  |  支持  |
|  scalbln  | 计算$x \* FLT\_RADIX^{exp}$的double类型值 | double类型的浮点数xlong类型的指数exp |  支持  |
|  scalblnf  | 计算$x \* FLT\_RADIX^{exp}$的float类型值 | float类型的浮点数xlong类型的指数exp |  支持  |
|  scalblnl  | 计算$x \* FLT\_RADIX^{exp}$的long double类型值 | long double类型的浮点数xlong类型的指数exp |  支持  |
|  scalbn  | 计算$x \* FLT\_RADIX^{exp}$的double类型值 | double类型的浮点数xint类型的指数exp |  支持  |
|  scalbnf  | 计算$x \* FLT\_RADIX^{exp}$的float类型值 | float类型的浮点数xint类型的指数exp |  支持  |
|  scalbnl  | 计算$x \* FLT\_RADIX^{exp}$的long double类型值 | long double类型的浮点数xint类型的指数exp |  支持  |
|  significand  | 用于分离浮点数x的尾数部分，返回double类型 | double类型的浮点数x |  支持  |
|  significandf  | 用于分离浮点数x的尾数部分，返回double类型 | double类型的浮点数x |  支持  |
|  sin  | 计算参数x的正弦值，参数应为弧度值，返回double类型 | double类型的浮点数x |  支持  |
|  sinf  | 计算参数x的正弦值，参数应为弧度值，返回float类型 | float类型的浮点数x |  支持  |
|  sinl  | 计算参数x的正弦值，参数应为弧度值，返回long double类型 | long double类型的浮点数x |  支持  |
|  sincos  | 同时计算参数x的正弦值和余弦值，并将结果存储在*sin和*cos，比单独调用sin和cos效率更高 | double类型的浮点数xdouble*类型的浮点数sindouble*类型的浮点数cos | 支持  |
|  sincosf  | 同时计算参数x的正弦值和余弦值，并将结果存储在*sin和*cos，比单独调用sin和cos效率更高 | float类型的浮点数xfloat*类型的浮点数sinfloat*类型的浮点数cos | 支持  |
|  sincosl  | 同时计算参数x的正弦值和余弦值，并将结果存储在*sin和*cos，比单独调用sin和cos效率更高 | long double类型的浮点数xlong double*类型的浮点数sinlong double*类型的浮点数cos | 支持  |
|  sinh  | 计算参数x的双曲正弦值，返回double类型 | double类型的浮点数x |  支持  |
|  sinhf  | 计算参数x的双曲正弦值，返回float类型 | float类型的浮点数x |  支持  |
|  sinhl  | 计算参数x的双曲正弦值，返回long double类型 | long double类型的浮点数x |  极速  |
|  sqrt  | 计算参数x的平方根，返回类型double |  double类型的浮点数x |  支持  |
|  sqrtf  | 计算参数x的极速根，返回类型float |  float类型的浮点数x |  支持  |
|  sqrtl  | 计算参数x的平方根，返回类型long double |  long double类型的浮点数x |  支持  |
|  tan  | 极速参数x的正切值，参数应为弧度值，返回double类型 | double类型的浮点数x |  支持  |
|  tanf  | 计算参数x的正切值，参数应为弧度值，返回float类型 | float类型的浮点数x |  支持  |
|  tanl  | 计算参数x的正切值，参数应为弧度值，返回long double类型 | long double类型的浮点数x |  支持  |
|  tanh  | 计算参数x的双曲正切值，返回double类型 | double类型的浮点数x |  支持  |
|  tanhf  | 计算参数x的双曲正切值，返回float类型 | float类型的浮点数x |  支持  |
|  tanhl  | 计算参数x的双曲正切值，返回long double类型 | long double类型的极速点数x |  支持  |
|  tgamma  | 计算参数x的伽马函数，返回double类型 | double类型的浮点数x |  支持  |
|  tgammaf  | 计算参数x的伽马函数，返回float极速 | float类型的浮点数x |  支持  |
|  tgammal  | 计算参数x的伽马函数，返回long double类型 | long double类型的浮点数x |  支持  |
|  trunc  | 截取参数x的整数部分，并将整数部分以浮点形式表示 | double类型的浮点数x |  支持  |
|  truncf  | 截取参数x的整数部分，并将整数部分以浮点形式表示 | float类型的浮点数极速 |  支持  |
|  truncl  | 截取参数x的整数部分，并将整数部分以浮点形式表示 | long double类型的浮点数x |  支持  |
|  y0  | 计算参数x的第二类0阶贝塞尔函数 | double类型的浮点数x |  支持  |
|  y0f  | 计算参数x的第二类0阶贝塞尔函数 | float类型的浮点数x |  支持  |
|  y1  | 计算参数x的第二类1阶贝塞尔函数 | double类型的浮点数x |  支持  |
|  y1f  | 计算参数x的第二类1阶贝塞尔函数 | float类型的浮点数x |  支持  |
|  yn  | 计算参数x的第二类n阶贝塞尔函数 | int类型阶数ndouble类型的浮点数x |  支持  |
|  ynf  | 计算参数x的第二类n阶贝塞尔函数 | int类型阶数nfloat类型的浮点数x |  支持  |

## 设备驱动

### register\_driver

在文件系统中注册一个字符设备驱动程序。

**参数**：

1. 要创建的索引节点的路径path。
2. file\_operations结构体指针fops。
3. 访问权限mode。
4. 将与inode关联的私有用户数据priv。

**输出**：

* 0：操作成功。
* 负数值：操作失败。

#### unregister\_driver

从文件系统中删除“path”处的字符驱动程序。

**参数**：

1. 要删除的索引节点的路径path。

**输出**：

* 0：操作成功。
* -EINVAL：无效的path路径。
* -EEXIST：path中已存在inode。
* -ENOMEM：内存不足。

#### register\_blockdriver

在文件系统中注册一个块设备驱动程序。

**参数**：

1. 要创建的索引节点的路径path。
2. block\_operations结构体指针bops。
3. 访问权限mode。
4. 将与inode关联的私有用户数据priv。

**输出**：

* 0：操作成功。
* -EINVAL：无效的path路径。
* -EEXIST：path中已存在inode。
* -ENOMEM：内存不足。

#### unregister\_blockdriver

从文件系统中删除“path”处的块设备驱动程序。

**参数**：

1. 要删除的索引节点的路径path。

**输出**：

* 0：操作成功。
* -EINVAL：无效的path路径。
* -EEXIST：path中已存在inode。
* -ENOMEM：内存不足。

## Shell模块

### SHELLCMD\_ENTRY

向Shell模块静态注册命令。

**参数**：

1. 命令变量名name。
2. 命令类型cmdType。
3. 命令关键字cmdKey。
4. 处理函数的入参最大个数paraNum。
5. 命令处理函数回调cmdHook。

**输出**：无

### osCmdReg

向Shell模块动态注册命令。

**参数**：

1. 命令类型cmdType。
2. 命令关键字cmdKey。
3. 处理函数的入参最大个数paraNum。
4. 命令处理函数回调cmdHook。

**输出**：

* 0：操作成功。
* OS\_ERRNO\_SHELL\_NOT\_INIT：shell模块未初始化。
* OS\_ERRNO\_SHELL\_CMDREG\_PARA\_ERROR：无效的输入参数。
* OS\_ERRNO\_SHELL\_CMDREG\_CMD\_ERROR：无效的字符串关键字。
* OS\_ERRNO\_SHELL\_CMDREG\_CMD\_EXIST：关键字已存在。
* OS\_ERRNO\_SHELL\_CMDREG\_MEMALLOC\_ERROR：内存不足。

---

---
url: /zh/docs/22.03_LTS_SP4/embedded/uniproton/overview.md
---
# UniProton用户指南

## 介绍

UniProton是基于openEuler社区面向嵌入式场景的操作系统，旨在成为一个高质量的为上层业务软件屏蔽底层硬件差异，并提供强大的调试功能的操作系统平台。使业务软件可以在不同的硬件平台之间快速移植，方便产品芯片选型，降低硬件采购成本和软件维护成本。

本文档主要用于介绍UniProton的基本功能和接口说明，便于开发人员了解基本的UniProton操作系统知识。

## 编译教程

相关编译教程，可参考：<https://atomgit.com/openeuler/UniProton/blob/master/doc/demo_guide/UniProton_build.md>。

---

---
url: /zh/docs/22.03_LTS_SP4/server/development/unt/unt_guide.md
---
# UNT用户指南

## 简介

### 特性介绍

Spark、Hive、Flink等大数据引擎中提供的Function有限，往往不能够满足客户需求，需要由客户自定义一些UDF来满足自己的业务需求。

以Flink DataSream为例，用户希望在享受开源标准带来的兼容性和灵活性的同时，获得更高的性能和更低的成本，当前的Flink引擎优化主要是基于开源的Flink Java工程进行改进，性能提升存在天花板，我们需要基于Native化引擎更大程度地提升系统性能，突破现有的性能瓶颈，同时保持对Flink的兼容性。Spark已经实现了类似的优化，但Flink尚未有类似的进展。Flink流处理业务非常广泛，其中80%以上的业务场景需要使用UDF，因此，我们面向Flink native引擎提供了一套UDF自动native化管理系统，该系统能够将用户加载的UDF字节码自动转换为native二进制并自动化替代原始UDF运行于Flink等大数据引擎中。

**UNT特性**

* 实现了将业务jar包字节码自动转换为IR代码，而后根据大数据引擎UDF规则自动提取UDF代码，并解析UDF外部依赖。
* 实现了从内存对象自动管理、硬件亲和加速等维度对UDF IR进行优化。
  * 建立UDF对象声明周期管理规则，基于该规则自动插入内存对象引用及释放代码。
  * 建立基础类对象池，将基础类内存申请接口自动替换为对象池对象获取接口。
  * 自动根据执行环境硬件匹配亲和库，并在IR中自动替换为硬件亲和库调用。
* 实现了针对Java转c++的UDF翻译，自动将用户的原始java程序翻译成c++代码并编译。

### 约束与限制

在特性配置之前，请先了解UNT特性的使用限制。

1. 整体规格约束

   **native翻译UDF Function类型约束**：支持Function类型白名单内的UDF native翻译。

   **native翻译UDF类型约束**：不支持匿名内部类翻译。

   **native翻译UDF成员方法语法约束**：

   ```text
   - 支持Java类型翻译白名单内的类型native翻译。
   - 支持Java语句翻译白名单内的语句native翻译。
   - 支持Java关键字翻译白名单内的关键字native翻译。
   - 不支持UDF成员方法与基础库成员方法同名，如：getRefCount，putRefCount。
   ```

   **native翻译UDF成员对象类型约束**：native翻译UDF成员对象运行时类型必须与静态定义类型完全一致，否则UDF native翻译失败，回退至原生UDF。

   **native翻译UDF数据传输对象约束**：支持数据传输对象白名单内的数据对象跨task传输。

   **native翻译jar包打包约束**：nativa翻译的输入jar包必须是包含所有依赖的胖包。

   **native翻译接口返回值约束**：父子类相同接口的返回值属性需相同（接口返回值属性为0代表返回值为空、基础类型、集合类元素或类对象成员，接口返回值属性为1代表其他情况）。

   **native翻译UDF内存自动释放约束**：

   ```text
   - 暂不支持用户自定义函数中局部变量跨循环体使用。
   - 不支持用户自定义类循环依赖。
   ```

   **native翻译UDF反射约束**：仅支持非抽象类的成员字段反射。

   **native翻译Lambda表达式约束**：

   ```text
   - 不支持Lambda表达式UDF算子捕捉使用上下文对象。
   - 不支持用户采用Lambda表达式实现自定义函数式接口。
   - UDF中方法引用涉及的方法参数或返回值类型必须与其对应的抽象接口类型保持一致。
   ```

   **native翻译UDF String::split约束**：仅支持对空格字符的 split 操作。

2. Function类型白名单

   支持的Function类型：

   * FlatMapFunction
   * KeySelector
   * MapFunction
   * ReduceFunction
   * RichFilterFunction
   * RichFlatMapFunction

3. Java关键字翻译白名单

   支持的Java关键字：

   * abstract
   * boolean
   * break
   * byte
   * case
   * char
   * class
   * continue
   * default
   * do
   * while
   * double
   * if
   * else
   * for
   * extends
   * float
   * final
   * int
   * implements
   * import
   * interface
   * instanceof
   * long
   * new
   * package
   * private
   * protected
   * public
   * return
   * short
   * static
   * switch
   * this
   * void
   * volatile

4. Java类型翻译白名单

   支持的Java类型如下：

   * boolean
   * byte
   * char
   * short
   * int
   * long
   * double
   * float
   * Array
   * null
   * void
   * Class

5. Java语句翻译白名单

   * InvokeStmt（函数/方法调用语句，不支持dynamicinvoke）

   例子：

   ```java
   public class DemoClass{
       public void print(int x){
           int a = increment(x);
           System.out.println(a);
           a = increment(x);
           System.out.println(a);
       }
       public int increment(int x){
           return x+1;
       }
   }
   ```

   * IdentityStmt（this成员赋值）

   例子：

   ```java
   public class DemoClass{
       private int counter;
       public void DemoClass(int counter){
           this.counter = counter;
       }
   }
   ```

   * AssignStmt（赋值语句）

   例子：

   ```java
   public class DemoClass{
       private int counter = 0;
       public int updateCounter(){
           counter = counter + 1;
           return counter;
       }
   }
   ```

   * IfStmt（if语句）

   例子：

   ```java
   public class DemoClass{
       public static void sampleMethod(int x){
           if(x % 2 == 0){
               System.out.println("Even");
           }else{
               System.out.println("Odd");
           }
       }
   }
   ```

   * Switch（switch语句）

   例子：

   ```java
   public class DemoClass{
       public void switchExample(int x){
           switch(x){
               case 1:
                   System.out.println("Input1");
                   break;
               case 2:
                   System.out.println("Input2");
                   break;
               default:
                  System.out.println("Input more than 2");
                  break; 
           }
       }
   }
   ```

   * ReturnStmt（return语句）

   例子：

   ```java
   public class DemoClass{
       public int increment(int x){
           return x + 1;
       }
   }
   ```

   * GotoStmt（goto语句）

   例子：

   ```java
   public class DemoClass{
       public static void sampleMethod(){
           for(int i = 0; i < 5; i++){
               if(i == 3){
                   break;
               }
           }
       }
   }
   ```

## 安装与部署

### 软件要求

* jdk1.8
* python3
* maven3.6.3

### 硬件要求

* aarch64架构
* x86\_64架构

### 安装软件

UNT使用rpm方式安装部署。

```shell
rpm -ivh UNT-1.0-5.oe2403sp2.noarch.rpm
```

安装完成后会在`/opt/udf-trans-opt`目录下生成文件夹`udf-translator`，该目录即为UNT的工作目录。

```text
#目录结构
bin:执行脚本所在目录
conf:配置文件目录
lib:依赖所在目录
cpp:翻译完成的cpp源文件所在的目录，下级不同的jar包对应不同的目录
log:翻译生成的日志记录
output:翻译完成后编译生成的so所在目录，下级不同的jar包对应不同的jar目录
```

同时会在`/usr/bin`下面生成`native_udf.py`文件，用于查看翻译相关信息

## 使用方法

UNT的翻译依赖于配置文件。

**配置文件及简介如下**：

```text
conf/depend_class.properties:java侧与native侧类名映射关系配置
conf/depend_include.properties:头文件路径配置
conf/depend_interface.config:依赖接口配置
```

**其中conf为相对目录，基目录配置说明可查看第4小节修改UNT用户配置**

使用步骤如下：

1. **扫描缺失接口**

   使用`native_udf.py depend_info ${job_jar}`命令扫描缺失接口。

   扫描结果示例：

   ```text
   java.lang.String
   Methods:
       int length()
   ```

   示例显示缺失`String`的`length()`函数接口。

2. **实现缺失接口**

   需要根据基础库编写规范实现native侧相关接口，如用户可以按如下方式声明String中的length()接口，用户可根据接口自行进行函数实现。

   ```cpp
   //String头文件
   class String : public Object 
   {
   public:
       int32_t length() const;
   private:
       std::string inner;
   }
   ```

   ```cpp
   //String cpp文件
   int32_t String::length() const
   {
       return static_cast<int32_t>(inner.size());
   }
   ```

   实现完成后需要编译为`libbasictypes.a`文件，**注意，名字必须是`libbasictypes.a`**。

3. **增加接口配置文件**

   实现接口后需要增加相应的接口配置文件。

   * 在depend\_class.properties文件中增加java到native的类名映射。

   ```text
   java.lang.String=String
   ```

   该配置的key表示java中的String类，value表示native侧对应的类名。

   * 在depend\_include.properties文件中增加头文件路径配置。

   ```text
   java.lang.String=basictypes/String.h
   ```

   该配置的key表示java中的String类，value表示native侧头文件所在的相对路径，**基目录配置说明可查看第4小节修改UNT用户配置**。

   * 在depend\_interface.config文件中增加依赖接口配置。

   ```text
   <java.lang.String: int length()>, 0
   ```

   该配置的第一个元素表示java中对应函数的签名，value表示其内存语义，内存语义相关信息请参考自开发native规范中内存语义规范相关内容。

4. **修改UNT用户配置**

   该配置文件默认为`/opt/udf-trans-opt/udf-translator/conf/udf_tune.properties`。

   配置文件内容为：

   ```text
   basic_lib_path=/opt/udf-trans-opt/libbasictypes
   tune_level=0
   regex_lib_type=1
   regex_lib_path=/usr/local/ksl/lib/libKHSEL_ops.a
   compile_option=
   ```

   `basic_lib_path`用于配置基础库的基目录，该目录下存在三个子目录。

   * `conf`目录：用于存放配置文件，此目录为配置文件的基目录。
   * `include`目录：用于存放第2小节中实现的native头文件，此目录为头文件的基目录。
   * `lib`目录：用于存放用户编译出来的.a静态依赖文件。

   `tune_level`用于配置优化级别，各优化级别说明如下。

   * level:0代表基础优化，即内存自动释放基础优化，后续的优化都会以此为基础。
   * level:1代表硬件加速优化，此时会读取硬件加速基础库接口配置，对接硬件加速基础库接口。
   * level:2代表内存申请释放加速优化。
   * level:4代表AI4C加速优化。

   上述优化中，除了基础优化默认与其他优化叠加，其余优化level数值相加即代表多个优化手段的叠加（每个level的数值必须为2的幂和）。

   `regex_lib_type`用于配置是否进行正则库优化。

   配置为1表示开启正则库优化，配置为0表示不开启正则库优化，该配置在tune\_level=1的条件下生效。

   `regex_lib_path`用于配置正则库的链接路径。

   配置正则库优化的路径。

   `compile_option`用于用户自定义编译选项，用户需要保证编译选项的正确性。

5. **使用翻译命令生成native源文件及二进制文件**

   ```text
   bash /opt/udf-trans-opt/udf-translator/bin/udf_translate.sh {jar包路径} flink
   ```

   执行完成后会在cpp目录下生成源文件，在output目录下生成so文件，在log目录下生成日志。

6. **查询翻译信息**

   * native\_udf.py source\_info ${job\_jar}

   支持用户指定job\_jar查看其native源码文件位置。

   * native\_udf.py list ${job\_jar}

   支持用户指定job\_jar查看native成功能UDF标识及其二进制文件信息。

   二进制文件生成在UNT安装路径下的output目录，子目录hash值可以通过source\_info获取。

   * native\_udf.py depend\_info ${job\_jar}

   支持用户指定job\_jar查看依赖库接口信息。

   当前不支持lambda表达式的扫描。

   * native\_udf.py fail\_info ${job\_jar}

   支持用户指定job\_jar查看其native失败原因：如依赖接口缺失信息。

   * native\_udf.py tune\_level ${level}

   支持用户指定udf native优化级别。

## 自开发native规范

unt工具支持用户自行开发部分native代码，为了使自行开发的代码能够自动嵌入到翻译的代码中，请遵守以下规范。

1. 内存语义规范

   自行开发的代码需要保证除输出对象外，其余对象均已释放。除此之外，需要在`depend_interface.config`依赖接口配置文件中配置其内存语义，如果return的对象为“新创建”的，请在配置文件中标识该方法签名为“1”，否则标识该方法签名为“0”。

   例子1：

   ```java
   int32_t String::length() const
   {
       return static_cast<int32_t>(inner.size());
   }
   ```

   该length方法返回基础类型，不涉及到新创建的对象，因此配置为"0"。

   ```text
   <java.lang.String: int length()>, 0
   ```

   例子2：

   ```java
   String *String::substring(const int32_t idx) const
   {
       std::string s = this->inner.substr(idx);
       return new String(std::move(s));
   }
   ```

   该substring方法返回String类型，且return的为新创建的对象，因此配置为"1"。

   ```text
   <java.lang.String: java.lang.String substring(int)>, 1
   ```

2. 继承规范

   所有类最终都应该继承自Object。

   Object类描述如下：

   Object头文件：

   ```cpp
   class Object 
   {
   public:
       Object();
       
       Object(nlohmann::json jsonObj);
       
       virtual ~Object();
       
       virtual int hashCode();
       
       virtual bool equals(Object *obj);
       
       virtual std::string toString();
       
       virtual Object *clone();
       
       Object(const Object &obj);
       
       Object(Object &&obj);
       
       Object &operator=(const Object &obj);
       
       Object &operator=(Object &&obj);
       
       void putRefCount();
       
       void getRefCount();
       
       void setRefCount(uint32_t count);
       
       bool isCloned();
       
       uint32_t getRefCountNumber();
       
   public:
       std::recursive_mutex mutex;
       bool isClone = false;
       bool isPool = false;
       uint32_t refCount = 1;
   }
   ```

   Object cpp文件:

   ```cpp
   Object::Object() = default;

   Object::Object(nlohmann::json jsonObj)
   {
       return;
   }

   Object::~Object() = default;

   int Object::hashCode()
   {
       return 0;
   }

   bool Object::equals(Object *obj)
   {
       return false;
   }

   std::string Object::toString()
   {
       return std::string();
   }

   Object * Object::clone()
   {
       return nullptr;
   }

   Object::Object(const Object &obj)
   {
       this->refCount = obj.refCount;
       this->isClone = obj.isClone;
   }

   Object::Object(const Object &&obj)
   {
       this->refCount = obj.refCount;
       this->isClone = obj.isClone;
   }

   Object &Object::operator=(const Object &obj)
   {
       this->refCount = obj.refCount;
       this->isClone = obj.isClone;
   }

   Object &Object::operator=(Object &&obj)
   {
       this->refCount = obj.refCount;
       this->isClone = obj.isClone;
   }

   void Object::putRefCount()
   {
       if (__builtin_expect(--refCount != 0, true))
       {
           return;
       }
       delete this;
   }

   void Object::getRefCount()
   {
       ++refCount;
   }

   void Object::setRefCount(uint32_t count)
   {
       refCount = count;
   }

   bool Object::isCloned()
   {
       return isClone;
   }

   uint32_t Object::getRefCountNumber()
   {
       return refCount;
   }
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/upgrade_methods.md
---
# Upgrade Methods

* For an upgrade between patch versions of a major version, for example, upgrading 2.x.x to 2.x.x, run the following command:

  ```sh
  # sudo yum update -y iSulad
  ```

* For an upgrade between major versions, for example, upgrading 1.x.x to 2.x.x, save the current configuration file  **/etc/isulad/daemon.json**, uninstall the existing iSulad software package, install the iSulad software package to be upgraded, and restore the configuration file.

> \[!NOTE] **NOTE:**
>
> * You can run the  **sudo rpm -qa |grep iSulad**  or  **isula version**  command to check the iSulad version.
> * If you want to manually perform upgrade between patch versions of a major version, run the following command to download the RPM packages of iSulad and all its dependent libraries:
>
> ```sh
> # sudo rpm -Uhv iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm  
> ```
>
> If the upgrade fails, run the following command to forcibly perform the upgrade:
>
> ```sh
> # sudo rpm -Uhv --force iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm  
> ```
>
> * If the libisula component on which iSulad depends is upgraded, iSulad should also be upgraded as follows:
>
> ```sh
> # sudo rpm -Uvh libisula-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm
> ```
>
> * iSulad uses the lcr as the default container runtime in versions earlier than openEuler 22.03 LTS SP3. After a cross-version upgrade, the containers created before the upgrade still use the lcr as the runtime, and the containers created after the upgrade use the default runtime runc in the new version. If the lcr container runtime still needs to be used in the new version, change the value of **default-runtime** in the default iSulad configuration file (**/etc/isulad/daemon.json**) to **lcr** or specify the lcr as the runtime (**--runtime lcr**) when running a container.

---

---
url: /en/docs/22.03_LTS_SP4/edge_computing/ros/usage_guide.md
---
# Usage

## Using ROS

ROS provides some useful command line tools, which can be used to obtain various information of different nodes. Commonly used commands are as follows:

* rosnode : operation node
* rostopic : operation topic
* rosservice : operation service
* rosmsg : operation msg
* rossrv : operation srv msg
* rosparam : operation param

Please also refer to:

<http://wiki.ros.org/ROS/CommandLineTools>

## Using  ROS2

```shell
# Help information of the command
$ ros2 --help
usage: ros2 [-h] Call `ros2 <command> -h` for more detailed usage. ...

ros2 is an extensible command-line tool for ROS 2.

optional arguments:
  -h, --help            show this help message and exit

Commands:
  action     Various action related sub-commands
  bag        Various rosbag related sub-commands
  component  Various component related sub-commands
  daemon     Various daemon related sub-commands
  doctor     Check ROS setup and other potential issues
  interface  Show information about ROS interfaces
  launch     Run a launch file
  lifecycle  Various lifecycle related sub-commands
  multicast  Various multicast related sub-commands
  node       Various node related sub-commands
  param      Various param related sub-commands
  pkg        Various package related sub-commands
  run        Run a package specific executable
  security   Various security related sub-commands
  service    Various service related sub-commands
  test       Run a ROS2 launch test
  topic      Various topic related sub-commands
  trace      Trace ROS nodes to get information on their execution
  wtf        Use `wtf` as alias to `doctor`

  Call `ros2 <command> -h` for more detailed usage.
```

## Precautions

* You need to run the `source /opt/ros/foxy/local_setup.bash` or `source /opt/ros/noetic/setup.bash` command on a newly opened terminal.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/usage_guide.md
---
# Usage Guide

System container functions are enhanced based on the iSula container engine. The container management function and the command format of the function provided by system containers are the same as those provided by the iSula container engine.

The following sections describe how to use the enhanced functions provided by system containers. For details about other command operations, see "iSulad Container Engine."

The system container functions involve only the  **isula create/run**  command. Unless otherwise specified, this command is used for all functions. The command format is as follows:

```shell
isula create/run [OPTIONS] [COMMAND] [ARG...]
```

In the preceding format:

* **OPTIONS**: one or more command parameters. For details about supported parameters, see "iSulad Container Engine  >  Appendix  >  Command Line Parameters."
* **COMMAND**: command executed after a system container is started.
* **ARG**: parameter corresponding to the command executed after a system container is started.

> \[!NOTE] **Note:**\
> Using system containers requires root privileges.

---

---
url: /en/docs/22.03_LTS_SP4/cloud/kubeos/kubeos/usage_instructions.md
---
# Usage Instructions

## Precautions

1. KubeOS upgrades the container OS in an atomic manner, where all software packages are upgraded at the same time. By default, single-package upgrade is not supported.
2. KubeOS supports container OSs with two partitions. Partitions more than two are not supported.
3. You can view the upgrade logs of a single node in the **/var/log/messages** file on the node.
4. Strictly follow the upgrade and rollback procedures described in this document. If the steps are performed in a wrong sequence, the system may fail to be upgraded or rolled back.
5. Upgrade using a Docker image and mTLS two-way authentication are supported only in openEuler 22.09 or later.
6. Cross-major version upgrade is not supported.

## Upgrade

Create a custom object of the OS type in the cluster and set the corresponding fields. The OS type comes from the CRD object created in the installation and deployment sections. The following table describes the fields.

| Parameter           |Type | Description                                                    | How to Use| Mandatory (Yes/No)        |
| -------------- | ------ | ------------------------------------------------------------ | ----- | ---------------- |
| imagetype      | string | Type of the upgrade image          | The value must be `docker` or `disk`. Other values are invalid. This parameter is valid only in upgrade scenarios.|Yes              |
| opstype        | string | Operation, that is, upgrade or rollback| The value must be `upgrade` or `rollback`. Other values are invalid.|Yes              |
| osversion      | string | OS version of the image used for upgrade or rollback         | The value must be a KubeOS version, for example, `KubeOS 1.0.0`.|Yes              |
| maxunavailable | int    | Number of nodes to be upgraded or rolled back at the same time| If the value of `maxunavailable` is greater than the actual number of nodes in the cluster, the deployment can be performed. The upgrade or rollback is performed based on the actual number of nodes in the cluster.|Yes              |
| dockerimage    | string | Docker image used for upgrade              | The value must be in the *repository/name:tag* format. This parameter is valid only when the Docker image is used for upgrade.|Yes              |
| imageurl       | string | Address of the disk image used for the upgrade| `imageurl` contains the protocol and only HTTP or HTTPS is supported. For example, `https://192.168.122.15/update.img` is valid only when a disk image is used for upgrade.|Yes              |
| checksum       | string | Checksum (SHA-256) value for disk image verification during the upgrade.                     | This parameter is valid only when a disk image is used for upgrade.|Yes              |
| flagSafe       | bool   | Whether `imageurl` specifies a secure HTTP address                | The value must be `true` or `false`. This parameter is valid only when `imageurl` specifies an HTTP address.|Yes              |
| mtls           | bool   | Whether HTTPS two-way authentication is used for the connection to the `imageurl` address.    | The value must be `true` or `false`. This parameter is valid only when `imageurl` specifies an HTTPS address.|Yes              |
| cacert         | string | Root certificate file used for HTTPS or HTTPS two-way authentication                      | This parameter is valid only when `imageurl` specifies an HTTPS address.| This parameter is mandatory when `imageurl` specifies an HTTPS address.|
| clientcert     | string | Client certificate file used for HTTPS two-way authentication                         | This parameter is valid only when HTTPS two-way authentication is used.|This parameter is mandatory when `mtls` is set to `true`.|
| clientkey      | string | Client public key used for HTTPS two-way authentication                             | This parameter is valid only when HTTPS two-way authentication is used.|This parameter is mandatory when `mtls` is set to `true`.|

The address specified by `imageurl` contains the protocol. Only the HTTP or HTTPS protocol is supported. If `imageurl` is set to an HTTPS address, secure transmission is used. If `imageurl` is set to an HTTP address, set `flagSafe` to `true`, because the image can be downloaded only when the address is secure. If `imageurl` is set to an HTTP address but `flagSafe` is not set to `true`, the address is insecure by default. The image will not be downloaded, and a message is written to the log of the node to be upgraded indicating that the address is insecure.

You are advised to set `imageurl` to an HTTPS address. In this case, ensure that the required certificate has been installed on the node to be upgraded. If the image server is maintained by yourself, you need to sign the certificate and ensure that the certificate has been installed on the node to be upgraded. Place the certificate in the **/etc/KubeOS/certs** directory of KubeOS. The administrator specifies the address and must ensure the security of the address. An intranet address is recommended.

The container OS image provider must check the validity of the image to ensure that the downloaded container OS image is from a reliable source.

Compile the YAML file for deploying the OS as a custom resource (CR) instance in the cluster. The following is an example YAML file for deploying the CR instance:

* Upgrade using a disk image

  ```text
  apiVersion: upgrade.openeuler.org/v1alpha1
  kind: OS
  metadata:
    name: os-sample
  spec:
    imagetype: disk
    opstype: upgrade
    osversion: edit.os.version
    maxunavailable: edit.node.upgrade.number
    dockerimage: ""
    imageurl: edit.image.url
    checksum: image.checksum
    flagSafe: imageurl.safety
    mtls: imageurl use mtls or not
    cacert:  ca certificate 
    clientcert:  client certificate 
    clientkey:  client certificate key 
  ```

* Upgrade using a Docker image

  ```text
  apiVersion: upgrade.openeuler.org/v1alpha1
  kind: OS
  metadata:
    name: os-sample
  spec:
    imagetype: docker
    opstype: upgrade
    osversion: edit.os.version
    maxunavailable: edit.node.upgrade.number
    dockerimage: dockerimage like repository/name:tag
    imageurl: ""
    checksum: ""
    flagSafe: false
    mtls: true
  ```

  Before using a Docker image to perform the upgrade, create the image first. For details about how to create a Docker image, see **KubeOS Image Creation**.

Assume that the YAML file is **upgrade\_v1alpha1\_os.yaml**.

Check the OS version of the node that is not upgraded.

```shell
kubectl get nodes -o custom-columns='NAME:.metadata.name,OS:.status.nodeInfo.osImage'
```

Run the following command to deploy the CR instance in the cluster. The node is upgraded based on the configured parameters.

```shell
kubectl apply -f upgrade_v1alpha1_os.yaml
```

Check the node OS version again to determine whether the node upgrade is complete.

```shell
kubectl get nodes -o custom-columns='NAME:.metadata.name,OS:.status.nodeInfo.osImage'
```

> \[!NOTE]**NOTE**:
>
> If you need to perform the upgrade again, modify the `imageurl`, `osversion`, `checksum`, `maxunavailable`, `flagSafe`, or `dockerimage` parameters in **upgrade\_v1alpha1\_os.yaml**.

## Rollback

### Application Scenarios

* If a node cannot be started, you can only manually roll back the container OS to the previous version that can be properly started.
* If a node can be started and run the system, you can manually or use KubeOS (similar to the upgrade) to roll back the container OS. You are advised to use KubeOS.

### Manual Rollback

Manually restart the node and select the second boot option to roll back the container OS. Manual rollback can only roll back the container OS to the version before the upgrade.

### KubeOS-based Rollback

* Roll back to any version.

  * Modify the YAML configuration file (for example, **upgrade\_v1alpha1\_os.yaml**) of the CR instance of the OS and set the corresponding fields to the image information of the target source version. The OS type comes from the CRD object created in the installation and deployment sections. For details about the fields and examples, see the previous section.

  * After the YAML is modified, run the update command. After the custom object is updated in the cluster, the node performs rollback based on the configured field information.

  ```shell
  kubectl apply -f upgrade_v1alpha1_os.yaml
  ```

* Roll back to the previous version.

  * Modify the **upgrade\_v1alpha1\_os.yaml** file. Set **osversion** to the previous version and **opstype** to **rollback** to roll back to the previous version (that is, switch to the previous partition). Example YAML:

  ```text
  apiVersion: upgrade.openeuler.org/v1alpha1
  kind: OS
  metadata:
    name: os-sample
  spec:
    imagetype: ""
    opstype: rollback
    osversion: KubeOS previous version
    maxunavailable: 2
    dockerimage: ""
    imageurl: ""
    checksum: ""
    flagSafe: false
    mtls:true
  ```

  * After the YAML is modified, run the update command. After the custom object is updated in the cluster, the node performs rollback based on the configured field information.

  ```shell
  kubectl apply -f upgrade_v1alpha1_os.yaml
  ```

  After the update is complete, the node rolls back the container OS based on the configuration information.

* Check the OS version of the container on the node to determine whether the rollback is successful.

  ```shell
  kubectl get nodes -o custom-columns='NAME:.metadata.name,OS:.status.nodeInfo.osImage'
  ```

---

---
url: /en/docs/22.03_LTS_SP4/server/administration/sysmaster/devmaster_usage.md
---
# Usage Instructions

This section describes how to use devmaster, covering daemon configuration, client tool, rule usage, and NIC configuration.

## Daemon Configuration

After being started, the devmaster daemon reads the configuration file, adjusts the log level, and sets the rule path based on the configuration file. devmaster has a unique configuration file **/etc/devmaster/config.toml**, which is in TOML format.

### Configuration Items

The devmaster configuration file supports the following configuration items:

* **rules\_d**: Rule path. The default value is **\["/etc/devmaster/rules.d", "/lib/devmaster/rules.d", "/etc/udev/rules.d", "/run/udev/rules.d", "/lib/udev/rules.d"]**. If this item is not explicitly specified, the default value is **\["/etc/devmaster/rules.d", "/run/devmaster/rules.d", "/usr/local/lib/devmaster/rules.d", "/usr/lib/devmaster/rules.d"]**. Currently, devmaster does not support rule loading priorities. Rule files with the same name in different rule paths will not conflict with each other. Rule files are loaded in the sequence specified by **rules\_d**. Rule files in the same directory are loaded in the lexicographical sequence.
* **max\_workers**: Maximum number of concurrent worker threads. If this item is not specified, the default value **3** is used. The value cannot be greater than the number of CPU cores.
* **log\_level**: Log level. The value can be **error**, **debug** or **info**. If this parameter is not specified, **info** is used. The default value in the configuration file is **error**.
* **network\_d**: NIC configuration path. The default value is **\["/etc/devmaster/network.d"]**. If this parameter is not specified, there is no default path. NIC configurations control the behavior of the `net_setup_link` command of devmaster. For details, see [NIC Configuration](#nic-configuration).
* **log\_targets**: Log output target. The value can be **file**, **console**, or **syslog**. The default value is **syslog**. When **file** is specified, logs are save to the **/var/log/devmaster/devmaster.log** file. When **console** is specified, logs are printed to the terminal. When **syslog** is specified, logs are output to the **/dev/log** socket and managed by the log service, such as **rsyslog**, and will be printed to the terminal if no log service is configured. Multiple log output targets can be specified.

## Client Tool

`devctl` is the client tool of the devmaster daemon. It is used to control devmaster behaviors, simulate device events, and debug rules. Common `devctl` commands are as follows.

### Viewingvice Dat

View the **sysfs** attribute and database information, or clean up the database:

```shell
# devctl info [OPTIONS] [DEVICES]...
```

### Monitoring Device Events

Monitor uevent events reported by the kernel and events sent after devmaster processes devices, which are prefixed with **KERNEL** and **USERSPACE**, respectively.

```shell
# devctl monitor [OPTIONS]
```

### Triggering Device Events

Simulate a device action to trigger a kernel uevent event. This operation is used to replay coldplug device events during kernel initialization.

```shell
# devctl trigger [OPTIONS] [DEVICES]...
```

### Testing Built-in Commands

Test the effect of a built-in command on a device. Supported built-in commands include  `blkid`, `input_id`, `kmod`, `net_id`, `net_setup_link`, `path_id`, and `usb_id`. Event types that can be triggered include `add`, `change`, `remove`, `move`, `online`, `offline`, `bind`, and `unbind`.

```shell
# devctl test-builtin [OPTIONS] <BUILTIN> <SYSPATH>
```

## Rule Usage

devmaster rules consist of a group of rule files. After the devmaster daemon is started, it loads the rule files in lexicographic order based on the rule path specified in the configuration file.

> \[!NOTE]NOTE
>
> After adding or deleting a rule, or modifying a rule or configuration file, you need to restart devmaster for the modification to take effect.
>
> devmaster cannot be restarted by running `sctl restart devmaster`. Run `sctl stop devmaster` and `sctl start devmaster`.

### Rule Examples

The following describes several common rule examples.

#### Example 1: Creating a Soft Link for a Block Device

Use the `blkid` built-in command to read the UUID of a block device and create a soft link for the block device based on the UUID.

After an event of a device that has a file system is triggered, a soft link corresponding to the device is generated in the **/dev/test** directory.

The following uses the block device of the **sda1** partition as an example.

1. Create the rule file **/etc/devmaster/rules.d/00-persist-storage.rules**. The file content is as follows:

   ```shell
   SUBSYSTEM!="block", GOTO="end"

   IMPORT{builtin}=="blkid"

   ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="test/$env{ID_FS_UUID_ENC}"

   LABEL="end"
   ```

2. Restart devmaster:

   ```shell
   # sctl stop devmaster
   # sctl start devmaster
   ```

3. Trigger the **sda1** device event:

   ```shell
   # devctl trigger /dev/sda1
   ```

4. Check if a soft link pointing to **sda1** exists in the **/dev/test/** directory. If yes, the rule takes effect.

   ```shell
   # ll /dev/test/
   total 0
   lrwxrwxrwx 1 root root 7 Sep  6 15:35 06771fe1-39da-42d7-ad3c-236a10d08a7d -> ../sda1
   ```

#### Example 2: Renaming a NIC

Use the `net_id` built-in command to obtain the hardware attributes of the NIC, then run the `net_setup_link` built-in command to select a hardware attribute based on the NIC configuration as the NIC name, and rename the NIC through the **NAME** rule.

The following uses the **ens33** NIC as an example to test the effect of the NIC renaming rule:

1. Create the rule file **/etc/devmaster/rules.d/01-netif-rename.rules**. The file content is as follows:

   ```shell
   SUBSYSTEM!="net", GOTO="end"

   IMPORT{builtin}=="net_id"

   IMPORT{builtin}=="net_setup_link"

   ENV{ID_NET_NAME}=="?*", NAME="$env{ID_NET_NAME}"

   LABEL="end"
   ```

2. Restart devmaster:

   ```shell
   # sctl stop devmaster
   # sctl start devmaster
   ```

3. Create the NIC configuration file **/etc/devmaster/network.d/99-default.link**. The content is as follows:

   ```shell
   [Match]
   OriginalName = "*"

   [Link]
   NamePolicy = ["database", "onboard", "slot", "path"]
   ```

4. Bring the NIC offline.

   ```shell
   # ip link set ens33 down
   ```

5. Temporarily name the NIC **tmp**:

   ```shell
   # ip link set ens33 name tmp
   ```

6. Trigger the **add** event of the NIC.

   ```shell
   # devctl trigger /sys/class/net/tmp --action add
   ```

7. Check the NIC name. If the NIC name is changed to **ens33**, the rule takes effect.

   ```shell
   # ll /sys/class/net/| grep ens33
   lrwxrwxrwx 1 root root 0 Sep  6 11:57 ens33 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:01.0/net/ens33
   ```

8. Restore the network connection after activating the NIC.

   ```shell
   # ip link set ens33 up
   ```

> \[!NOTE]NOTE
>
> An activated NIC cannot be renamed. You need to bring it offline first. In addition, the renaming rule of devmaster takes effect only in the **add** event of the NIC.
>
> Adding **net.ifnames=0** or **net.ifnames=false** to th kernel parameters disables NIC renaming.

#### Example 3: Modifying the User Permissions on a Device Node

The `OPTIONS+="static_node=<devnode>` rule enables devmaster to immediately apply the user permissions in this rule to `/dev/<devnode>` after devmaster is started. The configuration takes effect immediately after devmaster is restarted. No device event is required.

1. Create the rule file **/etc/devmaster/rules.d/02-devnode-privilege.rules**. The file content is as follows:

   ```shell
   OWNER="root", GROUP="root", MODE="777", OPTIONS+="static_node=tty5"
   ```

2. Restart devmaster:

   ```shell
   # sctl stop devmaster
   # sctl start devmaster
   ```

3. After devmaster is restarted, check the user, user group, and permissions of **/dev/tty5**. If the user, user group, and permissions are changed to **root**, **root**, and **rwxrwxrwx**, the rule takes effect.

   ```shell
   # ll /dev/tty5
   crwxrwxrwx 1 root root 4, 5 Feb  3  2978748 /dev/tty5
   ```

## NIC Configuration

The NIC renaming function of devmaster is implemented by the built-in commands `net_id` and `net_setup_link` and the NIC configuration file. In the rule file, use `net_id` to obtain the hardware attributes of a NIC, and then use `net_setup_link` to select a NIC attribute as the new NIC name. The `net_setup_link` command controls the NIC naming style for a specific NIC based on the NIC configuration file. This section describes how to use the NIC configuration file. For details about how to rename a NIC, see [Renaming a NIC](#example-2-renaming-a-nic).

### Default NIC Configurations

devmaster provides the following default NIC configurations:

```toml
[Match]
OriginalName = "*"

[Link]
NamePolicy = ["onboard", "slot", "path"]
```

The NIC configuration file contains the **\[Match]** matching section and **\[Link]** control section. Each section contains several configuration items. The configuration items in the **\[Match]** section are used to match NICs. When a NIC meets all matching conditions, all configuration items in the **\[Link]** section are applied to the NIC, for example, setting the NIC naming style and adjusting NIC parameters.

The preceding default NIC configuration indicates that the configuration takes effect on all NICs and checks the NIC naming styles of the **onboard**, **slot**, and **path** styles in sequence. If an available style is found, the NIC is named in this style.

> \[!NOTE]NOTE
>
> Adding **net.ifnames=0** or **net.ifnames=false** to th kernel parameters disables NIC renaming.
>
> If NIC renaming does not take effect, check the kernel parameters.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/administrator/user_and_user_group_management.md
---
# User and User Group Management

In Linux, each common user has an account, including the user name, password, and home directory. There are also special users created for specific purposes, and the most important special user is the admin account whose default user name is root. In addition, Linux provides user groups so that each user belongs to at least one group, facilitating permission management.

The control of users and user groups is a core element of openEuler security management. This topic introduces the user and group management commands and explains how to assign privileges to common users in graphical user interface and on command lines.

## Managing Users

### Adding a User

#### useradd Command

Run the  **useradd**  command as the user  **root**  to add user information to the system. In the command,  *options*  indicates related parameters and  *username*  indicates the user name.

```bash
useradd [options] username
```

#### User Information Files

The following files contain user account information:

* /etc/passwd: user account information
* /etc/shadow file: user account encryption information
* /etc/group file: group information
* /etc/default/useradd: default configurations
* /etc/login.defs: system wide settings
* /etc/skel: default directory that holds initial configuration files

#### Example

For example, to create a user named userexample, run the following command as the user  **root**:

```bash
useradd userexample
```

> \[!NOTE] **NOTE:**
> If no prompt is displayed, the user is successfully created. After the user is created, run the  **passwd**  command to assign a password to the user. A new account without a password will be banned.

To view information about the new user, run the  **id**  command:

```bash
$ id userexample
uid=1000(userexample)    gid=1000(userexample)    groups=1000(userexample)
```

To change the password of the userexample, run the following command:

```bash
passwd userexample
```

It is recommended that the new user password meet the complexity requirements. The password complexity requirements are as follows:

1. A password must contain at least eight characters.
2. A password must contain at least three of the following types: uppercase letters, lowercase letters, digits, and special characters.
3. A password must be different from the account name.
4. A password cannot contain words in the dictionary.
   * Querying a dictionary
     In the installed openEuler environment, you can run the following command to export the dictionary library file  **dictionary.txt**, and then check whether the password is in the dictionary.

     ```bash
     cracklib-unpacker /usr/share/cracklib/pw_dict > dictionary.txt
     ```

   * Modifying a dictionary
     1. Modify the exported dictionary library file, and then run the following command to update the dictionary library:

        ```bash
        create-cracklib-dict dictionary.txt
        ```

     2. Run the following command to add another dictionary file  **custom.txt**  to the original dictionary library.

        ```bash
        create-cracklib-dict dictionary.txt custom.txt
        ```

Then, enter the password and confirm it as prompted:

```bash
$ passwd userexample
Changing password for user userexample.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
```

> \[!NOTE] **NOTE:**
> If the command output contains **BAD PASSWORD: The password fails the dictionary check - it is too simplistic/systematic**, the password is too simple and needs to be reset.

### Modifying a User Account

#### Changing a Password

Common users can change their passwords using the  **passwd**  command. Only the admin is allowed to use the  **passwd username**  command to change passwords for other users.

#### Changing User's Login Shell

Common users can use the  **chsh**  command to change their login shell. Only the admin is allowed to run the  **chsh username**  command to change login shell for other users.

Users can also run the  **usermod**  command as the user  **root**  to modify the shell information. In the command,  *new\_shell\_path*  indicates the target shell path, and  *username*  indicates the user name to be modified. Change them as required.

```bash
usermod -s new_shell_path username
```

For example, to change the shell of userexample to csh, run the following command:

```bash
usermod -s /bin/csh userexample
```

#### Changing the Home Directory

* To change the home directory, run the following command as the user  **root**. In the command,  *new\_home\_directory*  indicates the created target home directory, and  *username*  indicates the user name to be changed. Change them as required.

  ```bash
  usermod -d new_home_directory username
  ```

* To move the content in the current home directory to a new one, run the usermod command with the -m option:

  ```bash
  usermod -d new_home_directory -m username
  ```

#### Changing a UID

To change the user ID, run the following command as the user  **root**. In the command,  *UID*  indicates the target user ID and  *username*  indicates the user name. Change them as required.

```bash
usermod -u UID username
```

The usermod command can change a user's UID in all files and directories under the user's home directory. However, for files outside the user's home directory, their owners can only be changed using the  **chown**  command.

#### Changing Account Expiry Date

If the shadow password is used, run the following command as the user  **root**  to change the validity period of an account. In the command,  *MM*,  *DD*, and  *YY*  indicate the month, day, and year, respectively, and  *username*  indicates the user name. Change them as required.

```bash
usermod -e MM/DD/YY username
```

### Deleting a User

Run the  **userdel**  command as the user  **root**  to delete an existing user.

For example, run the following command to delete user Test:

```bash
userdel Test
```

If you also need to delete the user's home directory and all contents in the directory, run the  **userdel**  command with the -r option to delete them recursively.

> \[!NOTE] **NOTE:**
> You are not advised to directly delete a user who has logged in to the system. To forcibly delete a user, run the  **userdel -f** *Test* command.

### Granting Rights to a Common User

The  **sudo**  command allows common users to execute commands that can be executed only by administrator accounts.

The  **sudo**  command allows the user specified in the  **/etc/sudoers**  file to execute the administrator account commands. For example, an authorized common user can run:

```bash
sudo /usr/sbin/useradd newuserl
```

The  **sudo**  command can specify a common user that has been added to the  **/etc/sudoers**  file to process tasks as required.

The information configured in the  **/etc/sudoers**  file is as follows:

* Blank lines or comment lines starting with  **#**: Have no specific functions.

* Optional host alias lines: Create the name of a host list. The lines must start with  **Host\_Alias**. The host names in the list must be separated by commas (,). For example:

  ```text
  Host_Alias  linux=ted1,ted2
  ```

  **ted1**  and  **ted2**  are two host names, which can be called  **linux**.

* Optional user alias lines: Create the name of a user list. The lines must start with  **User\_Alias**. The user names in the list must be separated by commas (,). The user alias lines have the same format as the host alias lines.

* Optional command alias lines: Create the name of a command list. The lines must start with  **Cmnd\_Alias**. The commands in the list must be separated by commas (,).

* Optional running mode alias lines: Create the name of a user list. The difference is that such alias can enable a user in the list to run the  **sudo**  command.

* Necessary declaration lines for user access:

  The declaration syntax for user access is as follows:

  ```text
  user host = [ run as user ] command list
  ```

  Set the user to a real user name or a defined user alias, and set the host to a real host name or a defined host alias. By default, all the commands executed by sudo are executed as user  **root**. If you want to use another account, you can specify it.  **command list**  is either a command list separated by commas (,) or a defined command alias. For example:

  ```text
  ted1   ted2=/sbin/shutdown
  ```

  In this example, **ted1** can run the **shutdown** command on **ted2**.

  ```text
  newuser1 ted1=(root) /usr/sbin/useradd,/usr/sbin/userdel
  ```

  This indicates that **newuser1** on the **ted1** host can run the  **useradd**  and  **userdel**  commands as the user  **root**.

  > \[!NOTE] **NOTE:**
  >
  > * You can define multiple aliases in a line and separate them with colons (:).
  > * You can add an exclamation mark (!) before a command or a command alias to make the command or the command alias invalid.
  > * There are two keywords: **ALL** and **NOPASSWD**. ALL indicates all files, hosts, or commands, and **NOPASSWD** indicates that no password is required.
  > * By modifying user access, you can change the access permission of a common user to be the same as that of the user  **root**. Then, you can grant rights to the common user.

The following is an example of the  **sudoers**  file:

```bash
#sudoers files
#User alias specification
User_Alias ADMIN=ted1:POWERUSER=globus,ted2
#user privilege specification
ADMIN ALL=ALL
POWERUSER ALL=ALL,!/bin/su
```

In the preceding information:

* User\_Alias ADMIN=ted1:POWERUSER=globus,ted2

  Two aliases ADMIN and POWERUSER are defined.

* ADMIN ALL=ALL

  ADMIN can run all commands as the user  **root**  on all hosts.

* POWERUSER ALL=ALL,!/bin/su

  POWERUSER can run all commands except the  **su**  command as the user  **root**  on all hosts.

## Managing User Groups

### Adding a User Group

#### groupadd Command

Run the  **groupadd**  command as the **root** user to add user group information to the system. In the command,  *options*  indicates related parameters and  *groupname*  indicates the group name.

```bash
groupadd [options] groupname
```

#### Example

For example, to create a user group named groupexample, run the following command as the **root** user:

```bash
groupadd groupexample
```

#### User Group Information Files

The following files contain user group information:

* /etc/gshadow file: user group encryption information
* /etc/group file: group information
* /etc/login.defs: system wide settings

### Modifying a User Group

#### Changing a GID

To change the user group ID, run the following command as the **root** user. In the command, *GID*  indicates the target user group ID and  *groupname*  indicates the user group name. Change them as required.

```bash
groupmod -g GID groupname
```

#### Changing a User Group Name

To change the user group name, run the following command as the **root** user. In the command, *newgroupname*  indicates the user group new name and  *oldgroupname*  indicates the user group name. Change them as required.

```bash
groupmod -n newgroupname oldgroupname
```

### Deleting a User Group

Run the  **groupdel**  command as the **root** user to delete an existing user group.

For example, run the following command to delete user group Test:

```bash
groupdel Test
```

> \[!NOTE] **NOTE:**
>
> Each user has only one primary group, which is created by default when creating a user. The user's primary group cannot be directly deleted. To forcibly delete a user's primary group, run the  **groupdel -f** *Test*  command.

### Adding a User to a Group or Removing a User from a Group

Run the  **gpasswd**  command as the **root** user to add a user to a group or remove a user from a group.

For example, run the following command to add the user userexample to the user group Test:

```bash
gpasswd -a userexample Test
```

For example, run the following command to remove the user userexample from the user group Test:

```bash
gpasswd -d userexample Test
```

### Changing the Current Group of a User to a Specified Group

If a user belongs to multiple user groups, the user can run the **newgrp** command to switch to another user group after logging in to the system. Then, the user has the permission of the corresponding group.

For example, run the following command to change the current group of the user **userexample** to the user group **Test**:

```bash
newgrp Test
```

---

---
url: /en/docs/22.03_LTS_SP4/server/security/safeguard/safeguard_user_guide.md
---
# User Guide

## Configuration

The safeguard configuration file is a YAML file that contains `key:value` or `key:[value list]` pairs.

## Configuration Items

| Configuration Item | Type | Description |
|:------:|:----|:-----------:|
| `network` | List  | Rule for network restrictions. |
| `files` | List  | Rule for file access restrictions. |
| `process` | List  | Rule for process restrictions. |
| `mount` | List  | Rule for mount restrictions. |
| `dns_proxy` | List  | DNS proxy configurations. |
| `log` | List containing the following sub-keys: `format: [json\|text]``output: <path>``max_size`: Maximum size to rotate (MB). Default: 100MB`max_age`: Period for which logs are kept. Default: 365`labels`: Key/Value to be added to the log.| Log configuration. |

## Network

| Configuration Item | Type | Description |
|:------:|:----|:-----------:|
| `enable` | Enum with the following possible values: `true`, `false` | Whether to enable restrictions or not. Default is `true`. |
| `mode` | Enum with the following possible values: `monitor`, `block` | If `monitor` is specified, events are only logged. If `block` is specified, network access is blocked. |
| `target` | Enum with the following possible values: `host`, `container` | Selecting `host` will apply the restriction to hosts. Selecting `container` will apply the restriction only to containers. |
| `cidr` | List containing the following sub-keys:`allow: [cidr list]``deny: [cidr list]`| Allow or deny CIDRs. |
| `domain` | List containing the following sub-keys:`allow: [domain list]``deny: [domain list]`| Allow or deny domains. |
| `command` | List containing the following sub-keys:`allow: [command list]``deny: [command list]`| Allow or deny commands. |
| `uid` | List containing the following sub-keys:`allow: [uid list]``deny: [uid list]`| Allow or deny UIDs. |
| `gid` | List containing the following sub-keys:`allow: [gid list]``deny: [gid list]`| Allow or deny GIDs. |

### Examples

#### Allowing All Network Connections

Allow all network communications and monitor their connections.

```yaml
network:
  mode: monitor
  target: host
  cidr:
    allow: ['0.0.0.0/0']
```

#### Blocking Specified Private Networks

Block access to `192.168.1.1/24` and `10.0.1.1/24`.

```yaml
network:
  mode: block
  target: host
  cidr:
    allow: ['0.0.0.0/0']
    deny:
      - 192.168.1.1/24
      - 10.0.1.1/24
```

#### Blocking Metadata Service API

Block access to the public cloud Metadata Service. This is a mitigation measure against SSRF, etc.

```yaml
network:
  mode: block
  target: host
  cidr:
    allow: ['0.0.0.0/0']
    deny:
      - 169.254.169.254/32
```

#### Blocking Connections to a Specified Domain

Block connections to `example.com`. safeguard periodically looks up IP addresses to keep up with IP address changes.

```yaml
network:
  mode: block
  target: host
  cidr:
    allow: ['0.0.0.0/0']
  domain:
    deny:
      - example.com
```

#### Blocking Network Connections of Containers

Allow communication from hosts, but block communication from containers.

```yaml
network:
  mode: block
  target: container
  cidr:
    allow: ['0.0.0.0/0']
  domain:
    deny:
    - example.com
```

!!! example

```shell
vagrant@ubuntu-impish:~$ curl -I https://example.com
HTTP/2 200

vagrant@ubuntu-impish:~$ sudo docker run --rm -it curlimages/curl https://example.com
curl: (7) Couldn't connect to server
```

#### Blocking All Connections from cURL

```yaml
network:
  mode: monitor
  target: container
  cidr:
    allow: ['0.0.0.0/0']
  command:
    deny: ['curl']
```

!!! example

```shell
vagrant@ubuntu-impish:~$ curl -I https://example.com
curl: (6) Could not resolve host: example.com

vagrant@ubuntu-impish:~$ wget https://example.com -O /dev/null
--2022-03-09 14:45:11--  http://example.com/
Resolving example.com (example.com)... 93.184.216.34
Connecting to example.com (example.com)|93.184.216.34|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1256 (1.2K) [text/html]
Saving to: '/dev/null'

/dev/null               100%[============================>]   1.23K  --.-KB/s    in 0s

2022-03-09 14:45:12 (70.1 MB/s) - '/dev/null' saved [1256/1256]
```

#### Blocking All Connections from the User Whose UID Is 1000

Block network access of the user whose UID is 1000, but allow network access of the user whose UID is 0.

```yaml
network:
  mode: monitor
  target: container
  cidr:
    allow: ['0.0.0.0/0']
  uid:
    allow: [0]
    deny: [1000]
```

!!! example

```shell
vagrant@ubuntu-impish:~$ id
uid=1000(vagrant) gid=1000(vagrant) groups=1000(vagrant)

vagrant@ubuntu-impish:~$ curl -I https://example.com
curl: (6) Could not resolve host: example.com

vagrant@ubuntu-impish:~$ sudo curl -I https://example.com
HTTP/2 200
```

## Files

Linux kernel 5.13 is required to use these options.

| Config | Type | Description |
|:------:|:----|:-----------:|
| `enable` | Enum with the following possible values: `true`, `false` | Whether to enable restrictions or not. The default value is `true`. |
| `mode` | Enum with the following possible values: `monitor`, `block` | If `monitor` is specified, events are only logged. If `block` is specified, network access is blocked. |
| `target` | Enum with the following possible values: `host`, `container` | Selecting `host` will apply the restriction to hosts. Selecting `container` will apply the restriction to containers. |
| `allow` | List of allowed file paths| |
| `deny` | List of denied file paths| |

### Examples

#### Allowing Access to All Files

```yaml
file:
  mode: monitor
  target: host
  allow:
    - /
```

#### Blocking Access to `/etc/passwd`

```yaml
file:
  mode: block
  target: host
  allow:
    - /
  deny:
    - /etc/passwd
```

#### Blocking All Accesses to `/root/.ssh`

```yaml
file:
  mode: block
  target: host
  allow:
    - /
  deny:
    - /root/.ssh
```

#### Blocking Access to `/proc/sys` in Containers

```yaml
file:
  mode: block
  target: container
  allow:
    - /
  deny:
    - /proc/sys
```

!!! example

```shell
root@ubuntu-impish:/# ls /proc/sys
abi  debug  dev  fs  kernel  net  user  vm

root@ubuntu-impish:/# docker run --privileged --rm -it ubuntu:latest bash
root@9cf961922b00:/# ls /proc/sys
ls: cannot open directory '/proc/sys': Operation not permitted
```

#### Blocking Escapes from Privileged Containers

```yaml
file:
  mode: block
  target: container
  allow:
    - /
  deny:
    - /proc/sysrq-trigger
    - /sys/kernel
    - /proc/sys/kernel
```

!!! example

```shell
root@ubuntu-impish:/# docker run --privileged --rm -it ubuntu:latest bash
root@e3b2ffe5b284:/# echo c > /proc/sysrq-trigger
bash: /proc/sysrq-trigger: Operation not permitted

root@e3b2ffe5b284:/# echo '/path/to/evil' > /sys/kernel/uevent_helper
bash: /sys/kernel/uevent_helper: Operation not permitted

root@e3b2ffe5b284:/# echo '|/path/to/evil' > /proc/sys/kernel/core_pattern
bash: /proc/sys/kernel/core_pattern: Operation not permitted
```

## Processes

| Configuration Item | Type | Description |
|:------:|:----|:-----------:|
| `enable` | Enum with the following possible values: `true`, `false` | Whether to enable restrictions or not. The default value is `true`. |
| `mode` | Enum with the following possible value: `monitor` | If `monitor` is specified, events are only logged.  |
| `target` | Enum with the following possible values: `host`, `container` | Selecting `host` will apply the restriction to hosts. Selecting `container` will apply the restriction to containers. |

### Examples

```yaml
mount:
  mode: monitor
  target: host
```

## Mount

| Configuration Item | Type | Description |
|:------:|:----|:-----------:|
| `enable` | Enum with the following possible values: `true`, `false` | Whether to enable restrictions or not. The default value is `true`. |
| `mode` | Enum with the following possible values: `monitor`, `block` | If `monitor` is specified, events are only logged. If `block` is specified, accesses are blocked. |
| `target` | Enum with the following possible values: `host`, `container` | Selecting `host` will apply the restriction to hosts. Selecting `container` will apply the restriction to containers. |
| `deny` | List of allowed mount paths | |

### Examples

#### Blocking the Mount of `/var/run/docker.sock` to Containers

```yaml
mount:
  mode: block
  target: host
  deny:
    - /var/run/docker.sock
```

---

---
url: /en/docs/22.03_LTS_SP4/server/releasenotes/user_notice.md
---
# User Notice

* The version number counting rule of openEuler is changed from openEuler  *x.x*  to openEuler  *year*.*month*. For example, openEuler 21.03 indicates that the version is released in March 2021.
* The [Python core team](https://www.python.org/dev/peps/pep-0373/#update) has stopped maintaining Python 2 in January 2020. Python 2 reached end of maintenance (EOM) on December 31, 2020. In 2021, openEuler 21.03 fixed only the critical CVEs related to Python 2. Please switch to Python 3 as soon as possible.
* From openEuler 22.03 LTS, only Python 3 is supported.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/administrator/using_dnf_to_manage_software_packages.md
---
# Using DNF to Manage Software Packages

DNF is a Linux software package management tool used to manage RPM software packages. The DNF can query software package information, obtain software packages from a specified software library, automatically process dependencies to install or uninstall software packages, and update the system to the latest available version.

> \[!NOTE] **NOTE:**
>
> * DNF is fully compatible with YUM and provides YUM-compatible command lines and APIs for extensions and plug-ins.
> * You must have the administrator rights to use the DNF. All commands in this chapter must be executed by the administrator.

## Configuring the DNF

### The DNF Configuration File

The main configuration file of the DNF is /etc/dnf/dnf.conf which consists of two parts:

* The  **main**  part in the file stores the global settings of the DNF.

* The **repository** part in the file stores the settings of the software source. You can add one or more  **repository**  sections to the file.

In addition, the /etc/yum.repos.d directory stores one or more repo source files, which define different repositories.

You can configure a software source by either directly configuring the /etc/dnf/dnf.conf file or configuring the .repo file in the /etc/yum.repos.d directory.

#### Configuring the main Part

The /etc/dnf/dnf.conf file contains the  **main**  part. The following is an example of the configuration file:

```bashconf
[main]
gpgcheck=1
installonly_limit=3
clean_requirements_on_remove=True
best=True
skip_if_unavailable=False
```

Common options are as follows:

**Table  1**  main parameter description

#### Configuring the repository Part

The repository part allows you to customize openEuler software source repositories. The name of each repository must be unique. Otherwise, conflicts may occur. You can configure a software source by either directly configuring the /etc/dnf/dnf.conf file or configuring the .repo file in the /etc/yum.repos.d directory.

* Configuring the /etc/dnf/dnf.conf file

  The following is a minimum configuration example of the \[repository] section:

  ```text
  [repository]
  name=repository_name
  baseurl=repository_url
  ```

  > \[!NOTE] **NOTE:**
  > openEuler provides an online image source at  <https://repo.openeuler.org/>. For example, if the openEuler 22.03 LTS SP4 version is aarch64, the  **baseurl**  can be set to  <https://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/aarch64/>.

  Common options are as follows:

  **Table  2**  repository parameter description&#x20;

* Configuring the .repo file in the /etc/yum.repos.d directory
  openEuler provides multiple repo sources for users online. For details about the repo sources, see [Installing the OS](./../../releasenotes/os_installation.md).

  For example, run the following command as the **root** user to add the openeuler repo source to the openEuler.repo file.

  ```bash
  vi /etc/yum.repos.d/openEuler.repo
  ```

  ```text
  [OS]
  name=openEuler-$releasever - OS
  baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/
  enabled=1
  gpgcheck=1
  gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler
  ```

  > \[!NOTE] **NOTE:**
  >
  > * **enabled** indicates whether to enable the software source repository. The value can be **1** or **0**. The default value is **1**, indicating that the software source repository is enabled.
  > * **gpgkey** is the public key used to verify the signature.

#### Displays the Current Configuration

* To display the current configuration information, run the following command:

  ```bash
  dnf config-manager --dump
  ```

* To display the configuration of a software source, query the repo id:

  ```bash
  dnf repolist
  ```

  Run the following command to display the software source configuration of the corresponding ID. In the command,  *repository*  indicates the repository ID.

  ```bash
  dnf config-manager --dump repository
  ```

* You can also use a global regular expression to display all matching configurations.

  ```bash
  dnf config-manager --dump glob_expression
  ```

### Creating a Local Software Repository

To create a local repository of software sources, perform the following steps.

1. Install the createrepo software package. Run the following command as the root user:

   ```bash
   dnf install createrepo
   ```

2. Copy the required software packages to a directory, for example, /mnt/local\_repo/.

3. Run the following command to create a software source:

   ```bash
   createrepo /mnt/local_repo
   ```

### Adding, Enabling, and Disabling Software Sources

This section describes how to add, enable, and disable the software source repository by running the  **dnf config-manager**  command.

#### Adding Software Source

To define a new software repository, you can add the repository part to the /etc/dnf/dnf.conf file or add the .repo file to the /etc/yum.repos.d/ directory. You are advised to add the .repo file. Each software source has its own .repo file. The following describes how to add the .repo file.

To add such a source to your system, run the following command as the user  **root**. After the command is executed, the corresponding .repo file is generated in the  **/etc/yum.repos.d/**  directory. In the command,  *repository\_url*  indicates the repo source address. For details, see  [Table 2](#en-us_topic_0151921080_a4a0b069bbf624b09be3bdd08567c0445).

```bash
dnf config-manager --add-repo repository_url
```

#### Enabling a Software Repository

To enable the software source, run the following command as the user  **root**. In the command,  *repository*  indicates the repository ID in the new .repo file. You can run the  **dnf repolist**  command to query the repository ID.

```bash
dnf config-manager --set-enable repository
```

You can also use a global regular expression to enable all matching software sources. In the command,  *glob\_expression*  indicates the regular expression used to match multiple repository IDs.

```bash
dnf config-manager --set-enable glob_expression
```

#### Disabling a Software Repository

To disable a software source, run the following command as the user  **root**:

```bash
dnf config-manager --set-disable repository
```

You can also use a global regular expression to disable all matching software sources.

```bash
dnf config-manager --set-disable glob_expression
```

## Managing Software Package

The DNF enables you to query, install, and delete software packages.

### Searching for Software Packages

You can search for the required RPM package by its name, abbreviation, or description. The command is as follows:

```bash
dnf search term
```

The following is an example:

```bash
$   dnf search httpd
========================================== N/S matched: httpd ==========================================
httpd.aarch64 : Apache HTTP Server
httpd-devel.aarch64 : Development interfaces for the Apache HTTP server
httpd-manual.noarch : Documentation for the Apache HTTP server
httpd-tools.aarch64  : Tools for use with the Apache HTTP Server
libmicrohttpd.aarch64  : Lightweight library for embedding a webserver in applications
mod_auth_mellon.aarch64  : A SAML 2.0 authentication module for the Apache Httpd Server
mod_dav_svn.aarch64  : Apache httpd module for Subversion server
```

### Listing Software Packages

To list all installed and available RPM packages in the system, run the following command:

```bash
dnf list all
```

To list a specific RPM package in the system, run the following command:

```bash
dnf list glob_expression...
```

The following is an example:

```bash
$ dnf list httpd
Available Packages
httpd.aarch64              2.4.51-17.oe2203SP4           Local
```

### Displaying RPM Package Information

To view information about one or more RPM packages, run the following command:

```bash
dnf info package_name...
```

The following is a command example:

```bash
$ dnf info httpd
Available Packages
Name        : httpd
Version     : 2.4.51
Release     : 17.oe2203SP4
Arch        : aarch64
Size        : 1.2 M
Repo        : Local
Summary     : Apache HTTP Server
URL         : http://httpd.apache.org/
License     : ASL 2.0
Description : The Apache HTTP Server is a powerful, efficient, and extensible
            : web server.
```

### Installing an RPM Package

To install a software package and all its dependencies that have not been installed, run the following command as the user  **root**:

```bash
dnf install package_name
```

You can also add software package names to install multiple software packages at the same time. Add the  **strict=False**  parameter to the /etc/dnf/dnf.conf configuration file and run the  **dnf**  command to add --setopt=strict=0. Run the following command as the user  **root**:

```bash
dnf install package_name package_name... --setopt=strict=0
```

The following is an example:

```bash
dnf install httpd
```

> \[!NOTE] **NOTE:**
>
> * If the RPM package fails to be installed, see  [Installation Failure Caused by Software Package Conflict, File Conflict, or Missing Software Package](https://docs.openeuler.openatom.cn/en/docs/common/faq/server/administration_faqs.html#_5-installation-failure-caused-by-software-package-conflict-file-conflict-or-missing-software-package).
> * Do not install the install-scripts software package. This software package is used by imageTailor for tailoring ISO images. openEuler will fail to boot if install-scripts is installed.

### Downloading Software Packages

To download the software package using the DNF, run the following command as the user  **root**:

```bash
dnf download package_name
```

If you need to download the dependency packages that are not installed, add  **--resolve**. The command is as follows:

```bash
dnf download --resolve package_name
```

The following is an example:

```bash
dnf download --resolve httpd
```

### Deleting a Software Package

To uninstall the software package and related dependent software packages, run the following command as the user  **root**:

```bash
dnf remove package_name...
```

The following is an example:

```bash
dnf remove totem
```

## Managing Software Package Groups

A software package set is a group of software packages that serve a common purpose, for example, a system tool set. You can use the DNF to install or delete software package groups, improving operation efficiency.

### Listing Software Package Groups

The summary parameter can be used to list the number of all installed software package groups, available groups, and available environment groups in the system. The command is as follows:

```bash
dnf groups summary
```

The following is an example:

```bash
$ dnf groups summary
Last metadata expiration check: 0:11:56 ago on Sat 17 Aug 2019 07:45:14 PM CST.
Available Groups: 8
```

To list all software package groups and their group IDs, run the following command:

```bash
dnf group list
```

The following is an example:

````bash
$ dnf group list
Last metadata expiration check: 0:10:32 ago on Sat 17 Aug 2019 07:45:14 PM CST.
Available Environment Groups:
   Minimal Install
   Custom Operating System
   Server
Available Groups:
   Development Tools
   Graphical Administration Tools
   Headless Management
   Legacy UNIX Compatibility
   Network Servers
   Scientific Support
   Security Tools
   System Tools

```bash

### Displaying the Software Package Group Information

To list the mandatory and optional packages contained in a software package group, run the following command:

```bash
dnf group info glob_expression...
````

The following is an example of displaying the Development Tools information:

```bash
$ dnf group info "Development Tools"
Last metadata expiration check: 0:14:54 ago on Wed 05 Jun 2019 08:38:02 PM CST.

Group: Development Tools
 Description: A basic development environment.
 Mandatory Packages:
   binutils
   glibc-devel
   make
   pkgconf
   pkgconf-m4
   pkgconf-pkg-config
   rpm-sign
 Optional Packages:
   expect
```

### Installation Software Package Group

Each software package group has its own name and corresponding group ID. You can use the software package group name or its ID to install the software package.

To install a software package group, run the following command as the user  **root**:

```bash
dnf group install group_name
```

or

```bash
dnf group install groupid
```

For example, to install the software package group of Development Tools, run the following command:

```bash
dnf group install "Development Tools"
```

or

```bash
dnf group install development
```

### Deleting a Software Package Group

To uninstall a software package group, you can use the group name or ID to run the following command as the user  **root**:

```bash
dnf group remove group_name
```

```bash
dnf group remove groupid
```

For example, to delete the software package group of Development Tools, run the following command:

```bash
dnf group remove "Development Tools"
```

```bash
dnf group remove development
```

## Check and Update

You can use the DNF to check whether any software package in your system needs to be updated. You can use the DNF to list the software packages to be updated. You can choose to update all packages at a time or update only specified packages.

### Checking for Update

To list all currently available updates, run the following command:

```bash
dnf check-update
```

The following is an example:

```bash
$ dnf check-update
Last metadata expiration check: 0:02:10 ago on Sun 21 May 2023 11:28:07 PM  CST.

anaconda-core.aarch64       36.15.5-17.oe2203SP4             update
anaconda-tui.aarch64        36.15.5-17.oe2203SP4             update
anaconda-user-help.aarch64  26.1-10.oe2203SP4                update
bind-libs.aarch64           9.16.23-18.oe2203SP4             update
bind-license.noarch         9.16.23-18.oe2203SP4             update
bind-utils.aarch64          9.16.23-18.oe2203SP4             updatey
...
```

### Upgrade

To upgrade a single software package, run the following command as the user  **root**:

```bash
dnf update package_name
```

For example, to upgrade the RPM package, run the following command:

```bash
$ dnf update anaconda-gui.aarch64
Last metadata expiration check: 0:02:10 ago on Sun 01 Sep 2019 11:30:27 PM  CST.
Dependencies Resolved
================================================================================
 Package                  Arch         Version              Repository     Size
================================================================================
Updating:
 anaconda-gui             aarch64      19.31.123-1.14       updates       461 k
 anaconda-core            aarch64      19.31.123-1.14       updates       1.4 M
 anaconda-tui             aarch64      19.31.123-1.14       updates       274 k
 anaconda-user-help       aarch64      19.31.123-1.14       updates       315 k
 anaconda-widgets         aarch64      19.31.123-1.14       updates       748 k

Transaction Summary
================================================================================
Upgrade  5 Package

Total download size: 3.1 M
Is this ok [y/N]:
```

Similarly, to upgrade a software package group, run the following command as the user  **root**:

```bash
dnf group update group_name
```

### Updating All Packages and Their Dependencies

To update all packages and their dependencies, run the following command as the user  **root**:

```bash
dnf update
```

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/gala/using_gala_anteater.md
---
# Using gala-anteater

gala-anteater is an AI-based operating system exception detection platform. It provides functions such as time series data preprocessing, exception detection, and exception reporting. Based on offline pre-training, online model incremental learning and model update, it can be well adapted to multi-dimensional and multi-modal data fault diagnosis.

This chapter describes how to deploy and use the gala-anteater service.

## Installation

Mount the repositories.

```basic
[everything]
name=everything
baseurl=http://121.36.84.172/dailybuild/EBS-openEuler-22.03-LTS-SP4/EBS-openEuler-22.03-LTS-SP4/everything/$basearch/
enabled=1
gpgcheck=0
priority=1

[EPOL]
name=EPOL
baseurl=http://repo.openeuler.org/EBS-openEuler-22.03-LTS-SP4/EPOL/main/$basearch/
enabled=1
gpgcheck=0
priority=1

```

Install gala-anteater.

```bash
# yum install gala-anteater
```

## Configuration

> Note: Some gala-anteater parameters can be configured in **/etc/gala-anteater/config/gala-anteater.yaml**.

### Startup Parameters

| Parameter| Parameter Full Name| Type| Mandatory (Yes/No)| Default Value| Name| Description|
|---|---|---|---|---|---|---|
| -ks | --kafka\_server | string | True |  | KAFKA\_SERVER | IP address of the Kafka server, for example, **localhost / xxx.xxx.xxx.xxx**.|
| -kp | --kafka\_port | string | True |  | KAFKA\_PORT | Port number of the Kafka server, for example, **9092**.|
| -ps | --prometheus\_server | string | True |  | PROMETHEUS\_SERVER | IP address of the Prometheus server, for example, **localhost / xxx.xxx.xxx.xxx**.|
| -pp | --prometheus\_port | string | True |  | PROMETHEUS\_PORT | Port number of the Prometheus server, for example, **9090**.|
| -m | --model | string | False | vae | MODEL | Exception detection model. Currently, two exception detection models are supported: **random\_forest** and **vae**.**random\_forest**: random forest model, which does not support online learning**vae**: Variational Atuoencoder (VAE), which is an unsupervised model and supports model update based on historical data during the first startup.|
| -d | --duration | int | False | 1 | DURATION | Frequency of executing the exception detection model. The unit is minute, which means that the detection is performed every *x* minutes.|
| -r | --retrain | bool | False | False | RETRAIN | Whether to use historical data to update and iterate the model during startup. Currently, only the VAE model is supported.|
| -l | --look\_back | int | False | 4 | LOOK\_BACK | Whether to update the model based on the historical data of the last *x* days.|
| -t | --threshold | float | False | 0.8 | THRESHOLD | Threshold of the exception detection model, ranging from 0 to 1. A larger value can reduce the false positive rate of the model. It is recommended that the value be greater than or equal to 0.5.|
| -sli | --sli\_time | int | False | 400 | SLI\_TIME | Application performance metric. The unit is ms. A larger value can reduce the false positive rate of the model. It is recommended that the value be greater than or equal to 200.For scenarios with a high false positive rate, it is recommended that the value be greater than 1000.|

## Start

Start gala-anteater.

> Note: gala-anteater can be started and run in command line mode, but cannot be started and run in systemd mode.

* Running in online training mode (recommended)

```bash
gala-anteater -ks {ip} -kp {port} -ps {ip} -pp {port} -m vae -r True -l 7 -t 0.6 -sli 400
```

* Running in common mode

```bash
gala-anteater -ks {ip} -kp {port} -ps {ip} -pp {port} -m vae -t 0.6 -sli 400
```

Query the gala-anteater service status.

If the following information is displayed, the service is started successfully. The startup log is saved to the **logs/anteater.log** file in the current running directory.

```log
2022-09-01 17:52:54,435 - root - INFO - Run gala_anteater main function...
2022-09-01 17:52:54,436 - root - INFO - Start to try updating global configurations by querying data from Kafka!
2022-09-01 17:52:54,994 - root - INFO - Loads metric and operators from file: xxx\metrics.csv
2022-09-01 17:52:54,997 - root - INFO - Loads metric and operators from file: xxx\metrics.csv
2022-09-01 17:52:54,998 - root - INFO - Start to re-train the model based on last day metrics dataset!
2022-09-01 17:52:54,998 - root - INFO - Get training data during 2022-08-31 17:52:00+08:00 to 2022-09-01 17:52:00+08:00!
2022-09-01 17:53:06,994 - root - INFO - Spends: 11.995422840118408 seconds to get unique machine_ids!
2022-09-01 17:53:06,995 - root - INFO - The number of unique machine ids is: 1!                            
2022-09-01 17:53:06,996 - root - INFO - Fetch metric values from machine: xxxx.
2022-09-01 17:53:38,385 - root - INFO - Spends: 31.3896164894104 seconds to get get all metric values!
2022-09-01 17:53:38,392 - root - INFO - The shape of training data: (17281, 136)
2022-09-01 17:53:38,444 - root - INFO - Start to execute vae model training...
2022-09-01 17:53:38,456 - root - INFO - Using cpu device
2022-09-01 17:53:38,658 - root - INFO - Epoch(s): 0     train Loss: 136.68      validate Loss: 117.00
2022-09-01 17:53:38,852 - root - INFO - Epoch(s): 1     train Loss: 113.73      validate Loss: 110.05
2022-09-01 17:53:39,044 - root - INFO - Epoch(s): 2     train Loss: 110.60      validate Loss: 108.76
2022-09-01 17:53:39,235 - root - INFO - Epoch(s): 3     train Loss: 109.39      validate Loss: 106.93
2022-09-01 17:53:39,419 - root - INFO - Epoch(s): 4     train Loss: 106.48      validate Loss: 103.37
...
2022-09-01 17:53:57,744 - root - INFO - Epoch(s): 98    train Loss: 97.63       validate Loss: 96.76
2022-09-01 17:53:57,945 - root - INFO - Epoch(s): 99    train Loss: 97.75       validate Loss: 96.58
2022-09-01 17:53:57,969 - root - INFO - Schedule recurrent job with time interval 1 minute(s).
2022-09-01 17:53:57,973 - apscheduler.scheduler - INFO - Adding job tentatively -- it will be properly scheduled when the scheduler starts
2022-09-01 17:53:57,974 - apscheduler.scheduler - INFO - Added job "partial" to job store "default"
2022-09-01 17:53:57,974 - apscheduler.scheduler - INFO - Scheduler started
2022-09-01 17:53:57,975 - apscheduler.scheduler - DEBUG - Looking for jobs to run
2022-09-01 17:53:57,975 - apscheduler.scheduler - DEBUG - Next wakeup is due at 2022-09-01 17:54:57.973533+08:00 (in 59.998006 seconds)
```

## Output Data

If gala-anteater detects an exception, it sends the result to Kafka. The output data format is as follows:

```json
{
   "Timestamp":1659075600000,
   "Attributes":{
      "entity_id":"xxxxxx_sli_1513_18",
      "event_id":"1659075600000_1fd37742xxxx_sli_1513_18",
      "event_type":"app"
   },
   "Resource":{
      "anomaly_score":1.0,
      "anomaly_count":13,
      "total_count":13,
      "duration":60,
      "anomaly_ratio":1.0,
      "metric_label":{
         "machine_id":"1fd37742xxxx",
         "tgid":"1513",
         "conn_fd":"18"
      },
      "recommend_metrics":{
         "gala_gopher_tcp_link_notack_bytes":{
            "label":{
               "__name__":"gala_gopher_tcp_link_notack_bytes",
               "client_ip":"x.x.x.165",
               "client_port":"51352",
               "hostname":"localhost.localdomain",
               "instance":"x.x.x.172:8888",
               "job":"prometheus-x.x.x.172",
               "machine_id":"xxxxxx",
               "protocol":"2",
               "role":"0",
               "server_ip":"x.x.x.172",
               "server_port":"8888",
               "tgid":"3381701"
            },
            "score":0.24421279500639545
         },
         ...
      },
      "metrics":"gala_gopher_ksliprobe_recent_rtt_nsec"
   },
   "SeverityText":"WARN",
   "SeverityNumber":14,
   "Body":"TimeStamp, WARN, APP may be impacting sli performance issues."
}
```

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/gala/using_gala_gopher.md
---
# Using gala-gopher

As a data collection module, gala-gopher provides OS-level monitoring capabilities, supports dynamic probe installation and uninstallation, and integrates third-party probes in a non-intrusive manner to quickly expand the monitoring scope.

This chapter describes how to deploy and use the gala-gopher service.

## Installation

Mount the repositories.

```basic
[oe-22.03-lts-SP4-everything] # openEuler 22.03-LTS-SP4 官方发布源
name=oe-2203-lts-SP4-everything
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-update] # openEuler 22.03-LTS-SP4 Update 官方发布源
name=oe-22.03-lts-SP4-epol-update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/update/main/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-main] # openEuler 22.03-LTS-SP4 EPOL 官方发布源
name=oe-22.03-lts-SP4-epol-main
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/x86_64/
enabled=1
gpgcheck=0
priority=1
```

Install gala-gopher.

```bash
yum install gala-gopher
```

## Configuration

### Configuration Description

The configuration file of gala-gopher is **/opt/gala-gopher/gala-gopher.conf**. The configuration items in the file are described as follows (the parts that do not need to be manually configured are not described):

The following configurations can be modified as required:

* `global`: gala-gopher global configuration information.
  * `log_file_name`: gala-gopher log file name.
  * `log_level`: gala-gopher log level. This configuration is not available currently.
  * `pin_path`: path for storing the map shared by the eBPF probe. You are advised to retain the default value.
* `metric`: metric output mode.
  * `out_channel`: metric output channel. The value can be `web_server` or `kafka`. If this parameter is left empty, the output channel is disabled.
  * `kafka_topic`: topic configuration information if the output channel is Kafka.
* `event`: output mode of abnormal events.
  * `out_channel`: event output channel. The value can be `logs` or `kafka`. If this parameter is left empty, the output channel is disabled.
  * `kafka_topic`: topic configuration information if the output channel is Kafka.
* `meta`: metadata output mode.
  * `out_channel`: metadata output channel. The value can be `logs` or `kafka`. If this parameter is left empty, the output channel is disabled.
  * `kafka_topic`: topic configuration information if the output channel is Kafka.
* `imdb`: cache specification configuration.
  * `max_tables_num`: maximum number of cache tables. In the **/opt/gala-gopher/meta** directory, each meta corresponds to a table.
  * `max_records_num`: maximum number of records in each cache table. Generally, each probe generates at least one observation record in an observation period.
  * `max_metrics_num`: maximum number of metrics contained in each observation record.
  * `record_timeout`: aging time of the cache table. If a record in the cache table is not updated within the aging time, the record is deleted. The unit is second.
* `web_server`: configuration of the web\_server output channel.
  * `port`: listening port.
* `kafka`: configuration of the Kafka output channel.
  * `kafka_broker`: IP address and port number of the Kafka server.
* `logs`: configuration of the logs output channel.
  * `metric_dir`: path for storing metric data logs.
  * `event_dir`: path for storing abnormal event data logs.
  * `meta_dir`: metadata log path.
  * `debug_dir`: path of gala-gopher run logs.
* `probes`: native probe configuration.
  * `name`: probe name, which must be the same as the native probe name. For example, the name of the **example.probe** probe is **example**.
  * `param`: probe startup parameters. For details about the supported parameters, see [Startup Parameters](#startup-parameters).
  * `switch`: whether to start a probe. The value can be `on` or `off`.
* `extend_probes`: third-party probe configuration.
  * `name`: probe name.
  * `command`: command for starting a probe.
  * `param`: probe startup parameters. For details about the supported parameters, see [Startup Parameters](#startup-parameters).
  * `start_check`: If `switch` is set to `auto`, the system determines whether to start the probe based on the execution result of `start_check`.
  * `switch`: whether to start a probe. The value can be `on`, `off`, or `auto`. The value `auto` determines whether to start the probe based on the result of `start_check`.

### Startup Parameters

| Parameter| Description                                                        |
| ------ | ------------------------------------------------------------ |
| -l     | Whether to enable the function of reporting abnormal events.                                        |
| -t     | Sampling period, in seconds. By default, the probe reports data every 5 seconds.            |
| -T     | Delay threshold, in ms. The default value is **0**.                       |
| -J     | Jitter threshold, in ms. The default value is **0**.                       |
| -O     | Offline time threshold, in ms. The default value is **0**.                       |
| -D     | Packet loss threshold. The default value is **0**.                                   |
| -F     | If this parameter is set to `task`, data is filtered by **task\_whitelist.conf**. If this parameter is set to the PID of a process, only the process is monitored.|
| -P     | Range of probe programs loaded to each probe. Currently, the tcpprobe and taskprobe probes are involved.|
| -U     | Resource usage threshold (upper limit). The default value is **0** (%).                              |
| -L     | Resource usage threshold (lower limit). The default value is **0** (%).                              |
| -c     | Whether the probe (TCP) identifies `client_port`. The default value is **0** (no).           |
| -N     | Name of the observation process of the specified probe (ksliprobe). The default value is **NULL**.             |
| -p     | Binary file path of the process to be observed, for example, `nginx_probe`. You can run `-p /user/local/sbin/nginx` to specify the Nginx file path. The default value is **NULL**.|
| -w     | Filtering scope of monitored applications, for example, `-w /opt/gala-gopher/task_whitelist.conf`. You can write the names of the applications to be monitored to the **task\_whitelist.conf** file. The default value is **NULL**, indicating that the applications are not filtered.|
| -n     | NIC to mount tc eBPF. The default value is **NULL**, indicating that all NICs are mounted. Example: `-n eth0`|

### Configuration File Example

* Select the data output channels.

  ```yaml
  metric =
  {
      out_channel = "web_server";
      kafka_topic = "gala_gopher";
  };

  event =
  {
      out_channel = "kafka";
      kafka_topic = "gala_gopher_event";
  };

  meta =
  {
      out_channel = "kafka";
      kafka_topic = "gala_gopher_metadata";
  };
  ```

* Configure Kafka and Web Server.

  ```yaml
  web_server =
  {
      port = 8888;
  };

  kafka =
  {
      kafka_broker = "<Kafka server IP address>:9092";
  };
  ```

* Select the probe to be enabled. The following is an example.

  ```yaml
  probes =
  (
      {
          name = "system_infos";
          param = "-t 5 -w /opt/gala-gopher/task_whitelist.conf -l warn -U 80";
          switch = "on";
      },
  );
  extend_probes =
  (
      {
          name = "tcp";
          command = "/opt/gala-gopher/extend_probes/tcpprobe";
          param = "-l warn -c 1 -P 7";
          switch = "on";
      }
  );
  ```

## Start

After the configuration is complete, start gala-gopher.

```bash
systemctl start gala-gopher.service
```

Query the status of the gala-gopher service.

```bash
systemctl status gala-gopher.service
```

If the following information is displayed, the service is started successfully: Check whether the enabled probe is started. If the probe thread does not exist, check the configuration file and gala-gopher run log file.

![gala-gopher-start-success](./figures/gala-gopher-start-success.png)

> Note: The root permission is required for deploying and running gala-gopher.

## How to Use

### Deployment of External Dependent Software

![gopher-arch](./figures/gopher-arch.png)

As shown in the preceding figure, the green parts are external dependent components of gala-gopher. gala-gopher outputs metric data to Prometheus, metadata and abnormal events to Kafka. gala-anteater and gala-spider in gray rectangles obtain data from Prometheus and Kafka.

> Note: Obtain the installation packages of Kafka and Prometheus from the official websites.

### Output Data

* **Metric**

  Prometheus Server has a built-in Express Browser UI. You can use PromQL statements to query metric data. For details, see [Using the expression browser](https://prometheus.io/docs/prometheus/latest/getting_started/#using-the-expression-browser) in the official document. The following is an example.

  If the specified metric is `gala_gopher_tcp_link_rcv_rtt`, the metric data displayed on the UI is as follows:

  ```text
  gala_gopher_tcp_link_rcv_rtt{client_ip="x.x.x.165",client_port="1234",hostname="openEuler",instance="x.x.x.172:8888",job="prometheus",machine_id="1fd3774xx",protocol="2",role="0",server_ip="x.x.x.172",server_port="3742",tgid="1516"} 1
  ```

* **Metadata**

  You can directly consume data from the Kafka topic `gala_gopher_metadata`. The following is an example.

  ```bash
  # Input request
  ./bin/kafka-console-consumer.sh --bootstrap-server x.x.x.165:9092 --topic gala_gopher_metadata
  # Output data
  {"timestamp": 1655888408000, "meta_name": "thread", "entity_name": "thread", "version": "1.0.0", "keys": ["machine_id", "pid"], "labels": ["hostname", "tgid", "comm", "major", "minor"], "metrics": ["fork_count", "task_io_wait_time_us", "task_io_count", "task_io_time_us", "task_hang_count"]}
  ```

* **Abnormal events**

  You can directly consume data from the Kafka topic `gala_gopher_event`. The following is an example.

  ```bash
  # Input request
  ./bin/kafka-console-consumer.sh --bootstrap-server x.x.x.165:9092 --topic gala_gopher_event
  # Output data
  {"timestamp": 1655888408000, "meta_name": "thread", "entity_name": "thread", "version": "1.0.0", "keys": ["machine_id", "pid"], "labels": ["hostname", "tgid", "comm", "major", "minor"], "metrics": ["fork_count", "task_io_wait_time_us", "task_io_count", "task_io_time_us", "task_hang_count"]}
  ```

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/gala/using_gala_spider.md
---
# Using gala-spider

This chapter describes how to deploy and use gala-spider and gala-inference.

## gala-spider

gala-spider provides the OS-level topology drawing function. It periodically obtains the data of all observed objects collected by gala-gopher (an OS-level data collection software) at a certain time point and calculates the topology relationship between them. The generated topology is saved to the graph database ArangoDB.

### Installation

Mount the Yum repositories.

```basic
[oe-22.03-lts-SP4-everything] # openEuler 22.03-LTS-SP4 官方发布源
name=oe-2203-lts-SP4-everything
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-update] # openEuler 22.03-LTS-SP4 Update 官方发布源
name=oe-22.03-lts-SP4-epol-update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/update/main/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-main] # openEuler 22.03-LTS-SP4 EPOL 官方发布源
name=oe-22.03-lts-SP4-epol-main
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/x86_64/
enabled=1
gpgcheck=0
priority=1
```

Install gala-spider.

```sh
yum install gala-spider
```

### Configuration

#### Configuration File Description

The configuration file of gala-spider is **/etc/gala-spider/gala-spider.yaml**. The configuration items in this file are described as follows:

* `global`: global configuration information.
  * `data_source`: database for collecting observation metrics. Currently, only `prometheus` is supported.
  * `data_agent`: agent for collecting observation metrics. Currently, only `gala_gopher` is supported.
* `spider`: spider configuration information.
  * `log_conf`: log configuration information.
    * `log_path`: log file path.
    * `log_level`: level of the logs to be printed. The value can be `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL`.
    * `max_size`: log file size, in MB.
    * `backup_count`: number of backup log files.
* `storage`: configuration information about the topology storage service.
  * `period`: storage period, in seconds, indicating the interval for storing the topology.
  * `database`: graph database for storage. Currently, only `arangodb` is supported.
  * `db_conf`: configuration information of the graph database.
    * `url`: IP address of the graph database server.
    * `db_name`: name of the database where the topology is stored.
* `kafka`: Kafka configuration information.
  * `server`: Kafka server address.
  * `metadata_topic`: topic name of the observed metadata messages.
  * `metadata_group_id`: consumer group ID of the observed metadata messages.
* `prometheus`: Prometheus database configuration information.
  * `base_url`: IP address of the Prometheus server.
  * `instant_api`: API for collecting data at a single time point.
  * `range_api`: API for collecting data in a time range.
  * `step`: collection time step, which is configured for `range_api`.

#### Configuration File Example

```yaml
global:
    data_source: "prometheus"
    data_agent: "gala_gopher"

prometheus:
    base_url: "http://localhost:9090/"
    instant_api: "/api/v1/query"
    range_api: "/api/v1/query_range"
    step: 1

spider:
    log_conf:
        log_path: "/var/log/gala-spider/spider.log"
        # log level: DEBUG/INFO/WARNING/ERROR/CRITICAL
        log_level: INFO
        # unit: MB
        max_size: 10
        backup_count: 10

storage:
    # unit: second
    period: 60
    database: arangodb
    db_conf:
        url: "http://localhost:8529"
        db_name: "spider"

kafka:
    server: "localhost:9092"
    metadata_topic: "gala_gopher_metadata"
    metadata_group_id: "metadata-spider"
```

### Start

* Run the following command to start gala-spider.

  ```sh
  spider-storage
  ```

* Use the systemd service to start gala-spider.

  ```sh
   systemctl start gala-spider
  ```

### How to Use

#### Deployment of External Dependent Software

The running of gala-spider depends on multiple external software for interaction. Therefore, before starting gala-spider, you need to deploy the software on which gala-spider depends. The following figure shows the software dependency of gala-spider.

![gala-spider-arch](./figures/gala-spider-arch.png)

The dotted box on the right indicates the two functional components of gala-spider. The green parts indicate the external components that gala-spider directly depends on, and the gray rectangles indicate the external components that gala-spider indirectly depends on.

* **spider-storage**: core component of gala-spider, which provides the topology storage function.
  1. Obtains the metadata of the observation object from Kafka.
  2. Obtains information about all observation object instances from Prometheus.
  3. Saves the generated topology to the graph database ArangoDB.
* **gala-inference**: core component of gala-spider, which provides the root cause locating function. It subscribes to abnormal KPI events from Kafka to trigger the root cause locating process of abnormal KPIs, constructs a fault propagation graph based on the topology obtained from the ArangoDB, and outputs the root cause locating result to Kafka.
* **prometheus**: time series database. The observation metric data collected by the gala-gopher component is reported to Prometheus for further processing.
* **kafka**: messaging middleware, which is used to store the observation object metadata reported by gala-gopher, exception events reported by the exception detection component gala-anteater, and root cause locating results reported by the cause-inference component.
* **arangodb**: graph database, which is used to store the topology generated by spider-storage.
* **gala-gopher**: data collection component. It must be deployed in advance.
* **arangodb-ui**: UI provided by ArangoDB, which can be used to query topologies.

The two functional components in gala-spider are released as independent software packages.

**spider-storage**: corresponds to the gala-spider software package in this section.

**gala-inference**: corresponds to the gala-inference software package.

For details about how to deploy the gala-gopher software, see [Using gala-gopher](./using_gala_gopher.md). This section only describes how to deploy ArangoDB.

The current ArangoDB version is 3.8.7, which has the following requirements on the operating environment:

* Only the x86 system is supported.
* GCC 10 or later

For details about ArangoDB deployment, see [Deployment](https://www.arangodb.com/docs/3.9/deployment.html) in the ArangoDB official document.

The RPM-based ArangoDB deployment process is as follows:

1. Configure the Yum repositories.

   ```basic
   [oe-22.03-lts-SP4-everything] # openEuler 22.03-LTS-SP4 官方发布源
   name=oe-2203-lts-SP4-everything
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
   enabled=1
   gpgcheck=0
   priority=1

   [oe-22.03-lts-SP4-epol-main] # openEuler 22.03-LTS-SP4 EPOL 官方发布源
   name=oe-22.03-lts-SP4-epol-main
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/x86_64/
   enabled=1
   gpgcheck=0
   priority=1
   ```

2. Install arangodb3.

   ```sh
   yum install arangodb3
   ```

3. Modify the configurations.

   The configuration file of the arangodb3 server is **/etc/arangodb3/arangod.conf**. You need to modify the following configurations:

   * `endpoint`: IP address of the arangodb3 server.
   * `authentication`: whether identity authentication is required for accessing the arangodb3 server. Currently, gala-spider does not support identity authentication. Therefore, set `authentication` to `false`.

   The following is an example.

   ```yaml
   [server]
   endpoint = tcp://0.0.0.0:8529
   authentication = false
   ```

4. Start arangodb3.

   ```sh
    systemctl start arangodb3
   ```

#### Modifying gala-spider Configuration Items

After the dependent software is started, you need to modify some configuration items in the gala-spider configuration file. The following is an example.

Configure the Kafka server address.

```yaml
kafka:
    server: "localhost:9092"
```

Configure the Prometheus server address.

```yaml
prometheus:
    base_url: "http://localhost:9090/"
```

Configure the IP address of the ArangoDB server.

```yaml
storage:
    db_conf:
        url: "http://localhost:8529"
```

#### Starting the Service

Run `systemctl start gala-spider` to start the service. Run `systemctl status gala-spider` to check the startup status. If the following information is displayed, the startup is successful:

```sh
$ systemctl status gala-spider
● gala-spider.service - a-ops gala spider service
     Loaded: loaded (/usr/lib/systemd/system/gala-spider.service; enabled; vendor preset: disabled)
     Active: active (running) since Tue 2022-08-30 17:28:38 CST; 1 day 22h ago
   Main PID: 2263793 (spider-storage)
      Tasks: 3 (limit: 98900)
     Memory: 44.2M
     CGroup: /system.slice/gala-spider.service
             └─2263793 /usr/bin/python3 /usr/bin/spider-storage
```

#### Output Example

You can query the topology generated by gala-spider on the UI provided by ArangoDB. The procedure is as follows:

1. Enter the IP address of the ArangoDB server in the address box of the browser, for example, **<http://localhost:8529>**. The ArangoDB UI is displayed.

2. Click **DB** in the upper right corner of the page to switch to the spider database.

3. On the **COLLECTIONS** page, you can view the collections of observation object instances and topology relationships stored in different time segments, as shown in the following figure.

   ![spider topology](./figures/spider_topology.png)

4. You can query the stored topology using the AQL statements provided by ArangoDB. For details, see the [AQL Documentation](https://www.arangodb.com/docs/3.8/aql/).

## gala-inference

gala-inference provides the capability of locating root causes of abnormal KPIs. It uses the exception detection result and topology as the input and outputs the root cause locating result to Kafka. The gala-inference component is archived in the gala-spider project.

### Installation

Mount the Yum repositories.

```basic
[oe-22.03-lts-SP4-everything] # openEuler 22.03-LTS-SP4 官方发布源
name=oe-2203-lts-SP4-everything
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-update] # openEuler 22.03-LTS-SP4 Update 官方发布源
name=oe-22.03-lts-SP4-epol-update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/update/main/x86_64/
enabled=1
gpgcheck=0
priority=1

[oe-22.03-lts-SP4-epol-main] # openEuler 22.03-LTS-SP4 EPOL 官方发布源
name=oe-22.03-lts-SP4-epol-main
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/x86_64/
enabled=1
gpgcheck=0
priority=1
```

Install gala-inference.

```sh
yum install gala-inference
```

### Configuration

#### Configuration File Description

The configuration items in the gala-inference configuration file **/etc/gala-inference/gala-inference.yaml** are described as follows:

* `inference`: configuration information about the root cause locating algorithm.
  * `tolerated_bias`: tolerable time offset for querying the topology at the exception time point, in seconds.
  * `topo_depth`: maximum depth for topology query.
  * `root_topk`: yop *K* root cause metrics generated in the root cause locating result.
  * `infer_policy`: root cause derivation policy, which can be `dfs` or `rw`.
  * `sample_duration`: sampling period of historical metric data, in seconds.
  * `evt_valid_duration`: valid period of abnormal system metric events during root cause locating, in seconds.
  * `evt_aging_duration`: aging period of abnormal metric events during root cause locating, in seconds.
* `kafka`: Kafka configuration information.
  * `server`: IP address of the Kafka server.
  * `metadata_topic`: configuration information about the observed metadata messages.
    * `topic_id`: topic name of the observed metadata messages.
    * `group_id`: consumer group ID of the observed metadata messages.
  * `abnormal_kpi_topic`: configuration information about abnormal KPI event messages.
    * `topic_id`: topic name of the abnormal KPI event messages.
    * `group_id`: consumer group ID of the abnormal KPI event messages.
  * `abnormal_metric_topic`: configuration information about abnormal metric event messages.
    * `topic_id`: topic name of the abnormal metric event messages.
    * `group_id`: consumer group ID of the abnormal system metric event messages.
    * `consumer_to`: timeout interval for consuming abnormal system metric event messages, in seconds.
  * `inference_topic`: configuration information about the output event messages of the root cause locating result.
    * `topic_id`: topic name of the output event messages of the root cause locating result.
* `arangodb`: configuration information about the ArangoDB graph database, which is used to query sub-topologies required for root cause locating.
  * `url`: IP address of the graph database server.
  * `db_name`: name of the database where the topology is stored.
* `log_conf`: log configuration information.
  * `log_path`: log file path.
  * `log_level`: level of the logs to be printed. The value can be `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL`.
  * `max_size`: log file size, in MB.
  * `backup_count`: number of backup log files.
* `prometheus`: Prometheus database configuration information, which is used to obtain historical time series data of metrics.
  * `base_url`: IP address of the Prometheus server.
  * `range_api`: API for collecting data in a time range.
  * `step`: collection time step, which is configured for `range_api`.

#### Configuration File Example

```yaml
inference:
  # Tolerable time offset for querying the topology at the exception time point, in seconds.
  tolerated_bias: 120
  topo_depth: 10
  root_topk: 3
  infer_policy: "dfs"
  # Unit: second
  sample_duration: 600
  # Valid period of abnormal metric events during root cause locating, in seconds.
  evt_valid_duration: 120
  # Aging period of abnormal metric events, in seconds.
  evt_aging_duration: 600

kafka:
  server: "localhost:9092"
  metadata_topic:
    topic_id: "gala_gopher_metadata"
    group_id: "metadata-inference"
  abnormal_kpi_topic:
    topic_id: "gala_anteater_hybrid_model"
    group_id: "abn-kpi-inference"
  abnormal_metric_topic:
    topic_id: "gala_anteater_metric"
    group_id: "abn-metric-inference"
    consumer_to: 1
  inference_topic:
    topic_id: "gala_cause_inference"

arangodb:
  url: "http://localhost:8529"
  db_name: "spider"

log:
  log_path: "/var/log/gala-inference/inference.log"
  # log level: DEBUG/INFO/WARNING/ERROR/CRITICAL
  log_level: INFO
  # unit: MB
  max_size: 10
  backup_count: 10

prometheus:
  base_url: "http://localhost:9090/"
  range_api: "/api/v1/query_range"
  step: 5
```

### Start

* Run the following command to start gala-inference.

  ```sh
   gala-inference
  ```

* Use the systemd service to start gala-inference.

  ```sh
   systemctl start gala-inference
  ```

### How to Use

#### Dependent Software Deployment

The running dependency of gala-inference is the same as that of gala-spider. For details, see [Deployment of External Dependent Software](#deployment-of-external-dependent-software). In addition, gala-inference indirectly depends on the running of [gala-spider](#gala-spider) and [gala-anteater](./using_gala_anteater.md). Deploy gala-spider and gala-anteater in advance.

#### Modify configuration items

Modify some configuration items in the gala-inference configuration file. The following is an example.

Configure the Kafka server address.

```yaml
kafka:
  server: "localhost:9092"
```

Configure the Prometheus server address.

```yaml
prometheus:
  base_url: "http://localhost:9090/"
```

Configure the IP address of the ArangoDB server.

```yaml
arangodb:
  url: "http://localhost:8529"
```

#### Starting the Service

Run `systemctl start gala-inference` to start the service. Run `systemctl status gala-inference` to check the startup status. If the following information is displayed, the startup is successful:

```sh
[root@openEuler ~]# systemctl status gala-inference
● gala-inference.service - a-ops gala inference service
     Loaded: loaded (/usr/lib/systemd/system/gala-inference.service; enabled; vendor preset: disabled)
     Active: active (running) since Tue 2022-08-30 17:55:33 CST; 1 day 22h ago
   Main PID: 2445875 (gala-inference)
      Tasks: 10 (limit: 98900)
     Memory: 48.7M
     CGroup: /system.slice/gala-inference.service
             └─2445875 /usr/bin/python3 /usr/bin/gala-inference
```

#### Output Example

When the exception detection module gala-anteater detects a KPI exception, it exports the corresponding abnormal KPI event to Kafka. The gala-inference keeps monitoring the message of the abnormal KPI event. If gala-inference receives the message of the abnormal KPI event, root cause locating is triggered. The root cause locating result is exported to Kafka. You can view the root cause locating result on the Kafka server. The basic procedure is as follows:

1. If Kafka is installed using the source code, go to the Kafka installation directory.

   ```sh
   cd /root/kafka_2.13-2.8.0
   ```

2. Run the command for consuming the topic to obtain the output of root cause locating.

   ```sh
   ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic gala_cause_inference
   ```

   Output example:

   ```json
   {
     "Timestamp": 1661853360000,
     "event_id": "1661853360000_1fd37742xxxx_sli_12154_19",
     "Attributes": {
       "event_id": "1661853360000_1fd37742xxxx_sli_12154_19"
     },
     "Resource": {
       "abnormal_kpi": {
         "metric_id": "gala_gopher_sli_rtt_nsec",
         "entity_id": "1fd37742xxxx_sli_12154_19",
         "timestamp": 1661853360000,
         "metric_labels": {
           "machine_id": "1fd37742xxxx",
           "tgid": "12154",
           "conn_fd": "19"
         }
       },
       "cause_metrics": [
         {
           "metric_id": "gala_gopher_proc_write_bytes",
           "entity_id": "1fd37742xxxx_proc_12154",
           "metric_labels": {
             "__name__": "gala_gopher_proc_write_bytes",
             "cmdline": "/opt/redis/redis-server x.x.x.172:3742",
             "comm": "redis-server",
             "container_id": "5a10635e2c43",
             "hostname": "openEuler",
             "instance": "x.x.x.172:8888",
             "job": "prometheus",
             "machine_id": "1fd37742xxxx",
             "pgid": "12154",
             "ppid": "12126",
             "tgid": "12154"
           },
           "timestamp": 1661853360000,
           "path": [
             {
               "metric_id": "gala_gopher_proc_write_bytes",
               "entity_id": "1fd37742xxxx_proc_12154",
               "metric_labels": {
                 "__name__": "gala_gopher_proc_write_bytes",
                 "cmdline": "/opt/redis/redis-server x.x.x.172:3742",
                 "comm": "redis-server",
                 "container_id": "5a10635e2c43",
                 "hostname": "openEuler",
                 "instance": "x.x.x.172:8888",
                 "job": "prometheus",
                 "machine_id": "1fd37742xxxx",
                 "pgid": "12154",
                 "ppid": "12126",
                 "tgid": "12154"
               },
               "timestamp": 1661853360000
             },
             {
               "metric_id": "gala_gopher_sli_rtt_nsec",
               "entity_id": "1fd37742xxxx_sli_12154_19",
               "metric_labels": {
                 "machine_id": "1fd37742xxxx",
                 "tgid": "12154",
                 "conn_fd": "19"
               },
               "timestamp": 1661853360000
             }
           ]
         }
       ]
     },
     "SeverityText": "WARN",
     "SeverityNumber": 13,
     "Body": "A cause inferring event for an abnormal event"
   }
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/application_dev/using_gcc_for_compilation.md
---
# Using GCC for Compilation

This chapter describes the basic knowledge of GCC compilation and provides examples for demonstration. For more information about GCC, run the  **man gcc**  command.

## Overview

The GNU Compiler Collection (GCC) is a powerful and high-performance multi-platform compiler developed by GNU. The GCC compiler can compile and link source programs, assemblers, and target programs of C and C++ into executable files. By default, the GCC software package is installed in the openEuler OS.

## Basics

### File Type

For any given input file, the file type determines which compilation to perform.  [Table 1](#table634145764320)  describes the common GCC file types.

**Table  1**  Common GCC file types

### Compilation Process

Using GCC to generate executable files from source code files requires preprocessing, compilation, assembly, and linking.

1. Preprocessing: Preprocess the source program (such as a **.c** file) to generate an **.i** file.
2. Compilation: Compile the preprocessed **.i** file into an assembly language to generate an **.s** file.
3. Assemble: Assemble the assembly language file to generate the target file **.o**.
4. Linking: Link the **.o** files of each module to generate an executable program file.

The **.i**, **.s**, and **.o** files are intermediate or temporary files. If the GCC is used to compile programs in C language at a time, these files will be deleted.

### Compilation Options

GCC compilation command format:  **gcc**  \[*options*] \[*filenames*]

In the preceding information:

*options* : compilation options.

*filenames* : file name.

GCC is a powerful compiler. It has many  *options*, but most of them are not commonly used.  [Table 2](#table1342946175212)  describes the common  *options*.

**Table  2**  Common GCC compilation options

### Multi-file Compilation

There are two methods provided for compiling multiple source files.

* Multiple source files are compiled at the same time. All files need to be recompiled during compilation.

  Example: Compile  **test1.c**  and  **test2.c**  and link them to the executable file  **test**.

  ```shell
  $ gcc test1.c test2.c -o test
  ```

* Compile each source file, and then link the target files generated after compilation. During compilation, only modified files need to be recompiled.

  For example, compile  **test1.c**  and  **test2.c**, and link the target files  **test1.o**  and  **test2.o**  to the executable file  **test**.

  ```shell
  $ gcc -c test1.c
  $ gcc -c test2.c
  $ gcc test1.o test2.o -o test
  ```

## Libraries

A library is mature and reusable code that has been written for use. Each program depends on many basic underlying libraries.

The library file name is prefixed with lib and suffixed with .so (dynamic library) or .a (static library). The middle part is the user-defined library file name, for example, libfoo.so or libfoo.a. Because all library files comply with the same specifications, the  **lib**  prefix can be omitted when the  **-l**  option specifies the name of the linked library file. That is, when GCC processes  **-lfoo**, the library file  **libfoo.so**  or  **libfoo.a**  is automatically linked. When creating a library, you must specify the full file name  **libfoo.so**  or  **libfoo.a**.

Libraries are classified into static libraries and dynamic libraries based on the linking time. The static library links and packs the target file .o generated by assembly and the referenced library into an executable file in the linking phase. The dynamic library is not linked to the target code when the program is compiled, but is loaded when the program is run. The differences are as follows:

* The resource usage is different.

  The static library is a part of the generated executable file, while the dynamic library is a separate file. Therefore, the sizes and occupied disk space of the executable files of the static library and dynamic library are different, which leads to different resource usage.

* The scalability and compatibility are different.

  If the implementation of a function in the static library changes, the executable file must be recompiled. For the executable file generated by dynamic linking, only the dynamic library needs to be updated, and the executable file does not need to be recompiled.

* The dependency is different.

  The executable file of the static library can run without depending on any other contents, while the executable file of the dynamic library must depend on the dynamic library. Therefore, the static library is convenient to migrate.

* The loading speeds are different.

  Static libraries are linked together with executable files, while dynamic libraries are linked only when they are loaded or run. Therefore, for the same program, static linking is faster than dynamic linking.

### Dynamic Link Library

You can use the  **-shared**  and  **-fPIC**  options to create a dynamic link library (DLL) with the source file, assembly file, or target file. The  **-fPIC**  option is used in the compilation phase. This option is used when the target file is generated, so as to generate location-independent code.

Example 1: Generate a DLL from the source file.

```shell
$ gcc -fPIC -shared test.c -o libtest.so
```

Example 2: Generate a DLL from the target file.

```shell
$ gcc -fPIC -c test.c -o test.o
$ gcc -shared test.o -o libtest.so
```

To link a DLL to an executable file, you need to list the name of the DLL in the command line.

Example: Compile  **main.c**  and  **libtest.so**  into  **app.out**. When  **app.out**  is running, the link library  **libtest.so**  is dynamically loaded.

```shell
$ gcc main.c libtest.so -o app.out
```

In this mode, the  **libtest.so**  file in the current directory is used.

If you choose to search for a DLL, to ensure that the DLL can be linked when the program is running, you must implement by using one of the following methods:

* Save the DLL to a standard directory, for example,  **/usr/lib**.

* Add the DLL path  **libraryDIR**  to the environment variable  **LD\_LIBRARY\_PATH**.

  $ export LD\_LIBRARY\_PATH=libraryDIR:$LD\_LIBRARY\_PATH

  > \[!NOTE] **NOTE:**\
  > **LD\_LIBRARY\_PATH**  is an environment variable of the DLL. If the DLL is not in the default directories (**/lib**  and  **/usr/lib**), you need to specify the environment variable  **LD\_LIBRARY\_PATH**.

* Add the DLL path  **libraryDIR**  to  **/etc/ld.so.conf**  and run  **ldconfig**, or use the DLL path  **libraryDIR**  as a parameter to run  **ldconfig**.

```shell
$ gcc main.c -L libraryDIR -ltest -o app.out
$ export LD_LIBRARY_PATH=libraryDIR:$LD_LIBRARY_PATH
```

### Static Link Library

To create a static link library (SLL), you need to compile the source file to the target file, and then run the  **ar**  command to compress the target file into an SLL.

Example: Compile and compress source files  **test1.c**,  **test2.c**, and  **test3.c**  into an SLL.

```shell
$ gcc -c test1.c test2.c test3.c
$ ar rcs libtest.a test1.o test2.o test3.o
```

The  **ar**  command is a backup compression command. You can compress multiple files into a backup file (also called an archive file) or extract member files from the backup file. The most common use of  **ar**  is to compress the target files into an SLL.

The format of the  **ar**  command to compress the target files into an SLL is as follows:

ar rcs  *Sllfilename* *Targetfilelist*

* *Sllfilename* : Name of the static library file.
* *Targetfilelist* : Target file list.
* **r**: replaces the existing target file in the library or adds a new target file.
* **c**: creates a library regardless of whether the library exists.
* **s**: creates the index of the target file. The speed can be improved when a large library is created.

Example: Create a main.c file to use the SLL.

```shell
$ gcc main.c -L libraryDIR -ltest -o test.out
```

In the preceding command,  **libraryDIR**  indicates the path of the libtest.a library.

## Examples

### Example for Using GCC to Compile C Programs

1. Run the  **cd**  command to go to the code directory. The  **~/code**  directory is used as an example. The command is as follows:

   ```shell
   $ cd ~/code 
   ```

2. Compile the Hello World program and save it as  **helloworld.c**. The following uses the Hello World program as an example. The command is as follows:

   ```shell
   $ vi helloworld.c
   ```

   Code example:

   ```c
   #include <stdio.h>    
   int main()    
   {    
          printf("Hello World!\n");       
          return 0;    
   }
   ```

3. Run the following command to compile the code in the code directory:

   ```shell
   $ gcc helloworld.c -o helloworld
   ```

   If no error is reported, the execution is successful.

4. After the compilation is complete, the helloworld file is generated. Check the compilation result. The following is an example:

   ```shell
   $ ./helloworld
   Hello World!
   ```

### Example for Creating and Using a DLL Using GCC

1. Run the  **cd**  command to go to the code directory. The  **~/code**  directory is used as an example. Create the  **src**,  **lib**, and  **include**  subdirectories in the directory to store the source file, DLL file, and header file, respectively.

   ```shell
   $ cd ~/code
   $ mkdir src lib include
   ```

2. Run the  **cd**  command to go to the  **~/code/src**  directory and create two functions  **add.c**  and  **sub.c**  to implement addition and subtraction, respectively.

   ```shell
   $ cd ~/code/src
   $ vi add.c
   $ vi sub.c
   ```

   The following is an example of the  **add.c**  code:

   ```c
   #include "math.h"
   int add(int a, int b)
   {
           return a+b;
   }
   ```

   The following is an example of the  **sub.c**  code:

   ```c
   #include "math.h"
   int sub(int a, int b)
   {
           return a-b;
   }
   ```

3. Compile the source files add.c and sub.c into the DLL libmath.so, and store the DLL in the  **~/code/lib**  directory.

   ```shell
   $ gcc -fPIC -shared add.c sub.c -o ~/code/lib/libmath.so
   ```

4. Go to the  **~/code/include**  directory, create a header file  **math.h**, and declare the header file of the function.

   ```shell
   $ cd ~/code/include
   $ vi math.h
   ```

   The following is an example of the  **math.h**  code:

   ```c
   #ifndef __MATH_H_
   #define __MATH_H_
   int add(int a, int b);
   int sub(int a, int b);
   #endif
   ```

5. Run the  **cd**  command to go to the  **~/code/src**  directory and create a  **main.c**  function that invokes add() and sub().

   ```shell
   $ cd ~/code/src
   $ vi main.c
   ```

   The following is an example of the  **math.c**  code:

   ```c
   #include <stdio.h>
   #include "math.h"
   int main()
   {
           int a, b;
           printf("Please input a and b:\n");
           scanf("%d %d", &a, &b);
           printf("The add: %d\n", add(a,b));
           printf("The sub: %d\n", sub(a,b));
           return 0;
   }
   ```

6. Compile  **main.c**  and  **libmath.so**  into  **math.out**.

   ```shell
   $ gcc main.c -I ~/code/include -L ~/code/lib -lmath -o math.out
   ```

7. Add the path of the DLL to the environment variable.

   ```shell
   $ export LD_LIBRARY_PATH=~/code/lib:$LD_LIBRARY_PATH
   ```

8. Run the following command to execute  **math.out**:

   ```shell
   $ ./math.out
   ```

   The command output is as follows:

   ```text
   Please input a and b:
   9 2
   The add: 11
   The sub: 7
   ```

### Example for Creating and Using an SLL Using GCC

1. Run the  **cd**  command to go to the code directory. The  **~/code**  directory is used as an example. Create the  **src**,  **lib**, and  **include**  subdirectories in the directory to store the source file, SLL file, and header file respectively.

   ```shell
   $ cd ~/code
   $ mkdir src lib include
   ```

2. Run the  **cd**  command to go to the  **~/code/src**  directory and create two functions  **add.c**  and  **sub.c**  to implement addition and subtraction, respectively.

   ```shell
   $ cd ~/code/src
   $ vi add.c
   $ vi sub.c
   ```

   The following is an example of the  **add.c**  code:

   ```c
   #include "math.h"
   int add(int a, int b)
   {
           return a+b;
   }
   ```

   The following is an example of the  **sub.c**  code:

   ```c
   #include "math.h"
   int sub(int a, int b)
   {
           return a-b;
   }
   ```

3. Compile the source files  **add.c**  and  **sub.c**  into the target files  **add.o**  and  **sub.o**.

   ```shell
   $ gcc -c add.c sub.c
   ```

4. Run the  **ar**  command to compress the  **add.o**  and  **sub.o**  target files into the SLL  **libmath.a**  and save the SLL to the  **~/code/lib**  directory.

   ```shell
   $ ar rcs ~/code/lib/libmath.a add.o sub.o
   ```

5. Go to the  **~/code/include**  directory, create a header file  **math.h**, and declare the header file of the function.

   ```shell
   $ cd ~/code/include
   $ vi math.h
   ```

   The following is an example of the  **math.h**  code:

   ```c
   #ifndef __MATH_H_
   #define __MATH_H_
   int add(int a, int b);
   int sub(int a, int b);
   #endif
   ```

6. Run the  **cd**  command to go to the  **~/code/src**  directory and create a  **main.c**  function that invokes add() and sub().

   ```shell
   $ cd ~/code/src
   $ vi main.c
   ```

   The following is an example of the  **math.c**  code:

   ```c
   #include <stdio.h>
   #include "math.h"
   int main()
   {
           int a, b;
           printf("Please input a and b:\n");
           scanf("%d %d", &a, &b);
           printf("The add: %d\n", add(a,b));
           printf("The sub: %d\n", sub(a,b));
           return 0;
   }
   ```

7. Compile  **main.c**  and  **libmath.a**  into  **math.out**.

   ```shell
   $ gcc main.c -I ~/code/include -L ~/code/lib -lmath -o math.out
   ```

8. Run the following command to execute  **math.out**:

   ```shell
   $ ./math.out
   ```

   The command output is as follows:

   ```text
   Please input a and b:
   9 2
   The add: 11
   The sub: 7
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/application_dev/using_jdk_for_compilation.md
---
# Using JDK for Compilation

## Overview

A Java Development Kit (JDK) is a software package required for Java development. It contains the Java Runtime Environment (JRE) and compilation and commissioning tools. On the basis of OpenJDK, openEuler optimizes GC, enhances concurrency stability, and enhances security, improving the performance and stability of Java applications on ARM.

## Basics

### File Type and Tool

For any given input file, the file type determines which tool to use for processing. The common file types and tools are described in  [Table 1](#table634145764320)  and  [Table 2](#table103504146433).

**Table  1**  Common JDK file types

**Table  2**  Common JDK tools

### Java Program Generation Process

To generate a program from Java source code files and run the program using Java, compilation and run are required.

1. Compilation: Use the Java compiler (javac) to compile Java source code files (.java files) into .class bytecode files.
2. Run: Execute the bytecode files on the Java virtual machine (JVM).

### Common JDK Options

#### Javac Compilation Options

The command format for javac compilation is as follows:  **javac**  \[*options*] \[*sourcefiles*] \[*classes*] \[@*argfiles*]

In the preceding information:

*options*: command options.

*sourcefiles*: one or more source files to be compiled.

*classes*: one or more classes to be processed as comments.

@*argfiles*: one or more files that list options and source files. The  **-J**  option is not allowed in these files.

Javac is a Java compiler. It has many  *options*, but most of them are not commonly used.  [Table 3](#table1342946175212)  describes the common options values.

**Table  3**  Common javac options

#### Java Running Options

The Java running format is as follows:

Running class file:  **java**  \[*options*]  *classesname*  \[args]

Running Java file:  **java**  \[*options*] -jar  *filename*  \[args]

In the preceding information:

*options*: command options, which are separated by spaces.

*classname*: name of the running .class file.

*filename*: name of the running .jar file.

args: parameters transferred to the main() function. The parameters are separated by spaces.

Java is a tool for running Java applications. It has many  *options*, but most of them are not commonly used.  [Table 4](#table371918587238)  describes the common options.

**Table  4**  Common Java running options

#### JAR Options

The JAR command format is as follows:  **jar**  {c | t | x | u}\[vfm0M] \[*jarfile*] \[*manifest*] \[-C  *dir*]  *file*...

[Table 5](#table3691718114817)  describes the parameters in the  **jar**  command.

**Table  5**  JAR parameter description

## Class Library

The Java class library is implemented as a package, which is a collection of classes and interfaces. The Java compiler generates a bytecode file for each class, and the file name is the same as the class name. Therefore, conflicts may occur between classes with the same name. In the Java language, a group of classes and interfaces are encapsulated in a package. Class namespaces can be effectively managed by package. Classes in different packages do not conflict even if they have the same name. This solves the problem of conflicts between classes with the same name and facilitates the management of a large number of classes and interfaces. It also ensures the security of classes and interfaces.

In addition to many packages provided by Java, developers can customize packages by collecting compiled classes and interfaces into a package for future use.

Before using a custom package, you need to declare the package.

### Package Declaration

The declaration format of a package is package pkg1\[.pkg2\[.pkg3...]].

To declare a package, you must create a directory. The subdirectory name must be the same as the package name. Then declare the package at the beginning of the class file that needs to be placed in the package, indicating that all classes of the file belong to the package. The dot (.) in the package declaration indicates the directory hierarchy. If the source program file does not contain the package statement, the package is specified as an anonymous package. An anonymous package does not have a path. Generally, Java still stores the classes in the source file in the current working directory (that is, the directory where the Java source files are stored).

The package declaration statement must be added to the beginning of the source program file and cannot be preceded by comments or spaces. If you use the same package declaration statement in different source program files, you can include the classes in different source program files in the same package.

### Package Reference

In Java, there are two methods to use the common classes in the package provided by Java or the classes in the custom package.

* Add the package name before the name of the class to be referenced.

  For example, name.A obj=new name.A ()

  **name**  indicates the package name,  **A**  indicates the class name, and  **obj**  indicates the object. This string indicates that class  **A**  in the  **name**  package is used to define an object  **obj**  in the program.

  Example: Create a test object of the Test class in the example package.

  ```java
  example.Test test = new example.Test();
  ```

* Use  **import**  at the beginning of the file to import the classes in the package.

  The format of the  **import**  statement is import pkg1\[.pkg2\[.pkg3...]].(classname | \*).

  **pkg1\[.pkg2\[.pkg3...]]**  indicates the package level, and  **classname**  indicates the class to be imported. If you want to import multiple classes from a package, you can use the wildcard (\*) instead.

  Example: Import the  **Test**  class in the  **example**  package.

  ```java
  import example.Test;
  ```

  Example: Import the entire  **example**  package.

  ```java
  import example.*;
  ```

## Examples

### Compiling a Java Program Without a Package

1. Run the  **cd**  command to go to the code directory. The  **~/code**  directory is used as an example. The command is as follows:

   ```shell
   cd ~/code 
   ```

2. Compile the Hello World program and save it as  **HelloWorld.java**. The following uses the Hello World program as an example. The command is as follows:

   ```shell
   vi HelloWorld.java
   ```

   Code example:

   ```java
   public class HelloWorld {     
        public static void main(String[] args) {         
            System.out.println("Hello World");     
         } 
   }
   ```

3. Run the following command to compile the code in the code directory:

   ```shell
   javac HelloWorld.java 
   ```

   If no error is reported, the execution is successful.

4. After the compilation is complete, the HelloWorld.class file is generated. You can run the  **java**  command to view the result. The following is an example:

   ```shell
   $ java HelloWorld
   Hello World
   ```

### Compiling a Java Program with a Package

1. Run the  **cd**  command to go to the code directory. The  **~/code**  directory is used as an example. Create the  **~/code/Test/my/example**,  **~/code/Hello/world/developers**, and  **~/code/Hi/openos/openeuler**  subdirectories in the directory to store source files.

   ```shell
   cd ~/code
   mkdir -p Test/my/example
   mkdir -p Hello/world/developers
   mkdir -p Hi/openos/openeuler
   ```

2. Run the  **cd**  command to go to the  **~/code/Test/my/example**  directory and create  **Test.java**.

   ```shell
   cd ~/code/Test/my/example
   vi Test.java
   ```

   The following is an example of the Test.java code:

   ```java
   package my.example;
   import world.developers.Hello;
   import openos.openeuler.Hi;
   public class Test {
     public static void main(String[] args) {
      Hello me = new Hello();
      me.hello();
      Hi you = new Hi();
      you.hi();
     }
   }
   ```

3. Run the  **cd**  command to go to the  **~/code/Hello/world/developers**  directory and create  **Hello.java**.

   ```shell
   cd ~/code/Hello/world/developers
   vi Hello.java
   ```

   The following is an example of the Hello.java code:

   ```java
   package world.developers;
   public class Hello {
     public void hello(){
      System.out.println("Hello, openEuler.");
     }
   }
   ```

4. Run the  **cd**  command to go to the  **~/code/Hi/openos/openeuler**  directory and create  **Hi.java**.

   ```shell
   cd ~/code/Hi/openos/openeuler
   vi Hi.java
   ```

   The following is an example of the Hi.java code:

   ```java
   package openos.openeuler;
   public class Hi {
     public void hi(){
      System.out.println("Hi, the global developers.");
     }
   }
   ```

5. Run the  **cd**  command to go to the **~/code**  directory and use javac to compile the source file.

   ```shell
   cd ~/code
   javac -classpath Hello:Hi Test/my/example/Test.java
   ```

   After the command is executed, the  **Test.class**,  **Hello.class**, and  **Hi.class**  files are generated in the  **~/code/Test/my/example**,  **~/code/Hello/world/developers**, and  **~/code/Hi/openos/openeuler**  directories.

6. Run the  **cd**  command to go to the  **~/code**  directory and run the  **Test**  program using Java.

   ```shell
   cd ~/code
   java -classpath Test:Hello:Hi my/example/Test
   ```

   The command output is as follows:

   ```console
   Hello, openEuler.
   Hi, the global developers.
   ```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/installation_upgrade/installation/using_kickstart_for_automatic_installation.md
---
# Using Kickstart for Automatic Installation

## Introduction

### Overview

You can use the kickstart tool to automatically install the openEuler OS in either of the following ways:

* Semi-automatic installation: You only need to specify the location of the kickstart file. Kickstart automatically configures OS attributes such as keyboard, language, and partitions.
* Automatic installation: The OS is automatically installed.

### Advantages and Disadvantages

[Table 1](#table1388812373315)  lists the advantages and disadvantages of semi-automatic installation and full-automatic installation using kickstart. You can select an installation mode as required.

**Table  1**  Advantages and disadvantages

### Background

#### Kickstart

Kickstart is an unattended installation mode. The principle of kickstart is to record typical parameters that need to be manually entered during the installation and generate the configuration file **ks.cfg**. During the installation, the installation program searches the **ks.cfg** configuration file first for required parameters. If no matching parameters are found, you need to manually configure these parameters. If all required parameters are covered by the kickstart file, automatic installation can be achieved by only specifying the path of the kickstart file.

Both full-automatic or semi-automatic installation can be achieved by kickstart.

kickstart uses the open source software [Pykickstart](https://github.com/pykickstart/pykickstart) to parse **ks.cfg** files.

#### PXE

Pre-boot Execution Environment (PXE) works in client/server network mode. The PXE client can obtain an IP address from the DHCP server during the startup and implement client boot and installation through the network based on protocols such as trivial file transfer protocol (TFTP).

#### TFTP

TFTP is used to transfer simple and trivial files between clients and the server.

## Semi-automatic Installation Guide

### Environment Requirements

The environment requirements for semi-automatic installation of openEuler OS using kickstart are as follows:

* PM/VM (For details about how to create VMs, see the documents from corresponding vendors): includes the computer where kickstart is used for automatic installation and the computer where the kickstart tool is installed.
* httpd: deploys the kickstart file and system installation file.
* ISO: openEuler-*{version}-{architecture}*-dvd.iso. In this document, **openEuler-22.03-LTS-SP4-aarch64-dvd.iso** is used as an example.

### Procedure

To use kickstart to perform semi-automatic installation of openEuler, perform the following steps:

#### Environment Preparation

> \[!NOTE] **NOTE:**
> Before the installation, ensure that the firewall of the HTTP server is disabled. Run the following command to disable the firewall:
>
> ```shell
> iptables -F  
> ```

1. Install httpd and start the service.

   ```shell
   dnf install httpd -y
   systemctl start httpd
   systemctl enable httpd
   ```

2. Run the following commands to prepare the kickstart file:

   ```shell
   $ mkdir /var/www/html/ks
   $ vim /var/www/html/ks/openEuler-ks.cfg # The file can be obtained by modifying the  **anaconda-ks.cfg**  file automatically generated from openEuler OS.
   ====================================
   ***Modify the following information as required.***
   #version=DEVEL
   ignoredisk --only-use=sda
   autopart --type=lvm
   # Partition clearing information
   clearpart --none --initlabel
   # Use graphical install
   graphical
   # Use CDROM installation media
   cdrom
   # Keyboard layouts
   keyboard --vckeymap=cn --xlayouts='cn'
   # System language
   lang zh_CN.UTF-8

   # Network information
   network  --bootproto=dhcp --device=enp4s0 --ipv6=auto --activate
   network  --hostname=openeuler.com
   # Root password
   rootpw --iscrypted $6$fQE83lxEZ48Or4zc$j7/PlUMHn29yTjCD4Fi44WTZL/RzVGxJ/7MGsZMl6QfE3KjIVT7M4UrhFXbafvRq2lUddAFcyWHd5WRmXfEK20
   # Run the Setup Agent on first boot
   firstboot --enable
   # Do not configure the X Window System
   skipx
   # System services
   services --disabled="chronyd"
   # System timezone
   timezone Asia/Shanghai --isUtc--nontp

   %packages
   @^minimal-environment
   @standard

   %end

   %post
   #enable kdump
   sed  -i "s/ ro / ro crashkernel=1024M,high /" /boot/efi/EFI/openEuler/grub.cfg
   %end
   =====================================
   ```

   > \[!NOTE] **NOTE:**
   > The method of generating the password ciphertext is as follows:
   >
   > ```py
   > # python3  
   > Python 3.7.0 (default, Apr  1 2019, 00:00:00)  
   > [GCC 7.3.0] on linux  
   > Type "help", "copyright", "credits" or "license" for more information.  
   > >>> import crypt  
   > >>> passwd = crypt.crypt("myPasswd")  
   > >>> print (passwd)  
   > $6$63c4tDmQGn5SDayV$mZoZC4pa9Jdt6/ALgaaDq6mIExiOO2EjzomB.Rf6V1BkEMJDcMddZeGdp17cMyc9l9ML9ldthytBEPVcnboR/0  
   > ```

3. Mount the ISO image file to the CD-ROM drive of the computer where openEuler is to be installed.

   If you want to install openEuler through the NFS, specify the path (which is  **cdrom**  by default) of installation source in the kickstart file.

#### Installing the System

1. The installation selection dialog box is displayed.
   1. On the installation wizard page in [Starting the Installation](./../installation/installation_guide.md#starting-the-installation), select  **Install openEuler 22.03-LTS-SP4**  and press  **e**.

   2. Add  \*\*inst.ks=<http://server_ip/ks/openEuler-ks.cfg**>  to the startup parameters.

      ![startparam.gif](./figures/startparam.gif)

   3. Press  **Ctrl**+**x**  to start the automatic installation.

2. Verify that the installation is complete.

   After the installation is complete, the system automatically reboots. If the first boot option of the system is set to the CD\_ROM, the installation page is displayed again. Shut down the computer and change startup option to start from the hard disk preferentially.

   ![](./figures/completing-the-automatic-installation.png)

## Full-automatic Installation Guide

### Environment Requirements

The environment requirements for full-automatic installation of openEuler using kickstart are as follows:

* PM/VM (For details about how to create VMs, see the documents from corresponding vendors): includes the computer where kickstart is used for automatic installation and the computer where the kickstart tool is installed.
* httpd: stores the kickstart file.
* TFTP: provides vmlinuz and initrd files.
* DHCPD/PXE: provides the DHCP service.
* ISO: openEuler-*{version}-{architecture}*-dvd.iso. In this document, **openEuler-22.03-LTS-SP4-aarch64-dvd.iso** is used as an example.

### Procedure

To use kickstart to perform full-automatic installation of openEuler, perform the following steps:

#### Environment Preparation

> \[!NOTE] **NOTE:**
> Before the installation, ensure that the firewall of the HTTP server is disabled. Run the following command to disable the firewall:
>
> ```shell
> iptables -F  
> ```

1. Install httpd and start the service.

   ```shell
   dnf install httpd -y
   systemctl start httpd
   systemctl enable httpd
   ```

2. Install and configure TFTP.

   ```shell
   $ dnf install tftp-server xinetd -y
   $ vim /etc/xinetd.d/tftp
   service tftp
   {
   socket_type = dgram
   protocol = udp
   wait = yes
   user = root
   server = /usr/sbin/in.tftpd
   server_args = -s /var/lib/tftpboot
   disable = no
   per_source = 11
   cps = 100 2
   flags = IPv4
   }
   $ systemctl start tftp
   $ systemctl enable tftp
   $ systemctl start xinetd
   $ systemctl status xinetd
   $ systemctl enable xinetd
   ```

3. Prepare the installation source.

   ```shell
   mount openEuler-22.03-LTS-SP4-aarch64-dvd.iso /mnt
   cp -r /mnt/* /var/www/html/openEuler/
   ```

4. Set and modify the kickstart configuration file  **openEuler-ks.cfg**. Select the HTTP installation source by referring to  [3](#en-us_topic_0229291289_l1692f6b9284e493683ffa2ef804bc7ca).

   ```shell
   $ vim  /var/www/html/ks/openEuler-ks.cfg
   ====================================
   ***Modify the following information as required.***
   #version=DEVEL
   ignoredisk --only-use=sda
   autopart --type=lvm
   # Partition clearing information
   clearpart --none --initlabel
   # Use graphical install
   graphical
   # Keyboard layouts
   keyboard --vckeymap=cn --xlayouts='cn'
   # System language
   lang zh_CN.UTF-8
   #Use http installation source
   url  --url=//192.168.122.1/openEuler/
   %post
   #enable kdump
   sed  -i "s/ ro / ro crashkernel=1024M,high /" /boot/efi/EFI/openEuler/grub.cfg
   %end
   ...
   ```

5. Modify the PXE configuration file  **grub.cfg**  as follows. (Note: Currently, openEuler does not support the cfg file in bls format. If the x86\_64 architecture is used, replace **grubaa64.efi** with **grubx64.efi**.)

   ```shell
   $ cp -r /mnt/images/pxeboot/* /var/lib/tftpboot/
   $ cp /mnt/EFI/BOOT/grubaa64.efi /var/lib/tftpboot/
   $ cp /mnt/EFI/BOOT/grub.cfg /var/lib/tftpboot/
   $ ls /var/lib/tftpboot/
   grubaa64.efi  grub.cfg  initrd.img  TRANS.TBL  vmlinuz
   $ vim /var/lib/tftpboot/grub.cfg
   set default="1"

   function load_video {
     if [ x$feature_all_video_module = xy ]; then
       insmod all_video
     else
       insmod efi_gop
       insmod efi_uga
       insmod ieee1275_fb
       insmod vbe
       insmod vga
       insmod video_bochs
       insmod video_cirrus
     fi
   }

   load_video
   set gfxpayload=keep
   insmod gzio
   insmod part_gpt
   insmod ext2

   set timeout=60

   ### BEGIN /etc/grub.d/10_linux ###
   menuentry 'Install openEuler 22.03-LTS-SP4' --class red --class gnu-linux --class gnu --class os {
           set root=(tftp,192.168.122.1)
           linux /vmlinuz ro inst.geoloc=0 console=ttyAMA0 console=tty0 rd.iscsi.waitnet=0 inst.ks=http://192.168.122.1/ks/openEuler-ks.cfg
           initrd /initrd.img
   }
   ```

6. Configure DHCP, which can be replaced by DNSmasq. If the x86\_64 architecture is used, replace **grubaa64.efi** with **grubx64.efi**.

   ```shell
   $ dnf install dhcp -y
   $ vim /etc/dhcp/dhcpd.conf
   # DHCP Server Configuration file.
   #   see /usr/share/doc/dhcp-server/dhcpd.conf.example
   #   see dhcpd.conf(5) man page
   ddns-update-style interim;
   ignore client-updates;
   filename "grubaa64.efi";  # location of the pxelinux startup file;
   next-server 192.168.122.1;     # (IMPORTANT) IP address of the TFTP server;
   subnet 192.168.122.0 netmask 255.255.255.0 {
   option routers 192.168.111.1; # Gateway address
   option subnet-mask 255.255.255.0; # Subnet mask
   range dynamic-bootp 192.168.122.50 192.168.122.200; # Dynamic IP address range
   default-lease-time 21600;
   max-lease-time 43200;
   }
   $ systemctl start dhcpd
   $ systemctl enable dhcpd
   ```

#### Installing the System

1. On the  **Start boot option**  screen, press  **F2**  to boot from the PXE and start automatic installation.

   ![](./figures/en-us_image_0229291270.png)

   ![](./figures/en-us_image_0229291286.png)

   ![](./figures/en-us_image_0229291247.png)

2. The automatic installation window is displayed.

3. Verify that the installation is complete.

   ![](./figures/completing-the-automatic-installation.png)

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/development/application_dev/using_make_for_compilation.md
---
# Using Make for Compilation

This chapter describes the basic knowledge of make compilation and provides examples for demonstration. For more options of `make`, run the  `make --help`  command, or see the [GNU official document](https://www.gnu.org/software/make/manual/make.html#Overview).

## Overview

The GNU make utility (usually abbreviated as make) is a tool for controlling the generation of executable files from source files. make automatically identifies which parts of the complex program have changed and need to be recompiled. Make uses a configuration file called makefile to control how the program is built.

## Basics

### File Type

[Table 1](#table634145764320)  describes the file types that may be used in the makefile file.

**Table  1**  File types

### make Work Process

The process of deploying make to generate an executable file from the source code file is described as follows:

1. The make command reads the makefiles, including the files named GNUmakefile, makefile, and Makefile in the current directory, the included makefile, and the rule files specified by the  **-f**,  **--file**, and  **--makefile**  options.
2. Initialize variables.
3. Derive implicit rules, analyze dependencies, and create a dependency chain.
4. Determine which targets need to be regenerated based on the dependency chain.
5. Run a command to generate the final file.

### make Options

make command format:  **make**  \[*option*]... \[*target*]...

In the preceding command:

*option* : parameter option.

*target* : target specified in Makefile.

[Table 2](#table261872312343)  describes the common make options.

**Table  2**  Common make options

## Makefiles

Make is a tool that uses makefiles for compilation, linking, installation, and cleanup, so as to generate executable files and other related files from source code files. Therefore, makefiles describe the compilation and linking rules of the entire project, including which files need to be compiled, which files do not need to be compiled, which files need to be compiled first, which files need to be compiled later, and which files need to be rebuilt. The makefiles automate project compilation. You do not need to manually enter a large number of source files and parameters each time.

This chapter describes the structure and main contents of makefiles. For more information about makefiles, run the  **info make**  command.

### Makefile Structure

The makefile file structure is as follows:

*targets*:*prerequisites*

*command*

or

*targets*:*prerequisites*;*command*

*command*

In the preceding information:

* *targets* : targets, which can be target files, executable files, or tags.
* *prerequisites* : dependency files, which are the files or targets required for generating the  *targets*. There can be multiple or none of them.
* *command* : command (any shell command) to be executed by make. Multiple commands are allowed, and each command occupies a line.
* Use colons (:) to separate the target files from the dependency files. Press  **Tab**  at the beginning of each command line.

The makefile file structure indicates the output target, the object on which the output target depends, and the command to be executed for generating the target.

### Makefile Contents

A makefile file consists of the following contents:

* Explicit rule

  Specify the dependency, such as the file to be generated, dependency file, and generated command.

* Implicit rule

  Specify the rule that is automatically derived by make. The make command supports the automatic derivation function.

* Variable definition

* File indicator

  The file indicator consists of three parts:

  * Inclusion of other makefiles, for example, include xx.md
  * Selective execution, for example, #ifdef
  * Definition of multiple command lines, for example, define...endef. (define ... endef)

* Comment

  The comment starts with a number sign (#).

## Examples

### Example of Using Makefile to Implement Compilation

1. Run the  **cd**  command to go to the code directory. The  **~/code**  directory is used as an example.

   ```shell
   cd ~/code
   ```

2. Create a header file  **hello.h**  and two functions  **hello.c**  and  **main.c**.

   ```shell
   vi hello.h
   vi hello.c
   vi main.c
   ```

   The following is an example of the  **hello.h**  code:

   ```c
   #pragma once
   #include <stdio.h>
   void hello();
   ```

   The following is an example of the  **hello.c**  code:

   ```c
   #include "hello.h"
   void hello()
   {
           int i=1;
           while(i<5)
           {
                   printf("The %dth say hello.\n", i);
                   i++;
           }
   }
   ```

   The following is an example of the  **main.c**  code:

   ```c
   #include "hello.h"
   #include <stdio.h>
   int main()
   {
           hello();
           return 0;
   }
   ```

3. Create the makefile.

   ```shell
   vi Makefile
   ```

   The following provides an example of the makefile content:

   ```text
   main:main.o hello.o
           gcc -o main main.o hello.o
   main.o:main.c
           gcc -c main.c
   hello.o:hello.c
           gcc -c hello.c
   clean:
           rm -f hello.o main.o main
   ```

4. Run the  **make**  command.

   ```shell
   make
   ```

   After the command is executed, the commands executed in makefile are printed. If you do not need to print the information, add the  **-s**  option to the  **make**  command.

   ```shell
   gcc -c main.c
   gcc -c hello.c
   gcc -o main main.o hello.o
   ```

5. Execute the ./main target.

   ```shell
   ./main
   ```

   After the command is executed, the following information is displayed:

   The 1th say hello.

   The 2th say hello.

   The 3th say hello.

   The 4th say hello.

---

---
url: /en/docs/22.03_LTS_SP4/server/maintenance/syscare/using_syscare.md
---
# Using SysCare

This chapter describes how to use SysCare on openEuler, including patch creation, application, activation, deactivation, acceptation, uninstallation, save and restore, conflict detection, overwriting, and query.

## Prerequisites

openEuler 22.03 LTS SP4 has been installed.

## Live Patch Creation

Run `syscare build` to create kernel- and user-mode live patches from RPM packages. Patches are encapsulated into RPM packages.

### Command Parameters

```shell
USAGE:
    syscare build [OPTIONS] --patch-name <PATCH_NAME> --source <SOURCE>... --debuginfo <DEBUGINFO>... --patch <PATCH>...

OPTIONS:
    -n, --patch-name <PATCH_NAME>                  Patch name
        --patch-arch <PATCH_ARCH>                  Patch architecture [default: aarch64]
        --patch-version <PATCH_VERSION>            Patch version [default: 1]
        --patch-release <PATCH_RELEASE>            Patch release [default: 1]
        --patch-description <PATCH_DESCRIPTION>    Patch description [default: (none)]
        --patch-requires <PATCH_REQUIRES>...       Patch requirements
    -s, --source <SOURCE>...                       Source package(s)
    -d, --debuginfo <DEBUGINFO>...                 Debuginfo package(s)
    -p, --patch <PATCH>...                         Patch file(s)
        --work-dir <WORK_DIR>                      Working directory [default: /var/run/syscare]
        --build-root <BUILD_ROOT>                  Build temporary directory [default: .]
    -o, --output <OUTPUT>                          Output directory [default: .]
    -j, --jobs <JOBS>                              Parallel build jobs [default: 64]
        --skip-compiler-check                      Skip compiler version check (not recommended)
        --skip-cleanup                             Skip post-build cleanup
    -v, --verbose                                  Provide more detailed info
    -h, --help                                     Print help information
    -V, --version                                  Print version information
```

### Command Options

|Name|Description|Type|Note|
| ---- | ---- | ---- | ---- |
|-n, --patch-name *\<PATCH\_NAME>*|Patch name|String|Mandatory. The value must comply with the RPM package naming convention.|
|--patch-arch *\<PATCH\_ARCH>*|Patch architecture|String|The default value is the current architectures. The value must comply with the RPM package naming convention.|
|--patch-version *\<PATCH\_VERSION>*|Patch version|String|The default value is **1**. The value must comply with the RPM package naming convention.|
|--patch-release *\<PATCH\_RELEASE>*|Patch release|Integer|The default value is **1**. The value must comply with the RPM package naming convention.|
|--patch-description *\<PATCH\_DESCRIPTION>*|Patch description|String|The default value is **none**.|
|--patch-requires *\<PATCH\_REQUIRES>*|Additional patch dependencies|String|The default value is empty. The name must meed RPM specifications.|
|-s, --source *\<SOURCE>*|Target software **src.rpm** package path|String|Mandatory. The value must be a valid path.|
|-d, --debuginfo *\<DEBUGINFO>*|Target software **debuginfo** package path|String|Mandatory. The value must be a valid path.|
|-p, --patch *\<PATCH>*|Patch file|String|This option is mandatory. The value must be a valid path.|
|--workdir *\<WORKDIR>*|Temporary directory|String|The default value is **/var/run/syscare**. The value must be a valid path.|
|--build-root *\<BUILD\_ROOT>*|Temporary build path|String|The default value is the current directory. The value must be a valid path.|
|-o, --output *\<OUTPUT>*|Patch output directory|String|The default value is the current directory. The value must be a valid path.|
|-j, --jobs *\<N>*|Number of parallel compilation jobs|Integer|The default value is the number of CPU threads|
|--skip-compiler-check|Skip compiler check|Flag|-|
|--skip-cleanup|Skip temporary file cleanup|Flag|-|
|-v, --verbose|Print detail information|Flag|-|
|-h, --help|Print help information|Flag|-|
|-V, --version|Print version information|Flag|-|

An example command is as follows:

```shell
syscare build \
    --patch-name "HP001" \
    --patch-description "CVE-2021-32675 - When parsing an incoming Redis Standard Protocol (RESP) request, Redis allocates memory according to user-specified values which determine the number of elements (in the multi-bulk header) and size of each element (in the bulk header). An attacker delivering specially crafted requests over multiple connections can cause the server to allocate significant amount of memory. Because the same parsing mechanism is used to handle authentication requests, this vulnerability can also be exploited by unauthenticated users." \
    --source ./redis-6.2.5-1.src.rpm \
    --debuginfo ./redis-debuginfo-6.2.5-1.x86_64.rpm \
    --output ./output \
```

### Live Patch Making Process

1. Prepare the source package (source RPM) and debugging information package (debuginfo RPM) of the target software.

   Example:

   ```shell

   yumdownloader kernel --source

   yumdownloader kernel --debuginfo
   ```

2. Ensure that the related software build dependencies are installed.

   Example:

   ```shell
   dnf install make gcc bison flex openssl-devel dwarves python3-devel elfutils-libelf-devel
   ```

3. Run the `syscare-build` command.

   Example:

   ```shell
   syscare build \
           --patch-name HP001 \
           --source kernel-5.10.0-60.66.0.91.oe2203.src.rpm \
           --debuginfo kernel-debuginfo-5.10.0-60.66.0.91.oe2203.x86_64.rpm \
           --output output \
           --patch 001-kernel-patch-test.patch
   ```

   During patch making, a temporary folder whose name starts with **syscare-build** is created in the directory specified by `--workdir` (the current directory by default) to store temporary files and build logs.

   Example:

   ```shell
   $ ls -l syscare-build.111602/
   total 100
   -rw-r--r--. 1 dev dev 92303 Nov 12 00:00 build.log
   drwxr-xr-x. 6 dev dev 4096 Nov 12 00:00 package
   drwxr-xr-x. 4 dev dev 4096 Nov 12 00:00 patch
   ```

   Build logs (**build.log**) are generated in the temporary folder.

   ```shell
   $ cat syscare-build.111602/build.log | less
   ...
   ```

   If the patch is created successfully and `--skip-compiler-check` is not specified, the temporary folder will be deleted after patch making.

4. Check the build result.

   Example:

   ```shell
   $ ls -l
   total 189680
   -rw-r--r--. 1 dev dev 194218767 Nov 12 00:00 kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.src.rpm
   -rw-r--r--. 1 dev dev     10937 Nov 12 00:00 patch-kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.rpm
   ```

   In the output:

   **patch-kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86\_64.rpm** is the live patch package.

   **kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86\_64.src.rpm** is the live patch source package.

5. Install the patch.

   ```shell
   dnf install patch-xxx.rpm
   ```

   After the patch is installed, files in the patch are stored in the **/usr/lib/syscare/patches/target\_software\_package\_name/patch\_name** directory

6. Uninstall the patch.

   ```shell
   dnf remove patch-xxx
   ```

   The patch package will be uninstalled when the patch is in the **ACTIVED** or **ACCEPTED** state.

### Patch Output

Two RPM packages are generated:

* A live patch package that contains the binary file of the live patch and meta information. This package is used to install the live patch.

* A live patch source package that contains the target software source code and the new patch. This package is used to create live patches for new versions.

Naming rules:

* Live patch package: patch-*TARGET\_SOFTWARE\_FULL\_NAME*-*PATCH\_NAME*-*PATCH\_VERSION*-*PATCH\_RELEASE*.*ARCHITECTURE*.rpm
* Live patch source code package: *TARGET\_SOFTWARE\_FULL\_NAME*-*PATCH\_NAME*-*PATCH\_VERSION*-*PATCH\_RELEASE*.*ARCHITECTURE*.src.rpm

### Error Handling

If an error occurs, see the build logs:

Error output example:

```text
...
Building patch, this may take a while
ERROR: Process '/usr/libexec/syscare/upatch-build' exited unsuccessfully, exit_code=255
```

## Live Patch Management

Run the `syscare` command to manage patches. SysCare searches for the patch that matches the input description and performs operations as instructed.
The search pattern is **package\_name/patch\_name**, where **package\_name** can be omitted if **patch\_name** is unique. UUID can also be used.

### Querying Live Patches

Run `syscare list` to query all installed live patches.

Example:

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
d81bce18-04bd-499f-91e9-8b9d7b94a76b     glibc-2.34-112.oe2203/HP001-1-1/libc.so.6                    NOT-APPLIED
3e7cb90d-9a4c-4fdf-a389-29d5e863f4b0     kernel-5.10.0-153.12.0.92.oe2203sp2/ACC-1-1/vmlinux          NOT-APPLIED
64fa88bd-def5-4994-85de-a7903c526109     kernel-5.10.0-60.91.0.115.oe2203/HP-50801-1-1/vmlinux        NOT-APPLIED
78268c3b-39a2-4d5c-ae96-206d8c62977a     kernel-5.10.0-60.91.0.115.oe2203/HP-50802-1-1/vmlinux        NOT-APPLIED
c568f31d-acfb-4fdb-8d2c-bde3facab5a2     kernel-5.10.0-60.91.0.115.oe2203/HP001-1-1/vmlinux           NOT-APPLIED
35b5ece6-8b67-407e-93fe-d576a78ab499     nginx-1.21.5-4/HP001-1-1/nginx                               NOT-APPLIED
074734fc-034f-4e40-b943-6a76d766939b     openssl-libs-1.1.1m-22.oe2203/HP001-1-1/libcrypto.so.1.1.1m  NOT-APPLIED
ae124f00-206a-4385-a341-c7b2f7e19482     qemu-7.0.0-2/HP001-1-1/qemu-system-aarch64                   NOT-APPLIED
11316483-dc62-4caf-bd5d-c51801dcb032     qemu-7.0.0-2/HP001-1-1/qemu-system-arm                       NOT-APPLIED
b382ea35-6713-4cfc-bb72-038feefb8173     qemu-7.0.0-2/HP001-1-1/qemu-system-i386                      NOT-APPLIED
6aaec566-a220-4b60-8020-8077b6adc6a6     qemu-7.0.0-2/HP001-1-1/qemu-system-mips                      NOT-APPLIED
2bc0158e-fc42-4ea1-8f5c-e6891d10098b     qemu-7.0.0-2/HP001-1-1/qemu-system-mips64                    NOT-APPLIED
17e00bf0-b389-46d3-a036-933aeb41e0cb     qemu-7.0.0-2/HP001-1-1/qemu-system-mips64el                  NOT-APPLIED
8481a911-d80b-4099-b9a2-a4d3c63de06d     qemu-7.0.0-2/HP001-1-1/qemu-system-mipsel                    NOT-APPLIED
d8305d00-6f45-4c38-b7a4-844b4a667d89     qemu-7.0.0-2/HP001-1-1/qemu-system-ppc                       NOT-APPLIED
d10dc5f8-1692-4da4-8908-d2075c47d62b     qemu-7.0.0-2/HP001-1-1/qemu-system-ppc64                     NOT-APPLIED
77dbfd01-dee4-405b-930f-9711a0ad43c4     qemu-7.0.0-2/HP001-1-1/qemu-system-x86_64                    NOT-APPLIED
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      NOT-APPLIED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            NOT-APPLIED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         NOT-APPLIED
789f0052-b932-4d9d-961d-7003bece1a3a     redis-6.2.5-1/HP002-1-1/redis-benchmark                      NOT-APPLIED
e4aee980-1596-43d9-be9a-07fc6f668970     redis-6.2.5-1/HP002-1-1/redis-cli                            NOT-APPLIED
feb13c9a-02b3-4109-a2f3-c3e9fe41e9ad     redis-6.2.5-1/HP002-1-1/redis-server                         NOT-APPLIED
```

### Querying Live Patch Metadata

Run `syscare info` to query the metadata of one or more live patches.

The following information is included in the live patch metadata:

| Field | Description |
| ----------- | ---------------------- |
| name | Live patch name |
| version | Live patch version |
| release | Live patch release |
| arch | Live patch architecture |
| type | Live patch type |
| target | Target software |
| license | Target software license |
| description | Live patch description |
| entities | Target binary file of the live patch |
| patch| Live patch file list |

Example:

```shell
root@dev:[~]$ syscare info redis-6.2.5-1/HP001-1-1
-------------------------------------------
Patch: redis-6.2.5-1/HP001-1-1
-------------------------------------------
name:        HP001
version:     1
release:     1
arch:        x86_64
type:        UserPatch
target:      redis-6.2.5-1
license:     BSD and MIT
description: CVE-2021-32675 - When parsing an incoming Redis Standard Protocol (RESP) request, Redis allocates memory according to user-specified values which determine the number of elements (in the multi-bulk header) and size of each element (in the bulk header). An attacker delivering specially crafted requests over multiple connections can cause the server to allocate significant amount of memory. Because the same parsing mechanism is used to handle authentication requests, this vulnerability can also be exploited by unauthenticated users.
entities:
* redis-server
* redis-benchmark
* redis-cli
patches:
* 0001-Prevent-unauthenticated-client-from-easily-consuming.patch
-------------------------------------------
```

### Querying Live Patch Status

Run `syscare status` to query the status of one or more live patches.

Example:

```shell
root@dev:[~]$ syscare status status redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-server: NOT-APPLIED
redis-6.2.5-1/HP001-1-1/redis-benchmark: NOT-APPLIED
redis-6.2.5-1/HP001-1-1/redis-cli: NOT-APPLIED
```

### Loading and Activating Live Patches

Run `syscare apply` to load and activate one or more live patches in the **NOT-APPLIED** state.

Example:

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      NOT-APPLIED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            NOT-APPLIED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         NOT-APPLIED

root@dev:[~]$ syscare apply redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-cli: ACTIVED
redis-6.2.5-1/HP001-1-1/redis-benchmark: ACTIVED
redis-6.2.5-1/HP001-1-1/redis-server: ACTIVED
```

### Activating Live Patches

Run `syscare active` to activate one or more live patches in the **DEACTIVED** state.

Example:

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      DEACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            DEACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         DEACTIVED

root@dev:[~]$ syscare active redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-cli: ACTIVED
redis-6.2.5-1/HP001-1-1/redis-benchmark: ACTIVED
redis-6.2.5-1/HP001-1-1/redis-server: ACTIVED
```

### Deactivating Live Patches

Run `syscare deactive` to deactivate one or more live patches in the **ACTIVED** state.

Example:

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED

root@dev:[~]$ syscare deactive redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-cli: DEACTIVED
redis-6.2.5-1/HP001-1-1/redis-benchmark: DEACTIVED
redis-6.2.5-1/HP001-1-1/redis-server: DEACTIVED
```

### Accepting Live Patches

Run `syscare accept` to accept one or more live patches in the **ACTIVED** state.

Accepted live patches are activated automatically after the system is restarted.

Example:

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED

root@dev:[~]$ syscare accept redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-cli: ACCEPTED
redis-6.2.5-1/HP001-1-1/redis-benchmark: ACCEPTED
redis-6.2.5-1/HP001-1-1/redis-server: ACCEPTED
```

### Uninstalling Live Patches

Run `syscare remove` to uninstall one or more live patches in any state.

Example:

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      DEACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         ACCEPTED

root@dev:[~]$ syscare remove redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-cli: NOT-APPLIED
redis-6.2.5-1/HP001-1-1/redis-benchmark: NOT-APPLIED
redis-6.2.5-1/HP001-1-1/redis-server: NOT-APPLIED
```

### Supporting Multiple Live Patches

SysCare allows multiple live patches to be applied to one user-mode binary file.

Example:

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED
789f0052-b932-4d9d-961d-7003bece1a3a     redis-6.2.5-1/HP002-1-1/redis-benchmark                      ACTIVED
e4aee980-1596-43d9-be9a-07fc6f668970     redis-6.2.5-1/HP002-1-1/redis-cli                            ACTIVED
feb13c9a-02b3-4109-a2f3-c3e9fe41e9ad     redis-6.2.5-1/HP002-1-1/redis-server                         ACTIVED
```

#### Detecting Live Patch Conflicts

If one or more live patches to be applied have functions that conflict with existing patches, a message indicating the patch conflict is displayed.

Example:

```shell
root@dev:[~]$ syscare apply redis-6.2.5-1/HP002-1-1
Error: Operation failed

Caused by:
    1. Transaction "Apply patch 'redis-6.2.5-1/HP002-1-1'" failed

    Caused by:
        0: Driver: Patch "redis-6.2.5-1/HP002-1-1/redis-cli" check failed
        1: Upatch: Patch is conflicted with "0e776e26-58cd-42ce-85e4-046481acad09"
```

Run `syscare check` to determine the live patch to be applied conflict with existing live patches.

Example:

```shell
root@dev:[~]$ syscare check redis-6.2.5-1/HP002-1-1
Error: Operation failed

Caused by:
    1. Driver: Patch "redis-6.2.5-1/HP002-1-1/redis-server" check failed

    Caused by:
        Upatch: Patch is conflicted with "d9432f08-65cf-4849-a9af-ba20e9b6c7dc"
```

#### Overwriting Live Patches

Use the `--force` option to overwrite existing live patches with the current one in case of a live patch conflict.

Example:

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED
789f0052-b932-4d9d-961d-7003bece1a3a     redis-6.2.5-1/HP002-1-1/redis-benchmark                      NOT-APPLIED
e4aee980-1596-43d9-be9a-07fc6f668970     redis-6.2.5-1/HP002-1-1/redis-cli                            NOT-APPLIED
feb13c9a-02b3-4109-a2f3-c3e9fe41e9ad     redis-6.2.5-1/HP002-1-1/redis-server                         NOT-APPLIED

root@dev:[~]$ syscare apply redis-6.2.5-1/HP002-1-1
Error: Operation failed

Caused by:
    1. Transaction "Apply patch 'redis-6.2.5-1/HP002-1-1'" failed

    Caused by:
        0: Driver: Patch "redis-6.2.5-1/HP002-1-1/redis-cli" check failed
        1: Upatch: Patch is conflicted with "0e776e26-58cd-42ce-85e4-046481acad09"

root@dev:[~]$ syscare apply redis-6.2.5-1/HP002-1-1 --force
redis-6.2.5-1/HP002-1-1/redis-cli: ACTIVED
redis-6.2.5-1/HP002-1-1/redis-benchmark: ACTIVED
redis-6.2.5-1/HP002-1-1/redis-server: ACTIVED
```

### Saving and Restoring Live Patches

SysCare supports saving and restoring of live patch status.

Example:

```shell
[root@2203sp2-85 syscare]# syscare list
Uuid                                     Name                                                         Status
eebc3155-9a5b-4a09-9561-6a94080de2ce     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
96666521-4606-4aa0-b663-1b455fe586da     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
1e98d692-cc51-4f83-9176-c547ed1db20b     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED
beffae33-1e1a-4bd5-8758-ab6a5f2f1a7c     redis-6.2.5-1/HP002-1-1/redis-benchmark                      NOT-APPLIED
24b01b18-5132-4cae-a379-71d2b0e6d832     redis-6.2.5-1/HP002-1-1/redis-cli                            NOT-APPLIED
a84934de-4a89-4e77-b646-125d1e2c98b4     redis-6.2.5-1/HP002-1-1/redis-server                         ACTIVED
[root@2203sp2-85 syscare]# syscare save
[root@2203sp2-85 syscare]# systemctl restart syscare
[root@2203sp2-85 syscare]# syscare list
Uuid                                     Name                                                         Status
eebc3155-9a5b-4a09-9561-6a94080de2ce     redis-6.2.5-1/HP001-1-1/redis-benchmark                      NOT-APPLIED
96666521-4606-4aa0-b663-1b455fe586da     redis-6.2.5-1/HP001-1-1/redis-cli                            NOT-APPLIED
1e98d692-cc51-4f83-9176-c547ed1db20b     redis-6.2.5-1/HP001-1-1/redis-server                         NOT-APPLIED
beffae33-1e1a-4bd5-8758-ab6a5f2f1a7c     redis-6.2.5-1/HP002-1-1/redis-benchmark                      NOT-APPLIED
24b01b18-5132-4cae-a379-71d2b0e6d832     redis-6.2.5-1/HP002-1-1/redis-cli                            NOT-APPLIED
a84934de-4a89-4e77-b646-125d1e2c98b4     redis-6.2.5-1/HP002-1-1/redis-server                         NOT-APPLIED
[root@2203sp2-85 syscare]# syscare restore
[root@2203sp2-85 syscare]# syscare list
Uuid                                     Name                                                         Status
eebc3155-9a5b-4a09-9561-6a94080de2ce     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
96666521-4606-4aa0-b663-1b455fe586da     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
1e98d692-cc51-4f83-9176-c547ed1db20b     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED
beffae33-1e1a-4bd5-8758-ab6a5f2f1a7c     redis-6.2.5-1/HP002-1-1/redis-benchmark                      NOT-APPLIED
24b01b18-5132-4cae-a379-71d2b0e6d832     redis-6.2.5-1/HP002-1-1/redis-cli                            NOT-APPLIED
a84934de-4a89-4e77-b646-125d1e2c98b4     redis-6.2.5-1/HP002-1-1/redis-server                         ACTIVED
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/using_systemd_to_start_a_container.md
---
# Using systemd to Start a Container

## Function Description

The init process started in system containers differs from that in common containers. Common containers cannot start system services through systemd. However, system containers have this capability. You can enable the systemd service by specifying the  **--system-container**  parameter when starting a system container.

## Parameter Description

## Constraints

* The systemd service needs to call some special system APIs, including mount, umount2, unshare, reboot, and name\_to\_handle\_at. Therefore, permissions to call the preceding APIs are enabled for system containers when the privileged container tag is disabled.
* All system containers are started by the init process. The init process does not respond to the SIGTERM signal which indicates normal exit. By default, the  **stop**  command forcibly kills the container 10 seconds later. If you need a quicker stop, you can manually specify the timeout duration of the  **stop**  command.
* **--system-container**  must be used together with  **--external-rootfs**.
* Various services can run in a system container. The  **systemctl**  command is used to manage the service starting and stopping. Services may depend on each other. As a result, when an exception occurs, some service processes are in the D or Z state so that the container cannot exit properly.
* Some service processes in a system container may affect other operation results. For example, if the NetworkManager service is running in the container, adding NICs to the container may be affected (the NICs are successfully added but then stopped by the NetworkManger), resulting in unexpected results.
* Currently, system containers and hosts cannot be isolated by using udev events. Therefore, the  **fstab**  file cannot be configured.
* The systemd service may conflict with the cgconfig service provided by libcgroup. You are advised to delete the libcgroup-related packages from a container or set  **Delegate**  of the cgconfig service to  **no**.

## Example

* Specify the  **--system-container**  and  **--external-rootfs**  parameters to start a system container.

  ```sh
  [root@localhost ~]# isula run -tid -n systest01 --system-container --external-rootfs /root/myrootfs none init
  ```

* After the preceding commands are executed, the container is running properly. You can run the  **exec**  command to access the container and view the process information. The command output indicates that the systemd service has been started.

  ```sh
  [root@localhost ~]# isula exec -it systest01 bash
  [root@localhost /]# ps -ef
  UID        PID  PPID  C STIME TTY          TIME CMD
  root         1     0  2 06:49 ?        00:00:00 init
  root        14     1  2 06:49 ?        00:00:00 /usr/lib/systemd/systemd-journal
  root        16     1  0 06:49 ?        00:00:00 /usr/lib/systemd/systemd-network
  dbus        23     1  0 06:49 ?        00:00:00 /usr/bin/dbus-daemon --system --
  root        25     0  0 06:49 ?        00:00:00 bash
  root        59    25  0 06:49 ?        00:00:00 ps –ef
  ```

* Run the  **systemctl**  command in the container to check the service status. The command output indicates that the service is managed by systemd.

  ```sh
  [root@localhost /]# systemctl status dbus
  ● dbus.service - D-Bus System Message Bus
     Loaded: loaded (/usr/lib/systemd/system/dbus.service; static; vendor preset:
  disabled)
     Active: active (running) since Mon 2019-07-22 06:49:38 UTC; 2min 5
  8s ago
       Docs: man:dbus-daemon(1)
   Main PID: 23 (dbus-daemon)
     CGroup: /system.slice/dbus.service
             └─23 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidf
  ile --systemd-activation --syslog-only
   
  Jul 22 06:49:38 localhost systemd[1]: Started D-Bus System Message Bus.
  ```

* Run the  **systemctl**  command in the container to stop or start the service. The command output indicates that the service is managed by systemd.

  ```sh
  [root@localhost /]# systemctl stop dbus
  Warning: Stopping dbus.service, but it can still be activated by:
    dbus.socket
  [root@localhost /]# systemctl start dbus
  ```

---

---
url: /en/docs/22.03_LTS_SP4/server/performance/kae/using_the_kae.md
---
# Using the Kunpeng Accelerator Engine (KAE)

## Overview

Kunpeng Accelerator Engine (KAE) is a software acceleration library of openEuler, which provides hardware acceleration engine function on the Kunpeng 920 processor. It supports symmetric encryption, asymmetric encryption, and digital signature. It is ideal for accelerating SSL/TLS applications, reducing processor consumption and improving processor efficiency. In addition, users can quickly migrate existing services through the standard OpenSSL interface.

The KAE supports the following algorithms:

* Digest algorithm SM3, which supports asynchronous mode.
* Symmetric encryption algorithm SM4, which supports asynchronous, CTR, XTS, and CBC modes.
* Symmetric encryption algorithm AES, which supports asynchronous, ECB, CTR, XTS, and CBC modes.
* Asymmetric algorithm RSA, which supports asynchronous mode and key sizes 1024, 2048, 3072, and 4096.
* Key negotiation algorithm DH, which supports asynchronous mode and key sizes 768, 1024, 1536, 2048, 3072, and 4096.

## Application Scenarios

The KAE applies to the following scenarios, as shown in  [Table 1](#table11915824163418).

**Table  1**  Application scenarios

## Installing, Running, and Uninstalling the KAE

### Installing the Accelerator Software Packages

#### Preparing for Installation

##### Environment Requirements

* The accelerator engine is enabled on TaiShan 200 servers.

> \[!NOTE] **NOTE:**
>
> * You need to import the accelerator license. For details, see section "License Management" in the  [TaiShan Rack Server iBMC (V500 or Later) User Guide](https://support.huawei.com/enterprise/en/doc/EDOC1100121685/426cffd9?idPath=7919749|9856522|21782478|8060757).
> * If the accelerator is used in the physical machine scenario, the SMMU must be disabled. For details, see the  [TaiShan 200 Server BIOS Parameter Reference](https://support.huawei.com/enterprise/en/doc/EDOC1100088647).

* CPU: Kunpeng 920
* OS: openEuler-22.03\_LTS\_SP4-aarch64-dvd.iso

##### KAE Software Description

**Table  2**  RPM software packages of the KAE

#### Installing the Accelerator Software Package

##### Prerequisites

* The remote SSH login tool has been installed on the local PC.
* The openEuler OS has been installed.
* The RPM tool is running properly.
* OpenSSL 1.1.1a or a later version has been installed.

  You can run the following commands to query the version number of OpenSSL:

  * openssl version

##### Procedure

1. Log in to the openEuler OS CLI as user  **root**.

2. Create a directory for storing accelerator engine software packages.

3. Use SSH to copy all accelerator engine software packages to the created directory.

4. In the directory, run the  **rpm -ivh**  command to install the accelerator engine software packages.

   > \[!NOTE] **NOTE:**
   > Install the  **libwd**  package first because the  **libkae**  package installation depends on the  **libwd**  package.

   ```shell
   rpm -ivh uacce*.rpm hisi*.rpm libwd-*.rpm libkae*.rpm
   ```

   ```text
   Verifying...                          ################################# [100%]
   Preparing...                          ################################# [100%]
   checking installed modules
   uacce modules start to install
   Updating / installing...
      1:uacce-1.2.10-4.oe1               ################################# [ 14%]
   uacce modules installed
      2:libwd-1.2.10-3.oe1               ################################# [ 29%]
      3:libkae-1.2.10-3.oe1              ################################# [ 43%]
   checking installed modules
   hisi_hpre modules start to install
      4:hisi_hpre-1.2.10-4.oe1           ################################# [ 57%]
   hisi_hpre modules installed
   checking installed modules
   hisi_rde modules start to install
      5:hisi_rde-1.2.10-4.oe1            ################################# [ 71%]
   hisi_rde modules installed
   checking installed modules
   hisi_sec2 modules start to install
      6:hisi_sec2-1.2.10-4.oe1           ################################# [ 86%]
   hisi_sec2 modules installed
   checking installed modules
   hisi_zip modules start to install
      7:hisi_zip-1.2.10-4.oe1            ################################# [100%]
   hisi_zip modules installed
   ```

5. Run the  **rpm -qa**  command to check whether the accelerator software packages have been installed successfully. Run the  **rpm -ql**  command to check whether files in the software packages are correct. The following is an example:

   ```shell
   rpm -qa|grep -E "hisi|uacce|libwd|libkae"
   ```

   ```text
   hisi_rde-1.2.10-4.oe1.aarch64
   hisi_sec2-1.2.10-4.oe1.aarch64
   libkae-1.2.10-3.oe1.aarch64
   hisi_hpre-1.2.10-4.oe1.aarch64
   uacce-1.2.10-4.oe1.aarch64
   libwd-1.2.10-3.oe1.aarch64
   hisi_zip-1.2.10-4.oe1.aarch64
   ```

   ```shell
   rpm -ql uacce hisi* libwd* libkae
   ```

   ```text
   /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_qm.ko
   /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/uacce.ko
   /etc/modprobe.d/hisi_hpre.conf
   /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_hpre.ko
   /etc/modprobe.d/hisi_rde.conf
   /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_rde.ko
   /etc/modprobe.d/hisi_sec2.conf
   /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_sec2.ko
   /etc/modprobe.d/hisi_zip.conf
   /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_zip.ko
   /usr/include/warpdrive/config.h
   /usr/include/warpdrive/include/uacce.h
   /usr/include/warpdrive/smm.h
   /usr/include/warpdrive/wd.h
   /usr/include/warpdrive/wd_bmm.h
   /usr/include/warpdrive/wd_cipher.h
   /usr/include/warpdrive/wd_comp.h
   /usr/include/warpdrive/wd_dh.h
   /usr/include/warpdrive/wd_digest.h
   /usr/include/warpdrive/wd_rsa.h
   /usr/lib64/libwd.so.1.2.10
   /usr/local/lib/engines-1.1/libkae.so.1.2.10
   ```

6. Restart the system or run commands to manually load the accelerator engine drivers to the kernel in sequence, and check whether the drivers are successfully loaded.

   ```shell
   modprobe uacce 
   lsmod | grep uacce 
   modprobe hisi_qm
   lsmod | grep hisi_qm 
   modprobe hisi_qm
   modprobe hisi_sec2 # Loads the hisi_sec2 driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_sec2.conf.
   modprobe hisi_hpre # Loads the hisi_hpre driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_hpre.conf.
   ```

##### Environment Variables Setup

Run the following command to export the environment variables (If you have specified the installation directory, set **/usr/local** to the actual one):

```shell
export OPENSSL_ENGINES=/usr/local/lib/engines-1.1
```

##### Post-Installation Check

Run the  **rpm -qa**  command to check whether the accelerator engine software packages are successfully installed.

If the command output contains  *software package name***-***version number***-**, the software packages are successfully installed. The following is an example:

```shell
rpm -qa|grep -E "hisi|uacce|libwd|libkae"
```

```text
hisi_rde-1.2.10-4.oe1.aarch64
hisi_sec2-1.2.10-4.oe1.aarch64
libkae-1.2.10-3.oe1.aarch64
hisi_hpre-1.2.10-4.oe1.aarch64
uacce-1.2.10-4.oe1.aarch64
libwd-1.2.10-3.oe1.aarch64
hisi_zip-1.2.10-4.oe1.aarch64
```

#### Required Operations After Installation

##### Testing the OpenSSL Accelerator Engine

You can run the following commands to test some accelerator functions.

* Use the OpenSSL software algorithm to test the RSA performance.

  ```shell
  $ ./openssl speed -elapsed rsa2048
  ...
                   sign    verify    sign/s verify/s
  rsa 2048 bits 0.001384s 0.000035s   724.1  28365.8.
  ```

* Use the KAE to test the RSA performance.

  ```shell
  $ ./openssl speed -elapsed -engine kae rsa2048
  ....
                   sign    verify    sign/s verify/s
  rsa 2048 bits 0.000355s 0.000022s   2819.0  45478.4
  ```

> \[!NOTE] **NOTE:**
> After the KAE is used, the signature performance is improved from 724.1 sign/s to 2819 sign/s.

* Use the OpenSSL software algorithm to test the asynchronous RSA performance.

  ```shell
  $ ./openssl speed -elapsed -async_jobs 36 rsa2048 
  ....
                    sign    verify    sign/s verify/s
  rsa 2048 bits 0.001318s 0.000032s    735.7  28555
  ```

* Use the KAE to test the asynchronous RSA performance.

  ```shell
  $ ./openssl speed -engine kae -elapsed -async_jobs 36 rsa2048 
  .... 
                    sign    verify    sign/s verify/s
  rsa 2048 bits 0.000018s 0.000009s  54384.1 105317.0
  ```

> \[!NOTE] **NOTE:**
> After the KAE is used, the asynchronous RSA signature performance is improved from 735.7 sign/s to 54384.1 sign/s.

* Use the OpenSSL software algorithm to test the performance of the SM4 CBC mode.

  ```shell
  $ ./openssl speed -elapsed -evp sm4-cbc
  You have chosen to measure elapsed time instead of user CPU time.
  ....
  Doing sm4-cbc for 3s on 10240 size blocks: 2196 sm4-cbc's in 3.00s  ....
  type          51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes
  sm4-cbc          82312.53k    85196.80k    85284.18k    85000.85k    85284.18k    85261.26k
  ```

* Use the KAE to test the SM4 CBC mode performance.

  ```shell
  $ ./openssl speed -elapsed -engine kae -evp sm4-cbc
  engine "kae" set. 
  You have chosen to measure elapsed time instead of user CPU time.
  ...
  Doing sm4-cbc for 3s on 1048576 size blocks: 11409 sm4-cbc's in 3.00s
  ...
  type          51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes
  sm4-cbc         383317.33k   389427.20k   395313.15k   392954.73k   394264.58k   394264.58k
  ```

> \[!NOTE] **NOTE:**
> After the KAE is used, the SM4 CBC mode performance is improved from 82312.53 kbit/s to 383317.33 kbit/s when the input data block size is 8 MB.

* Use the OpenSSL software algorithm to test the SM3 mode performance.

  ```shell
  $ ./openssl speed -elapsed -evp sm3
  You have chosen to measure elapsed time instead of user CPU time.
  Doing sm3 for 3s on 102400 size blocks: 1536 sm3's in 3.00s
  ....
  type          51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes
  sm3              50568.53k    52428.80k    52428.80k    52428.80k    52428.80k    52428.80k
  ```

* Use the KAE to test the SM3 mode performance.

  ```shell
  $ ./openssl speed -elapsed -engine kae -evp sm3
  engine "kae" set.
  You have chosen to measure elapsed time instead of user CPU time.
  Doing sm3 for 3s on 102400 size blocks: 19540 sm3's in 3.00s
  ....
  type            51200 bytes  102400 bytes  1048576 bytes 2097152 bytes 4194304 bytes 8388608 bytes
  sm3             648243.20k   666965.33k   677030.57k   678778.20k   676681.05k   668292.44k
  ```

> \[!NOTE] **NOTE:**
> After the KAE is used, the SM3 algorithm performance is improved from 52428.80 kbit/s to 668292.44 kbit/s when the input data block size is 8 MB.

* Use the OpenSSL software algorithm to test the asynchronous performance of the AES algorithm in CBC mode.

  ```shell
  $ ./openssl speed -elapsed -evp aes-128-cbc -async_jobs 4
  You have chosen to measure elapsed time instead of user CPU time.
  Doing aes-128-cbc for 3s on 51200 size blocks: 65773 aes-128-cbc's in 3.00s
  Doing aes-128-cbc for 3s on 102400 size blocks: 32910 aes-128-cbc's in 3.00s
  ....
  type          51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes
  aes-128-cbc    1122525.87k  1123328.00k  1120578.22k  1121277.27k  1119879.17k  1115684.86k
  ```

* Use the KEA engine to test the asynchronous performance of the AES algorithm in CBC mode.

  ```shell
  $ ./openssl speed -elapsed -evp aes-128-cbc -async_jobs 4 -engine kae
  engine "kae" set.
  You have chosen to measure elapsed time instead of user CPU time.
  Doing aes-128-cbc for 3s on 51200 size blocks: 219553 aes-128-cbc's in 3.00s
  Doing aes-128-cbc for 3s on 102400 size blocks: 117093 aes-128-cbc's in 3.00s
  ....
  type          51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes
  aes-128-cbc    3747037.87k  3996774.40k  1189085.18k  1196774.74k  1196979.11k  1199570.94k
  ```

> \[!NOTE] **NOTE:**
>
> * The AES algorithm supports only asynchronous mode when the data length is 256 KB or less.
> * After the KAE is used, the AES algorithm performance is improved from 1123328.00 kbit/s to 3996774.40 kbit/s when the input data block size is 100 KB.

### Upgrading the Accelerator Software Packages

#### Scenario

You can run the  **rpm -Uvh**  command to upgrade the accelerator software.

#### Procedure

1. Download the latest accelerator engine software packages from the openEuler community.

2. Use SSH to log in to the Linux CLI as user  **root**.

3. Save the downloaded software packages to a directory.

4. In the directory, run the  **rpm -Uvh**  command to upgrade the accelerator driver package and engine library package. The following is an example:

   The command and output are as follows:

   ![](./figures/en-us_image_0231143189.png)

   ![](./figures/en-us_image_0231143191.png)

5. Run the  **rpm -qa**  command to check whether the upgrade is successful. Ensure that the queried version is the latest version.

   ![](./figures/en-us_image_0231143193.png)

   ![](./figures/en-us_image_0231143195.png)

6. Restart the system or run the following commands to manually uninstall the drivers of the earlier version, load the drivers of the latest version, and check whether the new drivers are successfully loaded.

   ```shell
   # Uninstall the existing drivers.
   $ lsmod | grep uacce 
   uacce                 262144  3 hisi_hpre,hisi_sec2,hisi_qm 
   $ 
   $ rmmod hisi_hpre 
   $ rmmod hisi_sec2 
   $ rmmod hisi_qm 
   $ rmmod uacce 
   $ lsmod | grep uacce 
   $ 
   # Load the new drivers.
   $ modprobe uacce
   $ modprobe hisi_qm 
   $ modprobe hisi_sec2 # Loads the hisi_sec2 driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_sec2.conf.
   $ modprobe hisi_hpre  # Loads the hisi_hpre driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_hpre.conf.
   $ lsmod | grep uacce 
   uacce                36864  3 hisi_sec2,hisi_qm,hisi_hpre
   ```

### Uninstalling the Accelerator Software Packages

#### Scenario

You do not need the accelerator engine software or you want to install a new one.

#### Procedure

1. Use SSH to log in to the Linux CLI as user  **root**.

2. Restart the system or run commands to manually uninstall the accelerator drivers loaded to the kernel, and check whether the drivers are successfully uninstalled.

   ```shell
   # lsmod | grep uacce 
   uacce                36864  3 hisi_sec2,hisi_qm,hisi_hpre 
   # rmmod hisi_hpre 
   # rmmod hisi_sec2  
   # rmmod hisi_qm 
   # rmmod uacce 
   # lsmod | grep uacce 
   #
   ```

3. Run the  **rpm -e**  command to uninstall the accelerator engine software packages. The following is an example:

   > \[!NOTE] **NOTE:**
   > Due to the dependency relationships, the  **libkae**  package must be uninstalled before the  **libwd**  package.

   ![](./figures/en-us_image_0231143196.png)

   ![](./figures/en-us_image_0231143197.png)

4. Run the  **rpm -qa |grep**  command to check whether the uninstallation is successful.

   ![](./figures/en-us_image_0231143198.png)

## Querying Logs

[Table 3](#table52821836)  lists log information related to the accelerator engine.

**Table  3**  Log information

## Acceleration Engine Application

> \[!NOTE] **NOTE:**
> If you have not purchased the engine license, you are advised not to use the KAE to invoke the corresponding algorithms. Otherwise, the performance of the OpenSSL encryption algorithm may be affected.

### Example Code for the KAE

```c
#include <stdio.h> 

#include <stdlib.h> 

/* OpenSSL headers */ 

#include <openssl/bio.h> 

#include <openssl/ssl.h> 

#include <openssl/err.h> 

#include <openssl/engine.h> 

int main(int argc, char **argv) 

{ 

    /* Initializing OpenSSL */ 

    SSL_load_error_strings(); 

    ERR_load_BIO_strings(); 

    OpenSSL_add_all_algorithms(); 

    /*You can use ENGINE_by_id Function to get the handle of the Huawei Accelerator Engine*/ 

    ENGINE *e = ENGINE_by_id("kae"); 

    /* Enable the accelerator asynchronization function. This parameter is optional. The value 0 indicates disabled, and the value 1 indicates enabled. The asynchronous function is enabled by default. */

    ENGINE_ctrl_cmd_string(e, "KAE_CMD_ENABLE_ASYNC", "1", 0) 

    ENGINE_init(e); 

    RSA*rsa=RSA_new_method(e);#Specify the engine for RSA encryption and decryption.

    /*The user code*/ 

    ...... 

; 

    ENGINE_free(e); 

; 

}
```

### Usage of the KAE in the OpenSSL Configuration File openssl.cnf

Create the  **openssl.cnf**  file and add the following configuration information to the file:

```text
openssl_conf=openssl_def 
[openssl_def] 
engines=engine_section 
[engine_section] 
kae=kae_section 
[kae_section] 
engine_id=kae 
dynamic_path=/usr/local/lib/engines-1.1/kae.so 
KAE_CMD_ENABLE_ASYNC=1 #The value 0 indicates that the asynchronous function is disabled. The value 1 indicates that the asynchronous function is enabled. The asynchronous function is enabled by default.
default_algorithms=ALL 
init=1
```

Export the environment variable  **OPENSSL\_CONF**.

```shell
export OPENSSL_CONF=/home/app/openssl.cnf #Path for storing the openssl.cnf file
```

The following is an example of the OpenSSL configuration file:

```c
#include <stdio.h>  

#include <stdlib.h>  

/* OpenSSL headers */  

#include <openssl/bio.h>  

#include <openssl/ssl.h>  

#include <openssl/err.h>  

#include <openssl/engine.h>  

int main(int argc, char **argv)  

{  

    /* Initializing OpenSSL */  

    SSL_load_error_strings();  

    ERR_load_BIO_strings();  

#Load openssl configure 

OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CONFIG, NULL);    OpenSSL_add_all_algorithms();  

    /*You can use ENGINE_by_id Function to get the handle of the Huawei Accelerator Engine*/  

    ENGINE *e = ENGINE_by_id("kae"); 

    /*The user code*/  

    ......  

;  

    ENGINE_free(e);  

;
}
```

## Troubleshooting

### Failed to Initialize the Accelerator Engine

#### Symptom

The accelerator engine is not completely loaded.

#### Solution

1. Check whether the accelerator drivers are loaded successfully. Specifically, run the  **lsmod**  command to check whether uacce.ko, qm.ko, sgl.ko, hisi\_sec2.ko, hisi\_hpre.ko, hisi\_zip.ko, and hisi\_rde.ko exist.

   ```shell
   $ lsmod | grep uacce
   uacce                  262144  2 hisi_hpre,hisi_qm,hisi_sec2,hisi_zip,hisi_rde
   ```

2. Check whether the accelerator engine library exists in  **/usr/lib64**  (directory for RPM installation) or  **/usr/local/lib**  (directory for source code installation) and the OpenSSL installation directory, and check whether the correct soft link is established.

   ```shell
   $ ll /usr/local/lib/engines-1.1/ |grep kae 
   # Check whether the KAE has been correctly installed and whether a soft link has been established. If yes, the displayed information is as follows:
   lrwxrwxrwx. 1 root root     22 Nov 12 02:33 kae.so -> kae.so.1.0.1
   lrwxrwxrwx. 1 root root     22 Nov 12 02:33 kae.so.0 -> kae.so.1.0.1
   -rwxr-xr-x. 1 root root 112632 May 25  2019 kae.so.1.0.1
   $
   $ ll /usr/lib64/ | grep libwd  
   # Check whether libwd has been correctly installed and whether a soft link has been established. If yes, the displayed information is as follows:
   lrwxrwxrwx.  1 root root       14 Nov 12 02:33 libwd.so -> libwd.so.1.0.1
   lrwxrwxrwx.  1 root root       14 Nov 12 02:33 libwd.so.0 -> libwd.so.1.0.1
   -rwxr-xr-x.  1 root root   137120 May 25  2019 libwd.so.1.0.1
   $
   ```

3. Check whether the path of the OpenSSL engine library can be exported by running the  **export**  command.

   ```shell
   $ echo $OPENSSL_ENGINES 
   $ export OPENSSL_ENGINES=/usr/local/lib/engines-1.1
   $ echo $OPENSSL_ENGINES
   /usr/local/lib/engines-1.1
   ```

### Failed to Identify Accelerator Devices After the Acceleration Engine Is Installed

#### Symptom

After the acceleration engine is installed, the accelerator devices cannot be identified.

#### Solution

1. Check whether the device exists in the virtual file system. Normally, the following accelerator devices are displayed:

   ```shell
   $ ls -al /sys/class/uacce/
   total 0
   lrwxrwxrwx. 1 root root 0 Nov 14 03:45 hisi_hpre-2 -> ../../devices/pci0000:78/0000:78:00.0/0000:79:00.0/uacce/hisi_hpre-2
   lrwxrwxrwx. 1 root root 0 Nov 14 03:45 hisi_hpre-3 -> ../../devices/pci0000:b8/0000:b8:00.0/0000:b9:00.0/uacce/hisi_hpre-3
   lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_rde-4 -> ../../devices/pci0000:78/0000:78:01.0/uacce/hisi_rde-4
   lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_rde-5 -> ../../devices/pci0000:b8/0000:b8:01.0/uacce/hisi_rde-5
   lrwxrwxrwx. 1 root root 0 Nov 14 08:39 hisi_sec-0 -> ../../devices/pci0000:74/0000:74:01.0/0000:76:00.0/uacce/hisi_sec-0
   lrwxrwxrwx. 1 root root 0 Nov 14 08:39 hisi_sec-1 -> ../../devices/pci0000:b4/0000:b4:01.0/0000:b6:00.0/uacce/hisi_sec-1
   lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_zip-6 -> ../../devices/pci0000:74/0000:74:00.0/0000:75:00.0/uacce/hisi_zip-6
   lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_zip-7 -> ../../devices/pci0000:b4/0000:b4:00.0/0000:b5:00.0/uacce/hisi_zip-7
   ```

2. If you want to use the HPRE device but the device is not found in  [1](#li1760055514614), check whether the accelerator software is correctly installed by referring to  [Failed to Upgrade the Accelerator Drivers](#failed-to-upgrade-the-accelerator-drivers).

3. If the accelerator software is correctly installed, run the  **lspci**  command to check whether the physical device exists.

   ```shell
   $ lspci | grep HPRE
   79:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon HPRE Engine (rev 21)
   b9:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon HPRE Engine (rev 21)
   $ lspci | grep SEC
   76:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon SEC Engine (rev 21)
   b6:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon SEC Engine (rev 21)
   $ lspci | grep RDE
   78:01.0 RAID bus controller: Huawei Technologies Co., Ltd. HiSilicon RDE Engine (rev 21)
   b8:01.0 RAID bus controller: Huawei Technologies Co., Ltd. HiSilicon RDE Engine (rev 21)
   $ lspci | grep ZIP
   75:00.0 Processing accelerators: Huawei Technologies Co., Ltd. HiSilicon ZIP Engine (rev 21)
   b5:00.0 Processing accelerators: Huawei Technologies Co., Ltd. HiSilicon ZIP Engine (rev 21)
   $
   ```

4. If no physical device is found in  [3](#li1560012551369), perform the following operations:
   * Check whether the accelerator license has been imported. If no, import the accelerator license. For details, see "License Management" in the  [TaiShan Rack Server iBMC (V500 or Later) User Guide](https://support.huawei.com/enterprise/en/doc/EDOC1100121685/426cffd9?idPath=7919749|9856522|21782478|8060757). After the accelerator license is imported, power off and restart the iBMC to enable the license.
   * Check whether the iBMC and BIOS versions support the accelerator feature.

### Failed to Upgrade the Accelerator Drivers

#### Symptom

After the accelerator drivers are upgraded, the driver version is not changed after the system is restarted.

#### Possible Cause

Before the accelerator drivers are upgraded, the system upgrades other driver packages. These driver packages may update the boot file system initramfs, and update the accelerator drivers to initramfs before upgrade. For example, if the NIC driver is updated or initramfs is manually updated, the system loads the accelerator drivers from initramfs first during restart.

#### Solution

After the accelerator drivers are upgraded, run the  **dracut --force**  command to update initramfs again.

---

---
url: /en/docs/22.03_LTS_SP4/server/administration/compa_command/utshell_guide.md
---
# utshell User Guide

## Introduction

utshell is a shell compatible with Bash, capable of executing basic built-in commands and starting external commands. It also implements functions such as task, pipe, and signal processing.

## Installation and Uninstallation

### Installing utshell

Run the `rpm` command to install utshell. Assume that openEuler 23.09 is used.

![](./media/image1.png)

Enter **y** as prompted to install.

![](./media/image2.png)

### Uninstalling utshell

Run `rpm -e utshell` to uninstall utshell.

```shell
rpm -e utshell
```

![](./media/image3.png)

## Usage

### Using Common Commands

In the utshell environment, enter a command to execute.

utshell has the following built-in commands:

![](./media/image4.png)

### Defining and Using Variables

#### Defining a Variable

Use **=** to define a variable. No space is allowed in the expression.

```shell
var=4
```

#### Using a Variable

```shell
echo ${var}
```

### Defining and Using Arrays

#### Defining an Array

```shell
distros=(ubuntu fedora suse "arch linux")
```

#### Using an Array

```shell
echo ${distros[2]}
```

### Defining and Using Functions

#### Defining a Function

```shell
func() { echo $1; }
```

#### Using a Function

```shell
func 1
```

#### Passing Parameters to a Function

When calling a function, use a space to separate the function and the parameters.

```shell
func firstParam secondParam
```

In the function body, use **${number}** to represent the parameters, for example, $1 for the first parameter and $2 for the second parameter. For the tenth and subsequent parameters, the number must be enclosed in braces.

```shell
func() {
echo $1 ${10} # Ten parameters are required.
}
# Call the function.
func 1 2 3 4 5 6 7 8 9 0
```

### Using Logical Conditions

#### if

The syntax is as follows:

```shell
if condition; then
do-if-true;
elif second-condition; then
do-else-if-true
elif third-condition; then
do-else-if-third-true
else
do-else-false
fi
```

**condition** can be a command, for example:

```shell
if [ "$s" = "string" ]; then
echo "string is equivalent to $s"
else
echo "string is not equivalent to $s"
fi
```

**condition** can also be a conditional operator.

Some conditional operators are as follows.

```shell
-f: Checks whether a file exists and is a regular file.
-d: Checks whether the provided argument is a directory.
-h: Checks whether the provided argument is a symbolic link.
-s: Checks whether a file exists and is not empty.
-r: Checks whether a file is readable.
-w: Checks whether a file is writable.
-x: Checks whether a file is executable.
```

The following conditional operators can be used for comparing numbers.

```shell
-lt: less than
-gt: greater than
-ge: greater than or equal to
-le: less than or equal to
-ne: not equal to
```

The following conditional operators can be used for comparing strings.

```shell
==: Whether two strings are identical.
=: Whether two strings are identical (same as ==).
!=: Whether two strings are different.
-z: Returns true if the string is empty.
-n: Returns true if the string length is not 0.
```

### Using Loops

#### for

```shell
for number in 1 2 3 4 5
do
echo $number
done
# When used with a list:
for number in {1..500..2}
do
echo $number
done
```

**{1..500..2}** indicates that the start number is 1, the end number is 500 (included), and the step is 2.

#### until

```shell
until [condition]; do
commands
done
```

When the condition is true, the loop is executed.

#### while

```shell
while [ condition ]; do
commands
done
```

When the condition is true, the loop is executed.

---

---
url: /zh/docs/22.03_LTS_SP4/server/administration/compa_command/utshell_guide.md
---
# utshell 用户手册

## 介绍

utshell 是一个与 bash 兼容的 shell。它实现了基本的内建命令执行和启动外部命令。同时也实现了任务、管道和信号处理等功能。

## 安装和卸载

### 安装

utshell 使用 rpm 命令进行安装，我们假设使用的是欧拉的 2309 系统：

进入命令行界面执行：

![截图.png](./media/image1.png)

根据提示输入"y"，即可安装成功。

![截图.png](./media/image2.png)

### 卸载

在命令行执行`rpm -e utshell`即可卸载 utshell。

```shell

rpm -e utshell

```

![截图.png](./media/image3.png)

## 使用

### 一般命令

在 utshell 环境下，直接键入命令名即可执行对应的命令。

utshell 支持如下内建命令：

![截图.png](./media/image4.png)

### 变量定义和使用

#### 变量定义

变量定义直接用"="，中间不能有空格。

```shell

var=4

```

#### 变量使用

```shell

echo ${var}

```

### 数组的定义和使用

#### 数组的定义

```shell

distros=(ubuntu fedora suse "arch linux")

```

#### 数组的使用

```shell

echo ${distros[2]}

```

### 函数定义和使用

#### 函数的定义

```shell

func() { echo $1; }

```

#### 函数的调用

```shell

func 1

```

#### 给函数传递参数

调用函数时，在函数名后面直接以空格分割参数：

`func firstParam secondParam`

在函数体中使用$1,$2......其中$1 表示第一个参数，$2 表示第二个参数，大于 9 的需要用大括号括起来。

```shell

func() {

echo $1 ${10} #需要传递 10 个参数

}

#调用

func 1 2 3 4 5 6 7 8 9 0

```

### 逻辑判断

#### if 语句

语法为：

```shell

if condition; then

do-if-true;

elif second-condition; then

do-else-if-true

elif third-condition; then

do-else-if-third-true

else

do-else-false

fi

```

其中 condition 可以是命令，如：

```shell

if [ "$s" = "string" ]; then

echo "string is equivalent to \$s"

else

echo "string is not equivalent to \$s"

fi

```

也可以是测试条件操作符：

下面简单介绍些条件操作符：

```shell

    -f 检查文件是否存在并且它是一个普通文件。

    -d 检查提供的参数是否是目录。

    -h 检查提供的参数是否是符号链接。

    -s 检查文件是否存在且不为空。

    -r 检查文件是否可读。

    -w 检查文件是否可写。

    -x 检查文件是否可执行。

```

如果用于数字比较,可以用如下测试条件操作符：

```shell

    -lt 小于

    -gt 大于

    -ge 大于等于

    -le 小于等于

    -ne 不等于

```

如果用于字符串比较,可以用如下测试条件操作符：

```shell

    == 两个字符串相同

    = 两个字符串相同(同==)

    != 两个字符串不同

    -z 空字符串,返回 true

    -n 长度不是 0,则返回 true

```

### 循环

#### for 循环

```shell

for number in 1 2 3 4 5

do

echo $number

done

使用列表:

for number in {1..500..2}

do

echo $number

done

```

其中{1..500..2}表示起始数字为 1，结束数字为 500(包括)，步长为 2。

#### until 循环

```shell

until [ condition ]; do

commands

done

```

当条件为真时，执行循环；

#### while 循环

```shell

while [ condition ]; do

commands

done

```

当条件为真时，执行循环。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/compa_command/utsudo_user_guide.md
---
# utsudo User Guide

This document describes how to install and use utsudo. utsudo is fully compatible with sudo in terms of parameter functions and plug-in usage, greatly reducing users' learning costs.

This document is intended for utsudo developers, testers, and common users.

## utsudo Introduction

The utsudo project was initiated in June 2022. It aims to reconstruct sudo using the Rust language. utsudo is an efficient, secure, and flexible privilege escalation tool. The modules of utsudo include the common tool library, overall framework, and plug-in functions.

## utsudo Installation

In version 0.0.4, some files of utsudo conflict with those of sudo. Therefore, you need to use `yumdownloader` to download the binary RPM package of utsudo, and then run `rpm` to install the package with conflicts allowed.

Run `yumdownloader utsudo` to download the utsudo binary RPM package.

Then, run `sudo rpm -ivh utsudo-0.0.1-0.04.x86_64.rpm --replacefiles` to install utsudo. The execution process is as follows.

![](./figures/image-20230828094539717.png)

After the installation is complete, run `rpm -qa | grep utsudo` to check whether utsudo is properly installed, as shown in the following figure.

![](./figures/image-20230828094723153.png)

As shown in the preceding figure, utsudo has been installed and the version is **0.0.1-0.04**.

utsudo will be continuously updated in the future.

## utsudo Usage

`utsudo` has various options. Some options are as follows. You can run `utsudo -h` for details.

```shell
-e, --edit                    Edit a file instead of running a command.
-k, --reset-timestamp         Invalidate the timestamp file.
-l, --list                    List user privileges or check a specific command. Use the option twice for the longer format.
```

### `-e`

The `-e` option is used to edit files.

`utsudo -e` is equivalent to `sudoedit`. When the command is executed, a common user is used to edit a file. A file in the writable directory of the calling user cannot be edited unless the user is **root**.

In a directory on which the current user does not have write permission, a file **test.txt** exists on which the current user does not have write permission. When you edit the **test.txt** file as a common user, a message is displayed indicating that you do not have the permission. You can run `utsudo -e` to edit the file. The following figure shows the execution process.

![](./figures/image-20230828135001624.png)

As shown in the figure, the content of the **test.txt** file is successfully modified. (**utsudo -e is okay!!** was added in the editor.)

### `-k`

The `-k` option invalidates the timestamp.

By default, you need to enter the password when you run the `utsudo` command for the first time and every five minutes. The `-k` parameter forces the user to enter the password the next time the `utsudo` command is executed.

![](./figures/image-20230828140355863.png)

By default, you do not need to enter the password for five minutes after running `utsudo` for the first time.

However, as shown in the figure, the `utsudo -k` command invalidates the timestamp of the `utsudo` command.

### `-l`

The `-l` option displays the commands that the current user can execute by using `utsudo`.

The execution process is as follows:

![](./figures/image-20230828140709441.png)

As shown in the figure, the **test** user can run the following commands:

```shell
(ALL) ALL
```

That is, all commands can be executed by user **test**, indicating that the **/etc/sudoers** file does not restrict the user.

This section briefly describes how to use `utsudo`. Other functions and options of utsudo are not listed.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/administration/compa_command/utsudo_user_guide.md
---
# utsudo 使用指南

本文档主要介绍`utsudo`工具的安装和简单使用，帮助用户快速上手。`utsudo`从参数功能到插件使用都是完全兼容`sudo`，大大降低了用户的学习成本，欢迎大家使用。

本文档主要适用于`utsudo`的开发人员、测试人员、以及普通用户。

## utsudo 介绍

utsudo 诞生于 2022 年 6 月份，是一个目前正在进行的使用 Rust 语言重构 Sudo 的项目。utsudo 旨在提供一个更加高效、安全、灵活的提权工具，涉及的模块主要有：通用工具库、整体框架和插件功能等。

## utsudo 安装

在`0.0.4`版本中，`utsudo`与`sudo`还存在部分文件冲突。需要先使用`yum`命令，把`utsudo`的二进制`rpm`包下载到本地，再使用`rpm`命令进行安装 ，以允许与`sudo`的文件冲突。

首先使用`yumdownloader  utsudo`命令下载`utsudo`二进制包到本地。

然后使用`sudo rpm -ivh utsudo-0.0.1-0.04.x86_64.rpm  --replacefiles`命令安装`utsudo`，执行过程如下所示。

![image-20230828094539717](./figures/image-20230828094539717.png)

安装完成后，使用`rpm -qa | grep utsudo`命令查看`utsudo`是否正常安装，如下图所示。

![image-20230828094723153](./figures/image-20230828094723153.png)

由上图可知，`utsudo`已正常安装，安装的版本是`0.0.1-0.04`。

`utsudo`后续还会有版本更新，大家以自己安装的版本为准。

## utsudo使用

下面给大家介绍一下`utsudo`的简单使用。

`utsudo`参数较多，下面简单列出部分参数，详细内容可使用`utsudo  -h`列出。

```shell

-e, --edit                    编辑文件而非执行命令
-k, --reset-timestamp         无效的时间戳文件
-l, --list                    列出用户权限或检查某个特定命令；对于长格式，使用两次

```

### `-e` 参数

`-e`参数的功能：编辑文件。

`utsudo -e`相当于`sudoedit`命令，执行时会调用普通用户进行编辑，而位于调用用户可写目录中的文件是无法编辑的，除非该用户是`root`用户。

在当前用户没有可写权限的目录`e`中，存在一个当前用户没有可编辑权限的文件：`test.txt`。使用普通用户编辑`test.txt`文件，会提示没有权限。使用`utsudo -e`后，可以正常编辑。具体执行过程如下图所示。

![image-20230828135001624](./figures/image-20230828135001624.png)

由上图可知，成功修改了`test.txt`文件的内容。（其中`utsudo -e is okay !!`，是我们在编辑器中自己添加的。）

### `-k`参数

`-k`参数功能：使时间戳无效。

默认使用`utsudo`执行命令时，第一次是需要输入密码的，短时间内（默认是5分钟）再次执行`sudo`命令则不需要再次输入密码。使用`-k`参数可以强迫使用者在下一次执行`utsudo`时询问密码。

![image-20230828140355863](./figures/image-20230828140355863.png)

`utsudo`输入密码后，默认`5`分钟之内不需要输入密码。

但是，由上图可知，`utsudo -k`使得`utsudo`的时间戳失效了。

### `-l`参数

`-l`参数功能：用于显示当前用户可以用`utsudo`执行哪些命令。

执行过程如下所示：

![image-20230828140709441](./figures/image-20230828140709441.png)

上图显示了`test`用户，可以运行如下命令：

```shell

(ALL) ALL

```

也就是所有命令，说明`/etc/sudoers`文件中，并没有对`test`用户做过多的限制。

`utsudo`的使用，就简单介绍这些，除了上面介绍到的，`utsudo`还有很多其他的功能和参数，此处就不一一列举了。欢迎大家讨论。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/server/administration/administrator/viewing_system_information.md
---
# Viewing System Information

* View the system information.

  ```bash
  cat /etc/os-release
  ```

  For example, the command and output are as follows:

  ```bash
  $ cat /etc/os-release
  NAME="openEuler"
  VERSION="22.03 (LTS-SP4)"
  ID="openEuler"
  VERSION_ID="22.03"
  PRETTY_NAME="openEuler 22.03 (LTS-SP4)"
  ANSI_COLOR="0;31"
  ```

* View system resource information.

  Run the following command to view the CPU information:

  ```bash
  lscpu
  ```

  Run the following command to view the memory information:

  ```bash
  free
  ```

  Run the following command to view the disk information:

  ```bash
  fdisk -l
  ```

  View the real-time system resource information.

  ```bash
  top
  ```

---

---
url: /en/docs/22.03_LTS_SP4/virtualization.md
---


---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/vm_configuration.md
---
# VM Configuration

## Introduction

### Overview

Libvirt tool uses XML files to describe a VM feature, including the VM name, CPU, memory, disk, NIC, mouse, and keyboard. You can manage a VM by modifying configuration files. This section describes the elements in the XML configuration file to help users configure VMs.

### Format

The VM XML configuration file uses domain as the root element, which contains multiple other elements. Some elements in the XML configuration file can contain corresponding attributes and attribute values to describe VM information in detail. Different attributes of the same element are separated by spaces.

The basic format of the XML configuration file is as follows. In the format,  **label**  indicates the label name,  **attribute**  indicates the attribute, and  **value**  indicates the attribute value. Change them based on the site requirements.

```xml
<domain type='kvm'>
    <name>VMName</name>
    <memory attribute='value'>8</memory>
    <vcpu>4</vcpu>
    <os>
       <label attribute='value' attribute='value'>
         ...
       </label>
    </os>
    <label attribute='value' attribute='value'>
      ...
    </label>  
</domain>
```

### Process

1. Create an XML configuration file with domain root element.

2. Use the name tag to specify a unique VM name based on the naming rule.

3. Configure system resources such as the virtual CPU (vCPU) and virtual memory.

4. Configure virtual devices.
   1. Configure storage devices.
   2. Configure network devices.
   3. Configure the external bus structure.
   4. Configure external devices such as the mouse.

5. Save the XML configuration file.

## VM Description

### Overview

This section describes how to configure the VM  **domain**  root element and VM name.

### Elements

* **domain**: Root element of a VM XML configuration file, which is used to configure the type of the hypervisor that runs the VM.

  **type**: Type of a domain in virtualization. In the openEuler virtualization, the attribute value is  **kvm**.

* **name**: VM name.

  The VM name is a unique character string on the same host. The VM name can contain only digits, letters, underscores (\_), hyphens (-), and colons (:), but cannot contain only digits. The VM name can contain a maximum of 64 characters.

### Configuration Example

For example, if the VM name is  **openEuler**, the configuration is as follows:

```xml
<domain type='kvm'>
    <name>openEuler</name>
    ...
</domain>
```

## vCPU and Virtual Memory

### Overview

This section describes how to configure the vCPU and virtual memory.

### Elements

* **vcpu**: The number of virtual processors.

* **memory**: The size of the virtual memory.

  **unit**: The memory unit. The value can be KiB (210  bytes), MiB (220  bytes), GiB (230  bytes), or TiB (240  bytes).

* **cpu**: The mode of the virtual processor.

  **mode**: The mode of the vCPU.

  * **host-passthrough**: indicates that the architecture and features of the virtual CPU are the same as those of the host.

  * **custom**: indicates that the architecture and features of the virtual CPU are configured by the **cpu** element.

  Sub-element  **topology**: A sub-element of the element cpu, used to describe the topology structure of a vCPU mode.

  * The attributes **socket**, **cores**, and **threads** of the sub-element topology describe the number of CPU sockets of a VM, the number of processor cores included in each CPU socket, and the number of threads included in each processor core, respectively. The attribute value is a positive integer, and the product of the three values equals the number of vCPUs.
  * The ARM architecture supports the virtual hyper-threading function. The virtual CPU hot add and the virtual hyper-threading function are mutually exclusive.
    Sub-element  **model**: A sub-element of the element cpu, used to describe the CPU model when **mode** is custom.

  Sub-element  **feature**: A sub-element of the element cpu, used to enable/disable a CPU feature when **mode** is custom.  The attribute **name** describes the name of the CPU feature.  And whether enable the CPU feature is controlled by the attribute **policy**:

  * **force**: force enable the CPU feature regardless of it being supported by host CPU.

  * **require**: enable the CPU feature.

  * **optional**: the CPU feature will be enabled if and only if it is supported by host CPU.

  * **disable**: disable the CPU feature.

  * **forbid**: disable the CPU feature and guest creation will fail if the feature is supported by host CPU.

### Configuration Example

For example, if the number of vCPUs is 4, the processing mode is host-passthrough, the virtual memory is 8 GiB, the four CPUs are distributed in two CPU sockets, and hyperthreading  is not supported, the configuration is as follows:

```xml
<domain type='kvm'>
    ...
    <vcpu>4</vcpu>
    <memory unit='GiB'>8</memory>
    <cpu mode='host-passthrough'>
        <topology sockets='2' cores='2' threads='1'/>
    </cpu>
...
</domain>
```

If the virtual memory is 8 GiB, the number of vCPUs is 4, the processing mode is custom, the CPU model is Kunpeng-920, and pmull is disabled, the configuration is as follows:

```xml
<domain type='kvm'>
    ...
    <vcpu>4</vcpu>
    <memory unit='GiB'>8</memory>
    <cpu mode='custom'>
        <model>Kunpeng-920</model>
        <feature policy='disable' name='pmull'/>
    </cpu>
    ...
</domain>
```

## Virtual Device Configuration

The VM XML configuration file uses the  **devices**  elements to configure virtual devices, including storage devices, network devices, buses, and mouse devices. This section describes how to configure common virtual devices.

### Storage Devices

#### Overview

This section describes how to configure virtual storage devices, including floppy disks, disks, and CD-ROMs and their storage types.

#### Elements

The XML configuration file uses the  **disk**  element to configure storage devices.  [Table 1](#table14200183410353)  describes common  **disk**  attributes.  [Table 2](#table4866134925114)  describes common subelements and their attributes.

**Table  1**  Common attributes of the  **disk**  element

**Table  2**  Common subelements and attributes of the  **disk**  element

| Subelement | Subelement Description                                       | Attribute Description                                        |
| ---------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| source     | Specifies the backend storage medium, which corresponds to the type specified by the **type** attribute of the **disk** element. | **file**: file type. The value is the fully qualified path of the corresponding file.**dev**: block type. The value is the fully qualified path of the corresponding host device.**dir**: directory type. The value is the fully qualified path of the disk directory.**protocol**: protocol in use. **name**: RBD disk name. The format is as follows: $pool/$volume .**host name**: mon address. **port**: port of the mon address. |
| driver     | Details about the specified backend driver                   | **type**: disk format type. The value can be **raw** or **qcow2**, which must be the same as that of source.**io**: disk I/O mode. The options are **native** and **threads**.**cache**: disk cache mode. The value can be **none**, **writethrough**, **writeback**, or **directsync**.**iothread**: I/O thread allocated to the disk.**error\_policy**: processing policy when an I/O write error occurs. The value can be stop, report, ignore, enospace, or retry. **rerror\_policy**: processing policy when an I/O read error occurs. The value can be stop, report, ignore, enospac, or retry. **retry\_interval**: I/O retry interval. The value ranges from 0 to MAX\_INT, in milliseconds. This parameter can be set only when error\_policy or rerror\_policy is set to retry. **retry\_timeout**: I/O retry timeout interval. The value ranges from 0 to MAX\_INT, in milliseconds. This parameter can be set only when error\_policy or rerror\_policy is set to retry. |
| target     | The bus and device that a disk presents to a VM.             | **dev**: specifies the logical device name of a disk, for example, sd\[a-p] for SCSI, SATA, and USB buses and hd\[a-d] for IDE disks.**bus**: specifies the type of a disk. Common types include scsi, usb, sata, and virtio. |
| boot       | The disk can be used as the boot disk.                       | **order**: specifies the disk startup sequence.              |
| readonly   | The disk is read-only and cannot be modified by the VM. Generally, it is used together with the CD-ROM drive. | -                                                            |

#### Configuration Example

After the VM image is prepared according to  [Preparing a VM Image](./environment_preparation.md#preparing-a-vm-image), you can use the following XML configuration file to configure the virtual disk for the VM.

For example, this example configures two I/O threads for the virtual machine, one for a block disk device, one for an optical disc device, and one for an RBD disk, and the first I/O thread is allocated to the block disk device for use. The backend medium of the disk device is in qcow2 format and is used as the preferred boot disk.
Before using an RBD disk, ensure that the qemu-block-rbd driver is installed. Run the following command as the **root** user to install the driver:

```sh
yum install qemu-block-rbd
```

Configuration example:

```xml
<domain type='kvm'>
    ...
    <iothreads>2</iothreads>
    <devices>
        <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2' cache='none' io='native' iothread="1"/>
        <source file='/mnt/openEuler-image.qcow2'/>
        <target dev='vda' bus='virtio'/>
        <boot order='1'/>
    </disk>
    <disk type='file' device='cdrom'>
        <driver name='qemu' type='raw' cache='none' io='native'/>
        <source file='/mnt/openEuler-22.03-LTS-SP4-aarch64-dvd.iso'/>
        <target dev='sdb' bus='scsi'/>
        <readonly/>
        <boot order='2'/>
    </disk>
    <disk type='network' device='disk'>
        <driver name='qemu' type='raw' cache='none' />
        <source protocol="rbd" name="rbd/vol2"/>
        <host name="192.168.0.2" port="6789" />
        <target dev='sdc' bus='scsi'/>
        <boot order='3'/>
    </disk>
         ...
    </devices>
</domain>
```

### Network Devices

#### Overview

The XML configuration file can be used to configure virtual network devices, including the ethernet mode, bridge mode, and vhostuser mode. This section describes how to configure vNICs.

#### Elements

In the XML configuration file, the element  **interface**  is used, and its attribute  **type**  indicates the mode of the vNIC. The options are  **ethernet**,  **bridge**, and  **vhostuser**. The following uses the vNIC in bridge mode as an example to describe its subelements and attributes.

**Table  1**  Common subelements of a vNIC in bridge mode

#### Configuration Example

* After creating the Linux bridge br0 by referring to  [Preparing a VM Image](./environment_preparation.md#preparing-a-vm-image), configure a vNIC of the VirtIO type bridged on the br0 bridge. The corresponding XML configuration is as follows:

  ```xml
  <domain type='kvm'>
      ...
      <devices>
          <interface type='bridge'>
       <source bridge='br0'/>
              <model type='virtio'/>
   </interface>
          ...
      </devices>
  </domain>
  ```

* After an OVS network bridge is created according to  [Preparing a VM Image](./environment_preparation.md#preparing-a-vm-image), configure a VirtIO vNIC device that uses the vhost driver and has four queues.

  ```xml
  <domain type='kvm'>
      ...
      <devices>
          <interface type='bridge'>
       <source bridge='br0'/>
              <virtualport type='openvswitch'/> 
              <model type='virtio'/>
              <driver name='vhost' queues='4'/> 
   </interface>
          ...
      </devices>
  </domain>
  ```

### Bus Configuration

#### Overview

The bus is a channel for information communication between components of a computer. An external device needs to be mounted to a corresponding bus, and each device is assigned a unique address (specified by the subelement  **address**). Information exchange with another device or a central processing unit (CPU) is completed through the bus network. Common device buses include the ISA bus, PCI bus, USB bus, SCSI bus, and PCIe bus.

The PCIe bus is a typical tree structure and has good scalability. The buses are associated with each other by using a controller. The following uses the PCIe bus as an example to describe how to configure a bus topology for a VM.

> \[!NOTE] **NOTE:**
> The bus configuration is complex. If the device topology does not need to be precisely controlled, the default bus configuration automatically generated by libvirt can be used.

#### Elements

In the XML configuration of libvirt, each controller element (**controller**) represents a bus, and one or more controllers or devices can be mounted to one controller depending on the VM architecture. This topic describes common attributes and subelements.

**controller**: controller element, which indicates a bus.

* Attribute  **type**: bus type, which is mandatory for the controller. The common values are  **pci**,  **usb**,  **scsi**,  **virtio-serial**,  **fdc**, and  **ccid**.

* Attribute  **index**: bus number of the controller (the number starts from 0), which is mandatory for the controller. This attribute can be used in the  **address**  element.

* Attribute  **model**: specific model of the controller, which is mandatory for the controller. The available values are related to the value of  **type**. For details about the mapping and description, see  [Table 4](#table191911761111).

* Subelement  **address**: mount location of a device or controller on the bus network.
  * Attribute  **type**: device address type. The common values are  **pci**,  **usb**, or  **drive**. The attribute varies according to the  **type**  of the  **address**. For details about the common  **type**  attribute value and the corresponding  **address**  attribute, see  [Table 5](#table1200165711314).

* Subelement  **model**: name of a controller model.
  * Attribute  **name**: name of a controller model, which corresponds to the  **model**  attribute in the parent element controller.

**Table  4**  Mapping between the common values of  **type**  and  **model**  for the controller.

**Table  5**  Attributes of the  **address**  element in different devices.

#### Configuration Example

This example shows the topology of a PCIe bus. Three PCIe-Root-Port controllers are mounted to the PCIe root node (BUS 0). The multifunction function is enabled for the first PCIe-Root-Port controller (BUS 1). A PCIe-to-PCI-bridge controller is mounted to the first PCIe-Root-Port controller to form a PCI bus (BUS 3). A virtio-serial device and a USB 2.0 controller are mounted to the PCI bus. A SCSI controller is mounted to the second PCIe-Root-Port controller (BUS 2). No device is mounted to the third PCIe-Root-Port controller (BUS 0). The configuration details are as follows:

```xml
<domain type='kvm'>
    ...
    <devices>
        <controller type='pci' index='0' model='pcie-root'/>
 <controller type='pci' index='1' model='pcie-root-port'>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
 </controller>
 <controller type='pci' index='2' model='pcie-root-port'>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
 </controller>
 <controller type='pci' index='3' model='pcie-to-pci-bridge'>
     <model name='pcie-pci-bridge'/>
     <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
 </controller>
 <controller type='pci' index='4' model='pcie-root-port'>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
 </controller>
 <controller type='scsi' index='0' model='virtio-scsi'>
     <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
 </controller>
 <controller type='virtio-serial' index='0'>
     <address type='pci' domain='0x0000' bus='0x03' slot='0x02' function='0x0'/>
 </controller>
 <controller type='usb' index='0' model='ehci'>
     <address type='pci' domain='0x0000' bus='0x03' slot='0x01' function='0x0'/>
 </controller>
 ...
 </devices>
</domain>
```

### Other Common Devices

#### Overview

In addition to storage devices and network devices, some external devices need to be specified in the XML configuration file. This section describes how to configure these elements.

#### Elements

* **serial**: serial port device

  Attribute  **type**: specifies the serial port type. The common attribute values are  **pty**,  **tcp**,  **pipe**, and  **file**.

* **video**: media device

  Attribute **type**: media device type. The common attribute value of the AArch64 architecture is  **virtio**, and that of the x86\_64 architecture is  **vga**  or  **cirrus**.

  Subelement  **model**: subelement of  **video**, which is used to specify the media device type.

  In the subelement  **model**, if  **type**  is set to  **vga**, a Video Graphics Array (VGA) video card is configured.  **vram**  indicates the size of the video RAM, in KB by default.

  For example, if a 16 MB VGA video card is configured for an x86\_64 VM, configuration in the XML file is as follows. In the example, the value of  **vram**  is the size of video RAM, in KB by default.

  ```xml
  <video>
      <model type='vga' vram='16384' heads='1' primary='yes'/>
  </video>
  ```

* **input**: input device

  **type**  attribute: specifies the type of the input device. The common attribute values are  **tablet**  and  **keyboard**, indicating that the output device is the tablet and keyboard respectively.

  **bus**: specifies the bus to be mounted. The common attribute value is  **USB**.

* **emulator**: emulator application path

* **graphics**: graphics device

  **type**  attribute: specifies the type of a graphics device. The common attribute value is  **vnc**.

  **listen**  attribute: specifies the IP address to be listened to.

#### Configuration Example

For example, in the following example, the VM emulator path, pty serial port, VirtIO media device, USB tablet, USB keyboard, and VNC graphics device are configured.

> \[!NOTE] **NOTE:**
> When  **type**  of  **graphics**  is set to  **VNC**, you are advised to set the  **passwd**  attribute, that is, the password for logging in to the VM using VNC.

```xml
<domain type='kvm'>
    ...
    <devices>
        <emulator>/usr/libexec/qemu-kvm</emulator>
        <console type='pty'/>
        <video>
            <model type='virtio'/>
        </video>
        <input type='tablet' bus='usb'/>
        <input type='keyboard' bus='usb'/>
        <graphics type='vnc' listen='0.0.0.0' passwd='n8VfjbFK'/>
 ...
 </devices>
</domain>
```

## Configurations Related to the System Architecture

### Overview

The XML configuration file contains configurations related to the system architecture, which cover the mainboard, CPU, and some features related to the architecture. This section describes meanings of these configurations.

### Elements

* **os**: defines VM startup parameters.

  Subelement  **type**: specifies the VM type. The attribute  **arch**  indicates the architecture type, for example, AArch64. The attribute  **machine**  indicates the type of VM chipset. Supported chipset type can be queried by running the  **qemu-kvm -machine ?**  command. For example, the AArch64 architecture supports the  **virt**  type.

  Subelement  **loader**: specifies the firmware to be loaded, for example, the UEFI file provided by the EDK. The  **readonly**  attribute indicates whether the file is read-only. The value can be  **yes**  or  **no**. The  **type**  attribute indicates the  **loader**  type. The common values are  **rom**  and  **pflash**.

  Subelement  **nvram**: specifies the path of the  **nvram**  file, which is used to store the UEFI startup configuration.

* **features**: Hypervisor controls some VM CPU/machine features, such as the advanced configuration and power interface (ACPI) and the GICv3 interrupt controller specified by the ARM processor.

### Example for AArch64 Architecture

The VM is of the  **aarch64**  type and uses  **virt**  chipset. The VM configuration started using UEFI is as follows:

```xml
<domain type='kvm'>
    ...
    <os>
        <type arch='aarch64' machine='virt'>hvm</type>
        <loader readonly='yes' type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader>
        <nvram>/var/lib/libvirt/qemu/nvram/openEulerVM.fd</nvram>
    </os>
    ...
</domain>
```

Configure ACPI and GIC V3 interrupt controller features for the VM.

```xml
<features>
    <acpi/>
    <gic version='3'/>
</features>
```

### Example for x86\_64 Architecture

The x86\_64 architecture supports both BIOS and UEFI boot modes. If  **loader**  is not configured, the default BIOS boot mode is used. The following is a configuration example in which the UEFI boot mode and Q35 chipsets are used.

```xml
<domain type='kvm'>
    ...
    <os>
        <type arch='x86_64' machine='q35'>hvm</type>
        <loader type='rom'>/usr/share/edk2/ovmf/OVMF.fd</loader>
    </os>
    ...
</domain>
```

## Other Common Configuration Items

### Overview

In addition to system resources and virtual devices, other elements need to be configured in the XML configuration file. This section describes how to configure these elements.

### Elements

* **iothreads**: specifies the number of  **iothread**, which can be used to accelerate storage device performance.

* **on\_poweroff**: action taken when a VM is powered off.

* **on\_reboot**: action taken when a VM is rebooted.

* **on\_crash**: action taken when a VM is on crash.

* **clock**: indicates the clock type.

  **offset**  attribute: specifies the VM clock synchronization type. The value can be  **localtime**,  **utc**,  **timezone**, or  **variable**.

### Configuration Example

Configure two  **iothread**  for the VM to accelerate storage device performance.

```xml
<iothreads>2</iothreads>
```

Destroy the VM when it is powered off.

```xml
<on_poweroff>destroy</on_poweroff>
```

Restart the VM.

```xml
<on_reboot>restart</on_reboot>
```

Restart the VM when it is crashed.

```xml
<on_crash>restart</on_crash>
```

The clock uses the  **utc**  synchronization mode.

```xml
<clock offset='utc'/>
```

## XML Configuration File Example

### Overview

This section provides XML configuration files of a basic AArch64 VM and a x86\_64 VM as two examples for reference.

### Example 1

An XML configuration file of AArch64 VM, which contains basic elements. The following is a configuration example:

```xml
<domain type='kvm'>
    <name>openEulerVM</name>
    <memory unit='GiB'>8</memory>
    <vcpu>4</vcpu>
    <os>
    <type arch='aarch64' machine='virt'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/openEulerVM.fd</nvram>
    </os>
    <features>
    <acpi/>
    <gic version='3'/>
    </features>
    <cpu mode='host-passthrough'>
        <topology sockets='2' cores='2' threads='1'/>
    </cpu>
    <iothreads>1</iothreads>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>restart</on_crash>
    <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2' iothread="1"/>
        <source file='/mnt/openEuler-image.qcow2'/>
        <target dev='vda' bus='virtio'/>
        <boot order='1'/>
    </disk>
    <disk type='file' device='cdrom'>
        <driver name='qemu' type='raw'/>
        <source file='/mnt/openEuler-22.03-LTS-SP4-aarch64-dvd.iso'/>
        <readonly/>
        <target dev='sdb' bus='scsi'/>
        <boot order='2'/>
    </disk>
    <interface type='bridge'>
        <source bridge='br0'/>
        <model type='virtio'/>
    </interface>
    <console type='pty'/>
        <video>
           <model type='virtio'/>
        </video>
        <controller type='scsi' index='0' model='virtio-scsi'/>
    <controller type='usb' model='ehci'/>
    <input type='tablet' bus='usb'/>
    <input type='keyboard' bus='usb'/>
    <graphics type='vnc' listen='0.0.0.0' passwd='n8VfjbFK'/>
    </devices>
    <seclabel type='dynamic' model='dac' relabel='yes'/>
</domain>
```

### Example 2

An XML configuration file of x86\_64 VM, which contains basic elements and bus elements. The following is a configuration example:

```xml
<domain type='kvm'>
  <name>openEulerVM</name>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>8388608</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <iothreads>1</iothreads>
  <os>
    <type arch='x86_64' machine='pc-i440fx-4.0'>hvm</type>
  </os>
  <features>
    <acpi/>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='2' cores='2' threads='1'/>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' iothread='1'/>
      <source file='/mnt/openEuler-image.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </disk>
    <controller type='scsi' index='0' model='virtio-scsi'>
    </controller>
    <controller type='virtio-serial' index='0'>
    </controller>
    <controller type='usb' index='0' model='ehci'>
    </controller>
    <controller type='sata' index='0'>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <interface type='bridge'>
      <mac address='52:54:00:c1:c4:23'/>
      <source bridge='virbr0'/>
      <model type='virtio'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='keyboard' bus='usb'>
      <address type='usb' bus='0' port='2'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='vga' vram='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
    </memballoon>
  </devices>
</domain>
```

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/vm_live_migration.md
---
# VM Live Migration

## Introduction

### Overview

When a VM is running on a physical machine, the physical machine may be overloaded or underloaded due to uneven resource allocation. In addition, operations such as hardware replacement, software upgrade, networking adjustment, and troubleshooting are performed on the physical machine. Therefore, it is important to complete these operations without interrupting services. The VM live migration technology implements load balancing or the preceding operations on the premise of service continuity, improving user experience and work efficiency. VM live migration is to save the running status of the entire VM and quickly restore the VM to the original or even different hardware platforms. After the VM is restored, it can still run smoothly without any difference to users. Because the VM data can be stored on the current host or a shared remote storage device, openEuler supports shared and non-shared storage live migration.

## Application Scenarios

Shared and non-shared storage live migration applies to the following scenarios:

* When a physical machine is faulty or overloaded, you can migrate the running VM to another physical machine to prevent service interruption and ensure normal service running.
* When most physical machines are underloaded, migrate and integrate VMs to reduce the number of physical machines and improve resource utilization.
* When the hardware of a physical server becomes a bottleneck, such as the CPU, memory, and hard disk, replace the hardware with better performance or add devices. However, you cannot stop the VM or stop services.
* Server software upgrade, such as virtualization platform upgrade, allows the VM to be live migrated from the old virtualization platform to the new one.

Non-shared storage live migration can also be used in the following scenarios:

* If a physical machine is faulty and the storage space is insufficient, migrate the running VM to another physical machine to prevent service interruption and ensure normal service running.
* When the storage device of the physical machine is aged, the performance cannot support the current service data processing and becomes the bottleneck of the system performance. In this case, a storage device with higher performance needs to be used, but the VM cannot be shut down or stopped. The running VM needs to be migrated to a physical machine with better performance.

## Precautions and Restrictions

* During the live migration, ensure that the network is in good condition. If the network is interrupted, live migration is suspended until the network is recovered. If the network connection times out, live migration fails.

* During the migration, do not perform operations such as VM life cycle management and VM hardware device management.

* During VM migration, ensure that the source and destination servers are not powered off or restarted unexpectedly. Otherwise, the live migration fails or the VM may be powered off.

* Do not shut down, restart, or restore the VM during the migration. Otherwise, the live migration may fail. If you perform live migration when the VM is rebooted in ACPI mode, the VM will be shut down.

* Only homogeneous live migration is supported. That is, the CPU models of the source and destination must be the same.

* A VM can be successfully migrated across service network segments. However, network exceptions may occur after the VM is migrated to the destination. To prevent this problem, ensure that the service network segments to be migrated are the same.

* If the number of vCPUs on the source VM is greater than that on the destination physical machine, the VM performance will be affected after the migration. Ensure that the number of vCPUs on the destination physical machine is greater than or equal to that on the source VM.

Precautions for live migration of non-shared storage:

* The source and destination cannot be the same disk image file. You need to perform special processing on such migration to prevent image damage caused by data overwriting.
* Shared disks cannot be migrated. You need to perform foolproof operations on such migration.
* The destination image supports only files and does not support raw devices. You need to perform foolproof processing on the migration of raw devices.
* The size and number of disk images created on the destination must be the same as those on the source. Otherwise, the migration fails.
* In hybrid migration scenarios, the disks to be migrated must not include shared and read-only disks.

## Live Migration Operations

### Prerequisites

* Before live migration, ensure that the source and destination hosts can communicate with each other and have the same resource permissions. That is, the source and destination hosts can access the same storage and network resources.

* Before VM live migration, perform a health check on the VM and ensure that the destination host has sufficient CPU, memory, and storage resources.

### Prediction of the Dirty Page Rate During Live Migration (Optional)

Before migrating a VM, you can use the dirtyrate function to obtain the dirty page change rate of the live migration and evaluate whether the VM is suitable for migration or configure proper migration parameters based on the VM memory usage.

Procedure:

For example, if the VM name is **openEulerVM** and the calculation time is 1s, run the following command:

```shell
virsh qemu-monitor-command openEulerVM '{"execute":"calc-dirty-rate", "arguments": {"calc-time": 1}}'
```

After 1s, run the following command to query the dirty page change rate:

```shell
virsh qemu-monitor-command openEulerVM '{"execute":"query-dirty-rate"}'
```

### (Optional) Setting Live Migration Parameters

Before live migration, run the  **virsh migrate-setmaxdowntime**  command to specify the maximum tolerable downtime during VM live migration. This is an optional configuration item.

For example, to set the maximum downtime of the VM named  **openEulerVM**  to  **500 ms**, run the following command:

```shell
virsh migrate-setmaxdowntime openEulerVM 500
```

In addition, you can run the  **virsh migrate-setspeed**  command to limit the bandwidth occupied by VM live migration. This prevents VM live migration from occupying too much bandwidth and affecting other VMs or services on the host. This operation is also optional for live migration.

For example, to set the live migration bandwidth of the VM named  **openEulerVM**  to  **500 Mbit/s**, run the following command:

```shell
virsh migrate-setspeed openEulerVM --bandwidth 500
```

You can run the  **migrate-getspeed**  command to query the maximum bandwidth during VM live migration.

```shell
$ virsh migrate-getspeed openEulerVM
500
```

You can use migrate-set-parameters to set parameters related to live migration. The following table lists the parameters related to live migration compression:

1. compress-level: Compression level. The default value is 1.
2. compress-threads: Number of compression threads. The default value is 8.
3. compress-wait-thread: Whether to wait for the compression thread. The default value is true.
4. decompress-threads: Number of decompression threads. The default value is 2.
5. compress-method: Compression algorithm (zlib or zstd). The default value is zlib.

For example, set the live migration algorithm of the VM named *openEulerVM* to zstd and retain the default values for other parameters.

```shell
virsh qemu-monitor-command openeulerVM '{ "execute": "migrate-set-parameters", "arguments": {"compress-method": "zstd"}}'
```

You can run the query-migrate-parameters command to query parameters related to live migration.

```shell
$ virsh qemu-monitor-command openeulerVM '{ "execute": "query-migrate-parameters"}' --pretty

{
  "return": {
    "xbzrle-cache-size": 67108864,
    "cpu-throttle-initial": 20,
    "announce-max": 550,
    "decompress-threads": 2,
    "compress-threads": 8,
    "compress-level": 1,
    "compress-method": "zstd",
    "multifd-channels": 2,
    "announce-initial": 50,
    "block-incremental": false,
    "compress-wait-thread": true,
    "downtime-limit": 300,
    "tls-authz": "",
    "announce-rounds": 5,
    "announce-step": 100,
    "tls-creds": "",
    "max-cpu-throttle": 99,
    "max-postcopy-bandwidth": 0,
    "tls-hostname": "",
    "max-bandwidth": 33554432,
    "x-checkpoint-delay": 20000,
    "cpu-throttle-increment": 10
  },
  "id": "libvirt-18"
}
```

### Live Migration Operations (Shared Storage Scenario)

1. Check whether the storage device is shared.

   ```shell
   $ virsh domblklist <VMInstance>
    Target   Source
   --------------------------------------------
   sda      /dev/mapper/open_euleros_disk
   sdb      /mnt/nfs/images/openeuler-test.qcow2
   ```

   Run the  **virsh domblklist**  command to query the storage device information of the VM. For example, the preceding query result shows that the VM is configured with two storage devices: sda and sdb. Then, check whether the backend storage of the two devices is local storage or remote storage, if all storage devices are on the remote shared storage, the VM is a shared storage VM. Otherwise, the VM is a non-shared storage VM.

2. Run the following command for VM live migration:

   For example, run the  **virsh migrate**  command to migrate VM  **openEulerVM**  to the destination host.

   ```shell
   virsh migrate --live --unsafe openEulerVM qemu+ssh://<destination-host-ip>/system
   ```

   **\<destination-host-ip>**  indicates the IP address of the destination host. Before live migration, SSH authentication must be performed to obtain the source host management permission.

   In addition, the  **virsh migrate**  command has the  **--auto-converge**  and  **--timeout**  sub-options to ensure successful migration.

   Related sub-options:

   The  **--unsafe**  command forcibly performs live migration and skips the security check step.

   The  **--auto-converge**  command reduces the CPU frequency to ensure that the live migration process can be converged.

   The  **--timeout**  command specifies the live migration timeout period. If the live migration exceeds the specified period, the VM is forcibly suspended to reduce the live migration.

3. After the live migration is complete, the VM is running properly on the destination host.

### Live Migration Operations (Non-Shared Storage Scenario)

1. Query the VM storage device list to ensure that the VM uses non-shared storage.

   For example, the  **virsh domblklist**  command output shows that the VM to be migrated has a disk sda in qcow2 format. The XML configuration of sda is as follows:

   ```xml
       <disk type='file' device='disk'>
         <driver name='qemu' type='qcow2' cache='none' io='native'/>
         <source file='/mnt/sdb/openeuler/openEulerVM.qcow2'/>
         <target dev='sda' bus='scsi'/>
         <address type='drive' controller='0' bus='0' target='0' unit='0'/>
       </disk>
   ```

   Before live migration, create a virtual disk file in the same disk directory on the destination host. Ensure that the disk format and size are the same.

   ```shell
   qemu-img create -f qcow2 /mnt/sdb/openeuler/openEulerVM.qcow2 20G
   ```

2. Run the  **virsh migrate**  command on the source to perform live migration. During the migration, the storage is also migrated to the destination.

   ```shell
   $ virsh migrate --live  --unsafe --copy-storage-all --migrate-disks sda \
   openEulerVM qemu+ssh://<dest-host-ip>/system
   ```

3. After the live migration is complete, the command output indicates that the VM is running properly on the destination host and the storage device is migrated to the destination host.

### Live Migration Operations (Encrypted Transmission)

1. Overview

   To better encrypt data during VM live migration, openEuler provides the TLS encryption feature. Almost all network services in QEMU can use TLS to encrypt session data and use X509 certificates to perform simple identity authentication on clients.

2. Application Scenarios

   A typical application scenario is to ensure data security when VM data is transmitted between the source end and the target end during live migration.

3. Precautions

   Before using TLS to live migrate VMs, you need to apply for certificates and set certificates on the source and destination ends. Before using the TLS function, you need to enable the peer authentication configuration item and set migrate\_tls\_x509\_verify to 1 in /etc/libvirt/qemu.conf.

   The service interruption duration and migration duration of single-channel TLS live migration increase significantly. The upper limit of the migration bandwidth is 100 MB/s to 200 MB/s. As a result, the migration may fail.

   The multiFd can be used to perform multi-channel TLS migration. However, the CPU overhead increases (two more migration threads are enabled), which may affect VM running. You are advised to set the CPU affinity of the live migration thread to isolate the CPU resources used by the live migration thread from the CPU resources bound to the VM process. You are advised to bind two CPUs to each VM to be migrated.

4. How to Use

   Single-channel live migration encryption transmission command

```shell
virsh migrate --live --unsafe --tls --domain openEulerVM --desturi qemu+tcp://<destination-host-ip>/system --migrateuri tcp://<destination-host-ip>
```

Encrypted transmission commands for multi-channel live migration

```shell
virsh migrate --live --unsafe --parallel --tls --domain openEulerVM --desturi qemu+tcp://<destination-host-ip>/system --migrateuri tcp://<destination-host-ip>
```

### Live Migration Operations (Migration Core Pinning)

1. Overview

   To improve migration performance under certain conditions, you need to configure CPU affinity for live migration threads. By default, a live migration thread shares CPU resources with a VM process. This function allows you to set the CPU resources available to the live migration thread beyond the CPU range pinned to the VM process.

2. Application Scenarios

   A typical application scenario is that the CPU pinned to the migration thread can be controlled during live migration to prevent other services from being affected due to CPU resource contention during live migration.

3. Precautions

   At present, you need to run the corresponding script to set the parameters for migration core pinning. The script is at the end of this section.

4. How to Use

   Run the migration core pinning script:

```shell
python migrateToURI3.py domain_name dst_ip xml
```

Python script:

```python
#encoding=UTF-8

import libvirt
import sys

conn = None
domain = None

default_bandwidth = 30

default_flags = 27 + 512 # To perform multifd migration, add 131072 to the flags.

if len(sys.argv) < 4:
        print("usage: python migrateToURI3.py domain_name dst_ip xml")
        sys.exit(1)
else:
        try:
                domain_name = sys.argv[1]
                dst_ip = sys.argv[2]
                xml_file = sys.argv[3]
                f = open(xml_file, 'r')
                dst_xml = f.read()
                f.close()
                disks = []

                dconnuri = "qemu+tcp://" +dst_ip + "/system"
                conn = libvirt.open("qemu:///system")
                domain = conn.lookupByName(domain_name)
                params = {}
                params["migrate_uri"] = "tcp://" + dst_ip
                params["destination_xml"] = dst_xml
                params["persistent_xml"] = dst_xml
                params["bandwidth"] = default_bandwidth
                params["migration.pin"] = "0-1"

                flags = default_flags
                if len(sys.argv) > 4 and sys.argv[4].isdigit():
                        flags = int(sys.argv[4])
                        start = 5
                else:
                        start = 4
                for i in range(start, len(sys.argv)):
                        disks.append(sys.argv[i])
                if len(disks) > 0:
                        params["migrate_disks"] = disks
                domain.migrateToURI3(dconnuri, params, flags)
        except:
                error = libvirt.virGetLastError()
                if error == None:
                        print ("error_number is NULL.")
                        raise
                else:
                        print ('%s err_code %s' %(domain_name, error[0]))
                        print ('%s err_domain %s' %(domain_name, error[1]))
                        print ('%s err_message %s' %(domain_name, error[2]))
                        print ('%s err_level %s' %(domain_name, error[3]))
                        print ('%s err_str1 %s' %(domain_name, error[4]))
                        print ('%s err_str2 %s' %(domain_name, error[5]))
                        print ('%s err_str3 %s' %(domain_name, error[6]))
                        print ('%s err_int1 %s' %(domain_name, error[7]))
                        print ('%s err_int2 %s' %(domain_name, error[8]))
        print ('Done')
if domain is not None:
        domain.__del__()
if conn is not None:
        conn.close()
```

### Live Migration Operations (Dirty Block Chunking)

1. Overview

   To improve the VM migration performance, the current kernel of openEuler supports dirty VM blocks on huge pages, which effectively reduces the performance loss during the migration. Dirty block chunking is that with KVM\_DIRTY\_LOG\_MANUAL\_PROTECT\_ENABLE set, KVM\_GET\_DIRTY\_LOG will not automatically clear and write-protect all pages that are returned as dirty. Rather, the user space will have to do this operation separately using KVM\_CLEAR\_DIRTY\_LOG.

2. Application Scenarios

   A typical application scenario is hugepage VM migration.

3. Precautions

   None.

4. How to Use

   Enable KVM\_DIRTY\_LOG\_MANUAL\_PROTECT\_ENABLE in the kernel.

---

---
url: >-
  /en/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/vm_maintainability_managment.md
---
# VM Maintainability Management

## VM NMI Watchdog

### Overview

The NMI Watchdog is a mechanism used to detect hardlockup in Linux. Even if normal interrupts are disabled, non-maskable interrupt (NMI) can interrupt the code execution and further detect hardlockup. The current Arm architecture does not support native NMI, so it enables Pseudo-NMI based on the interrupt priority and configures Performance Monitoring Interrupt (PMI) as NMI to implement NMI Watchdog (PMU Watchdog).

### Precautions

* The VM OS needs to support Pseudo-NMI and corresponding kernel parameters needs to be configured.
* The configuration of the VM PMU Watchdog is the same as that of the host PMU Watchdog. The XML file configuration is not required.
* Both SDEI Watchdog and PMU Watchdog are NMI Watchdogs, whereas the SDEI Watchdog has a higher priority than PMU Watchdog. Therefore, the PMU Watchdog can be enabled only when the SDEI Watchdog is disabled. The VM does not support SDEI Watchdog, so you need to disable it by configuring kernel parameters.

### Procedure

To configure the NMI Watchdog for a VM in ARM architecture,perform the following steps:

1. Add the following parameters to the VM boot configuration file grub.cfg:

2. Check whether the PMU Watchdog is successfully loaded on the VM.
   If the loading is successful, information similar to the following is displayed in the dmesg log of the kernel:

   ```conf
    [2.1173222] NMI watchdog: CPU0 freq probed as 2399999942 HZ.
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/ai/euler-copilot-framework/witty_assistant/witty_shell/user_guide/user_guide.md
---
# Witty CLI 使用介绍

## 引言

Witty CLI 是 openEuler 终端场景下的 AI 助手。它提供两种使用方式：在普通 Bash 提示符下直接输入自然语言的 **Shell 直输**，以及运行 `witty` 进入的 **REPL** 交互式 CLI。两种方式共享同一套会话、渲染、权限与展示核心。

## 快速上手

### Shell 直输（最简方式）

安装完成后，直接在 Bash 终端输入自然语言即可触发 AI 助手：

```bash
检查系统内存
```

Witty CLI 会自动识别这是自然语言而非 shell 命令，调用 AI 助手并流式输出回答：

```bash
systemctl 怎么看 nginx 日志
```

以已知命令开头但包含中文疑问词的输入，也会被正确路由到 AI 助手。

### REPL 模式

```bash
witty
```

进入交互式 REPL，提示符格式为 `witty [<agent>:<model>] >`（末尾含空格）。可以在其中连续对话：

```text
witty [witty-builtin-agent] > 帮我分析一下 dmesg 里的 OOM 日志
...（流式输出回答）...
witty [witty-builtin-agent] > 那怎么优化内存配置呢
...（基于上一轮上下文继续回答）...
```

输入 `/exit` 或按 `Ctrl+D` 退出 REPL。

### 单次提问

```bash
witty ask "检查系统内存"
```

单次请求，流式输出回答后退出。

### 通过管道传入

```bash
echo "explain this function" | witty ask
```

```bash
dmesg | witty ask "分析这段内核日志中的异常"
```

### 连接 Provider 并切换模型

```bash
witty provider connect deepseek
Enter API key: ****（输入不可见）
connected provider deepseek
```

随后进入 REPL，输入 `/model` 打开模型选择器：

```text
witty [witty-builtin-agent] > /model

Select model:

❯ 1) opencode/north-mini-code-free  — North Mini Code Free
  2) opencode/ling-3.0-flash-free   — Ling-3.0-flash Free
  ...
  8) deepseek/deepseek-v4-flash     — DeepSeek V4 Flash
  9) deepseek/deepseek-v4-pro       — DeepSeek V4 Pro

  ↑/↓ navigate  ↵ select  esc cancel
```

## 命令参考

### 全局选项

以下选项可用于所有子命令：

| 选项 | 类型 | 默认值 | 说明 |
| ---- | ---- | ------ | ---- |
| `--config` | string | `""` | 指定配置文件路径 |
| `--server-url` | string | `""` | 指定 opencode server 地址 |
| `--agent` | string | `""` | 指定默认 agent |
| `--model` | string | `""` | 指定默认模型（格式 `provider/model`） |
| `--variant` | string | `""` | 指定模型变体（如推理级别） |
| `--debug` | bool | `false` | 启用调试日志 |
| `--no-color` | bool | `false` | 禁用彩色输出 |
| `--version`, `-v` | — | — | 打印版本信息 |

### `witty`（无子命令）— 启动 REPL

```bash
witty
```

启动交互式 REPL，自动恢复当前目录的最近会话；若无则新建。

### `witty ask`

```bash
witty ask [prompt]
```

发送一条消息并流式输出回答，回答结束后退出。

**选项**：

| 选项 | 类型 | 说明 |
| ---- | ---- | ---- |
| `--new` | bool | 创建新会话，不复用当前目录会话 |
| `--session` | string | 继续指定会话 ID |

> `--new` 和 `--session` 不能同时使用。

### `witty init bash`

```bash
eval "$(witty init bash)"
```

输出 Bash 集成脚本到 stdout，通常配合 `eval` 使用。RPM 安装后已通过 `/etc/profile.d/witty.sh` 自动完成集成。

### `witty session`

```bash
witty session list
witty session continue <id>
```

列出全部会话，或切换到指定会话。也可使用顶层快捷命令 `witty continue <id>`。

### `witty provider`

Provider 管理，详见下文「Provider 管理」。

### `witty server`

Server 管理，详见下文「Server 生命周期管理」。

### `witty doctor`

环境诊断，详见下文「环境诊断」。

### `witty version`

打印版本号、commit 和构建日期。

## Shell 集成

### 工作原理

Shell 集成通过 Bash 的 `DEBUG` trap（配合 `extdebug` 选项）实现，在每条命令即将执行前拦截，由分类器判断路由：

| 路由 | 行为 |
| ---- | ---- |
| **shell** | 正常交给 Bash 执行，零侵入 |
| **agent** | 转交给 `witty ask` 处理 |
| **control** | 转交给 `witty` 解释 slash 控制命令 |

分类遵循「Shell 优先、Agent 兜底」原则——无法明确识别为自然语言的输入默认交给 Shell 执行。

### 路由示例

| 输入 | 路由 | 判定依据 |
| ---- | ---- | -------- |
| `检查系统内存` | Agent | CJK 字符 |
| `systemctl 怎么看 nginx 日志` | Agent | 中文触发词「怎么看」 |
| `systemctl status nginx` | Shell | 已知命令 + 无自然语言特征 |
| `grep error /var/log/messages` | Shell | 已知命令 + 无自然语言特征 |
| `cat /etc/os-release \| grep NAME` | Shell | 管道 |
| `/session` | Control | 白名单 slash 命令，显示 `/session` 详细用法 |
| `/ask systemctl 怎么看 nginx 日志` | Control | 白名单 slash 命令，内部强制走 Agent 提问 |
| `/usr/bin/ls` | Shell | 显式路径 |
| `for i in 1; do` | Shell | Shell 语法（含 `;` 等强语法特征） |
| `how do I restart nginx` | Agent | 英文触发词 |

### 启用与禁用

* **默认状态**：RPM 安装后对所有用户默认启用。
* **临时禁用**（当前终端会话）：`export WITTY_SHELL_ENABLE=0`
* **永久禁用**：在 `~/.bashrc` 中添加 `export WITTY_SHELL_ENABLE=0`
* **调试模式**：`export WITTY_SHELL_DEBUG=1`，在 stderr 输出分类结果

### 强制走 Agent

如果分类器误将自然语言识别为 shell 命令，可使用 `/ask` 逃生口强制走 Agent：

```bash
/ask docker 镜像怎么删
```

### 安装前提

Shell 集成仅在以下条件满足时安装：当前 shell 为 Bash、交互式会话（`$-` 含 `i`）、stdin/stdout 为 TTY。非交互式脚本、zsh / fish 等不支持的 shell 不会安装集成。

## Slash 控制命令

在 REPL 和 Shell 直输模式下均可使用以下 slash 命令：

| 命令 | 作用 |
| ---- | ---- |
| `/help` | 显示帮助 |
| `/exit`、`/quit`、`/q` | 退出 REPL（不接受参数） |
| `/ask <prompt>` | 强制将当前输入作为 Agent 请求 |
| `/agent [name]` | 显示或切换默认 agent |
| `/model [provider/model]` | 显示或切换默认模型 |
| `/new` | 下一次提问时新建会话 |
| `/session` | 显示 `/session` 详细用法 |
| `/session list` | 列出会话 |
| `/session continue <id>` | 继续指定会话 |

### `/agent` — 切换 Agent

不带参数时，如果当前是 TTY 终端，会显示交互式 agent 选择器（列出所有非隐藏、非 subagent 型 agent）。带参数时直接切换：

```text
/agent build
```

切换后会持久化到用户配置文件：

```text
[agent] set to "build" (saved to ~/.config/witty/config.toml)
```

### `/model` — 切换模型

模型格式必须为 `provider/model`：

```text
/model openai/gpt-4
```

不带参数时显示交互式模型选择器，仅列出已连接 provider 的模型。如果选中的模型支持变体（variant），会进一步显示变体选择器。选择完成后输出确认并持久化到配置文件：

```text
[model] set to "deepseek/deepseek-v4-flash" (variant: max, saved to ~/.config/witty/config.toml)
```

### 参数校验规则

| 命令 | 参数要求 |
| ---- | -------- |
| `/ask` | **必须**带 prompt |
| `/agent` | 可选参数 |
| `/model` | 可选参数 |
| `/session` | 无参数，显示 `/session` 详细用法 |
| `/session list` | 无参数 |
| `/session continue` | **必须**带 session id |
| `/new` | 无参数 |
| `/help` | 无参数 |
| `/exit`、`/quit`、`/q` | 无参数 |

## 配置

### 配置文件

Witty CLI 使用 TOML 格式配置文件，搜索顺序（后者覆盖前者）：

1. `/etc/witty/config.toml`（系统级配置）
2. `~/.config/witty/config.toml`（用户级配置）

也可通过 `--config <path>` 或环境变量 `WITTY_CONFIG` 指定。

### 完整配置项

```toml
# 顶层配置
default_agent = "witty-builtin-agent"  # 默认 agent
default_model = ""                     # 默认模型（格式 provider/model）
default_variant = ""                   # 模型变体（如推理级别）
debug = false                          # 调试日志
theme = "auto"                         # 主题（auto）
no_color = false                       # 禁用彩色输出
stream_mode = 1                        # 流式输出模式（1=分段渲染，2=逐字回显）

# Server 生命周期管理
[server]
auto_start = true              # 自动启动 opencode serve
port = 0                       # 0 = 自动选择端口，正整数 = 固定端口
hostname = "127.0.0.1"         # 绑定地址
startup_timeout_seconds = 10   # 等待 server 启动的最大时间（秒）
idle_timeout_minutes = 30      # 闲置超时自动停止（0 = 禁用）

# REPL 配置
[repl]
auto_resume = true             # 自动恢复当前目录最近会话

# Shell 集成配置
[shell]
enabled = true                 # 启用 Shell 集成
debug = false                  # Shell 集成调试模式

# 诊断配置
[doctor]
timeout_seconds = 5            # 诊断超时时间（秒）

# 显示配置
[display]
show_reasoning = "show"        # 推理过程展示：show / minimal / hide
tool_mode = "compact"          # 工具调用展示：compact / verbose
group_context_tools = true     # 合并连续的读取类工具调用展示
step_style = "line"            # 步骤展示风格：line / minimal / none
```

### 配置覆盖类环境变量

环境变量优先于配置文件，CLI 选项优先于环境变量。

| 环境变量 | 对应配置项 | 说明 |
| -------- | ---------- | ---- |
| `WITTY_CONFIG` | — | 指定配置文件路径 |
| `WITTY_AGENT` | `default_agent` | 默认 agent |
| `WITTY_MODEL` | `default_model` | 默认模型 |
| `WITTY_VARIANT` | `default_variant` | 模型变体 |
| `WITTY_DEBUG` | `debug` | 调试日志（`true`/`false`） |
| `WITTY_SHELL_ENABLE` | `shell.enabled` | 启用 Shell 集成（`true`/`false`） |
| `WITTY_SHELL_DEBUG` | `shell.debug` | Shell 集成调试模式（`true`/`false`） |
| `WITTY_SERVER_AUTO_START` | `server.auto_start` | 自动启动（`true`/`false`） |
| `WITTY_SERVER_PORT` | `server.port` | server 端口 |
| `WITTY_SERVER_HOSTNAME` | `server.hostname` | 绑定地址 |
| `WITTY_DISPLAY_SHOW_REASONING` | `display.show_reasoning` | 推理过程展示 |
| `WITTY_DISPLAY_STEP_STYLE` | `display.step_style` | 步骤展示风格 |
| `NO_COLOR` | `no_color` | 设置任意值即禁用彩色输出（遵循 NO\_COLOR 标准） |

## Provider 管理

### 列出 Provider

```bash
witty provider list
```

列出所有支持 API Key 认证的 provider，标注连接状态：

```text
STATUS     ID        NAME          DEFAULT_MODEL
-          openai    OpenAI        gpt-4
connected  deepseek  DeepSeek      deepseek-chat
```

仅列出已连接的 provider：

```bash
witty provider list --connected
```

### 连接 Provider

```bash
witty provider connect <provider> --key <api-key>
```

`<provider>` 可以是 provider 的 ID 或名称（支持不区分大小写匹配）。

**API Key 的传入方式**（按优先级）：

1. `--key` 参数
2. 交互式终端提示（如果 stdin 是 TTY，会提示 `Enter API key:`，输入不可见）
3. stdin 管道读取
4. 环境变量（如 `OPENAI_API_KEY`、`DEEPSEEK_API_KEY`）

连接成功后输出 `connected provider deepseek`。如果 provider 仍不可用，witty 会自动重启 server 以重新加载凭据。

## Server 生命周期管理

Witty CLI 自动管理 `opencode serve` 进程的完整生命周期，通常无需用户干预。

### 自动启动

当 `server.auto_start = true`（默认）时，witty 首次使用时自动启动 `opencode serve` 子进程。启动后：

* server 作为守护进程运行，witty 退出后继续运行
* 状态持久化到 `~/.local/state/witty/server-state.json`（权限 0600）
* 下次启动 witty 时通过状态文件无缝复用

### 查看状态

```bash
witty server status
```

```text
Running:    yes
Port:       4096
PID:        12345
Managed:    no
StartedAt:  2026-08-19T10:30:00+08:00
```

* `Running`：server 是否正在运行——判断 server 状态以此字段为准
* `Managed`：表示「是否由当前进程启动」；由于 `witty server status` 是只读命令，该命令下 `Managed` 恒为 `no`，属预期行为，请以 `Running` 为准

### 停止 / 重启

```bash
witty server stop
witty server restart
```

`witty server restart` 在安装或修改 agent / skill / MCP 配置文件后使用，让 server 重新加载配置：

```bash
sudo dnf install witty-diagnosis-agent
witty server restart
```

### 关闭自动启动

```toml
[server]
auto_start = false
```

关闭后需手动启动 `opencode serve`，witty 将探测该地址寻找已有 server。若需连接其他地址的 server，请使用 `--server-url` CLI 标志。

### 闲置超时

默认 30 分钟无活动后自动停止 server（`server.idle_timeout_minutes = 30`）。设为 `0` 禁用此功能。

### 多用户安全隔离

同一台服务器上，不同用户的 server 实例互相隔离：

* 每个用户有独立的随机密码（HTTP Basic Auth）
* 状态文件权限 0600，其他用户无法读取
* 密码仅通过环境变量传递给子进程，不出现在命令行参数中

## 环境诊断

```bash
witty doctor
```

检查以下项目，每项状态为 `OK`、`WARN`、`FAIL` 或 `SKIP`：

| 检查项 | 说明 |
| ------ | ---- |
| **config** | 配置文件加载情况、当前 server/agent/model/shell 设置 |
| **server reachable** | opencode server 是否可达 |
| **/doc endpoint** | OpenAPI 文档端点是否可用 |
| **server management** | server 管理状态（managed/discovered/manual） |
| **shell integration** | Shell 集成是否已加载 |
| **bash environment** | `TERM` 设置、Bash 版本（需 ≥ 4.0） |
| **terminal** | stdout TTY 状态、终端宽度、颜色支持 |

`witty doctor` 不会启动 server（无副作用），适合用于排查问题。

## 常见问题

### Shell 直输不生效

1. 运行 `witty doctor`，检查 `shell integration` 项
2. 如果显示 `not loaded`，手动执行 `eval "$(witty init bash)"`
3. 检查是否设置了 `WITTY_SHELL_ENABLE=0`

### 自然语言被当成 shell 命令执行

分类器采用「Shell 优先」策略，部分边界场景可能误判。可使用 `/ask` 强制走 Agent：

```bash
/ask docker 镜像怎么删
```

### 提示 server 不可达

* 检查 `opencode` 是否已安装：`which opencode`
* 运行 `witty doctor` 查看详细诊断
* 尝试 `witty server restart`
* 若关闭了自动启动：报错提示需手动启动 `opencode serve --port 4096`，或通过 `--server-url` 指定地址

### Provider 连接后仍不可用

```bash
witty server restart
witty provider list --connected  # 确认连接状态
```

### 会话上下文丢失

Witty CLI 按当前目录管理会话：

* 同一目录下执行 `witty` 或 Shell 直输会自动恢复最近会话
* 切换目录后会话不共享
* 使用 `/new` 或 `witty ask --new` 可强制新建会话
* 使用 `/session list` 查看可用会话，`/session continue <id>` 切换

### 如何完全禁用 Witty CLI

```bash
export WITTY_SHELL_ENABLE=0   # 禁用 Shell 集成
witty server stop             # 停止后台 server
sudo dnf remove witty         # 卸载
```

卸载后自动移除 `/etc/profile.d/witty.sh`，零残留。

### REPL 中如何取消当前请求

按 `Ctrl+C` 可取消正在进行的 AI 请求（输出 `[cancelled]`），不会退出 REPL。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/ai/euler-copilot-framework/witty_assistant/witty_shell/deploy_guide/deployment.md
---
# Witty CLI 安装指南

本手册适用于 Witty CLI 3.x。当前最新版本已发布至 **openEuler 24.03 LTS SP3 与 SP4**，RPM 包名为 `witty`。

## 环境要求

* **操作系统**：openEuler 24.03 LTS SP3 或更高版本（x86\_64 / aarch64）
* **内存**：建议至少 8GB RAM
* **存储**：建议至少 20GB 可用磁盘空间
* **网络**：稳定的互联网连接（用于安装与拉取弱依赖；如需离线部署请提前准备）
* **大模型服务**：
  * 线上：需具备支持工具调用的线上大模型 API 访问权限，例如百炼、DeepSeek Chat、智谱、MiniMax、硅基流动等
  * 本地：支持部署在本地的 LLM 服务，如 llama-server、ollama、vLLM、LM Studio 等，需使用支持工具调用的大模型，例如 Qwen3、DeepSeek 3.2、GLM-4.7、MiniMax M2.1、Kimi K2 等
* **系统权限**：需具备 sudo 权限，以便安装必要的软件包与依赖项

## 依赖仓库与弱依赖说明

`witty` 的完整能力依赖若干由 **update / everything / EPOL / EPOL update** 仓库提供的软件包。其中部分为 **弱依赖（Recommends）**，默认会随 `witty` 一起安装。以下为在 openEuler 24.03 LTS 上实际解析到的包来源（已排除测试仓库）：

| 软件包 | 来源仓库 | 作用 |
| ------ | -------- | ---- |
| `witty` / `witty-builtin-agent` / `witty-agent-loader`（3.x 主包） | **EPOL/update**（SP4）、**update**（SP3） | Witty CLI 主程序、内置 Agent 与托管配置 loader |
| `opencode` | **update** | opencode CLI，作为 `witty` 的执行后端 |
| `nodejs >= 20` | **everything** | 运行 opencode 与相关工具所需 |
| `euler-copilot-rag` / `witty-lite-rag` / `witty-log-detection` | **EPOL** | RAG、经验技能与日志检测能力 |
| `witty-experience-skill` | **EPOL**（`euler-copilot-rag` 子包） | 经验技能管理服务（skill 核心知识引擎） |

因此，除了默认已启用的 `OS` / `update` / `everything` 仓库外，还需要配置 **EPOL（main）** 与 **EPOL update** 仓库，才能解析完整依赖。

## 配置 repo 源

### openEuler 24.03 LTS SP3

在 `/etc/yum.repos.d/` 下创建仓库配置文件（例如 `witty.repo`），加入以下内容：

```ini
[EPOL]
name=EPOL
baseurl=https://repo.openeuler.org/openEuler-24.03-LTS-SP3/EPOL/main/$basearch/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-24.03-LTS-SP3/OS/$basearch/RPM-GPG-KEY-openEuler

[update-EPOL]
name=update-EPOL
baseurl=https://repo.openeuler.org/openEuler-24.03-LTS-SP3/EPOL/update/main/$basearch/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-24.03-LTS-SP3/OS/$basearch/RPM-GPG-KEY-openEuler
```

### openEuler 24.03 LTS SP4

同理，SP4 将上述路径中的 `openEuler-24.03-LTS-SP3` 替换为 `openEuler-24.03-LTS-SP4`：

```ini
[EPOL]
name=EPOL
baseurl=https://repo.openeuler.org/openEuler-24.03-LTS-SP4/EPOL/main/$basearch/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-24.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler

[update-EPOL]
name=update-EPOL
baseurl=https://repo.openeuler.org/openEuler-24.03-LTS-SP4/EPOL/update/main/$basearch/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-24.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler
```

> 若您使用了国内镜像源（如华为云、清华镜像），请以实际镜像路径为准。为正确解析完整依赖，请确保以下仓库均已启用：**OS**、**update**、**everything**、**EPOL**、**EPOL/update**。
>
> 说明：`opencode` 来自 **update** 仓库、`nodejs` 来自 **everything** 仓库、`euler-copilot-rag`/`witty-lite-rag`/`witty-log-detection` 来自 **EPOL** 仓库；`witty` 主包在 SP3 位于 **update**、在 SP4 位于 **EPOL/update**。

## 安装 Witty CLI

开始使用前，建议先更新软件包缓存与系统软件包：

```bash
sudo dnf clean all
sudo dnf makecache
sudo dnf update -y
```

安装 Witty CLI：

```bash
sudo dnf install witty
```

安装完成后，一键部署即自动完成，`witty` 包会一并安装以下组件：

* `witty`（CLI 主二进制）与 `wittyd`（守护进程），以及 systemd 服务 `wittyd.service`
* `witty-builtin-agent`：内置智能体、5 个核心 Skill 与 openEuler Portal MCP 配置
* `witty-agent-loader`：托管配置 loader，负责在 `/etc/opencode/opencode.json` 中聚合生成 opencode 配置

## 验证安装

```bash
witty version
```

输出示例：

```text
version: 3.1.5
commit: c859a1e
date: 2026-08-19T08:47:19Z
```

运行环境诊断，检查配置、server、Shell 集成与终端环境：

```bash
witty doctor
```

启动交互式 CLI：

```bash
witty
```

在普通 Bash 终端直接输入自然语言即可触发 AI 助手（Shell 集成随 RPM 默认启用）：

```bash
检查系统内存
```

## 配置大模型与模型切换

连接模型服务商：

```bash
witty provider connect <provider> --key <api-key>
```

进入 REPL 后切换模型：

```bash
witty
/model
```

详细说明参见「使用说明」中的 Provider 管理与模型切换章节。

## 配置参数

若需调整配置，请编辑 `/etc/witty/config.toml`（系统级）或 `~/.config/witty/config.toml`（用户级）。完整配置项参见「使用说明」配置章节。

修改 agent / skill / MCP 相关配置后，使 server 重新加载配置：

```bash
witty server restart
```

## 卸载

```bash
sudo dnf remove witty
```

卸载后自动移除 `/etc/profile.d/witty.sh` 与 Shell 集成，零残留。

## 附录

### 其他环境使用 Witty CLI

若需在非 openEuler 或非官方仓库环境手动安装，可通过源码构建或使用 openEuler 构建系统产出的 RPM 包。源码位于 `https://atomgit.com/openeuler/euler-copilot-shell`，构建与打包参见仓库 README 与 `packaging/`。

### 内网大模型 API 服务

若内网大模型服务需通过 HTTPS 访问，请确保证书已在系统中正确配置，以避免证书验证失败。如无法配置有效证书，可在环境变量中添加以下内容以跳过证书验证：

```bash
export OI_SKIP_SSL_VERIFY=true
```

### 本地部署大模型服务

若无可用大模型服务，可在本地部署一个支持工具调用能力的大模型服务。若本地设备无 GPU，建议使用激活参数较少的 MoE 模型（如 Qwen3-30B-A3B）；本地部署需要较大计算资源，建议至少 32GB 内存和多核 CPU。

### FAQ

#### Q1：安装后 Shell 直输不生效怎么办？

**A1**：运行 `witty doctor` 检查 `shell integration`；若未加载，执行 `eval "$(witty init bash)"`；并确认未设置 `WITTY_SHELL_ENABLE=0`。

#### Q2：提示找不到 `opencode` 或 server 不可达？

**A2**：`opencode` 是 `witty` 的弱依赖，确认已安装：`which opencode`。若未安装，执行 `sudo dnf install opencode nodejs`；随后运行 `witty doctor` 诊断，或执行 `witty server restart`。

#### Q3：EPOL / EPOL update 源无法访问怎么办？

**A3**：确认已正确配置 EPOL 与 EPOL update 仓库，并检查网络与镜像源是否可用。可改用国内镜像源（如华为云镜像的 `mirrors.huaweicloud.com/openeuler/...` 路径）。

---

---
url: >-
  /en/docs/22.03_LTS_SP4/cloud/container_form/system_container/writable_namespace_kernel_parameters.md
---
# Writable Namespace Kernel Parameters

## Function Description

For services running in containers, such as databases, big data, and common applications, some kernel parameters need to be set and adjusted to obtain the optimal performance and reliability. The modification permission of all kernel parameters must be disabled or enabled simultaneously (by using privileged container).

When the modification permission is disabled, only the --sysctl external interface is provided and parameters cannot be flexibly modified in a container.

When the modification permission is enabled, some kernel parameters are globally valid. If some parameters are modified in a container, all programs on the host will be affected, harming security.

System containers provide the  **--ns-change-opt**  parameter, which can be used to dynamically set namespace kernel parameters in a container. The parameter value can be  **net**  or  **ipc**.

## Parameter Description

## Constraints

* If both  **--privileged**  (privileged container) and  **--ns-change-opt**  are specified during container startup,  **--ns-change-opt**  does not take effect.

## Example

Start a container and set  **--ns-change-opt**  to  **net**.

```sh
[root@localhost ~]# isula run -tid --ns-change-opt net --system-container --external-rootfs /root/myrootfs none init
4bf44a42b4a14fdaf127616c90defa64b4b532b18efd15b62a71cbf99ebc12d2
[root@localhost ~]# isula exec -it 4b mount | grep /proc/sys
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime)
```

The mount point  **/proc/sys/net**  in the container has the  **rw**  option, indicating that the  **net**-related namespace kernel parameters have the read and write permissions.

Start another container and set  **--ns-change-opt**  to  **ipc**.

```sh
[root@localhost ~]# isula run -tid --ns-change-opt ipc --system-container --external-rootfs /root/myrootfs none init
c62e5e5686d390500dab2fa76b6c44f5f8da383a4cbbeac12cfada1b07d6c47f
[root@localhost ~]# isula exec -it c6 mount | grep /proc/sys
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/shmmax type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/shmmni type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/shmall type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/shm_rmid_forced type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/msgmax type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/msgmni type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/msgmnb type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/kernel/sem type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/fs/mqueue type proc (rw,nosuid,nodev,noexec,relatime)
```

The mount point information of  **ipc**-related kernel parameters in the container contains the  **rw**  option, indicating that the  **ipc**-related namespace kernel parameters have the read and write permissions.

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/xfce/xfce_user_guide.md
---
# Xfce Desktop Environment

## 1. Overview

Xfce is a lightweight desktop environment running on Unix-like operating systems. Xfce provides multiple functional components, including all applications. This document describes how to use Xfce.

The following figure shows the WebUI.

![Figure 1 Main screen of the desktop - big](./figures/xfce-1.png)

## 2. Desktop

### 2.1 Desktop Icons

By default, icons such as the file system, main folder, and mount directory are placed. You can double-click the icons to open the page.

![Figure 2 Default desktop icons - big](./figures/xfce-2.png)

### 2.2 Shortcut Menu

Right-click in the blank area on the desktop. The shortcut menus are displayed, as shown in the following figure.

![Figure 3 Shortcut menu](./figures/xfce-3.png)

The following table describes some options.

| Parameter| Description|
|:----------|:----------|
| Open in New Window| Open the **Desktop** directory of the login user.|
| Create Launcher| Create a launcher.|
| Create URL Link| Create a URL link.|
| Create Folder| Create a folder.|
| Create Document| Create a text file.|
| Open Terminal Here| Create a terminal.|
| Arrange Desktop Icons| Automatically arrange desktop icons.|
| Desktop Settings| Set the background, menus, and icons.|
| Applications| Applications|

## 3. Taskbar

### 3.1 Basic Functions

The taskbar is located at the top, including application, window display area, multi-view switch, and tray menus.

![Figure 4 Taskbar](./figures/xfce-4.png)

| Component| Description|
|:----------|:----------|
| Applications| Display all applications and settings, and allow you to search for applications and settings.|
| Window display area| The blank area in the middle of the horizontal bar, which displays running programs or opened documents. You can minimize, maximize, close, or pin the window to the top.|
| Switching views| Perform operations in multiple workspaces without interfering with each other.|
| Tray| Set the network connection, sound, power, notification center, calendar, and login user actions.|

#### 3.1.1 Applications

![Figure 5 All applications - big](./figures/xfce-5.png)

#### 3.1.2 Window Display Area

![Figure 6 Window display area - big](./figures/xfce-6.png)

#### 3.1.3 Multi-View Switching

Click ![](./figures/xfce-7.png) in the taskbar to enter the corresponding work area.

For example, you can use the mouse to switch among multiple workspaces to select the operation area that you want to work in.

![Figure 7 Switching among multiple views - big](./figures/xfce-71.png)

#### 3.1.4 Tray

![Figure 8 Tray menu - big](./figures/xfce-8.png)

##### 3.1.4.1 Network

You can click ![](./figures/xfce-81.png) on the taskbar and select a network connection mode as required.

![Figure 9 Network connection page](./figures/xfce-811.png)

Network settings dialog box

Right-click the network icon ![](./figures/xfce-81.png) on the taskbar. The network setting menu is displayed.

![Figure 10 Network settings](./figures/xfce-812.png)

Click **Edit Connections**. The network setting dialog box is displayed.

![Figure 11 Network setting dialog box](./figures/xfce-813.png)

Double-click the specified network connection, for example, **enp1s0**. The page for setting the connection is displayed.

![Figure 12 Setting the wired network](./figures/xfce-814.png)

##### 3.1.4.2 Volume

Click the volume icon ![](./figures/xfce-82.png) on the taskbar to open the sound page.

![Figure 13 Volume setting window](./figures/xfce-821.png)

##### 3.1.4.3 Power supply

Click ![](./figures/xfce-83.png) on the taskbar.

![Figure 14 Power supply devices](./figures/xfce-831.png)

You can click **Power Manager Settings** to configure the display and nodes.

![Figure 15 Setting the power manager](./figures/xfce-832.png)

##### 3.1.4.4 Notification Center

Click ![](./figures/xfce-84.png) on the taskbar.

![Figure 16 Notification center - big](./figures/xfce-841.png)

You can disable the notification function by selecting **Do not disturb**.

The notification center displays the latest important information list. You can click **Clear log** to clear the information list.

You can click **Notification settings** to go to the notification setting page of the control panel and set the applications to be displayed and the number of messages to be displayed.

![Figure 17 Notification center - big](./figures/xfce-842.png)

##### 3.1.4.5 Calendar

You can click the date and time on the taskbar to display the calendar window and view the calendar, month calendar, and annual calendar.

You can choose a year, a month and a day to view the information of a specific day.

![Figure 18 Calendar-big](./figures/xfce-85.png)

Right-click the time and date on the taskbar and click **Properties** to set the time.

![Figure 19 Date setting - big](./figures/xfce-851.png)

#### 3.1.4.6 Advanced Settings

Right-click the taskbar and choose **Panel** from the shortcut menu.

![Figure 20 Shortcut menu on the taskbar](./figures/xfce-86.png)

You can set the layout of the taskbar and add or delete items.

![Figure 21 Shortcut menu on the taskbar](./figures/xfce-861.png)

##### 3.1.4.7 Login User Actions

Click the login user on the task bar to view related actions.

![Figure 22 Actions of a login user](./figures/xfce-87.png)

###### 3.1.4.7.1 Lock Screen

If you use the computer currently, you can lock the screen (which does not affect the current running status of the system) to prevent misoperations. After locking the screen, you can enter the password to log in to the system again.

By default, the system automatically locks the screen after a period of idle time.

###### 3.1.4.7.2 Switch User

If you want to log in to the computer as another user, choose **Switch User**.

Then, the system closes all running applications. Therefore, before performing this operation, save the current work.

###### 3.1.4.7.3 Suspend

For the sake of environmental protection and energy saving, you can select **Suspend**\*\*.\*\*

After that, the related data is read into the memory. Do not switch the power supply.

###### 3.1.4.7.3 Shut Down

You can choose **Shut Down** to shut down a computer.

Before performing this operation, save the current work.

###### 3.1.4.7.3 Log Out

To log out of the GUI, click **Log Out**.

Then, the system closes all running applications. Therefore, before performing this operation, save the current work.

## 4. Shortcut Operation Bar

### 4.1 Basic Functions

The shortcut operation bar is located at the bottom, including the icons for displaying all the desktops, terminals, file managers, web browsers, application search, and user home directories.

![Figure 23 Shortcut operation bar](./figures/xfce-9.png)

| Component| Description |
|:----------|:----------|
| Show Desktop| Hide all windows and show the desktop. Click again to restore the window.|
| Terminal| Open a terminal.|
| File Manager| Open a file manager.|
| Web Browser| Open a web browser.|
| Application Finder| Open the application search window.|
| User Home Directory| Open the home directory of the login user.|

#### 4.1.1 Show Desktop

Click ![](./figures/xfce-91.png) on the shortcut operation bar to display the desktop.

![Figure 24 Showing the desktop - big](./figures/xfce-911.png)

#### 4.1.2 Terminal

Click ![](./figures/xfce-92.png) on the shortcut operation bar to open a terminal.

![Figure 25 Terminal-big](./figures/xfce-921.png)

#### 4.1.3 File Manager

You can click the ![](./figures/xfce-93.png) icon on the shortcut operation bar to open a file manager.

![Figure 26 File manager - big](./figures/xfce-931.png)

#### 4.1.4 Web Browser

You can click the ![](./figures/xfce-94.png) icon on the shortcut operation bar to open a web browser.

![Figure 27 Web browser - big](./figures/xfce-941.png)

#### 4.1.5 Application Finder

You can click the ![](./figures/xfce-95.png) icon on the shortcut operation bar to open an application program search interface.

![Figure 28 Searching for an application - big](./figures/xfce-951.png)

#### 4.1.6 User Home Directory

Click ![](./figures/xfce-96.png) on the shortcut operation bar and click **Open File**. The user home directory page is displayed.

![Figure 29 User home directory - big](./figures/xfce-961.png)

Click the ![](./figures/xfce-96.png) icon on the shortcut operation bar, and then click **Open in Terminal** to open a terminal. The current directory is the home directory of the user.

![Figure 30 User home directory - big](./figures/xfce-962.png)

---

---
url: /en/docs/22.03_LTS_SP4/tools/desktop/xfce/xfce_installation.md
---
# Xfce Installation

Xfce is a lightweight Linux desktop. In the current version, all components have been updated from GTK2 to GTK3 and from D-Dbus Glib to GDBus. Most components support GObject Introspection (GI), which is used to generate and parse the API meta information of the C program library, so that the dynamic language (or managed language) can be bound to the program library based on C + GObject. In the current version, user experience is optimized, new features are added, and a large number of bugs are fixed. Xfce occupies fewer memory and CPU resources than other UIs (GNOME and KDE), providing smoother and more efficient user experience.

Xfce supports the x86\_64 and AArch64 architectures.

You are advised to create an administrator during the installation.

1. [Download](https://openeuler.org/en/download/) the openEuler ISO image and install the OS. Run the following command to update the software source. You are advised to configure the Everything source and the EPOL source. This document describes how to install Xfce in the minimum installation scenario.

   ```shell
   sudo dnf update
   ```

2. Install the font library.

   ```shell
   sudo dnf install dejavu-fonts liberation-fonts gnu-*-fonts google-*-fonts
   ```

3. Install Xorg.

   ```shell
   sudo dnf install xorg-*
   ```

4. Install Xfce and related components.

   ```shell
   sudo dnf install xfwm4 xfdesktop xfce4-* xfce4-*-plugin network-manager-applet *fonts
   ```

5. Install the login manager.

   ```shell
   sudo dnf install lightdm lightdm-gtk
   ```

6. Run the following command as the root user to set Xfce as the default desktop environment:

   ```shell
   echo 'user-session=xfce' >> /etc/lightdm/lightdm.conf.d/60-lightdm-gtk-greeter.conf
   ```

7. Start Xfce using the login manager.

   ```shell
   sudo systemctl start lightdm
   ```

   After the login manager is started, choose **Xfce Session** in the upper right corner and enter the user name and password to log in.

8. Set the GUI to start upon system boot.

   ```shell
   sudo systemctl enable lightdm
   sudo systemctl set-default graphical.target
   ```

   If GDM is installed by default, you are advised to disable GDM.

   ```shell
   systemctl disable gdm
   ```

   Restart the machine.

   ```shell
   sudo reboot
   ```

## FAQs

**Why Is the Background Color of the LightDM Login Page Black?**

The login page is black because **background** is not set in the default configuration file **/etc/lightdm/lightdm-gtk-greeter.conf** of lightdm-gtk.
Set **background=/usr/share/backgrounds/xfce/xfce-blue.jpg** in the **greeter** section at the end of the configuration file, and then run the `systemctl restart lightdm` command.

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/xfce/xfce_user_guide.md
---
# Xfce 用户指南

## 一 概述

Xfce是运行在类Unix操作系统中的一款轻量级桌面环境。Xfce提供了多个功能部件，包括所有应用程序等，本文主要描述 Xfce 的使用。

界面如下图所示。

![图 1 桌面主界面-big](./figures/xfce-1.png)

## 二 桌面

### 2.1 桌面图标

系统默认放置了文件系统、主文件夹、挂载目录等图标，鼠标左键双击即可打开页面。

![图 2 主界面默认图标-big](./figures/xfce-2.png)

### 2.2 右键菜单

在桌面空白处单击鼠标右键，出现的菜单如下图所示，为用户提供了一些快捷功能。

![图 3 右键菜单](./figures/xfce-3.png)

部分选项说明如表。

| 选项 | 说明|
| :------------ | :------------ |
| 在新窗口中打开 | 打开对应登录用户的Desktop目录 |
| 创建启动器 | 启动器的自行创建 |
| 创建URL链接 | URL链接的自行创建 |
| 创建文件夹 | 新建文件夹 |
| 创建文档 | 新建文本文档 |
| Open Terminal Here | 新建终端 |
| 排列桌面图标 | 自动排列桌面图标 |
| 桌面设置 | 提供关于背景、菜单、图标的设置 |
| 应用程序 | 所有应用程序 |

## 三 任务栏

### 3.1 基本功能

任务栏位于顶部，包括所有应用程序菜单、窗口显示区、多视图切换、托盘菜单。

![图 4 任务栏](./figures/xfce-4.png)

| 组件| 说明 |
| :------------ | :------------ |
| 所有应用程序 | 用于弹出所有程序以及设置，可查找应用和设置。|
| 窗口显示区 | 横条中间空白部分；显示正在运行的程序或打开的文档，可进行最小化、最大化、关闭窗口、窗口置顶等操作。|
| 多视图切换 | 可在多个工作区互不干扰进行操作。 |
| 托盘 | 包含了对网络连接、声音、电源、通知中心、日历、登录用户动作的设置。|

#### 3.1.1 所有应用程序

![图 5 所有应用程序-big](./figures/xfce-5.png)

#### 3.1.2 窗口显示区

![图 6 窗口显示区-big](./figures/xfce-6.png)

#### 3.1.3 多视图切换

点击任务栏“![](./figures/xfce-7.png)”中的每个区域图标，即可进入对应的工作区域。

例如，通过鼠标在多个工作区内切换选择当下需要工作的操作区。

![图 7 多视图切换-big](./figures/xfce-71.png)

#### 3.1.4 托盘

![图 8 托盘菜单-big](./figures/xfce-8.png)

##### 3.1.4.1 网络

用户通过鼠标左键点击任务栏上的网络“![](./figures/xfce-81.png)”图标，可根据需要选择网络连接方式。

![图 9 网络连接界面](./figures/xfce-811.png)

网络设置窗口

用户通过鼠标右键点击任务栏上的网络“![](./figures/xfce-81.png)”图标，弹出网络设置菜单。

![图 10 网络设置](./figures/xfce-812.png)

点击 编辑连接，即刻进入网络设置窗口。

![图 11 网络设置窗口](./figures/xfce-813.png)

双击 指定的网络连接 ，例如enp1s0，进入该连接的设置界面。

![图 12 有线网络设置窗口](./figures/xfce-814.png)

##### 3.1.4.2 音量

用户通过鼠标左键点击任务栏上的音量“![](./figures/xfce-82.png)”图标，打开声音界面。

![图 13 音量设置窗口](./figures/xfce-821.png)

##### 3.1.4.3 电源

用户通过点击鼠标左键任务栏中电源“![](./figures/xfce-83.png)”图标。

![图 14 电源设备](./figures/xfce-831.png)

用户通过点击 电源管理器设置 进行 显示、节点 等配置。

![图 15 电源管理器设置](./figures/xfce-832.png)

##### 3.1.4.4 通知中心

用户通过点击鼠标左键任务栏中通知“![](./figures/xfce-84.png)”图标。

![图 16 通知中心-big](./figures/xfce-841.png)

用户可通过选择“请勿打扰”关闭通知。

通知中心将会显示重要的近期最新的重要信息列表，选择“清除日志”可将信息列表清空。

用户可通过选择“通知设置”跳转进入控制面板的通知设置界面，能设置显示信息的应用，以及信息的数量。

![图 17 通知中心-big](./figures/xfce-842.png)

##### 3.1.4.5 日历

用户通过鼠标左键点击任务栏上的时间日期弹出日历窗口，查看日历、月历、年历窗口。

用户可通过筛选年 > 月 > 日查看一日信息。

![图 18 日历-big](./figures/xfce-85.png)

用户通过鼠标右键点击任务栏上的时间日期，点击 属性 进行时间设置。

![图 19 日期设置-big](./figures/xfce-851.png)

#### 3.1.4.6 高级设置

右键单击任务栏，出现的菜单中点击 面板。

![图 20 任务栏右键菜单](./figures/xfce-86.png)

用户可对任务栏的布局进行设定，可进行项目的添加、删除等相关设置。

![图 21 任务栏右键菜单](./figures/xfce-861.png)

##### 3.1.4.7 登录用户动作

用户通过鼠标左键点击任务栏上的登录用户，查看相关动作。

![图 22 登录用户动作](./figures/xfce-87.png)

###### 3.1.4.7.1 锁屏

当用户暂时不需要使用计算机时，可以选择锁屏（不会影响系统当前的运行状态），防止误操作；用户返回后，输入密码即可重新进入系统。

在默认设置下，系统在一段空闲时间后，将自动锁定屏幕。

###### 3.1.4.7.2 切换用户

选择其他用户登录使用计算机时，可选择“切换用户”。

此时，系统会关闭所有正在运行的应用；所以，在执行此操作前，请先保存当前工作。

###### 3.1.4.7.3 挂起

处于环保节能考虑，可选择“挂起”。

此时，相关数据读入内存，注意不能切换电源。

###### 3.1.4.7.3 关机

用户选择关闭计算机时，可以选择“关机”。

在执行此操作前，建议先保存当前工作。

###### 3.1.4.7.3 注销

选择退出本次图形界面登录时，可选择“注销”。

此时，系统会关闭所有正在运行的应用；所以，在执行此操作前，请先保存当前工作。

## 四 快捷操作栏

### 4.1 基本功能

快捷操作栏位于底部，包括所有显示桌面、终端、文件管理器、网络浏览器、应用程序查找、用户家目录。

![图 23 快捷操作栏](./figures/xfce-9.png)

| 组件  | 说明 |
| :------------ | :------------ |
| 显示桌面 | 最小化桌面的所有窗口，返回桌面；再次单击将恢复窗口 |
| 终端 | 快速打开一个终端 |
| 文件管理器 | 快速打开一个文件管理器 |
| 网络浏览器 | 快速打开一个网络浏览器 |
| 应用程序查找 | 快速打开应用程序查找窗口 |
| 用户家目录 | 快速打开登录用户的家目录 |

#### 4.1.1 显示桌面

用户通过鼠标左键点击快捷操作栏上的“![](./figures/xfce-91.png)”图标，执行 显示桌面 相关操作。

![图 24 显示桌面-big](./figures/xfce-911.png)

#### 4.1.2 终端

用户通过鼠标左键点击快捷操作栏上的“![](./figures/xfce-92.png)”图标，打开一个终端。

![图 25 终端-big](./figures/xfce-921.png)

#### 4.1.3 文件管理器

用户通过鼠标左键点击快捷操作栏上的“![](./figures/xfce-93.png)”图标，打开一个文件管理器。

![图 26 文件管理器-big](./figures/xfce-931.png)

#### 4.1.4 网络浏览器

用户通过鼠标左键点击快捷操作栏上的“![](./figures/xfce-94.png)”图标，打开一个网络浏览器。

![图 27 网络浏览器-big](./figures/xfce-941.png)

#### 4.1.5 应用程序查找

用户通过鼠标左键点击快捷操作栏上的“![](./figures/xfce-95.png)”图标，打开一个应用程序查找界面。

![图 28 应用程序查找-big](./figures/xfce-951.png)

#### 4.1.6 用户家目录

用户通过鼠标左键点击快捷操作栏上的“![](./figures/xfce-96.png)”图标，点击 打开文件，打开一个用户家目录界面。

![图 29 用户家目录-big](./figures/xfce-961.png)

用户通过鼠标左键点击快捷操作栏上的“![](./figures/xfce-96.png)”图标，点击 在终端中打开，打开一个终端，当前目录为用户家目录。

![图 30 用户家目录-big](./figures/xfce-962.png)

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/tools/ai/euler-copilot-framework/xpu_turbo/deploy_guide/deployment.md
---
# XPU Turbo 部署指南

## 1. 概述

XPU Turbo 原名 sysHAX， 功能定位为K+X异构融合推理加速，主要包含两部分功能：

* 推理动态调度
* CPU推理加速

**推理动态调度**：

* 对于推理任务，prefill阶段属于计算密集型任务，decode阶段属于访存密集型任务。因此从计算资源的角度来说，prefill阶段适合在GPU/NPU等硬件上执行，而decode阶段可以放在CPU等硬件上执行。

* 此外，针对MOE模型，XPU Turbo还支持AF分离功能。XPU Turbo针对MOE模型可以将FFN阶段卸载至CPU侧，将Attention阶段卸载至xPU侧。通过卸载部分任务到CPU侧，从而带来总性能上的提升。

**CPU推理加速**：在CPU上通过NUMA亲和、并行优化、算子优化等方式，来加速CPU推理性能。

XPU-Turbo共包含两部分交付件：

![xpu-turbo-deploy](pictures/xpu-turbo-deploy.png "xpu-turbo-deploy")

交付件包括：

* sysHAX：负责请求的处理和prefill、decode请求的调度
* vllm：vllm是一款大模型推理服务，在部署时包含GPU/NPU和CPU两种，分别用于处理prefill和decode请求。从开发者使用易用性角度来说，vllm将使用容器化发布。

vllm是一款**高吞吐、低内存占用**的**大语言模型（LLM）推理与服务引擎**，支持**CPU 计算加速**，提供高效的算子下发机制，包括：

* Schedule（调度）：优化任务分发，提高并行计算效率
* Prepare Input（准备数据）：高效的数据预处理，加速输入构建
* Ray 框架：利用分布式计算提升推理吞吐
* Sample（模型后处理）：优化采样策略，提升生成质量
* 框架后处理：融合多种优化策略，提升整体推理性能

该引擎结合**高效计算调度与优化策略**，为 LLM 推理提供**更快、更稳定、更可扩展**的解决方案。

## 2. 环境准备

| KEY        |  VALUE                                   |
| ---------- | ---------------------------------------- |
| 服务器型号  |  鲲鹏920系列CPU                           |
| GPU        |  Nvidia A100                              |
| 操作系统    | openEuler 24.03 LTS SP1                 |
| python     | 3.9及以上                                 |
| docker     | 25.0.3及以上                              |

* docker 25.0.3可通过 `dnf install moby` 进行安装。

## 3. PD分离-部署流程

XPU Turbo目前支持在Nvidia、Ascend上进行部署。下文以在Nvidia上进行部署举例，另外读者也可以参考：

* PD分离：[Nvidia部署](https://gitcode.com/openeuler/sysHAX/blob/dev/docs/sysHAX_online_deployment_guide_on_CPU+GPU.md)

* PD分离：[Ascend部署（310p）](https://gitcode.com/openeuler/sysHAX/blob/dev/docs/sysHAX_online_deployment_guide_on_CPU+NPU.md)

如下为XPU Turbo在Nvidia上部署的流程：

### 3.1 安装NVIDIA Container Toolkit（容器引擎插件）

首先需要通过 `nvidia-smi` 和 `nvcc -V` 检查是否已经安装好了nvidia驱动和cuda驱动，如果没有的话需要首先安装nvidia驱动和cuda驱动。

已经安装NVIDIA Container Toolkit可忽略该步骤。否则按照如下流程安装：

<https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html>

* 执行 `systemctl restart docker` 命令重启docker，使容器引擎插件在docker配置文件中添加的内容生效。

### 3.2 容器场景vllm搭建

如下流程为在GPU容器中部署vllm。

```shell
docker pull hub.oepkgs.net/neocopilot/syshax/syshax-vllm-gpu:0.2.1

docker run --name vllm_gpu \
    --ipc="shareable" \
    --shm-size=64g \
    --gpus=all \
    -p 8001:8001 \
    -v /home/models:/home/models \
    -w /home/ \
    -itd hub.oepkgs.net/neocopilot/syshax/syshax-vllm-gpu:0.2.1 bash
```

在上述脚本中：

* `--ipc="shareable"`：允许容器共享IPC命名空间，可进行进程间通信。
* `--shm-size=64g`：设置容器共享内存为64G。
* `--gpus=all`：允许容器使用宿主机所有GPU设备
* `-p 8001:8001`：端口映射，将宿主机的8001端口与容器的8001端口进行映射，开发者可自行修改。
* `-v /home/models:/home/models`：目录挂载，将宿主机的 `/home/models` 映射到容器内的 `/home/models` 内，实现模型共享。开发者可自行修改映射目录。

```shell
vllm serve /home/models/DeepSeek-R1-Distill-Qwen-32B \
    --served-model-name=ds-32b \
    --host 0.0.0.0 \
    --port 8001 \
    --dtype=auto \
    --swap_space=16 \
    --block_size=16 \
    --preemption_mode=swap \
    --max_model_len=8192 \
    --tensor-parallel-size 2 \
    --gpu_memory_utilization=0.8 \
    --enable-auto-pd-offload
```

在上述脚本中：

* `--tensor-parallel-size 2`：启用张量并行，将模型拆分到2张GPU上运行，需至少2张GPU，开发者可自行修改。
* `--gpu_memory_utilization=0.8`：限制显存使用率为80%，避免因为显存耗尽而导致服务崩溃，开发者可自行修改。
* `--enable-auto-pd-offload`：启动在swap out时触发PD分离。

如下流程为在CPU容器中部署vllm。

```shell
docker pull hub.oepkgs.net/neocopilot/syshax/syshax-vllm-cpu:0.2.1

docker run --name vllm_cpu \
    --ipc container:vllm_gpu \
    --shm-size=64g \
    --privileged \
    -p 8002:8002 \
    -v /home/models:/home/models \
    -w /home/ \
    -itd hub.oepkgs.net/neocopilot/syshax/syshax-vllm-cpu:0.2.1 bash
```

在上述脚本中：

* `--ipc container:vllm_gpu`共享名为vllm\_gpu的容器的IPC（进程间通信）命名空间。允许此容器直接通过共享内存交换数据，避免跨容器复制。

```shell
NRC=4 INFERENCE_OP_MODE=fused OMP_NUM_THREADS=160 CUSTOM_CPU_AFFINITY=0-159 SYSHAX_QUANTIZE=q4_0 \
vllm serve /home/models/DeepSeek-R1-Distill-Qwen-32B \
    --served-model-name=ds-32b \
    --host 0.0.0.0 \
    --port 8002 \
    --dtype=half \
    --block_size=16 \
    --preemption_mode=swap \
    --max_model_len=8192 \
    --enable-auto-pd-offload
```

在上述脚本中：

* `INFERENCE_OP_MODE=fused`：启动CPU推理加速
* `OMP_NUM_THREADS=160`：指定CPU推理启动线程数为160，该环境变量需要在指定INFERENCE\_OP\_MODE=fused后才能生效
* `CUSTOM_CPU_AFFINITY=0-159`：指定CPU绑核方案，后续会详细介绍。
* `SYSHAX_QUANTIZE=q4_0`：指定量化方案为q4\_0。当前版本支持2种量化方案：`q8_0`、`q4_0`。
* `NRC=4`：GEMV算子分块方式，该环境变量在920系列处理器上具有较好的加速效果。

需要注意的是，必须先启动GPU的容器，才能启动CPU的容器。

通过lscpu来检查当前机器的硬件情况，其中重点需要关注的是：

```shell
Architecture:             aarch64
  CPU op-mode(s):         64-bit
  Byte Order:             Little Endian
CPU(s):                   160
  On-line CPU(s) list:    0-159
Vendor ID:                HiSilicon
  BIOS Vendor ID:         HiSilicon
  Model name:             -
    Model:                0
    Thread(s) per core:   1
    Core(s) per socket:   80
    Socket(s):            2
NUMA:
  NUMA node(s):           4
  NUMA node0 CPU(s):      0-39
  NUMA node1 CPU(s):      40-79
  NUMA node2 CPU(s):      80-119
  NUMA node3 CPU(s):      120-159
```

该机器共有160个物理核心，没有开启SMT，共有4个NUMA，每个NUMA上有40个核心。

通过这两条脚本来设定绑核方案：`OMP_NUM_THREADS=160 CUSTOM_CPU_AFFINITY=0-159`。在这两条环境变量中，第一条为CPU推理启动线程数，第二条为绑定的CPU的ID。在 CPU推理加速中为实现NUMA亲和，需要进行绑核操作，要遵循如下规则：

* 启动线程数要与绑定的CPU个数相同；
* 每一个NUMA上使用的CPU个数需要相同，以保持负载均衡。

例如，在上述脚本中，绑定了0-159号CPU。其中，0-39属于0号NUMA节点，40-79属于1号NUMA节点，80-119属于2号NUMA节点，120-159属于3号NUMA节点。每个NUMA均使用了40个CPU，保证了每个NUMA的负载均衡。

### 3.3 sysHAX安装

sysHAX安装有两种方式，可以通过dnf安装rpm包。注意，使用该方法需要将openEuler升级至openEuler 24.03 LTS SP2及以上版本：

```shell
dnf install sysHAX
```

或者直接使用源码启动：

```shell
git clone -b v0.2.0 https://gitee.com/openeuler/sysHAX.git
```

在启动sysHAX之前需要进行一些基础配置：

```shell
# 使用 dnf install sysHAX 安装sysHAX时
syshax init
syshax config services.gpu.port 8001
syshax config services.cpu.port 8002
syshax config services.conductor.port 8010
syshax config models.default ds-32b
```

```shell
# 使用 git clone -b v0.2.0 https://gitee.com/openeuler/sysHAX.git 时
python3 cli.py init
python3 cli.py config services.gpu.port 8001
python3 cli.py config services.cpu.port 8002
python3 cli.py config services.conductor.port 8010
python3 cli.py config models.default ds-32b
```

此外，也可以通过 `syshax config --help` 或者 `python3 cli.py config --help` 来查看全部配置命令。

配置完成后，通过如下命令启动sysHAX服务：

```shell
# 使用 dnf install sysHAX 安装sysHAX时
syshax run
```

```shell
# 使用 git clone -b v0.2.0 https://gitee.com/openeuler/sysHAX.git 时
python3 main.py
```

启动sysHAX服务的时候，会进行服务连通性测试。sysHAX符合openAPI标准，待服务启动完成后，即可API来调用大模型服务。可通过如下脚本进行测试：

```shell
curl http://0.0.0.0:8010/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "ds-32b",
    "messages": [
        {
            "role": "user",
            "content": "介绍一下openEuler。"
        }
    ],
    "stream": true,
    "max_tokens": 1024
}'
```

## 4. AF分离-部署流程

AF分离目前支持在Ascend NPU（910B系列）和沐曦GPU（metax C500）上的部署。由于正处于beta阶段，功能暂未完善，请关注 [sysHAX-adapter](https://gitcode.com/openeuler/sysHAX-adapter) 。尝鲜版部署流程可参考：

* AF分离：[910B 部署流程](https://gitcode.com/openeuler/sysHAX-adapter/blob/dev/docs/sysHAX_AF_online_deployment_guide_on_CPU+NPU.md)
* AF分离：[metax C500部署流程](https://gitcode.com/openeuler/sysHAX-adapter/blob/dev/docs/sysHAX_AF_online_deployment_guide_on_CPU+MUXI.md)

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/aops/quick_deployment_of_aops.md
---
# 一键化部署指南

## 一、一键化部署介绍

Aops服务一键化部署采用docker容器技术，搭配docker-compose容器编排，简化部署难度，实现一键启动和暂停。

## 二、环境要求

建议使用2台openEuler 22.03 LTS SP4及以上机器完成部署（单台机器内存8G+），具体用途及部署方案如下：

* 机器A用于部署mysql、elasticsearch、kafka、redis、prometheus等，主要提供数据服务支持；
* 机器B用于部署A-Ops服务端，提供业务功能支持。部署A-Ops前端服务，提供展示、操作；

| 机器编号 | 配置IP      | 部署服务                                     |
| -------- | ----------- | -------------------------------------------- |
| 机器A    | 192.168.1.1 | mysql elasticsearch redis kafka prometheus   |
| 机器B    | 192.168.1.2 | aops-zeus aops-diana aops-apollo aops-hermes |

## 三、配置环境部署

### 1. 关闭机器A防火墙

```shell
systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld
```

### 2. 安装docker docker-compose

```shell
dnf install docker docker-compose
# 设置docker开机启动
systemctl enable docker
```

### 3. 安装aops-vulcanus aops-tools

```shell
dnf install aops-vulcanus aops-tools
```

### 4. 执行一键化部署

* 执行部署脚本

```shell
cd /opt/aops/scripts/deploy/container
# 执行run.sh部署脚本
bash run.sh
```

> 进入交互式命令行
>
> ```shell
> 1. Build the docker container (build).
> 2. Start the container orchestration service (start-service/start-env).
> 3. Stop all container services (stop-service/stop-env).
> run.sh: line 74: read: `Enter to exit the operation (Q/q).': not a valid identifier
> Select an operation procedure to continue:
>
> ```
>
> **build**: 部署基础服务（mysql、kafka等）不需要执行build操作
>
> **start-service**: 启动A-Ops服务及前端应用
>
> **start-env**: 启动基础服务，包括mysql、redis、kafka等
>
> **stop-service**: 停止A-Ops服务及前端应用
>
> **stop-env**: 停止基础服务（数据会依然保留）
>
> **Q/q**: 退出命令交互模式

* 部署A-Ops服务端

```shell
# 切换在机器B上执行部署脚本
cd /opt/aops/scripts/deploy/container
bash run.sh
# 交互式命令中执行start-service
```

* 更改服务配置文件

> **注意：当A-Ops服务和基础服务在同一台机器上部署时，则无需调整配置文件即可使用。若部署方案与本文档中类似（机器A、B），则需要将所有的配置文件中连接基础服务的配置项更改为机器A的ip**
>
> **默认的mysql连接字符串中使用无密码模式，基础服务的mysql配置了默认密码“123456”，视具体情况调整**

```shell
# 调整 apollo.ini diana.ini zeus.ini配置文件中连接mysql、elasticsearch、kafka、redis的ip地址
cd /etc/aops/
```

* **FAQ**

​**1. elasticsearch基础服务无法正常启动**

查看/opt/es文件夹的权限，默认权限需要调整为777，可执行 "chmod -R 777 /opt/es" 。

​**2. prometheus 基础服务无法正常启动**

查看/etc/prometheus目录下是否存在prometheus.yml配置文件，如果不存在，请添加配置文件。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/secure_container/configuring_network_for_a_secure_container.md
---
# 为安全容器配置网络

## tap设备网络支持

安全容器技术是基于Qemu VM实现的，对于物理机系统来说，安全容器就相当于是一个VM，所以安全容器可以在Neutron网络中将VM通过TAP技术接入外部网络。我们这里不需要关心TAP设备的创建和网桥对接等问题，只需要将指定的TAP设备（host已经存在）热插进pause容器的VM，并更新网卡信息即可。

相关命令行如下：

1. **为已经启动的容器添加一个tap网卡（interface）**

   ```sh
   $ cat ./test-iface.json | kata-runtime kata-network add-iface 6ec7a98 -
   ```

   其中：6ec7a98是容器ID的截断,test-iface.json是描述网卡信息的文件，举例如下：

   ```conf
   {
       "device": "tap-test", 
       "name": "eth-test", 
       "IPAddresses": [
           {
               "address": "172.16.0.3", 
               "mask": "16"
           }
       ], 
       "hwAddr":"02:42:20:6f:a3:69",
       "mtu": 1500,
       "vhostUserSocket":"/usr/local/var/run/openvswitch/vhost-user1",
       "queues":5
   }
   ```

   上述json文件中各个字段的含义说明如下：

   kata-runtime kata-network add-iface添加网卡命令执行返回结果说明：

   * 命令执行成功：从**命令的标准输出**返回json格式插入网卡的信息，json格式内容和传入的网卡信息相同。

     例如：

     ```sh
     $ kata-runtime kata-network add-iface <container-id> net.json 
     {"device":"tap_test","name":"eth-test","IPAddresses":[{"Family":2,"Address":"173.85.100.1","Mask":"24"}],"mtu":1500,"hwAddr":"02:42:20:6e:03:01","pciAddr":"01.0/00"}
     ```

   * 命令执行失败：从**命令的标准输出**返回字符串null。

     例如：

     ```sh
     $ kata-runtime kata-network add-iface <container-id> netbad.json 2>/dev/null
     null
     ```

   > !\[NOTE]说明\
   > 当网卡添加成功时，如果为其指定了IP地址，则kata会为新添加的网卡添加一条destination为同网段地址的默认路由，如上例中添加网卡之后容器内有如下路由被添加：
   >
   > ```conf
   > [root@6ec7a98 /]# ip route  
   > 172.16.0.0/16 dev eth-test proto kernel scope link src 172.16.0.3  
   > ```

2. **列出已经添加的网卡**

   ```sh
   $ kata-runtime kata-network list-ifaces 6ec7a98
   [{"name":"eth-test","mac":"02:42:20:6f:a3:69","ip":["172.16.0.3/16"],"mtu":1500}]
   ```

   可以查询到我们刚才添加的网卡信息。

   kata-runtime kata-network  list-ifaces列出已添加网卡命令执行返回结果说明：

   * 命令执行成功：从**命令的标准输出**返回json格式的Pod中所有插入网卡的信息。

     如果Pod中插入了多个网卡设备，返回的是一个json数组格式的网卡信息。

     ```sh
     $ kata-runtime kata-network list-ifaces <container-id>
     [{"name":"container_eth","mac":"02:42:20:6e:a2:59","ip":["172.17.25.23/8"],"mtu":1500},{"name":"container_eth_2","mac":"02:90:50:6b:a2:29","ip":["192.168.0.34/24"],"mtu":1500}]
     ```

     如果Pod中没有插入任何网卡设备，从**命令的标准输出**返回字符串null。

     ```sh
     $ kata-runtime kata-network list-ifaces <container-id>
     null
     ```

   * 命令执行失败：从**命令的标准输出**返回字符串null，从**命令的标准错误输出**返回错误描述信息。

     例如：

     ```sh
     $ kata-runtime kata-network list-ifaces <container-id>
     null
     ```

3. **为指定网卡添加一条路由**

   ```sh
   $ cat ./test-route.json | kata-runtime kata-network add-route 6ec7a98 -
   [{"dest":"default","gateway":"172.16.0.1","device":"eth-test"}]
   ```

   kata-runtime kata-network add-route为指定网卡添加一条路由命令执行返回结果说明：

   * 命令执行成功：从**命令的标准输出**返回json格式的添加的路由信息。

     例如：

     ```sh
     $ kata-runtime kata-network add-route <container-id> route.json 
     [{"dest":"177.17.0.0/24","gateway":"177.17.25.1","device":"netport_test_1"}]
     ```

   * 命令执行失败：从**命令的标准输出**返回字符串null，从**命令的标准错误输出**返回错误描述信息。

     例如：

     ```sh
     $ kata-runtime kata-network add-route <container-id> routebad.json 2>/dev/null
     null
     ```

   字段说明如下：

   * dest：设置路由对应的网段。格式为\<ip>/\<mask>，\<ip>必选。分如下3种情况：
     1. 配置\<ip>/\<mask>；
     2. 只配置\<ip>，则默认掩码为32；
     3. 若配置了"dest":"default"，默认无dest，需传入gateway。

   * gateway：设置路由的下一跳网关。设置"dest":"default"时，gateway必选；其他情况可选。

   * device：必选。设置路由对应的网卡名称，最长支持15字符。

   > \[!NOTE]说明
   > 如果为容器内的回环设备lo添加路由时，路由配置文件中的"device"字段对应的设备名称为"lo"。

4. **删除指定路由**

   ```sh
   $ cat ./test-route.json | kata-runtime kata-network del-route 6ec7a98 -
   ```

   test-route.json字段与添加路由输入json文件的字段相同。

   kata-runtime kata-network del-route删除指定路由命令执行返回结果说明：

   * 命令执行成功：从**命令的标准输出**返回json格式的添加的路由信息。

     例如：

     ```sh
     $ kata-runtime kata-network del-route <container-id> route.json 
     [{"dest":"177.17.0.0/24","gateway":"177.17.25.1","device":"netport_test_1"}]
     ```

   * 命令执行失败：从**命令的标准输出**返回字符串null，从**命令的标准错误输出**返回错误描述信息。

     例如：

     ```sh
     $ kata-runtime kata-network del-route <container-id> routebad.json 2>/dev/null
     null
     ```

   > \[!NOTE]说明
   >
   > * 输入字段中dest为必选，device/gateway均为可选。kata根据不同字段进行模糊匹配，删除对应的路由规则。例如指定了dest为某个IP，则所有该IP的规则都会被删除。
   > * 如果删除的是容器内回环设备lo的路由时，路由配置文件中的"device"字段对应的设备名称为"lo"。

5. **删除已经添加的网卡**

   ```sh
   $ cat ./test-iface.json | kata-runtime kata-network del-iface 6ec7a98 -
   ```

   > \[!NOTE]说明\
   > 删除网卡时，仅根据网卡容器内名称（name字段）来删除。即便填写其他字段，kata也不会使用。

   kata-runtime kata-network del-iface删除网卡命令执行返回结果说明：

   * 命令执行成功：从**命令标准输出**返回null字符串。

     例如：

     ```sh
     $ kata-runtime kata-network del-iface <container-id> net.json
     null
     ```

   * 命令执行失败：从**命令标准输出**返回删除失败网卡json格式的信息，从**命令的标准错误输出**返回错误描述信息。

     例如：

     ```sh
     $ kata-runtime kata-network del-iface <container-id> net.json
     {"device":"tapname_fun_012","name":"netport_test_1","IPAddresses":[{"Family":0,"Address":"177.17.0.1","Mask":"8"}],"mtu":1500,"hwAddr":"02:42:20:6e:a2:59","linkType":"tap"}
     ```

以上为常用场景和命令行示例，具体命令行接口请参见“附录 > 接口列表”。

## kata IPVS子系统

安全容器提供添加ipvs命令的接口，支持对容器设置ipvs规则。功能包含对虚拟服务的添加/编辑/删除、对真实服务器的添加/编辑/删除、查询ipvs服务信息、设置连接超时、清理系统连接缓存，并支持对规则的批量导入。

1. 为容器添加一个虚拟服务地址

   ```sh
   kata-runtime kata-ipvs ipvsadm --parameters "--add-service --tcp-service 172.17.0.7:80 --scheduler rr --persistent 3000" <container-id>
   ```

2. 修改容器虚拟服务参数

   ```sh
   kata-runtime kata-ipvs ipvsadm --parameters "--edit-service --tcp-service 172.17.0.7:80 --scheduler rr --persistent 5000" <container-id>
   ```

3. 删除容器虚拟服务地址

   ```sh
   kata-runtime kata-ipvs ipvsadm --parameters "--delete-service --tcp-service 172.17.0.7:80" <container-id>
   ```

4. 为虚拟服务地址添加一个真实服务器

   ```sh
   kata-runtime kata-ipvs ipvsadm --parameters "--add-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80 --weight 100" <container-id>
   ```

5. 修改容器真实服务器参数

   ```sh
   kata-runtime kata-ipvs ipvsadm --parameters "--edit-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80 --weight 200" <container-id>
   ```

6. 删除容器真实服务器

   ```sh
   kata-runtime kata-ipvs ipvsadm --parameters "--delete-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80" <container-id>
   ```

7. 查询服务信息

   ```sh
   kata-runtime kata-ipvs ipvsadm --parameters "--list" <container-id>
   ```

8. 逐条导入耗时太久，可将规则写入文件后，批量导入

   ```sh
   kata-runtime kata-ipvs ipvsadm --restore -  <  <规则文件路径> <container-id>
   ```

   > \[!NOTE]说明
   > 单条添加时默认使用NAT模式，批量导入时添加真实服务器需手动添加-m参数使用NAT模式。\
   > 规则文件内容示例：\
   > -A -t 10.10.11.12:100 -s rr -p  3000\
   > -a -t 10.10.11.12:100 -r  172.16.0.1:80 -m\
   > -a -t 10.10.11.12:100 -r  172.16.0.1:81 -m\
   > -a -t 10.10.11.12:100 -r 172.16.0.1:82  -m

9. 清理系统连接缓存

   ```sh
   kata-runtime kata-ipvs cleanup --parameters "--orig-dst 172.17.0.4 --protonum tcp" <container-id>
   ```

10. 为tcp/tcpfin/udp连接设置超时

    ```sh
    kata-runtime kata-ipvs ipvsadm --parameters "--set 100 100 200" <container-id>
    ```

    > \[!NOTE]说明
    >
    > 1. 每个容器支持iptables规则数量最大为20000条（5k service，3个server/service），add-service和add-server都算作规则。
    > 2. 批量导入前需清空已有规则。
    > 3. 不存在并发测试场景。
    > 4. 以上为常用命令示例，具体命令行接口请参见“附录 > 接口列表”。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/secure_container/configuring_resources_for_a_secure_container.md
---
# 为安全容器配置资源

安全容器运行于虚拟化隔离的轻量级虚拟机内，因此资源的配置应分为两部分：对轻量级虚拟机的资源配置，即Host资源配置；对虚拟机内容器的配置，即Guest容器资源配置。以下资源配置均分为这两部分。

## 资源共享

由于安全容器运行于虚拟化隔离的轻量虚拟机内，故无法访问Host上某些namespace下的资源，因此启动时不支持--net host，--ipc host，--pid host，--uts host。

当启动一个Pod时，同一个Pod中的所有容器默认共享同一个net namespace和ipc namespace。如果同一个Pod中的容器需要共享pid namespace，则可以通过Kubernetes进行配置，Kubernetes 1.11版本该值为默认关闭。

## 限制资源

对资源的限制建议在configuration.toml中进行配置。
常用的字段的有：

* default\_vcpus ：指定每个SB/VM的默认vCPU数量
* default\_maxvcpus ：指定每个SB/VM的默认最大vCPU数量
* default\_root\_ports ：指定每个SB/VM的默认Root Ports数量
* default\_bridges ：指定每个SB/VM的默认bridges数量
* default\_memory ：指定每个SB/VM的默认内存大小，默认为1024 MiB
* memory\_slots ：指定每个SB/VM的内存插槽数量，默认为10

## 热插拔限制内存资源

内存热插拔对于容器在部署时内存动态分配来说是一项关键特性。由于安全容器是运行在虚拟机当中的，所以这一项特性需要VMM和guest kernel两方面的支持。目前kata在arm64上默认使用的QEMU和guest kernel都是支持这一特性的。除了VMM和guest kernel，内存热插拔还取决于依赖着固件的ACPI。在x86上，由于ACPI可以隐式地随固件一起启动，所以可以直接用QEMU去启动一个打开了ACPI的虚拟机。不过在arm64上的话，在使用内存热插拔之前就需要手动去安装UEFI ROM。

```shell
$ pushd $GOPATH/src/github.com/kata-containers/tests
$ sudo .ci/aarch64/install_rom_aarch64.sh #仅限ubuntu
$ popd
```

---

---
url: /zh/docs/22.03_LTS_SP4/cloud.md
---


---

---
url: /zh/docs/22.03_LTS_SP4/server/security/secdetector/using_secdetector.md
---
# 使用 secDetector

secDetector 提供了SDK，一个so库，用户可以在自己的应用程序中集成该动态链接库从而通过简单的接口使用secDetector。本章介绍其使用方法。

## 基本用法

用户按照指南《[安装secDetector](./install_secdetector.md)》安装完secDetector之后，libsecDetectorsdk.so、secDetector\_sdk.h、secDetector\_topic.h就已经被部署到系统用户库默认路径中。

1. 使用 C 或 C++ 开发的应用程序确保include路径包含后，可以首先在程序中引用这两个头文件。

   ```c
   #include <secDetector/secDetector_topic.h>
   #include <secDetector/secDetector_sdk.h>
   ```

2. 参考指南《[接口参考](./api_reference.md)》调用SDK提供的接口访问secDetector。

   1. 首先调用订阅接口secSub，订阅所需的主题。
   2. 然后在独立线程中调用消息读取接口secReadFrom阻塞式的读取被订阅主题产生的信息。
   3. 最后当不需要使用secDetector时，调用退订接口secUnsub。退订时请严格使用订阅时的返回值。

## 代码示例

可以参考secDetector代码仓上的示例代码，由python语言编写。

1. 可以在如下链接中查看示例代码

   [examples/python · openEuler/secDetector (atomgit.com)](https://atomgit.com/openeuler/secDetector/tree/master/examples/python)

2. 也可以下载后参考

```shell
git clone https://atomgit.com/openeuler/secDetector.git
```

## 规格与约束

1. 部分功能（如内存修改探针-安全开关）依赖硬件体系结构，因此在不同指令集架构上的表现并不相同。
2. 从内核到用户态传输数据缓存空间为探针共享，缓冲区满会丢弃新采集的事件信息。缓存空间可配置范围为4~1024 MB, 必须为2的幂。
3. 服务进程secDetectord支持root用户运行，不支持多实例，非第一个运行的程序会退出。
4. 用户订阅连接数限制为5个。
5. 用户订阅后，读取消息时需要为消息读取接口提供一块缓冲区，超过缓冲区长度的消息将被截断。建议缓冲区长度不低于4096。
6. 对于文件名、节点名之类的描述字符串都有一定的长度限制，过长可能会被截断。
7. 应用程序单进程内不支持并行多连接 secDetectord 接收消息。只能一次订阅，单连接接受消息。去订阅后才能重新订阅。
8. secDetectord 进程应当等待所有应用程序的连接中断即完全退订所有主题后，才可以关闭退出。
9. 部分功能（如内存修改探针-安全开关）基于当前CPU状态。因此检测的基本功能是可以检测到当前CPU上的状态变化，其他CPU上的状态变化如果未能及时同步到当前CPU，则不会被检测到。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/administration/administrator/using_dnf_to_manage_software_packages.md
---
# 使用DNF管理软件包

DNF是一款Linux软件包管理工具，用于管理RPM软件包。DNF可以查询软件包信息，从指定软件库获取软件包，自动处理依赖关系以安装或卸载软件包，以及更新系统到最新可用版本。

> \[!NOTE]说明
>
> * DNF与YUM完全兼容，提供了YUM兼容的命令行以及为扩展和插件提供的API。
> * 使用DNF需要管理员权限，本章所有命令需要在管理员权限下执行。

## 配置DNF

### DNF配置文件

DNF的主要配置文件是/etc/dnf/dnf.conf，该文件包含两部分：

* “main”部分保存着DNF的全局设置。

* “repository”部分保存着软件源的设置，可以有零个或多个“repository”。

另外，在/etc/yum.repos.d目录中保存着零个或多个repo源相关文件，它们也可以定义不同的“repository”。

所以openEuler软件源的配置一般有两种方式，一种是直接配置/etc/dnf/dnf.conf文件中的“repository”部分，另外一种是在/etc/yum.repos.d目录下增加.repo文件。

#### 配置main部分

/etc/dnf/dnf.conf 文件包含的“main”部分，配置示例如下：

```text
[main]
gpgcheck=1
installonly_limit=3
clean_requirements_on_remove=True
best=True
skip_if_unavailable=False
```

常用选项说明：

**表 1**  main参数说明

#### 配置repository部分

repository部分允许您定义定制化的openEuler软件源仓库，各个仓库的名称不能相同，否则会引起冲突。配置repository部分有两种方式，一种是直接配置/etc/dnf/dnf.conf文件中的“repository”部分，另外一种是配置/etc/yum.repos.d目录下的.repo文件。

* 直接配置/etc/dnf/dnf.conf文件中的“repository”部分

  下面是\[repository]部分的一个最小配置示例：

  ```text
  [repository]
  name=repository_name
  baseurl=repository_url
  ```

  > \[!NOTE]说明\
  > openEuler提供在线的镜像源，地址：<https://repo.openeuler.org/>。以 openEuler 22.03 LTS SP4 的aarch64版本为例，baseurl可配置为<https://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/aarch64/>。

  选项说明：

  **表 2**  repository参数说明

* 配置/etc/yum.repos.d目录下的.repo文件

  openEuler提供了多种repo源供用户在线使用，各repo源含义可参考[系统安装](./../../releasenotes/os_installation.md)。使用root权限添加openEuler repo源，示例如下：

  ```shell
  # vi /etc/yum.repos.d/openEuler.repo
  ```

  ```shell
  [OS]
  name=OS
  baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/
  enabled=1
  gpgcheck=1
  gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler
  ```

  > \[!NOTE]说明
  >
  > * enabled为是否启用该软件源仓库，可选值为1和0。默认值为1，表示启用该软件源仓库。
  > * gpgkey为验证签名用的公钥。

#### 显示当前配置

* 要显示当前的配置信息：

  ```shell
  dnf config-manager --dump
  ```

* 要显示相应软件源的配置，首先查询repo id：

  ```shell
  dnf repolist
  ```

  然后执行如下命令，显示对应id的软件源配置，其中  *repository*  为查询得到的repo id：

  ```shell
  dnf config-manager --dump repository
  ```

* 您也可以使用一个全局正则表达式，来显示所有匹配部分的配置：

  ```shell
  dnf config-manager --dump glob_expression
  ```

### 创建本地软件源仓库

要建立一个本地软件源仓库，请按照下列步骤操作。

1. 安装createrepo软件包。在root权限下执行如下命令：

   ```shell
   dnf install createrepo
   ```

2. 将需要的软件包复制到一个目录下，如/mnt/local\_repo/ 。

3. 创建软件源，执行以下命令：

   ```shell
   createrepo /mnt/local_repo
   ```

### 添加、启用和禁用软件源

本节将介绍如何通过“dnf config-manager”命令添加、启用和禁用软件源仓库。

#### 添加软件源

要定义一个新的软件源仓库，您可以在 /etc/dnf/dnf.conf 文件中添加“repository”部分，或者在/etc/yum.repos.d/目录下添加“.repo”文件进行说明。建议您通过添加“.repo”的方式，每个软件源都有自己对应的“.repo”文件，以下介绍该方式的操作方法。

要在您的系统中添加一个这样的源，请在root权限下执行如下命令，执行完成之后会在/etc/yum.repos.d/目录下生成对应的repo文件。其中  *repository\_url*  为repo源地址，详情请参见[表2](#zh-cn_topic_0151921080_a4a0b069bbf624b09be3bdd08567c0445)。

```shell
dnf config-manager --add-repo repository_url
```

#### 启用软件源

要启用软件源，请在root权限下执行如下命令，其中  *repository*  为新增.repo文件中的repo id（可通过dnf repolist查询）：

```shell
dnf config-manager --set-enable repository
```

您也可以使用一个全局正则表达式，来启用所有匹配的软件源。其中  *glob\_expression*  为对应的正则表达式，用于同时匹配多个repo id：

```shell
dnf config-manager --set-enable glob_expression
```

#### 禁用软件源

要禁用软件源，请在root权限下执行如下命令：

```shell
dnf config-manager --set-disable repository
```

同样的，您也可以使用一个全局正则表达式来禁用所有匹配的软件源：

```shell
dnf config-manager --set-disable glob_expression
```

## 管理软件包

使用dnf能够让您方便的进行查询、安装、删除软件包等操作。

### 搜索软件包

您可以使用rpm包名称、缩写或者描述搜索需要的RPM包，使用命令如下：

```shell
dnf search term
```

示例如下：

```shell
$   dnf search httpd
========================================== N/S matched: httpd ==========================================
httpd.aarch64 : Apache HTTP Server
httpd-devel.aarch64 : Development interfaces for the Apache HTTP server
httpd-manual.noarch : Documentation for the Apache HTTP server
httpd-tools.aarch64  : Tools for use with the Apache HTTP Server
libmicrohttpd.aarch64  : Lightweight library for embedding a webserver in applications
mod_auth_mellon.aarch64  : A SAML 2.0 authentication module for the Apache Httpd Server
mod_dav_svn.aarch64  : Apache httpd module for Subversion server
```

### 列出软件包清单

要列出系统中所有已安装的以及可用的RPM包信息，使用命令如下：

```shell
dnf list all
```

要列出系统中特定的RPM包信息，使用命令如下：

```shell
dnf list glob_expression...
```

示例如下：

```shell
$ dnf list httpd
Available Packages
httpd.aarch64              2.4.51-17.oe2203SP4           Local
```

### 显示RPM包信息

要显示一个或者多个RPM包信息，使用命令如下：

```shell
dnf info package_name...
```

例如搜索，命令如下：

```shell
$ dnf info httpd
Available Packages
Name        : httpd
Version     : 2.4.51
Release     : 17.oe2203SP4
Arch        : aarch64
Size        : 1.2 M
Repo        : Local
Summary     : Apache HTTP Server
URL         : http://httpd.apache.org/
License     : ASL 2.0
Description : The Apache HTTP Server is a powerful, efficient, and extensible
            : web server.
```

### 安装RPM包

要安装一个软件包及其所有未安装的依赖，请在root权限下执行如下命令：

```shell
dnf install package_name
```

您也可以通过添加软件包名字同时安装多个软件包。配置文件/etc/dnf/dnf.conf添加参数strict=False，运行dnf命令参数添加--setopt=strict=0。请在root权限下执行如下命令：

```shell
dnf install package_name package_name... --setopt=strict=0
```

示例如下：

```shell
# dnf install httpd
```

> \[!NOTE]说明
>
> * 安装RPM包过程中，若出现安装失败，可参考[安装时出现软件包冲突、文件冲突或缺少软件包导致安装失败](https://docs.openeuler.openatom.cn/zh/docs/common/faq/server/administration_faqs.html#%E9%97%AE%E9%A2%985-%E5%AE%89%E8%A3%85%E6%97%B6%E5%87%BA%E7%8E%B0%E8%BD%AF%E4%BB%B6%E5%8C%85%E5%86%B2%E7%AA%81%E3%80%81%E6%96%87%E4%BB%B6%E5%86%B2%E7%AA%81%E6%88%96%E7%BC%BA%E5%B0%91%E8%BD%AF%E4%BB%B6%E5%8C%85%E5%AF%BC%E8%87%B4%E5%AE%89%E8%A3%85%E5%A4%B1%E8%B4%A5)。
> * 请勿安装install-scripts软件包，该包仅用于imageTailor工具裁剪ISO时使用，若在openEuler系统上安装该软件包会导致系统无法启动。

### 下载软件包

使用dnf下载软件包，请在root权限下输入如下命令：

```shell
dnf download package_name
```

如果需要同时下载未安装的依赖，则加上--resolve，使用命令如下：

```shell
dnf download --resolve package_name
```

示例如下：

```shell
# dnf download --resolve httpd
```

### 删除软件包

要卸载软件包以及相关的依赖软件包，请在root权限下执行如下命令：

```shell
dnf remove package_name...
```

示例如下：

```shell
# dnf remove totem
```

## 管理软件包组

软件包集合是服务于一个共同目的的一组软件包，例如系统工具集等。使用dnf可以对软件包组进行安装/删除等操作，使相关操作更高效。

### 列出软件包组清单

使用summary参数，可以列出系统中所有已安装软件包组、可用的组、可用的环境组的数量，命令如下：

```shell
dnf groups summary
```

使用示例如下：

```shell
# dnf groups summary
Last metadata expiration check: 0:11:56 ago on Sat 17 Aug 2019 07:45:14 PM CST.
Available Groups: 8
```

要列出所有软件包组和它们的组ID ，命令如下：

```shell
dnf group list
```

使用示例如下：

```shell
# dnf group list
Last metadata expiration check: 0:10:32 ago on Sat 17 Aug 2019 07:45:14 PM CST.
Available Environment Groups:
   Minimal Install
   Custom Operating System
   Server
Available Groups:
   Development Tools
   Graphical Administration Tools
   Headless Management
   Legacy UNIX Compatibility
   Network Servers
   Scientific Support
   Security Tools
   System Tools

```

### 显示软件包组信息

要列出包含在一个软件包组中必须安装的包和可选包，使用命令如下：

```shell
dnf group info glob_expression...
```

例如显示Development Tools信息，示例如下：

```shell
# dnf group info "Development Tools"
Last metadata expiration check: 0:14:54 ago on Wed 05 Jun 2019 08:38:02 PM CST.

Group: Development Tools
 Description: A basic development environment.
 Mandatory Packages:
   binutils
   glibc-devel
   make
   pkgconf
   pkgconf-m4
   pkgconf-pkg-config
   rpm-sign
 Optional Packages:
   expect
```

### 安装软件包组

每一个软件包组都有自己的名称以及相应的ID（groupid），您可以使用软件包组名称或它的ID进行安装。

要安装一个软件包组，请在root权限下执行如下命令：

```shell
dnf group install group_name
```

或

```shell
dnf group install groupid
```

例如安装Development Tools相应的软件包组，命令如下：

```shell
# dnf group install "Development Tools"
```

或

```shell
# dnf group install development
```

### 删除软件包组

要卸载软件包组，您可以使用软件包组名称或它的ID，在root权限下执行如下命令：

```shell
dnf group remove group_name
```

或

```shell
dnf group remove groupid
```

例如删除Development Tools相应的软件包组，命令如下：

```shell
# dnf group remove "Development Tools"
```

或

```shell
# dnf group remove development
```

## 检查并更新

dnf可以检查您的系统中是否有软件包需要更新。您可以通过dnf列出需要更新的软件包，并可以选择一次性全部更新或者只对指定包进行更新。

### 检查更新

如果您需要显示当前系统可用的更新，使用命令如下：

```shell
dnf check-update
```

使用示例如下：

```shell
# dnf check-update
Last metadata expiration check: 0:02:10 ago on Sun 21 May 2023 11:28:07 PM  CST.

anaconda-core.aarch64       36.15.5-17.oe2203SP4             update
anaconda-tui.aarch64        36.15.5-17.oe2203SP4             update
anaconda-user-help.aarch64  26.1-10.oe2203SP4                update
bind-libs.aarch64           9.16.23-18.oe2203SP4             update
bind-license.noarch         9.16.23-18.oe2203SP4             update
bind-utils.aarch64          9.16.23-18.oe2203SP4             update
...
```

### 升级

如果您需要升级单个软件包，在root权限下执行如下命令：

```shell
dnf update package_name
```

例如升级rpm包，示例如下：

```shell
# dnf update anaconda-gui.aarch64
Last metadata expiration check: 0:02:10 ago on Sun 21 May 2023 11:30:27 PM  CST.
Dependencies Resolved
================================================================================
 Package                  Arch         Version              Repository     Size
================================================================================
Updating:
 anaconda-core            aarch64      36.15.5-17.oe2203SP4       updates       1.4 M
 anaconda-tui             aarch64      36.15.5-17.oe2203SP4       updates       274 k
 anaconda-user-help       aarch64      26.1-10.oe2203SP4          updates       315 k

Transaction Summary
================================================================================
Upgrade  5 Package

Total download size: 3.1 M
Is this ok [y/N]:
```

如果您需要升级软件包组，在root权限下执行如下命令：

```shell
dnf group update group_name
```

### 更新所有的包和它们的依赖

要更新所有的包和它们的依赖，在root权限下执行如下命令：

```shell
dnf update
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/application_dev/using_gcc_for_compilation.md
---
# 使用GCC编译

本章介绍GCC编译的一些基本知识，并通过示例进行实际演示。更多的GCC知识请通过**man gcc**命令查询。

## 简介

GCC（GNU Compiler Collection）是GNU推出的功能强大、性能优越的多平台编译器。GCC编译器能将C、C++语言源程序、汇编程序和目标程序编译、链接成可执行文件。openEuler操作系统中已默认安装了GCC软件包。

## 基本规则

### 文件类型

对于任何给定的输入文件，文件类型决定进行何种编译。GCC常用的文件类型如[表1](#table634145764320)所示。

**表 1**  GCC常用的文件类型

### 编译流程

使用GCC将源代码文件生成可执行文件，需要经过预处理、编译、汇编和链接。

1. 预处理：将源程序（如 **.c** 文件）预处理，生成 **.i** 文件。
2. 编译：将预处理后的 **.i** 文件编译成为汇编语言，生成 **.s** 文件。
3. 汇编：将汇编语言文件经过汇编，生成目标文件 **.o** 文件。
4. 链接：将各个模块的 **.o** 文件链接起来生成一个可执行程序文件。

其中 **.i** 文件、**.s**文件、**.o**文件是中间文件或临时文件，如果使用GCC一次性完成C语言程序的编译，则这些文件会被删除。

### 编译选项

GCC编译的命令格式为：**gcc**  \[*options*] \[*filenames*]

其中：

*options* ：编译选项。

*filenames* ：文件名称。

GCC是一个功能强大的编译器，其 *options* 参数取值很多，但有些大部分并不常用，常用的 *options* 取值如[表2](#table1342946175212)所示。

**表 2**  GCC常用的编译选项

### 多源文件编译

多个源文件的编译方法有2种。

* 多个源文件一起编译。编译时需要所有文件重新编译。

  示例：将test1.c和test2.c分别编译后链接成test可执行文件。

  ```shell
  $ gcc test1.c test2.c -o test
  ```

* 分别编译各个源文件，之后对编译后输出的目标文件链接。编译时只重新编译修改的文件，未修改的文件不用重新编译。

  示例：分别编译test1.c，test2.c，再将二者的目标文件test1.o，test2.o链接成test可执行文件。

  ```shell
  $ gcc -c test1.c
  $ gcc -c test2.c
  $ gcc -o test1.o test2.o -o test
  ```

## 库

库是写好的、现有的、成熟的、可以复用的代码。每个程序都要依赖很多基础的底层库。

库文件在命名时约定，以lib为前缀，以.so（动态库）或.a（静态库）为后缀，中间为库文件名。如libfoo.so或libfoo.a。由于所有的库文件都遵循了同样的规范，因此当在链接库时，-l 选项指定链接的库文件名时可以省去lib前缀，即GCC 在对-lfoo 进行处理时，会自动去链接名为libfoo.so 或libfoo.a的库文件。而当在创建库时，必须指定完整文件名libfoo.so或libfoo.a。

根据链接时期的不同，库分为静态库和动态库。静态库是在链接阶段，将汇编生成的目标文件.o与引用到的库一起链接打包到可执行文件中；而动态库是在程序编译时并不会被链接到目标代码中，而是在程序运行时才被载入。二者有如下差异：

* 资源利用不一样

  静态库为生成的可执行文件的一部分，而动态库为单独的文件。所以使用静态库和和动态库的可执行文件大小和占用的磁盘空间大小不一样，导致资源利用不一样。

* 扩展性与兼容性不一样

  静态库中某个函数的实现变了，那么可执行文件必须重新编译，而对于动态链接生成的可执行文件，只需要更新动态库本身即可，不需要重新编译可执行文件。

* 依赖不一样

  静态库的可执行文件不需要依赖其他的内容即可运行，而动态库的可执行文件必须依赖动态库的存在。所以静态库更方便移植。

* 加载速度不一样

  静态库在链接时就和可执行文件在一块了，而动态库在加载或者运行时才链接，因此，对于同样的程序，静态链接的要比动态链接加载更快。

### 动态链接库

使用-shared选项 和-fPIC选项，可直接使用源文件、汇编文件或者目标文件创建一个动态库。其中-fPIC 选项作用于编译阶段，在生成目标文件时就需要使用该选项，以生成位置无关的代码。

示例1：从源文件生成动态链接库。

```shell
$ gcc -fPIC -shared test.c -o libtest.so
```

示例2：从目标文件生成动态链接库。

```shell
$ gcc -fPIC -c test.c -o test.o
$ gcc -shared test.o -o libtest.so
```

将一个动态库链接到可执行文件，需要在命令行中列出动态库的名称。

示例：将main.c和libtest.so一起编译成 app.out，当 app.out 运行时，会动态地加载链接库 libtest.so。

```shell
$ gcc main.c libtest.so -o app.out
```

这种方式是直接指定使用当前目录下的libtest.so文件。

若使用下面搜索动态库的方式，则为了确保程序在运行时能够链接到动态库，需要通过如下三种方法中的任一种实现。

* 将动态库保存在标准目录下，例如 /usr/lib。

* 把动态库所在路径libraryDIR增加到环境变量LD\_LIBRARY\_PATH中。

  ```shell
  $ export LD\_LIBRARY\_PATH=libraryDIR:$LD\_LIBRARY\_PATH
  ```

  > \[!NOTE]说明\
  > LD\_LIBRARY\_PATH为动态库的环境变量。当运行动态库时，若动态库不在缺省文件夹（/lib 和/usr/lib）下，则需要指定环境变量LD\_LIBRARY\_PATH。

* 把动态库所在路径libraryDIR添加到 /etc/ld.so.conf中然后执行ldconfig或者以动态库所在路径libraryDIR为参数执行ldconfig。

  ```shell
  $ gcc main.c -L libraryDIR -ltest -o app.out
  $ export LD_LIBRARY_PATH=libraryDIR:$LD_LIBRARY_PATH
  ```

### 静态链接库

创建一个静态链接库，需要先将源文件编译为目标文件，然后再使用ar命令将目标文件打包成静态链接库。

示例：将源文件test1.c，test2.c，test3.c编译并打包成静态库。

```shell
$ gcc -c test1.c test2.c test3.c
$ ar rcs libtest.a test1.o test2.o test3.o
```

其中ar是一个备份压缩命令，可以将多个文件打包成一个备份文件（也叫归档文件），也可以从备份文件中提取成员文件。ar最常见的用法是将目标文件打包为静态链接库。

ar将目标文件打包成静态链接库的命令格式为：

ar rcs  *Sllfilename* *Targetfilelist*

* *Sllfilename* ：静态库文件名。
* *Targetfilelist* ：目标文件列表。
* r： 替换库中已有的目标文件，或者加入新的目标文件。
* c： 创建一个库，不管库是否存在，都将创建。
* s： 创建目标文件索引，在创建较大的库时能提高速度。

示例：创建一个main.c文件来使用静态库。

```shell
$ gcc main.c -L libraryDIR -ltest -o test.out
```

其中libraryDIR为libtest.a库的路径。

## 示例

### 使用GCC编译C程序示例

1. cd到代码目录，以目录“~/code”进行举例。如下所示：

   ```shell
   $ cd ~/code 
   ```

2. 编写Hello World程序，保存为helloworld.c，此处以编译Hello World程序进行举例说明。示例如下：

   ```shell
   $ vi helloworld.c
   ```

   代码内容示例：

   ```shell
   #include <stdio.h>    
   int main()    
   {    
          printf("Hello World!\n");       
          return 0;    
   }
   ```

3. 在代码目录，执行编译，使用命令：

   ```shell
   $ gcc helloworld.c -o helloworld
   ```

   编译执行未报错，表明执行通过。

4. 编译完成后，会生成helloworld文件，查看编译结果，示例如下：

   ```shell
   $ ./helloworld
   Hello World!
   ```

### 使用GCC创建和使用动态链接库示例

1. cd到代码目录，以目录“~/code”进行举例。并在该目录下创建src，lib，include子目录，分别用于存放源文件，动态库文件和头文件。

   ```shell
   $ cd ~/code
   $ mkdir src lib include
   ```

2. cd到~/code/src目录，创建2个函数add.c、sub.c，分别实现加、减。

   ```c
   $ cd ~/code/src
   $ vi add.c
   $ vi sub.c
   ```

   add.c代码内容示例：

   ```c
   #include "math.h"
   int add(int a, int b)
   {
           return a+b;
   }
   ```

   sub.c代码内容示例：

   ```c
   #include "math.h"
   int sub(int a, int b)
   {
           return a-b;
   }
   ```

3. 将add.c、sub.c源文件创建为动态库libmath.so，并将该动态库存放在~/code/lib目录。

   ```shell
   $ gcc -fPIC -shared add.c sub.c -o ~/code/lib/libmath.so
   ```

4. cd到~/code/include目录，创建1个头文件math.h，声明函数的头文件。

   ```shell
   $ cd ~/code/include
   $ vi math.h
   ```

   math.h代码内容示例：

   ```shell
   #ifndef __MATH_H_
   #define __MATH_H_
   int add(int a, int b);
   int sub(int a, int b);
   #endif
   ```

5. cd到~/code/src目录，创建一个调用add()和sub()的main.c函数。

   ```shell
   $ cd ~/code/src
   $ vi main.c
   ```

   math.c代码内容示例：

   ```c
   #include <stdio.h>
   #include "math.h"
   int main()
   {
           int a, b;
           printf("Please input a and b:\n");
           scanf("%d %d", &a, &b);
           printf("The add: %d\n", add(a,b));
           printf("The sub: %d\n", sub(a,b));
           return 0;
   }
   ```

6. 将main.c和libmath.so一起编译成math.out。

   ```shell
   $ gcc main.c -I ~/code/include -L ~/code/lib -lmath -o math.out
   ```

7. 将动态链接库所在的路径加入到环境变量中。

   ```shell
   $ export LD_LIBRARY_PATH=~/code/lib:$LD_LIBRARY_PATH
   ```

8. 执行math.out。

   ```shell
   $ ./math.out
   ```

   执行结果如下所示：

   ```text
   Please input a and b:
   9 2
   The add: 11
   The sub: 7
   ```

### 使用GCC创建和使用静态链接库示例

1. cd到代码目录，以目录“~/code”进行举例。并在该目录下创建src，lib，include子目录，分别用于存放源文件，静态库文件和头文件。

   ```shell
   $ cd ~/code
   $ mkdir src lib include
   ```

2. cd到~/code/src目录，创建2个函数add.c、sub.c，分别实现加、减。

   ```shell
   $ cd ~/code/src
   $ vi add.c
   $ vi sub.c
   ```

   add.c代码内容示例：

   ```c
   #include "math.h"
   int add(int a, int b)
   {
           return a+b;
   }
   ```

   sub.c代码内容示例：

   ```c
   #include "math.h"
   int sub(int a, int b)
   {
           return a-b;
   }
   ```

3. 将add.c、sub.c源文件编译为目标文件add.o、sub.o。

   ```shell
   $ gcc -c add.c sub.c
   ```

4. 将add.o、sub.o目标文件通过ar命令打包成静态库libmath.a，并将该静态库存放在~/code/lib目录。

   ```shell
   $ ar rcs ~/code/lib/libmath.a add.o sub.o
   ```

5. cd到~/code/include目录，创建1个头文件math.h，声明函数的头文件。

   ```shell
   $ cd ~/code/include
   $ vi math.h
   ```

   math.h代码内容示例：

   ```shell
   #ifndef __MATH_H_
   #define __MATH_H_
   int add(int a, int b);
   int sub(int a, int b);
   #endif
   ```

6. cd到~/code/src目录，创建一个调用add()和sub()的main.c函数。

   ```shell
   $ cd ~/code/src
   $ vi main.c
   ```

   math.c代码内容示例：

   ```c
   #include <stdio.h>
   #include "math.h"
   int main()
   {
           int a, b;
           printf("Please input a and b:\n");
           scanf("%d %d", &a, &b);
           printf("The add: %d\n", add(a,b));
           printf("The sub: %d\n", sub(a,b));
           return 0;
   }
   ```

7. 将main.c和libmath.a一起编译成math.out。

   ```shell
   $ gcc main.c -I ~/code/include -L ~/code/lib -lmath -o math.out
   ```

8. 执行math.out。

   ```shell
   $ ./math.out
   ```

   执行结果如下所示：

   ```text
   Please input a and b:
   9 2
   The add: 11
   The sub: 7
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/application_dev/using_jdk_for_compilation.md
---
# 使用JDK编译

## 简介

JDK（Java Development Kit）是 Java 开发者进行 Java 开发所必需的软件包，包含 JRE（Java Runtime Environment）和编译、调测工具。openEuler在OpenJDK 的基础上进行了 GC 优化、并发稳定性增强、安全性增强等修改，提高了 Java 应用程序在 ARM 上的性能和稳定性。

## 基本规则

### 文件类型及工具

对于任何给定的输入文件，文件类型决定采用何种工具进行处理。JDK常用的文件类型如[表1](#table634145764320)所示，JDK常用的工具如[表2](#table103504146433)所示。

**表 1**  JDK常用的文件类型

**表 2**  JDK常用的工具

### java程序生成流程

通过JDK将java源代码文件生成并运行Java程序，需要经过编译和运行。

1. 编译：是指使用Java编译器（javac）将java源代码文件（.java文件）编译为.class的字节码文件。
2. 运行：是指在Java虚拟机上执行字节码文件。

### JDK常用工具选项

#### javac编译选项

javac编译的命令格式为：**javac**  \[*options*] \[*sourcefiles*] \[*classes*] \[@*argfiles*]

其中：

*options* ：命令选项。

*sourcefiles* ：一个或多个需要编译的源文件。

*classes* ：一个或多个要为注释处理的类。

@*argfiles* ：一个或多个列出选项和源文件的文件。这些文件中不允许有-J选项。

javac是java编译器，其 *options* 参数取值很多，但有些大部分并不常用，常用的 *options* 取值如[表3](#table1342946175212)所示。

**表 3**  javac常用的编译选项

#### java运行选项

java运行的格式为：

运行类文件：**java**  \[*options*]  *classesname*  \[args]

运行jar文件：**java**  \[*options*] -jar  *filename*  \[args]

其中：

*options* ：命令选项，选项之间用空格分隔。

*classname* ：运行的.class文件名。

*filename* ：运行的.jar文件名。

args：传递给main()函数的参数，参数之间用空格分隔。

java是运行java应用程序的工具，其 *options* 参数取值很多，但有些大部分并不常用，常用的 *options* 取值如[表4](#table371918587238)所示。

**表 4**  java常用的运行选项

#### jar打包选项

jar的命令格式为：**jar**  {c | t | x | u}\[vfm0M] \[*jarfile*] \[*manifest*] \[-C  *dir*]  *file*...

jar命令参数说明如[表5](#table3691718114817)所示。

**表 5**  jar命令参数说明

## 类库

java类库是以包的形式实现的，包是类和接口的集合。java编译器为每个类生成一个字节码文件，且文件名与类名相同，因此同名的类之间就有可能发生冲突。java语言中，把一组类和接口封装在一个包内，包可以有效地管理类名空间，位于不同包中的类即使同名也不会冲突，从而解决了同名类之间可能发生的冲突问题，为管理大量的类和接口提供了方便，也有利于类和接口的安全。

除java提供的许多包外，开发者也可以自定义包，把自己编写的类和接口等组成程序包的形式，以便后续使用。

自定义包需要先声明包，然后再使用包。

### 包的声明

包的声明格式为：package pkg1\[.pkg2\[.pkg3...]];

为了声明一个包，首先必须建立一个相应的目录结构，子目录与包名一致，然后在需要放入该包的类文件开头声明包，表示该文件的全部类都属于这个包。包声明中的“.”指明了目录的层次。如果源程序文件中没有package语句，则指定为无名包。无名包没有路径，一般情况下，java仍然会把源文件中的类存储在当前工作目录（即存放java源文件的目录）下。

包声明语句必须被加到源程序文件的起始部分，而且前面不能有注释和空格。如果在不同源程序文件中使用相同的包声明语句，就可以将不同源程序文件中的类都包含在相同的包中。

### 包的引用

在 Java 中，为了能使用java提供的包中的公用类，或者使用自定义的包中的类，有两种方法。

* 在要引用的类名前带上包名。

  如：name.A obj=new name.A ();

  其中，name为包名，A为类名，obj为对象。表示程序中用name包中的A类定义一个对象obj。

  示例：新建一个example包中Test类的test对象。

  ```shell
  example.Test test = new example.Test();
  ```

* 在文件开头使用import来导入包中的类。

  import语句的格式为：import pkg1\[.pkg2\[.pkg3...]].(classname | \*);

  其中，pkg1\[.pkg2\[.pkg3...]]表明包的层次，classname为所要导入的类。如果要从一个包中导入多个类，则可以使用通配符“\*”来替代。

  示例：导入example包中的Test类。

  ```java
  import example.Test;
  ```

  示例：将example 整个包导入。

  ```java
  import example.*;
  ```

## 示例

### 编译不带包的java程序示例

1. cd到代码目录，此处以“~/code”进行举例。如下所示：

   ```shell
   $ cd ~/code 
   ```

2. 编写Hello World程序，保存为HelloWorld.java，此处以编译Hello World程序进行举例说明。示例如下：

   ```shell
   $ vi HelloWorld.java
   ```

   代码内容示例：

   ```java
   public class HelloWorld {     
        public static void main(String[] args) {         
            System.out.println("Hello World");     
         } 
   }
   ```

3. 在代码目录，执行编译，使用命令：

   ```shell
   $ javac HelloWorld.java 
   ```

   编译执行未报错，表明执行通过。

4. 编译完成后，会生成 HelloWorld.class 文件，通过java命令可执行查看结果，示例如下：

   ```shell
   $ java HelloWorld
   Hello World
   ```

### 编译带包的java程序示例

1. cd到代码目录，此处以“~/code”进行举例。并在该目录下创建“~/code/Test/my/example”、“~/code/Hello/world/developers”、“~/code/Hi/openos/openeuler”子目录，分别用于存放源文件。

   ```shell
   $ cd ~/code
   $ mkdir -p Test/my/example
   $ mkdir -p Hello/world/developers
   $ mkdir -p Hi/openos/openeuler
   ```

2. cd到~/code/Test/my/example目录，创建Test.java。

   ```shell
   $ cd ~/code/Test/my/example
   $ vi Test.java
   ```

   Test.java代码内容示例：

   ```java
   package my.example;
   import world.developers.Hello;
   import openos.openeuler.Hi;
   public class Test {
     public static void main(String[] args) {
      Hello me = new Hello();
      me.hello();
      Hi you = new Hi();
      you.hi();
     }
   }
   ```

3. cd到~/code/Hello/world/developers目录，创建Hello.java。

   ```shell
   $ cd ~/code/Hello/world/developers
   $ vi Hello.java
   ```

   Hello.java代码内容示例：

   ```java
   package world.developers;
   public class Hello {
     public void hello(){
      System.out.println("Hello, openEuler.");
     }
   }
   ```

4. \~/code/Hi/openos/openeuler目录，创建Hi.java。

   ```shell
   $ cd ~/code/Hi/openos/openeuler
   $ vi Hi.java
   ```

   Hi.java代码内容示例：

   ```java
   package openos.openeuler;
   public class Hi {
     public void hi(){
      System.out.println("Hi, the global developers.");
     }
   }
   ```

5. cd到~/code目录，使用javac编译源文件。

   ```shell
   $ cd ~/code
   $ javac -classpath Hello:Hi Test/my/example/Test.java
   ```

   执行完命令后，会在“~/code/Test/my/example”、“~/code/Hello/world/developers”、“~/code/Hi/openos/openeuler”目录下分别生成Test.class、Hello.class、Hi.class文件。

6. cd到~/code目录，使用java运行Test程序。

   ```shell
   $ cd ~/code
   $ java -classpath Test:Hello:Hi my/example/Test
   ```

   执行结果如下所示：

   ```text
   Hello, openEuler.
   Hi, the global developers.
   ```

---

---
url: /zh/docs/22.03_LTS_SP4/server/performance/kae/using_the_kae.md
---
# 使用KAE加速引擎

## 简介

KAE加速引擎为openEuler的一个软件加速库，搭载在Kunpeng 920处理器上联合提供硬件加速引擎功能，包含了对称加密、非对称加密和数字签名，用于加速SSL/TLS应用，可以显著降低处理器消耗，提高处理器效率。此外，用户通过OpenSSL标准接口可实现业务快速迁移。

KAE加速引擎支持以下算法：

* 摘要算法SM3，支持异步模式。
* 对称加密算法SM4，支持异步模式，支持CTR/XTS/CBC模式。
* 对称加密算法AES，支持异步模式，支持ECB/CTR/XTS/CBC模式。
* 非对称算法RSA，支持异步模式，支持 Key Sizes 1024/2048/3072/4096。
* 密钥协商算法DH，支持异步模式，支持 Key Sizes 768/1024/1536/2048/3072/4096。

## 应用场景

KAE加速引擎主要有以下应用场景，如[表1](#table11915824163418)所示。

**表 1**  应用场景

## 安装、升级和卸载

### 安装加速器软件包

#### 安装前准备

##### 环境要求

* TaiShan 200服务器，开启加速引擎功能

> \[!NOTE]说明
>
> * 需要导入加速器许可证，具体操作请参考[《TaiShan 机架服务器 iBMC (V500及以上) 用户指南》](https://support.huawei.com/enterprise/zh/doc/EDOC1100121687)中“许可证管理”章节。
> * 物理机场景使用加速器需要关闭SMMU，具体操作请参考[《TaiShan 200服务器BIOS参数参考》](https://support.huawei.com/enterprise/zh/doc/EDOC1100088653)。

* CPU：Kunpeng 920
* 操作系统：openEuler-22.03\_LTS\_SP4-aarch64-dvd.iso

##### KAE加速引擎软件说明

**表 2**  加速引擎的rpm软件包

#### 安装加速器软件包

##### 前提条件

* 已在本地安装远程SSH登录工具。
* 已安装openEuler操作系统。
* RPM工具能正常使用。
* 已安装OpenSSL 1.1.1a或以上版本。

  使用如下命令查询OpenSSL的版本号：

  * openssl version

##### 安装步骤

1. 以root帐号登录openEuler OS命令行界面。

2. 新建目录用于存放加速器引擎软件包。

3. 使用SSH远程登录工具，将所有加速引擎软件包拷贝到已建好的目录下。

4. 在存放加速引擎软件包目录下，使用**rpm -ivh**命令安装加速器引擎软件包。

   > \[!NOTE]说明
   > 由于libkae包的安装依赖libwd包，所以libwd的安装必须先于libkae。

   ```shell
   rpm -ivh uacce*.rpm hisi*.rpm libwd-*.rpm libkae*.rpm
   ```

5. 使用**rpm -qa**命令，查看加速器软件包是否已正常安装到系统内。使用**rpm -ql**命令，查看软件包的文件是否正确。示例如下。

   ```shell
   rpm -qa|grep -E "hisi|uacce|libwd|libkae"
   ```

6. 重启系统或通过命令行手动依次加载加速器引擎驱动到内核，并查看是否加载成功。

   ```shell
   # modprobe uacce 
   # lsmod | grep uacce 
   # modprobe hisi_qm
   # lsmod | grep hisi_qm 
   # modprobe hisi_qm
   # modprobe hisi_sec2 #加载hisi_sec2驱动时将根据/etc/modprobe.d/hisi_sec2.conf 下的配置文件加载到内核
   # modprobe hisi_hpre #加载hisi_hpre驱动时将根据/etc/modprobe.d/hisi_hpre.conf 下的配置文件加载到内核
   ```

##### 设置环境变量

通过以下命令导出环境变量：如果用户指定安装路径，则下面/usr/local应根据实际安装路径进行修改。

```shell
export OPENSSL_ENGINES=/usr/local/lib/engines-1.1
```

##### 安装后检查

执行**rpm -qa**命令查看加速器引擎软件包是否安装成功。

打印信息中包含“ *软件包名-版本号-* ”表示该软件包安装成功。示例如下。

```shell
rpm -qa|grep -E "hisi|uacce|libwd|libkae"
```

#### 安装后操作

##### OpenSSL加速器引擎测试

用户可以通过以下命令测试部分加速器功能。

* 使用OpenSSL的软件算法测试RSA性能。

  ```shell
  linux-rmw4:/usr/local/bin # ./openssl speed -elapsed rsa2048
  ...
                   sign    verify    sign/s verify/s
  rsa 2048 bits 0.001384s 0.000035s   724.1  28365.8.
  ```

* 使用KAE引擎测试RSA性能。

  ```shell
  linux-rmw4:/usr/local/bin # ./openssl speed -elapsed -engine kae rsa2048
  ....
                   sign    verify    sign/s verify/s
  rsa 2048 bits 0.000355s 0.000022s   2819.0  45478.4
  ```

> \[!NOTE]说明
> 使用KAE引擎加速后签名性能从724.1sign/s提升到2819sign/s。

* 使用OpenSSL的软件算法测试异步RSA性能。

  ```shell
  linux-rmw4:/usr/local/bin # ./openssl speed -elapsed -async_jobs 36 rsa2048 
  ....
                    sign    verify    sign/s verify/s
  rsa 2048 bits 0.001318s 0.000032s    735.7  28555
  ```

* 使用KAE引擎测试异步RSA性能。

  ```shell
  linux-rmw4:/usr/local/bin # ./openssl speed -engine kae -elapsed -async_jobs 36 rsa2048 
  .... 
                    sign    verify    sign/s verify/s
  rsa 2048 bits 0.000018s 0.000009s  54384.1 105317.0
  ```

> \[!NOTE]说明
> 使用KAE引擎加速后异步RSA签名性能从735.7 sign/s提升到 54384.1sign/s。

* 使用OpenSSL的软件算法测试SM4 CBC模式性能。

  ```shell
  linux-rmw4:/usr/local/bin # ./openssl speed -elapsed -evp sm4-cbc
  You have chosen to measure elapsed time instead of user CPU time.
  ....
  Doing sm4-cbc for 3s on 10240 size blocks: 2196 sm4-cbc's in 3.00s  ....
  type          51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes
  sm4-cbc          82312.53k    85196.80k    85284.18k    85000.85k    85284.18k    85261.26k
  ```

* 使用KAE引擎测试SM4 CBC模式性能。

  ```shell
  linux-rmw4:/usr/local/bin # ./openssl speed -elapsed -engine kae -evp sm4-cbc
  engine "kae" set. 
  You have chosen to measure elapsed time instead of user CPU time.
  ...
  Doing sm4-cbc for 3s on 1048576 size blocks: 11409 sm4-cbc's in 3.00s
  ...
  type          51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes
  sm4-cbc         383317.33k   389427.20k   395313.15k   392954.73k   394264.58k   394264.58k
  ```

> \[!NOTE]说明
> 使用KAE加速后SM4 CBC模式在输入数据块大小为8M时，性能从82312.53k/s提升到383317.33k/s。

* 使用OpenSSL的软件算法测试SM3模式性能。

  ```shell
  linux-rmw4:/usr/local/bin # ./openssl speed -elapsed -evp sm3
  You have chosen to measure elapsed time instead of user CPU time.
  Doing sm3 for 3s on 102400 size blocks: 1536 sm3's in 3.00s
  ....
  type          51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes
  sm3              50568.53k    52428.80k    52428.80k    52428.80k    52428.80k    52428.80k
  ```

* 使用KAE引擎测试SM3模式性能。

  ```shell
  linux-rmw4:/usr/local/bin # ./openssl speed -elapsed -engine kae -evp sm3
  engine "kae" set.
  You have chosen to measure elapsed time instead of user CPU time.
  Doing sm3 for 3s on 102400 size blocks: 19540 sm3's in 3.00s
  ....
  type            51200 bytes  102400 bytes  1048576 bytes 2097152 bytes 4194304 bytes 8388608 bytes
  sm3             648243.20k   666965.33k   677030.57k   678778.20k   676681.05k   668292.44k
  ```

> \[!NOTE]说明
> 使用KAE加速后SM3算法在输入数据块大小为8M时，从52428.80 k/s提升到668292.44k/s。

* 使用OpenSSL软件算法测试AES算法CBC模式异步性能。

  ```shell
  linux-rmw4:/usr/local/bin # ./openssl speed -elapsed -evp aes-128-cbc -async_jobs 4
  You have chosen to measure elapsed time instead of user CPU time.
  Doing aes-128-cbc for 3s on 51200 size blocks: 65773 aes-128-cbc's in 3.00s
  Doing aes-128-cbc for 3s on 102400 size blocks: 32910 aes-128-cbc's in 3.00s
  ....
  type          51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes
  aes-128-cbc    1122525.87k  1123328.00k  1120578.22k  1121277.27k  1119879.17k  1115684.86k
  ```

* 使用的KEA引擎测试AES算法CBC模式异步性能。

  ```shell
  linux-rmw4:/usr/local/bin # ./openssl speed -elapsed -evp aes-128-cbc -async_jobs 4 -engine kae
  engine "kae" set.
  You have chosen to measure elapsed time instead of user CPU time.
  Doing aes-128-cbc for 3s on 51200 size blocks: 219553 aes-128-cbc's in 3.00s
  Doing aes-128-cbc for 3s on 102400 size blocks: 117093 aes-128-cbc's in 3.00s
  ....
  type          51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes
  aes-128-cbc    3747037.87k  3996774.40k  1189085.18k  1196774.74k  1196979.11k  1199570.94k
  ```

> \[!NOTE]说明
>
> * AES仅支持数据长度为256KB及以下场景的异步使用。
> * 使用KAE加速后AES算法在输入数据块为100K大小时，从1123328.00k/s提升到3996774.40 k/s 。

### 升级加速器软件包

#### 使用场景

当需要更新加速器软件版本时可以使用**rpm -Uvh**方式进行升级。

#### 操作步骤

1. 从openEuler社区下载最新版本的加速引擎软件包。

2. 使用SSH远程登录工具，以root帐号进入Linux操作系统命令行界面。

3. 将下载下来的最新版本的软件包都放在某个路径下。

4. 在存放软件包的路径下使用**rpm -Uvh**命令升级加速器驱动包及引擎库包。示例如下。

   命令和信息回显如下所示。

   ![](./figures/zh-cn_image_0231143189.png)

   ![](./figures/zh-cn_image_0231143191.png)

5. 使用**rpm -qa**命令查询是否升级成功。确认查询到的版本是最新的升级后版本。

   ![](./figures/zh-cn_image_0231143193.png)

   ![](./figures/zh-cn_image_0231143195.png)

6. 重启系统或通过命令行手动卸载旧版本驱动，然后加载新版本驱动，并查看是否加载成功。

   ```shell
   卸载旧驱动
   # lsmod | grep uacce 
   uacce                 262144  3 hisi_hpre,hisi_sec2,hisi_qm 
   # 
   # rmmod hisi_hpre 
   # rmmod hisi_sec2 
   # rmmod hisi_qm 
   # rmmod uacce 
   # lsmod | grep uacce 
   # 
   加载新驱动
   # modprobe uacce 
   # modprobe hisi_qm
   # modprobe hisi_sec2 #加载hisi_sec2驱动时将根据/etc/modprobe.d/hisi_sec2.conf 下的配置文件加载到内核
   # modprobe hisi_hpre  #加载hisi_hpre驱动时将根据/etc/modprobe.d/hisi_hpre.conf 下的配置文件加载到内核
   # lsmod | grep uacce 
   uacce                36864  3 hisi_sec2,hisi_qm,hisi_hpre
   ```

### 卸载加速器软件包

#### 使用场景

用户不再使用加速引擎软件，或进行新版本加速引擎软件的安装。

#### 操作步骤

1. 使用SSH远程登录工具，以root帐号进入Linux操作系统命令行界面。

2. 重启系统或通过命令行手动将已加载到内核的驱动卸载掉，并查看是否卸载成功。

   ```shell
   # lsmod | grep uacce 
   uacce                36864  3 hisi_sec2,hisi_qm,hisi_hpre 
   # rmmod hisi_hpre 
   # rmmod hisi_sec2  
   # rmmod hisi_qm 
   # rmmod uacce 
   # lsmod | grep uacce 
   #
   ```

3. 通过**rpm -e**命令卸载加速引擎软件包。示例如下。

   > \[!NOTE]说明
   > 由于存在依赖关系，卸载libwd前须先卸载libkae引擎软件包。

   ![](./figures/zh-cn_image_0231143196.png)

   ![](./figures/zh-cn_image_0231143197.png)

4. 使用**rpm -qa |grep**软件包名命令查询是否卸载成功。

   ![](./figures/zh-cn_image_0231143198.png)

## 日志查询

加速器引擎涉及日志信息如[表3](#table52821836)所示。

**表 3**  日志信息

## 加速引擎的应用

> \[!NOTE]说明
> 如果用户未购买引擎许可证，建议用户不要通过kae引擎调用相应算法，否则可能会影响openSSL加密算法的性能。

### KAE引擎使用示例代码

```c
#include <stdio.h> 

#include <stdlib.h> 

/* OpenSSL headers */ 

#include <openssl/bio.h> 

#include <openssl/ssl.h> 

#include <openssl/err.h> 

#include <openssl/engine.h> 

int main(int argc, char **argv) 

{ 

    /* Initializing OpenSSL */ 

    SSL_load_error_strings(); 

    ERR_load_BIO_strings(); 

    OpenSSL_add_all_algorithms(); 

    /*You can use ENGINE_by_id Function to get the handle of the Huawei Accelerator Engine*/ 

    ENGINE *e = ENGINE_by_id("kae"); 

    /*使能加速器异步功能，可选配置，设置为“0”表示不使能，设置为“1”表示使能，默认使能异步功能*/ 

    ENGINE_ctrl_cmd_string(e, "KAE_CMD_ENABLE_ASYNC", "1", 0) 

    ENGINE_init(e); 


    RSA *rsa = RSA_new_method(e);#指定引擎用于RSA加解密 

    /*The user code*/ 

    …… 

; 

    ENGINE_free(e); 

; 

}
```

### 通过OpenSSL配置文件openssl.cnf使用KAE引擎

新建openssl.cnf 需要添加如下配置信息

```cnf
openssl_conf=openssl_def 
[openssl_def] 
engines=engine_section 
[engine_section] 
kae=kae_section 
[kae_section] 
engine_id=kae 
dynamic_path=/usr/local/lib/engines-1.1/kae.so 
KAE_CMD_ENABLE_ASYNC=1    # 0表示不使能异步功能，1表示使能异步功能，默认使能 
default_algorithms=ALL 
init=1
```

导出OPENSSL\_CONF环境变量：

```shell
export OPENSSL_CONF=/home/app/openssl.cnf  #该路径为openssl.cnf存放路径
```

使用OpenSSL配置文件示例如下：

```c
#include <stdio.h>  

#include <stdlib.h>  

/* OpenSSL headers */  

#include <openssl/bio.h>  

#include <openssl/ssl.h>  

#include <openssl/err.h>  

#include <openssl/engine.h>  

int main(int argc, char **argv)  

{  

    /* Initializing OpenSSL */  

    SSL_load_error_strings();  

    ERR_load_BIO_strings();  

#Load openssl configure 

OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CONFIG, NULL);    OpenSSL_add_all_algorithms();  

    /*You can use ENGINE_by_id Function to get the handle of the Huawei Accelerator Engine*/  

    ENGINE *e = ENGINE_by_id("kae"); 

    /*The user code*/  

    ……  

;  

    ENGINE_free(e);  

;
}
```

## 故障处理

### 初始化失败

#### 故障现象

加速器引擎没有完全加载成功。

#### 处理步骤

1. 检查加速器驱动是否加载成功，运行lsmod 命令查看uacce.ko、qm.ko 、sgl.ko 、hisi\_sec2.ko 、hisi\_hpre.ko 、hisi\_zip.ko、 hisi\_rde.ko是否在位。

   ```shell
   # lsmod | grep uacce
   uacce                  262144  2 hisi_hpre,hisi_qm,hisi_sec2,hisi_zip,hisi_rde
   ```

2. 检查/usr/lib64（RPM方式安装时目录）或者/usr/local/lib（源码方式安装时目录）和OpenSSL安装目录是否有加速器引擎库，且建立正确的软链接。

   ```shell
   [root@localhost home]# ll /usr/local/lib/engines-1.1/ |grep kae 
   #查询kae是否正确安装并建立软链接，如果有正确安装显示如下内容
   # ll /usr/local/lib/engines-1.1/ |grep kae
   lrwxrwxrwx. 1 root root     22 Nov 12 02:33 kae.so -> kae.so.1.0.1
   lrwxrwxrwx. 1 root root     22 Nov 12 02:33 kae.so.0 -> kae.so.1.0.1
   -rwxr-xr-x. 1 root root 112632 May 25  2019 kae.so.1.0.1
   [root@localhost home]#
   [root@localhost home]# ll /usr/lib64/ | grep libwd  
   #查询libwd是否正确安装并建立软链接，如果有正确安装显示如下内容
   lrwxrwxrwx.  1 root root       14 Nov 12 02:33 libwd.so -> libwd.so.1.0.1
   lrwxrwxrwx.  1 root root       14 Nov 12 02:33 libwd.so.0 -> libwd.so.1.0.1
   -rwxr-xr-x.  1 root root   137120 May 25  2019 libwd.so.1.0.1
   [root@localhost home]#
   ```

3. 检查OpenSSL引擎库的路径是否能通过export命令进行导出。

   ```shell
   # echo $OPENSSL_ENGINES 
   # export OPENSSL_ENGINES=/usr/local/lib/engines-1.1
   #  echo $OPENSSL_ENGINES
   /usr/local/lib/engines-1.1
   ```

### 安装完加速器引擎之后，查找不到加速器设备

#### 故障现象

安装完加速器引擎之后，查找不到加速器设备。

#### 解决方法

1. 检查虚拟文件系统下是否有相应设备。正常情况下有如下相应的加速器设备。

   ```shell
   # ls -al /sys/class/uacce/
   total 0
   lrwxrwxrwx. 1 root root 0 Nov 14 03:45 hisi_hpre-2 -> ../../devices/pci0000:78/0000:78:00.0/0000:79:00.0/uacce/hisi_hpre-2
   lrwxrwxrwx. 1 root root 0 Nov 14 03:45 hisi_hpre-3 -> ../../devices/pci0000:b8/0000:b8:00.0/0000:b9:00.0/uacce/hisi_hpre-3
   lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_rde-4 -> ../../devices/pci0000:78/0000:78:01.0/uacce/hisi_rde-4
   lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_rde-5 -> ../../devices/pci0000:b8/0000:b8:01.0/uacce/hisi_rde-5
   lrwxrwxrwx. 1 root root 0 Nov 14 08:39 hisi_sec-0 -> ../../devices/pci0000:74/0000:74:01.0/0000:76:00.0/uacce/hisi_sec-0
   lrwxrwxrwx. 1 root root 0 Nov 14 08:39 hisi_sec-1 -> ../../devices/pci0000:b4/0000:b4:01.0/0000:b6:00.0/uacce/hisi_sec-1
   lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_zip-6 -> ../../devices/pci0000:74/0000:74:00.0/0000:75:00.0/uacce/hisi_zip-6
   lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_zip-7 -> ../../devices/pci0000:b4/0000:b4:00.0/0000:b5:00.0/uacce/hisi_zip-7
   ```

2. 若要使用hpre设备但是在[1](#li1760055514614)中未查询到，请按[初始化失败](#初始化失败)排查加速器软件是否已正确安装。

3. 若[2](#li1600175515610)已确认加速器软件正确安装，请排查通过lspci命令查看物理设备是否存在。

   ```shell
   # lspci | grep HPRE
   79:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon HPRE Engine (rev 21)
   b9:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon HPRE Engine (rev 21)
   ## lspci | grep SEC
   76:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon SEC Engine (rev 21)
   b6:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon SEC Engine (rev 21)
   ## lspci | grep RDE
   78:01.0 RAID bus controller: Huawei Technologies Co., Ltd. HiSilicon RDE Engine (rev 21)
   b8:01.0 RAID bus controller: Huawei Technologies Co., Ltd. HiSilicon RDE Engine (rev 21)
   ## lspci | grep ZIP
   75:00.0 Processing accelerators: Huawei Technologies Co., Ltd. HiSilicon ZIP Engine (rev 21)
   b5:00.0 Processing accelerators: Huawei Technologies Co., Ltd. HiSilicon ZIP Engine (rev 21)
   #
   ```

4. 若[3](#li1560012551369)未查询到相应的物理设备，请确认以下，不分先后：
   * 确认是否已导入加速器许可证，若未导入，请参考《[TaiShan 机架服务器 iBMC (V500及以上) 用户指南](https://support.huawei.com/enterprise/zh/doc/EDOC1100121687)》中“许可证管理”章节，导入加速器许可证。导入加速器许可证之后，需要掉电重启iBMC，使能License。
   * 确认iBMC和BIOS版本是否支持加速器特性。

### 升级加速器驱动失败

#### 故障现象

升级加速器驱动后，重启系统驱动版本仍为旧版本。

#### 可能原因

在升级加速器驱动前，系统更新了其他驱动包，这些驱动包可能重新更新了引导文件系统initramfs，将未升级前的加速器驱动一起更新到了initramfs文件系统中。例如系统更新了网卡驱动，或者人为更新了initramfs文件系统，导致系统重启时优先从initramfs文件系统中加载加速器驱动。

#### 处理步骤

升级加速器驱动版本后，通过执行dracut --force命令重新更新initramfs文件系统。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/installation_upgrade/installation/using_kickstart_for_automatic_installation.md
---
# 使用kickstart自动化安装

## 总体介绍

### 概述

用户可以使用kickstart工具进行openEuler系统的自动化安装，包括如下两种方式：

* 半自动化安装：安装人员不需要手动设定操作系统的键盘、语言、分区等具体属性（通过kickstart实现自动化），但是需要手动指定kickstart文件的位置。
* 全自动化安装：实现操作系统的安装过程全自动化。

### 优缺点对比

使用kickstart工具进行半自动化安装和全自动化安装的优缺点对比如[表1](#table1388812373315)所示，用户可以自行选择安装方式。

**表 1**  优缺点对比

### 背景知识

**kickstart**

kickstart是一种无人值守的安装方式。它的工作原理是在安装过程中记录典型的需要人工干预填写的各种参数，并生成一个配置文件（ks.cfg），在安装过程中，安装程序首先会查找ks配置文件，如果查找到合适的参数，就采用所找到的参数；如果未找到合适的参数，便需要安装者手工设定。所以，如果kickstart文件涵盖了安装过程中需要设定的所有参数，安装者只需要告诉安装程序从何处获取ks.cfg文件，就能实现系统安装的自动化。

kickstart 安装提供一个安装过程自动化的方法，可以是部分自动化，也可以是完全自动化。

kickstart 采用开源 [pykickstart](https://github.com/pykickstart/pykickstart)来对ks.cfg文件进行语法解析。

**PXE**

PXE（Pre-boot Execution Environment，预启动执行环境），工作于Client/Server的网络模式，支持PXE的客户端在启动过程中，能够从DHCP服务器获取IP结合TFTP（Trivial File Transfer Protocol）等协议可以实现客户端的网络引导和安装。

**TFTP**

TFTP（Trivial File Transfer Protocol，简单文件传输协议），该协议用来实现客户机与服务器之间的简单文件传输，它提供不复杂、开销不大的文件传输服务。

## 半自动化安装指导

### 环境要求

使用kickstart进行openEuler系统的半自动化安装的环境要求如下：

* 物理机/虚拟机（虚拟机创建可参考对应厂商的资料）： 包括使用kickstart工具进行自动化安装的计算机和被安装的计算机。
* httpd： 存放kickstart文件。
* ISO： openEuler-{版本号}-{架构}-dvd.iso。下文中均以openEuler-22.03-LTS-SP4-aarch64-dvd.iso作为示例。

### 操作步骤

使用kickstart进行openEuler系统的半自动化安装的操作步骤如下：

**环境准备**

> \[!NOTE]说明\
> 安装之前，请确保http服务器的防火墙处于关闭状态。关闭防火墙可参照如下命令：
>
> ```shell
> iptables -F  
> ```

1. httpd的安装与服务启动。

   ```shell
   # dnf install httpd -y
   # systemctl start httpd
   # systemctl enable httpd
   ```

2. kickstart文件的准备。

   ```shell
   # mkdir /var/www/html/ks
   # vim /var/www/html/ks/openEuler-ks.cfg ===>根据已安装openEuler系统自动生成的anaconda-ks.cfg修改得到
   ====================================
   ***以下内容需要根据实际需求进行修改***
   #version=DEVEL
   ignoredisk --only-use=sda
   autopart --type=lvm
   # Partition clearing information
   clearpart --none --initlabel
   # Use graphical install
   graphical
   # Use CDROM installation media
   cdrom
   # Keyboard layouts
   keyboard --vckeymap=cn --xlayouts='cn'
   # System language
   lang zh_CN.UTF-8

   # Network information
   network  --bootproto=dhcp --device=enp4s0 --ipv6=auto --activate
   network  --hostname=openeuler.com
   # Root password
   rootpw --iscrypted $6$fQE83lxEZ48Or4zc$j7/PlUMHn29yTjCD4Fi44WTZL/RzVGxJ/7MGsZMl6QfE3KjIVT7M4UrhFXbafvRq2lUddAFcyWHd5WRmXfEK20
   # Run the Setup Agent on first boot
   firstboot --enable
   # Do not configure the X Window System
   skipx
   # System services
   services --disabled="chronyd"
   # System timezone
   timezone Asia/Shanghai --isUtc--nontp

   %packages
   @^minimal-environment
   @standard

   %end

   %post
   #enable kdump
   sed  -i "s/ ro / ro crashkernel=1024M,high /" /boot/efi/EFI/openEuler/grub.cfg
   %end
   =====================================
   ```

   > \[!NOTE]说明\
   > 密码密文生成方式：
   >
   > ```shell
   > # python3  
   > Python 3.7.0 (default, Apr  1 2019, 00:00:00)  
   > [GCC 7.3.0] on linux  
   > Type "help", "copyright", "credits" or "license" for more information.  
   > >>> import crypt  
   > >>> passwd = crypt.crypt("myPasswd")  
   > >>> print (passwd)  
   > $6$63c4tDmQGn5SDayV$mZoZC4pa9Jdt6/ALgaaDq6mIExiOO2EjzomB.Rf6V1BkEMJDcMddZeGdp17cMyc9l9ML9ldthytBEPVcnboR/0  
   > ```

3. 将ISO镜像文件挂载到需要安装openEuler计算机的光驱上。

   另外，也可以选择NFS等网络安装，kickstart文件中需要指定安装源位置（默认是cdrom）。

**安装系统**

1. 启动系统进入安装选择界面。
   1. 在“[启动安装](./../installation/installation_guide.md#启动安装)”中的“安装引导界面”中选择“Install openEuler 22.03-LTS-SP4”，并按下“e”键。

   2. 启动参数中追加“inst.ks=http://{server\_ip}/ks/openEuler-ks.cfg”。

      ![](./figures/startparam.png)

   3. 按“Ctrl+x”，开始系统的自动安装。

2. 确认系统安装完毕。

   系统安装完毕以后会自动重启，如果优先从光驱启动，会再次进入到安装界面，此时关闭计算机，调整启动顺序（优先从硬盘启动）。

   ![](./figures/Automatic_installation_complete.png)

## 全自动化安装指导

### 环境要求

使用kickstart进行openEuler系统的全自动化安装的环境要求如下：

* 物理机/虚拟机（虚拟机创建可参考对应厂商的资料）：包括使用kickstart工具进行自动化安装的计算机和被安装的计算机。
* httpd：存放kickstart文件。
* tftp：提供vmlinuz和initrd文件。
* dhcpd/pxe：提供DHCP服务。
* ISO： openEuler-{版本号}-{架构}-dvd.iso。下文中均以openEuler-22.03-LTS-SP4-aarch64-dvd.iso作为示例。

### 操作步骤

使用kickstart进行openEuler系统的全自动化安装的操作步骤如下：

**环境准备**

> \[!NOTE]说明\
> 安装之前，请确保http服务器的防火墙处于关闭状态。关闭防火墙可参照如下命令：
>
> ```shell
> iptables -F  
> ```

1. httpd的安装与服务启动。

   ```shell
   # dnf install httpd -y
   # systemctl start httpd
   # systemctl enable httpd
   ```

2. tftp的安装与配置。

   ```shell
   # dnf install tftp-server xinetd -y
   # vim /etc/xinetd.d/tftp
   service tftp
   {
   socket_type = dgram
   protocol = udp
   wait = yes
   user = root
   server = /usr/sbin/in.tftpd
   server_args = -s /var/lib/tftpboot
   disable = no
   per_source = 11
   cps = 100 2
   flags = IPv4
   }
   # systemctl start tftp
   # systemctl enable tftp
   # systemctl start xinetd
   # systemctl status xinetd
   # systemctl enable xinetd
   ```

3. 安装源的制作。

   ```shell
   # mount openEuler-22.03-LTS-SP4-aarch64-dvd.iso /mnt
   # cp -r /mnt/* /var/www/html/openEuler/
   ```

4. 设置和修改kickstart配置文件 openEuler-ks.cfg，参考[3](#zh-cn_topic_0151920754_l1692f6b9284e493683ffa2ef804bc7ca)安装源的目录，此处选择http安装源。

   ```shell
   #vim  /var/www/html/ks/openEuler-ks.cfg
   ====================================
   ***以下内容根据实际需求进行修改***
   #version=DEVEL
   ignoredisk --only-use=sda
   autopart --type=lvm
   # Partition clearing information
   clearpart --none --initlabel
   # Use graphical install
   graphical
   # Keyboard layouts
   keyboard --vckeymap=cn --xlayouts='cn'
   # System language
   lang zh_CN.UTF-8
   #Use http installation source
   url --url=http://192.168.122.1/openEuler/
   %post
   #enable kdump
   sed  -i "s/ ro / ro crashkernel=1024M,high /" /boot/efi/EFI/openEuler/grub.cfg
   %end
   ...
   ```

5. 修改pxe配置文件grub.cfg， 可参考如下内容（注意：openEuler当前不支持bls格式的cfg文件，如果架构是x86\_64，请将grubaa64.efi替换成grubx64.efi）。

   ```shell
   # cp -r /mnt/images/pxeboot/* /var/lib/tftpboot/
   # cp /mnt/EFI/BOOT/grubaa64.efi /var/lib/tftpboot/
   # cp /mnt/EFI/BOOT/grub.cfg /var/lib/tftpboot/
   # ls /var/lib/tftpboot/
   grubaa64.efi  grub.cfg  initrd.img  TRANS.TBL  vmlinuz
   # vim /var/lib/tftpboot/grub.cfg
   set default="1"

   function load_video {
     if [ x$feature_all_video_module = xy ]; then
       insmod all_video
     else
       insmod efi_gop
       insmod efi_uga
       insmod ieee1275_fb
       insmod vbe
       insmod vga
       insmod video_bochs
       insmod video_cirrus
     fi
   }

   load_video
   set gfxpayload=keep
   insmod gzio
   insmod part_gpt
   insmod ext2

   set timeout=60


   ### BEGIN /etc/grub.d/10_linux ###
   menuentry 'Install openEuler 22.03-LTS-SP4' --class red --class gnu-linux --class gnu --class os {
           set root=(tftp,192.168.122.1)
           linux /vmlinuz ro inst.geoloc=0 console=ttyAMA0 console=tty0 rd.iscsi.waitnet=0 inst.ks=http://192.168.122.1/ks/openEuler-ks.cfg
           initrd /initrd.img
   }
   ```

6. DHCP的配置（可以使用dnsmasq代替，如果架构是x86\_64，请将grubaa64.efi替换成grubx64.efi）。

   ```shell
   # dnf install dhcp -y
   #
   # vim /etc/dhcp/dhcpd.conf
   #
   # DHCP Server Configuration file.
   #   see /usr/share/doc/dhcp-server/dhcpd.conf.example
   #   see dhcpd.conf(5) man page
   #
   ddns-update-style interim;
   ignore client-updates;
   filename "grubaa64.efi"; 　　 # pxelinux 启动文件位置;
   next-server 192.168.122.1;　　# (重要)TFTP Server 的IP地址;
   subnet 192.168.122.0 netmask 255.255.255.0 {
   option routers 192.168.122.1; # 网关地址
   option subnet-mask 255.255.255.0; # 子网掩码
   range dynamic-bootp 192.168.122.50 192.168.122.200; # 动态ip范围
   default-lease-time 21600;
   max-lease-time 43200;
   }
   # systemctl start dhcpd
   # systemctl enable dhcpd
   ```

**安装系统**

1. 在“Start boot option”界面按下“F2”选择从网络pxe启动，开始自动化安装。

   ![](./figures/zh-cn_image_0229291270.png)

   ![](./figures/zh-cn_image_0229291286.png)

   ![](./figures/zh-cn_image_0229291247.png)

2. 进入系统全自动化安装界面。

3. 确认系统安装完毕。

   ![](./figures/Automatic_installation_complete.png)

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/memory_storage/lvm/managing_drives_through_lvm.md
---
# 使用LVM管理硬盘

## LVM简介

LVM是逻辑卷管理（Logical Volume Manager）的简称，它是Linux环境下对磁盘分区进行管理的一种机制。LVM通过在硬盘和文件系统之间添加一个逻辑层，来为文件系统屏蔽下层硬盘分区布局，提高硬盘分区管理的灵活性。

使用LVM管理硬盘的基本过程如下：

1. 将硬盘创建为物理卷。
2. 将多个物理卷组合成卷组。
3. 在卷组中创建逻辑卷。
4. 在逻辑卷之上创建文件系统。

通过LVM管理硬盘之后，文件系统不再受限于硬盘的大小，可以分布在多个硬盘上，也可以动态扩容。

### 基本概念

* 物理存储介质（The physical media）：指系统的物理存储设备，如硬盘，系统中为/dev/hda、/dev/sda等等，是存储系统最低层的存储单元。

* 物理卷（Physical Volume，PV）：指硬盘分区或从逻辑上与磁盘分区具有同样功能的设备(如RAID)，是LVM的基本存储逻辑块。物理卷包括一个特殊的标签，该标签默认存放在第二个 512 字节扇区，但也可以将标签放在最开始的四个扇区之一。该标签包含物理卷的随机唯一识别符（UUID），记录块设备的大小和LVM元数据在设备中的存储位置。

* 卷组（Volume Group，VG）：由物理卷组成，屏蔽了底层物理卷细节。可在卷组上创建一个或多个逻辑卷且不用考虑具体的物理卷信息。

* 逻辑卷（Logical Volume，LV）：卷组不能直接用，需要划分成逻辑卷才能使用。逻辑卷可以格式化成不同的文件系统，挂载后直接使用。

* 物理块（Physical Extent，PE）：物理卷以大小相等的“块”为单位存储，块的大小与卷组中逻辑卷块的大小相同。

* 逻辑块（Logical Extent，LE）：逻辑卷以“块”为单位存储，在一卷组中的所有逻辑卷的块大小是相同的。

## 安装

> \[!NOTE]说明\
> openEuler操作系统默认已安装LVM。可通过**rpm -qa | grep lvm2**命令查询，若打印信息中包含“lvm2”信息，则表示已安装LVM，可跳过本章节内容；若无任何打印信息，则表示未安装，可参考本章节内容进行安装。

1. 配置本地yum源，详细信息请参考[搭建repo服务器](./../../administration/administrator/configuring_the_repo_server.md)。

2. 清除缓存。

   ```shell
   $ dnf clean all
   ```

3. 创建缓存。

   ```shell
   $ dnf makecache
   ```

4. 在root权限下安装LVM。

   ```shell
   # dnf install lvm2
   ```

5. 查看安装后的rpm包。

   ```shell
   $ rpm -qa | grep lvm2
   ```

## 管理物理卷

### 创建物理卷

可在root权限下通过pvcreate命令创建物理卷。

```shell
pvcreate [option] devname ...
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -f：强制创建物理卷，不需要用户确认。
  * -u：指定设备的UUID。
  * -y：所有的问题都回答“yes”。

* devname：指定要创建的物理卷对应的设备名称。如果需要批量创建，可以填写多个设备名称，中间以空格间隔。

示例1：将/dev/sdb、/dev/sdc创建为物理卷。

```shell
# pvcreate /dev/sdb /dev/sdc
```

示例2：将/dev/sdb1、/dev/sdb2创建为物理卷。

```shell
# pvcreate /dev/sdb1 /dev/sdb2
```

### 查看物理卷

可在root权限通过pvdisplay命令查看物理卷的信息，包括：物理卷名称、所属的卷组、物理卷大小、PE大小、总PE数、可用PE数、已分配的PE数和UUID。

```shell
pvdisplay [option] devname
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -s：以短格式输出。
  * -m：显示PE到LE的映射。

* devname：指定要查看的物理卷对应的设备名称。如果不指定物理卷名称，则显示所有物理卷的信息。

示例：显示物理卷/dev/sdb的基本信息。

```shell
# pvdisplay /dev/sdb
```

### 修改物理卷属性

可在root权限下通过pvchange命令修改物理卷的属性。

```shell
pvchange [option] pvname ...
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -u：生成新的UUID。
  * -x：是否允许分配PE。

* pvname：指定要修改属性的物理卷对应的设备名称。如果需要批量修改，可以填写多个设备名称，中间以空格间隔。

示例：禁止分配/dev/sdb物理卷上的PE。没有加入卷组的物理卷执行pvdisplay命令显示Allocatable属性为NO，需要加入卷组才能成功修改该属性。

```shell
# pvchange -x n /dev/sdb
```

### 删除物理卷

可在root权限下通过pvremove命令删除物理卷。

```shell
pvremove [option] pvname ...
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -f：强制删除物理卷，不需要用户确认。
  * -y：所有的问题都回答“yes”。

* pvname：指定要删除的物理卷对应的设备名称。如果需要批量删除，可以填写多个设备名称，中间以空格间隔。

示例：删除物理卷/dev/sdb。如果物理卷已经加入卷组，需要先删除卷组或者从卷组中移除，再删除物理卷。

```shell
# pvremove /dev/sdb
```

## 管理卷组

### 创建卷组

可在root权限下通过vgcreate命令创建卷组。

```shell
vgcreate [option] vgname pvname ...
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -l：卷组上允许创建的最大逻辑卷数。
  * -p：卷组中允许添加的最大物理卷数。
  * -s：卷组上的物理卷的PE大小。

* vgname：要创建的卷组名称。

* pvname：要加入到卷组中的物理卷名称。

示例：创建卷组vg1，并且将物理卷/dev/sdb和/dev/sdc添加到卷组中。

```shell
# vgcreate vg1 /dev/sdb /dev/sdc  
```

### 查看卷组

可在root权限下通过vgdisplay命令查看卷组的信息。

```shell
vgdisplay [option] [vgname]
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -s：以短格式输出。
  * -A：仅显示活动卷组的属性。

* vgname：指定要查看的卷组名称。如果不指定卷组名称，则显示所有卷组的信息。

示例：显示卷组vg1的基本信息。

```shell
# vgdisplay vg1
```

### 修改卷组属性

可在root权限下通过vgchange命令修改卷组的属性。

```shell
vgchange [option] vgname
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -a：设置卷组的活动状态。

* vgname：指定要修改属性的卷组名称。

示例：将卷组vg1状态修改为活动。

```shell
# vgchange -ay vg1
```

### 扩展卷组

可在root权限下通过vgextend命令动态扩展卷组。它通过向卷组中添加物理卷来增加卷组的容量。

```shell
vgextend [option] vgname pvname ...
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -d：调试模式。
  * -t：仅测试。

* vgname：要扩展容量的卷组名称。

* pvname：要加入到卷组中的物理卷名称。

示例：向卷组vg1中添加物理卷/dev/sdb。

```shell
# vgextend vg1 /dev/sdb
```

### 收缩卷组

可在root权限下通过vgreduce命令删除卷组中的物理卷来减少卷组容量。不能删除卷组中剩余的最后一个物理卷。

```shell
vgreduce [option] vgname pvname ...
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -a：如果命令行中没有指定要删除的物理卷，则删除所有的空物理卷。
  * \--removemissing：删除卷组中丢失的物理卷，使卷组恢复正常状态。

* vgname：要收缩容量的卷组名称。

* pvname：要从卷组中删除的物理卷名称。

示例：从卷组vg1中移除物理卷/dev/sdb2。

```shell
# vgreduce vg1 /dev/sdb2
```

### 删除卷组

可在root权限下通过vgremove命令删除卷组。

```shell
vgremove [option] vgname
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -f：强制删除卷组，不需要用户确认。

* vgname：指定要删除的卷组名称。

示例：删除卷组vg1。

```shell
# vgremove vg1
```

## 管理逻辑卷

### 创建逻辑卷

可在root权限下通过lvcreate命令创建逻辑卷。

```shell
lvcreate [option] vgname
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -L：指定逻辑卷的大小，单位为“kKmMgGtT”字节。
  * -l：指定逻辑卷的大小（LE数）。
  * -n：指定要创建的逻辑卷名称。
  * -s：创建快照。

* vgname：要创建逻辑卷的卷组名称。

示例1：在卷组vg1中创建10G大小的逻辑卷。

```shell
# lvcreate -L 10G vg1
```

示例2：在卷组vg1中创建200M的逻辑卷，并命名为lv1。

```shell
# lvcreate -L 200M -n lv1 vg1
```

### 查看逻辑卷

可在root权限下通过lvdisplay命令查看逻辑卷的信息，包括逻辑卷空间大小、读写状态和快照信息等属性。

```shell
lvdisplay [option] [lvname]
```

其中：

* option：命令参数选项。常用的参数选项有：

  * -v：显示LE到PE的映射。

* lvname：指定要显示属性的逻辑卷对应的设备文件。如果省略，则显示所有的逻辑卷属性。

  > \[!NOTE]说明\
  > 逻辑卷对应的设备文件保存在卷组目录下，例如：在卷组vg1上创建一个逻辑卷lv1，则此逻辑卷对应的设备文件为/dev/vg1/lv1。

示例：显示逻辑卷lv1的基本信息。

```shell
# lvdisplay /dev/vg1/lv1
```

### 调整逻辑卷大小

可在root权限下通过lvresize命令调整LVM逻辑卷的空间大小，可以增大空间和缩小空间。使用lvresize命令调整逻辑卷空间大小和缩小空间时需要谨慎，因为有可能导致数据丢失。

```shell
lvresize [option] vgname
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -L：指定逻辑卷的大小，单位为“kKmMgGtT”字节。
  * -l：指定逻辑卷的大小（LE数）。
  * -f：强制调整逻辑卷大小，不需要用户确认。

* lvname：指定要调整的逻辑卷名称。

示例1：为逻辑卷/dev/vg1/lv1增加200M空间。

```shell
# lvresize -L +200 /dev/vg1/lv1
```

示例2：为逻辑卷/dev/vg1/lv1减少200M空间。

```shell
# lvresize -L -200 /dev/vg1/lv1
```

### 扩展逻辑卷

可在root权限下通过lvextend命令动态在线扩展逻辑卷的空间大小，而不中断应用程序对逻辑卷的访问。

```shell
lvextend [option] lvname
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -L：指定逻辑卷的大小，单位为“kKmMgGtT”字节。
  * -l：指定逻辑卷的大小（LE数）。
  * -f：强制调整逻辑卷大小，不需要用户确认。

* lvname：指定要扩展空间的逻辑卷的设备文件。

示例：为逻辑卷/dev/vg1/lv1增加100M空间。

```shell
# lvextend -L +100M /dev/vg1/lv1
```

### 收缩逻辑卷

可在root权限下通过lvreduce命令减少逻辑卷占用的空间大小。使用lvreduce命令收缩逻辑卷的空间大小有可能会删除逻辑卷上已有的数据，所以在操作前必须进行确认。

```shell
lvreduce [option] lvname
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -L：指定逻辑卷的大小，单位为“kKmMgGtT”字节。
  * -l：指定逻辑卷的大小（LE数）。
  * -f：强制调整逻辑卷大小，不需要用户确认。

* lvname：指定要扩展空间的逻辑卷的设备文件。

示例：将逻辑卷/dev/vg1/lv1的空间减少100M。

```shell
# lvreduce -L -100M /dev/vg1/lv1
```

### 删除逻辑卷

可在root权限下通过lvremove命令删除逻辑卷。如果逻辑卷已经使用mount命令加载，则不能使用lvremove命令删除。必须使用umount命令卸载后，逻辑卷方可被删除。

```shell
lvremove [option] lvname
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -f：强制删除逻辑卷，不需要用户确认。

* lvname：指定要删除的逻辑卷设备文件名。

示例：删除逻辑卷/dev/vg1/lv1。

```shell
# lvremove /dev/vg1/lv1
```

## 创建并挂载文件系统

在创建完逻辑卷之后，需要在逻辑卷之上创建文件系统并挂载文件系统到相应目录下。

### 创建文件系统

可在root权限下通过mkfs命令创建文件系统。

```shell
mkfs [option] lvname
```

其中：

* option：命令参数选项。常用的参数选项有：
  * -t：指定创建的linux系统类型，如ext2，ext3，ext4等等，默认类型为ext2。

* lvname：指定要创建的文件系统对应的逻辑卷设备文件名。

示例：在逻辑卷/dev/vg1/lv1上创建ext4文件系统。

```shell
# mkfs -t ext4 /dev/vg1/lv1
```

### 手动挂载文件系统

手动挂载的文件系统仅在当时有效，一旦操作系统重启则会不存在。

可在root权限下通过mount命令挂载文件系统。

```shell
mount lvname mntpath
```

其中：

* lvname：指定要挂载文件系统的逻辑卷设备文件名。
* mntpath：挂载路径。

示例：将逻辑卷/dev/vg1/lv1挂载到/mnt/data目录。

```shell
# mount /dev/vg1/lv1 /mnt/data
```

### 自动挂载文件系统

手动挂载的文件系统在操作系统重启之后会不存在，需要重新手动挂载文件系统。但若在手动挂载文件系统后在root权限下进行如下设置，可以实现操作系统重启后文件系统自动挂载文件系统。

1. 执行blkid命令查询逻辑卷的UUID，逻辑卷以/dev/vg1/lv1为例。

   ```shell
   # blkid /dev/vg1/lv1
   ```

   查看打印信息，打印信息中包含如下内容，其中  *uuidnumber*  是一串数字，为UUID，  *fstype*  为文件系统。

   /dev/vg1/lv1: UUID="  *uuidnumber*  " TYPE="  *fstype*  "

2. 执行**vi /etc/fstab**命令编辑fstab文件，并在最后加上如下内容：

   ```shell
   UUID=uuidnumber  mntpath                   fstype    defaults        0 0
   ```

   内容说明如下：

   * 第一列：UUID，此处填写[1](#li65701520154311)查询的  *uuidnumber*  。
   * 第二列：文件系统的挂载目录  *mntpath*  。
   * 第三列：文件系统的文件格式，此处填写[1](#li65701520154311)查询的  *fstype*  。
   * 第四列：挂载选项，此处以“defaults”为例。
   * 第五列：备份选项，设置为“1”时，系统自动对该文件系统进行备份；设置为“0”时，不进行备份。此处以“0”为例。
   * 第六列：扫描选项，设置为“1”时，系统在启动时自动对该文件系统进行扫描；设置为“0”时，不进行扫描。此处以“0”为例。

3. 验证自动挂载功能。
   1. 执行umount命令卸载文件系统，逻辑卷以/dev/vg1/lv1为例。

      ```shell
      # umount /dev/vg1/lv1
      ```

   2. 执行如下命令，将/etc/fstab文件所有内容重新加载。

      ```shell
      # mount -a
      ```

   3. 执行如下命令，查询文件系统挂载信息，挂载目录以/mnt/data为例。

      ```shell
      # mount | grep /mnt/data
      ```

      查看打印信息，若信息中包含如下信息表示自动挂载功能生效。

      /dev/vg1/lv1 on /mnt/data

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/application_dev/using_make_for_compilation.md
---
# 使用make编译

本章介绍make编译的一些基本知识，并通过示例进行实际演示。更多的make选项请通过**man make**命令查询。

## 简介

GNU make实用程序（通常缩写为make）是一种用于控制从源文件生成可执行文件的工具。 make会自动确定复杂程序的哪些部分已更改并需要重新编译。 make使用称为Makefile的配置文件来控制程序的构建方式。

## 基本规则

### 文件类型

makefile文件中可能用到的文件类型如[表1](#table634145764320)所示。

**表 1**  文件类型

### make工作流程

使用make由源代码文件生成可执行文件，需要经过如下步骤：

1. make命令会读入Makefile文件，包括当前目录下命名为“GNUmakefile” 、“makefile” 、“Makefile”的文件、被include的makefile文件、参数-f、--file、--makefile指定的规则文件。
2. 初始化变量。
3. 推导隐含规则，分析依赖关系，并创建依赖关系链。
4. 根据依赖关系链，决定哪些目标需要重新生成。
5. 执行生成命令，最终输出终极文件。

### make选项

make命令格式为：**make**  \[*option*]... \[*target*]...

其中：

*option* ：参数选项。

*target* ：Makefile中指定的目标。

常用make的 *option* 取值如[表2](#table261872312343)所示。

**表 2**  常用的make选项

## Makefile

make是通过Makefile文件获取如何编译、链接和安装、清理的方法，从而实现将源代码文件生成可执行文件和其他相关文件的工具。因此，Makefile中描述了整个工程的编译和链接等规则，其中包含了哪些文件需要编译，哪些文件不需要编译，哪些文件需要先编译，哪些文件需要后编译，哪些文件需要重建等等。Makefile文件让工程编译实现了自动化，不需要每次都手动输入一堆源文件和参数。

本章简单介绍Makefile文件的结构和主要内容，更多Makefile的内容请通过**info make**命令查询。

### Makefile结构

Makefile文件结构如下所示：

*targets*:*prerequisites*

*command*

或者是：

*targets*:*prerequisites*;*command*

*command*

其中：

* *targets* ：目标，可以是目标文件、可执行文件或标签。
* *prerequisites* ：依赖文件，生成targets需要的文件或者是目标。可以是多个，也可以没有。
* *command* ：make需要执行的命令（任意的 shell 命令）。可以有多条命令，每一条命令占一行。
* 目标和依赖文件之间要使用“:”分隔，命令的开始一定要按“Tab”。

Makefile文件结构表明了输出的目标，输出目标的依赖对象和生成目标需要执行的命令。

### Makefile主要内容

一个Makefile文件主要由以下内容组成：

* 显式规则

  明确写出来的依赖关系，如要生成的文件，文件的依赖文件，生成的命令。

* 隐含规则

  由make自动推导的规则，make命令支持自动推导功能。

* 变量的定义

* 文件指示

  文件指示包括三部分：

  * include 其他 Makefile，如include xx.md。
  * 选择执行，如#ifdef。
  * 定义多行命令，如define...endef。

* 注释

  以 “#” 开头。

## 示例

### 使用Makefile实现编译的示例

1. cd到代码目录，以目录“~/code”进行举例。

   ```shell
   $ cd ~/code
   ```

2. 创建1个头文件hello.h和2个函数hello.c、main.c。

   ```c
   $ vi hello.h
   $ vi hello.c
   $ vi main.c
   ```

   hello.h代码内容示例：

   ```c
   #pragma once
   #include <stdio.h>
   void hello();
   ```

   hello.c代码内容示例：

   ```c
   #include "hello.h"
   void hello()
   {
           int i=1;
           while(i<5)
           {
                   printf("The %dth say hello.\n", i);
                   i++;
           }
   }

   ```

   main.c代码内容示例：

   ```c
   #include "hello.h"
   #include <stdio.h>
   int main()
   {
           hello();
           return 0;
   }
   ```

3. 创建Makefile文件。

   ```shell
   $ vi Makefile
   ```

   Makefile文件内容示例：

   ```text
   main:main.o hello.o
           gcc -o main main.o hello.o
   main.o:main.c
           gcc -c main.c
   hello.o:hello.c
           gcc -c hello.c
   clean:
           rm -f hello.o main.o main
   ```

4. 执行make命令。

   ```shell
   $ make
   ```

   命令执行后，会打印Makefile中执行的命令。如果不需要打印该信息，可以在执行make命令时加上参数-s。

   gcc -c main.c

   gcc -c hello.c

   gcc -o main main.o hello.o

5. 执行./main目标。

   ```shell
   $ ./main
   ```

   命令执行后，打印如下信息：

   The 1th say hello.

   The 2th say hello.

   The 3th say hello.

   The 4th say hello.

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/performance/powerapi/development_using_powerapi.md
---
# 使用powerapi进行开发

## 下载源码（可选）

源码下载：<https://atomgit.com/openeuler/powerapi>

源码中部分接口还处于实验阶段，并未发布，代码中使用 RELEASE\_MODE 对接口是否正式发布进行了隔离。

当前已发布接口参考源码库中的 API 文档。

## 安装软件包

使用以下命令安装 powerapi-devel 软件包，使用提供的接口进行开发。

```sh
yum install powerapi-devel
```

## 基本流程

查询类接口调用只需要注册，设置类接口调用前除了注册还需要请求控制权。按业务场景分，业务流程如下：\
离散型感知类业务场景：设置日志回调 -> 注册 -> 查询接口调用 -> 注销\
配置类业务场景：设置日志回调 -> 注册 -> 请求控制权 -> 配置接口调用 -> 释放控制权 -> 注销

## powerapi接口

### 通用接口

#### 设置日志回调函数

接口定义：PWR\_API int PWR\_SetLogCallback(void(LogCallback)(int level, const char \*fmt, va\_list vl))\
接口描述：设置日志打印回调，设置后，powerapi库将回调LogCallBack函数打印日志。如不设置，默认打印到控制台。该接口可在注册前调用。

参数描述：

| 参数        | 类型    | 描述                 |
| ----------- | ------- | -------------------- |
| LogCallBack | void(\*) | 日志打印回调函数指针 |

返回值：

| 类型 | 描述                                           |
| ---- | ---------------------------------------------- |
| int  | 返回码0：成功4：失败，回调函数指针为空 |

***

#### 设置服务端信息

接口定义：PWR\_API int PWR\_SetServerInfo(const char\* socketPath)\
接口描述：该接口用于设置unix domain socket通信服务端地址信息。\
参数描述：

| 参数       | 类型        | 描述               |
| ---------- | ----------- | ------------------ |
| socketPath | const char\* | 服务端地址路径信息 |

返回值：

| 类型 | 描述                                           |
| ---- | ---------------------------------------------- |
| int  | 返回码0：成功4：失败，回调函数指针为空 |

注：\
服务端sock文件默认路径为/etc/sysconfig/pwrapis/pwrserver.sock，若在pwrapis配置文件中修改默认路径，则需要保证目录权限为755，而文件权限为722。
修改路径后需要在注册前通过该接口来告知so修改后的路径，否则连接将失败。

***

#### 注册

接口定义：PWR\_API int PWR\_Register(void)\
接口描述：用于注册到powerapi service。\
参数描述：无参数\
返回值：

| 类型 | 描述                                               |
| ---- | -------------------------------------------------- |
| int  | 返回码0：成功1：失败，初始化sock客户端失败 |

***

#### 注销

接口定义：PWR\_API int PWR\_UnRegister(void)\
接口描述：用于从服务端注销。\
参数描述：无参数\
返回值：

| 类型 | 描述                  |
| ---- | --------------------- |
| int  | 返回码0：注销成功 |

***

#### 请求接管能效控制权

接口定义：PWR\_API int PWR\_RequestControlAuth(void)\
接口描述：该接口用于请求对系统能效控制权进行接管。上层应用接管后，系统不会自动进行能效调控，否则会自动进行能效调控。\
参数描述：无参数\
返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 放弃能效控制权

接口定义：PWR\_API int PWR\_ReleaseControlAuth(void)\
接口描述：该接口用于释放对系统能效的控制权。\
参数描述：无参数\
返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置PowerAPI客户端的socket文件路径

接口定义：int PWR\_SetClientSockPath(const char\* socketPath)

接口描述：该接口用于设置PowerAPI客户端的socket文件路径。 客户端sock文件默认路径为程序当前用户所在home目录(如/home/user/pwrclient.sock)。

参数描述：

| 参数       | 类型        | 描述                 |
| ---------- | ----------- | -------------------- |
| socketPath | const char\* | 客户端socket文件路径 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. 该接口必须在注册前调用才能让新设置的socket文件路径生效。
2. 调用该接口设置的路径必须是存在并且当前用户有读写权限的路径。

***

#### 设置数据采集任务的回调函数

接口定义：int PWR\_SetMetaDataCallback(void(MetaDataCallback)(const PWR\_COM\_CallbackData \*callbackData))

接口描述：Powerapi目前支持部分功耗相关数据采集，此接口用户可设置回调函数，在回调函数中处理这些数据。

参数描述：

| 参数             | 类型    | 描述                   |
| ---------------- | ------- | ---------------------- |
| MetaDataCallback | void(\*) | 采集数据的回调函数指针 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. 该接口必须在启动采集任务前进行设置，否则不处理数据。
2. 回调函数中应该尽量轻量化，不宜做过多逻辑处理，尤其是比较费时的逻辑处理。

***

#### 创建数据采集任务

接口定义：int PWR\_CreateDcTask(const PWR\_COM\_BasicDcTaskInfo \*basicDcTaskInfo)

接口描述：该接口用于创建周期性数据采集任务，目前暂时只支持两种类型的数据采集：CPU使用率和CPU性能数据。

参数描述：

| 参数            | 类型                     | 描述                                               |
| --------------- | ------------------------ | -------------------------------------------------- |
| basicDcTaskInfo | PWR\_COM\_BasicDcTaskInfo\* | 数据采集任务基础信息，包含采集的周期、采集数据类型 |
*PWR\_COM\_BasicDcTaskInfo* 结构体具体定义，可以查看头文件`pwrdata.h`中具体定义

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. 创建的采集任务是以数据类型为单位，多次创建同一数据类型的采集任务，内部是只有一个这个数据类型的采集任务。

***

#### 删除数据采集任务

接口定义：int PWR\_DeleteDcTask(PWR\_COM\_COL\_DATATYPE dataType)

接口描述：该接口根据数据类型删除数据采集任务。

参数描述：

| 参数     | 类型                 | 描述               |
| -------- | -------------------- | ------------------ |
| dataType | PWR\_COM\_COL\_DATATYPE | 数据采集的数据类型 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置事件回调函数

接口定义：int PWR\_SetEventCallback(void(EventCallback)(const PWR\_COM\_EventInfo\* eventInfo))

接口描述：PowerAPI内部定义了一些事件，用户可以设置回调函数，在回调函数中处理这些事件。目前支持认证失败和控制权被抢占两个事件。

参数描述：

| 参数          | 类型    | 描述             |
| ------------- | ------- | ---------------- |
| EventCallback | void(\*) | 事件回调函数指针 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. 回调函数的实现应轻量化，不宜做过多逻辑处理，尤其是比较费时的逻辑处理，否则可能导致由于数据拥塞而丢失数据。

***

### CPU

#### 获取CPU信息

接口定义：PWR\_API int PWR\_CPU\_GetInfo(PWR\_CPU\_Info \*cpuInfo)\
接口描述：获取CPU信息，包含CPU基础信息，NUMA信息。\
参数描述：

| 参数    | 类型          | 描述    |
| ------- | ------------- | ------- |
| cpuinfo | PWR\_CPU\_Info\* | CPU信息 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 获取CPU使用率

接口定义：int PWR\_CPU\_GetUsage(PWR\_CPU\_Usage \*usage, uint32\_t bufferSize)

接口描述：获取CPU全部核心的平均使用率，以及每个核心分别的使用率。

参数描述：

| 参数       | 类型           | 描述                    |
| ---------- | -------------- | ----------------------- |
| Usage      | PWR\_CPU\_Usage\* | CPU使用率信息           |
| bufferSize | uint32\_t       | CPU使用率信息内存块大小 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. 该接口会阻塞当前线程500ms，如果是比较频繁的获取CPU使用率，建议采用任务方式，参见[创建数据采集任务](#创建数据采集任务)

***

#### 获取CPU性能数据

接口定义：int PWR\_CPU\_GetPerfData(PWR\_CPU\_PerfData \*perfData)

接口描述：获取CPU性能数据，实时获取CPU单指令平均LLC miss数和IPC(统计时长100ms)

参数描述：

| 参数     | 类型              | 描述            |
| -------- | ----------------- | --------------- |
| perfData | PWR\_CPU\_PerfData\* | CPU性能数据信息 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. 该接口会阻塞当前线程100ms，如果是比较频繁的获取CPU性能数据，建议采用任务方式，参见[创建数据采集任务](#创建数据采集任务)

***

#### 获取CPU Frequency能力

接口定义：PWR\_API int PWR\_CPU\_GetFreqAbility(PWR\_CPU\_FreqAbility \*freqAbi, uint32\_t bufferSize)\
参数描述：查询可用的CPUFreq调频域信息，可用governor和当前使用的CPUFreq驱动。

| 参数       | 类型                 | 描述                                                                                                                                           |
| ---------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| freqAbi    | PWR\_CPU\_FreqAbility\* | CPUFreq能力信息                                                                                                                                |
| bufferSize | uint32\_t             | freqAbi内存块大小。该内存块建议大小：sizeof(PWR\_CPU\_FreqAbility)+CPU核心数\*(sizeof(int)+5)内存块太小，则只返回能容纳的调频域数据。 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 获取CPU Frequency governor

接口定义：PWR\_API int PWR\_CPU\_GetFreqGovernor(char gov\[], uint32\_t size)\
接口描述：获取系统当前使用的CPUFreq调频器（默认获取第一个调频域的governor）。\
参数描述：

| 参数 | 类型     | 描述                                                          |
| ---- | -------- | ------------------------------------------------------------- |
| gov  | char\[]   | governor的名称，最大长度31。                                  |
| size | uint32\_t | 给定gov数组空间大小，需大于等于PWR\_MAX\_ELEMENT\_NAME\_LEN(32)。 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置CPU Frequency governor

接口定义：PWR\_API int PWR\_CPU\_SetFreqGovernor(const char gov\[])\
接口描述：设置系统当前使用的CPUFreq调频器（设置时将所有调频域设置为输入的governor）。\
参数描述：

| 参数 | 类型   | 描述                                                                                                                               |
| ---- | ------ | ---------------------------------------------------------------------------------------------------------------------------------- |
| gov  | char\[] | governor的名称，最大长度31。取值如：conservativeondemanduserspacepowersaveperformanceschedutilseep |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 获取CPU Frequency governor所有属性值

接口定义：PWR\_API int PWR\_CPU\_GetFreqGovAttrs(PWR\_CPU\_FreqGovAttrs \*govAttrs)\
接口描述：获取当前所使用的CPUFreq调频器的所有属性信息。\
参数描述：

| 参数     | 类型                  | 描述           |
| -------- | --------------------- | -------------- |
| govAttrs | PWR\_CPU\_FreqGovAttrs\* | 调频器属性信息 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 获取CPU Frequency governor属性

接口定义：PWR\_API int PWR\_CPU\_GetFreqGovAttr(PWR\_CPU\_FreqGovAttr \*govAttr)\
接口描述：获取当前使用的CPUFreq属性（系统将获取第一个调频域（policy0）当前使用的governor对应的属性）。\
参数描述：

| 参数     | 类型                  | 描述           |
| -------- | --------------------- | -------------- |
| govAttrs | PWR\_CPU\_FreqGovAttrs\* | 调频器属性信息 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置CPU Frequency governor属性

接口定义：PWR\_API int PWR\_CPU\_SetFreqGovAttr(const PWR\_CPU\_FreqGovAttr \*govAttr)\
接口描述：设置当前使用的CPUFreq属性（系统将设置第一个调频域（policy0）当前使用的governor对应的属性）。\
参数描述：

| 参数     | 类型                  | 描述                                                  |
| -------- | --------------------- | ----------------------------------------------------- |
| govAttrs | PWR\_CPU\_FreqGovAttrs\* | 调频器属性信息。需指定要设置的attr名称及其value。 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注：\
不同的governor所支持的属性有所差异，governor所支持的属性一般在以下路径：/sys/devices/system/cpu/cpufreq/{gov}/，其中{gov}为当前使用的调频器名称。

***

#### 获取CPU工作频率范围

接口定义：PWR\_API int PWR\_CPU\_GetFreqRange(PWR\_CPU\_FreqRange \*freqRange)\
接口描述：获取CPU工作频率范围（默认获取第一个调频域的工作频率范围）。\
参数描述：

| 参数      | 类型               | 描述                 |
| --------- | ------------------ | -------------------- |
| freqRange | PWR\_CPU\_FreqRange\* | 所获取的工作频率范围 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置CPU工作频率范围

接口定义：PWR\_API int PWR\_CPU\_SetFreqRange(const PWR\_CPU\_FreqRange \*freqRange)\
接口描述：设置CPU工作频率范围（把所有调频域设置成指定的频率范围）。\
参数描述：

| 参数      | 类型               | 描述               |
| --------- | ------------------ | ------------------ |
| freqRange | PWR\_CPU\_FreqRange\* | 设置的工作频率范围 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 获取CPU当前频率

接口定义：PWR\_API int PWR\_CPU\_GetFreq(PWR\_CPU\_CurFreq curFreq\[], uint32\_t \*num, int spec)\
接口描述：获取调频域的当前频率。\
参数描述：

| 参数    | 类型              | 描述                                                                                                                               |
| ------- | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| curFreq | PWR\_CPU\_CurFreq\[] | 表示要查询的policy当前调频域的频率信息。spec=1时需要设置对应成员的policyId。输出调频域的当前频率。                         |
| num     | uint32\_t \*        | curFreq数组空间长度，代表要查询的policy的个数。输出时为系统实际返回的有效数据长度（表现为实际policy的个数与输入num的较小值）。 |
| spec    | int               | 释放获取特定某个或某几个调频域的信息。0：否1：是，此时需要在curFreq成员中设定具体的调频域所对应的policyId，如32，64等。    |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置CPU当前频率

接口定义：PWR\_API int PWR\_CPU\_SetFreq(const PWR\_CPU\_CurFreq curFreq\[], uint32\_t num)\
接口描述：设置调频域的工作频率（CPUFreq governor为userspace时才可设置）。\
参数描述：

| 参数    | 类型              | 描述                                      |
| ------- | ----------------- | ----------------------------------------- |
| curFreq | PWR\_CPU\_CurFreq\[] | 要设置的调频域及其频率列表。              |
| num     | uint32\_t          | curFreq数组空间长度，即调整policy的个数。 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 获取CPU休眠能力和状态信息

接口定义：PWR\_API int PWR\_CPU\_GetIdleInfo(PWR\_CPU\_IdleInfo \*idleInfo)\
接口描述：获取CPUIdele能力和状态信息。\
参数描述：

| 参数     | 类型              | 描述                   |
| -------- | ----------------- | ---------------------- |
| idleInfo | PWR\_CPU\_IdleInfo\* | CPUIdle能力和状态信息 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 获取CPU休眠governor

接口定义：PWR\_API int PWR\_CPU\_GetIdleGovernor(char idleGov\[], uint32\_t size)\
接口描述：获取CPU休眠模式。\
参数描述：

| 参数    | 类型     | 描述                                                  |
| ------- | -------- | ----------------------------------------------------- |
| idleGov | char\[]   | governor名称                                          |
| size    | uint32\_t | idleGov缓存区大小，最小值PWR\_MAX\_ELEMENT\_NAME\_LEN(32) |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置CPU休眠governor

接口定义：PWR\_API int PWR\_CPU\_SetIdleGovernor(const char idleGov\[])\
接口描述：设置CPU休眠模式。\
参数描述：

| 参数    | 类型   | 描述                                                    |
| ------- | ------ | ------------------------------------------------------- |
| idleGov | char\[] | governor名称，如：laddermenuteohaltpoll |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 获取CPU\&DMA时延

接口定义：PWR\_API int PWR\_CPU\_DmaGetLatency(int \*latency)\
接口描述：获取CPU\&DMA可忍受时延。\
参数描述：

| 参数    | 类型 | 描述               |
| ------- | ---- | ------------------ |
| latency | int\* | 时延（单位：us）取值范围：\[0, 2000000000] |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置CPU\&DMA时延

接口定义：PWR\_API int PWR\_CPU\_DmaSetLatency(int latency)\
接口描述：设置CPU\&DMA可忍受时延。\
参数描述：

| 参数    | 类型 | 描述                                       |
| ------- | ---- | ------------------------------------------ |
| latency | int\* | 时延（单位：us）。取值范围：\[0,2000000000] |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注：\
不同的C-state，被唤醒所需时间不同，睡得越深，唤醒时延越大，因而系统在进入C-state前会判断CPU\&DMA latency，如果对应C-state的时延大于CPU\&DMA latency，则不会进入C-state。\
各个C-state唤醒时延参考（单位：us）

| C-state name | latency |
| ------------ | ------- |
| C0 POLL      | 0       |
| C1           | 2       |
| C1E          | 10      |
| C3           | 40      |
| C6           | 133     |
| C7S          | 166     |
| C8           | 300     |
| C9           | 600     |
| C10          | 2600    |

***

### 系统类接口

#### 设置整系统电源状态

接口定义：int PWR\_SYS\_SetPowerState(int powerState)

接口描述：

1. 该接口用于进行系统级休眠

参数描述：

| 参数  | 类型              | 描述                                         |
| ----- | ----------------- | -------------------------------------------- |
| powerState | int | 1 - 休眠RAM2 - 休眠Disk |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. 系统进入休眠后，如果重新激活系统，需要通过其他方式(如通过BMC物理端口的IPMI接口)远程激活系统。

***

#### 获取系统当前功耗信息

接口定义：int PWR\_SYS\_GetRtPowerInfo(PWR\_SYS\_PowerInfo \* powerInfo)

接口描述：

1. 查询系统实时能耗信息，包括整机当前功率、CPU当前功率、内存当前功率

参数描述：

| 参数  | 类型              | 描述                                         |
| ----- | ----------------- | -------------------------------------------- |
| powerInfo | PWR\_SYS\_PowerInfo | 包含sysPower/cpuPower/memPower三部分，这里是最终获取到功耗信息的出参内存地址 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. ARM平台暂时无法获取到CPU功耗信息和内存功耗信息

***

### HBM类接口

#### 查询HBM能力和运行模式

接口定义：int PWR\_HBM\_GetSysState(PWR\_HBM\_SYS\_STATE \*state)
接口描述：

1. 查询当前设备是否支持HBM，并查询HBM当前运行模式为flat模式、还是cache模式或混合模式。
2. HBM一般支持flat和cache两种运行模式，前者OS会将HBM作为cpuless的numa节点使用，而后者当做LLC使用。两种方式都支持进行上下电操作。

参数描述：

| 参数  | 类型              | 描述                                         |
| ----- | ----------------- | -------------------------------------------- |
| state | PWR\_HBM\_SYS\_STATE | HBM系统状态，包括是否支持HBM以及当前运行模式 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 关闭/启动所有HBM

接口定义：int PWR\_HBM\_SetAllPwrState(int state)

接口描述：

1. 关闭/启动所有HBM，包括cache模式和flat模式。

参数描述：

| 参数  | 类型 | 描述                             |
| ----- | ---- | -------------------------------- |
| state | int  | 0：关闭所有HBM1：启动所有HBM |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

### proc进程相关接口

#### 根据关键字查找任务

接口定义: int PWR\_PROC\_QueryProcs(const char \*keywords, pid\_t procs\[], uint32\_t \*num)

接口描述：

1. 根据关键字查找进程，并返回进程列表。

参数描述：

| 参数     | 类型        | 描述                                                                                                                                                                         |
| -------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| keywords | const char\* | 关键字列表。多个关键字用"|"隔开，例如 "nginx|mysql", 注意，空格会被认为是关键字一部分输入NULL或空字符串时表示查询所有用户态的任务不包含结束符'\0'，长度范围\[0,999] |
| procs    | pid\_t\*      | 进程列表                                                                                                                                                                     |
| num      | uint32\_t\*   | 进程列表长度                                                                                                                                                                 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 查询瓦特调度开关状态

瓦特调度说明：

1. 瓦特调度是一种为节省功耗，在内核采取的调度机制。在负载比较低时，系统会将任务集中在部分numa节点，为其他节点创造进入低功耗的条件。随着负载变化，系统会动态地扩缩用于完成任务的numa节点。
2. 约束：当前瓦特调度在x86内核中是默认是不开启，如果需要开启需要QOS\_SCHED\_DYNAMIC\_AFFINITY和QOS\_SCHED\_SMART\_GRID这两个配置，重新编译内核并替换内核。

接口定义: int PWR\_PROC\_GetWattState(int \*state)

接口描述：

1. 查询瓦特调度开关状态。

参数描述：

| 参数  | 类型 | 描述                                   |
| ----- | ---- | -------------------------------------- |
| state | int\* | 瓦特调度开关状态，0表示关闭，1表示打开 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置瓦特调度开关

接口定义: int PWR\_PROC\_SetWattState(int state)

接口描述：

1. 设置瓦特调度开关状态。

参数描述：

| 参数  | 类型 | 描述                                   |
| ----- | ---- | -------------------------------------- |
| state | int  | 瓦特调度开关状态，0表示关闭，1表示打开 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置瓦特调度第一调度域

接口定义: int PWR\_PROC\_SetWattFirstDomain(int cpuId)

接口描述：

1. 该接口用于指定瓦特调度使用的第一个调度域。
2. 默认情况下，在启动瓦特调度时，系统选择最空闲的NUMA node作为第一个调度域(level0)，后续在扩缩算力时，根据NUMA node的亲和性进行扩缩(例如：对于2颗CPU芯片，4个NUMA node的设备，node2为第一个调度域时，优先会扩展同一个socket下的node3, 然后是另一个socket下的node1和node0)。
3. 有些情况下，由于IRQ亲和性等原因，为了保证性能，需要指定第一个调度域为IRQ所在的NUMA node。

参数描述：

| 参数  | 类型 | 描述                        |
| ----- | ---- | --------------------------- |
| cpuId | int  | 设置cpuId所在域为第一调度域 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. 设置的CPU是在线和可调度状态。
2. 瓦特调度需要是关闭状态。

***

#### 获取瓦特调度属性

瓦特调度属性说明：

| 参数名         | 取值                                                                                                                                                                                                             | 含义                                                                                                           | 实际路径                                               |
| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
| scaleThreshold | \[0, 100] 默认85                                                                                                                                                                                                  | 扩缩资源的阈值。 即CPU利用率达到该阈值后，瓦特调度自动扩展计算资源。 属性设置时，该值为0，表示不设置该属性 | /proc/sys/kernel/sched\_util\_low\_pct                    |
| domainMask     | 4个numa node场景（有Level0-3共四个等级，分别表示使用1/2/3/4个numa node），默认值为15，即二进制1111，表示打开所有资源等级。如果不想打开Level2(Level1不满足算力时，直接扩展为level3)，则可以设置为“1011”，即11 | 资源域掩码。属性设置时，该值为0，表示不设置该属性                                                          | /sys/fs/cgroup/cpu/watt\_sched/cpu.affinity\_domain\_mask |
| scaleInterval  | \[0，3600000] 默认2000ms                                                                                                                                                                                          | 系统使用该值作为设置定时器，触发瓦特调度扩缩算力资源。属性设置时，该值为0，表示不设置该属性                | /sys/fs/cgroup/cpu/watt\_sched/cpu.affinity\_period\_ms   |

接口定义: int PWR\_PROC\_GetWattAttrs(PWR\_PROC\_WattAttrs \*wattAttrs)

接口描述：

1. 获取瓦特调度属性。

参数描述：

| 参数      | 类型               | 描述         |
| --------- | ------------------ | ------------ |
| wattAttrs | PWR\_PROC\_WattAttrs | 瓦特调度属性 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置瓦特调度属性

接口定义: int PWR\_PROC\_SetWattAttrs(PWR\_PROC\_WattAttrs \*wattAttrs)

接口描述：

1. 设置瓦特调度属性。

参数描述：

| 参数      | 类型               | 描述         |
| --------- | ------------------ | ------------ |
| wattAttrs | PWR\_PROC\_WattAttrs | 瓦特调度属性 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 查询当前瓦特调度任务列表

接口定义：int PWR\_PROC\_GetWattProcs(pid\_t wattProcs\[], uint32\_t \*num)

接口描述：

1. 查询有哪些任务已经加入到瓦特调度中

参数描述：

| 参数      | 类型     | 描述                 |
| --------- | -------- | -------------------- |
| wattProcs | pid\_t    | 瓦特调度任务列表，最大5000个。需要和num个数匹配|
| num       | uint32\_t | 瓦特调度任务列表数量，最大值5000 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 添加瓦特调度任务

接口定义：int PWR\_PROC\_AddWattProcs(const pid\_t wattProcs\[], uint32\_t num)

接口描述：

1. 添加任务到瓦特调度中

参数描述：

| 参数      | 类型     | 描述                           |
| --------- | -------- | ------------------------------ |
| wattProcs | pid\_t    | 瓦特调度任务列表，最大5000个。需要和num个数匹配|
| num       | uint32\_t | 瓦特调度任务列表数量，最大5000 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. 子进程会继承父进程/线程的瓦特调度属性。即如果父进程/线程已经在瓦特调度组，则生成的子进程/线程也会在该调度组

***

#### 删除瓦特调度任务

接口定义：int PWR\_PROC\_DelWattProcs(const pid\_t wattProcs\[], uint32\_t num)

接口描述：

1. 删除任务从瓦特调度中

参数描述：

| 参数      | 类型     | 描述                           |
| --------- | -------- | ------------------------------ |
| wattProcs | pid\_t    | 瓦特调度任务列表               |
| num       | uint32\_t | 瓦特调度任务列表数量，最大5000 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 查询smart\_grid开关状态

smart\_grid说明：

1. smart\_grid智能网格是一种为节省功耗，在瓦特调度的基础上内核的一种CPU资源划分和标识方法。smart\_grid可以将算力资源和任务划分不同等级，高等级任务用高算力来运行，低等级任务用低算力运行。
2. 为使smart\_grid生效，需要先开启smart\_grid，并批量设置任务的smart\_grid\_level。
3. openEuler 22.03 SP2及以上版本才支持smart\_grid功能。
4. 开启smart\_grid前，开启瓦特调度才能让smart\_grid功能生效。

接口定义：int PWR\_PROC\_GetSmartGridState(int \*state)

接口描述：

1. 查询smart\_grid开关状态

参数描述：

| 参数  | 类型 | 描述               |
| ----- | ---- | ------------------ |
| state | int  | 0：关闭1：开启 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置smart\_grid开关状态

接口定义：int PWR\_PROC\_SetSmartGridState(int state)

接口描述：

1. 设置smart\_grid开关状态

参数描述：

| 参数  | 类型 | 描述               |
| ----- | ---- | ------------------ |
| state | int  | 0：关闭1：开启 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 根据smart\_grid\_level查询任务列表

smart\_grid等级说明：

1. 当前只支持两个等级：0-高等级， 1-低等级。
2. 处于高等级的任务将会被调度到性能域，而处于低等级的任务将会被调度到能效域。
3. 子进程/线程会继承父进程/线程的smart\_grid等级属性。即如果父进程/线程的等级是0，则生成的子进程/线程等级也是0。
4. 已经存在的子进程/线程不会随父进程/线程改变smart\_grid等级属性。

接口定义：int PWR\_PROC\_GetSmartGridProcs(PWR\_PROC\_SMART\_GRID\_LEVEL level, PWR\_PROC\_SmartGridProcs \*sgProcs)

接口描述：

1. 根据smart\_grid\_level查询任务列表

参数描述：

| 参数    | 类型                      | 描述                                                                                                         |
| ------- | ------------------------- | ------------------------------------------------------------------------------------------------------------ |
| level   | PWR\_PROC\_SMART\_GRID\_LEVEL | 0：高等级1：低等级                                                                                       |
| sgProcs | PWR\_PROC\_SmartGridProcs   | 任务列表。其中：procNum作为输入，指定procs空间所能存储的任务数量。作为输出，标识实际返回的任务数量。 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 批量设置任务的smart\_grid\_level

接口定义：int PWR\_PROC\_SetSmartGridLevel(const PWR\_PROC\_SmartGridProcs \*sgProcs)

接口描述：

1. 批量设置任务的smart\_grid等级

参数描述：

| 参数    | 类型                    | 描述     |
| ------- | ----------------------- | -------- |
| sgProcs | PWR\_PROC\_SmartGridProcs | 任务列表 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 获取smart\_grid的调频器

smart\_gird调频器可设置参数：

| 参数名       | 取值                                                                                  | 含义                                                                           | 实际路径                                                   |
| ------------ | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | ---------------------------------------------------------- |
| sgAgentstate | 0或1                                                                                  | smart\_grid调频器状态。smart\_grid调频器状态为1时会覆盖cpufreq设置的governor | /sys/devices/system/cpu/cpufreq/smart\_grid\_governor\_enable |
| sgLevel0Gov  | 最大长度31，取值范围：conservative/ondemand/userspace/powersave/performance/schedutil | level0的调频器，需要smart\_grid开启才有效                                       | /sys/devices/system/cpu/cpufreq/smart\_grid\_governor        |
| sgLevel1Gov  | 最大长度31，取值范围：conservative/ondemand/userspace/powersave/performance/schedutil | level1的调频器，需要smart\_grid开启才有效                                       | /sys/devices/system/cpu/cpufreq/smart\_grid\_governor        |

接口定义：int PWR\_PROC\_GetSmartGridGov(PWR\_PROC\_SmartGridGov \*sgGov)

接口描述：

1. 获取smart\_grid的调频器

参数描述：

| 参数  | 类型                  | 描述       |
| ----- | --------------------- | ---------- |
| sgGov | PWR\_PROC\_SmartGridGov | 调频器参数 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置smart\_grid的调频器

接口定义：int PWR\_PROC\_SetSmartGridGov(const PWR\_PROC\_SmartGridGov \*sgGov)

接口描述：

1. 设置smart\_grid的调频器

参数描述：

| 参数  | 类型                  | 描述       |
| ----- | --------------------- | ---------- |
| sgGov | PWR\_PROC\_SmartGridGov | 调频器参数 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 获取服务的状态

接口定义：int PWR\_PROC\_GetServiceState(PWR\_PROC\_ServiceStatus \*sStatus)

接口描述：

1. 获取服务的状态

参数描述：

| 参数  | 类型                  | 描述       |
| ----- | --------------------- | ---------- |
| state | PWR\_PROC\_ServiceStatus | 服务的状态，有 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

***

#### 设置服务的状态

接口定义：int PWR\_PROC\_SetServiceState(const PWR\_PROC\_ServiceState \*sState)

接口描述：

1. 设置服务的状态

参数描述：

| 参数  | 类型                  | 描述       |
| ----- | --------------------- | ---------- |
| state | PWR\_PROC\_ServiceState | 服务的状态，目前只支持停止(0)/开启(1)两种状态 |

返回值：

| 类型 | 描述                                          |
| ---- | --------------------------------------------- |
| int  | 返回码0：成功其他值：失败，参见错误码 |

注意：

1. 目前暂时只支持设置mpctool和eagle两个服务的状态。
2. 设置服务状态前，需要安装mpctool或者eagle两个软件，分别对应`python3-eagle-mpctool`和`eagle`软件包。
3. mpctool目前只支持以下arm服务器机型，x86下无法使用
   1. Taishan200 2280(VD)
   2. Taishan200 Pro 2280
   3. Taishan200 2280v2

***

## 使用用例

将以下代码保存为`powerapi_test.c`。

```sh
#include <stdio.h>
#include <unistd.h>
#include <signal.h>
#include <string.h>
#include <stdlib.h>
#include <pwrapic/powerapi.h>

#define MAIN_LOOP_INTERVAL 5
#define TEST_FREQ 2400
#define TEST_CORE_NUM 128
#define AVG_LEN_PER_CORE 5
#define TEST_CPU_DMA_LATENCY 2000
#define TASK_INTERVAL 1000
#define TASK_RUN_TIME 10
#define TEST_FREQ_RANGE_MIN 500
#define TEST_FREQ_RANGE_MAX 2500

static int g_run = 1;

static void PrintResult(char *function, int ret)
{
    int length = 24;
    printf("[TEST ]    ");
    printf("%-*s", length, function);
    printf(":");
    if (ret == PWR_SUCCESS) {
        printf("SUCCESS ret: %d\n", ret);
    } else {
        printf("ERROR   ret: %d\n", ret);
    }
}

enum {
    DEBUG = 0,
    INFO,
    WARNING,
    ERROR
};

static const char *GetLevelName(int level)
{
    static char debug[] = "DEBUG";
    static char info[] = "INFO";
    static char warning[] = "WARNING";
    static char error[] = "ERROR";
    switch (level) {
        case DEBUG:
            return debug;
        case INFO:
            return info;
        case WARNING:
            return warning;
        case ERROR:
            return error;
        default:
            return info;
    }
}

void LogCallback(int level, const char *fmt, va_list vl)
{
    char logLine[4096] = {0};
    char message[4000] = {0};
    int length = 5;

    if (vsnprintf(message, sizeof(message) - 1, fmt, vl) < 0) {
        return;
    }

    printf("[");
    printf("%-*s", length, GetLevelName(level));
    printf("]    %s\n", message);
}

static void SignalHandler(int none)
{
    g_run = 0;
}

static void SetupSignal(void)
{
    // regist signal handler
    (void)signal(SIGINT, SignalHandler);
    (void)signal(SIGUSR1, SignalHandler);
    (void)signal(SIGUSR2, SignalHandler);
    (void)signal(SIGTERM, SignalHandler);
    (void)signal(SIGKILL, SignalHandler);
}
/************************** COMMON ************************/
static void TEST_PWR_SetLogCallback(void)
{
    int ret = -1;
    ret = PWR_SetLogCallback(LogCallback);
    PrintResult("PWR_SetLogCallback", ret);
}

static void TEST_PWR_SetServerInfo(void)
{
    int ret = -1;
    char str[] = "/etc/sysconfig/pwrapis/pwrserver.sock";
    ret = PWR_SetServerInfo(str);
    PrintResult("PWR_SetServerInfo", ret);
}

static void TEST_PWR_Register(void)
{
    while (PWR_Register() != PWR_SUCCESS) {
        sleep(MAIN_LOOP_INTERVAL);
        PrintResult("PWR_Register", PWR_ERR_COMMON);
        continue;
    }
    PrintResult("PWR_Register", PWR_SUCCESS);
}

static void TEST_PWR_RequestControlAuth(void)
{
    int ret = -1;
    ret = PWR_RequestControlAuth();
    PrintResult("PWR_RequestControlAuth", ret);
}
/************************** COMMON END************************/

/***************************** CPU ***************************/
static void TEST_PWR_CPU_GetInfo(void)
{
    int ret = -1;
    PWR_CPU_Info *info = (PWR_CPU_Info *)malloc(sizeof(PWR_CPU_Info));
    if (!info) {
        return;
    }
    bzero(info, sizeof(PWR_CPU_Info));
    ret = PWR_CPU_GetInfo(info);
    PrintResult("PWR_CPU_GetInfo", ret);
    printf("    arch: %s\n    coreNum: %d\n    maxFreq: %f\n    minFreq: %f\n    "
        "modelName: %s\n    numaNum: %d\n    threadsPerCore: %d\n", info->arch,
        info->coreNum, info->maxFreq, info->minFreq, info->modelName, info->numaNum,
        info->threadsPerCore);
    for (int i = 0; i < info->numaNum; i++) {
        printf("    numa node[%d]  cpuList: %s\n", info->numa[i].nodeNo, info->numa[i].cpuList);
    }
    free(info);
}

static void TEST_PWR_CPU_GetFreq(void)
{
    int ret = -1;
    int num = 0;
    int spec = 0;
    int i = 0;

    /**
     * Test 1: spec = 0, get all policy freq.
     * Set the num to the number of CPU cores
     * (it is possible that one kernel corresponds to one policy)
     */
    num = TEST_CORE_NUM;
    spec = 0;
    PWR_CPU_CurFreq cpuCurFreq1[num];
    bzero(cpuCurFreq1, num * sizeof(PWR_CPU_CurFreq));
    ret = PWR_CPU_GetFreq(cpuCurFreq1, &num, spec);
    PrintResult("1  PWR_CPU_GetFreq", ret);
    for (i = 0; i < num; i++) {
        printf("    policy[%d]: %lf\n", cpuCurFreq1[i].policyId, cpuCurFreq1[i].curFreq);
    }

    /**
     * Test 2: spec = 0 num = 2. get the previous 2 policies freq
     */
    ret = -1;
    // 2: previous 2 policies
    num = 2;
    spec = 0;
    PWR_CPU_CurFreq cpuCurFreq2[num];
    bzero(cpuCurFreq2, num * sizeof(PWR_CPU_CurFreq));
    ret = PWR_CPU_GetFreq(cpuCurFreq2, &num, spec);
    PrintResult("2  PWR_CPU_GetFreq", ret);
    for (i = 0; i < num; i++) {
        printf("    policy[%d]: %lf\n", cpuCurFreq2[i].policyId, cpuCurFreq2[i].curFreq);
    }

    /**
     * Test 3: spec = 1, get the two target policy freq
     */
    ret = -1;
    // 2: previous 2 policies
    num = 2;
    spec = 1;
    PWR_CPU_CurFreq cpuCurFreq3[num];
    bzero(cpuCurFreq3, num * sizeof(PWR_CPU_CurFreq));
    cpuCurFreq3[0].policyId = 0;
    // 32 : the Id of the second policy.
    cpuCurFreq3[1].policyId = 32;
    ret = PWR_CPU_GetFreq(cpuCurFreq3, &num, spec);
    PrintResult("3  PWR_CPU_GetFreq", ret);
    for (i = 0; i < num; i++) {
        printf("    policy[%d]: %lf\n", cpuCurFreq3[i].policyId, cpuCurFreq3[i].curFreq);
    }
}

static void TEST_PWR_CPU_SetFreq(void)
{
    int ret = -1;
    int num = 1;
    PWR_CPU_CurFreq cpuCurFreq[num];
    bzero(cpuCurFreq, num * sizeof(PWR_CPU_CurFreq));
    cpuCurFreq[0].policyId = 0;
    cpuCurFreq[0].curFreq = TEST_FREQ;
    ret = PWR_CPU_SetFreq(cpuCurFreq, num);
    PrintResult("PWR_CPU_SetFreq", ret);

    int spec = 1;
    bzero(cpuCurFreq, num * sizeof(PWR_CPU_CurFreq));
    cpuCurFreq[0].policyId = 0;
    ret = PWR_CPU_GetFreq(cpuCurFreq, &num, spec);
    printf("    current policy[%d]: %lf\n", cpuCurFreq[0].policyId, cpuCurFreq[0].curFreq);
}
/*************************** CPU END *************************/

int main(int argc, const char *args[])
{
    /********** Common **********/
    TEST_PWR_SetServerInfo();
    TEST_PWR_SetLogCallback();
    TEST_PWR_Register();
    TEST_PWR_RequestControlAuth();

    /************ CPU ***********/
    TEST_PWR_CPU_GetInfo();
    TEST_PWR_CPU_GetFreq();
    TEST_PWR_CPU_SetFreq();

    PWR_ReleaseControlAuth();
    PWR_UnRegister();
    return 0;
}

```

使用gcc编译：

```sh
gcc powerapi_test.c -o powerapi_test -lpwrapi
```

运行查看结果：

```sh
./powerapi_test
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/syscare/using_syscare.md
---
# 使用SysCare

本章介绍在openEuler中使用SysCare的方法，包括使用SysCare制作及管理补丁，包括补丁制作、应用、激活、反激活、确认、卸载补丁、补丁状态保存与恢复、补丁冲突检测、补丁覆盖、补丁信息查询。

## 前提条件

安装openEuler 22.03 LTS SP4版本。

## 制作热补丁

使用`syscare build`命令制作热补丁，该命令为纯CLI工具，提供从RPM包生成热补丁包的功能，热补丁包以RPM包的形式封装维护，支持制作内核热补丁及用户态热补丁。

### 命令行参数

```shell
USAGE:
    syscare build [OPTIONS] --patch-name <PATCH_NAME> --source <SOURCE>... --debuginfo <DEBUGINFO>... --patch <PATCH>...

OPTIONS:
    -n, --patch-name <PATCH_NAME>                  Patch name
        --patch-arch <PATCH_ARCH>                  Patch architecture [default: aarch64]
        --patch-version <PATCH_VERSION>            Patch version [default: 1]
        --patch-release <PATCH_RELEASE>            Patch release [default: 1]
        --patch-description <PATCH_DESCRIPTION>    Patch description [default: (none)]
        --patch-requires <PATCH_REQUIRES>...       Patch requirements
    -s, --source <SOURCE>...                       Source package(s)
    -d, --debuginfo <DEBUGINFO>...                 Debuginfo package(s)
    -p, --patch <PATCH>...                         Patch file(s)
        --work-dir <WORK_DIR>                      Working directory [default: /var/run/syscare]
        --build-root <BUILD_ROOT>                  Build temporary directory [default: .]
    -o, --output <OUTPUT>                          Output directory [default: .]
    -j, --jobs <JOBS>                              Parallel build jobs [default: 64]
        --skip-compiler-check                      Skip compiler version check (not recommended)
        --skip-cleanup                             Skip post-build cleanup
    -v, --verbose                                  Provide more detailed info
    -h, --help                                     Print help information
    -V, --version                                  Print version information
```

### 命令行选项

|名称|描述|类型|备注|
| ---- | ---- | ---- | ---- |
|-n, --patch-name `<PATCH_NAME>`|补丁名称|字符串|必选参数，需符合RPM命名规范|
|--patch-arch `<PATCH_ARCH>`|补丁架构|字符串|默认为当前架构，需符合RPM命名规范|
|--patch-version `<PATCH_VERSION>`|补丁版本号|字符串|默认值为1，需符合RPM命名规范|
|--patch-release `<PATCH_RELEASE>`|补丁release|数字|默认值为1，需符合RPM命名规范|
|--patch-description `<PATCH_DESCRIPTION>`|补丁描述|字符串|默认为(none)|
|--patch-requires `<PATCH_REQUIRES>`|补丁额外依赖|字符串|默认为空，需符合RPM规范|
|-s, --source `<SOURCE>`|目标软件src.rpm源码包路径|字符串|必选参数，需为合法路径|
|-d, --debuginfo `<DEBUGINFO>`|目标软件debuginfo包路径|字符串|必选参数，需为合法路径|
|-p, --patch `<PATCH>`|补丁文件|字符串|必选参数，需为合法路径|
|--workdir `<WORKDIR>`|临时文件夹路径|字符串|默认为/var/run/syscare，需为合法路径|
|--build-root `<BUILD_ROOT>`|构建临时文件夹路径|字符串|默认为当前执行目录，需为合法路径|
|-o, --output `<OUTPUT>`|补丁输出文件夹|字符串|默认为当前执行目录，需为合法路径|
|-j, --jobs `<N>`|并行编译线程数|数字|默认为cpu线程数|
|--skip-compiler-check|跳过编译器检查|标识|-|
|--skip-cleanup|跳过临时文件清理|标识|-|
|-v, --verbose|打印详细信息|标识|-|
|-h, --help|打印帮助信息|标识|-|
|-V, --version|打印版本信息|标识|-|

示例如下：

```shell
syscare build \
    --patch-name "HP001" \
    --patch-description "CVE-2021-32675 - When parsing an incoming Redis Standard Protocol (RESP) request, Redis allocates memory according to user-specified values which determine the number of elements (in the multi-bulk header) and size of each element (in the bulk header). An attacker delivering specially crafted requests over multiple connections can cause the server to allocate significant amount of memory. Because the same parsing mechanism is used to handle authentication requests, this vulnerability can also be exploited by unauthenticated users." \
    --source ./redis-6.2.5-1.src.rpm \
    --debuginfo ./redis-debuginfo-6.2.5-1.x86_64.rpm \
    --output ./output \
    --patch ./0001-Prevent-unauthenticated-client-from-easily-consuming.patch
```

### 热补丁制作流程

1. 准备补丁目标软件源码包(source rpm)及软件调试信息包(debuginfo rpm)

   示例：

   ```shell
   yumdownloader kernel --source
   yumdownloader kernel --debuginfo
   ```

2. 确认满足对应软件编译依赖

   示例：

   ```shell
   dnf install make gcc bison flex openssl-devel dwarves python3-devel elfutils-libelf-devel
   ```

3. 执行SysCare build命令

   示例：

   ```shell
   syscare build \
           --patch-name HP001 \
           --source kernel-5.10.0-60.66.0.91.oe2203.src.rpm \
           --debuginfo kernel-debuginfo-5.10.0-60.66.0.91.oe2203.x86_64.rpm \
           --output output \
           --patch 001-kernel-patch-test.patch
   ```

   补丁制作过程将会在由`--workdir`参数所指定的目录中（默认为当前目录）创建以syscare-build开头的临时文件夹，用于存放临时文件及编译日志。

   示例：

   ```shell
   dev@openeuler-dev:[~]$ ls -l syscare-build.111602/
   total 100
   -rw-r--r--. 1 dev dev 92303 Nov 12 00:00 build.log
   drwxr-xr-x. 6 dev dev 4096 Nov 12 00:00 package
   drwxr-xr-x. 4 dev dev 4096 Nov 12 00:00 patch
   ```

   编译日志将会生成在临时文件夹中，名称为`build.log`

   ```shell
   dev@openeuler-dev:[~]$ cat syscare-build.111602/build.log | less
   ...
   ```

   若补丁制作成功，且未指定`--skip-compiler-check`参数，将不会保留该临时文件夹。

4. 检查编译结果

   示例：

   ```shell
   dev@openeuler-dev:[~]$ ls -l
   total 189680
   -rw-r--r--. 1 dev dev 194218767 Nov 12 00:00 kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.src.rpm
   -rw-r--r--. 1 dev dev     10937 Nov 12 00:00 patch-kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.rpm
   ```

   其中

   * `patch-kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.rpm`为热补丁包
   * `kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.src.rpm`为热补丁源码包

5. 补丁包安装

   ```shell
   dnf install patch-xxx.rpm
   ```

   补丁包安装后，相关文件在如下路径：

   ```shell
   /usr/lib/syscare/patches/目标软件包名/补丁名
   ```

6. 补丁包卸载

   ```shell
   dnf remove patch-xxx
   ```

   若补丁处于`ACTIVED`或`ACCEPTED`状态时，补丁将会被自动卸载。

### 输出件

​   热补丁制作工具将会生成两个RPM包：

* 热补丁包：包含SysCare热补丁的二进制及元信息，用于热补丁安装。

* 热补丁源码包：包含目标软件源码及新增补丁，用于新版本热补丁制作。

​   输出RPM包命名规则：

* 热补丁包：patch-目标软件全名-补丁名称-补丁版本-补丁release.架构名.rpm
* 热补丁源码包：目标软件全名-补丁名称-补丁版本-补丁release.架构名.src.rpm

### 错误处理

如果出现错误，请参考编译日志。

错误示例：

```shell
...
Building patch, this may take a while
ERROR: Process '/usr/libexec/syscare/upatch-build' exited unsuccessfully, exit_code=255
```

## 管理热补丁

使用`syscare`命令可以对任意补丁进行管理操作，SysCare将会通过用户输入的补丁描述符搜索补丁，并根据对应命令进行相应操作。
补丁管理搜索规则为：目标包名/补丁名，其中“目标包名/”在补丁名唯一的情况下可以省略，也可使用UUID来进行管理。

### 查询热补丁列表

使用`syscare list`命令可查询所有已安装在本地的热补丁列表。

示例：

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
d81bce18-04bd-499f-91e9-8b9d7b94a76b     glibc-2.34-112.oe2203/HP001-1-1/libc.so.6                    NOT-APPLIED
3e7cb90d-9a4c-4fdf-a389-29d5e863f4b0     kernel-5.10.0-153.12.0.92.oe2203sp2/ACC-1-1/vmlinux          NOT-APPLIED
64fa88bd-def5-4994-85de-a7903c526109     kernel-5.10.0-60.91.0.115.oe2203/HP-50801-1-1/vmlinux        NOT-APPLIED
78268c3b-39a2-4d5c-ae96-206d8c62977a     kernel-5.10.0-60.91.0.115.oe2203/HP-50802-1-1/vmlinux        NOT-APPLIED
c568f31d-acfb-4fdb-8d2c-bde3facab5a2     kernel-5.10.0-60.91.0.115.oe2203/HP001-1-1/vmlinux           NOT-APPLIED
35b5ece6-8b67-407e-93fe-d576a78ab499     nginx-1.21.5-4/HP001-1-1/nginx                               NOT-APPLIED
074734fc-034f-4e40-b943-6a76d766939b     openssl-libs-1.1.1m-22.oe2203/HP001-1-1/libcrypto.so.1.1.1m  NOT-APPLIED
ae124f00-206a-4385-a341-c7b2f7e19482     qemu-7.0.0-2/HP001-1-1/qemu-system-aarch64                   NOT-APPLIED
11316483-dc62-4caf-bd5d-c51801dcb032     qemu-7.0.0-2/HP001-1-1/qemu-system-arm                       NOT-APPLIED
b382ea35-6713-4cfc-bb72-038feefb8173     qemu-7.0.0-2/HP001-1-1/qemu-system-i386                      NOT-APPLIED
6aaec566-a220-4b60-8020-8077b6adc6a6     qemu-7.0.0-2/HP001-1-1/qemu-system-mips                      NOT-APPLIED
2bc0158e-fc42-4ea1-8f5c-e6891d10098b     qemu-7.0.0-2/HP001-1-1/qemu-system-mips64                    NOT-APPLIED
17e00bf0-b389-46d3-a036-933aeb41e0cb     qemu-7.0.0-2/HP001-1-1/qemu-system-mips64el                  NOT-APPLIED
8481a911-d80b-4099-b9a2-a4d3c63de06d     qemu-7.0.0-2/HP001-1-1/qemu-system-mipsel                    NOT-APPLIED
d8305d00-6f45-4c38-b7a4-844b4a667d89     qemu-7.0.0-2/HP001-1-1/qemu-system-ppc                       NOT-APPLIED
d10dc5f8-1692-4da4-8908-d2075c47d62b     qemu-7.0.0-2/HP001-1-1/qemu-system-ppc64                     NOT-APPLIED
77dbfd01-dee4-405b-930f-9711a0ad43c4     qemu-7.0.0-2/HP001-1-1/qemu-system-x86_64                    NOT-APPLIED
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      NOT-APPLIED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            NOT-APPLIED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         NOT-APPLIED
789f0052-b932-4d9d-961d-7003bece1a3a     redis-6.2.5-1/HP002-1-1/redis-benchmark                      NOT-APPLIED
e4aee980-1596-43d9-be9a-07fc6f668970     redis-6.2.5-1/HP002-1-1/redis-cli                            NOT-APPLIED
feb13c9a-02b3-4109-a2f3-c3e9fe41e9ad     redis-6.2.5-1/HP002-1-1/redis-server                         NOT-APPLIED
```

### 查询热补丁元信息

使用`syscare info`命令可以查询一个或多个热补丁的元信息。

补丁元信息中包含以下字段：

| 字段名称 | 字段描述 |
| ----------- | ---------------------- |
| name | 热补丁名称 |
| version | 热补丁版本 |
| release | 热补丁Release |
| arch | 热补丁架构 |
| type | 热补丁类型 |
| target | 目标软件名 |
| license | 目标软件许可证 |
| description | 热补丁描述 |
| entities | 热补丁目标二进制名称 |
| patch| 热补丁文件列表 |

示例：

```shell
root@dev:[~]$ syscare info redis-6.2.5-1/HP001-1-1
-------------------------------------------
Patch: redis-6.2.5-1/HP001-1-1
-------------------------------------------
name:        HP001
version:     1
release:     1
arch:        x86_64
type:        UserPatch
target:      redis-6.2.5-1
license:     BSD and MIT
description: CVE-2021-32675 - When parsing an incoming Redis Standard Protocol (RESP) request, Redis allocates memory according to user-specified values which determine the number of elements (in the multi-bulk header) and size of each element (in the bulk header). An attacker delivering specially crafted requests over multiple connections can cause the server to allocate significant amount of memory. Because the same parsing mechanism is used to handle authentication requests, this vulnerability can also be exploited by unauthenticated users.
entities:
* redis-server
* redis-benchmark
* redis-cli
patches:
* 0001-Prevent-unauthenticated-client-from-easily-consuming.patch
-------------------------------------------
```

### 查询热补丁状态

使用`syscare status`命令可以查询一个或多个热补丁状态。

示例：

```shell
root@dev:[~]$ syscare status status redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-server: NOT-APPLIED
redis-6.2.5-1/HP001-1-1/redis-benchmark: NOT-APPLIED
redis-6.2.5-1/HP001-1-1/redis-cli: NOT-APPLIED
```

### 加载并激活热补丁

使用`syscare apply`命令可以加载并激活一个或多个处于`NOT-APPLIED`状态的热补丁。

示例：

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      NOT-APPLIED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            NOT-APPLIED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         NOT-APPLIED

root@dev:[~]$ syscare apply redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-cli: ACTIVED
redis-6.2.5-1/HP001-1-1/redis-benchmark: ACTIVED
redis-6.2.5-1/HP001-1-1/redis-server: ACTIVED
```

### 激活热补丁

使用`syscare active`命令可以激活一个或多个处于`DEACTIVED`状态的热补丁。

示例：

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      DEACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            DEACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         DEACTIVED

root@dev:[~]$ syscare active redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-cli: ACTIVED
redis-6.2.5-1/HP001-1-1/redis-benchmark: ACTIVED
redis-6.2.5-1/HP001-1-1/redis-server: ACTIVED
```

### 去激活热补丁

使用`syscare deactive`命令可以去激活一个或多个处于`ACTIVED`状态的热补丁。

示例：

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED

root@dev:[~]$ syscare deactive redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-cli: DEACTIVED
redis-6.2.5-1/HP001-1-1/redis-benchmark: DEACTIVED
redis-6.2.5-1/HP001-1-1/redis-server: DEACTIVED
```

### 确认热补丁

使用`syscare accept`命令可以确认一个或多个处于`ACTIVED`状态的热补丁。

经过确认后的所有热补丁将会在系统重启后自动应用激活。

示例：

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED

root@dev:[~]$ syscare accept redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-cli: ACCEPTED
redis-6.2.5-1/HP001-1-1/redis-benchmark: ACCEPTED
redis-6.2.5-1/HP001-1-1/redis-server: ACCEPTED
```

### 卸载/移除热补丁

使用`syscare remove`命令可以卸载任意状态的热补丁。

示例：

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      DEACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         ACCEPTED

root@dev:[~]$ syscare remove redis-6.2.5-1/HP001-1-1
redis-6.2.5-1/HP001-1-1/redis-cli: NOT-APPLIED
redis-6.2.5-1/HP001-1-1/redis-benchmark: NOT-APPLIED
redis-6.2.5-1/HP001-1-1/redis-server: NOT-APPLIED
```

### 多补丁支持

目前SysCare支持对同一个用户态的二进制应用多个热补丁。

示例：

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED
789f0052-b932-4d9d-961d-7003bece1a3a     redis-6.2.5-1/HP002-1-1/redis-benchmark                      ACTIVED
e4aee980-1596-43d9-be9a-07fc6f668970     redis-6.2.5-1/HP002-1-1/redis-cli                            ACTIVED
feb13c9a-02b3-4109-a2f3-c3e9fe41e9ad     redis-6.2.5-1/HP002-1-1/redis-server                         ACTIVED
```

#### 热补丁冲突检测

在应用多个热补丁时，若当前应用的补丁与已打补丁存在函数冲突，则会出现补丁冲突的提示。

示例：

```shell
root@dev:[~]$ syscare apply redis-6.2.5-1/HP002-1-1
Error: Operation failed

Caused by:
    0. Transaction "Apply patch 'redis-6.2.5-1/HP002-1-1'" failed

    Caused by:
        0: Driver: Patch "redis-6.2.5-1/HP002-1-1/redis-cli" check failed
        1: Upatch: Patch is conflicted with "0e776e26-58cd-42ce-85e4-046481acad09"
```

SysCare提供`syscare check`命令，用以判断当前热补丁是否与已打热补丁存在冲突。

示例：

```shell
root@dev:[~]$ syscare check redis-6.2.5-1/HP002-1-1
Error: Operation failed

Caused by:
    0. Driver: Patch "redis-6.2.5-1/HP002-1-1/redis-server" check failed

    Caused by:
        Upatch: Patch is conflicted with "d9432f08-65cf-4849-a9af-ba20e9b6c7dc"
```

#### 热补丁强制覆盖

当出现热补丁冲突错误，可以使用`--force`选项使用当前热补丁强制覆盖原先的热补丁。

示例：

```shell
root@dev:[~]$ syscare list
Uuid                                     Name                                                         Status
777f15fe-cfc8-4b7a-96af-808a4518859f     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
0e776e26-58cd-42ce-85e4-046481acad09     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
d9432f08-65cf-4849-a9af-ba20e9b6c7dc     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED
789f0052-b932-4d9d-961d-7003bece1a3a     redis-6.2.5-1/HP002-1-1/redis-benchmark                      NOT-APPLIED
e4aee980-1596-43d9-be9a-07fc6f668970     redis-6.2.5-1/HP002-1-1/redis-cli                            NOT-APPLIED
feb13c9a-02b3-4109-a2f3-c3e9fe41e9ad     redis-6.2.5-1/HP002-1-1/redis-server                         NOT-APPLIED

root@dev:[~]$ syscare apply redis-6.2.5-1/HP002-1-1
Error: Operation failed

Caused by:
    0. Transaction "Apply patch 'redis-6.2.5-1/HP002-1-1'" failed

    Caused by:
        0: Driver: Patch "redis-6.2.5-1/HP002-1-1/redis-cli" check failed
        1: Upatch: Patch is conflicted with "0e776e26-58cd-42ce-85e4-046481acad09"

root@dev:[~]$ syscare apply redis-6.2.5-1/HP002-1-1 --force
redis-6.2.5-1/HP002-1-1/redis-cli: ACTIVED
redis-6.2.5-1/HP002-1-1/redis-benchmark: ACTIVED
redis-6.2.5-1/HP002-1-1/redis-server: ACTIVED
```

### 热补丁状态保存与恢复

SysCare支持对热补丁当前状态进行保存与恢复。

示例:

```shell
[root@2203sp2-85 syscare]# syscare list
Uuid                                     Name                                                         Status
eebc3155-9a5b-4a09-9561-6a94080de2ce     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
96666521-4606-4aa0-b663-1b455fe586da     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
1e98d692-cc51-4f83-9176-c547ed1db20b     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED
beffae33-1e1a-4bd5-8758-ab6a5f2f1a7c     redis-6.2.5-1/HP002-1-1/redis-benchmark                      NOT-APPLIED
24b01b18-5132-4cae-a379-71d2b0e6d832     redis-6.2.5-1/HP002-1-1/redis-cli                            NOT-APPLIED
a84934de-4a89-4e77-b646-125d1e2c98b4     redis-6.2.5-1/HP002-1-1/redis-server                         ACTIVED
[root@2203sp2-85 syscare]# syscare save
[root@2203sp2-85 syscare]# systemctl restart syscare
[root@2203sp2-85 syscare]# syscare list
Uuid                                     Name                                                         Status
eebc3155-9a5b-4a09-9561-6a94080de2ce     redis-6.2.5-1/HP001-1-1/redis-benchmark                      NOT-APPLIED
96666521-4606-4aa0-b663-1b455fe586da     redis-6.2.5-1/HP001-1-1/redis-cli                            NOT-APPLIED
1e98d692-cc51-4f83-9176-c547ed1db20b     redis-6.2.5-1/HP001-1-1/redis-server                         NOT-APPLIED
beffae33-1e1a-4bd5-8758-ab6a5f2f1a7c     redis-6.2.5-1/HP002-1-1/redis-benchmark                      NOT-APPLIED
24b01b18-5132-4cae-a379-71d2b0e6d832     redis-6.2.5-1/HP002-1-1/redis-cli                            NOT-APPLIED
a84934de-4a89-4e77-b646-125d1e2c98b4     redis-6.2.5-1/HP002-1-1/redis-server                         NOT-APPLIED
[root@2203sp2-85 syscare]# syscare restore
[root@2203sp2-85 syscare]# syscare list
Uuid                                     Name                                                         Status
eebc3155-9a5b-4a09-9561-6a94080de2ce     redis-6.2.5-1/HP001-1-1/redis-benchmark                      ACTIVED
96666521-4606-4aa0-b663-1b455fe586da     redis-6.2.5-1/HP001-1-1/redis-cli                            ACTIVED
1e98d692-cc51-4f83-9176-c547ed1db20b     redis-6.2.5-1/HP001-1-1/redis-server                         ACTIVED
beffae33-1e1a-4bd5-8758-ab6a5f2f1a7c     redis-6.2.5-1/HP002-1-1/redis-benchmark                      NOT-APPLIED
24b01b18-5132-4cae-a379-71d2b0e6d832     redis-6.2.5-1/HP002-1-1/redis-cli                            NOT-APPLIED
a84934de-4a89-4e77-b646-125d1e2c98b4     redis-6.2.5-1/HP002-1-1/redis-server                         ACTIVED
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/application_scenarios.md
---
# 使用指南

本章介绍iSula容器引擎的使用方法。

> !!\[NOTE]说明
> iSulad的所有使用操作均需要使用root权限。

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/container_form/system_container/usage_guide.md
---
# 使用指南

系统容器基于iSula容器引擎进行功能增强，提供系统容器相关功能。系统容器提供的容器管理功能和iSula容器引擎保持一致，其命令格式和功能与iSula容器引擎相同。

本文档仅描述系统容器提供的增强功能对应的使用方式，其他命令行操作请参考 "iSula容器引擎" 章节。

系统容器功能仅涉及isula create/run命令行，后续未特别说明，各功能均使用此命令行。其命令行格式如下所示：

```sh
# isula create/run [OPTIONS] [COMMAND] [ARG...]
```

其中：

* OPTIONS：命令参数，可以一个或者多个，可选参数请参见 "iSula容器引擎 > 附录 > 命令行参数说明"。
* COMMAND：系统容器启动后执行的命令。
* ARG：系统容器启动后执行命令对应的参数。

> \[!NOTE]说明
> 系统容器的使用需要root权限。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/secure_container/application_scenarios_2.md
---
# 使用方法

本章介绍安全容器的使用方法。

> !\[NOTE]说明\
> 安全容器的使用需要root权限。

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/kubeos/kubeos/usage_instructions.md
---
# 使用方法

## 注意事项

* 公共注意事项
  * 仅支持虚拟机和物理机x86和arm64 UEFI场景。
  * 使用kubectl apply通过YAML创建或更新OS的CR时，不建议并发apply，当并发请求过多时，kube-apiserver会无法处理请求导致失败。
  * 如用户配置了容器镜像仓的证书或密钥，请用户保证证书或密钥文件的权限最小。

* 升级注意事项
  * 升级为所有软件包原子升级，默认不提供单包升级能力。
  * 升级为双区升级的方式，不支持更多分区数量。
  * 当前暂不支持跨大版本升级。
  * 单节点的升级过程的日志可在节点的 /var/log/messages 文件查看。
  * 请严格按照提供的升级和回退流程进行操作，异常调用顺序可能会导致系统无法升级或回退。
  * 节点上containerd如需配置ctr使用的私有镜像，请将配置文件host.toml按照ctr指导放在/etc/containerd/certs.d目录下。
  * 使用OCI 镜像升级和mtls双向认证仅支持 openEuler 22.09 及之后的版本。
  * nodeselector、executionmode、timewindow和timeinterval 仅支持openEuler 24.09及之后版本。
  * KubeOS 24.03-LTS-SP1 版本与历史版本不兼容。
  * 使用从http/https服务器下载升级镜像功能需要同步使用对应版本镜像制作工具。

* 配置注意事项
  * 用户自行指定配置内容，用户需保证配置内容安全可靠 ，尤其是持久化配置（kernel.sysctl.persist、grub.cmdline.current、grub.cmdline.next、kubernetes.kubelet、container.containerd、pam.limits），KubeOS不对参数有效性进行检验。
  * opstype=config时，若osversion与当前集群节点的OS版本不一致，配置不会进行。
  * 当前仅支持kernel参数临时配置（kernel.sysctl）、持久化配置（kernel.sysctl.persist）和grub cmdline配置（grub.cmdline.current和grub.cmdline.next）、kubelet配置（kubernetes.kubelet）、containerd配置（container.containerd）和pam limits配置（pam.limits）。
  * 持久化配置会写入persist持久化分区，升级重启后配置保留；kernel参数临时配置重启后不保留。
  * 配置grub.cmdline.current或grub.cmdline.next时，如为单个参数（非key=value格式参数），请指定key为该参数，value为空。
  * 进行配置删除（operation=delete）时，key=value形式的配置需保证key、value和实际配置一致。
  * 配置不支持回退，如需回退，请修改配置版本和配置内容，重新下发配置。
  * 配置出现错误，节点状态陷入config时，请将配置版本恢复成上一版本并重新下发配置，从而使节点恢复至idle状态。 但是请注意：出现错误前已经配置完成的参数无法恢复。
  * 在配置grub.cmdline.current或grub.cmdline.next时，若需要将已存在的“key=value”格式的参数更新为只有key无value格式，比如将“rd.info=0”更新成rd.info，需要先删除“key=value”，然后在下一次配置时，添加key。不支持直接更新或者更新删除动作在同一次完成。

## OS CR参数说明

在集群中创建类别为OS的定制对象，设置相应字段。类别OS来自于[安装和部署章节](./installation_and_deployment.md)创建的CRD对象，字段及说明如下：

* imageurl指定的地址里包含协议，只支持http或https协议。imageurl为https协议时为安全传输，imageurl为http地址时，需指定flagSafe为true，即用户明确该地址为安全时，才会下载镜像。如imageurl为http地址且没有指定flagSafe为true，默认该地址不安全，不会下载镜像并且在升级节点的日志中提示用户该地址不安全。
* 对于imageurl，推荐使用https协议，使用https协议需要升级的机器已安装相应证书。如果镜像服务器由用户自己维护，需要用户自己进行签名，并保证升级节点已安装对应证书。用户需要将证书放在容器OS`/etc/KubeOS/certs`目录下。地址由管理员传入，管理员应该保证网址的安全性，推荐采用内网地址。
* 容器OS镜像的合法性检查需要由容器OS镜像服务提供者做合法性检查，确保下载的容器OS镜像来源可靠。
* 集群存在多OS版本即存在多个OS的实例时，OS的nodeselector字段需要与其他OS不同，即通过label区分的一类node只能对应一个OS实例：
  * 当有OS的nodeselector为all-label时，集群只能存在这一个OS的有效实例（有效实例为存在与这个OS对应的节点）。
  * nodeselector不配置的OS也只能有一个，因为nodeselector不配置时认为是对没有label的节点进行操作。
* timewinterval参数说明：

  * 参数不设置时默认为15s。
  * 参数设置为0时，由于k8s controller-runtime的rate limit限制，operator下发任务的时间间隔会逐渐增加直至1000s。
  * 并行时为每批次operator下发升级/配置的时间间隔。
  * 在串行时为每批次节点串行升级完毕后与下次升级/配置下发的时间间隔，批次内部的时间间隔为15s。
  * OS的实例字段进行更新会立刻触发operator。

  | 参数            |参数类型  | 参数说明                                                     | 使用说明 | 是否必选         |
  | -------------- | ------ | ------------------------------------------------------------ | ----- | ---------------- |
  | imagetype      | string | 升级镜像的类型           | 仅支持docker ，containerd ，或者是 disk，仅在升级场景有效。**注意**：若使用containerd，agent优先使用crictl工具拉取镜像，没有crictl时才会使用ctr命令拉取镜像。使用ctr拉取镜像时，镜像如果在私有仓内，需按照[官方文档](https://github.com/containerd/containerd/blob/main/docs/hosts.md)在/etc/containerd/certs.d目录下配置私有仓主机信息，才能成功拉取镜像。 |是               |
  | opstype        | string | 操作类型：升级,回退或者配置 | 仅支持upgrade ，config 或者 rollback |是               |
  | osversion      | string | 升级/回退的目标版本  | osversion需与节点的目标os版本对应（节点上/etc/os-release中PRETTY\_NAME字段或k8s检查到的节点os版本） 例如：KubeOS 1.0.0。 |是               |
  | maxunavailable | int    | 每批同时进行升级/回退/配置的节点数。 | maxunavailable值大于实际节点数时，取实际节点数进行升级/回退/配置。 |是               |
  | containerimage    | string | 用于升级的容器镜像               | 仅在imagetype是容器类型时生效，仅支持以下3种格式的容器镜像地址： repository/name repository/name@sha256:xxxx repository/name:tag |是               |
  | imageurl       | string | 用于升级的磁盘镜像的地址 | imageurl中包含协议，只支持http或https协议，例如：`https://192.168.122.15/update.img` ，仅在使用磁盘镜像升级场景下有效 |是               |
  | checksum       | string | 用于升级的磁盘镜像校验的checksum(SHA-256)值或者是用于升级的容器镜像的digests值                      | 仅在升级场景下有效 |是               |
  | flagSafe       | bool   | 当imageurl的地址使用http协议表示是否是安全的                 | 需为 true 或者 false ，仅在imageurl使用http协议时有效 |是               |
  | mtls           | bool   | 用于表示与imageurl连接是否采用https双向认证     | 需为 true 或者 false ，仅在imageurl使用https协议时有效|是               |
  | cacert         | string | https或者https双向认证时使用的根证书文件                       | 仅在imageurl使用https协议时有效| imageurl使用https协议时必选 |
  | clientcert     | string | https双向认证时使用的客户端证书文件                          | 仅在使用https双向认证时有效|mtls为true时必选 |
  | clientkey      | string | https双向认证时使用的客户端公钥                              | 仅在使用https双向认证时有效|mtls为true时必选 |
  | evictpodforce      | bool | 升级/回退时是否强制驱逐pod                            | 需为 true 或者 false ，仅在升级或者回退时有效| 必选 |
  | sysconfigs      | / | 配置设置                          | 1. “opstype=config”时只进行配置。  2.“opstype=upgrade/rollback”时，代表升级/回退后配置，即在升级/回退重启后进行配置，详细字段说明请见[配置（Settings）指导](#配置settings指导) | “opstype=config”时必选 |
  | upgradeconfigs | / | 升级前配置设置                       | 在升级或者回退时有效，在升级或者回退操作之前起效，详细字段说明请见[配置（Settings）指导](#配置settings指导)| 可选 |
  | nodeselector      | string | 需要进行升级/配置/回滚操作的节点label                           | 用于只对具有某些特定label的节点而不是集群所有worker节点进行运维的场景，需要进行运维操作的节点需要包含key为upgrade.openeuler.org/node-selector的label，nodeselector为该label的value值。注意事项： 1.此参数不配置时，或者配置为“no-label”时对没有upgrade.openeuler.org/node-selector的节点进行操作 2.此参数为“”时，对具有upgrade.openeuler.org/node-selector=“”的节点进行操作  3.如需忽略label，对所有节点进行操作，需指定此参数为all-label| 可选 |
  | timewindow      | / | 升级/配置/回滚操作的时间窗口                           |1.指定时间窗口时starttime和endtime都需指定，即二者需要同时为空或者同时不为空 2.starttime和endtime类型为string，需要为YYYY-MM-DD HH:MM:SS格式或者HH:MM:SS格式，且二者格式需一致 3.为HH:MM:SS格式时，starttime 小于endtime认为starttime是下一天的该时间 4.timewindow不配置时默认为不存在时间窗限制| 可选 |
  | timeinterval      | int | 升级/配置/回滚操作每批次任务下发的时间间隔                           |参数单位为秒，时间间隔为operator下发任务的时间间隔，如k8s集群繁忙无法立即响应operator请求，实际时间间隔可能会大于指定时间| 可选 |
  | executionmode     | string | 升级/配置/回滚操作执行的方式                           |仅支持serial或者parallel，即串行或者并行，当次参数不设置时，默认采用并行的方式| 可选 |

## 升级指导

1. 编写YAML文件，在集群中部署 OS 的cr实例，用于部署cr实例的YAML示例如下，假定将上面的YAML保存到upgrade\_v1alpha1\_os.yaml;

   * 使用磁盘镜像进行升级

     ```yaml
     apiVersion: upgrade.openeuler.org/v1alpha1
     kind: OS
     metadata:
       name: os-sample
     spec:
       imagetype: disk
       opstype: upgrade
       osversion: edit.os.version
       maxunavailable: edit.node.upgrade.number
       containerimage: ""
       evictpodforce: true/false
       imageurl: edit.image.url
       checksum: image.checksum
       flagSafe: imageurl.safety
       mtls: imageurl use mtls or not
       cacert:  ca certificate 
       clientcert:  client certificate 
       clientkey:  client certificate key 
     ```

   * 使用容器镜像进行升级

     * 使用容器镜像进行升级前请先制作升级所需的容器镜像，制作方式请见[《容器OS镜像制作指导》](./kubeos_image_creation.md)中 [KubeOS OCI 镜像制作](./kubeos_image_creation.md#kubeos-oci-镜像制作)。

     * 节点容器引擎为docker

     ```yaml
     apiVersion: upgrade.openeuler.org/v1alpha1
     kind: OS
     metadata:
       name: os-sample
     spec:
       imagetype: docker
       opstype: upgrade
       osversion: edit.os.version
       maxunavailable: edit.node.upgrade.number
       containerimage: container image like repository/name:tag
       evictpodforce: true/false
       imageurl: ""
       checksum: container image digests
       flagSafe: false
       mtls: true
     ```

     * 节点容器引擎为containerd

     ```yaml
     apiVersion: upgrade.openeuler.org/v1alpha1
     kind: OS
     metadata:
       name: os-sample
     spec:
       imagetype: containerd
       opstype: upgrade
       osversion: edit.os.version
       maxunavailable: edit.node.upgrade.number
       containerimage: container image like repository/name:tag
       evictpodforce: true/false
       imageurl: ""
       checksum: container image digests
       flagSafe: false
       mtls: true
     ```

     * 升级并且进行配置的示例如下：

       * 以节点容器引擎为containerd为例，升级方式对配置无影响，upgradeconfigs在升级前起效，sysconfigs在升级后起效，配置参数说明请见[配置（Settings）指导](#配置settings指导)。
       * 升级并且配置时opstype字段需为upgrade。
       * upgradeconfig为升级之前执行的配置，sysconfigs为升级机器重启后执行的配置，用户可按需进行配置。

         ```yaml
         apiVersion: upgrade.openeuler.org/v1alpha1
         kind: OS
         metadata:
             name: os-sample
         spec:
             imagetype: ""
             opstype: upgrade
             osversion: edit.os.version
             maxunavailable: edit.node.upgrade.number
             containerimage: ""
             evictpodforce: true/false
             imageurl: ""
             checksum: container image digests
             flagSafe: false
             mtls: false
             sysconfigs:
                 version: edit.os.version
                 configs:
                     - model: kernel.sysctl
                     contents:
                         - key: kernel param key1
                           value: kernel param value1
                         - key: kernel param key2
                           value: kernel param value2
                     - model: kernel.sysctl.persist
                       configpath: persist file path
                       contents:
                         - key: kernel param key3
                           value: kernel param value3
                         - key: ""
                           value: ""
             upgradeconfigs:
                 version: 1.0.0
                 configs:
                     - model: kernel.sysctl
                     contents:
                         - key: kernel param key4
                           value: kernel param value4          
         ```

     * 设置nodeselector、timewindow、timeinterval、executionmode升级部分节点示例如下：

       * 以节点容器引擎为containerd为例，升级方式对节点筛选无影响。
       * 需要进行升级的节点需包含key为`upgrade.openeuler.org/node-selector`的label，nodeselector的值为该label的value，即假定nodeselector值为kubeos，则只对包含`upgrade.openeuler.org/node-selector=kubeos`的label的worker节点进行升级。
       * nodeselector、timewindow、timeinterval、executionmode对配置和回滚同样有效。
       * 节点添加label、修改label、删除label和查看label命令示例如下：

       ```shell
       # 为节点kubeos-node1增加label
       kubectl label nodes kubeos-node1 upgrade.openeuler.org/node-selector=kubeos-v1
       # 修改节点kubeos-node1的label
       kubectl label --overwrite nodes kubeos-node1 upgrade.openeuler.org/node-selector=kubeos-v2
       # 删除节点kubeos-node1的label
       kubectl label nodes kubeos-node1 upgrade.openeuler.org/node-selector-
       # 查看节点的label
       kubectl get nodes --show-labels
       ```

       * yaml示例如下：

       ```yaml
       apiVersion: upgrade.openeuler.org/v1alpha1
       kind: OS
       metadata:
         name: os-sample
       spec:
         imagetype: containerd
         opstype: upgrade
         osversion: edit.os.version
         maxunavailable: edit.node.upgrade.number
         containerimage: container image like repository/name:tag
         evictpodforce: true/false
         imageurl: ""
         checksum: container image digests
         flagSafe: false
         mtls: true
         nodeselector: edit.node.label.key
         timewindow:
           starttime: "HH::MM::SS/YYYY-MM-DD HH::MM::SS"
           endtime: "HH::MM::SS/YYYY-MM-DD HH::MM::SS"
         timeinterval: time intervel like 30
         executionmode: serial/parallel
       ```

2.查看未升级的节点的 OS 版本。

```shell
kubectl get nodes -o custom-columns='NAME:.metadata.name,OS:.status.nodeInfo.osImage'
```

3.执行命令，在集群中部署cr实例后，节点会根据配置的参数信息进行升级。

```shell
kubectl apply -f upgrade_v1alpha1_os.yaml
```

4.再次查看节点的 OS 版本来确认节点是否升级完成。

```shell
kubectl get nodes -o custom-columns='NAME:.metadata.name,OS:.status.nodeInfo.osImage'
```

> \[!NOTE]说明
>
> 如果后续需要再次升级，与上面相同对 upgrade\_v1alpha1\_os.yaml 的 相应字段进行相应修改。

## 配置（Settings）指导

* Settings参数说明:

  基于示例YAML对配置的参数进行说明，示例YAML如下，配置的格式（缩进）需和示例保持一致：

  ```yaml
  apiVersion: upgrade.openeuler.org/v1alpha1
  kind: OS
  metadata:
    name: os-sample
  spec:
    imagetype: ""
    opstype: config
    osversion: edit.os.version
    maxunavailable: edit.node.config.number
    containerimage: ""
    evictpodforce: false
    checksum: ""
    sysconfigs:
        version: edit.sysconfigs.version
        configs:
            - model: kernel.sysctl
              contents: 
                - key: kernel param key1
                  value: kernel param value1
                - key: kernel param key2
                  value: kernel param value2
                  operation: delete
            - model: kernel.sysctl.persist
              configpath: persist file path
              contents:
                - key: kernel param key3
                  value: kernel param value3
            - model: grub.cmdline.current
              contents:
                - key: boot param key1
                - key: boot param key2
                  value: boot param value2
                - key: boot param key3
                  value: boot param value3
                  operation: delete
            - model: grub.cmdline.next
              contents:
                - key: boot param key4
                - key: boot param key5
                  value: boot param value5
                - key: boot param key6
                  value: boot param value6
                  operation: delete         
  ```

  配置的参数说明如下：

  | 参数       | 参数类型 | 参数说明                    | 使用说明                                                     | 配置中是否必选          |
  | ---------- | -------- | --------------------------- | ------------------------------------------------------------ | ----------------------- |
  | version    | string   | 配置的版本                  | 通过version是否相等来判断配置是否触发，version为空（为""或者没有值）时同样进行判断，所以不配置sysconfigs/upgradeconfigs时，继存的version值会被清空并触发配置。 | 是                      |
  | configs    | /        | 具体配置内容                | 包含具体配置项列表。                                         | 是                      |
  | model      | string   | 配置的类型                  | 支持的配置类型请看附录下的[Settings列表](#setting-列表)                 | 是                      |
  | configpath | string   | 配置文件路径                | 仅在kernel.sysctl.persist配置类型中生效，请看附录下的[Settings列表](#setting-列表)对配置文件路径的说明。 | 否                      |
  | contents   | /        | 具体key/value的值及操作类型 | 包含具体配置参数列表。                                       | 是                      |
  | key        | string   | 参数名称                    | key不能为空，不能包含"="，不建议配置含空格、tab键的字符串，具体请看附录下的[Settings列表](#setting-列表)中每种配置类型对key的说明。 | 是                      |
  | value      | string   | 参数值                      | key=value形式的参数中，value不能为空，不建议配置含空格、tab键的字符串，具体请看附录下的[Settings列表](#setting-列表)中对每种配置类型对value的说明。 | key=value形式的参数必选 |
  | operation  | string   | 对参数进行的操作            | 仅对kernel.sysctl.persist、grub.cmdline.current、grub.cmdline.next类型的参数生效。默认为添加或更新。仅支持配置为delete，代表删除已存在的参数（key=value需完全一致才能删除）。 | 否                      |

  * upgradeconfigs与sysconfigs参数相同，upgradeconfigs为升级/回退前进行的配置，仅在upgrade/rollback场景起效，sysconfigs既支持只进行配置，也支持在升级/回退重启后进行配置。

* 使用说明

  * 编写YAML文件，在集群中部署 OS 的cr实例，用于部署cr实例的YAML示例如上，假定将上面的YAML保存到upgrade\_v1alpha1\_os.yaml。

  * 查看配置之前的节点的配置的版本和节点状态（NODESTATUS状态为idle）。

  ```shell
  kubectl get osinstances -o custom-columns='NAME:.metadata.name,NODESTATUS:.spec.nodestatus,SYSCONFIG:status.sysconfigs.version,UPGRADECONFIG:status.upgradeconfigs.version'
  ```

  * 执行命令，在集群中部署cr实例后，节点会根据配置的参数信息进行配置，再次查看节点状态(NODESTATUS变成config)。

  ```shell
  kubectl apply -f upgrade_v1alpha1_os.yaml
  kubectl get osinstances -o custom-columns='NAME:.metadata.name,NODESTATUS:.spec.nodestatus,SYSCONFIG:status.sysconfigs.version,UPGRADECONFIG:status.upgradeconfigs.version'
  ```

  * 再次查看节点的配置的版本确认节点是否配置完成(NODESTATUS恢复为idle)。

  ```shell
  kubectl get osinstances -o custom-columns='NAME:.metadata.name,NODESTATUS:.spec.nodestatus,SYSCONFIG:status.sysconfigs.version,UPGRADECONFIG:status.upgradeconfigs.version'
  ```

* 如果后续需要再次配置，与上面相同对 upgrade\_v1alpha1\_os.yaml 的相应字段进行相应修改。

## 回退指导

### 使用场景

* 虚拟机无法正常启动时，可在grub启动项页面手动切换启动项，使系统回退至上一版本（即手动回退）。
* 虚拟机能够正常启动并且进入系统时，支持工具回退和手动回退，建议使用工具回退。
* 工具回退有两种方式：

  1. rollback模式直接回退至上一版本。
  2. upgrade模式重新升级至上一版本。

### 手动回退指导

* 手动重启虚拟机，进入启动项页面后，选择第二启动项进行回退，手动回退仅支持回退到上一个版本。

### 工具回退指导

* 回退至任意版本

  1. 修改 OS 的cr实例的YAML 配置文件（例如 upgrade\_v1alpha1\_os.yaml），设置相应字段为期望回退的老版本镜像信息。类别OS来自于安装和部署章节创建的CRD对象，字段说明及示例请见上一节升级指导。

  2. YAML修改完成后执行更新命令，在集群中更新定制对象后，节点会根据配置的字段信息进行回退

     ```shell
     kubectl apply -f upgrade_v1alpha1_os.yaml
     ```

* 回退至上一版本

  * OS回退至上一版本：修改upgrade\_v1alpha1\_os.yaml，设置osversion为上一版本，opstype为rollback，回退至上一版本（即切换至上一分区）。YAML示例如下：

    ```yaml
    apiVersion: upgrade.openeuler.org/v1alpha1
    kind: OS
    metadata:
    name: os-sample
    spec:
        imagetype: ""
        opstype: rollback
        osversion: KubeOS previous version
        maxunavailable: 2
        containerimage: ""
        evictpodforce: true/false
        imageurl: ""
        checksum: ""
        flagSafe: false
        mtls: true
    ```

  * 配置回退至上一版本：修改upgrade\_v1alpha1\_os.yaml，设置sysconfigs/upgradeconfigs的version为上一版本，回退至上一版本（已配置的参数无法回退）。YAML示例如下：

  ```yaml
  apiVersion: upgrade.openeuler.org/v1alpha1
  kind: OS
  metadata:
    name: os-sample
  spec:
    imagetype: ""
    opstype: config
    osversion: edit.os.version
    maxunavailable: edit.node.config.number
    containerimage: ""
    evictpodforce: true/false
    imageurl: ""
    checksum: ""
    flagSafe: false
    mtls: false
    sysconfigs:
        version: previous config version
        configs:
            - model: kernel.sysctl
              contents:
                - key: kernel param key1
                  value: kernel param value1
                - key: kernel param key2
                  value: kernel param value2
            - model: kernel.sysctl.persist
              configpath: persist file path
              contents:
                - key: kernel param key3
                  value: kernel param value3         
  ```

* YAML修改完成后执行更新命令，在集群中更新定制对象后，节点会根据配置的字段信息进行回退。

  ```shell
  kubectl apply -f upgrade_v1alpha1_os.yaml
  ```

  更新完成后，节点会根据配置信息回退容器 OS。

* 查看节点容器 OS 版本(回退OS版本)或节点config版本&节点状态为idle(回退config版本)，确认回退是否成功。

  ```shell
  kubectl get osinstances -o custom-columns='NAME:.metadata.name,NODESTATUS:.spec.nodestatus,SYSCONFIG:status.sysconfigs.version,UPGRADECONFIG:status.upgradeconfigs.version'
  ```

## 附录

### Setting 列表

#### kernel Settings

* kernel.sysctl：临时设置内核参数，重启后无效，key/value 表示内核参数的 key/value， key与value均不能为空且key不能包含“=”，该参数不支持删除操作（operation=delete）示例如下:

  ```yaml
  configs:
    - model: kernel.sysctl
      contents:
          - key: user.max_user_namespaces
            value: 16384
          - key: net.ipv4.tcp_tw_recycle
            value: 0
            operation: delete
  ```

* kernel.sysctl.persist: 设置持久化内核参数，key/value表示内核参数的key/value，key与value均不能为空且key不能包含“=”， configpath为配置文件路径，支持新建（需保证父目录存在），如不指定configpath默认修改/etc/sysctl.conf，示例如下：

  ```yaml
  configs:
    - model: kernel.sysctl.persist
      configpath : /etc/persist.conf
      contents:
          - key: user.max_user_namespaces
            value: 16384
          - key: net.ipv4.tcp_tw_recycle
            value: 0
            operation: delete
  ```

#### Grub Settings

* grub.cmdline.current/next: 设置grub.cfg文件中的内核引导参数，该行参数在grub.cfg文件中类似如下示例：

  ```shell
  linux   /boot/vmlinuz root=/dev/sda2 ro rootfstype=ext4 nomodeset quiet oops=panic softlockup_panic=1 nmi_watchdog=1 rd.shell=0 selinux=0 crashkernel=256M panic=3
  ```

  * 在dm-verity模式下，grub.cmdline配置下发无效。

  * KubeOS使用双分区，grub.cmdline.current/next支持对当前分区或下一分区进行配置：

    * grub.cmdline.current：对当前分区的启动项参数进行配置。
    * grub.cmdline.next：对下一分区的启动项参数进行配置。

  * 注意：升级/回退前后的配置，始终基于升级/回退操作下发时的分区位置进行current/next的区分。假设当前分区为A分区，下发升级操作并在sysconfigs（升级重启后配置）中配置grub.cmdline.current，重启后进行配置时仍修改A分区对应的grub cmdline。

  * grub.cmdline.current/next支持“key=value”（value不能为空），也支持单key。若value中有“=”，例如“root=UUID=some-uuid”，key应设置为第一个“=”前的所有字符，value为第一个“=”后的所有字符。 配置方法示例如下：

  ```yaml
  configs:
  - model: grub.cmdline.current
    contents:
        - key: selinux
          value: "0"
        - key: root
          value: UUID=e4f1b0a0-590e-4c5f-9d8a-3a2c7b8e2d94
        - key: panic
          value: "3"
          operation: delete
        - key: crash_kexec_post_notifiers
  - model: grub.cmdline.next
    contents:
        - key: selinux
          value: "0"
        - key: root
          value: UUID=e4f1b0a0-590e-4c5f-9d8a-3a2c7b8e2d94
        - key: panic
          value: "3"
          operation: delete
        - key: crash_kexec_post_notifiers
  ```

#### kubelet配置

* kubernetes.kubelet: 配置节点kubelet的配置文件中的参数，参数说明和约束如下：

  * 仅支持`KubeletConfiguration`中的配置参数。
  * 节点kubelet配置文件需要为yaml格式的文件。
  * 如不指定configpath，默认配置文件路径为`/var/lib/kubelet/config.yaml`，并且需要注意的是配置文件的路径需要与kubelet启动时的`-- config`参数指定的路径一致才能生效，用户需保证配置文件路径有效。
  * kubelet配置的value参数类型支持为空/null、int、float、string、boolean和数组。当为数组时，数组元素允许重复，数组参数进行更新时会追加到已有数组中。如需修改数组中的元素，需要先删除数组，再新增数组来完成修改。
  * 如配置存在嵌套，则通过`'.'`连接嵌套的key值，例如如果修改如下yaml示例中`cacheAuthorizedTTL`参数为1s。

  ```yaml
  authorization:
    mode: Webhook
    webhook:
      cacheAuthorizedTTL: 0s
  ```

  参数配置示例如下：

  ```yaml
  configs:
  - model: kubernetes.kubelet
    configpath: /etc/test.yaml
    contents:
      - key: authorization.webhook.cacheAuthorizedTTL
        value: 1s
  ```

  * kubernetes.kubelet进行删除时，不对value与配置文件中的值进行比较。

#### containerd配置

* container.containerd: 配置节点上containerd的配置文件中的参数，参数说明和约束如下：

  * containerd需要配置文件为toml格式，所以key为toml中该参数的表头.键名，例如希望修改如下toml示例中`no_shim`为true。

  ```toml
  [plugins."io.containerd.runtime.v1.linux"]
  no_shim=false
  runtime="runc"
  runtime_root="
  ```

  参数配置示例如下：

  ```yaml
  configs:
  - model: container.containerd
    configpath: /etc/test.toml
    contents:
      - key: plugins."io.containerd.runtime.v1.linux".no_shim
        value: true
  ```

  * toml使用`.`分割键，os-agent识别时与toml保持一致，所以当键名中包含`.`时，该键名需要使用`""`，例如上例中的`"io.containerd.runtime.v1.linux"`为一个键
  * 如不指定configpath，默认配置文件路径为`/etc/containerd/config.toml`，用户需要保证配置文件路径有效。
  * container.conatainerd配置的key和value均不能为空，value参数类型支持int、float、string、boolean和数组。当为数组时，数组元素允许重复，数组参数进行更新时会追加到已有数组中。如需修改数组中的元素，需要先删除数组，再新增数组来完成修改。
  * container.containerd进行删除时，不对value与配置文件中的值进行比较。

#### Pam Limits配置

* pam.limits：配置节点上/etc/security/limits.conf文件

  * key为domain值，value的格式需要为type.item.value（limits.conf文件要求每行格式为：\<domain> \<type> \<item> \<value>），例如：

  ```yaml
  configs:
  - model: pam.limits
    contents:
      - key: ftp
        value: soft.core.0  
  ```

  * 更新时，如不需要对type/item/value更新时，可以使用`_`，忽略对此参数的更新，但value必须为点隔的三段式，例如：

  ```yaml
  configs:
  - model: pam.limits
    contents:
      - key: ftp
        value: hard._.1  
  ```

  * pam.limits新增时，value中不允许包含`_`
  * pam.limits删除时，会对value进行校验，当value与配置文件中的值不同时，删除失败
  * pam.limits配置的key和value均不能为空

---

---
url: /zh/docs/22.03_LTS_SP4/edge_computing/ros/usage_guide.md
---
# 使用方法

## ROS使用

ROS 提供了一些实用的命令行工具，可以用于获取不同节点的各类信息，常用的命令如下:

* rosnode : 操作节点
* rostopic : 操作话题
* rosservice : 操作服务
* rosmsg : 操作msg消息
* rossrv : 操作srv消息
* rosparam : 操作参数

另请参考:

<http://wiki.ros.org/ROS/CommandLineTools>

## ROS2使用

```shell
# 命令help
[root@openEuler ~]# ros2 --help
usage: ros2 [-h] Call `ros2 <command> -h` for more detailed usage. ...

ros2 is an extensible command-line tool for ROS 2.

optional arguments:
  -h, --help            show this help message and exit

Commands:
  action     Various action related sub-commands
  bag        Various rosbag related sub-commands
  component  Various component related sub-commands
  daemon     Various daemon related sub-commands
  doctor     Check ROS setup and other potential issues
  interface  Show information about ROS interfaces
  launch     Run a launch file
  lifecycle  Various lifecycle related sub-commands
  multicast  Various multicast related sub-commands
  node       Various node related sub-commands
  param      Various param related sub-commands
  pkg        Various package related sub-commands
  run        Run a package specific executable
  security   Various security related sub-commands
  service    Various service related sub-commands
  test       Run a ROS2 launch test
  topic      Various topic related sub-commands
  trace      Trace ROS nodes to get information on their execution
  wtf        Use `wtf` as alias to `doctor`

  Call `ros2 <command> -h` for more detailed usage.
```

## 注意事项

* 新打开终端需要执行“source /opt/ros/foxy/local\_setup.bash”或“source /opt/ros/noetic/setup.bash”命令。

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/kernel_live_upgrade/usage_guide.md
---
# 使用方法

## 命令用法

* nvwa help

  打印帮助信息，打印的信息如下:

  ```shell
  NAME:
  nvwa - a tool used for openEuler kernel update.

  USAGE:
  nvwa [global options] command [command options] [arguments...]

  VERSION:
  0.1

  COMMANDS:
  update   specify kernel version for nvwa to update
  init     init nvwa running environment
  help, h  Shows a list of commands or help for one command

  GLOBAL OPTIONS:
  --help, -h     show help (default: false)
  --version, -v  print the version (default: false)
  ```

* nvwa update \<kernel version>

  热升级到内核某一版本，nvwa会去/boot目录下寻找内核镜像和ramfs，kernel的命名格式需为vmlinuz-\<kernel version>, rootfs命名格式需为initramfs-\<kernel version>.img

  需要注意的是，升级过程有可能会失败，如果失败，部分被dump的进程或者服务，将停止运行。

* nvwa init

  清除nvwa产生的现场信息以及对systemd的配置修改，用于nvwa执行前或者执行失败后，对现场进行清理。

## 使用限制

1. 对于需要通过nvwa保存的service，其配置中需要设置标准输出(StandardOutput)和错误输出(StandardError)，以redis为例:

   ```shell
   [Unit]
   Description=Redis persistent key-value database
   After=network.target
   [Service]
   ExecStart=/usr/bin/redis-server /etc/redis.conf --supervised systemd
   Type=notify
   User=redis
   Group=redis
   RuntimeDirectory=redis
   RuntimeDirectoryMode=0755
   StandardOutput=file:/root/log1.log
   StandardError=file:/root/log2.log
   [Install]
   WantedBy=multi-user.target
   ```

2. 使用加速特性需要修改cmdline以及分配合适的内存，参见[此处](#加速特性说明及使用)

3. 运行过程中需要关闭SELINUX

   理论上，仅需要在执行nvwa update之后和系统重启nvwa恢复现场这段时间前关闭。稳妥起见，建议全程关闭SELINUX。

## 加速特性说明及使用

1. cpu park(加速内核重启过程)

   cpu park，是在使用kexec过程，使cpu进入一种忙等的状态，更快的响应主核发送的中断请求，减少状态的变化。

   使用cpu park，需要在cmdline中加入"cpuparkmem=0x200000000"，其中0x200000000是一段未被其他程序使用的内存起始地址，cpuparkmem将占用从该地址开始，size为1M左右的内存空间。

   需要注意的是，在内存允许的情况下，此处的地址选择，建议范围在4G(0x100000000)之后，前4G通常被系统各组件预留，容易冲突。

2. quick kexec(加速内核启动过程)

   quick kexec，是对kexec加载镜像过程中的一种加速。

   使用quick kexec，需要在配置文件中使能相关选项，更多信息参考<<安装与部署>>(配置介绍)。

3. pin\_memory(加速现场保存恢复过程)

   pin memory，是对criu进行现场保存恢复过程中的一种加速。

   使用pin memory，需要在配置文件中使能相关选项，更多信息参考<<安装与部署>>(配置介绍)。

## 产生的日志信息

内核热升级工具产生的日志分为两部分:

* 运行过程产生的日志

  通过service nvwa status查看。

* 保留现场过程中产生的日志

  日志位于criu\_dir指定的路径对应命名的进程/服务文件夹中。

---

---
url: /zh/docs/22.03_LTS_SP4/server/performance/atune/usage_instructions.md
---
# 使用方法

用户可以通过命令行客户端atune-adm使用A-Tune提供的功能。本章介绍A-Tune客户端包含的功能和使用方法。

## 总体说明

* 使用A-Tune需要使用root权限。

* atune-adm支持的命令可以通过  **atune-adm help/--help/-h**  查询。

* 使用方法中所有命令的使用举例都是在单机部署模式下，如果是在分布式部署模式下，需要指定服务器IP和端口号，例如：

  ```sh
  #  atune-adm -a 192.168.3.196 -p 60001 list
  ```

* define、update、undefine、collection、train、upgrade不支持远程执行。

* 命令格式中，\[ ] 表示参数可选，<> 表示参数必选，具体参数由实际情况确定。

## 查询负载类型

### list

### 功能描述

查询系统当前支持的profile，以及当前处于active状态的profile。

### 命令格式

**atune-adm list**

### 使用示例

```sh
# atune-adm list 

Support profiles:
+------------------------------------------------+-----------+
| ProfileName                                    | Active    |
+================================================+===========+
| arm-native-android-container-robox             | false     |
+------------------------------------------------+-----------+
| basic-test-suite-euleros-baseline-fio          | false     |
+------------------------------------------------+-----------+
| basic-test-suite-euleros-baseline-lmbench      | false     |
+------------------------------------------------+-----------+
| basic-test-suite-euleros-baseline-netperf      | false     |
+------------------------------------------------+-----------+
| basic-test-suite-euleros-baseline-stream       | false     |
+------------------------------------------------+-----------+
| basic-test-suite-euleros-baseline-unixbench    | false     |
+------------------------------------------------+-----------+
| basic-test-suite-speccpu-speccpu2006           | false     |
+------------------------------------------------+-----------+
| basic-test-suite-specjbb-specjbb2015           | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-hdfs-dfsio-hdd                 | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-hdfs-dfsio-ssd                 | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-bayesian                 | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-kmeans                   | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql1                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql10                    | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql2                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql3                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql4                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql5                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql6                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql7                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql8                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-sql9                     | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-tersort                  | false     |
+------------------------------------------------+-----------+
| big-data-hadoop-spark-wordcount                | false     |
+------------------------------------------------+-----------+
| cloud-compute-kvm-host                         | false     |
+------------------------------------------------+-----------+
| database-mariadb-2p-tpcc-c3                    | false     |
+------------------------------------------------+-----------+
| database-mariadb-4p-tpcc-c3                    | false     |
+------------------------------------------------+-----------+
| database-mongodb-2p-sysbench                   | false     |
+------------------------------------------------+-----------+
| database-mysql-2p-sysbench-hdd                 | false     |
+------------------------------------------------+-----------+
| database-mysql-2p-sysbench-ssd                 | false     |
+------------------------------------------------+-----------+
| database-postgresql-2p-sysbench-hdd            | false     |
+------------------------------------------------+-----------+
| database-postgresql-2p-sysbench-ssd            | false     |
+------------------------------------------------+-----------+
| default-default                                | false     |
+------------------------------------------------+-----------+
| docker-mariadb-2p-tpcc-c3                      | false     |
+------------------------------------------------+-----------+
| docker-mariadb-4p-tpcc-c3                      | false     |
+------------------------------------------------+-----------+
| hpc-gatk4-human-genome                         | false     |
+------------------------------------------------+-----------+
| in-memory-database-redis-redis-benchmark       | false     |
+------------------------------------------------+-----------+
| middleware-dubbo-dubbo-benchmark               | false     |
+------------------------------------------------+-----------+
| storage-ceph-vdbench-hdd                       | false     |
+------------------------------------------------+-----------+
| storage-ceph-vdbench-ssd                       | false     |
+------------------------------------------------+-----------+
| virtualization-consumer-cloud-olc              | false     |
+------------------------------------------------+-----------+
| virtualization-mariadb-2p-tpcc-c3              | false     |
+------------------------------------------------+-----------+
| virtualization-mariadb-4p-tpcc-c3              | false     |
+------------------------------------------------+-----------+
| web-apache-traffic-server-spirent-pingpo       | false     |
+------------------------------------------------+-----------+
| web-nginx-http-long-connection                 | true      |
+------------------------------------------------+-----------+
| web-nginx-https-short-connection               | false     |
+------------------------------------------------+-----------+

```

> \[!NOTE]说明\
> Active为true表示当前激活的profile，示例表示当前激活的profile是web-nginx-http-long-connection。

## 分析负载类型并自优化

### analysis

### 功能描述

采集系统的实时统计数据进行负载类型识别，并进行自动优化。

### 命令格式

**atune-adm analysis** \[command options] \[APP\_NAME]

### 参数说明

* OPTIONS

### 使用示例

* 使用默认的模型进行应用识别

  ```sh
  # atune-adm analysis --characterization
  ```

* 使用默认的模型进行应用识别，并进行自动优化

  ```sh
  # atune-adm analysis
  ```

* 使用自训练的模型进行应用识别

  ```sh
  # atune-adm analysis --model /usr/libexec/atuned/analysis/models/new-model.m
  ```

## 自定义模型

A-Tune支持用户定义并学习新模型。定义新模型的操作流程如下：

1. 用define命令定义一个新应用的profile
2. 用collection命令收集应用对应的系统数据
3. 用train命令训练得到模型

### define

### 功能描述

添加用户自定义的应用场景，及对应的profile优化项。

### 命令格式

**atune-adm define**  \<service\_type> \<application\_name> \<scenario\_name> \<profile\_path>

### 使用示例

新增一个profile，service\_type的名称为test\_service，application\_name的名称为test\_app，scenario\_name的名称为test\_scenario，优化项的配置文件为example.conf。

```sh
# atune-adm define test_service test_app test_scenario ./example.conf
```

example.conf 可以参考如下方式书写（以下各优化项非必填，仅供参考），也可通过**atune-adm info**查看已有的profile是如何书写的。

```sh
 [main]
 # list its parent profile
 [kernel_config]
 # to change the kernel config
 [bios]
 # to change the bios config
 [bootloader.grub2]
 # to change the grub2 config
 [sysfs]
 # to change the /sys/* config
 [systemctl]
 # to change the system service status
 [sysctl]
 # to change the /proc/sys/* config
 [script]
 # the script extension of cpi
 [ulimit]
 # to change the resources limit of user
 [schedule_policy]
 # to change the schedule policy
 [check]
 # check the environment
 [tip]
 # the recommended optimization, which should be performed manunaly
```

### collection

### 功能描述

采集业务运行时系统的全局资源使用情况以及OS的各项状态信息，并将收集的结果保存到csv格式的输出文件中，作为模型训练的输入数据集。

> \[!NOTE]说明
>
> * 本命令依赖采样工具perf，mpstat，vmstat，iostat，sar。
> * CPU型号目前仅支持鲲鹏920，可通过dmidecode -t processor检查CPU型号。

### 命令格式

**atune-adm collection**  \<OPTIONS>

### 参数说明

* OPTIONS

### 使用示例

```shell
# atune-adm collection --filename name --interval 5 --duration 1200 --output_path /home/data --disk sda --network eth0 --app_type test_type 
```

### train

### 功能描述

使用采集的数据进行模型的训练。训练时至少采集两种应用类型的数据，否则训练会出错。

### 命令格式

**atune-adm train**  \<OPTIONS>

### 参数说明

* OPTIONS

### 使用示例

使用data目录下的csv文件作为训练输入，生成的新模型new-model.m存放在model目录下。

```shell
# atune-adm train --data_path /home/data --output_file /usr/libexec/atuned/analysis/models/new-model.m 
```

### undefine

### 功能描述

删除用户自定义的profile。

### 命令格式

**atune-adm undefine**  \<profile>

### 使用示例

删除自定义的profile。

```shell
# atune-adm undefine test_service-test_app-test_scenario
```

## 查询profile

### info

### 功能描述

查看对应的profile内容。

### 命令格式

**atune-adm info**  \<profile>

### 使用示例

查看web-nginx-http-long-connection的profile内容：

```shell
# atune-adm info web-nginx-http-long-connection

*** web-nginx-http-long-connection:

#
# nginx http long connection A-Tune configuration
#
[main]
include = default-default

[kernel_config]
#TODO CONFIG

[bios]
#TODO CONFIG

[bootloader.grub2]
iommu.passthrough = 1

[sysfs]
#TODO CONFIG

[systemctl]
sysmonitor = stop
irqbalance = stop

[sysctl]
fs.file-max = 6553600
fs.suid_dumpable = 1
fs.aio-max-nr = 1048576
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_local_port_range = 1024     65500
net.ipv4.tcp_max_tw_buckets = 5000
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_mem =  362619      483495   725238
net.ipv4.tcp_rmem = 4096         87380   6291456
net.ipv4.tcp_wmem = 4096         16384   4194304
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

[script]
prefetch = off
ethtool =  -X {network} hfunc toeplitz

[ulimit]
{user}.hard.nofile = 102400
{user}.soft.nofile = 102400

[schedule_policy]
#TODO CONFIG

[check]
#TODO CONFIG

[tip]
SELinux provides extra control and security features to linux kernel. Disabling SELinux will improve the performance but may cause security risks. = kernel
disable the nginx log = application
```

## 更新profile

用户根据需要更新已有profile。

### update

### 功能描述

将已有profile中原来的优化项更新为new.conf中的内容。

### 命令格式

**atune-adm update**  \<profile> \<profile\_path>

### 使用示例

更新名为test\_service-test\_app-test\_scenario的profile优化项为new.conf。

```shell
# atune-adm update test_service-test_app-test_scenario ./new.conf
```

## 激活profile

### profile

### 功能描述

手动激活profile，使其处于active状态。

### 命令格式

**atune-adm profile** \<profile>

### 参数说明

profile名参考list命令查询结果。

### 使用示例

激活web-nginx-http-long-connection对应的profile配置。

```sh
# atune-adm profile web-nginx-http-long-connection
```

## 回滚profile

### rollback

### 功能描述

回退当前的配置到系统的初始配置。

### 命令格式

**atune-adm rollback**

### 使用示例

```sh
# atune-adm rollback
```

## 更新数据库

### upgrade

### 功能描述

更新系统的数据库。

### 命令格式

**atune-adm upgrade**  \<DB\_FILE>

### 参数说明

* DB\_FILE

  新的数据库文件路径

### 使用示例

数据库更新为new\_sqlite.db。

```sh
# atune-adm upgrade ./new_sqlite.db
```

## 系统信息查询

### check

### 功能描述

检查系统当前的cpu、bios、os、网卡等信息。

### 命令格式

**atune-adm check**

### 使用示例

```sh
# atune-adm check
 cpu information:
     cpu:0   version: Kunpeng 920-6426  speed: 2600000000 HZ   cores: 64
     cpu:1   version: Kunpeng 920-6426  speed: 2600000000 HZ   cores: 64
 system information:
     DMIBIOSVersion: 0.59
     OSRelease: 4.19.36-vhulk1906.3.0.h356.eulerosv2r8.aarch64
 network information:
     name: eth0              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth1              product: HNS GE/10GE/25GE Network Controller
     name: eth2              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth3              product: HNS GE/10GE/25GE Network Controller
     name: eth4              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth5              product: HNS GE/10GE/25GE Network Controller
     name: eth6              product: HNS GE/10GE/25GE RDMA Network Controller
     name: eth7              product: HNS GE/10GE/25GE Network Controller
     name: docker0           product:
```

## 参数自调优

A-Tune提供了最佳配置的自动搜索能力，免去人工反复做参数调整、性能评价的调优过程，极大地提升最优配置的搜寻效率。

### tuning

### 功能描述

使用指定的项目文件对参数进行动态空间的搜索，找到当前环境配置下的最优解。

### 命令格式

> \[!NOTE]说明\
> 在运行命令前，需要满足如下条件：
>
> * 服务端的yaml配置文件已经编辑完成并放置于 atuned服务下 ****/etc/atuned/tuning/**** 的目录中。
> * 客户端的yaml配置文件已经编辑完成并放置于atuned客户端任意目录下。

**atune-adm tuning**  \[OPTIONS] \<PROJECT\_YAML>

### 参数说明

* OPTIONS

  > \[!NOTE]说明\
  > 当使用参数时，-p参数后需要跟具体的项目名称且必须指定该项目yaml文件。

* PROJECT\_YAML：客户端yaml配置文件。

### 配置说明

**表 1**  服务端yaml文件

**表 2**  object项配置说明

**表 3**  客户端yaml文件配置说明

**表 4**  evaluations项配置说明

### 配置示例

服务端yaml文件配置示例：

```Conf
project: "compress"
maxiterations: 500
startworkload: ""
stopworkload: ""
object :
  -
    name : "compressLevel"
    info :
        desc : "The compresslevel parameter is an integer from 1 to 9 controlling the level of compression"
        get : "cat /root/A-Tune/examples/tuning/compress/compress.py | grep 'compressLevel=' | awk -F '=' '{print $2}'"
        set : "sed -i 's/compressLevel=\\s*[0-9]*/compressLevel=$value/g' /root/A-Tune/examples/tuning/compress/compress.py"
        needrestart : "false"
        type : "continuous"
        scope :
          - 1
          - 9
        dtype : "int"
  -
    name : "compressMethod"
    info :
        desc : "The compressMethod parameter is a string controlling the compression method"
        get : "cat /root/A-Tune/examples/tuning/compress/compress.py | grep 'compressMethod=' | awk -F '=' '{print $2}' | sed 's/\"//g'"
        set : "sed -i 's/compressMethod=\\s*[0-9,a-z,\"]*/compressMethod=\"$value\"/g' /root/A-Tune/examples/tuning/compress/compress.py"
        needrestart : "false"
        type : "discrete"
        options :
          - "bz2"
          - "zlib"
          - "gzip"
        dtype : "string"
```

客户端yaml文件配置示例：

```yaml
project: "compress"
engine : "gbrt"
iterations : 20
random_starts : 10

benchmark : "python3 /root/A-Tune/examples/tuning/compress/compress.py"
evaluations :
  -
    name: "time"
    info:
        get: "echo '$out' | grep 'time' | awk '{print $3}'"
        type: "positive"
        weight: 20
  -
    name: "compress_ratio"
    info:
        get: "echo '$out' | grep 'compress_ratio' | awk '{print $3}'"
        type: "negative"
        weight: 80
```

### 使用示例

* 进行tuning调优

  ```sh
  # atune-adm tuning --project compress --detail compress_client.yaml
  ```

* 恢复tuning调优前的初始配置，compress为yaml文件中的项目名称

  ```sh
  # atune-adm tuning --restore --project compress
  ```

---

---
url: /zh/docs/22.03_LTS_SP4/server/administration/sysmaster/devmaster_usage.md
---
# 使用说明

本章介绍 `devmaster`的使用方法，包括常驻进程配置、客户端工具、规则使用说明和网卡配置。

## 常驻进程配置

常驻进程 `devmaster`启动后会读取配置文件，并根据配置文件内容，调整日志级别、设置规则加载路径等等。`devmaster`拥有唯一的配置文件，路径为 `/etc/devmaster/config.toml`，文件内容采用 `toml`格式。

### 配置选项

目前 `devmaster`配置文件中支持的配置选项如下：

* `rules_d`: 指定规则加载路径，默认配置中设置为 `["/etc/devmaster/rules.d", "/lib/devmaster/rules.d", "/etc/udev/rules.d", "/run/udev/rules.d", "/lib/udev/rules.d"]`，该配置项未显式指定时，默认值为 `["/etc/devmaster/rules.d", "/run/devmaster/rules.d", "/usr/local/lib/devmaster/rules.d", "/usr/lib/devmaster/rules.d"]`。 `devmaster`当前不支持规则加载优先级，不同规则路径下的同名规则文件不会发生覆盖。规则文件的加载顺序按照 `rules_d`配置项中指定的目录顺序，相同目录下按照规则文件的字典序进行加载。
* `max_workers`: 指定最大 `worker`线程并发数，默认配置文件中设置为3，该配置项未设置时默认为3。手动设置该配置项时，不能超过 `cpu`核数。
* `log_level`: 指定日志级别，支持 `error`、 `debug`和 `info`级别，默认配置中设置为 `error`级别，未设置时默认为 `info`级别。
* `network_d`: 指定网卡配置加载路径，默认配置中设置为 `["/etc/devmaster/network.d"]`，未指定时无默认加载路径。网卡配置用于控制 `devmaster`的内置命令 `net_setup_link`的行为，具体可参考[网卡配置说明](#网卡配置)。
* `log_targets`: 指定日志输出对象，支持 `file`、 `console`和 `syslog`，默认值为 `["syslog"]`。设置 `file`时，日志会保存在 `/var/log/devmaster/devmaster.log`文件中。设置 `console`时，日志直接在控制台显示。设置 `syslog`时，日志输出到 `/dev/log`套接字并由相关日志服务，比如 `rsyslog`间接管理，如果未配置日志服务，则日志直接在控制台显示。可以指定多个日志输出对象。

## 客户端工具

`devctl`是常驻进程 `devmaster`的客户端工具，用来控制 `devmaster`的行为、模拟设备事件、调试规则等等。以下介绍一些常用的 `devctl`使用命令：

### 查看设备数据库信息

查看设备的 `sysfs`属性和数据库信息，或清理数据库：

```shell
# devctl info [OPTIONS] [DEVICES]...
```

### 监听设备事件

监听内核上报的 `uevent`事件和 `devmaster`处理完设备后发出的事件，分别以 `KERNEL`和 `USERSPACE`作为前缀进行区分：

```shell
# devctl monitor [OPTIONS]
```

### 触发设备事件

模拟一个设备动作，使内核上报对应的uevent事件，用于重放内核初始化过程中的冷插(coldplug)设备事件：

```shell
# devctl trigger [OPTIONS] [DEVICES]...
```

### 测试内置命令

测试内置命令在某个设备上的执行效果，目前支持的内置命令包括 `blkid`、 `input_id`、 `kmod`、 `net_id`、 `net_setup_link`、 `path_id`、 `usb_id`，可以触发的事件类型包括 `add`、 `change`、 `remove`、 `move`、 `online`、 `offline`、 `bind`和 `unbind`：

```shell
# devctl test-builtin [OPTIONS] <BUILTIN> <SYSPATH>
```

## 规则使用说明

`devmaster`的规则由一组规则文件组成， `devmaster`常驻进程启动后会根据配置文件中指定的规则加载目录，按字典序依次加载各个规则文件。

> \[!NOTE]说明
>
> 增加、删除、修改规则或配置文件后，均需要重启 `devmaster`使之生效。
>
> 重启 `devmaster`需要执行 `sctl stop devmaster`和 `sctl start devmaster`，不能使用 `sctl restart devmaster`。

### 常用规则案例

以下介绍几种常见的规则应用案例。

#### 示例1: 创建块设备软链接

通过 `blkid`内置命令，读取块设备的 `uuid`，并基于 `uuid`创建块设备的软链接。

触发拥有文件系统的某块设备的事件后，在 `/dev/test`目录下生成该设备对应的软链接。

以 `sda1`分区块设备为例，测试规则效果：

1. 创建规则文件 `/etc/devmaster/rules.d/00-persist-storage.rules`，内容如下：

   ```shell
   SUBSYSTEM!="block", GOTO="end"

   IMPORT{builtin}=="blkid"

   ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="test/$env{ID_FS_UUID_ENC}"

   LABEL="end"
   ```

2. 重启 `devmaster`：

   ```shell
   # sctl stop devmaster
   # sctl start devmaster
   ```

3. 触发 `sda1`设备的事件：

   ```shell
   # devctl trigger /dev/sda1
   ```

4. 查看 `/dev/test/`目录下存在指向 `sda1`的软链接，表示规则生效：

   ```shell
   # ll /dev/test/
   total 0
   lrwxrwxrwx 1 root root 7 Sep  6 15:35 06771fe1-39da-42d7-ad3c-236a10d08a7d -> ../sda1
   ```

#### 示例2: 网卡重命名

使用 `net_id`内置命令，获取网卡设备的硬件属性，再使用 `net_setup_link`内置命令，基于网卡配置选择某个硬件属性作为网卡名，最后通过 `NAME`规则重命名网卡。

以 `ens33`网卡为例，测试网卡重命名规则的效果：

1. 创建规则文件 `/etc/devmaster/rules.d/01-netif-rename.rules`，内容如下：

   ```shell
   SUBSYSTEM!="net", GOTO="end"

   IMPORT{builtin}=="net_id"

   IMPORT{builtin}=="net_setup_link"

   ENV{ID_NET_NAME}=="?*", NAME="$env{ID_NET_NAME}"

   LABEL="end"
   ```

2. 重启 `devmaster`：

   ```shell
   # sctl stop devmaster
   # sctl start devmaster
   ```

3. 创建网卡配置`/etc/devmaster/network.d/99-default.link`，内容如下：

   ```shell
   [Match]
   OriginalName = "*"

   [Link]
   NamePolicy = ["database", "onboard", "slot", "path"]
   ```

4. 先将网卡设备下线：

   ```shell
   # ip link set ens33 down
   ```

5. 将网卡名临时命名为 `tmp`：

   ```shell
   # ip link set ens33 name tmp
   ```

6. 触发网卡设备的 `add`事件：

   ```shell
   # devctl trigger /sys/class/net/tmp --action add
   ```

7. 查看网卡名称，发现重新命名为 `ens33`，表示规则生效：

   ```shell
   # ll /sys/class/net/| grep ens33
   lrwxrwxrwx 1 root root 0 Sep  6 11:57 ens33 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:01.0/net/ens33
   ```

8. 激活网卡后恢复网络连接：

   ```shell
   # ip link set ens33 up
   ```

> \[!NOTE]说明
>
> 网卡设备处于激活状态下无法重命名，需要先将其下线。另外 `devmaster`仅在网卡设备的 `add`事件下对网卡重命名才会生效。
>
> 在内核参数中添加 `net.ifnames=0`或 `net.ifnames=false`参数会关闭网卡重命名功能。

#### 示例3: 修改设备节点的用户权限

`OPTIONS+="static_node=<devnode>`规则会使 `devmaster`启动后，立即将本规则行中的用户权限应用在 `/dev/<devnode>`设备节点上。重启 `devmaster`后立即生效，无需设备事件触发。

1. 创建规则文件`/etc/devmaster/rules.d/02-devnode-privilege.rules`，内容如下：

   ```shell
   OWNER="root", GROUP="root", MODE="777", OPTIONS+="static_node=tty5"
   ```

2. 重启 `devmaster`：

   ```shell
   # sctl stop devmaster
   # sctl start devmaster
   ```

3. 观察 `/dev/tty5`的用户、用户组和权限，变更为 `root`、`root`和 `rwxrwxrwx`，表示规则生效：

   ```shell
   # ll /dev/tty5
   crwxrwxrwx 1 root root 4, 5 Feb  3  2978748 /dev/tty5
   ```

## 网卡配置

`devmaster`的网卡重命名功能由内置命令 `net_id`、`net_setup_link`和网卡配置文件配合完成。在规则文件中，通过 `net_id`获取网卡的硬件属性，再使用 `net_setup_link`选择某个网卡属性作为新的网卡名。`net_setup_link`命令基于网卡配置，针对特定网卡设备，控制网卡命名的风格。本章主要介绍网卡配置文件的使用方法，网卡重命名的实施方法可参考[网卡重命名规则案例](#示例2-网卡重命名)。

### 默认网卡配置

`devmaster`提供了如下默认网卡配置：

```toml
[Match]
OriginalName = "*"

[Link]
NamePolicy = ["onboard", "slot", "path"]
```

网卡配置文件中包含 `[Match]`匹配节和 `[Link]`控制节，每节中包含若干配置项。匹配节的配置项用于匹配网卡设备，当网卡满足所有匹配条件时，将控制节中的所有配置项作用在网卡上，比如设置网卡名选取策略、调整网卡参数等等。

以上列举的默认网卡配置表示将该配置作用在所有网卡设备上，并依次检查 `onboard`、`slot`和 `path`风格的网卡命名风格，如果找到一个可用的风格，就以该风格对网卡进行命名。

> \[!NOTE]说明
>
> 在内核参数中添加 `net.ifnames=0`或 `net.ifnames=false`参数会关闭网卡重命名功能。
>
> 如果发现网卡重命名未生效，可以检查内核参数中是否配置了这两项。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/maintenance/common_skills/information_collection.md
---
# 信息收集

## 查看OS信息

（1）查看操作系统版本信息，可用下面3个命令进行查询：

命令1，如下：

```shell
cat /etc/openEuler-latest
```

命令2，如下：

```shell
cat /etc/os-release
```

命令3，如下：

```shell
cat /etc/openEuler-release
```

（2）查看内核版本信息

```shell
uname -a
```

## 查看硬件信息

（1）查看CPU的统计信息

```shell
lscpu
```

（2）查看CPU相关参数

```shell
cat /proc/cpuinfo
```

（3）查看系统内存信息

```shell
cat /proc/meminfo
```

（4）查看内存信息

```shell
dmidecode -t memory
```

（5）查看硬盘和分区分布

```shell
lsblk
```

（6）看硬盘和分区的详细信息

```shell
fdisk -l
```

（7）查看网卡硬件信息

```shell
lspci | grep -i 'eth'
```

（8）查看所有网络接口

```shell
ip a 或 ifconfig -a
```

（9）查看某个网络接口的详细信息

```shell
ethtool enp7s0 （以enp7s0为例）
```

（10）查看pci信息

```shell
lspci
```

（11）查看设备树

```shell
lspci -t
```

（12）查看BIOS信息

```shell
[root@openEuler ~]# dmidecode -t bios
# dmidecode 3.4
Getting SMBIOS data from sysfs.
SMBIOS 3.0.0 present.

Handle 0x0000, DMI type 0, 26 bytes
BIOS Information
        Vendor: EFI Development Kit II / OVMF
        Version: 0.0.0
        Release Date: 02/06/2015
        Address: 0xE8000
        Runtime Size: 96 kB
        ROM Size: 64 kB
        Characteristics:
                BIOS characteristics not supported
                Targeted content distribution is supported
                UEFI is supported\
                System is a virtual machine
        BIOS Revision: 0.0
```

## 查看软件信息

（1）查看软件包的详细信息

```shell
rpm -qi systemd（以systemd为例）
```

（2）查看软件包提供的模块

```shell
rpm -q --provides  （以systemd为例）
```

（3）查看所有已安装软件包

```shell
rpm -qa （以systemd为例）
```

（4）查看软件包文件列表

```shell
rpm -ql （以python3-rpm为例）
```

## 查看OS日志

（1）查看系统启动后的信息和错误日志

```shell
cat  /var/log/messages
```

（2）查看安全相关的日志信息

```shell
cat /var/log/secure
```

（3）查看邮件相关的日志信息

```shell
cat /var/log/maillog
```

（4）查看定时任务相关的日志信息

```shell
cat /var/log/cron
```

（5）查看UUCP和news设备相关的日志信息

```shell
cat /var/log/spooler
```

（6）查看系统启动日志消息

```shell
cat /var/log/boot.log
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/system_container/shared_memory_channels.md
---
# 共享内存通道

## 功能描述

系统容器提供容器与主机进程通过共享内存进行通信的功能，通过在容器创建时配置--host-channel参数，可以在容器与主机之间共享同一tmpfs，从而达到主机与容器间通信的功能。

## 参数说明

## 约束限制

* 宿主机上挂载的tmpfs的生命周期为从容器启动到容器删除，容器删除并解除对空间的占用后会移除这片空间。
* 容器删除时会将宿主机上挂载tmpfs的路径删除，所以不允许使用宿主机上已存在的目录。
* 为了宿主机上非root用户运行的进程能够与容器内进行通信，宿主机上tmpfs挂载的权限为1777。

## 使用示例

创建容器时指定--host-channel参数：

```sh
[root@localhost ~]# isula run --rm -it --host-channel /testdir:/testdir:rw:32M --system-container --external-rootfs /root/myrootfs none init 
root@3b947668eb54:/# dd if=/dev/zero of=/testdir/test.file bs=1024 count=64K 
dd: error writing '/testdir/test.file': No space left on device 
32769+0 records in 
32768+0 records out 
33554432 bytes (34 MB, 32 MiB) copied, 0.0766899 s, 438 MB/s
```

> \[!NOTE]说明
>
> * 使用--host-channel大小限制时，若在容器内创建共享文件，则会受到容器内的内存配额限制（在容器内存使用达到极限时可能会产生oom）。
> * 若用户在主机端创建共享文件，则不受容器内的内存配额限制。
> * 若用户需要在容器内创建共享文件，且业务为内存密集型，可以通过设置容器内存配额为在原本基础上加上--host-channel配置的大小来消除影响。

---

---
url: /zh/docs/22.03_LTS_SP4/server/releasenotes/key_features.md
---
# 关键特性

## AI专项

智能时代，操作系统需要面向AI不断演进。一方面，在操作系统开发、部署、运维全流程以AI加持，让操作系统更智能；另一方面，openEuler已支持Arm，x86，RISC-V等全部主流通用计算架构，在智能时代，openEuler也率先支持NVIDIA、昇腾等主流AI处理器，成为使能多样性算力的首选。

* **OS for AI**：openEuler兼容NVIDIA、Ascend等主流算力平台的软件栈，为用户提供高效的开发运行环境。通过将不同AI算力平台的软件栈进行容器化封装，即可简化用户部署过程，提供开箱即用的体验。同时，openEuler也提供丰富的AI框架，方便大家快速在openEuler上使用AI能力。

  * openEuler已兼容CANN、CUDA等硬件SDK，以及TensorFlow、PyTorch等相应的AI框架软件，支持AI应用在openEuler上高效开发与运行。
  * openEuler AI软件栈容器化封装优化环境部署过程，并面向不同场景提供以下三类容器镜像。
    1. SDK镜像：以openEuler为基础镜像，安装相应硬件平台的SDK，如Ascend平台的CANN或NVIDIA的CUDA软件。
    2. AI框架镜像：以SDK镜像为基础，安装AI框架软件，如PyTorch或TensorFlow。
    3. 模型应用镜像：在AI框架镜像的基础上，包含完整的工具链和模型应用。

  相关使用方式请参考[《openEuler AI 容器镜像用户指南》](https://docs.openeuler.openatom.cn/zh/docs/22.03_LTS_SP4/tools/ai/euler-copilot-framework/ai_full_stack/ai_container_image_userguide/ai_container_image_user_guide.html)。

* **AI for OS**：当前，欧拉和AI深度结合，一方面使用基础大模型，基于大量欧拉操作系统的代码和数据，训练出EulerCopilot，初步实现代码辅助生成、智能问题智能分析、系统辅助运维等功能，让openEuler更智能。EulerCopilot-智能问答：EulerCopilot智能问答平台目前支持web和智能shell两个入口。

  1. Web入口：操作简单，可咨询操作系统相关基础知识，openEuler动态数据、openEuler运维问题解决方案、openEuler项目介绍与使用指导等等。
  2. 智能Shell入口：自然语言和openEuler交互，启发式的运维。

  相关使用方式请参考[《openEuler 智能助手使用指南》](https://docs.openeuler.openatom.cn/zh/docs/22.03_LTS_SP4/tools/ai/euler-copilot-framework/openeuler_intelligence/intelligent_assistant/quick_start/smart_web/deploy_guide/offline.html)。

## AI for Compiler

AI For Compiler框架接受编译器相关信息作为输入，在框架内部进行编译器信息解析和ONNX环境准备之后，向ONNX模型发送推理请求。ONNX模型将根据编译器信息进行推理，并将推理结果返回AI For Compiler框架。AI For Compiler框架对推理结果进行解析、处理，将处理后的结果返回编译器，用于指导多种优化手段的实施。

## openEuler Embedded

openEuler Embedded 围绕工业和机器人领域持续深耕，通过行业项目垂直打通，不断完善和丰富嵌入式技术栈和生态。openEuler 22.03 LTS SP4 Embedded 支持嵌入式虚拟化弹性底座，提供 Jailhouse 虚拟化方案、openAMP 轻量化混合部署方案，用户可以根据自己的使用场景选择最优的部署方案。同时支持 ROS humble 版本，集成 ros-core、ros-base、SLAM 等核心软件包，满足 ROS2 运行时要求。未来 openEuler Embedded 将协同 openEuler 社区生态伙伴、用户、开发者，逐步扩展支持 RISC-V、龙芯等芯片架构，丰富工业中间件、ROS 中间件、仿真系统等能力，打造嵌入式领域操作系统解决方案。

* **南向生态**： openEuler Embedded 22.03 LTS SP4当前主要支持 ARM64、x86-64 两种芯片架构，未来openEuler Embedded Linux还将支持龙芯、飞腾等芯片。
* **嵌入式弹性虚拟化底座** ：openEuler Embedded的弹性虚拟化底座是为了在多核片上系统（SoC, System On Chip）上实现多个操作系统共同运行的一系列技术的集合，包含了裸金属、嵌入式虚拟化、轻量级容器、LibOS、可信执行环境（TEE）、异构部署等多种实现形态。
* **混合关键性部署框架**： openEuler Embedded打造了构建在融合弹性底座之上混合关键性部署框架，并命名为MICA（MIxed CriticAlity），旨在通过一套统一的框架屏蔽下层弹性底座形态的不同，从而实现Linux和其他OS运行时便捷地混合部署。依托硬件上的多核能力使得通用的Linux和专用的实时操作系统有效互补，从而达到全系统兼具两者的特点，并能够灵活开发、灵活部署。
* **北向生态**：600+嵌入式领域常用软件包的构建；支持ROS2 humble版本，集成ros-core、ros-base、SLAM 等核心包，并提供软实时能力，软实时中断响应时延微秒级。
* **UniProton硬实时系统**：是一款实时操作系统，具备极致的低时延和灵活的混合关键性部署特性，可以适用于工业控制场景，既支持微控制器 MCU，也支持算力强的多核 CPU。目前关键能力如下：
  * 支持Cortex-M、Arm64、x86\_64架构，支持M4、RK3568、RK3588、x86\_64、Hi3093、树莓派4B。
  * 支持树莓派4B、Hi3093、RK3588、x86\_64设备上通过裸金属模式和openEuler Embedded Linux混合部署。
  * 支持通过gdb在openEuler Embedded Linux侧远程调试。
  * 支持890+ POSIX接口，支持文件系统、设备管理、shell控制台和网络。

## openEuler 内核新特性

openEuler 22.03 LTS SP4 基于 Linux Kernel 5.10 构建，在此基础上，同时吸收了社区高版本的有益特性及社区创新特性。

### 体系结构

* 支持CPU在线巡检：静默数据损坏(SDC)可能导致数据丢失和数据被破坏。通过执行巡检指令，发现存在静默故障的核，提前对故障核进行隔离，避免出现更严重的故障，提高系统可靠性。

* 自适应NUMA特性：随着摩尔定律放缓，硬件架构Scale-up的收益空间收窄，需要解决资源“locality”的问题，Scale-out是硬件架构的主要发展方向。对于NUMA架构而言，跨NUMA访存时延相比本地访存时延大大增加，而linux操作系统以吞吐为中心，强调负载均衡，打破全局跨NUMA访存最优的状态。本特性是以业务亲缘关系为中心的负载均衡机制：突破Linux调度模型中负载均衡的约束，以亲缘关系为中心，在资源未达到瓶颈时，将具有亲缘关系的任务packing在一起，减少跨NUMA访存。

  主要技术点包含如下：

  1. 感知亲缘关系：通过软硬协同，感知线程间网络、内存的亲缘关系。
  2. 感知资源瓶颈：基于NUMA维度的资源瓶颈感知，感知NUMA资源是否达到瓶颈。
  3. 优化选核和负载均衡机制：基于可编程调度的框架，突破linux原有负载均衡约束，决策NUMA内还是NUMA 间调度，在没有资源瓶颈前提下，将具有亲缘关系的任务调度到相同NUMA上。

* xfs atomic write特性：数据库（如MySQL的InnoDB存储引擎）使用Doublewrite机制，写2次16KB数据到磁盘，来保护故障场景下数据不受部分写损坏的影响。xfs atomic write特性基于硬件的原子写能力，通过保证用户态软件发起的Direct IO原子性写入磁盘，可以替代数据库Doublewrite机制，以减少IO写数量，提升数据库的写入性能。

* 增强核隔离特性：在HPC场景，由于每个线程会频繁进行同步，OS背景噪声对性能的影响会随着节点数增多逐步放大，OS背景噪声包括背景守护进程、外设中断、内核背景线程等。增强核隔离特性将系统CPU分为housekeeping CPU和non-housekeeping CPU，将OS背景噪声集中在housekeeping CPU上，non-housekeeping CPU只运行业务计算任务，通过减少业务运行时背景噪声的干扰，提升业务性能。non-housekeeping CPU可以通过启动参数nohz\_full和isolcpus指定，增加enhanced\_isolcpus参数后可以进一步消除磁盘IO的噪声干扰。详情可参考文档 [kernel-parameters.txt](https://atomgit.com/openeuler/kernel/blob/openEuler-22.03-LTS/Documentation/admin-guide/kernel-parameters.txt)。

### 调度

支持功耗感知调度：在面向业务层面收集访存带宽，CPU负载等数据，确保业务关键线程资源得到满足。引入物理拓扑，调压域感知新的维度，减少单DIE调频、单DIE调压带来的调频降功耗的局限，保障在低负载下能最小化功耗。具体如下：

1. 根据物理拓扑构建逃逸通道，根据CPU负载和访存带宽瓶颈自动调节节能级别，低负载集中业务减少功耗，高负载扩散业务的策略保证QOS。
2. 定时器收集负载，带宽等信息，感知访存带宽，避免跨DIE访问。
3. OS新增调压域、idle静态功耗感知、业务标签等机制，实现智能感知调频、CPU idle、DVFS，来优化业务性能。
4. 提供一套任务标签化的机制来为每个用户态进程来标注自己的QoS等级，并且支持将标记为不同优先等级的任务分别绑定到不同的分区上进行调度，以此来保证业务QoS，每个QoS的功耗策略也可以通过cpufreq模块进行配置，以达到不影响业务QoS的前提下保障节能的需求。

### 内存

* **cgroup粒度的KSM使能增强**：通过memory.ksm接口为memcg中的进程使能KSM时，仅配置已在memcg中的进程，后续新加入memcg的进程不会自动使能KSM，需要重新调用该接口使能。该特性支持memcg的KSM状态位，向memory.ksm接口写入1后，新加入memcg的进程自动使能KSM，简化了为进程使能KSM的操作。
* **支持地址随机化区域指定**：内核增加CONFIG\_NOKASLR\_MEM\_RANGE选项，允许用户指定至多4个物理内存区域，系统在启动时避开这4个内存区域进行物理地址随机化。arm64和x86\_64架构的物理地址随机化支持从启动参数指定不随机化的范围，避免跟kdump预留内存冲突，解决内存多占用导致内存不足问题。
* **bufferio限速自动绑定memcg和blkcg**：该特性由美团贡献到openEuler社区。当前，cgroup v1中的子系统独立，无法协同工作，如blkio cgroup在限制资源时无法感知memory cgroup中的资源使用，需手动创建映射关系。改进特性允许在内核态自动绑定同一进程的blkio cgroup和memory cgroup映射关系。用户可在编译阶段通过CONFIG\_CGROUP\_V1\_BIND\_BLKCG\_MEMCG开关选择启用，也可在系统运行时通过sysctl\_bind\_memcg\_blkcg\_enable动态开关调整。

### 虚拟化

* **KVM TDP MMU**：该特性由Intel贡献到openEuler社区，是Linux 5.10之后，内存虚拟化领域中用于提升KVM可扩展性的一个改进方案。相对于传统的KVM MMU，TDP MMU提供了更高效的并发Page Fault处理机制，从而使得KVM对大型虚拟机（多vCPU，大内存）有了更好的支持。同时，通过采用新的EPT/NPT遍历接口，KVM TDP MMU摒弃了传统内存虚拟化中对rmap数据结构的依赖，使得主机内存有了更好的利用率。继openEuler 22.03-SP3中提供了Linux 5.10至5.15版本的KVM TDP MMU支持之后，openEuler 22.03-SP4版本增加了Linux 5.15至6.2版本间对KVM MMU的后续优化和bugfix，如提供优化大幅降低KVM MMU unloading的次数以及unloading的流程，优化KVM MMU page fault的处理流程，以及memslot更新时的相关优化等。
* **Dirty ring**：该特性由麒麟软件贡献到openEuler社区。原始内核中对vcpu访问内存⻚的统计是通过dirty bitmap来实现的，dirty bitmap的统计粒度是⼀个kvm\_mem\_slot。而dirty ring的统计粒度更细，可以精确到⼀个vcpu甚⾄⼀个内存⻚，并且统计可以在⽤⼾空间进⾏，dirty ring的扩展性较好，可以在⼤内存规格的虚机上开启此特性。

## NestOS 容器操作系统

NestOS是在openEuler社区孵化的云底座操作系统，集成了rpm-ostree支持、ignition配置等技术。采用双根文件系统、原子化更新的设计思路，使用nestos-assembler快速集成构建，并针对K8S、OpenStack等平台进行适配，优化容器运行底噪，使系统具备十分便捷的集群组建能力，可以更安全的运行大规模的容器化工作负载。

1. **开箱即用的容器平台**：NestOS集成适配了iSulad、Docker、Podman等主流容器引擎，为用户提供轻量级、定制化的云场景OS。
2. **简单易用的配置过程**：NestOS通过ignition技术，可以以相同的配置方便地完成大批量集群节点的安装配置工作。
3. **安全可靠的包管理**：NestOS使用rpm-ostree进行软件包管理，搭配openEuler软件包源，确保原子化更新的安全稳定状态。
4. **友好可控的更新机制**：NestOS使用zincati提供自动更新服务，可实现节点自动更新与重新引导，实现集群节点有序升级而服务不中断。
5. **紧密配合的双根文件系统**：NestOS采用双根文件系统的设计实现主备切换，确保NestOS运行期间的完整性与安全性。

## 机密计算远程证明服务：远程证明TEE插件框架

随着机密计算的逐渐成熟，机密计算硬件众多，信任根及远程证明差异，成为阻塞TEE之间建立信任关系的关键，构建远程证明统一框架，提供证明代理、TEE插件框架等组件，实现不同TEE证明报告统一验证流程。

* 当前版本证明代理支持鲲鹏Trustzone、virtCCA证明报告获取。
* TEE报告校验插件框架支持运行时兼容鲲鹏Trustzone、virtCCA证明报告验证，并支持证明代理集成TEE报告校验插件框架，实现无证明服务时的点对点验证。

## DPU 场景 vDPA 新增支持磁盘，并支持配置vDPA网卡和磁盘的虚机热迁移

内核态vDPA框架，为设备虚拟化提供了一种性能与直通持平，且支持跨硬件厂商热迁移的方案。新增对于DPU卡形态的硬件支持，支持DPU卡呈现给前端主机的网络/存储设备，接入通用vDPA框架，并支持热迁移。

DPU场景vDPA需求继承openEuler 2203 LTS SP1/SP3支持vDPA网卡设备以及支持vDPA网卡直通热迁移特性。新增支持对于DPU卡形态呈现存储设备的支持，并继承支持vDPA磁盘直通热迁移特性。详细范围说明如下:

* 支持虚拟机配置vDPA磁盘作为数据盘。
* 支持虚拟机热插拔vDPA磁盘。
* 支持同时配置vDPA网卡和vDPA磁盘热迁移。

## virtCCA 机密虚机特性合入

virtCCA机密虚机特性在TEE侧实现机密虚机能力，实现现有普通虚机中的软件栈无缝迁移到机密环境中。

基于Arm CCA标准接口，在Trustzone固件基础上构建TEE虚拟化管理模块，实现机密虚机间的内存隔离、上下文管理、生命周期管理和页表管理等机制，支持客户应用无缝迁移到机密计算环境中。

**技术约束:**

1. 机密虚机每次启动需要对Guest Kernel等内存信息进行度量，因此不支持reboot能力.
2. 基于安全性考虑，宿主机BIOS中CPU超线程需要关闭。

## PowerAPI 功耗管理统一API

PowerAPI是一组用于支持系统功率管理的 API，它们提供了一种标准化方法来管理系统的功率使用情况，包括监控，调整和优化系统的功率消耗。这些 API 可以帮助系统管理员更好地管理系统的能源消耗，从而提升系统的效率和可靠性，并减少能源成本。

PowerAPI含pwrapis服务程序和libpwrapi.so动态库。PowerAPI实际上是对OS提供的各类接口的统一封装，方便上层管理软件对功耗进行统一管理，其目标为：

* 屏蔽底层sci/sysfs/procs/impi等接口差异，屏蔽平台差异。
* 屏蔽功耗管理场景下，root特权需求(如集群代理Agent不要求采用root用户运行)。

**PowerAPI接口已提供功能：**

* 数据采集：系统功耗、CPU利用率、LLC cache Miss等。
* CPU调频、CPU休眠管理。
* 瓦特调度管理、分域调度管理 。

## eagle 实现整机能耗管理

Eagle（Energy Aware intelliGent scheduler）智能功耗感知调节器是一个以提升能效为目标的调优服务。用户可通过配置策略的方式，让eagle实现整机功耗管理，进而提升能效，实现降本增效。Eagle基于PowerAPI构建，实现基于策略的能效调优。

1. **调优策略**：用户可根据业务情况，定制调优策略。
2. **智能场景感知功耗调节**：eagle通过powerapi实时感知并结合场景特点(CPU密集/访存密集/时延敏感)，应用优先级，通过智能调度、调频、Turbo、休眠等调节手段，保证业务体验的前提下，自适应地实现能效最优。
   * 基于能效域的智能调度：通过划分能效域，按需激活资源、均衡调度、智能Turbo等实现能效最优。
   * 性能损失可控的调频：通过实时感知业务性能变化，严格控制CPU调频导致的性能下降。
   * 更精确的智能休眠：实时感知业务特点，制定更适合业务的CPU、外设休眠策略。
   * 多维度功耗封顶控制：在功耗限额下，通过多维度、多设备的控制，保证高优先级应用的前提下，实现能效最优。
   * 基于预测的风扇调速： 通过建模，智能预测整机热变化，风扇提前响应，提升整机热可靠性，节省能耗。
3. **人机交互**：管理员可通过eagle-CLI实时监控系统状态，或对调优策略等进行配置。

## gala-ragdoll 支持配置变更溯源

利用ragdoll内部插件ragdoll-filetrace可以实现对配置文件的实时监控，当配置文件被修改的时候，会抓取操作文件的系统调用的信息保存到mysql数据库表中，ragdoll通过接口形式将数据库的数据提供给aops-hermes，用户可以在页面端查看配置文件的变更信息，比如：修改时间，修改方式，进程名称。目前可以支持监控系统中常用文本编辑工具，比如vi/vim、sed、echo、mv等。

---

---
url: /zh/docs/22.03_LTS_SP4/server/security/secharden/kernel_parameters.md
---
# 内核参数

## 加固内核参数

### 说明

内核参数决定配置和应用特权的状态。内核提供用户可配置的系统控制，这一系统控制可微调或配置，该功能特性可通过控制各种可配置的内核参数，来提高操作系统的安全特性。比如：通过微调或配置网络选项，可有效提高系统的安全性。

### 实现

1. 将[表1](#zh-cn_topic_0152100187_t69b5423c26644b26abe94d88d38878eb)中的加固项写入/etc/sysctl.conf文件中。

   > \[!NOTE]说明\
   > 写入方式如下：
   >
   > ```text
   > net.ipv4.icmp_echo_ignore_broadcasts = 1  
   > net.ipv4.conf.all.rp_filter = 1  
   > net.ipv4.conf.default.rp_filter = 1  
   > ```

   **表 1**  内核参数加固策略说明

2. 加载sysctl.conf文件中设置的内核参数。

   ```sh
   sysctl -p  /etc/sysctl.conf
   ```

### 其他安全建议

* net.ipv4.icmp\_echo\_ignore\_all：忽略ICMP请求。

  出于安全考虑，建议开启此项（当前默认值为0，开启将值设为1）。

  但开启后会忽略所有接收到的icmp echo请求的包(会导致机器无法ping通)，建议用户根据实际组网场景决定是否开启此项。

* net.ipv4.conf.all.log\_martians/net.ipv4.conf.default.log\_martians：对于仿冒/源路由/重定向数据包开启日志记录。

  出于安全考虑，建议开启此项（当前默认值为0，开启将值设为1）。

  但是开启后会记录带有不允许的地址的数据到内核日志中，存在冲日志风险，建议用户根据实际使用场景决定是否开启此项。

* net.ipv4.tcp\_timestamps：关闭tcp\_timestamps。

  出于安全考虑，建议关闭tcp\_timestamps（当前默认值为1，关闭将值设为0）。

  但是关闭此项会影响TCP超时重发的性能，建议用户根据实际使用场景决定是否关闭此项。

* net.ipv4.tcp\_max\_syn\_backlog：决定了SYN\_RECV状态队列的数量。

  该参数决定了SYN\_RECV状态队列的数量，超过这个数量，系统将不再接受新的TCP连接请求，一定程度上可以防止系统资源耗尽。建议由用户根据实际使用场景配置合适的值。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/maintenance/kernel_live_upgrade/kernel_live_upgrade.md
---
# 内核热升级用户指南

本文档介绍openEuler系统内核热升级特性的安装部署和使用方法，openEuler的内核热替换特性通过快速重启内核和程序热迁移实现，我们提供了一个用户态工具以自动化这一过程。

本文档适用于使用openEuler系统并希望了解和使用内核热升级的社区开发者、开源爱好者以及相关合作伙伴。使用人员需要具备基础的Linux操作系统知识。

## 使用场景

内核热升级的目标，是实现在秒级的端到端时延下，实现进程运行现场的保存和恢复。

使用场景通常符合以下两个条件：

1. 内核由于漏洞修复，版本更新等原因，需要重新启动。
2. 运行在内核之上的业务能够在内核重启后快速恢复状态。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/best_practices.md
---
# 最佳实践

## 性能最佳实践

### halt-polling

#### 概述

在计算资源充足的情况下，为使虚拟机获得接近物理机的性能，可以使用halt-polling特性。没有使用halt-polling特性时，当vCPU空闲退出后，主机会把CPU资源分配给其他进程使用。当主机开启halt-polling特性时，虚拟机vCPU处于空闲时会polling一段时间，polling的时间由具体配置决定。若该vCPU在polling期间被唤醒，可以不从主机侧调度而继续运行，减少了调度流程的开销，从而在一定程度上提高了虚拟机系统的性能。

> !\[!NOTE]说明
> halt-polling的机制保证虚拟机的vCPU线程的及时响应，但在虚拟机空载的时候，主机侧也会polling，导致主机看到vCPU所在CPU占用率比较高，而实际虚拟机内部CPU占用率并不高。

#### 操作指导

系统默认开启了halt-polling特性，polling的时间默认为500000ns。用户可以通过文件halt\_poll\_ns内容动态修改vCPU用于halt-polling的时间，单位为ns。

例如设置polling时间为400000，使用root用户执行命令如下：

```sh
# echo 400000 > /sys/module/kvm/parameters/halt_poll_ns
```

### IOThread配置

#### 概述

KVM平台上，对虚拟磁盘的读写在后端默认由QEMU主线程负责处理。这样会造成如下问题：

* 虚拟机的I/O请求都由一个QEMU主线程进行处理，因此单线程的CPU利用率成为虚拟机I/O性能的瓶颈。
* 虚拟机I/O在QEMU主线程处理时会持有QEMU全局锁(qemu\_global\_mutex)，一旦I/O处理耗时较长，QEMU主线程长时间占有全局锁，会导致虚拟机vCPU无法正常调度，影响虚拟机整体性能及用户体验。

可以为virtio-blk磁盘或者virtio-scsi控制器配置IOThread属性，在QEMU后端单独开辟IOThread线程处理虚拟磁盘读写请求，IOThread线程和virtio-blk磁盘或virtio-scsi控制器可配置成一对一的映射关系，尽可能地减少对QEMU主线程的影响，提高虚拟机整体I/O性能，提升用户体验。

#### 配置说明

使用IOThread线程处理虚拟机磁盘读写请求，需要修改虚拟机配置，这里给出具体的配置说明。

* 配置虚拟机高性能虚拟磁盘的总数。例如通过\<iothreads>配置IOThread线程的总数为4：

  ```conf
  <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>   
       <name>VMName</name>
       <memory>4194304</memory>
       <currentMemory>4194304</currentMemory>
       <vcpu>4</vcpu>
       <iothreads>4</iothreads>
  ```

* 给virtio-blk磁盘配置IOThread属性。<**iothread**>表示IOThread线程编号，编号从1开始配置，最大为\<iothreads>的配置值，且编号不能重复使用。例如将编号为2的IOThread配置给virtio-blk磁盘使用：

  ```conf
  <disk type='file' device='disk'>
        <driver name='qemu' type='raw' cache='none' io='native' iothread='2'/>
        <source file='/path/test.raw'/>
        <target dev='vdb' bus='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
  </disk>
  ```

* 给virtio-scsi控制器配置IOThread属性。例如将编号为2的IOThread配置给virtio-scsi控制器使用：

  ```conf
  <controller type='scsi' index='0' model='virtio-scsi'>
        <driver iothread='2'/>
        <alias name='scsi0'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
  </controller>
  ```

* IOThread线程绑定物理CPU

  虚拟磁盘IOThread线程的绑核配置，将IOThread线程绑定到用户指定的物理CPU范围内，不影响vCPU线程的资源占用诉求。\<iothread>表示IOThread线程编号，\<cpuset>表示绑定的物理CPU编号。

  ```conf
  <cputune>
  <iothreadpin iothread='1' cpuset='1-3,5,7-12' />
  <iothreadpin iothread='2' cpuset='1-3,5,7-12' />
  </cputune>
  ```

### 裸设备映射

#### 概述

配置虚拟机存储设备时，除了将文件配置给虚拟机作为虚拟磁盘使用外，还可以将块设备（物理LUN、逻辑卷等）直接配置给虚拟机使用，从而提升存储性能。该配置方式称为裸设备映射。在该配置方式下，虚拟磁盘向虚拟机呈现为一个SCSI（Small Computer System Interface，小型计算机系统接口）设备，且支持大部分SCSI命令。

裸设备映射根据后端实现特点，分为虚拟裸设备映射和物理裸设备映射，物理裸设备映射相对虚拟裸设备映射具有更优秀的性能和更丰富的SCSI命令，但物理裸设备映射需要将整块SCSI磁盘挂载给虚拟机使用，如果使用分区、逻辑卷等方式配置，虚拟机将无法识别。

#### 配置示例

裸设备映射需要修改虚拟机配置文件，这里给出配置示例。

* 虚拟裸设备映射

  将主机上存在的SCSI磁盘“/dev/sdc”挂载给虚拟机作为虚拟裸设备的配置示例如下：

  ```conf
  <domain type='kvm'>
   <devices>
      ...
      <controller type='scsi' model='virtio-scsi' index='0'/>
      <disk type='block' device='disk'>
          <driver name='qemu' type='raw' cache='none' io='native'/>
          <source dev='/dev/sdc'/>
          <target dev='sdc' bus='scsi'/>
          <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      </disk>
      ...
   </devices>
  </domain>
  ```

* 物理裸设备映射

  将主机上存在的SCSI磁盘“/dev/sdc”挂载给虚拟机作为物理裸设备的配置示例如下：

  ```conf
  <domain type='kvm'>
   <devices>
      ...
      <controller type='scsi' model='virtio-scsi' index='0'/>
      <disk type='block' device='lun' rawio='yes'>
          <driver name='qemu' type='raw' cache='none' io='native'/>
          <source dev='/dev/sdc'/>
          <target dev='sdc' bus='scsi'/>
          <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      </disk>
      ...
   </devices>
  </domain>
  ```

### kworker隔离绑定

#### 概述

kworker是Linux内核实现的per-CPU线程，用来执行系统中的workqueue请求。kworker线程会和vCPU线程争抢物理核资源，导致虚拟化业务性能抖动。为了使虚拟机能够稳定的运行，减少kworker线程对虚拟机的干扰，可以将主机上的kworker线程绑定到特定的CPU上运行。

#### 操作步骤

用户可以通过修改/sys/devices/virtual/workqueue/cpumask文件，将workqueue中的任务绑定到cpumask中指定的CPU上。cpumask中的掩码以十六进制表示，例如将kworker绑定到CPU0~CPU7上，对应掩码为ff，使用root用户执行命令如下：

```sh
# echo ff > /sys/devices/virtual/workqueue/cpumask
```

### 内存大页

#### 概述

相比传统的4K内存分页，openEuler也支持2MB/1GB的大内存分页。内存大页可以有效减少TLB miss，显著提升内存访问密集型业务的性能。openEuler使用两种技术来实现内存大页。

* 静态大页

  静态大页要求宿主机操作系统在加载前提前预留一个静态大页池，虚拟机创建时通过修改xml配置文件的方式，指定虚拟机的内存从静态大页池中分配。静态大页能保证虚拟机的所有内存在host上始终以大页形式存在，保证物理连续，但增加了部署的困难，修改静态大页池的页面大小后需要重启host才能生效。静态大页的页面大小支持2M或1G。

* 透明大页

  如果开启透明大页模式THP（Transparent Huge Pages），虚拟机分配内存时自动选择可用的2M连续页，同时自动完成大页的拆分合并，当没有可用的2M连续页时，它会选择可用的64K（AArch64架构）或4K（x86\_64架构）页面进行分配。透明大页的好处是不需要用户感知，同时能尽量使用2M大页以提升内存访问性能。

在虚拟机完全使用静态大页的场景下，可以通过关闭透明大页的方法，减少宿主机操作系统的开销，以便虚拟机获得更稳定的性能。

#### 操作指导

* 使用静态大页

  在创建虚拟机之前通过修改XML的方式，为虚拟机配置使用静态大页。

  ```conf
    <memoryBacking>
      <hugepages>
        <page size='1' unit='GiB'/>
      </hugepages>
    </memoryBacking>
  ```

  以上XML片段表示为虚拟机配置1G静态大页。

  ```conf
    <memoryBacking>
      <hugepages>
        <page size='2' unit='MiB'/>
      </hugepages>
    </memoryBacking>
  ```

  以上XML片段表示为虚拟机配置2M静态大页。

* 使用透明大页

  通过sysfs可以动态开启使用透明大页：

  ```sh
  # echo always > /sys/kernel/mm/transparent_hugepage/enabled
  ```

  动态关闭使用透明大页：

  ```sh
  # echo never > /sys/kernel/mm/transparent_hugepage/enabled
  ```

### PV-qspinlock

#### 概述

PV-qspinlock主要是针对虚拟化CPU超分场景自旋锁的优化，允许hypervisor将处于锁上下文中的vCPU置于block状态，并在锁释放后将对应的vCPU唤醒，在超分场景下能够更好地利用pCPU资源，对于编译的应用场景有一定的优化，可以减少编译应用的时长。

#### 操作指导

修改虚拟机/boot/efi/EFI/openEuler/grub.cfg配置文件，在命令行启动参数添加arm\_pvspin，重启虚拟机后生效。PV-qspinlock生效后，虚拟机内部使用dmesg命令可以查到如下日志打印：

```conf
[    0.000000] arm-pv: PV qspinlocks enabled
```

> !\[!NOTE]说明
> PV-qspinlock仅限于宿主机和虚拟机操作系统均为openEuler 20.09及以上版本支持，且虚拟机内核编译选项需要配置CONFIG\_PARAVIRT\_SPINLOCKS=y（openEuler默认配置）。

### Guest-Idle-Haltpoll

#### 概述

为了保证公平性及降低功耗，当虚拟机vCPU空闲时，虚拟机将执行WFx/HLT指令退出到宿主机中，并触发上下文切换。宿主机将决定在物理CPU上调度其他进程或vCPU，或进入节能模式。但是，虚拟机和宿主机之间的切换、额外的上下文切换以及唤醒IPI中断开销较大，在频繁睡眠和唤醒的业务中该问题尤为突出。Guest-Idle-Haltpoll技术是指当虚拟机vCPU空闲时，不立刻执行WFx/HLT并发生VM-exit，而是在虚拟机内部轮询（polling）一段时间。在该时间段内，其他共享LLC的vCPU在该vCPU上的任务被唤醒不需要发送IPI中断，减少了发送和接收处理IPI的开销及虚拟机陷出（VM-exit）的开销，从而降低任务唤醒的时延。

> !\[!NOTE]说明
> 由于vCPU在虚拟机内部执行idle-haltpoll会增加vCPU在宿主机的CPU开销，所以开启该特性建议vCPU在宿主机独占物理核。

#### 操作指导

Guest-Idle-Haltpoll特性默认关闭，这里给出开启该特性的操作指导。

1. 使能Guest-Idle-Haltpoll特性。
   * 若宿主机处理器架构为x86，可以在宿主机的虚拟机XML中配置“hint-dedicated”使能该特性，通过虚拟机XML配置将vCPU独占物理核的状态传递给虚拟机。vCPU独占物理核的状态由宿主机保证。

     ```conf
     <domain type='kvm'>
      ...
      <features>
        <kvm>
          ...
          <hint-dedicated state='on'/>
        </kvm>
      </features>
       ...
     </domain>
     ```

     或者登录到虚拟机内部以虚拟机为粒度进行在线配置。该方法不依赖宿主机配置vCPU独占物理核。

     ```sh
     echo Y > /sys/module/cpuidle_haltpoll/parameters/force
     ```

   * 若宿主机处理器架构为AArch64，当前只支持在虚拟机内部进行在线配置的方式使能该特性。

     ```sh
     echo Y > /sys/module/cpuidle_haltpoll/parameters/force
     ```

2. 确认Guest-Idle-Haltpoll特性是否生效。在虚拟机中执行如下命令，若返回haltpoll，说明特性已经生效。

   ```sh
   # cat /sys/devices/system/cpu/cpuidle/current_driver
   ```

3. （可选）配置Guest-Idle-Haltpoll参数。
   虚拟机的/sys/module/haltpoll/parameters/路径下提供了如下配置文件，用于调整配置参数，用户可以根据业务特点选择调整。

   * guest\_halt\_poll\_ns: 全局参数，指vCPU空闲后polling的最大时长，默认值为200000（单位ns）。
   * guest\_halt\_poll\_shrink:  当唤醒事件发生在全局guest\_halt\_poll\_ns时间之后，用于收缩当前vCPU guest\_halt\_poll\_ns的除数因子，默认值为2。
   * guest\_halt\_poll\_grow:  当唤醒事件发生在当前vCPU guest\_halt\_poll\_ns之后且在全局guest\_halt\_poll\_ns之前，用于扩展当前vCPU guest\_halt\_poll\_ns的乘数因子，默认值为2。
   * guest\_halt\_poll\_grow\_start: 当系统空闲时，每个vCPU的guest\_halt\_poll\_ns最终会达到零。该参数用于设置当前vCPU guest\_halt\_poll\_ns的初始值，以便vCPU polling时长的收缩和扩展。默认值为50000（单位ns）。
   * guest\_halt\_poll\_allow\_shrink: 允许每个vCPU guest\_halt\_poll\_ns收缩的开关，默认值是Y（Y表示允许收缩，N表示禁止收缩）。

   可以使用root权限，参考如下命令修改参数值。其中 *value* 表示需要设置的参数值， *configFile* 为对应的配置文件。

   ```sh
   # echo value > /sys/module/haltpoll/parameters/configFile
   ```

   例如设置全局guest\_halt\_poll\_ns为200000ns的命令如下：

   ```sh
   # echo 200000 > /sys/module/haltpoll/parameters/guest_halt_poll_ns
   ```

### Nvme磁盘直通

#### 概述

设备直通技术是一种基于硬件的虚拟化解决方案，通过该技术，虚拟机可以直接连接到指定的物理直通设备上。对于用户来说，如果需要提升虚拟机存储性能，可以采用将 Nvme 磁盘通过 PCI 直通技术直通给虚拟机的办法，从而获得更高的性能表现。

#### 操作指导

1. 使用前准备。

   * 确认 Guest OS 内安装 Nvme 磁盘供应商所提供的驱动程序，否则 Nvme 磁盘无法正常工作。
   * 确认 Host OS 开启CPU的 VT-d 和 VT-x 支持。
   * 确认 Host OS 开启内核的 IOMMU 功能。
   * 确认 Host OS 开启内核的中断重映射功能。

2. 获取 Nvme 磁盘的 PCI BDF 信息。

   在Host上通过 **lspci** 命令获取主机上pci设备的资源列表，具体命令如下所示：

   ```sh
   # lspci -vmm
   Slot: 81:00.1
   Class: Non-Volatile memory controller
   ...
   ```

   命令回显其中的 **Slot** 选项即对应了 Nvme 磁盘的 PCI BDF 号，以上方命令为例，每个值的对应关系即81-bus号，00-slot号，1-function号。

3. 挂载 PCI 直通 Nvme 磁盘至虚拟机中。

   创建虚拟机时，在其对应的 xml 配置文件中加入 PCI Nvme 磁盘直通的配置选项。具体配置文件如下所示：

   ```conf
   <hostdev mode='subsystem' type='pci' managed='yes'>
       <source>
           <address domain='0x0000' bus='0x81' slot='0x00' function='0x1' />
       </source>
   </hostdev>
   ```

   * hostdev.source.address.domain： Host OS 上 PCI 设备的 domain 号。
   * hostdev.source.address.bus： Host OS 上 PCI 设备的 bus 号。
   * hostdev.source.address.slot： Host OS 上 PCI 设备的 slot 号。
   * hostdev.source.address.function： Host OS 上 PCI 设备的 function 号。

4. 指定 Nvme 磁盘的 PCI bar 空间。

   为了进一步将 Nvme 磁盘的性能发挥到极致，需要指定直通 Nvme 磁盘在 Guest OS 内 PCI MSI-X 中断的 Bar 空间。具体配置如下：

   ```conf
   <hostdev mode='subsystem' type='pci' managed='yes'>
       <source>
           <address domain='0x0000' bus='0x01' slot='0x00' function='0x0' />
       </source>
       <alias name='ua-sm2262'/>
           <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
   </hostdev>
   <qemu:commandline>
       <qemu:arg value='-set'/>
       <qemu:arg value='device.ua-sm2262.x-msix-relocation=bar2'/>
   </qemu:commandline>
   ```

   以上xml配置将直通的 Nvme 磁盘的中断信息处理指定在第2号 Bar 上，增加该项配置可以使 Guest OS 内的 Nvme 磁盘性能达到与 Host OS 上的 Nvme 磁盘性能几乎一致。

## 安全最佳实践

### Libvirt鉴权

#### 简介

用户使用libvirt远程调用功能时，如果不进行任何鉴权校验，所有连接到主机所在网络的第三方程序都可以通过libvirt的远程调用操作虚拟机，存在安全隐患。为了提升系统安全性，openEuler提供了libvirt鉴权功能，即用户通过libvirt远程调用操作虚拟机前，必须经过身份校验，只有特定用户允许访问虚拟机，从而保护组网中的虚拟机。

#### 开启libvirt鉴权

openEuler默认关闭libvirt远程调用功能，这里给出开启libvirt远程调用和libvirt鉴权功能的方法。

1. 使用root用户登录主机。

2. 修改libvirt服务配置文件/etc/libvirt/libvirtd.conf，开启libvirt远程调用和libvirt鉴权功能。例如使用基于SASL（Simple Authentication and Security Layer）协议的TCP远程调用配置参考如下：

   ```sh
   # 传输层安全协议，0表示关闭，1表示开启，由用户自行配置
   listen_tls = 0
   # 开启基于TCP的远程调用，开启libvirt远程调用和libvirt鉴权功能必须配置为1     
   listen_tcp = 1
   # TCP远程调用所使用的协议，由用户自行配置，此处以sasl为例    
   auth_tcp = "sasl" 
   ```

3. 修改/etc/sasl2/libvirt.conf配置文件，设置SASL认证机制和sasldb数据库。

   ```sh
   # sasl协议的认证机制
   mech_list: digest-md5
   # 存放用户和用户密码的数据库
   sasldb_path: /etc/libvirt/passwd.db
   ```

4. 添加用于SASL验证的用户并设置其密码，假设用户名为userName，命令参考如下：

   ```sh
   # saslpasswd2 -a libvirt userName
   Password:
   Again (for verification):
   ```

5. 修改/etc/sysconfig/libvirtd配置文件，开启libvirt侦听选项。

   ```sh
   LIBVIRTD_ARGS="--listen"
   ```

6. 重启libvirtd服务，使修改生效。

   ```sh
   # systemctl restart libvirtd
   ```

7. 确认libvirt远程调用的鉴权功能是否生效。根据提示输入用户名和密码能够成功连接libvirt服务，说明开启成功。

   ```sh
   # virsh -c qemu+tcp://192.168.0.1/system
   Please enter your authentication name: openeuler
   Please enter your password:
   Welcome to virsh, the virtualization interactive terminal.

   Type:  'help' for help with commands
          'quit' to quit

   virsh #
   ```

#### 管理SASL

这里给出管理SASL用户的操作，请使用root用户操作。

* 查询数据库中存在的用户

  ```sh
  # sasldblistusers2 -f /etc/libvirt/passwd.db
  user@localhost.localdomain: userPassword
  ```

* 从数据库中删除用户user

  ```sh
  # saslpasswd2 -a libvirt -d user
  ```

### qemu-ga

#### 概述

qemu-ga（Qemu Guest Agent）它是运行在虚拟机内部的守护进程，它允许用户在host OS上通过QEMU提供带外通道实现对guest OS的多种管理操作：包括文件操作（open、read、write、close，seek、flush等）、内部关机、虚拟机休眠（suspend-disk、suspend-ram、suspend-hybrid），获取虚拟机内部的信息（包括内存，CPU，网卡，OS等相关信息 ）等。

在一些对安全要求较高的使用场景，为了防止虚拟机内部信息泄露，qemu-ga提供了黑名单功能，用户可以通过黑名单选择性屏蔽qemu-ga提供的部分功能。

> !\[!NOTE]说明
> qemu-ga对应的安装包是qemu-guest-agent-xx.rpm，openEuler默认不安装。xx为实际版本号。

#### 操作方法

请使用root用户按照如下操作步骤添加qemu-ga黑名单：

1. 登录虚拟机，确定qemu-guest-agent服务存在且处于运行状态：

   ```sh
   # systemctl status qemu-guest-agent |grep Active
      Active: active (running) since Wed 2018-03-28 08:17:33 CST; 9h ago
   ```

2. 查询qemu-ga哪些命令可以加入黑名单：

   ```sh
   # qemu-ga --blacklist ?
   guest-sync-delimited
   guest-sync
   guest-ping
   guest-get-time
   guest-set-time
   guest-info
   ...
   ```

3. 设置黑名单。通过修改/usr/lib/systemd/system/qemu-guest-agent.service，将需要屏蔽的命令添加到该文件的--blacklist中，不同命令使用空格分隔。例如将guest-file-open和guest-file-close命令加入黑名单的配置参考如下：

   ```conf
   [Service]
   ExecStart=-/usr/bin/qemu-ga \
         --blacklist=guest-file-open guest-file-close
   ```

4. 重启qemu-guest-agent服务：

   ```sh
   # systemctl daemon-reload
   # systemctl restart qemu-guest-agent
   ```

5. 确认虚拟机开启qemu-ga黑名单功能是否生效，即qemu-ga进程配置的参数--blacklist是否正确：

   ```sh
   # ps -ef|grep qemu-ga|grep -E "blacklist=|b="
   root       727     1  0 08:17 ?        00:00:00 /usr/bin/qemu-ga --method=virtio-serial --path=/dev/virtio-ports/org.qemu.guest_agent.0 --blacklist=guest-file-open guest-file-close guest-file-read guest-file-write guest-file-seek guest-file-flush -F/etc/qemu-ga/fsfreeze-hook
   ```

   > !\[!NOTE]说明
   > 更多关于qemu-ga的资料可以参见<https://wiki.qemu.org/Features/GuestAgent>。

### sVirt保护

#### 概述

在只使用自由访问控制DAC（Discretionary Access Control）策略的虚拟化环境中，主机上运行的恶意虚拟机可能存在攻击hypervisor或其他虚拟机的情况。为了提升虚拟化场景的安全性，openEuler使用了sVirt保护。sVirt是基于SELinux，适用于KVM虚拟化场景的安全防护技术。虚拟机本质是主机操作系统上的普通进程，sVirt机制在hypervisor将虚拟机对应的QEMU进程进行SELinux标记分类，除了使用type表示虚拟化专有进程和文件，还用不同的category（在seclevel区间）表示不同虚拟机，每个虚拟机只能访问自身相同category的文件设备，防止虚拟机访问非授权的主机或其他虚拟机的文件和设备，从而防止虚拟机逃逸，提升主机和虚拟机的安全性。

#### 开启sVirt保护

**一、使用root用户按照如下操作步骤开启主机的SELinux**

1. 登录主机。

2. 开启主机SELinux功能。
   1. 修改系统启动的grub.cfg，将selinux设置为1。

      ```conf
      selinux=1
      ```

   2. 修改/etc/selinux/config，将SELINUX模式设置为enforcing。

      ```conf
      SELINUX=enforcing
      ```

3. 重启主机。

   ```sh
   # reboot
   ```

**二、创建开启sVirt功能的虚拟机**

1. 虚拟机配置文件中添加如下配置：

   ```conf
   <seclabel type='dynamic' model='selinux' relabel='yes'/>
   ```

   或确认没有下述配置：

   ```conf
   <seclabel type='none' model='selinux'/>
   ```

2. 创建虚拟机。

   ```sh
   # virsh define openEulerVM.xml
   ```

**三、确认sVirt开启成功**

执行下述命令检查运行中的虚拟机QEMU进程是否已经启用sVirt防护，若存在"svirt\_t:s0:c"表示已经启用sVirt防护。

```sh
# ps -eZ|grep qemu |grep "svirt_t:s0:c"
system_u:system_r:svirt_t:s0:c200,c947 11359 ? 00:03:59 qemu-kvm
system_u:system_r:svirt_t:s0:c427,c670 13790 ? 19:02:07 qemu-kvm
```

### 虚拟机可信启动

#### 概述

可信启动包含度量启动和远程证明。其中虚拟化组件主要提供度量启动功能，远程证明由用户自己在虚拟机中安装相关软件（RA client）及搭建远程证明服务器（RA server)进行使能。

度量启动的两个基本要素是信任根和信任链，其基本思想是首先在计算机系统中建立一个信任根，信任根的可信性由物理安全、技术安全和管理安全共同确保，即CRTM(Core Root of Trust for Measurement)。然后建立一条信任链，从信任根开始到BIOS/BootLoader、操作系统、再到应用，一级度量认证一级，一级信任一级，最终把这种信任扩展到整个系统。上述过程看起来如同一根链条一样环环相扣，因此称之为“信任链”。

CRTM是度量启动的根，是系统启动的首个组件，没有其他代码来检查CRTM本身的完整性。所以，作为信任链的起点，必须保证它是绝对可信的信任源。因此，技术上需要将CRTM设计成一段只读或更新严格受限的代码，抵御BIOS攻击，防止远程注入恶意代码或在操作系统上层修改启动代码。通常物理主机中由CPU中的微码作为CRTM，在虚拟化环境中，一般选择vBIOS的sec部分为CRTM。

启动过程中，前一个部件度量（计算HASH值）后一个部件，然后把度量值扩展入可信存储区例如TPM的PCR中。CRTM度量BootLoader把度量值扩展到PCR中，BootLoader度量OS把度量值扩展到PCR中。

#### 配置vTPM设备，使能度量启动

**一、安装swtpm和libtpms软件**

swtpm提供了一个可集成到虚拟化环境中的TPM仿真器（TPM1.2和TPM2.0）。到目前为止，它已经集成到了QEMU中，同时也作为RunC中的原型系统。swtpm是利用libtpms来提供TPM1.2和TPM2.0的模拟功能。
目前openEuler 22.03 LTS 版本中提供了libtpms和swtpm的源，可以直接使用yum命令安装。

```sh
# yum install libtpms swtpm swtpm-devel swtpm-tools
```

**二、虚拟机配置vTPM设备**

1. 虚拟机配置文件中添加如下配置：

   ```conf
   <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    ...
       <devices>
           ...
     <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0'/>
     </tpm>
           ...
    </devices>
       ...
   </domain>
   ```

   > \[!NOTE]说明
   > 目前，openEuler 20.09 版本 AArch64 架构上的虚拟机可信启动不支持 ACPI 特性，所以虚拟机请勿配置 ACPI 特性，否则启动虚拟机后无法识别 vTPM 设备。AArch64 架构在openEuler 22.03 LTS 之前的版本，tpm model 配置为 \<tpm model='tpm-tis-device'>。

2. 创建虚拟机。

   ```sh
   # virsh define MeasuredBoot.xml
   ```

3. 启动虚拟机

   启动虚拟机前需要使用chmod命令给目录/var/lib/swtpm-localca/赋予如下权限，否则libvirt无法拉起swtpm。

```sh
   # chmod -R 777 /var/lib/swtpm-localca/
   #
   # virsh start MeasuredbootVM
```

**三、确认度量启动使能成功**

度量启动功能使能与否由vBIOS决定，目前 openEuler 22.03 LTS 版本中的vBIOS已经具备了度量启动的能力。若宿主机采用其他版本的edk2组件，请确认其是否支持度量启动功能。

使用root用户登录虚拟机，确认虚拟机中是否安装了tpm驱动、tpm2-tss协议栈及tpm2-tools工具。
openEuler 22.03 LTS 版本中默认安装了tpm驱动(tpm\_tis.ko)、tpm2-tss协议栈和tpm2-tools工具。若使用其他操作系统，可以使用如下命令检查是否安装了驱动和相关工具。

```sh
# lsmod |grep tpm
# tpm_tis          16384   0
#
# yum list installed | grep -E 'tpm2-tss|tpm2-tools'
#
# yum install tpm2-tss tpm2-tools
```

可以使用tpm2\_pcrread(低版本tpm2\_tools中使用tpm2\_pcrlist)命令列出所有的pcr值。

```sh
# tpm2_pcrread
sha1 :
  0  : fffdcae7cef57d93c5f64d1f9b7f1879275cff55
  1  : 5387ba1d17bba5fdadb77621376250c2396c5413
  2  : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236
  3  : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236
  4  : e5d40ace8bb38eb170c61682eb36a3020226d2c0
  5  : 367f6ea79688062a6df5f4737ac17b69cd37fd61
  6  : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236
  7  : 518bd167271fbb64589c61e43d8c0165861431d8
  8  : af65222affd33ff779780c51fa8077485aca46d9
  9  : 5905ec9fb508b0f30b2abf8787093f16ca608a5a
  10 : 0000000000000000000000000000000000000000
  11 : 0000000000000000000000000000000000000000
  12 : 0000000000000000000000000000000000000000
  13 : 0000000000000000000000000000000000000000
  14 : 0000000000000000000000000000000000000000
  15 : 0000000000000000000000000000000000000000
  16 : 0000000000000000000000000000000000000000
  17 : ffffffffffffffffffffffffffffffffffffffff
  18 : ffffffffffffffffffffffffffffffffffffffff
  19 : ffffffffffffffffffffffffffffffffffffffff
  20 : ffffffffffffffffffffffffffffffffffffffff
  21 : ffffffffffffffffffffffffffffffffffffffff
  22 : ffffffffffffffffffffffffffffffffffffffff
  23 : 0000000000000000000000000000000000000000
sha256 :
  0  : d020873038268904688cfe5b8ccf8b8d84c1a2892fc866847355f86f8066ea2d
  1  : 13cebccdb194dd916f2c0c41ec6832dfb15b41a9eb5229d33a25acb5ebc3f016
  2  : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969
  3  : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969
  4  : 07f9074ccd4513ef1cafd7660f9afede422b679fd8ad99d25c0659eba07cc045
  5  : ba34c80668f84407cd7f498e310cc4ac12ec6ec43ea8c93cebb2a688cf226aff
  6  : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969
  7  : 65caf8dd1e0ea7a6347b635d2b379c93b9a1351edc2afc3ecda700e534eb3068
  8  : f440af381b644231e7322babfd393808e8ebb3a692af57c0b3a5d162a6e2c118
  9  : 54c08c8ba4706273f53f90085592f7b2e4eaafb8d433295b66b78d9754145cfc
  10 : 0000000000000000000000000000000000000000000000000000000000000000
  11 : 0000000000000000000000000000000000000000000000000000000000000000
  12 : 0000000000000000000000000000000000000000000000000000000000000000
  13 : 0000000000000000000000000000000000000000000000000000000000000000
  14 : 0000000000000000000000000000000000000000000000000000000000000000
  15 : 0000000000000000000000000000000000000000000000000000000000000000
  16 : 0000000000000000000000000000000000000000000000000000000000000000
  17 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  18 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  19 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  20 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  21 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  22 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  23 : 0000000000000000000000000000000000000000000000000000000000000000
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/system_container/maximum_number_of_handles.md
---
# 最大句柄数限制

## 功能描述

系统容器支持对容器内使用文件句柄数进行限制，文件句柄包括普通文件句柄和网络套接字，启动容器时，可以通过指定--files-limit参数限制容器内打开的最大句柄数。

## 参数说明

## 约束限制

* 如果--files-limit指定的值太小，可能会导致系统容器无法通过exec执行命令，报"open too many  files"错误，所以files limit的值应该设置大一些。
* 文件句柄包括普通文件句柄和网络套接字。

## 使用示例

使用--files-limit限制容器内打开文件句柄数需要内核支持files cgroup，可以执行以下命令查看：

```sh
[root@localhost ~]# cat /proc/1/cgroup | grep files
10:files:/
```

结果显示files，说明内核支持files cgroup。

容器启动指定--files-limit参数，并检查files.limit参数是否成功写入：

```sh
[root@localhost ~]# isula run  -tid --files-limit 1024 --system-container  --external-rootfs /tmp/root-fs  empty init 01e82fcf97d4937aa1d96eb8067f9f23e4707b92de152328c3fc0ecb5f64e91d 
[root@localhost ~]# isula exec -it 01e82fcf97d4 bash 
[root@localhost ~]# cat /sys/fs/cgroup/files/files.limit 
1024 

```

可以看出，容器内文件句柄数被成功限制。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/environment_preparation.md
---
# 准备使用环境

## 准备虚拟机镜像

### 概述

虚拟机镜像是一个文件，包含了已经完成安装并且可启动操作系统的虚拟磁盘。虚拟机镜像具有不同格式，常见的有raw格式和qcow2格式。qcow2格式镜像相比raw格式，具有占用更小的空间，支持快照、Copy-On-Write、AES加密、zlib压缩等特性，但性能略逊于raw格式镜像。镜像文件的制作借助于qemu-img工具，本节以qcow2格式镜像文件为例，介绍虚拟机镜像制作过程。

### 制作镜像

制作qcow2格式镜像文件的操作步骤如下：

1. 使用root用户安装qemu-img软件包。

   ```sh
   # yum install -y qemu-img
   ```

2. 使用qemu-img工具的create命令，创建镜像文件，命令格式为：

   ```sh
   # qemu-img create -f <imgFormat> -o <fileOption> <fileName> <diskSize>
   ```

   其中，各参数含义如下：

   * *imgFormat*：镜像格式，取值为raw, qcow2等。
   * *fileOption*：文件选项，用于设置镜像文件的特性，如指定后端镜像文件，压缩，加密等特性。
   * *fileName*：文件名称。
   * *diskSize*：磁盘大小，用于指定块磁盘设备的大小，支持的单位有K、M、G、T，分别代表KiB、MiB、GiB、TiB。

   例如，创建一个磁盘设备大小为4GB、格式为qcow2的镜像文件openEuler-image.qcow2，命令和回显如下：

   ```sh
   # qemu-img create -f qcow2 openEuler-image.qcow2 4G
   Formatting 'openEuler-image.qcow2', fmt=qcow2 size=4294967296 cluster_size=65536 lazy_refcounts=off refcount_bits=16
   ```

### 修改镜像磁盘空间大小

当虚拟机需要更大的磁盘空间时，可以使用qemu-img工具，修改虚拟机镜像磁盘空间的大小，修改方法如下。

1. 查询当前虚拟机镜像磁盘空间大小，命令如下：

   ```sh
   # qemu-img info <imgFiLeName>
   ```

   例如，查询openEuler-image.qcow2镜像磁盘空间大小的命令和回显如下，说明该镜像磁盘空间大小为4GiB。

   ```sh
   # qemu-img info openEuler-image.qcow2 
   image: openEuler-image.qcow2
   file format: qcow2
   virtual size: 4.0G (4294967296 bytes)
   disk size: 196K
   cluster_size: 65536
   Format specific information:
       compat: 1.1
       lazy refcounts: false
       refcount bits: 16
       corrupt: false
   ```

2. 修改镜像磁盘空间大小，命令如下，其中imgFiLeName为镜像名称，“+”和“-”分别表示需要增加或减小的镜像磁盘空间大小，单位为K、M、G、T，代表KiB、MiB、GiB、TiB。

   ```sh
   # qemu-img resize <imgFiLeName> [+|-]<size>
   ```

   例如，将上述openEuler-image.qcow2镜像磁盘空间大小扩展到24GiB，即在原来4GiB基础上增加20GiB，命令和回显如下：

   ```sh
   # qemu-img resize openEuler-image.qcow2 +20G
   Image resized.
   ```

3. 查询修改后的镜像磁盘空间大小，确认是否修改成功，命令如下：

   ```sh
   # qemu-img info <imgFiLeName>
   ```

   例如，上述openEuler-image.qcow2镜像磁盘空间已扩展到24GiB，命令和回显如下：

   ```sh
   # qemu-img info openEuler-image.qcow2 
   image: openEuler-image.qcow2
   file format: qcow2
   virtual size: 24G (25769803776 bytes)
   disk size: 200K
   cluster_size: 65536
   Format specific information:
       compat: 1.1
       lazy refcounts: false
       refcount bits: 16
       corrupt: false
   ```

## 准备虚拟机网络

### 概述

为了使虚拟机可以与外部进行网络通信，需要为虚拟机配置网络环境。KVM虚拟化支持Linux网桥、Open vSwitch网桥等多种类型的网桥。如[图1](#fig1785384714917)所示，数据传输路径为“虚拟机 -> 虚拟网卡设备  -> Linux网桥或Open vSwitch网桥 -> 物理网卡”。创建网桥，除了为虚拟机配置虚拟网卡设备外，为主机创建网桥是连接虚拟机网络的关键。

本节给出搭建Linux网桥和Open vSwitch网桥的方法，使虚拟机连接到网络，用户可以根据情况选择搭建网桥的类型。

**图 1**  虚拟网络结构图&#x20;
![](./figures/virtual-network-structure.png)

### 搭建Linux网桥

以物理网卡eth0绑定到Linux网桥br0的操作为例，使用root用户执行如下命令搭建Linux网桥：

1. 安装bridge-utils软件包。

   Linux网桥通常通过brctl工具管理，其对应的安装包为bridge-utils，安装命令如下：

   ```sh
   # yum install -y bridge-utils
   ```

2. 创建网桥br0。

   ```sh
   # brctl addbr br0
   ```

3. 将物理网卡eth0绑定到Linux网桥。

   ```sh
   # brctl addif br0 eth0
   ```

   > !\[!NOTE]说明
   > 若在ssh远程工具中操作命令brctl addif br0 eth0，ssh远程连接会断开，需要到iBMC界面进行后续如下操作以完成虚拟化网路配置。

4. eth0与网桥连接后，不再需要IP地址，安装net-tools软件包，将eth0的IP设置为0.0.0.0。

   ```sh
   # yum install -y net-tools
   # ifconfig eth0 0.0.0.0
   ```

5. 设置br0的IP地址。
   * 如果有DHCP服务器，可以通过dhclient设置动态IP地址。

     ```sh
     # dhclient br0
     ```

   * 如果没有DHCP服务器，给br0配置静态IP，例如设置静态IP为192.168.1.2，子网掩码为255.255.255.0。

     ```sh
     # ifconfig br0 192.168.1.2 netmask 255.255.255.0
     ```

### 搭建Open vSwitch网桥

Open vSwitch网桥，具有更便捷的自动化编排能力。搭建Open vSwitch网桥需要安装网络虚拟化组件，这里介绍总体操作。

**一、安装Open vSwitch组件**

使用Open vSwitch提供虚拟网络，需要安装Open vSwitch网络虚拟化组件，使用root用户执行如下命令：

1. 安装Open vSwitch组件。

   ```sh
   # yum install -y openvswitch
   ```

2. 启动Open vSwitch服务。

   ```sh
   # systemctl start openvswitch
   ```

**二、确认安装是否成功**

确认Open vSwitch组件是否安装成功。

1. 确认openvswitch组件是否安装成功。若安装成功，可以看到软件包相关信息，命令和回显如下：

   ```sh
   $ rpm -qi openvswitch
   Name        : openvswitch
   Version     : 2.12.4
   Release     : 3.oe2203SP4
   Architecture: x86_64
   Install Date: Tue 09 May 2023 10:58:53 AM CST
   Group       : Unspecified
   Size        : 7920016
   License     : ASL 2.0 and ISC
   Signature   : RSA/SHA256, Wed 19 Apr 2023 09:40:31 AM CST, Key ID 007fb747fb37bc6f
   Source RPM  : openvswitch-2.12.4-3.oe2203SP4.src.rpm
   Build Date  : Wed 19 Apr 2023 09:39:49 AM CST
   Build Host  : dc-64g.compass-ci
   Packager    : http://openeuler.org
   URL         : http://www.openvswitch.org/
   Summary     : Production Quality, Multilayer Open Virtual Switch
   Description :
   Open vSwitch is a production quality, multilayer virtual switch licensed under
   the open source Apache 2.0 license.
   ```

2. 查看Open vSwitch服务是否启动成功。若服务处于“Active”状态，说明服务启动成功，可以正常使用Open vSwitch提供的命令行工具，命令和回显如下：

   ```sh
   $ systemctl status openvswitch
   ● openvswitch.service - LSB: Open vSwitch switch
      Loaded: loaded (/etc/rc.d/init.d/openvswitch; generated)
      Active: active (running) since Sat 2019-08-17 09:47:14 CST; 4min 39s ago
        Docs: man:systemd-sysv-generator(8)
     Process: 54554 ExecStart=/etc/rc.d/init.d/openvswitch start (code=exited, status=0/SUCCESS)
       Tasks: 4 (limit: 9830)
      Memory: 22.0M
      CGroup: /system.slice/openvswitch.service
              ├─54580 ovsdb-server: monitoring pid 54581 (healthy)
              ├─54581 ovsdb-server /etc/openvswitch/conf.db -vconsole:emer -vsyslog:err -vfile:info --remote=punix:/var/run/openvswitch/db.sock --private-key=db:Open_vSwitch,SSL,private_key --certificate>
              ├─54602 ovs-vswitchd: monitoring pid 54603 (healthy)
              └─54603 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/open>
   ```

**三、搭建Open vSwitch网桥**

以创建Open vSwitch一层网桥br0为例，介绍搭建方法，使用root用户执行如下命令：

1. 创建Open vSwitch网桥br0。

   ```sh
   # ovs-vsctl add-br br0
   ```

2. 将物理网卡eth0添加到br0。

   ```sh
   # ovs-vsctl add-port br0 eth0
   ```

3. eth0与网桥连接后，不再需要IP地址，将eth0的IP设置为0.0.0.0。

   ```sh
   # ifconfig eth0 0.0.0.0
   ```

4. 为OVS网桥br0分配IP。
   * 如果有DHCP服务器，可以通过dhclient设置动态IP地址。

     ```sh
     # dhclient br0
     ```

   * 如果没有DHCP服务器，给br0配置静态IP，例如192.168.1.2。

     ```sh
     # ifconfig br0 192.168.1.2
     ```

## 准备引导固件

### 概述

针对不同的架构，引导的方式有所差异。x86支持UEFI（Unified Extensible Firmware Interface）和Legacy方式启动，AArch64仅支持UEFI方式启动。openEuler默认已安装BIOS启动对应的引导文件，不需要用户额外操作。所以这里仅介绍UEFI启动方式的安装方法。

统一的可扩展固件接口UEFI是一种全新类型的接口标准，用于开机自检、引导操作系统的启动，是传统BIOS的一种替代方案。EDK II是一套实现了UEFI标准的开源代码，在虚拟化场景中，通常利用EDK II工具集，通过UEFI的方式启动虚拟机。使用EDK II工具需要在虚拟机启动之前安装对应的软件包 ，本节介绍EDK II的安装方法。

### 安装方法

如果使用UEFI方式引导，需要安装工具集EDK II，AArch64架构对应的安装包为edk2-aarch64，x86架构对应的安装包为edk2-ovmf。这里以AArch64架构为例，给出具体的安装方法，x86架构仅需将edk2-aarch64替换为edk2-ovmf。

1. 安装edk软件包，使用root用户执行如下命令：

   在AArch64架构下edk2的包名为edk2-aarch64

   ```sh
   # yum install -y edk2-aarch64
   ```

   在x86\_64架构下edk2的包名为edk2-ovmf

   ```sh
   # yum install -y edk2-ovmf
   ```

2. 查询edk软件是否安装成功，命令如下：

   在AArch64架构下查询如下：

   ```sh
   $ rpm -qi edk2-aarch64
   ```

   若edk软件安装成功，回显示例如下：

   ```text
   Name        : edk2-aarch64
   Version     : 202011
   Release     : 11.oe2203SP4
   Architecture: noarch
   Install Date: Tue 09 May 2023 11:28:22 AM CST
   Group       : Unspecified
   ```

   在x86\_64架构下查询如下：

   ```sh
   $ rpm -qi edk2-ovmf
   ```

   若edk软件安装成功，回显类似如下：

   ```text
   Name        : edk2-ovmf
   Version     : 202011
   Release     : 11.oe2203SP4
   Architecture: noarch
   Install Date: Tue 09 May 2023 11:06:06 AM CST
   ```

## 非root用户配置

### 概述

openEuler虚拟化使用virsh管理虚拟机。如果希望在非root用户使用virsh命令管理虚拟机，在使用之前需要进行相关配置，这里给出配置指导。

### 操作指导

允许非root用户使用virsh命令管理虚拟机的配置操作如下，以下命令中的userName请改为实际的非root用户名称：

1. 使用root用户登录主机。

2. 将非root用户添加到libvirt用户组。

   ```sh
   # usermod -a -G libvirt userName
   ```

3. 切换到非root用户。

   ```sh
   # su userName
   ```

4. 配置非root用户的环境变量。使用vim打开~/.bashrc文件。

   ```sh
   $ vim ~/.bashrc
   ```

   并在末尾加上如下内容后保存。

   ```sh
   export LIBVIRT_DEFAULT_URI="qemu:///system"
   ```

   执行如下命令，使配置生效。

   ```sh
   $ source ~/.bashrc
   ```

5. 在虚拟机XML配置文件中的domain根元素中添加如下内容，使qemu-kvm进程可以访问磁盘镜像文件。

   ```conf
   <seclabel type='dynamic' model='dac' relabel='yes'/>
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/prepare_env.md
---
# 准备环境

## 使用说明

* StratoVirt仅支持运行于x86\_64和AArch64处理器架构下并启动相同架构的Linux虚拟机。
* 建议在 openEuler 22.03 LTS SP4 版本编译、调测和部署该版本 StratoVirt。
* StratoVirt支持以非root权限运行。

## 环境要求

运行StratoVirt需要具备如下环境：

* /dev/vhost-vsock设备（用于实现mmio）
* nmap工具
* Kernel镜像和rootfs镜像

## 准备设备和工具

* StratoVirt运行需要实现mmio设备，所以运行之前确保存在设备`/dev/vhost-vsock`

  查看该设备是否存在：

  ```sh
  $ ls /dev/vhost-vsock
  /dev/vhost-vsock
  ```

  若该设备不存在，请执行如下命令生成/dev/vhost-vsock设备。

  ```sh
  $ modprobe vhost_vsock
  ```

* 为了能够使用QMP命令，需要安装nmap工具，在配置yum源的前提下，可执行如下命令安装nmap。

  ```sh
  # yum install nmap
  ```

## 准备镜像

### 制作kernel镜像

当前版本的StratoVirt仅支持x86\_64和AArch64平台的PE格式内核镜像。此格式内核映像可通过以下方法生成。

1. 获取openEuler的kernel源代码，参考命令如下：

   ```sh
   $ git clone https://atomgit.com/openeuler/kernel.git
   $ cd kernel
   ```

2. 查看并切换kernel的版本到openEuler-22.03-LTS-SP4，参考命令如下：

   ```sh
   $ git checkout openEuler-22.03-LTS-SP4
   ```

3. 配置并编译Linux kernel。目前有两种方式可以生成配置文件：1. 使用推荐配置（[获取配置文件](https://atomgit.com/openeuler/stratovirt/tree/master/docs/kernel_config)），将指定版本的推荐文件复制到kernel路径下并重命名为`.config`， 并执行命令`make olddefconfig`更新到最新的默认配置（否则后续编译可能有选项需要手动选择）。2. 通过以下命令进行交互，根据提示完成kernel配置，可能会提示缺少指定依赖，按照提示使用`yum install`命令进行安装。

   ```sh
   $ make menuconfig
   ```

4. 使用下面的命令制作并转换kernel镜像为PE格式，转化后的镜像为vmlinux.bin。

   ```sh
   $ make -j vmlinux && objcopy -O binary vmlinux vmlinux.bin
   ```

5. 如果想在x86平台使用bzImzge格式的kernel，可以使用如下命令进行编译。

   ```sh
   $ make -j bzImage
   ```

## 制作rootfs镜像

rootfs镜像是一种文件系统镜像，在StratoVirt启动时可以装载带有init的ext4格式的镜像。下面是制作ext4 rootfs镜像的简单方法。

1. 准备一个大小合适的文件（例如在/home中创建10GiB空间大小的文件）。

   ```sh
   $ cd /home
   $ dd if=/dev/zero of=./rootfs.ext4 bs=1G count=10
   ```

2. 在此文件上创建空的ext4文件系统。

   ```sh
   $ mkfs.ext4 ./rootfs.ext4
   ```

3. 挂载文件镜像。创建/mnt/rootfs，使用root权限，将rootfs.ext4挂载到/mnt/rootfs目录。

   ```sh
   $ mkdir /mnt/rootfs
   # 返回刚刚创建文件系统的目录（如/home）
   $ cd /home
   $ sudo mount ./rootfs.ext4 /mnt/rootfs && cd /mnt/rootfs
   ```

4. 获取对应处理器架构的最新alpine-mini rootfs。

   * 对于AArch64处理器架构，从[alpine](http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/)网站获取最新alpine-mini rootfs，例如：alpine-minirootfs-3.16.0-aarch64.tar.gz ，参考命令如下：

   ```sh
   $ wget http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/aarch64/alpine-minirootfs-3.16.0-aarch64.tar.gz
   $ tar -zxvf alpine-minirootfs-3.16.0-aarch64.tar.gz
   $ rm alpine-minirootfs-3.16.0-aarch64.tar.gz
   ```

   * 对于x86\_64处理器架构，从[alpine](http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/)网站获取指定架构最新alpine-mini rootfs，例如：alpine-minirootfs-3.16.0-x86\_64.tar.gz，参考命令如下：

   ```sh
   $ wget http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/x86_64/alpine-minirootfs-3.16.0-x86_64.tar.gz
   $ tar -zxvf alpine-minirootfs-3.16.0-x86_64.tar.gz
   $ rm alpine-minirootfs-3.16.0-x86_64.tar.gz
   ```

5. 为ext4文件镜像制作一个简单的/sbin/init，参考命令如下：

   ```sh
   $ rm sbin/init; touch sbin/init && cat > sbin/init <<EOF
   #! /bin/sh
   mount -t devtmpfs dev /dev
   mount -t proc proc /proc
   mount -t sysfs sysfs /sys
   ip link set up dev lo

   exec /sbin/getty -n -l /bin/sh 115200 /dev/ttyS0
   poweroff -f
   EOF

   sudo chmod +x sbin/init
   ```

6. 卸载rootfs镜像。

   ```sh
   $ cd /home; umount /mnt/rootfs
   ```

   至此， rootfs制作成功，已可使用ext4 rootfs镜像文件rootfs.ext4，该文件在/home目录下。

## 获取标准启动所需固件

固件 (firmware) 是指设备内部保存的设备驱动程序。操作系统只有通过固件才能按照标准启动的方式进行启动。 StratoVirt 当前只支持在 x86\_64 和 aarch64 架构下按照 UEFI (Unified Extensible Firmware Interface) 接口进行标准启动。

EDK II 是实现了 UEFI 标准的开源软件，StratoVirt 使用 EDK II 作为标准启动的固件。因此需要获取EDK II的固件文件。可以通过 yum 命令来进行 EDK II 固件文件的安装，具体命令如下。

x86\_64 架构上运行以下命令：

```shell
$ sudo yum install -y edk2-ovmf
```

aarch64 架构上运行以下命令：

```shell
$ sudo yum install -y edk2-aarch64
```

EDK II 的固件包括两个文件：一个文件用于保存可执行代码，另一个文件用于保存启动配置信息。安装成功之后，在 x86\_64 架构上，固件文件 `OVMF_CODE.fd` 与固件配置文件 `OVMF_VARS.fd` 会保存在 `/usr/share/edk2/ovmf` 路径下；在 aarch64 架构上, 固件文件 `QEMU_EFI-pflash.raw` 和固件配置文件 `vars-template-pflash.raw` 则是保存在 `/usr/share/edk2/aarch64` 路径下。

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/preparing_vms.md
---
# 准备虚拟机

本章介绍使用 virt  manager 安装虚拟机的方法，如果您已经准备好虚拟机，可以跳过本章节。

## 安装依赖工具

安装虚拟机，会依赖相关工具，安装依赖并使能 libvirtd 服务的参考命令如下（如果需要代理，请先配置代理）：

```bash
$ dnf install virt-install virt-manager libvirt-daemon-qemu edk2-aarch64.noarch virt-viewer
$ systemctl start libvirtd
$ systemctl enable libvirtd
```

## 准备虚拟机磁盘文件

```bash
$ dnf install -y qemu-img
$ virsh pool-define-as vmPool --type dir --target /mnt/vm/images/
$ virsh pool-build vmPool
$ virsh pool-start vmPool
$ virsh pool-autostart  vmPool
$ virsh vol-create-as --pool vmPool --name master0.img --capacity 200G --allocation 1G --format qcow2
$ virsh vol-create-as --pool vmPool --name master1.img --capacity 200G --allocation 1G --format qcow2
$ virsh vol-create-as --pool vmPool --name master2.img --capacity 200G --allocation 1G --format qcow2
$ virsh vol-create-as --pool vmPool --name node1.img --capacity 300G --allocation 1G --format qcow2
$ virsh vol-create-as --pool vmPool --name node2.img --capacity 300G --allocation 1G --format qcow2
$ virsh vol-create-as --pool vmPool --name node3.img --capacity 300G --allocation 1G --format qcow2
```

## 打开 VNC 防火墙端口

**方法一**

1. 查询端口

   ```shell
   $ netstat -lntup | grep qemu-kvm
   ```

2. 打开 VNC 的防火墙端口。假设端口从 5900 开始，参考命令如下：

   ```shell
   $ firewall-cmd --zone=public --add-port=5900/tcp
   $ firewall-cmd --zone=public --add-port=5901/tcp
   $ firewall-cmd --zone=public --add-port=5902/tcp
   $ firewall-cmd --zone=public --add-port=5903/tcp
   $ firewall-cmd --zone=public --add-port=5904/tcp
   $ firewall-cmd --zone=public --add-port=5905/tcp
   ```

**方法二**

直接关闭防火墙

```shell
$ systemctl stop firewalld
```

## 准备虚拟机配置文件

创建虚拟机需要虚拟机配置文件。假设配置文件为 master.xml ，以虚拟机 hostname 为 k8smaster0 的节点为例，参考配置如下：

```bash
 cat master.xml

<domain type='kvm'>
    <name>k8smaster0</name>
    <memory unit='GiB'>8</memory>
    <vcpu>8</vcpu>
    <os>
 <type arch='aarch64' machine='virt'>hvm</type>
 <loader readonly='yes' type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader>
 <nvram>/var/lib/libvirt/qemu/nvram/k8smaster0.fd</nvram>
    </os>
    <features>
 <acpi/>
 <gic version='3'/>
    </features>
    <cpu mode='host-passthrough'>
        <topology sockets='2' cores='4' threads='1'/>
    </cpu>
    <iothreads>1</iothreads>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>restart</on_crash>
    <devices>
 <emulator>/usr/libexec/qemu-kvm</emulator>
 <disk type='file' device='disk'>
     <driver name='qemu' type='qcow2' iothread="1"/>
     <source file='/mnt/vm/images/master0.img'/>
     <target dev='vda' bus='virtio'/>
     <boot order='1'/>
 </disk>
 <disk type='file' device='cdrom'>
     <driver name='qemu' type='raw'/>
     <source file='/mnt/openEuler-{version}-everything-aarch64-dvd.iso'/>
     <readonly/>
     <target dev='sdb' bus='scsi'/>
     <boot order='2'/>
 </disk>
        <interface type='network'>
           <mac address='52:54:00:00:00:80'/>
           <source network='default'/>
           <model type='virtio'/>
        </interface>
 <console type='pty'/>
        <video>
           <model type='virtio'/>
        </video>
        <controller type='scsi' index='0' model='virtio-scsi'/>
 <controller type='usb' model='ehci'/>
 <input type='tablet' bus='usb'/>
 <input type='keyboard' bus='usb'/>
 <graphics type='vnc' listen='0.0.0.0'/>
    </devices>
    <seclabel type='dynamic' model='dac' relabel='yes'/>
</domain>
```

由于虚拟机相关配置必须唯一，新增虚拟机需要适配修改如下内容，保证虚拟机的唯一性：

* name：虚拟机 hostname，建议尽量小写。例中为 `k8smaster0`
* nvram：nvram的句柄文件路径，需要全局唯一。例中为  `/var/lib/libvirt/qemu/nvram/k8smaster0.fd`
* disk 的 source file：虚拟机磁盘文件路径。例中为  `/mnt/vm/images/master0.img`
* interface 的 mac address：interface 的 mac 地址。例中为 `52:54:00:00:00:80`

## 安装虚拟机

1. 创建并启动虚拟机

   ```shell
   $ virsh define master.xml
   $ virsh start k8smaster0
   ```

2. 获取虚拟机的 VNC 端口号

   ```shell
   $ virsh vncdisplay k8smaster0
   ```

3. 使用虚拟机连接工具，例如 VNC Viewer 远程连接虚拟机，并根据提示依次选择配置，完成系统安装

4. 设置虚拟机 hostname，例如设置为 k8smaster0

   ```shell
   $ hostnamectl set-hostname k8smaster0
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/preparing_certificates.md
---
# 准备证书

**声明：本文使用的证书为自签名，不能用于商用环境**

部署集群前，需要生成集群各组件之间通信所需的证书。本文使用开源 CFSSL 作为验证部署工具，以便用户了解证书的配置和集群组件之间证书的关联关系。用户可以根据实际情况选择合适的工具，例如 OpenSSL 。

## 编译安装 CFSSL

编译安装 CFSSL 的参考命令如下（需要互联网下载权限，需要配置代理的请先完成配置，需要配置 go语言环境）:

```bash
$ wget --no-check-certificate  https://github.com/cloudflare/cfssl/archive/v1.5.0.tar.gz
$ tar -zxf v1.5.0.tar.gz
$ cd cfssl-1.5.0/
$ make -j6
# cp bin/* /usr/local/bin/
```

## 生成根证书

编写 CA 配置文件，例如 ca-config.json：

```bash
$ cat ca-config.json | jq
{
  "signing": {
    "default": {
      "expiry": "8760h"
    },
    "profiles": {
      "kubernetes": {
        "usages": [
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ],
        "expiry": "8760h"
      }
    }
  }
}
```

编写 CA CSR 文件，例如 ca-csr.json：

```bash
$ cat ca-csr.json  | jq
{
  "CN": "Kubernetes",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "openEuler",
      "OU": "WWW",
      "ST": "BinJiang"
    }
  ]
}
```

生成 CA 证书和密钥：

```bash
$ cfssl gencert -initca ca-csr.json | cfssljson -bare ca
```

得到如下证书：

```bash
ca.csr  ca-key.pem  ca.pem
```

## 生成 admin 帐户证书

admin 是 K8S 用于系统管理的一个帐户，编写 admin 帐户的 CSR 配置，例如 admin-csr.json：

```bash
cat admin-csr.json | jq
{
  "CN": "admin",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:masters",
      "OU": "Containerum",
      "ST": "BinJiang"
    }
  ]
}
```

生成证书：

```bash
$ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin
```

结果如下：

```bash
admin.csr  admin-key.pem  admin.pem
```

## 生成 service-account 帐户证书

编写 service-account 帐户的 CSR 配置文件，例如 service-account-csr.json：

```bash
cat service-account-csr.json | jq
{
  "CN": "service-accounts",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "Kubernetes",
      "OU": "openEuler k8s install",
      "ST": "BinJiang"
    }
  ]
}
```

生成证书：

```bash
$ cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes service-account-csr.json | cfssljson -bare service-account
```

结果如下：

```bash
service-account.csr  service-account-key.pem  service-account.pem
```

## 生成 kube-controller-manager 组件证书

编写 kube-controller-manager 的 CSR 配置：

```bash
{
  "CN": "system:kube-controller-manager",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:kube-controller-manager",
      "OU": "openEuler k8s kcm",
      "ST": "BinJiang"
    }
  ]
}
```

生成证书：

```bash
$ cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager
```

结果如下：

```bash
kube-controller-manager.csr  kube-controller-manager-key.pem  kube-controller-manager.pem
```

## 生成 kube-proxy 证书

编写 kube-proxy 的 CSR 配置：

```bash
{
  "CN": "system:kube-proxy",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:node-proxier",
      "OU": "openEuler k8s kube proxy",
      "ST": "BinJiang"
    }
  ]
}
```

生成证书：

```bash
$ cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
```

结果如下：

```bash
kube-proxy.csr  kube-proxy-key.pem  kube-proxy.pem
```

## 生成 kube-scheduler 证书

编写 kube-scheduler 的 CSR 配置：

```bash
{
  "CN": "system:kube-scheduler",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:kube-scheduler",
      "OU": "openEuler k8s kube scheduler",
      "ST": "BinJiang"
    }
  ]
}
```

生成证书：

```bash
$ cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler
```

结果如下：

```bash
kube-scheduler.csr  kube-scheduler-key.pem  kube-scheduler.pem
```

## 生成 kubelet 证书

由于证书涉及到 kubelet 所在机器的 hostname 和 IP 地址信息，因此每个 node 节点配置不尽相同，所以编写脚本完成，生成脚本如下：

```bash
$ cat node_csr_gen.bash

#!/bin/bash

nodes=(k8snode1 k8snode2 k8snode3)
IPs=("192.168.122.157" "192.168.122.158" "192.168.122.159")

for i in "${!nodes[@]}"; do

cat > "${nodes[$i]}-csr.json" <<EOF
{
  "CN": "system:node:${nodes[$i]}",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:nodes",
      "OU": "openEuler k8s kubelet",
      "ST": "BinJiang"
    }
  ]
}
EOF

 # generate ca
 echo "generate: ${nodes[$i]} ${IPs[$i]}"
 cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -hostname=${nodes[$i]},${IPs[$i]} -profile=kubernetes ${nodes[$i]}-csr.json | cfssljson -bare ${nodes[$i]}
done
```

说明：如果节点存在多个 IP 或者其他别名，-hostname 可以增加其他的 IP 或者 hostname

结果如下：

```bash
k8snode1.csr       k8snode1.pem       k8snode2-key.pem  k8snode3-csr.json
k8snode1-csr.json  k8snode2.csr       k8snode2.pem      k8snode3-key.pem
k8snode1-key.pem   k8snode2-csr.json  k8snode3.csr      k8snode3.pem
```

CSR 配置信息，以 k8snode1 为例如下：

```bash
$ cat k8snode1-csr.json
{
  "CN": "system:node:k8snode1",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "system:nodes",
      "OU": "openEuler k8s kubelet",
      "ST": "BinJiang"
    }
  ]
}
```

注意：由于每个 node 所属的帐户组为 `system:node`，因此 CSR 的 CN 字段都为 `system:node` 加上`hostname`。

## 生成 kube-apiserver 证书

编写 kube api server 的 CSR 配置文件：

```bash
$ cat kubernetes-csr.json | jq
{
  "CN": "kubernetes",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "Kubernetes",
      "OU": "openEuler k8s kube api server",
      "ST": "BinJiang"
    }
  ]
}
```

生成证书和密钥：

```bash
cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -hostname=10.32.0.1,192.168.122.154,192.168.122.155,192.168.122.156,127.0.0.1,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.svc.cluster.local -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes
```

结果如下：

```bash
kubernetes.csr  kubernetes-key.pem  kubernetes.pem
```

*说明：10.32.0.1 是内部 services 使用的 IP 地址区间，可以设置为其他值，后面启动 apiserver 服务时，会设置该参数。*

## 生成 etcd 证书（可选）

部署 etcd 有两种方式：

* 在每个 api-server 对应的机器都启动一个 etcd 服务
* 独立部署一个 etcd 集群服务

如果是和 api-server 一起部署，那么直接使用上面生成的 `kubernetes-key.pem` 和  `kubernetes.pem`  证书即可。

如果是独立的etcd集群，那么需要创建证书如下：

编写 etcd 的 CSR 配置：

```bash
cat etcd-csr.json | jq
{
  "CN": "ETCD",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "HangZhou",
      "O": "ETCD",
      "OU": "openEuler k8s etcd",
      "ST": "BinJiang"
    }
  ]
}
```

生成证书：

```bash
$ cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -hostname=192.168.122.154,192.168.122.155,192.168.122.156,127.0.0.1 -profile=kubernetes etcd-csr.json | cfssljson -bare etcd
```

*说明：假设 etcd 集群的 IP地址是 192.168.122.154,192.168.122.155,192.168.122.156*

结果如下：

```bash
etcd.csr  etcd-key.pem  etcd.pem
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/development/distributed/overview.md
---
# 分布式中间件用户指南

本章节介绍了openEuler上多设备通信连接的软总线应用开发使用指南，以及基于软总线生态衍生的分布式文件的相关介绍。

通过这些文章可以了解到openEuler边缘服务器、嵌入式设备级OpenHarmony设备之间原生的发现连接方式，以及相关的拓展应用。

使用人员需要具备以下经验和技能：

* 熟悉Linux基本操作。

* 了解OpenHarmony及openEuler的相关开发测试流程。

* 熟悉IPC及PRC的相关背景知识，对分布式架构中的数据同步相关知识有一定的了解。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/distributed/distributed_data_management.md
---
# 分布式数据管理

分布式数据管理基于分布式软总线，实现了应用程序数据和用户数据的分布式管理。用户数据不再与单一物理设备绑定，业务逻辑与数据存储分离，应用跨设备运行时数据无缝衔接。

分布式数据管理由上游 `OpenHarmony 3.2 Release` 版本移植而来，目前包含四个组件：

| 组件 | 对应openEuler软件包 | 功能介绍 |
| --- | --- | --- |
| 分布式数据服务（Distributed Data Service） | distributeddatamgr\_datamgr\_service | 提供不同设备间数据库数据分布式的能力 |
| 键值对数据库（KV store） | distributeddatamgr\_kv\_store | 为设备应用提供键值对数据管理能力 |
| 关系型数据库（Relational store） | distributeddatamgr\_relational\_store | 基于关系模型来管理数据的数据库 |
| 分布式数据对象部件（Data object） | distributeddatamgr\_data\_object | 具备多设备数据同步的面向对象的内存数据管理框架 |

更多分布式数据管理介绍，可参考[上游 OpenHarmony 文档 “数据管理” 相关章节](https://gitcode.com/openharmony/docs/blob/master/zh-cn/application-dev/database/data-mgmt-overview.md)。

## 安装

分布式数据管理在 openEuler-22.03-LTS-SP4 版本已经默认集成，直接安装即可：

```shell
dnf install distributeddatamgr_kv_store distributeddatamgr_relational_store distributeddatamgr_datamgr_service distributeddatamgr_data_object
```

## 启动服务

1. 分布式数据服务可以通过 start\_services.sh 脚本来启动。

   ```shell
   /system/bin/start_services.sh datamgr
   ```

2. 可以通过 ps 命令查看分布式数据管理服务是否拉起。

   ```shell
   ps -ef | grep distributeddata
   ```

## 使用分布式数据功能

1. 新建demo运行需要的/data目录。因为OpenHarmony下app运行有指定的目录，需要在openEuler下先新增。

   ```shell
   mkdir -p /data/app/el0/0/database/com.example.distributed.rdb/rdb
   mkdir -p /data/app/el1/0/database/distributeddata/kvdb
   mkdir -p /data/service/el1/public/database/distributeddata/meta
   mkdir -p /data/service/el1/public/database/distributeddata/kvdb
   mkdir -p /data/service/el1/public/database/distributeddata/meta/backup
   ```

2. 编写三个数据库`kv_store/data_object/relational_store`的客户端程序，使用分布式数据功能。

   客户端程序编写可参考[仓库](https://gitee.com/heppen/distributed-data-files)每个数据库的 demo 源代码，demo 在这个仓库每个数据库同名目录下的 demo目录下，如`kv_store/demo`，可使用对应目录下的 build.sh 脚本进行编译。

   > **注意**
   >
   > demo 指定的数据 db 路径等，请最好保持和所给示例 demo 的路径一致，否则可能会存在路径不存在或者权限不足问题。

3. 暂停分布式管理服务 datamgr\_service。

   ```shell
   ./stop_services.sh all # 停止所有服务
   ./stop_services.sh datamgr # 停止datamgr一个服务
   ```

## 常见问题

* 启动各个服务日志一直在报错 `Binder Driver died`。

  原因：说明系统未开启 binder，可以查看 `/dev/binder` 文件是否存在，如果不存在则说明未开启 binder。

  解决方法：启动binder功能，参考[communication\_ipc仓库说明](https://atomgit.com/src-openeuler/communication_ipc/blob/46d83ed1462e521ce356aec48ef980dbf84cff80/README.md)开启binder功能。

* `softbus_server` 服务未成功起来，报错 `GetNetworkIfIp ifName:eth0 fail`。

  原因：使用命令 `ip a` 查看当前系统的网卡名称，查看是否有 `eth0` 有线网卡名。 因为 `softbus_server` 是通过 `eth0` 这个有线网卡名来获取 ip 等信息，如果没有 `eth0` 网卡则无法启动 `softbus_server`。

  解决方法一：修改网卡名称为 `eth0`。

  解决方法二：修改 `softbus_server` 源码，将依赖的有线网卡名称改成当前系统的网卡名。

## 参考

[分布式文件系统架构](https://docs.openeuler.org/zh/docs/22.03_LTS_SP4/docs/Distributed/%E5%88%86%E5%B8%83%E5%BC%8F%E6%96%87%E4%BB%B6%E7%B3%BB%E7%BB%9F%E6%9E%B6%E6%9E%84.html)

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/nestos/nestos/feature_description.md
---
# 功能特性描述

## 容器技术

NestOS通过容器化 (containerized) 的运算环境向应用程序提供运算资源，应用程序之间共享系统内核和资源，但是彼此之间又互不可见。这意味着应用程序将不会再被直接安装到操作系统中，而是通过 Docker 运行在容器中。大大降低了操作系统、应用程序及运行环境之间的耦合度。相对于传统的应用程序部署方式而言，在NestOS 集群中部署应用程序更加灵活便捷，应用程序运行环境之间的干扰更少，而且操作系统自身的维护也更加容易。

## rpm-ostree

### 系统更新

rpm-ostree是一种镜像/包混合系统，可以看成是rpm和OSTree的合体。一方面它提供了基于rpm的软件包安装管理方式，另一方面它提供了基于OSTree的操作系统更新升级。rpm-ostree将这两种操作都视为对操作系统的更新，每次对系统的更新都向rpm-ostree提交“Transaction-事务”，从而确保更新全部成功或全部失败，允许在更新系统后回滚到更新前的状态。

rpm-ostree在更新操作系统的时候会有2个bootable区域，分别为更新前和更新后的，对系统的更新升级只有在重启操作系统后才生效。如果软件安装或升级出现问题，通过rpm-ostree回滚会使NestOS系统返回到先前的状态。我们可以查看NestOS的“/ostree/”和“/boot/”目录，它们是OSTree Repository环境并且可以观察到boot使用哪个OSTree。

### 文件系统

在rpm-ostree的文件系统布局中，只有/etc和/var是唯一可写的目录，/var中的任何数据不会被触及，而是在升级过程中共享。在系统升级的过程中采用新的默认值/etc，并将更改添加到顶部。这意味着升级将会接收/etc中新的默认文件，这是一个非常关键的特性。

OSTree旨在可以并行安装多个独立操作系统的版本。OSTree需要新建一个顶级目录“ostree”。然而实际上，OSTree支持在现有操作系统或发行版本的根目录（/）上，并行安装其他操作系统版本。每台客户机和每组部署都存储在 /ostree/deploy/STATEROOT/CHECKSUM 上，而且还有一个OSTree存储库存储在 /ostree/repo 中。每个部署主要由一组指向存储库的硬链接组成，这意味着每个版本都进行了重复数据的删除并且升级过程中只消耗了与新文件成比例的磁盘空间加上一些恒定的开销。

OSTree模型强调的是OS只读内容保存在 /usr 中，它附带了用于创建Linux只读安装以防止无意损坏的代码，对于给定的操作系统，每个部署之间都有一个 /var 共享的可供读写的目录。OSTree核心代码不触及该目录的内容，如何管理和升级状态取决于每个操作系统中的代码。

### 系统扩展

出于安全性和可维护性的考虑，NestOS让基础镜像尽可能保持小巧和精简。但是在某些情况下，需要向基本操作系统本身添加软件，例如驱动软件，VPN等等，因为它们比较难容器化。这些包拓展了操作系统的功能，为此，rpm-ostree将这些包视为拓展，而不是仅仅在用户运行时提供。也就是说，目前NestOS对于实际安装哪些包没有限制，默认情况下，软件包是从openEuler仓库下载的。

要对软件包进行分层，需要重新编写一个systemd单元来执行rpm-ostree命令安装所需要的包，所做的更改应⽤于新部署，重新启动才能生效。

## nestos-installer

nestos-installer是一个帮助安装NestOS的程序，它可以执行以下操作：

（1）安装操作系统到一个目标磁盘，可使用Ignition和首次引导内核参数对其进行自定义(nestos-installer install)

（2）下载并验证各种云平台、虚拟化或者裸机平台的操作系统映像(nestos-installer download)

（3）列出可供下载的NestOS镜像(nestos-installer list-stream)

（4）在ISO中嵌入一个Ignition配置，以自定义地从中启动操作系统(nestos-installer iso ignition)

（5）将Ignition配置包装在initd映像中，该映像可以被加入到PXE initramfs中以自定义从中启动的操作系统(nestos-installer pxe ignition)

## zincati

Zincati是NestOS自动更新的代理，它作为Cincinnati和rpm-ostree的客户端，负责自动更新/重启机器。Zincati有如下特点：

（1）支持自动更新代理，支持分阶段推出

（2）通过toml配置文件支持运行时自定义，用户自定义配置文件可覆盖默认配置

（3）多种更新策略

（4）通过维护窗口每周在特定时间范围内进行更新的策略

（5）收集和导出本地运行的zincati内部指标，可提供给Prometheus以减轻跨大量节点的监控任务

（6）具有可配置优先级的日志记录

（7）通过Cincinnati协议支持复杂的更新图

（8）通过外部锁管理器支持集群范围的重启编排

## 系统初始化（Ignition）

Ignition 是一个与分发无关的配置实用程序，不仅用于安装，还读取配置文件（JSON 格式）并根据该配置初始化NestOS系统。可配置的组件包括存储和文件系统、systemd单元和用户。

Ignition仅在系统第一次引导期间运行一次（在initramfs中）。因为 Ignition 在启动过程的早期运行，所以它可以在用户空间开始启动之前重新分区磁盘、格式化文件系统、创建用户和写入文件。 因此，systemd 服务在 systemd 启动时已经写入磁盘，从而加快了启动速度。

（1）Ignition 仅在第一次启动时运行

Ignition定位是资源发放工具，而不是配置管理工具。Ignition面向稳定不变的基础设施，如果用户对机器作了修改，用户需要丢弃旧节点，重新发放新机器。

（2）Ignition不是在任何情况下都可以完成配置

Ignition 执行它需要的操作，使系统与 Ignition 配置中描述的状态相匹配。 如果由于某种原因 Ignition 无法提供配置要求的确切机器，Ignition 会阻止机器成功启动。

（3）Ignition只是声明性配置

Ignition配置只描述了系统的状态，没有列出 Ignition 应该采取的一系列步骤。

Ignition 配置不允许用户提供任意逻辑（包括 Ignition 运行的脚本）。用户只需描述哪些文件系统必须存在、哪些文件必须创建、哪些用户必须存在等等。任何进一步的定制都必须使用由 Ignition 创建的 systemd 服务。

（4）Ignition配置不应手写

Ignition 配置被设计为人类可读但难以编写，是为了阻止用户尝试手动编写配置。可以使用Butane或类似工具生成或转化生成Ignition 配置。

## Afterburn

Afterburn是类似于云平台一样的一次性代理，可以用于与特定的提供者的元数据端点进行交互，通常和Ignition结合使用。

Afterburn包含了很多可以在实例生命周期中不同时间段运行的模块。下面的服务是根据特定的平台可能在第一次启动时在initramfs中运行的：

（1）设置本地主机名

（2）加入网络命令行参数

以下的功能是在一定条件下，作为systemd服务单元在一些平台上才能使用的：

（1）为本地系统用户安装公共SSH密钥

（2）从实例元数据中检索属性

（3）给提供者登记以便报道成功的启动或实例供应

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/system_container/dynamically_loading_the_kernel_module.md
---
# 动态加载内核模块

## 功能描述

容器内业务可能依赖某些内核模块，可通过设置环境变量的方式，在系统容器启动前动态加载容器中业务需要的内核模块到宿主机，此特性需要配合syscontainer-hooks一起使用，具体使用可参看"容器资源动态管理（syscontainer-tools）"章节。

## 参数说明

## 约束限制

* 如果加载的内核模块是未经过验证的，或者跟宿主机已有模块冲突的场景，会导致宿主机出现不可预知问题，在做加载内核模块时需要谨慎操作。
* 动态加载内核模块通过将需要加载的内核模块传递给容器，此功能是依靠syscontainer-tools捕获到容器启动的环境变量实现，依赖syscontainer-tools的正确安装部署。
* 加载的内核模块需要手动进行删除。

## 使用示例

启动系统容器时，指定-e KERNEL\_MODULES参数，待系统容器启动后，可以看到ip\_vs模块被成功加载到内核中。

```sh
[root@localhost ~]# lsmod | grep ip_vs
[root@localhost ~]# isula run -tid -e KERNEL_MODULES=ip_vs,ip_vs_wrr --hook-spec /etc/syscontainer-tools/hookspec.json --system-container --external-rootfs /root/myrootfs none init
ae18c4281d5755a1e153a7bff6b3b4881f36c8e528b9baba8a3278416a5d0980
[root@localhost ~]# lsmod | grep ip_vs
ip_vs_wrr              16384  0
ip_vs                 176128  2 ip_vs_wrr
nf_conntrack          172032  7 xt_conntrack,nf_nat,nf_nat_ipv6,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink,ip_vs
nf_defrag_ipv6         20480  2 nf_conntrack,ip_vs
libcrc32c              16384  3 nf_conntrack,nf_nat,ip_vs
```

> \[!NOTE]说明
>
> * 宿主机需要安装syscontainer-tools。
> * 需要指定--hooks-spec为syscontainer hooks。

---

---
url: /zh/docs/22.03_LTS_SP4/server/development/gcc/dfot_user_guide.md
---
# 动态反馈优化框架用户指南

## 介绍

D-FOT是动态反馈优化框架（**D**ynamic **F**eedback-oirected **O**ptimization **T**ool），目标是实现应用全自动、无感知使能反馈优化特性，全面提高反馈优化特性的易用性和性能表现。
本框架计划实现两类动态优化特性：启动时优化和运行时优化，当前已实现基于oeAware在线调优框架的启动时优化功能。
启动时优化：应用运行时自动执行采样和反馈优化，完成优化后自动接管下一次启动，不需要用户修改启动流程，应用重启后自动拉起优化版本。
运行时优化：应用运行时自动执行采样和反馈优化，不需要用户操作重启，整个过程无用户介入，仅少量中断后即可使能优化版本。

## 软件架构说明

本框架包含以下几个模块：

* 基于libkperf的采样数据处理
* 基于sysboost的启动接管和二进制优化特性使能
* 基于llvm-bolt的二进制反馈优化
* oeAware调优插件dfot\_tuner\_sysboost实现

## 依赖项

系统OS要求：当前仅支持openEuler-22.03-LTS-SP4
以下依赖组件均可通过yum安装

| 组件              | 代码仓                                                                                        | 分支或版本                             | 说明          |
| --------------- | ------------------------------------------------------------------------------------------ | --------------------------------- | ----------- |
| oeAware-manager | <https://atomgit.com/openeuler/oeAware-manager> | openEuler-22.03-LTS-SP4 v2.0.0-3  | 业务在线无感调优框架  |
| libkperf        | <https://atomgit.com/openeuler/libkperf>               | openEuler-22.03-LTS-SP4 v1.2-2    | 轻量级全内存化采集工具 |
| sysboost        | <https://atomgit.com/openeuler/sysboost>               | openEuler-22.03-LTS-SP4           | 微架构优化工具     |
| llvm-bolt       | <https://atomgit.com/src-openeuler/llvm-bolt>                                                  | openEuler-22.03-LTS-SP4 17.0.6-12 | 二进制优化器      |

## 使用流程

### 被优化应用的必要条件

1. 被优化二进制需要带有重定位信息。使用自编译的软件：需要在编译时增加`-Wl,-q`链接选项，如MySQL：`cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DBUILD_CONFIG=mysql_release -DWITH_BOOST=../boost -DCMAKE_C_LINK_FLAGS="-Wl,-q" -DCMAKE_CXX_LINK_FLAGS="-Wl,-q"`；使用oe软件包的场景：后续会提供对应应用的relocation包，直接安装即可。
2. 如何判断目标二进制是否带有重定位信息：`-Wl,-q`生效后，二进制会增加RELA段，可以通过`readelf -SW /path/to/bin`来判断，如MySQL加选项之前，仅有`.rela.dyn`和`.rela.plt`段，增加后会出现`.rela.text`、`.rela.eh_frame`等10+ RELA段；如果`-Wl,-q`未生效，则在手动perf采样并执行`perf2bolt`时，或者执行`llvm-bolt`优化时（无论是否通过sysboost机制）会有告警：`BOLT-WARNING: non-relocation mode for AArch64 is not fully supported`。

### D-FOT准备

yum安装D-FOT或通过以下命令源码构建（注意如果libkperf或oeAware-manager也是源码构建，执行cmake时需要额外指定lib和include路径）。

```shell
# 依赖包安装
yum install boost-devel log4cplus-devel numactl-devel libkperf-devel oeAware-manager-devel
# 下载源码并编译
git clone https://atomgit.com/openeuler/D-FOT.git
cd D-FOT
mkdir build && cd build
cmake ../ -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_SKIP_RPATH=TRUE
make -j`nproc`
cd ..

cp build/libdfot.so /lib64/oeAware-plugin/
chmod 440 /lib64/oeAware-plugin/libdfot.so
mkdir -p /etc/dfot/
cp configs/dfot.ini /etc/dfot/
```

### 配置修改

修改`/etc/dfot/dfot/ini`，内容项参考如下描述：
公共配置： \[general]

| 配置项                           | 值范围                                 | 是否可用 | 说明                                                                                     |
| ----------------------------- | ----------------------------------- | ---- | -------------------------------------------------------------------------------------- |
| LOG\_LEVEL                     | `<FATAL, ERROR, WARN, INFO, DEBUG>` | 可用   | 优化服务日志级别，级别依次递减，默认为INFO，注意级别越低打印的日志越多                                                                 |
| COLLECTOR\_SAMPLING\_STRATEGY   | `0`                               | 不可用  | 采样策略0表示插件enable后即持续低频采样1表示enable时启动监控线程，只有负载达到阈值场景才采样当前由oeaware控制采样流程，仅支持0 |
| COLLECTOR\_HIGH\_LOAD\_THRESHOLD | `[0, cpus*100]`                     | 不可用  | 仅在采样策略1场景下生效，使用HIGH\_LOAD\_THRESHOLD作为触发采样的应用CPU使用率阈值，当前不支持                              |
| COLLECTOR\_SAMPLING\_PERIOD     | `100`                               | 不可用  | collector执行run间隔，每隔COLLECTOR\_SAMPLING\_PERIOD ms执行一次，当前统一由oeaware配置                     |
| COLLECTOR\_SAMPLING\_FREQ       | `100`                               | 不可用  | 采样频率，每秒采样COLLECTOR\_SAMPLING\_FREQ次，当前统一由oeaware配置                                       |
| COLLECTOR\_DATA\_AGING\_TIME     | 按实际需要确定                             | 可用   | 采样数据老化时间，当前数据与最老数据时间差值达到阈值时，丢弃累积采样数据，单位ms                                              |
| TUNER\_TOOL                    | `["sysboost"]`                      | 不可用  | 二进制优化器，当前仅支持sysboost                                                                   |
| TUNER\_CHECK\_PERIOD            | `[10, max]`                         | 可用   | 优化插件检查时间间隔，每隔一段时间判断内部收集数据是否满足新一轮优化条件，单位ms，无上限，但尽量不要太大，以免优化不及时，默认1000                                                |
| TUNER\_PROFILE\_DIR             | 按实际需要确定                             | 可用   | 采样数据存放位置，profile文件被命名为`[app_name]_[full_path_hash]_[threshold].profile`                |
| TUNER\_OPTIMIZING\_STRATEGY     | `[0, 1]`                            | 可用   | 优化策略，0表示只优化一次，1表示只要采样信息在刷新，可以持续多次优化                                                    |
| TUNER\_OPTIMIZING\_CONDITION    | `0`                            | 不可用  | 触发优化的条件，0表示应用退出后即开始优化，1表示低负载时优化，2表示应用退出且低负载时优化，当前仅支持0                                  |
应用配置： \[app]

| 配置项                           | 值范围            | 是否可用 | 说明                                                                                         |
| ----------------------------- | -------------- | ---- | ------------------------------------------------------------------------------------------ |
| FULL\_PATH                     | 按实际需要确定        | 可用   | 应用二进制文件绝对路径                                                                                |
| DEFAULT\_PROFILE               | 按实际需要确定        | 可用   | 应用的开箱profile文件，用于冷启动时使能二进制优化，没有则留空                                                         |
| COLLECTOR\_DUMP\_DATA\_THRESHOLD | `[10000, max]` | 可用   | 采样数据达到该阈值行数时触发数据导出到profile，数值越大需要的采集的samples越多，该值需要依据实际应用和测试场景确定，mysql参考值：65000                                             |
| BOLT\_DIR                      | `/usr/bin`       | 不可用  | BOLT工具路径,留空则默认/usr/bin，内部会调用`${BOLT_DIR}/perf2bolt和${BOLT_DIR}/llvm-bolt`当前由sysboost确定 |
| BOLT\_OPTIONS                  | 按实际需要确定        | 可用   | BOLT优化选项，配置该项可以覆盖内置的默认选项，用于针对性的选项调优                                                        |
| UPDATE\_DEBUG\_INFO             | `[0, 1]`       | 可用   | 优化时是否同步更新调试信息，1表示更新，0表示不更新，注意更新调试信息会有额外耗时                                                  |

### 插件使用

参考<https://atomgit.com/openeuler/oeAware-manager/tree/master/docs>

yum安装D-FOT的场景，可以直接执行以下命令来使能插件实例：

```shell
# 重启oeaware服务
systemctl restart oeaware
# enable采集插件
oeawarectl -e pmu_sampling_collector
# enable反馈优化调优插件
oeawarectl -e dfot_tuner_sysboost

# disable两个插件
oeawarectl -d pmu_sampling_collector
oeawarectl -d dfot_tuner_sysboost
```

源码构建D-FOT场景，需要将libdfot.so拷贝至oeaware的默认插件目录`/lib64/oeAware-plugin/`，然后加载激活插件实例，注意插件权限得是440：

```shell
# 可以重启oeaware服务（自动触发/lib64/oeAware-plugin/下的插件加载）
systemctl restart oeaware
# 或者不重启服务，手动调用oeawarectl加载插件
oeawarectl -l libdfot.so
# 后续可以进行上述enable操作
```

其他操作步骤可以查阅oeAware用户手册。

### 使用举例

以 mysql-8.0.25 测试tpcc场景为例：

```shell
# 1. 环境准备
yum install libkperf oeaware-manager sysboost llvm-bolt D-FOT

# 2. 构建mysql
# -Wl,-q是优化必要条件；-fPIE和-Wl,-zmax-page-size=0x200000 -Wl,-zcommon-page-size=0x200000可选，如果需要sysboost大页预加载优化则必选
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DBUILD_CONFIG=mysql_release -DWITH_BOOST=../boost -DCMAKE_C_FLAGS="-fPIE" -DCMAKE_CXX_FLAGS="-fPIE" -DCMAKE_C_LINK_FLAGS="-Wl,-q -Wl,-zmax-page-size=0x200000 -Wl,-zcommon-page-size=0x200000 -pie" -DCMAKE_CXX_LINK_FLAGS="-Wl,-q -Wl,-zmax-page-size=0x200000 -Wl,-zcommon-page-size=0x200000 -pie"

# 3. 启动oeaware和优化插件
# 需按应用实际信息填写/etc/dfot/dfot.ini，再执行以下命令
systemctl start sysboost
systemctl start oeaware
oeawarectl -e pmu_sampling_collector
oeawarectl -e dfot_tuner_sysboost

# 4. 执行TPCC测试
...

# 5. 重启mysql
# 可提前通过/etc/dfot/dfot.log确认是否已经执行优化，应用重启后即是优化版本
```

## 约束限制

1. 受限于sysboost的实现，暂不支持在虚拟化场景下使用D-FOT（KVM虚拟机：支持使用但不支持热迁移；docker容器：不支持）。
2. 待优化的目标应用需要具备重定位信息。
3. 作为oeAware调优插件使用D-FOT时，当前对应的oeAware采集插件pmu\_sampling\_collector固定采样频率为100，可能会导致采样时间较久（作为参考：perf默认采样频率为4000）。

## 未来规划

* \[ ]  虚拟化场景支持
* \[ ]  运行时优化支持
* \[ ]  进一步提升二进制优化效果

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/upgrade_methods.md
---
# 升级

* 若为相同大版本之间的升级，例如从2.x.x版本升级到2.x.x版本，请执行如下命令：

  ```sh
  # sudo yum update -y iSulad
  ```

* 若为不同大版本之间的升级，例如从1.x.x版本升级到2.x.x版本，请先保存当前的配置文件/etc/isulad/daemon.json，并卸载已安装的iSulad软件包，然后安装待升级的iSulad软件包，随后恢复配置文件。

> !\[NOTE]说明
>
> * 可通过**sudo rpm -qa |grep iSulad**  或  **isula version**  命令确认当前iSulad的版本号。
> * 相同大版本之间，如果希望手动升级，请下载iSulad及其所有依赖的RPM包进行升级，参考命令如下：
>
> ```sh
> # sudo rpm -Uhv iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm  
> ```
>
> 若升级失败，可通过--force选项进行强制升级，参考命令如下：
>
> ```sh
> # sudo rpm -Uhv --force iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm  
> ```
>
> * 如若iSulad依赖的libisula组件发生升级，iSulad应该与对应版本的libisula一起升级，参考命令如下：
>
> ```sh
> # sudo rpm -Uvh libisula-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm
> ```
>
> * iSulad在openeuler 22.03-LTS-SP3之前的版本使用lcr作为默认容器运行时。因此，跨此版本升级时，在升级之前创建的容器仍是使用lcr作为容器运行时，只有在升级之后创建的容器才会采用新版本的默认运行时runc。若在新版本中仍需使用lcr容器运行时，需要修改isulad默认配置文件（默认为/etc/isulad/daemon.json）中的default-runtime为lcr或者在运行容器时指定容器运行时为lcr（--runtime lcr）, 在升级时若对应的lcr、lxc版本发生升级，同样应该与iSulad一起升级。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/installation_upgrade/upgrade/openeuler_22.03_lts_upgrade_and_downgrade_guide.md
---
# 升降级指导

## **1**. **系统安装**

获取openEuler-22.03-LTS-SP4镜像，参考安装指南，完成openEuler操作系统的安装。

查看当前环境openEuler、kernel版本。

```sh
[root@openEuler ~]# cat /etc/openEuler-latest
```

## **2**. **升级兼容性**

### 2.1 新增openEuler-22.03-LTS-SP4 repo源 (openEuler-22.03-LTS-SP4.repo)

```bash
vi /etc/yum.repos.d/openEuler-22.03-LTS-SP4.repo
```

输入以下openEuler-22.03-LTS-SP4 repo源信息并保存退出。

```conf
[SP4_OS]
name=SP4_OS
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler

[SP4_everything]
name=SP4_everything
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/$basearch/RPM-GPG-KEY-openEuler

[SP4_EPOL]
name=EPOL
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/$basearch/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler

[SP4_debuginfo]
name=debuginfo
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/debuginfo/$basearch/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/debuginfo/$basearch/RPM-GPG-KEY-openEuler

[SP4_source]
name=source
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/source/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/source/RPM-GPG-KEY-openEuler

[SP4_update]
name=SP4_update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/update/$basearch/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/$basearch/RPM-GPG-KEY-openEuler

[SP4_update-source]
name=SP4_update-source
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/update/source/
metadata_expire=1h
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/source/RPM-GPG-KEY-openEuler
```

### 2.2 执行升级

```bash
dnf update | tee update_log
```

\[!NOTE]说明

1. 安装报错时，通过执行 dnf update --skip-broken -x conflict\_pkg1 |tee update\_log 规避安装冲突问题，如果有多个包冲突，添加多个-x conflict\_pkg1 -x conflict\_pkg2 -x conflict\_pkg3，待升级完成后，对跳过的软件包单独进行分析、验证、升级。
2. 参数释义：
   * \--allowerasing：通过卸载已安装的软件包解决依赖关系。
   * \--skip-broken：通过跳过软件包解决依赖问题。
   * -x：跟--skip-broken配合使用，后边跟需要跳过的软件包名。

### 2.3 重启系统

```shell
reboot
```

## **3**. **升级结果验证**

查看当前环境的openEuler、kernel版本。

```sh
[root@openEuler ~]# cat /etc/openEuler-latest
```

## **4**. **降级兼容性**

### 4.1 执行降级

```shell
dnf downgrade | tee downgrade_log
```

### 4.2 重启系统

```shell
reboot
```

## **5**. **降级结果验证**

查看当前环境的openEuler、kernel版本。

```sh
[root@openEuler ~]# cat /etc/openEuler-latest
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/uninstallation.md
---
# 卸载

卸载iSulad的操作步骤如下：

1. 卸载iSulad及其依赖软件包
   * 若使用yum方式安装，卸载的参考命令如下：

     ```sh
     # yum remove iSulad
     ```

   * 若使用rpm方式安装，需卸载iSulad及其依赖包，卸载单个RPM包的参考命令如下：

     ```sh
     # rpm -e iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm
     ```

2. 镜像、容器、volumes以及相关配置文件不会自动删除，需要手动删除。参考命令如下：

   ```sh
   # rm -rf /var/lib/iSulad
   ```

---

---
url: /zh/docs/22.03_LTS_SP4/server/releasenotes/contribution.md
---
# 参与贡献

作为openEuler用户，你可以通过多种方式协助openEuler社区。参与社区贡献的方法请参见[贡献攻略](https://www.openeuler.org/zh/community/contribution/)，这里简单列出部分方式供参考。

## 特别兴趣小组

openEuler将拥有共同兴趣的人们聚在一起，组成了不同的特别兴趣小组（SIG）。当前已有的SIG请参见[SIG列表](https://www.openeuler.org/zh/sig/sig-list/)。

我们欢迎并鼓励你加入已有的SIG或创建新的SIG，创建方法请参见[SIG管理指南](https://atomgit.com/openeuler/community/blob/master/zh/technical-committee/governance/)。

## 邮件列表和任务

欢迎你积极地帮助用户解决在[邮件列表](https://www.openeuler.org/zh/community/mailing-list/)和issue任务（包括[代码仓任务](https://gitee.com/organizations/openeuler/issues)和[软件包仓任务](https://gitee.com/organizations/src-openeuler/issues)） 中提出的问题。另外，我们也欢迎你提出问题。这些都将帮助openEuler社区更好地发展。

## 文档

你不仅可以通过提交代码参与社区贡献，我们也欢迎你反馈遇到的问题、困难，或者对文档易用性、完整性的改进建议等。例如获取软件或文档过程中的问题，使用系统过程中的难点。欢迎关注并改进[openEuler社区](https://openeuler.org/zh/)的文档模块。

## IRC

openEuler也在IRC开辟了频道，作为提供社区支持和交互的额外渠道。详情请参见[openEuler IRC](https://atomgit.com/openeuler/community/tree/master/zh/communication)。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/installation_upgrade/installation/more_resources.md
---
# 参考资料

* 如何构建树莓派镜像文件
* 如何使用树莓派

---

---
url: /zh/docs/22.03_LTS_SP4/server/security/trusted_computing/trusted_computing.md
---
# 可信计算

## 可信计算基础

### 可信计算

不同国际组织对可信（Trusted）做了不同的定义。

1. 可信计算组织（TCG）的定义：

   一个实体是可信的，它的行为总是以预期的方式达到预期的目标。

2. 国际标准化组织与国际电子技术委员会定义（1999）：

   参与计算的组件、操作或过程在任意的条件下是可预测的，并能够抵御病毒和一定程度的物理干扰。

3. IEEE Computer Society Technical Committee on Dependable Computing 定义：

   所谓可信，是指计算机系统所提供的服务是可被论证其是可信赖的，可信赖主要是指系统的可靠性和可用性。

简而言之，可信就是系统按照预定的设计和策略运行，不做其他事情。

一个可信计算系统由信任根、可信硬件平台、可信操作系统和可信应用组成，它的基本思想是首先创建一个安全信任根（TCB），然后建立从硬件平台、操作系统到应用的信任链，在这条信任链上从根开始，前一级认证后一级，实现信任的逐级扩展，从而实现一个安全可信的计算环境。

![](./figures/trusted_chain.png)

相比于传统安全机制的“头痛医头，脚痛医脚”，发现一个病毒消灭一个病毒，可信计算采用的是白名单机制，即只允许经过认证的内核、内核模块、应用程序等在系统上运行，如果发现程序已发生更改（或本来就是一个未知的程序），就拒绝其执行。

## 内核完整性度量（IMA）

### 概述

#### IMA

IMA，全称 Integrity Measurement Architecture（完整性度量架构），是内核中的一个子系统，能够基于自定义策略对通过 execve()、mmap() 和 open() 系统调用访问的文件进行度量，度量结果可被用于**本地/远程证明**，或者和已有的参考值比较以**控制对文件的访问**。

根据 IMA wiki 的定义，内核完整性子系统的功能可以被分为三部分：

* 度量（measure）：检测对文件的意外或恶意修改，无论远程还是本地。
* 评估（appraise）：度量文件并与一个存储在扩展属性中的参考值作比较，控制本地文件完整性。
* 审计（audit）：将度量结果写到系统日志中，用于审计。

可以看到，相比于 IMA 度量作为一个“只记录不干涉”的观察员，IMA 评估更像是一位严格的保安人员，它的职责是拒绝对所有“人证不一”的程序的访问。

#### EVM

EVM，全称 Extended Verification Module（扩展验证模块），它的作用就是将系统当中某个文件的安全扩展属性，包括 security.ima 、security.selinux 等合起来计算一个哈希值，然后使用 TPM 中存储的密钥或其他可信环境中的密钥对其进行签名，签名之后的值存在 security.evm 中，这个签名后的值是不能被篡改的，如果被篡改，再次访问的时候就会验签失败。

总而言之，EVM 的作用就是通过对安全扩展属性计算摘要和签名并将其存储在 security.evm 中，提供对安全扩展属性的离线保护。

#### IMA Digest Lists

IMA Digest Lists（IMA 摘要列表扩展）是 openEuler 对内核原生完整性保护机制的增强，它取代了原生 IMA 机制为文件完整性提供保护。

“摘要列表”（digest lists）是一种特殊格式的二进制数据文件，它与 rpm 包一一对应，记录了 rpm 包中受保护文件（即可执行文件和动态库文件）的哈希值。

当正确配置启动参数后，内核将维护一个哈希表（对用户空间不可见），并通过 securityfs 对外提供更新哈希表的接口（digest\_list\_data 和 digest\_list\_data\_del）。摘要列表在构建阶段经过私钥签名，通过接口上传到内核时，需经过内核中的公钥验证。

![](./figures/ima_digest_list_update.png)

在开启 IMA 评估的情况下，每当访问一个可执行文件或动态库文件，就会调用内核中的钩子，计算文件内容和扩展属性的哈希值，并在内核哈希表中进行搜索，如果匹配就允许文件的执行，否则就拒绝访问。

![1599719649188](./figures/ima_verification.png)

相比内核社区原生 IMA 机制，openEuler 内核提供的 IMA 摘要列表扩展从安全性、性能、易用性三个方面进行了改良，助力完整性保护机制在生产环境下落地：

* **具备完整的信任链，安全性好**

  原生 IMA 机制要求在现网环境下预先生成并标记文件扩展属性，访问文件时将文件扩展属性作为参考值，信任链不完整。

  IMA 摘要列表扩展将文件参考摘要值保存在内核空间中，构建阶段通过摘要列表的形式携带在发布的 rpm 包中，安装 rpm 包的同时导入摘要列表并执行验签，确保了参考值来自于软件发行商，实现了完整的信任链。

* **惊艳的性能**

  由于 TPM 芯片是一种低速芯片，因此 PCR 扩展操作成为了 IMA 度量场景的性能瓶颈。摘要列表扩展在确保安全性的前提下，减少了不必要的 PCR 扩展操作，相比原生 IMA 性能提升高达 65%。

  IMA 评估场景下，摘要列表扩展将签名验证统一移动到启动阶段进行，避免每次访问文件时都执行验签，相比原生 IMA 评估场景提升运行阶段文件访问的性能约 20%。

* **快速部署，平滑升级**

  原生 IMA 机制在初次部署或每次更新软件包时，都需要切换到 fix 模式手动标记文件扩展属性后再重启进入 enforce 模式，才能正常访问安装的程序。

  摘要列表扩展可实现安装完成后开箱即用，且允许直接在 enforce 模式下安装或升级 rpm 包，无需重启和手动标记即可使用，实现了用户感知最小化，适合现网环境下的快速部署和平滑升级。

需要注意的是，IMA 摘要列表扩展将原生 IMA 的验签过程提前到启动阶段进行，也引入了一个假设，即内核空间的内存无法被篡改，这就使得 IMA 也依赖于其他安全机制（内核模块安全启动和内存动态度量）以保护内核内存的完整性。

但无论社区原生 IMA 机制还是 IMA 摘要列表扩展，都只是可信计算信任链中的一环，无法孤立地保证系统的安全性，安全自始至终都是一个构建纵深防御的系统工程。

### 约束限制

1. 当前 IMA 评估模式仅支持保护系统中的不可变文件（包括可执行文件和动态库文件）。
2. IMA 提供的是应用层的完整性度量，它的安全性依赖于之前环节的可信。
3. 当前阶段 IMA 不支持第三方应用摘要列表的导入。
4. 启动日志中可能存在 `Unable to open file: /etc/keys/x509_ima.der` 字样，该报错来自于开源社区，不影响 IMA 摘要列表特性的使用。
5. ARM 版本中 IMA 开启日志模式可能存在一些 audit 报错信息，这是由于 modprobe 在摘要列表未导入时加载内核模块所致，不影响正常功能。

### 使用场景

#### IMA measurement

IMA measurement（IMA 度量）的目的是检测对系统文件的意外或恶意修改，度量结果可被用于本地证明或远程证明。

如果系统中存在 TPM 芯片，度量结果将被扩展到 TPM 芯片的指定 PCR 寄存器中，由于 PCR 扩展的单向性以及 TPM 芯片的硬件安全性，用户无法修改已被扩展的度量结果，这就确保了度量结果的真实性。

IMA 度量的文件范围和触发条件可以由用户通过 IMA 策略自行配置。

默认情况下 IMA 不启用，但系统会前往 `/etc/ima/` 路径下寻找 ima-policy 策略文件，如果找到，就会按照策略在启动时度量系统中的文件。如果不想手动编写策略文件，也可以在启动参数中配置 `ima_policy=tcb` 使用默认策略（更多策略参数请参考附录“IMA启动参数”章节）。

系统当前加载的 IMA 策略可以在 `/sys/kernel/security/ima/policy` 文件中查看，IMA 度量日志则位于`/sys/kernel/security/ima/ascii_runtime_measurements` 文件中，如下所示：

```shell
$ head /sys/kernel/security/ima/ascii_runtime_measurements
10 ddee6004dc3bd4ee300406cd93181c5a2187b59b ima-ng sha1:9797edf8d0eed36b1cf92547816051c8af4e45ee boot_aggregate
10 180ecafba6fadbece09b057bcd0d55d39f1a8a52 ima-ng sha1:db82919bf7d1849ae9aba01e28e9be012823cf3a /init
10 ac792e08a7cf8de7656003125c7276968d84ea65 ima-ng sha1:f778e2082b08d21bbc59898f4775a75e8f2af4db /bin/bash
10 0a0d9258c151356204aea2498bbca4be34d6bb05 ima-ng sha1:b0ab2e7ebd22c4d17d975de0d881f52dc14359a7 /lib64/ld-2.27.so
10 0d6b1d90350778d58f1302d00e59493e11bc0011 ima-ng sha1:ce8204c948b9fe3ae67b94625ad620420c1dc838 /etc/ld.so.cache
10 d69ac2c1d60d28b2da07c7f0cbd49e31e9cca277 ima-ng sha1:8526466068709356630490ff5196c95a186092b8 /lib64/libreadline.so.7.0
10 ef3212c12d1fbb94de9534b0bbd9f0c8ea50a77b ima-ng sha1:f80ba92b8a6e390a80a7a3deef8eae921fc8ca4e /lib64/libc-2.27.so
10 f805861177a99c61eabebe21003b3c831ccf288b ima-ng sha1:261a3cd5863de3f2421662ba5b455df09d941168 /lib64/libncurses.so.6.1
10 52f680881893b28e6f0ce2b132d723a885333500 ima-ng sha1:b953a3fa385e64dfe9927de94c33318d3de56260 /lib64/libnss_files-2.27.so
10 4da8ce3c51a7814d4e38be55a2a990a5ceec8b27 ima-ng sha1:99a9c095c7928ecca8c3a4bc44b06246fc5f49de /etc/passwd
```

每一条记录从左到右分别是：

1. PCR：用于扩展度量结果的 PCR 寄存器，默认是 10，只在系统装了 TPM 芯片的情况下有意义。
2. 模板哈希值：最终被用于扩展的哈希值，组合了文件内容哈希和文件路径的长度和值。
3. 模板：扩展度量值的模板，如 ima-ng。
4. 文件内容哈希值：被度量的文件内容的哈希值。
5. 文件路径：被度量的文件路径。
6. 本版本使能了ko压缩特性，当加载被压缩ko文件时，如需IMA使能appraise func=MODULE\_CHECK策略，需要在启动参数配置module.sig\_enforce=1。

#### IMA appraisal

IMA appraisal（IMA 评估）的目的是通过与标准参考值的比较，控制对本地文件的访问。

IMA 首先使用安全扩展属性 security.ima 和 security.evm 存储文件完整性度量的参考值：

* security.ima：存储文件内容的哈希值；
* security.evm：存储文件扩展属性的哈希值签名。

访问受保护文件时，将会触发内核中的钩子，依次验证文件扩展属性和内容的完整性：

1. 使用内核 keyring 中的公钥对文件 security.evm 扩展属性中的签名值验签，与当前文件扩展属性的哈希值比较，如果匹配就证明文件的扩展属性是完整的（包括 security.ima）。
2. 在文件扩展属性完整的前提下，将文件 security.ima 扩展属性的内容与当前文件内容的摘要值比较，如果匹配就允许对文件的访问。

同样，IMA 评估的文件范围和触发条件也可以由用户通过 IMA 策略自行配置。

#### IMA Digest Lists

IMA 摘要列表扩展当前提供对以下三种启动参数组合的支持：

* IMA measurement 度量模式：

  ```shell
  ima_policy=exec_tcb ima_digest_list_pcr=11
  ```

* IMA appraisal 日志模式 + IMA measurement 度量模式：

  ```shell
  ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=log evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest
  ```

* IMA appraisal 强制模式 + IMA measurement 度量模式：

  ```shell
  ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=enforce-evm evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest
  ```

#### IMA measurement virtCCA

在ARM virtCCA机密虚拟机环境中开启IMA度量功能时，支持将度量结果扩展至virtCCA环境的REM0度量寄存器，并将RIM度量寄存器的值记录为boot\_aggregate度量日志。该流程在IMA初始化流程中检测并配置，无需用户手动修改。

### 操作指导

#### 原生 IMA 场景初次部署

第一次启动时，需要在启动参数中配置：

```shell
ima_appraise=fix ima_policy=appraise_tcb
```

`fix` 模式会允许系统在没有参考值的情况下启动，`appraise_tcb` 对应了一种 IMA 策略，具体可参考附录中的“IMA 启动参数”章节。

接下来，你需要访问所有需要被校验的文件，从而为它们添加 IMA 扩展属性：

```shell
$ time find / -fstype ext4 -type f -uid 0 -exec dd if='{}' of=/dev/null count=0 status=none \;
```

该过程会花费一定时间，请耐心等待。命令执行完成后，你可以从受保护文件的扩展属性中看到参考值已被标记：

```shell
$ getfattr -m - -d /sbin/init
# file: sbin/init
security.ima=0sAXr7Qmun5mkGDS286oZxCpdGEuKT
security.selinux="system_u:object_r:init_exec_t"
```

最后，配置以下启动参数并重新启动系统：

```shell
ima_appraise=enforce ima_policy=appraise_tcb
```

#### 摘要列表场景初次部署

1. 配置内核参数进入 log 模式。

   编辑 `/boot/efi/EFI/openEuler/grub.cfg` 文件，加入以下参数：

   ```shell
   ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=log evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest
   ```

   使用 `reboot` 重启系统进入 log 模式，该模式下已开启完整性校验，但不会因校验失败而无法启动。

2. 安装依赖包。

   使用 yum 安装 digest-list-tools 和 ima-evm-utils，确认不低于以下版本：

   ```shell
   $ yum install digest-list-tools ima-evm-utils
   $ rpm -qa | grep digest-list-tools
   digest-list-tools-0.3.93-1.oe1.x86_64
   $ rpm -qa | grep ima-evm-utils
   ima-evm-utils-1.2.1-9.oe1.x86_64
   ```

3. 因签名密钥切换，对已发布的版本，需要手动重新设置下密钥ID，具体操作如下：

   ```shell
   $ setfattr -n security.ima -v "0x$(getfattr -n security.ima --only-values /etc/ima/digest_lists/0-parser_list-compact-libexec | xxd -p | tr -d '\n' | sed 's/b675600b/fb37bc6f/g')" /etc/ima/digest_lists/0-parser_list-compact-libexec
   $ setfattr -n security.ima -v "0x$(getfattr -n security.ima --only-values /etc/ima/digest_lists/0-metadata_list-compact-digest-list-tools-* | xxd -p | tr -d '\n' | sed 's/b675600b/fb37bc6f/g')" /etc/ima/digest_lists/0-metadata_list-compact-digest-list-tools-*
   ```

4. 执行 `dracut` 重新生成 initrd：

   ```shell
   $ dracut -f -e xattr
   ```

   编辑 `/boot/efi/EFI/openEuler/grub.cfg` 文件，将 ima\_appraise=log 改为 ima\_appraise=enforce-evm：

   ```shell
   ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sha256 ima_appraise=enforce-evm evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest
   ```

   使用 reboot 重启即可完成初次部署。

#### 在 OBS 上进行摘要列表构建

OBS 全称 Open Build Service，是一种编译系统，最早在 openSUSE 用于软件包的构建，能够支持多架构的分布式编译。

进行摘要列表构建之前，首先确保您的工程包含以下 rpm 包，且来自 openEuler：

* digest-list-tools
* pesign-obs-integration
* selinux-policy
* rpm
* openEuler-rpm-config

在交付件工程中增加 Project Config：

```shell
Preinstall: pesign-obs-integration digest-list-tools selinux-policy-targeted
Macros:
%__brp_digest_list /usr/lib/rpm/openEuler/brp-digest-list %{buildroot}
:Macros
```

* 在 Preinstall 中新增 digest-list-tools 用于生成摘要列表，pesign-obs-integration 用于生成摘要列表的签名，新增 selinux-policy-targeted 用于确保生成摘要列表时构建环境内 SELinux 标签正确。
* 在 Macros 中定义宏 %\_\_brp\_digest\_list，rpm 将在构建阶段通过这个宏执行命令为编译完成的二进制文件生成摘要列表。这个宏可以作为一个开关控制工程中的摘要列表是否生成。

配置完成后，OBS 会自动执行全量构建，正常情况下构建完成后，软件包中会新增以下两个文件：

* /etc/ima/digest\_lists/0-metadata\_list-compact-\[包名]-\[版本号]
* /etc/ima/digest\_lists.tlv/0-metadata\_list-compact\_tlv-\[包名]-\[版本号]

#### 在 Koji 上进行摘要列表构建

Koji 是 Fedora 社区的编译系统，openEuler 社区将在后续支持，敬请期待。

### FAQ

1. 为什么进入 enforce 模式后系统无法启动或启动后命令无法执行/服务不正常？

   enforce 模式下 IMA 会对文件访问做控制，如果访问文件的内容或扩展属性不完整，就会被拒绝访问，当影响启动的关键命令无法执行时，就会造成系统无法启动。

   请确认是否存在以下问题：

   * **摘要列表是否被加入到 initrd 中？**

     初次部署时是否执行了 dracut 命令将摘要列表加入内核？如果摘要列表没有加入 initrd，启动阶段就无法导入摘要列表，从而导致启动失败。

   * **是否使用官方提供的 rpm 包？**

     如果使用的是非 openEuler 官方提供的 rpm 包，rpm 包可能没有携带摘要列表，或者对摘要列表签名的私钥与内核中的验签公钥不匹配，从而导致摘要列表没有被导入内核。

   如果原因还不明确，可以进入 log 模式启动，从错误日志中寻找原因：

   ```shell
   $ dmesg | grep appraise
   ```

2. 为什么 enforce 模式下没有对系统文件做访问控制？

   系统没有按照预期对文件执行访问控制，首先查看启动参数中的 IMA 策略是否已被正确配置：

   ```shell
   $ cat /proc/cmdline
   ...ima_policy=exec_tcb|appraise_exec_tcb|appraise_exec_immutable...
   ```

   其次查看当前内核中 IMA 策略是否已生效：

   ```shell
   $ cat /sys/kernel/security/ima/policy
   ```

   如果 policy 文件是空的，证明策略没有设置成功，系统也就不会进行访问控制。

3. 初次部署完成后，安装/升级/卸载软件包后还需要手动执行 dracut 生成 initrd 吗？

   不需要。rpm 包提供的 digest\_list.so 插件能够在 rpm 包粒度提供摘要列表的自动更新，可以实现用户对摘要列表的无感知。

### 附录

#### IMA securityfs 接口说明

原生 IMA 提供的 securityfs 接口如下：

> 注：以下接口路径都位于 `/sys/kernel/security/` 目录下。

| 路径                           | 权限 | 说明                                     |
| ------------------------------ | ---- | ---------------------------------------- |
| ima/policy                     | 600  | IMA 策略接口                             |
| ima/ascii\_runtime\_measurement  | 440  | ascii 码形式表示的 IMA 度量结果          |
| ima/binary\_runtime\_measurement | 440  | 二进制形式表示的 IMA 度量结果            |
| ima/runtime\_measurement\_count  | 440  | 度量结果数量统计                         |
| ima/violations                 | 440  | IMA 度量结果冲突数                       |
| evm                            | 660  | EVM 模式，即校验文件扩展属性完整性的方式 |

其中，`/sys/kernel/security/evm` 的取值有以下四种：

* 0：EVM 未初始化；
* 1：使用 HMAC（对称加密）方式校验扩展属性完整性；
* 2：使用公钥验签（非对称加密）方式校验扩展属性完整性；
* 6：关闭扩展属性完整性校验（openEuler 使用此方式）。

IMA 摘要列表扩展额外提供的 securityfs 接口如下：

| 路径                     | 权限 | 说明                                    |
| ------------------------ | ---- | --------------------------------------- |
| ima/digests\_count        | 440  | 显示系统哈希表中的总摘要数量（IMA+EVM） |
| ima/digest\_list\_data     | 200  | 摘要列表新增接口                        |
| ima/digest\_list\_data\_del | 200  | 摘要列表删除接口                        |

#### IMA 策略语法

每条 IMA 策略语句都必须以 action 关键字代表的**动作**开头，后接**筛选条件**：

* action：表示该条策略具体的动作，一条策略只能选一个 action。

  > 注：实际书写时**可忽略 action 字样**，直接书写 dont\_measure，不需要写成 action=dont\_measure。

* func：表示被度量或鉴定的文件类型，常和 mask 匹配使用，一条策略只能选一个 func。

  * FILE\_CHECK 只能同 MAY\_EXEC、MAY\_WRITE、MAY\_READ 匹配使用。
  * MODULE\_CHECK、MMAP\_CHECK、BPRM\_CHECK 只能同 MAY\_EXEC 匹配使用。
  * 匹配关系以外的组合不会产生效果。

* mask：表示文件在做什么操作时将被度量或鉴定，一条策略只能选一个 mask。

* fsmagic：表示文件系统类型的十六进制魔数，定义在 `/usr/include/linux/magic.h` 文件中。

  > 注：默认情况下度量所有文件系统，除非使用 dont\_measure/dont\_appraise 标记不度量某文件系统。

* fsuuid：表示系统设备 uuid 的 16 位的十六进制字符串。

* objtype：表示文件类型，一条策略只能选一个文件类型。

  > 注：objtype 相比 func 而言，划分的粒度更细，比如 obj\_type=nova\_log\_t 表示 nova log 类型的文件。

* uid：表示哪个用户（用用户 id 表示）对文件进行操作，一条策略只能选一个 uid。

* fowner：表示文件的属主（用用户 id 表示）是谁，一条策略只能选一个 fowner。

关键字的具体取值及说明如下：

| 关键字        | 值                 | 说明                                                         |
| ------------- | ------------------ | ------------------------------------------------------------ |
| action        | measure            | 开启 IMA 度量                                                |
|               | dont\_measure       | 禁用 IMA 度量                                                |
|               | appraise           | 开启 IMA 评估                                                |
|               | dont\_appraise      | 禁用 IMA 评估                                                |
|               | audit              | 开启审计                                                     |
| func          | FILE\_CHECK         | 将要被打开的文件                                             |
|               | MODULE\_CHECK       | 将要被装载的内核模块文件                                     |
|               | MMAP\_CHECK         | 将要被映射到进程内存空间的动态库文件                         |
|               | BRPM\_CHECK         | 将要被执行的文件（不含通过 `/bin/bash` 等程序打开的脚本文件） |
|               | POLICY\_CHECK       | 将要被作为补充 IMA 策略装载的文件                            |
|               | FIRMWARE\_CHECK     | 将要被加载到内存中的固件                                     |
|               | DIGEST\_LIST\_CHECK  | 将要被加载到内核中的摘要列表文件                             |
|               | KEXEC\_KERNEL\_CHECK | 将要切换的 kexec 内核                                        |
| mask          | MAY\_EXEC           | 执行文件                                                     |
|               | MAY\_WRITE          | 写文件。不建议使用，受限于 echo、vim 等开源机制（修改本质是新建临时文件再重命名），并不是每次修改都会触发 MAY\_WRITE 的 IMA 度量。 |
|               | MAY\_READ           | 读文件                                                       |
|               | MAY\_APPEND         | 扩展文件属性                                                 |
| fsmagic       | fsmagic=xxx        | 表示文件系统类型的十六进制魔数                               |
| fsuuid        | fsuuid=xxx         | 表示系统设备 uuid 的 16 位的十六进制字符串                   |
| fowner        | fowner=xxx         | 文件属主的用户 id                                            |
| uid           | uid=xxx            | 操作文件的用户 id                                            |
| obj\_type      | obj\_type=xxx\_t     | 表示文件的类型（基于 SELinux 标签）                          |
| pcr           | pcr=\<num>          | 选择 TPM 中用于扩展度量值的 PCR（默认为 10）                 |
| appraise\_type | imasig             | 基于签名进行 IMA 评估                                        |
|               | meta\_immutable     | 基于签名进行文件扩展属性的评估（支持摘要列表）               |

> 注：PATH\_CHECK 等同于 FILE\_CHECK，FILE\_MMAP 等同于 MMAP\_CHECK，不在本表提及。

#### IMA 原生启动参数

原生 IMA 的内核启动参数列表如下：

| 参数名称         | 取值         | 功能                                                         |
| ---------------- | ------------ | ------------------------------------------------------------ |
| ima\_appraise     | off          | 关闭 IMA 评估模式，在访问文件时不进行完整性校验，也不为文件生成新的参考值。 |
|                  | enforce      | 开启 IMA 评估强制模式，在访问文件时进行完整性校验，即计算文件摘要值并与参考值比对，如果比对失败就拒绝对文件的访问。IMA 会为新文件生成新的参考值。 |
|                  | fix          | 开启 IMA 修复模式，在该模式下允许更新受保护文件的参考值。    |
|                  | log          | 开启 IMA 评估日志模式，在访问文件时进行完整性校验，但即使校验失败也允许执行命令，只进行日志记录。 |
| ima\_policy       | tcb          | 度量所有文件执行、动态库映射、内核模块导入以及设备驱动加载，此外，root 用户读文件的行为也会被度量。 |
|                  | appraise\_tcb | 对所有 root 属主的文件进行评估。                             |
|                  | secure\_boot  | 对所有内核模块导入、硬件驱动加载、kexec 内核切换以及 IMA 策略进行评估，前提是这些文件都具有 IMA 签名。 |
| ima\_tcb          | 无           | 等价于 ima\_policy=tcb                                        |
| ima\_appraise\_tcb | 无           | 等价于 ima\_policy=appraise\_tcb                               |
| ima\_hash         | sha1/md5/... | IMA 摘要算法，默认为 sha1                                    |
| ima\_template     | ima          | IMA 度量扩展模板                                             |
|                  | ima-ng       | IMA 度量扩展模板                                             |
|                  | ima-sig      | IMA 度量扩展模板                                             |
| integrity\_audit  | 0            | 基础完整性审计信息（默认）                                   |
|                  | 1            | 额外完整性审计信息                                           |

> 注：ima\_policy 参数可以同时指定多个值，例如 ima\_policy=tcb|appraise\_tcb，启动后系统的 IMA 策略就是这两种参数对应的策略的总和。

启动参数 `ima_policy=tcb` 对应的 IMA 策略为：

```ini
# PROC_SUPER_MAGIC = 0x9fa0
dont_measure fsmagic=0x9fa0
# SYSFS_MAGIC = 0x62656572
dont_measure fsmagic=0x62656572
# DEBUGFS_MAGIC = 0x64626720
dont_measure fsmagic=0x64626720
# TMPFS_MAGIC = 0x01021994
dont_measure fsmagic=0x1021994
# DEVPTS_SUPER_MAGIC=0x1cd1
dont_measure fsmagic=0x1cd1
# BINFMTFS_MAGIC=0x42494e4d
dont_measure fsmagic=0x42494e4d
# SECURITYFS_MAGIC=0x73636673
dont_measure fsmagic=0x73636673
# SELINUX_MAGIC=0xf97cff8c
dont_measure fsmagic=0xf97cff8c
# SMACK_MAGIC=0x43415d53
dont_measure fsmagic=0x43415d53
# CGROUP_SUPER_MAGIC=0x27e0eb
dont_measure fsmagic=0x27e0eb
# CGROUP2_SUPER_MAGIC=0x63677270
dont_measure fsmagic=0x63677270
# NSFS_MAGIC=0x6e736673
dont_measure fsmagic=0x6e736673 
measure func=MMAP_CHECK mask=MAY_EXEC 
measure func=BPRM_CHECK mask=MAY_EXEC 
measure func=FILE_CHECK mask=MAY_READ uid=0 
measure func=MODULE_CHECK 
measure func=FIRMWARE_CHECK
```

启动参数 `ima_policy=tcb_appraise` 对应的 IMA 策略为：

```ini
# PROC_SUPER_MAGIC = 0x9fa0
dont_appraise fsmagic=0x9fa0
# SYSFS_MAGIC = 0x62656572
dont_appraise fsmagic=0x62656572
# DEBUGFS_MAGIC = 0x64626720 
dont_appraise fsmagic=0x64626720 
# TMPFS_MAGIC = 0x01021994
dont_appraise fsmagic=0x1021994
# RAMFS_MAGIC
dont_appraise fsmagic=0x858458f6 
# DEVPTS_SUPER_MAGIC=0x1cd1
dont_appraise fsmagic=0x1cd1
# BINFMTFS_MAGIC=0x42494e4d
dont_appraise fsmagic=0x42494e4d 
# SECURITYFS_MAGIC=0x73636673
dont_appraise fsmagic=0x73636673
# SELINUX_MAGIC=0xf97cff8c
dont_appraise fsmagic=0xf97cff8c 
# SMACK_MAGIC=0x43415d53
dont_appraise fsmagic=0x43415d53 
# NSFS_MAGIC=0x6e736673
dont_appraise fsmagic=0x6e736673 
# CGROUP_SUPER_MAGIC=0x27e0eb
dont_appraise fsmagic=0x27e0eb 
# CGROUP2_SUPER_MAGIC=0x63677270
dont_appraise fsmagic=0x63677270 
appraise fowner=0
```

启动参数 `ima_policy=secure_boot` 对应的 IMA 策略为：

```ini
appraise func=MODULE_CHECK appraise_type=imasig 
appraise func=FIRMWARE_CHECK appraise_type=imasig 
appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig 
appraise func=POLICY_CHECK appraise_type=imasig
```

#### IMA 摘要列表启动参数

IMA 摘要列表特性额外引入的内核启动参数如下：

| 参数名称                 | 取值                    | 功能                                                         |
| ------------------------ | ----------------------- | ------------------------------------------------------------ |
| integrity                | 0                       | IMA 特性总开关关闭（默认）                                   |
|                          | 1                       | IMA 特性总开关打开                                           |
| ima\_appraise             | off                     | 关闭 IMA 评估模式                                            |
|                          | enforce-evm             | IMA 评估强制模式，在访问文件时进行完整性校验并进行访问控制   |
| ima\_appraise\_digest\_list | digest                  | 当 EVM 被禁用时，使用摘要列表进行 IMA appraise，摘要列表同时保护文件内容和扩展属性 |
|                          | digest-nometadata       | 在EVM摘要值不存在的情况下，仅基于IMA摘要值进行完整性校验（不保护文件扩展属性） |
| evm                      | fix                     | 允许任何对扩展属性的修改（即使修改会导致扩展属性完整性校验失败） |
|                          | ignore                  | 只有在扩展属性不存在或不正确的情况下才允许修改               |
| ima\_policy               | exec\_tcb                | IMA 度量策略，详见下文策略说明。                             |
|                          | appraise\_exec\_tcb       | IMA 评估策略，详见下文策略说明。                             |
|                          | appraise\_exec\_immutable | IMA 评估策略，详见下文策略说明。                             |
| ima\_digest\_list\_pcr      | 11                      | 使用 PCR 11 替代 PCR 10，仅使用摘要列表进行度量              |
|                          | +11                     | 依然保留 PCR 10 的度量，在有TPM芯片时也往TPM芯片写度量结果   |
| initramtmpfs             | 无                      | 添加对 tmpfs 的支持                                          |

启动参数 `ima_policy=exec_tcb` 对应的 IMA 策略为：

```ini
dont_measure fsmagic=0x9fa0 
dont_measure fsmagic=0x62656572 
dont_measure fsmagic=0x64626720 
dont_measure fsmagic=0x1cd1 
dont_measure fsmagic=0x42494e4d 
dont_measure fsmagic=0x73636673 
dont_measure fsmagic=0xf97cff8c 
dont_measure fsmagic=0x43415d53 
dont_measure fsmagic=0x27e0eb 
dont_measure fsmagic=0x63677270 
dont_measure fsmagic=0x6e736673 
measure func=MMAP_CHECK mask=MAY_EXEC 
measure func=BPRM_CHECK mask=MAY_EXEC 
measure func=MODULE_CHECK 
measure func=FIRMWARE_CHECK 
measure func=POLICY_CHECK 
measure func=DIGEST_LIST_CHECK 
measure parser
```

启动参数 `ima_policy=appraise_exec_tcb` 对应的 IMA 策略为：

```ini
appraise func=MODULE_CHECK appraise_type=imasig 
appraise func=FIRMWARE_CHECK appraise_type=imasig 
appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig 
appraise func=POLICY_CHECK appraise_type=imasig 
appraise func=DIGEST_LIST_CHECK appraise_type=imasig 
dont_appraise fsmagic=0x9fa0 
dont_appraise fsmagic=0x62656572 
dont_appraise fsmagic=0x64626720 
dont_appraise fsmagic=0x858458f6 
dont_appraise fsmagic=0x1cd1 
dont_appraise fsmagic=0x42494e4d 
dont_appraise fsmagic=0x73636673 
dont_appraise fsmagic=0xf97cff8c 
dont_appraise fsmagic=0x43415d53 
dont_appraise fsmagic=0x6e736673 
dont_appraise fsmagic=0x27e0eb 
dont_appraise fsmagic=0x63677270
```

启动参数 `ima_policy=appraise_exec_immutable`  对应的 IMA 策略为：

```ini
appraise func=BPRM_CHECK appraise_type=imasig appraise_type=meta_immutable 
appraise func=MMAP_CHECK 
appraise parser appraise_type=imasig
```

#### IMA 内核编译选项详解

原生 IMA 提供的编译选项如下：

| 编译选项                         | 功能                        |
| -------------------------------- | --------------------------- |
| CONFIG\_INTEGRITY                 | IMA/EVM 总编译开关          |
| CONFIG\_INTEGRITY\_SIGNATURE       | 使能 IMA 签名校验           |
| CONFIG\_INTEGRITY\_ASYMMETRIC\_KEYS | 使能 IMA 非对称签名校验     |
| CONFIG\_INTEGRITY\_TRUSTED\_KEYRING | 使能 IMA/EVM 密钥环         |
| CONFIG\_INTEGRITY\_AUDIT           | 编译 IMA audit 审计模块     |
| CONFIG\_IMA                       | IMA 总编译开关              |
| CONFIG\_IMA\_WRITE\_POLICY          | 允许在运行阶段更新 IMA 策略 |
| CONFIG\_IMA\_MEASURE\_PCR\_IDX       | 允许指定 IMA 度量 PCR 序号  |
| CONFIG\_IMA\_LSM\_RULES             | 允许配置 LSM 规则           |
| CONFIG\_IMA\_APPRAISE              | IMA 评估总编译开关          |
| IMA\_APPRAISE\_BOOTPARAM           | 启用 IMA 评估启动参数       |
| CONFIG\_EVM                       | EVM 总编译开关              |

IMA 摘要列表扩展额外提供的编译选项如下：

| 编译选项           | 功能                      |
| ------------------ | ------------------------- |
| CONFIG\_DIGEST\_LIST | 开启 IMA 摘要列表特性开关 |

#### IMA 性能参考数据

下图对比了不开启 IMA、开启原生 IMA、开启 IMA 摘要列表特性时的性能：

![img](./figures/ima_performance.gif)

#### IMA 根证书配置

当前openEuler使用RPM密钥对IMA摘要列表进行签名，为保证IMA功能开箱可用，openEuler内核编译时默认将RPM根证书（PGP证书）导入内核。当前共包含两本PGP证书，分别为旧版本使用的OBS证书和openEuler 22.03 LTS SP4版本切换的openEuler证书：

```shell
$ cat /proc/keys | grep PGP
1909b4ad I------     1 perm 1f030000     0     0 asymmetri private OBS b25e7f66: PGP.rsa b25e7f66 []
2f10cd36 I------     1 perm 1f030000     0     0 asymmetri openeuler <openeuler@compass-ci.com> fb37bc6f: PGP.rsa fb37bc6f []
```

由于当前内核不支持导入PGP子公钥，而切换后的openEuler证书采用子密钥签名，因此openEuler内核编译前对证书进行了预处理，抽取子公钥并导入内核，具体处理流程可见内核软件包代码仓内的process\_pgp\_certs.sh脚本文件：<https://atomgit.com/src-openeuler/kernel/blob/openEuler-22.03-LTS-SP4/process_pgp_certs.sh>。

如果用户不使用IMA摘要列表功能或使用其他密钥实现签名/验签，则可将相关代码移除，自行实现内核根证书配置。

## 远程证明（鲲鹏安全库）

### 介绍

本项目开发运行在鲲鹏处理器上的基础安全软件组件，前期主要聚焦在远程证明等可信计算相关领域，使能社区安全开发者。

### 软件架构

在未使能TEE的平台上，本项目可提供平台远程证明特性，其软件架构如下图所示：

![img](./figures/RA-arch-1.png)

在已使能TEE的平台上，本项目可提供TEE远程证明特性，其软件架构如下图所示：

![img](./figures/RA-arch-2.png)

### 安装配置

1. 使用yum安装程序的rpm包，命令如下：

   ```shell
   $ yum install kunpengsecl-ras kunpengsecl-rac kunpengsecl-rahub kunpengsecl-qcaserver kunpengsecl-attester kunpengsecl-tas kunpengsecl-devel
   ```

2. 准备数据库环境：进入 `/usr/share/attestation/ras` 目录，执行 `prepare-database-env.sh` 脚本进行自动化的数据库环境配置。

3. 程序运行时依赖的配置文件有三个路径，分别为：当前路径 `./config.yaml` ，家路径 `${HOME}/.config/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)/config.yaml` ，以及系统路径 `/etc/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)/config.yaml` 。

4. （可选）如果需要创建家目录配置文件，可在安装好rpm包后，执行位于 `/usr/share/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)` 下的脚本 `prepare-ras(rac)(hub)(qca)(attester)(tas)conf-env.sh` 从而完成家目录配置文件的部署。

### 相关参数

#### RAS启动参数

命令行输入 `ras` 即可启动RAS程序。请注意，在当前目录下需要提供**ECDSA**公钥并命名为 `ecdsakey.pub` 。相关参数如下：

```text
  -H  --https         http/https模式开关，默认为https(true)，false=http
  -h  --hport         https模式下RAS监听的restful api端口
  -p, --port string   RAS监听的client api端口
  -r, --rest string   http模式下RAS监听的restful api端口
  -T, --token         生成一个测试用的验证码并退出
  -t, --test          以测试模式启动
  -v, --verbose       打印更详细的RAS运行时日志信息
  -V, --version       打印RAS版本并退出
```

**注意：**

> 1.若要使用TEE远程证明特性，需要预装由TEE团队提供的**libqca.so**库和**libteec.so**库。\
> 2.若不使用TEE远程证明特性，需要将 `${DESTDIR}/usr/share/attestation/qcaserver` 目录下的libqca.so库和libteec.so库复制到 `/usr/lib` 或 `/usr/lib64` 目录。

#### RAC启动参数

命令行输入 `sudo raagent` 即可启动RAC程序，请注意，物理TPM模块的开启需要sudo权限。相关参数如下：

```text
  -s, --server string   指定待连接的RAS服务端口
  -t, --test            以测试模式启动
  -v, --verbose         打印更详细的RAC运行时日志信息
  -V, --version         打印RAC版本并退出
  -i, --imalog          指定ima文件路径
  -b, --bioslog         指定bios文件路径
  -T, --tatest          以TA测试模式启动
```

**注意：**

> 1.若要使用TEE远程证明特性，需要以非TA测试模式启动RAC，并将待证明TA的uuid、是否使用TCB、mem\_hash和img\_hash按序放入RAC执行路径下的**talist**文件内。**talist**文件格式如下：
>
> ```text
> e08f7eca-e875-440e-9ab0-5f381136c600 false ccd5160c6461e19214c0d8787281a1e3c4048850352abe45ce86e12dd3df9fde 46d5019b0a7ffbb87ad71ea629ebd6f568140c95d7b452011acfa2f9daf61c7a
> ```
>
> 2.若不使用TEE远程证明特性，需要TA测试模式启动RAC。\
> 3.若无法使能物理TPM模块，需要以测试模式启动RAC。我们提供了一组测试模式下的平台基准值文件供RAC读取，在启动RAC前需要将 `$(DESTDIR)/etc/attestation/default_test` 目录下的文件复制到RAC运行目录下。

#### QCA启动参数

命令行输入 `${DESTDIR}/usr/bin/qcaserver` 即可启动QCA程序，请注意，这里必须要使用qcaserver的完整路径以正常启动QTA，同时需要使QTA中的CA路径参数与该路径保持相同。相关参数如下：

```text
  -C, --scenario int    设置程序的应用场景，默认为no_as场景(0)，1=as_no_daa场景，2=as_with_daa场景
  -S, --server string   指定开放的服务器地址/端口
```

#### ATTESTER启动参数

命令行输入 `attester` 即可启动ATTESTER程序。相关参数如下：

```text
  -B, --basevalue string   设置基准值文件读取路径
  -M, --mspolicy int       设置度量策略，默认为-1，需要手动指定。1=仅比对img-hash值，2=仅比对hash值，3=同时比对img-hash和hash两个值
  -S, --server string      指定待连接的服务器地址
  -U, --uuid int           指定待验证的可信应用
  -V, --version            打印程序版本并退出
  -T, --test               读取固定的nonce值以匹配目前硬编码的可信报告
```

#### TAS启动参数

命令行输入 `tas` 即可启动TAS程序。相关参数如下：

```text
  -T, --token         生成一个测试用的验证码并退出
```

**注意：**

> 1.若要启用TAS服务，需要先为TAS配置好私钥。可以按如下命令修改家目录下的配置文件：
>
> ```shell
> $ cd ${HOME}/.config/attestation/tas
> $ vim config.yaml
> # 如下DAA_GRP_KEY_SK_X和DAA_GRP_KEY_SK_Y的值仅用于测试，正常使用前请务必更新其内容以保证安全。
> tasconfig:
>  port: 127.0.0.1:40008
>  rest: 127.0.0.1:40009
>  akskeycertfile: ./ascert.crt
>  aksprivkeyfile: ./aspriv.key
>  huaweiitcafile: ./Huawei IT Product CA.pem
>  DAA_GRP_KEY_SK_X: 65a9bf91ac8832379ff04dd2c6def16d48a56be244f6e19274e97881a776543c65a9bf91ac8832379ff04dd2c6def16d48a56be244f6e19274e97881a776543c
>  DAA_GRP_KEY_SK_Y: 126f74258bb0ceca2ae7522c51825f980549ec1ef24f81d189d17e38f1773b56126f74258bb0ceca2ae7522c51825f980549ec1ef24f81d189d17e38f1773b56
> ```
>
> 之后再输入`tas`启动TAS程序。
>
> 2.在有TAS环境中，为提高QCA配置证书的效率，并非每一次启动都需要访问TAS以生成相应证书，而是通过证书的本地化存储，即读取QCA侧 `config.yaml` 中配置的证书路径，通过 `func hasAKCert(s int) bool` 函数检查是否已有TAS签发的证书保存于本地，若成功读取证书，则无需访问TAS，若读取证书失败，则需要访问TAS，并将TAS返回的证书保存于本地。

### 接口定义

#### RAS接口

为了便于管理员对目标服务器、RAS以及目标服务器上部署的TEE中的用户 TA 进行管理，本程序设计了以下接口可供调用：

| 接口                                              | 方法                        |
| ---------------------------------                 | --------------------------- |
| /                                                 | GET                         |
| /{id}                                             | GET、POST、DELETE           |
| /{from}/{to}                                      | GET                         |
| /{id}/reports                                     | GET                         |
| /{id}/reports/{reportid}                          | GET、DELETE                 |
| /{id}/basevalues                                  | GET                         |
| /{id}/newbasevalue                                | POST                        |
| /{id}/basevalues/{basevalueid}                    | GET、POST、DELETE           |
| /{id}/ta/{tauuid}/status                          | GET                         |
| /{id}/ta/{tauuid}/tabasevalues                    | GET                         |
| /{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}    | GET、POST、DELETE           |
| /{id}/ta/{tauuid}/newtabasevalue                  | POST                        |
| /{id}/ta/{tauuid}/tareports                       | GET                         |
| /{id}/ta/{tauuid}/tareports/{tareportid}          | GET、POST、DELETE           |
| /{id}/basevalues/{basevalueid}                    | GET、DELETE                 |
| /version                                          | GET                         |
| /config                                           | GET、POST                   |
| /{id}/container/status                            | GET                         |
| /{id}/device/status                               | GET                         |

上述接口的具体用法分别介绍如下。

若需要查询所有服务器的信息，可以使用`"/"`接口。

```shell
$ curl -X GET -H "Content-Type: application/json" http://localhost:40002/
```

***

若需要查询目标服务器的详细信息，可以使用`"/{id}"`接口的`GET`方法，其中{id}是RAS为目标服务器分配的唯一标识号。

```shell
$ curl -X GET -H "Content-Type: application/json" http://localhost:40002/1
```

***

若需要修改目标服务器的信息，可以使用`"/{id}"`接口的`POST`方法，其中$AUTHTOKEN是事先使用`ras -T`自动生成的身份验证码。

```go
type clientInfo struct {
 Registered   *bool `json:"registered"`  // 目标服务器注册状态
 IsAutoUpdate *bool `json:"isautoupdate"`// 目标服务器基准值更新策略
}
```

```shell
$ curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1 -d '{"registered":false, "isautoupdate":false}'
```

***

若需要删除目标服务器，可以使用`"/{id}"`接口的`DELETE`方法。

> **注意：**
> 使用该方法并非删除目标服务器的所有信息，而是把目标服务器的注册状态置为`false`！

```shell
$ curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1
```

***

若需要查询指定范围内的所有服务器信息，可以使用`"/{from}/{to}"`接口的`GET`方法。

```shell
$ curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/9
```

***

若需要查询目标服务器的所有可信报告，可以使用`"/{id}/reports"`接口的`GET`方法。

```shell
$ curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/reports
```

***

若需要查询目标服务器指定可信报告的详细信息，可以使用`"/{id}/reports/{reportid}"`接口的`GET`方法，其中{reportid}是RAS为目标服务器指定可信报告分配的唯一标识号。

```shell
$ curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/reports/1
```

***

若需要删除目标服务器指定可信报告，可以使用`"/{id}/reports/{reportid}"`接口的`DELETE`方法。\
**注意：**

> 使用该方法将删除指定可信报告的所有信息，将无法再通过接口对该报告进行查询！

```shell
$ curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/reports/1
```

***

若需要查询目标服务器的所有基准值，可以使用`"/{id}/basevalues"`接口的`GET`方法。

```shell
$ curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/basevalues
```

***

若需要给目标服务器新增一条基准值信息，可以使用`"/{id}/newbasevalue"`接口的`POST`方法。

```go
type baseValueJson struct {
 BaseType   string `json:"basetype"`   // 基准值类型
 Uuid       string `json:"uuid"`       // 容器或设备的标识号
 Name       string `json:"name"`       // 基准值名称
 Enabled    bool   `json:"enabled"`    // 基准值是否可用
 Pcr        string `json:"pcr"`        // PCR值
 Bios       string `json:"bios"`       // BIOS值
 Ima        string `json:"ima"`        // IMA值
 IsNewGroup bool   `json:"isnewgroup"` // 是否为一组新的基准值
}
```

```shell
$ curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/newbasevalue -d '{"name":"test", "basetype":"host", "enabled":true, "pcr":"testpcr", "bios":"testbios", "ima":"testima", "isnewgroup":true}'
```

***

若需要查询目标服务器指定基准值的详细信息，可以使用`"/{id}/basevalues/{basevalueid}"`接口的`GET`方法，其中{basevalueid}是RAS为目标服务器指定基准值分配的唯一标识号。

```shell
$ curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/basevalues/1
```

***

若需要修改目标服务器指定基准值的可用状态，可以使用`"/{id}/basevalues/{basevalueid}"`接口的`POST`方法。

```shell
$ curl -X POST -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40002/1/basevalues/1 -d '{"enabled":true}'
```

***

若需要删除目标服务器指定基准值，可以使用`"/{id}/basevalues/{basevalueid}"`接口的`DELETE`方法。\
**注意：**

> 使用该方法将删除指定基准值的所有信息，将无法再通过接口对该基准值进行查询！

```shell
$ curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/basevalues/1
```

***

若需要查询目标服务器上特定用户 TA 的可信状态，可以使用`"/{id}/ta/{tauuid}/status"`接口的GET方法。其中{id}是RAS为目标服务器分配的唯一标识号，{tauuid}是特定用户 TA 的身份标识号。

```shell
$ curl -X GET -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40002/1/ta/test/status
```

***

若需要查询目标服务器上特定用户 TA 的所有基准值信息，可以使用`"/{id}/ta/{tauuid}/tabasevalues"`接口的GET方法。

```shell
$ curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tabasevalues
```

***

若需要查询目标服务器上特定用户 TA 的指定基准值的详细信息，可以使用`"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"`接口的GET方法。其中{tabasevalueid}是RAS为目标服务器上特定用户 TA 的指定基准值分配的唯一标识号。

```shell
$ curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tabasevalues/1
```

***

若需要修改目标服务器上特定用户 TA 的指定基准值的可用状态，可以使用`"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"`接口的`POST`方法。

```shell
$ curl -X POST -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN"  http://localhost:40002/1/ta/test/tabasevalues/1 --data '{"enabled":true}'
```

***

若需要删除目标服务器上特定用户 TA 的指定基准值，可以使用`"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"`接口的`DELETE`方法。\
**注意：**

> 使用该方法将删除指定基准值的所有信息，将无法再通过接口对该基准值进行查询！

```shell
$ curl -X DELETE -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" -k http://localhost:40002/1/ta/test/tabasevalues/1
```

***

若需要给目标服务器上特定用户 TA 新增一条基准值信息，可以使用`"/{id}/ta/{tauuid}/newtabasevalue"`接口的`POST`方法。

```go
type tabaseValueJson struct {
 Uuid      string `json:"uuid"`       // 用户 TA 的标识号
 Name      string `json:"name"`       // 基准值名称
 Enabled   bool   `json:"enabled"`    // 基准值是否可用
 Valueinfo string `json:"valueinfo"`  // 镜像哈希值和内存哈希值
}
```

```shell
$ curl -X POST -H "Content-Type: application/json" -H "Authorization: $AUTHTOKEN" -k http://localhost:40002/1/ta/test/newtabasevalue -d '{"uuid":"test", "name":"testname", "enabled":true, "valueinfo":"test info"}'
```

***

若需要查询目标服务器上特定用户 TA 的所有可信报告，可以使用`"/{id}/ta/{tauuid}/tareports"`接口的`GET`方法。

```shell
$ curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports
```

***

若需要查询目标服务器上特定用户 TA 的指定可信报告的详细信息，可以使用`"/{id}/ta/{tauuid}/tareports/{tareportid}"`接口的`GET`方法，其中{tareportid}是RAS为目标服务器上特定用户 TA 的指定可信报告分配的唯一标识号。

```shell
$ curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports/2
```

***

若需要删除目标服务器上特定用户 TA 的指定可信报告，可以使用`"/{id}/ta/{tauuid}/tareports/{tareportid}"`接口的`DELETE`方法。\
**注意：**

> 使用该方法将删除指定可信报告的所有信息，将无法再通过接口对该报告进行查询！

```shell
$ curl -X DELETE -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports/2
```

***

若需要获取本程序的版本信息，可以使用`"/version"`接口的`GET`方法。

```shell
$ curl -X GET -H "Content-Type: application/json" http://localhost:40002/version
```

***

若需要查询目标服务器/RAS/数据库的配置信息，可以使用`"/config"`接口的`GET`方法。

```shell
$ curl -X GET -H "Content-Type: application/json" http://localhost:40002/config
```

***

若需要修改目标服务器/RAS/数据库的配置信息，可以使用`"/config"`接口的`POST`方法。

```go
type cfgRecord struct {
  // 目标服务器配置
    HBDuration      string `json:"hbduration" form:"hbduration"`
    TrustDuration   string `json:"trustduration" form:"trustduration"`
    DigestAlgorithm string `json:"digestalgorithm" form:"digestalgorithm"`
  // RAS配置
    MgrStrategy     string `json:"mgrstrategy" form:"mgrstrategy"`
    ExtractRules    string `json:"extractrules" form:"extractrules"`
    IsAllupdate     *bool  `json:"isallupdate" form:"isallupdate"`
    LogTestMode     *bool  `json:"logtestmode" form:"logtestmode"`
}
```

```shell
$ curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/config -d '{"hbduration":"5s","trustduration":"20s","DigestAlgorithm":"sha256"}'
```

#### TAS接口

为了便于管理员对TAS服务的远程控制，本程序设计了以下接口可供调用：

| 接口                | 方法               |
| --------------------| ------------------|
| /config             | GET、POST         |

若需要查询TAS的配置信息，可使用`"/config"`接口的`GET`方法：

```shell
$ curl -X GET -H "Content-Type: application/json" http://localhost:40009/config
```

***

若需要修改TAS的配置信息，可使用`"/config"`接口的`POST`方法：

```shell
curl -X POST -H "Content-Type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40009/config -d '{"basevalue":"testvalue"}' 
```

**注意：**

> TAS的配置信息读取与修改目前仅支持基准值

### FAQ

1. RAS安装后，为什么无法启动？

   > 因为在当前RAS的设计逻辑中，程序启动后需要从当前目录查找一份名为 `ecdsakey.pub` 的文件进行读取并作为之后访问该程序的身份验证码，若当前目录没有该文件，则RAS启动会报错。
   >
   > > 解决方法一：运行 `ras -T` 生成测试用token后会生成 `ecdsakey.pub` 。
   > > 解决方法二：自行部署oauth2认证服务后，将对应JWT token生成方对应的验证公钥保存为 `ecdsakey.pub` 。

2. 为什么RAS启动后，通过restapi无法访问？

   > 因为RAS默认以https模式启动，您需要向RAS提供合法的证书才能正常访问，而http模式下启动的RAS则不需要提供证书。

3. 为什么RAS/ATTESTER启动后，会出现 “WARNING: failed to verify x509 cert” 的问题？

   > 因为缺少CA证书。
   >
   > > 解决方法：将位于 `$(DESTDIR)/usr/bin` 目录下的华为IT产品证书 `Huawei IT Product CA.pem` 复制到RAS/ATTESTER的运行目录下。

## 可信平台控制模块（TPCM）

### 背景

可信计算在近40年的研究过程中，经历了不断的发展和完善，已经成为信息安全的一个重要分支。中国的可信计算技术近年发展迅猛，在可信计算2.0的基础上解决了可信体系与现有体系的融合问题、可信管理问题以及可信开发的简化问题，形成了基于主动免疫体系的可信计算技术--可信计算3.0。相对于可信计算2.0被动调用的外挂式体系结构，可信计算3.0提出了以自主密码为基础、控制芯片为支柱、双融主板为平台、可信软件为核心、可信连接为纽带、策略管控成体系、安全可信保应用的全新的可信体系框架，在网络层面解决可信问题。

可信平台控制模块（Trusted Platform Control Module，TPCM）是一种可集成在可信计算平台中，用于建立和保障信任源点的基础核心模块。它作为中国可信计算3.0中的创新点之一和主动免疫机制的核心，实现了对整个平台的主动可控。

TPCM可信计算3.0架构为双体系架构，分为防护部件和计算部件，以可信密码模块为基础，通过可信平台控制模块对防护部件和计算部件及组件的固件进行可信度量，可信软件基(Trusted Software Base，TSB)对系统软件及应用软件进行可信度量，同时TPCM管理平台实现对可信度量的验证及可信策略同步和管理。

### 功能描述

如下图所示，整体系统方案由防护部件、计算部件和可信管理中心三部分组成。

![](./figures/TPCM.png)

* 可信管理中心：对可信计算节点的防护策略和基准值进行制定、下发、维护、存储等操作的集中管理平台，可信管理中心由第三方厂商提供。

* 防护部件：独立于计算部件执行，为可信计算平台提供具有主动度量和主动控制特征的可信计算防护功能，实现运算的同时进行安全防护。防护部件包括可信平台控制模块、可信软件基，以及可信密码模块（Trusted Cryptography Module，TCM）。TPCM是可信计算节点中实现可信防护功能的关键部件，可以采用多种技术途径实现，如板卡、芯片、IP核等，其内部包含中央处理器、存储器等硬件，固件，以及操作系统与可信功能组件等软件，支撑其作为一个独立于计算部件的防护部件组件，并行于计算部件按内置防护策略工作，对计算部件的硬件、固件及软件等需防护的资源进行可信监控，是可信计算节点中的可信根。

* 计算部件：主要包括硬件、操作系统和应用层软件。其中操作系统分为引导阶段和运行阶段，在引导阶段openEuler的shim和grub2支持可信度量能力，可实现对shim、grub2以及操作系统内核、initramfs等启动文件的可信度量防护；在运行阶段，openEuler操作系统支持部署可信验证要素代理（由第三方厂商可信华泰提供），它负责将数据发送给TPCM模块，用以实现运行阶段的可信度量防护。

其中，TPCM作为可信计算节点中实现可信防护功能的关键部件，需要与TSB、TCM、可信管理中心和可信计算节点的计算部件交互，交互方式如下：

1. TPCM的硬件、固件与软件为TSB提供运行环境，设置的可信功能组件为TSB按策略库解释要求实现度量、控制、支撑与决策等功能提供支持。
2. TPCM通过访问TCM获取可信密码功能，完成对防护对象可信验证、度量和保密存储等计算任务，并提供TCM服务部件以支持对TCM的访问。
3. TPCM通过管理接口连接可信管理中心，实现防护策略管理、可信报告处理等功能。
4. TPCM通过内置的控制器和I/O端口，经由总线与计算部件的控制器交互，实现对计算部件的主动监控。
5. 计算部件操作系统中内置的防护代理获取预设的防护对象有关代码和数据提供给TPCM，TPCM将监控信息转发给TSB，由TSB依据策略库进行分析处理。

### 约束限制

适配服务器：TaiShan 200（型号2280）VF\
适配BMC插卡型号：BC83SMMC

### 应用场景

通过TPCM特性构成一个完整的信任链，保障系统启动以后进入一个可信的计算环境。

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop.md
---


---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/dde/dde_installation.md
---
# 在 openEuler 上安装 DDE

## 简介

DDE是统信软件团队研发的一款功能强大的桌面环境。包含数十款功能强大的桌面应用，是真正意义上的自主自研桌面产品。

## 安装方法

1. [下载](https://openeuler.org/zh/download/)openEuler ISO镜像并安装系统

2. 更新软件源

   ```bash
   sudo dnf update
   ```

3. 安装DDE

   ```bash
   sudo dnf install dde
   ```

4. 设置以图形界面的方式启动

   ```bash
   sudo systemctl set-default graphical.target
   ```

5. 重启

   ```bash
   sudo reboot
   ```

6. 在重启完成后，使用安装过程中创建的用户或openeuler用户登录桌面

   > ![notes](./figures/icon99-o.svg)说明：
   >
   > * dde桌面无法使用root帐号登录。
   > * dde内置了openeuler用户，此用户的密码为openeuler。

现在您可以尽情的使用dde桌面了。

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/gnome/gnome_installation.md
---
# 在 openEuler 上安装 GNOME

GNOME是运行在类Unix操作系统中最常用桌面环境。是一个功能完善、操作简单，界面友好，集使用和开发为一身的桌面环境，是GNU计划的正式桌面。

从用户的角度看，GNOME是一个集成桌面环境和应用程序的套件。从开发者的角度看，它是一个应用程序开发框架(由数目众多的实用函数库组成)。即使用户不运行GNOME桌面环境，用GNOME编写的应用程序也可以正常运行。

GNOME既包含文件管理器，应用商店，文本编辑器等基础软件，也包含系统采样分析，系统日志，软件工程IDE，web浏览器，简洁虚拟机监视器，开发者文档浏览器等高级应用和工具。

安装时，建议新建一个管理员用户。

1. [下载](https://openeuler.org/zh/download/)openEuler ISO镜像并安装系统，更新软件源（需要配置Everything源，以及EPOL源，下面命令是在最小化安装系统的情况下安装GNOME）

   ```sh
   sudo dnf update
   ```

2. 安装字库

   ```sh
   sudo dnf install dejavu-fonts liberation-fonts gnu-*-fonts google-*-fonts
   ```

3. 安装Xorg

   ```sh
   sudo dnf install xorg-*
   ```

   这可能会安装很多没用的包，可以使用下面的命令安装必要的xorg相关包

   ```sh
   sudo dnf install xorg-x11-apps xorg-x11-drivers xorg-x11-drv-ati \
   xorg-x11-drv-dummy xorg-x11-drv-evdev xorg-x11-drv-fbdev xorg-x11-drv-intel \
   xorg-x11-drv-libinput xorg-x11-drv-nouveau xorg-x11-drv-qxl \
   xorg-x11-drv-synaptics-legacy xorg-x11-drv-v4l xorg-x11-drv-vesa \
   xorg-x11-drv-vmware xorg-x11-drv-wacom xorg-x11-fonts xorg-x11-fonts-others \
   xorg-x11-font-utils xorg-x11-server xorg-x11-server-utils xorg-x11-server-Xephyr \
   xorg-x11-server-Xspice xorg-x11-util-macros xorg-x11-utils xorg-x11-xauth \
   xorg-x11-xbitmaps xorg-x11-xinit xorg-x11-xkb-utils
   ```

4. 安装GNOME及组件

   ```sh
   sudo dnf install adwaita-icon-theme atk atkmm at-spi2-atk at-spi2-core baobab \
   abattis-cantarell-fonts cheese clutter clutter-gst3 clutter-gtk cogl dconf \
   dconf-editor devhelp eog epiphany evince evolution-data-server file-roller folks \
   gcab gcr gdk-pixbuf2 gdm gedit geocode-glib gfbgraph gjs glib2 glibmm24 \
   glib-networking gmime30 gnome-autoar gnome-backgrounds gnome-bluetooth \
   gnome-builder gnome-calculator gnome-calendar gnome-characters \
   gnome-clocks gnome-color-manager gnome-contacts gnome-control-center \
   gnome-desktop3 gnome-disk-utility gnome-font-viewer gnome-getting-started-docs \
   gnome-initial-setup gnome-keyring gnome-logs gnome-menus gnome-music \
   gnome-online-accounts gnome-online-miners gnome-photos gnome-remote-desktop \
   gnome-screenshot gnome-session gnome-settings-daemon gnome-shell \
   gnome-shell-extensions gnome-software gnome-system-monitor gnome-terminal \
   gnome-tour gnome-user-docs gnome-user-share gnome-video-effects \
   gnome-weather gobject-introspection gom grilo grilo-plugins \
   gsettings-desktop-schemas gsound gspell gssdp gtk3 gtk4 gtk-doc gtkmm30 \
   gtksourceview4 gtk-vnc2 gupnp gupnp-av gupnp-dlna gvfs json-glib libchamplain \
   libdazzle libgdata libgee libgnomekbd libgsf libgtop2 libgweather libgxps libhandy \
   libmediaart libnma libnotify libpeas librsvg2 libsecret libsigc++20 libsoup \
   mm-common mutter nautilus orca pango pangomm libphodav python3-pyatspi \
   python3-gobject rest rygel simple-scan sushi sysprof tepl totem totem-pl-parser \
   tracker3 tracker3-miners vala vte291 yelp yelp-tools \
   yelp-xsl zenity
   ```

5. 启动gdm显示管理器

   ```sh
   sudo systemctl enable gdm
   ```

6. 设置系统默认以图形界面登录

   ```sh
   sudo systemctl set-default graphical.target
   ```

   重启验证

   ```sh
   sudo reboot
   ```

7. 当gdm不能工作

   如果默认安装了gdm，则停用gdm

   ```sh
   sudo systemctl disable gdm
   ```

   安装lightdm显示管理器替代

   ```sh
   sudo dnf install lightdm lightdm-gtk
   ```

   设置默认桌面为GNOME，通过root权限用户设置

   ```sh
   echo 'user-session=gnome' >> /etc/lightdm/lightdm.conf.d/60-lightdm-gtk-greeter.conf
   ```

   启动lightdm显示管理器

   ```sh
   sudo systemctl enable lightdm
   ```

   设置系统默认以图形界面登录

   ```sh
   sudo systemctl set-default graphical.target
   ```

   重启验证

   ```sh
   sudo reboot
   ```

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/kiran/kiran_installation.md
---
# 在 openEuler 上安装 Kiran

## 简介

kiran 桌面是湖南麒麟信安团队以用户和市场需求为导向，研发的一个安全、稳定、高效、易用的桌面环境。Kiran 可以支持 x86 和 aarch64 架构。

## 安装方法

安装时建议使用 root 用户或者新建一个管理员用户。

1.下载 openEuler 22.03 LTS SP4 镜像并安装系统。

2.更新软件源：

```shell
sudo dnf update
```

3.安装 kiran-desktop：

```shell
sudo dnf -y install kiran-desktop
```

4.设置以图形界面的方式启动，并重启（`reboot`）。

```shell
systemctl set-default graphical.target
```

重启系统即可通过 Kiran 桌面登录，您就可以尽情使用 Kiran 桌面了。

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/ukui/ukui_installation.md
---
# 在 openEuler 上安装 UKUI

UKUI是麒麟软件团队历经多年打造的一款Linux 桌面，主要基于 GTK 和 QT开发。与其他UI界面相比，UKUI更加注重易用性和敏捷度，各元件相依性小，可以不依赖其他套件而独自运行，给用户带来亲切和高效的使用体验。

UKUI支持x86\_64和aarch64两种架构。

安装时，建议新建一个管理员用户。

1.下载 openEuler 22.03 LTS SP4 镜像并安装系统。

```shell
sudo dnf update
```

2.安装UKUI。

```shell
sudo dnf install ukui
```

3.在确认正常安装后，如果希望以图形界面的方式启动，请在命令行运行以下代码，并重启（`reboot`）。

```shell
systemctl set-default graphical.target
```

目前UKUI版本还在不断的更新，最新的安装方法请查阅：
[openEuler UKUI Issue库](https://gitee.com/openkylin-backup/ukui-issues?_from=gitee_search#/openkylin-backup/ukui-issues/blob/master/openEuler_20.09_UKUI_install_cn.md)。

---

---
url: /zh/docs/22.03_LTS_SP4/tools/desktop/xfce/xfce_installation.md
---
# 在 openEuler 上安装 XFCE

XFCE是一款轻量级 Linux 桌面，当前版本已经将所有部件从 GTK2 更新到 GTK3，从D-Dbus Glib更新到GDBus，大部分组件支持GObject Introspection（简称 GI，用于产生与解析 C 程序库 API 元信息，以便于动态语言（或托管语言）绑定基于 C + GObject 的程序库）。优化用户体验，加入新特性，并修补大量BUG。与其他UI界面（GNOME、KDE）相比，XFCE占用的内存和CPU使用量非常小，给用户带来亲切和高效的使用体验。

XFCE支持x86\_64和aarch64两种架构。

安装时，建议新建一个管理员用户。

1. [下载](https://openeuler.org/zh/download/)openEuler ISO镜像并安装系统，更新软件源（需要配置Everything源，以及EPOL源，下面命令是在最小化安装系统的情况下安装XFCE）

   ```sh
   # sudo dnf update
   ```

2. 安装字库

   ```sh
   # sudo dnf install dejavu-fonts liberation-fonts gnu-*-fonts google-*-fonts
   ```

3. 安装Xorg

   ```sh
   # sudo dnf install xorg-*
   ```

4. 安装XFCE及组件

   ```sh
   # sudo dnf install xfwm4 xfdesktop xfce4-* xfce4-*-plugin network-manager-applet *fonts
   ```

5. 安装登录管理器

   ```sh
   # sudo dnf install lightdm lightdm-gtk
   ```

6. 设置默认桌面为XFCE
   通过root权限用户设置

   ```sh
   # echo 'user-session=xfce' >> /etc/lightdm/lightdm.conf.d/60-lightdm-gtk-greeter.conf
   ```

7. 使用登录管理器登录XFCE

   ```sh
   # sudo systemctl start lightdm
   ```

   登录管理器启动后，在右上角左侧选择"xfce-session"，输入用户名、密码登录。

8. 设置开机自启动图形界面

   ```sh
   # sudo systemctl enable lightdm
   # sudo systemctl set-default graphical.target
   ```

   如果默认安装了gdm，建议停用gdm

   ```sh
   # systemctl disable gdm
   ```

   重启验证

   ```sh
   # sudo reboot
   ```

FAQ

Q:为什么lightdm登录界面背景是黑色的？

A:登录界面是黑色的是因为lightdm-gtk默认配置文件/etc/lightdm/lightdm-gtk-greeter.conf中没有设置background。
可以在该配置文件最后的\[greeter]段中设置 background=/usr/share/backgrounds/xfce/xfce-blue.jpg
然后使用“systemctl restart lightdm”命令就可以看到背景了。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/distributed/expanding_the_ecosystem_through_distributed_soft_bus.md
---
# 基于分布式软总线扩展生态互联

## 背景简介

openEuler秉承着打造“数字化基础设施操作系统”的愿景，为促进与OpenHarmony生态的合作与互通，实现端边领域的互通和协同，首次在嵌入式领域引入分布式软总线技术。

分布式软总线是OpenHarmony社区开源的分布式设备通信基座，为设备之间的互通互联提供统一的分布式协同能力，实现设备无感发现和高效传输。OpenHarmony主要面向有强交互需求的智能终端、物联网终端和工业终端。openEuler主要面向有高可靠、高性能等需求的服务器、边缘计算、云和嵌入式设备，二者各有侧重。通过以分布式软总线为代表的技术进行生态互通，一起实现“1+1>2”的效果，支撑社区用户开拓更广阔的行业空间。

分布式软总线工作原理和流程介绍可参考：[openEuler分布式软总线](https://pages.openeuler.openatom.cn/embedded/docs/build/html/master/features/distributed_softbus.html)

## 环境准备

**硬件设备**：

| 设备名称             | 操作系统                | 说明                            |
| -------------------- | ----------------------- | ------------------------------- |
| 树莓派 4B            | openEuler22.03-LTS-SP4  | 安装openEuler操作系统的树莓派   |
| DAYU200 rk3568开发板 | OpenHarmony 3.1 Release | 安装OpenHarmony操作系统的开发板 |

**软件环境**：

| 配置项                  | 下载路径                                                     | 使用说明                                                     |
| ----------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| openEuler22.03-LTS-SP4  | <https://www.openeuler.org/zh/download/>                       | openEuler操作系统，下载raspi\_img路径中对应的image，用于烧录到树莓派，烧录可参考：[树莓派openEuler的安装](https://www.cnblogs.com/rocedu/p/14615565.html)。 |
| OpenHarmony 3.1 Release | <https://gitee.com/openharmony/manifest.git>                   | OpenHarmony操作系统，下载、编译、烧录指导：[标准系统入门](https://docs.openharmony.cn/pages/v3.1/zh-cn/device-dev/quick-start/quickstart-ide-standard-overview.md/)。|
| HUAWEI DevEco Studio    | <https://developer.harmonyos.com/cn/develop/deveco-studio#download> | OpenHarmony应用开发编辑器，使用指导：[《DevEco Studio使用指南（OpenHarmony）》](https://developer.harmonyos.com/cn/docs/documentation/doc-guides-V3/ohos-deveco-studio-overview-0000001263280421-V3?catalogVersion=V3)，本文档中代码需使用OpenHarmony SDK 9。 |

## 代码获取

该文档使用两台设备进行生态互联演示：一台树莓派（openEuler）和一台RK3568开发板（OpenHarmony）。

**openEuler侧（树莓派）代码获取**：

```shell
# openEuler侧（树莓派）操作均在树莓派中进行，下同。

# 服务端软件包已集成至openEuler22.03-LTS-SP4系统中，所以可直接使用如下命令进行安装：

dnf install dsoftbus -y

# 客户端代码可使用“项目简介”中“openEuler分布式软总线”中的“softbus_client_main.c”
```

**OpenHarmony侧（RK3568开发板）代码获取**：

```shell
# OpenHarmony侧（RK3568开发板）操作均在windows机器上进行，下同。

# 分布式软总线服务端能力已集成到OpenHarmony 3.1 Release镜像中，所以无需手动获取服务端代码。

# 客户端代码使用“Native C++”工程编写，可使用git命令下载参考代码。代码说明在仓库的README.md中。

git clone https://gitee.com/liheavy/softbus_client_app.git
```

## 代码编译

**openEuler侧（树莓派）代码编译**：

```shell
# 安装dsoftbus后，将softbus_client_main.c保存到任意路径下，使用如下gcc命令进行编译,编译成功后会在路径下生成softbus_client_main可执行文件，添加执行权限:

gcc softbus_client_main.c -I/usr/include/dsoftbus/ -L /usr/lib64 -lsoftbus_client.z -lboundscheck -o softbus_client_main

chmod u+x softbus_client_main
```

**OpenHarmony侧（RK3568开发板）代码编译**：

* 首先需要根据“环境准备”中 HUAWEI DevEco Studio 配置指导将 IDE 环境配置完成。

* 使用 DevEco 将下载的"softbus\_client\_app/softbus\_client\_sample"参考代码打开，DevEco 将会根据配置文件自动完成项目初始化。

  初始化完成后需要生成项目的签名文件：File -->Project Structure --> Project --> Signing Configs -->Automatically generate signature。

  ![image-20221201101959764](figures/image-20221201101959764.png)

* 由于 OpenHarmony SDK 9 暂未提供分布式软总线的应用 API，所以该项目直接在应用中使用分布式软总线服务的 API ，需要将依赖的 so文件（libsoftbus\_client.z.so、libsec\_shared.z.so）拷贝到本地OpenHarmony SDK 的动态库路径中。so 文件会随源码一起下载到本地(required\_so文件夹中)；也可通过hdc\_std工具拷贝RK3568中的两个 so 文件，so 文件路径/lib/xxx.so。

  本地OpenHarmony SDK安装路径可通过File --> Settings --> OpenHarmony SDK查看。

  ![image-20221201103709581](figures/image-20221201103709581.png)

  将libsoftbus\_client.z.so、libsec\_shared.z.so拷贝到 “OpenHarmony SDK安装路径/native/x.x.x.x/sysroot/usr/lib/arm-linux-ohos/"路径中。

  完成后便可在DevEco中进行代码编译 Build --> Build Hap(s)/App(s)  --> Build Hap(s) 。

  ![image-20221201111227158](figures/image-20221201111227158.png)

## 代码运行

### 前置准备

**配置设备ID**

**openEuler侧（树莓派）**：

由于当前openEuler版本的分布式软总线暂时去除了对设备管理模块的依赖，为适配和简化获取设备ID的代码，采用了读取/etc/SN文件中内容作为设备ID的方式，所以在启动分布式软总线服务侧之前需要将设备ID（自定义数字）写入/etc/SN，注意多个设备的ID不要设置成同一个值，否则可能造成认证和通信失败问题。

```shell
# 举例设置该设备ID为1
echo "1" >>/etc/SN
```

**连接网络**

两台设备需要在同一局域网内，两台设备的 ip可以相互 ping 通，可在各自操作系统中使用ifconfig查看ip。

**openEuler侧（树莓派）**：

* 启动分布式软总线服务

  ```shell
  # dsoftbus会提供softbus_server_main命令，可直接执行

  softbus_server_main
  ```

**OpenHarmony侧（RK3568开发板）**：

* 电脑连接 RK3568 开发板

  使用 USB 线一端连接 RK3568 开发板的 “OTG” 类型的 USB 接口，一端连接电脑的 USB 接口。打开 DevEco，界面右上方设备列表识别到开发板即为连接成功。

  ![image-20221201115003525](figures/image-20221201115003525.png)

* 修改分布式软总线权限配置文件

  由于 OpenHarmony 分布式软总线服务对使用其功能的应用有权限限制。故运行 demo 需要修改分布式软总线权限配置文件。

  ```shell
  # 可使用hdc_std工具进行权限配置文件替换，hdc_std工具会在OpenHarmony SDK安装时一起安装，安装路径为：

  OpenHarmony SDK安装路径（可根据上一章指导，通过DevEco查看）/toolchains/x.x.x.x/hdc_std.exe

  # 由于RK3568中分布式软总线权限配置文件为只读模式，所以需要使用如下命令将系统文件修改为可读可写模式：

  hdc_std.exe shell "mount -o remount,rw /"

  # 如下使用softbus_client_app仓库中softbus_trans_permission.json替换RK3568中对应的文件。

  hdc_std.exe file send softbus_trans_permission.json /system/etc/communication/softbus/
  ```

* 重启分布式软总线服务

  ```shell
  # 使用如下命令重启分布式软总线服务，使修改的权限配置文件生效。

  ps -ef | grep softbus_server | grep -v grep

  kill -9 上面查询到的进程号
  ```

### 设备认证

OpenHarmony 设备和 openEuler 设备在组网过程中，需要两台设备提前将对方添加为可信设备。所以在两台设备互联互通之前，需要通过 Hichain 模块进行认证。详情请参考[可信设备添加](https://pages.openeuler.openatom.cn/embedded/docs/build/html/master/features/distributed_softbus.html#id7)，如下使用“pin码弹窗”方式认证的demo演示，以下操作均在树莓派中进行操作。

* 依赖动态库软链接

  ```shell
  # 该方式需要依赖libsec_shared.z.so，该动态库在openEuler的dsoftbus中被替换为libboundscheck.so，所以需要通过建立软链接的方式使用libsec_shared.z.so。
  ln -s /usr/lib64/libboundscheck.so /usr/lib64/libsec_shared.z.so

  # 在ARM环境下该方法还需要依赖/lib64/1d-linux-aarch64.so.1，否则会出现no such file的报错。
  ln -s /lib/ld-linux-aarch64.so.1 /lib64/ld-linux-aarch64.so.1
  ```

* 运行认证客户端

  ```shell
  # 认证客户端demo可执行文件同样在softbus_client_app仓库中，hichain_sample—>devicemanager

  chmod u+x devicemanager
  ./devicemanager
  ```

  根据提示输入“l”列出同一局域网内设备。

  ![image-20221201145904897](figures/image-20221201145904897.png)

  根据提示输入 OpenHarmony 设备编号进行认证，输入后在 RK3568 开发板中会弹出手动确认弹窗“是否允许对端设备连接”，点击“允许”，在树莓派侧再输入“l”，会发现设备的状态从“discovery”变成了“online”，表示认证成功。

  ![image-20221201150148988](figures/image-20221201150148988.png)

### 设备通信

* OpenHarmony侧（RK3568开发板）客户端烧录、运行。

  电脑连接 RK3568 开发板，点击 DevEco 上方的“Run”按钮。

  ![image-20221201151951393](figures/image-20221201151951393.png)

  烧录结束后 RK3568 将直接运行 APP。

  ![image-20221130193253259](figures/image-20221130193253259.png)

* openEuler侧（树莓派）发送数据

  ```shell
  # 直接运行“代码编译”阶段编译出的softbus_client_main可执行文件

  ./softbus_client_main
  ```

  运行后会显示组网设备：

  ![image-20221201162316137](figures/image-20221201162316137.png)

  根据提示输入“c”：

  ![image-20221201163115659](figures/image-20221201163115659.png)

  根据提示输入OpenHarmony设备的编号：

  ![image-20221201163310745](figures/image-20221201163310745.png)

​  此时在 RK3568 开发板屏幕上方将显示接收到的字符串：

![](figures/接收数据.png)

* OpenHarmony侧（RK3568开发板）展示终端信息：

  点击app页面中的“刷新”图标，会显示与该设备连接的终端设备，点击终端图标将会显示设备基本信息。

  ![终端信息](figures/终端信息.png)

* OpenHarmony侧（RK3568开发板）发送数据：

  点击app页面“发送数据”按钮，将会向openEuler设备发送一条数据。

![发送数据](figures/发送数据.png)

​在openEuler侧（树莓派），将会展示接收到的信息。

![image-20221201165948378](figures/image-20221201165948378.png)

​至此，openEuler和OpenHarmony通过分布式软总线互联互通demo演示完成。

## 总结

该文档主要演示了 OpenHarmony 设备和 openEuler 设备如何通过分布式软总线进行互联互通。服务端代码正在持续优化，以支持更多分布式场景；客户端所用代码均属于 demo，仍需优化：如OpenHarmony APP调用分布式软总线的方式，标准流程为开发 System Ability 来调用分布式软总线服务端 API，集成到 SDK 中并提供 API 给APP 进行调用，由于时间关系采用 APP 直接调用分布式软总线 API 的方式，欢迎各位开发爱好者参与，一起丰富openEuler的嵌入式能力。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/distributed/dsoftbus_application_based_on_containers.md
---
# 基于容器的软总线应用指南

## 背景简介

当前用户软件迁向容器化已经是大势所趋。本文档介绍了基于容器部署软总线的使用指导，简化软总线客户端的安装部署，以及方便地与业务软件兼容。

## 环境准备

**硬件设备**：

| 设备名称             | 操作系统                | 说明                            | 数量 |
| -------------------- | ----------------------- | ------------------------------- | --- |
| 树莓派 4B            | openEuler 22.03-LTS-SP4  | 安装 openEuler 操作系统的树莓派   | 2 |

## 代码获取

<https://atomgit.com/openeuler/distributed-codelabs.git>

## 说明

### 服务安装

以下步骤在设备 A/B 上都需要操作。

1. 物理机环境安装 openEuler 22.03 LTS SP2 以上版本，环境需要准备好 binder 驱动 [binder驱动安装指导](https://atomgit.com/src-openeuler/communication_ipc/blob/openEuler-22.03-LTS-SP2/README.md)。

2. 编译 softbus\_client。

   ```sh
   bash distributed-codelabs/build-repo/demo/dsoftbus/build.sh
   ```

3. 安装 softbus\_server。

   ```sh
   dnf install dsoftbus -y
   ```

4. 启动 softbus\_server。

   ```sh
   /system/bin/start_services.sh all
   ```

### 容器多 client 场景测试

1. 执行脚本在物理机A打包加载容器镜像。

   ```sh
   bash distributed-codelabs/build-repo/demo/dsoftbus/docker_img_build.sh
   ```

2. 在物理机A启动容器镜像 softbus\_client，并将相关SDK、binder驱动映射到容器中。

   ```sh
   docker run  -it  --privileged --net=host --name=softbus  -v /dev/binderfs/binder:/dev/binder  -v  /system:/system  -v /usr/lib64:/usr/lib64  -p 5684:5684/udp softbus_client_image  bash
   ```

3. 镜像中写`/etc/SI`文件，作为该容器中 softclient 的标志，注意需要和组网中其他 client 的不一致，避免冲突。

   ```sh
   echo 123 > /etc/SI
   ```

4. 容器中启动 softbus\_client。

   ```sh
   ./home/softbus_client
   ```

5. 在本节点上可重复步骤2-3启动不同的 client 容器。

6. 在物理机B启动 softbus。

   ```sh
   ./build-repo/demo/dsoftbus/softbus_client
   ```

7. 在设备A中的各个中 client 分别执行命令打开所有连接。

   ```sh
   openA
   ```

8. 在设备B中 client 查看所有已经打开的 session。

   ```sh
   conDevices
   ```

   回显说明，已经打开了两个 session，id 分别是4、3，此 sessionid 是本机的 softbus\_server 分配的，后续发送消息需要用到该 id。

   ```sh
   conDevices
   12-11 20:18:58.350 2780335 2780335 I A0fffe/SOFTBUS_DEMO: [SOFTBUS_DEMO]::PrintConnectedDevicesInfo: sessionId:4, networkId: 3c95f61941b81c48ecd73fef881262b82fcbc58e9b1f545e2097b0dc6fecea37
   12-11 20:18:58.350 2780335 2780335 I A0fffe/SOFTBUS_DEMO: [SOFTBUS_DEMO]::PrintConnectedDevicesInfo: sessionId:3, networkId: 3c95f61941b81c48ecd73fef881262b82fcbc58e9b1f545e2097b0dc6fecea37
   ```

9. 设备B发送使用“session 4”消息至设备A的 client。

   ```sh
   send 4 "hello4"
   12-11 20:19:14.975 2780335 2780335 I C015c0/dsoftbus: [TRAN]SendBytes: sessionId=4
   ```

   观察设备A中哪个 client 有回显，说明设备B使用 session 4与该 client 通信，同理可以使用 session 3 发送消息至另一个 client。

10. 设备A容器中 client 发送使用消息至设备B的 client，查看打开的 session id。

    ```sh
    conDevices
    12-11 20:25:48.995 344047 344047 I A0fffe/SOFTBUS_DEMO: [SOFTBUS_DEMO]::PrintConnectedDevicesInfo: sessionId:1, networkId: e69eab4e2d657264dfbb2006fdfa15524f4a27edeff0baa26d5d2a2b9502f300
    ```

    说明使用 session 1 通信。

11. 发送消息测试，在设备B中接受到该字段说明消息接收成功。

    ```sh
    send 1 "hello1"
    ```

12. 设备A物理机上 client 发送使用消息至设备B的 client，查看打开的 session id。

    ```sh
    conDevices
    12-11 20:37:24.823 3512580 3512580 I A0fffe/SOFTBUS_DEMO: [SOFTBUS_DEMO]::PrintConnectedDevicesInfo: sessionId:1, networkId: e69eab4e2d657264dfbb2006fdfa15524f4a27edeff0baa26d5d2a2b9502f300
    ```

    说明使用 session 1 通信。

13. 发送消息测试，在设备B中接受到该字段说明消息接收成功。

    ```sh
    send 1 "hello1"
    ```

### session id 说明

在上个章节的测试中，发现两个 client 之间通信使用的 session id 不一样。原因是 session id 是本机 server 分配的，只需要在本机范围内互斥。同样容器与虚机也做了 session name，group name 的隔离，所以本机容器与虚机上 client 的 session id 是一致的。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/administration/administrator/basic_configuration.md
---
# 基础配置

## 设置语言环境

您可以通过localectl修改系统的语言环境，对应的参数设置保存在/etc/locale.conf文件中。这些参数会在系统启动过程中被systemd的守护进程读取。

### 显示当前语言环境状态

显示当前语言环境，命令如下：

```shell
$ localectl status
```

例如显示系统当前的设置，命令和输出如下：

```shell
$ localectl status
   System Locale: LANG=zh_CN.UTF-8
       VC Keymap: cn
      X11 Layout: cn
```

### 列出可用的语言环境

显示当前可用的语言环境，命令如下：

```shell
$ localectl list-locales
```

例如显示当前系统中所有可用的中文环境，命令和输出如下：

```shell
$ localectl list-locales | grep zh
zh_CN.UTF-8
```

### 设置语言环境

要设置语言环境，在root权限下执行如下命令，其中  *locale*  是您要设置的语言类型，取值范围可通过**localectl list-locales**获取，请根据实际情况修改。

```shell
# localectl set-locale LANG=locale
```

例如设置为简体中文语言环境，在root权限下执行如下命令：

```shell
# localectl set-locale LANG=zh_CN.UTF-8
```

> \[!NOTE]说明\
> 修改后需要重新登录或者在root权限下执行`source /etc/locale.conf`命令刷新配置文件，使修改生效。

## 设置键盘

您可以通过localectl修改系统的键盘设置，对应的参数设置保存在/etc/locale.conf文件中。这些参数，会在系统启动的早期被systemd的守护进程读取。

### 显示当前设置

显示当前键盘设置，命令如下：

```shell
$ localectl status
```

例如显示系统当前的设置，命令和输出如下：

```shell
$ localectl status
   System Locale: LANG=zh_CN.UTF-8
       VC Keymap: cn
      X11 Layout: cn
```

### 列出可用的键盘布局

显示当前可用的键盘布局，命令如下：

```shell
$ localectl list-keymaps
```

例如显示系统当前的中文键盘布局，命令和输出如下：

```shell
$ localectl list-keymaps | grep cn
cn
```

### 设置键盘布局

设置键盘布局，在root权限下执行如下命令，其中  *map*  是您想要设置的键盘类型，取值范围可通过**localectl list-keymaps**获取，请根据实际情况修改：

```shell
# localectl set-keymap map
```

此时设置的键盘布局同样也会应用到图形界面中。

设置完成后，查看当前状态：

```shell
$ localectl status
   System Locale: LANG=zh_CN.UTF-8
       VC Keymap: cn
      X11 Layout: us
```

## 设置日期和时间

本节介绍如何通过timedatectl、date、hwclock命令来设置系统的日期、时间和时区等。

### 使用timedatectl命令设置

#### 显示日期和时间

显示当前的日期和时间，命令如下：

```shell
$ timedatectl
```

例如显示系统当前的日期和时间，命令和输出如下：

```shell
$ timedatectl
               Local time: Mon 2023-05-20 04:05:00 EDT
           Universal time: Mon 2023-05-20 08:05:00 UTC
                 RTC time: Mon 2023-05-20 08:05:00
                Time zone: China Standard Time (CST), UTC +8
System clock synchronized: no
              NTP service: inactive
          RTC in local TZ: no
```

#### 通过远程服务器进行时间同步

您可以启用NTP远程服务器进行系统时钟的自动同步。是否启用NTP，可在root权限下执行如下命令进行设置。其中  *boolean*  可取值yes和no，分别表示启用和不启用NTP进行系统时钟自动同步，请根据实际情况修改。

> \[!NOTE]说明\
> 若启用了NTP远程服务器进行系统时钟自动同步，则不能手动修改日期和时间。若需要手动修改日期或时间，则需确保已经关闭NTP系统时钟自动同步。可执行**timedatectl set-ntp no** 命令进行关闭。

```shell
# timedatectl set-ntp boolean
```

例如开启自动远程时间同步，命令如下：

```shell
# timedatectl set-ntp yes
```

#### 修改日期

> \[!NOTE]说明\
> 修改日期前，请确保已经关闭NTP系统时钟自动同步。

修改当前的日期，在root权限下执行如下命令，其中  *YYYY*  代表年份，*MM*  代表月份，*DD*  代表某天，请根据实际情况修改：

```shell
# timedatectl set-time YYYY-MM-DD
```

例如修改当前的日期为2019年8月14号，命令如下：

```shell
# timedatectl set-time '2019-08-14'
```

#### 修改时间

修改时间前，请确保已经关闭NTP系统时钟自动同步。查看NTP服务是否运行，命令如下：

```shell
systemctl status ntpd
```

关闭NTP服务，命令如下：

```shell
systemctl disable ntpd
```

修改当前的时间，在root权限下执行如下命令，其中  *HH*  代表小时，*MM*  代表分钟，*SS*  代表秒，请根据实际情况修改：

```shell
# timedatectl set-time HH:MM:SS
```

例如修改当前的时间为15点57分24秒，命令如下：

```shell
# timedatectl set-time 15:57:24
```

#### 修改时区

显示当前可用时区，命令如下：

```shell
$ timedatectl list-timezones
```

要修改当前的时区，在root权限下执行如下命令，其中  *time\_zone*  是您想要设置的时区，请根据实际情况修改：

```shell
# timedatectl set-timezone time_zone
```

例如修改当前的时区，首先查询所在地域的可用时区，此处以Asia为例：

```shell
# timedatectl list-timezones | grep Asia
Asia/Aden
Asia/Almaty
Asia/Amman
Asia/Anadyr
Asia/Aqtau
Asia/Aqtobe
Asia/Ashgabat
Asia/Baghdad
Asia/Bahrain
……

Asia/Seoul
Asia/Shanghai
Asia/Singapore
Asia/Srednekolymsk
Asia/Taipei
Asia/Tashkent
Asia/Tbilisi
Asia/Tehran
Asia/Thimphu
Asia/Tokyo
```

然后修改当前的时区为“Asia/Shanghai”，命令如下：

```shell
# timedatectl set-timezone Asia/Shanghai
```

### 使用date命令设置

#### 显示当前的日期和时间

显示当前的日期和时间，命令如下：

```shell
$ date
```

默认情况下，date命令显示本地时间。要显示UTC时间，添加--utc或-u参数：

```shell
$ date --utc
```

要自定义对应的输出信息格式，添加 +"format" 参数：

```shell
$ date +"format"
```

**表 1**  参数说明

实际使用示例如下：

* 显示当前的日期和本地时间。

  ```shell
  $ date 
  2023年 05月 20日 星期六 17:26:34 CST
  ```

* 显示当前的日期和UTC时间。

  ```shell
  $ date --utc
  2023年 05月 20日 星期六 09:26:18 UTC
  ```

* 自定义date命令的输出。

  ```shell
  $ date +"%Y-%m-%d %H:%M"
  2023-05-20 17:24
  ```

#### 修改时间

要修改当前的时间，请添加--set或者-s参数。在root权限下执行如下命令，其中  *HH*  代表小时，*MM*  代表分钟，*SS*  代表秒，请根据实际情况修改：

```shell
# date --set HH:MM:SS
```

默认情况下， date命令设置本地时间。要设置UTC时间，添加--utc或-u参数：

```shell
# date --set HH:MM:SS --utc
```

例如修改当前的时间为23点26分00秒，在root权限下执行如下命令：

```shell
# date --set 23:26:00
```

#### 修改日期

修改当前的日期，添加--set或者-s参数。在root权限下执行如下命令，其中  *YYYY*  代表年份，*MM*  代表月份，*DD*  代表某天，请根据实际情况修改：（注意，执行修改日期操作后，相应的时间会重置为00:00:00）

```shell
# date --set YYYY-MM-DD
```

例如修改当前的日期为2019年11月2日，命令如下：

```shell
# date --set 2019-11-02
```

### 使用hwclock命令设置

可以使用 hwclock 命令设置硬件时钟RTC (Real Time Clock) 。

#### 硬件时钟和系统时钟

Linux 将时钟分为：

* 系统时钟 (System Clock) ：当前Linux Kernel中的时钟。
* 硬件时钟 RTC：主板上由电池供电的主板硬件时钟，该时钟可以在BIOS的 "Standard BIOS Feature" 项中进行设置。

当Linux启动时，会读取硬件时钟，并根据硬件时间来设置系统时间。

#### 显示日期和时间

显示当前硬件的日期和时间，在root权限下执行如下命令：

```shell
# hwclock
```

例如显示当前硬件的日期和时间，命令和输出如下：

```shell
# hwclock
2023-05-26 10:18:42.528948+08:00
```

#### 设置日期和时间

修改当前硬件的日期和时间，在root权限下执行如下命令，其中  *dd*  表示日，*mm*  表示月份，*yyyy*  表示年份，*HH*  表示小时，*MM*  表示分钟，请根据实际情况修改：

```shell
# hwclock --set --date "dd mm yyyy HH:MM"
```

例如修改当前的时间为2023年05月21日21点17分，命令如下：

```shell
# hwclock --set --date "21 May 2023 21:17" --utc
```

## 设置kdump

本节介绍如何设置kdump预留内存及修改kdump配置文件参数。

### 设置kdump预留内存

#### 预留内存参数格式

kdump预留内存参数必须添加到内核启动参数中，配置文件为/boot/efi/EFI/openEuler/grub.cfg(UEFI引导模式)或/boot/grub2/grub.cfg(legacy引导模式)，openEuler发布版本中默认已经添加，可以根据实际使用情况调整。添加和修改启动参数后，重启系统生效。kdump预留内存参数格式如下：

| 内核启动参数             | 描述                                        | 默认值           | 备注                                                         |
|--------------------|-------------------------------------------|---------------|------------------------------------------------------------|
| crashkernel=x      | 在4G以下的物理内存预留x大小的内存给kdump使用。               | x86版本默认配置512M | 该配置方法只在4G以下内存预留，必须保证4G以下连续可用内存足够预留。                        |
| crashkernel=x@y    | 在y起始地址预留x大小的内存给kdump使用。                   | 未使用           | 需要确保y起始地址的x大小的内存未被其他模块预留。                                  |
| crashkernel=x,high | 在4G以下的物理内存中预留256M内存，在4G以上预留x大小内存给kdump使用。 | arm64版本默认配置1024M,high           | 需要确保4G以下有256M连续可用的物理内存，4G以上有连续的x大小的连续物理内存。实际预留内存大小为256M+x。 |
| crashkernel=x,low crashkernel=y,high | 在4G以下的物理内存中预留x大小，在4G以上预留y大小内存给kdump使用。                                          | 未使用              | 需要确保4G以下有连续的x大小物理内存，4G以上有连续的y大小物理内存。                                                           |

### 预留内存推荐值

| 推荐方案 | 预留参数                   | 参数说明                                         |
|------|------------------------|----------------------------------------------|
| 通用方案 | crashkernel=2048M,high | 4G以下预留256M，4G以上预留2048M内存给kdump使用。共256+2048M。 |
| 经济方案 | crashkernel=1024M,high | 4G以下预留256M，4G以上预留1024M内存给kdump使用。共256+1024M。 推荐系统512M内存以内的场景，并不使用网络转储kdump文件。对于虚拟机场景，可以适当减少内存预留值，推荐虚拟机设置为crashkernel=512M或者crashkernel=256M,high。 |

> \[!NOTE]说明\
> 不通过网络转储kdump文件时，需要设置kdump文件系统不打包网络相关驱动。网络驱动加载需要申请较大内存，可能导致预留内存不足，kdump失败。因此建议禁用网络相关驱动。

### 禁用网络相关驱动

kdump配置文件（/etc/kdump.conf）中，dracut参数可以设置裁剪的驱动模块，可以将网络驱动配置到裁剪驱动列表中，让kdump文件系统中不加载该驱动，修改配置文件后，重启kdump服务生效。dracut参数配置如下所示：

```Conf
dracut_args --omit-drivers "mdio-gpi usb_8dev et1011c rt2x00usb bcm-phy-lib mac80211_hwsim rtl8723be rndis_host hns3_cae amd vrf rtl8192cu mt76x02-lib int51x1 ppp_deflate team_mode_loadbalance smsc911x aweth bonding mwifiex_usb hnae dnet rt2x00pci vaser_pci hdlc_ppp marvell rtl8xxxu mlxsw_i2c ath9k_htc rtl8150 smc91x cortina at803x rockchip cxgb4 spi_ks8995 mt76x2u smsc9420 mdio-cavium bnxt_en ch9200 dummy macsec ice mt7601u rtl8188ee ixgbevf net1080 liquidio_vf be2net mlxsw_switchx2 gl620a xilinx_gmii2rgmii ppp_generic rtl8192de sja1000_platform ath10k_core cc770_platform realte igb c_can_platform c_can ethoc dm9601 smsc95xx lg-vl600 ifb enic ath9 mdio-octeon ppp_mppe ath10k_pci cc770 team_mode_activebackup marvell10g hinic rt2x00lib mlx4_en iavf broadcom igc c_can_pci alx rtl8192se rtl8723ae microchip lan78xx atl1c rtl8192c-common almia ax88179_178a qed netxen_nic brcmsmac rt2800usb e1000 qla3xxx mdio-bitbang qsemi mdio-mscc-miim plx_pci ipvlan r8152 cx82310_eth slhc mt76x02-usb ems_pci xen-netfront usbnet pppoe mlxsw_minimal mlxsw_spectrum cdc_ncm rt2800lib rtl_usb hnae3 ath9k_common ath9k_hw catc mt76 hns_enet_drv ppp_async huawei_cdc_ncm i40e rtl8192ce dl2 qmi_wwan mii peak_usb plusb can-dev slcan amd-xgbe team_mode_roundrobin ste10Xp thunder_xcv pptp thunder_bgx ixgbe davicom icplus tap tun smsc75xx smsc dlci hns_dsaf mlxsw_core rt2800mmi softing uPD60620 vaser_usb dp83867 brcmfmac mwifiex_pcie mlx4_core micrel team macvlan bnx2 virtio_net rtl_pci zaurus hns_mdi libcxgb hv_netvsc nicvf mt76x0u teranetics mlxfw cdc_eem qcom-emac pppox mt76-usb sierra_net i40evf bcm87xx mwifiex pegasus rt2x00mmi sja1000 ena hclgevf cnic cxgb4vf ppp_synctty iwlmvm team_mode_broadcast vxlan vsockmon hdlc_cisc rtl8723-common bsd_comp fakelb dp83822 dp83tc811 cicada fm10 8139t sfc hs geneve hclge xgene-enet-v2 cdc_mbim hdlc asix netdevsim rt2800pci team_mode_random lxt ems_usb mlxsw_pci sr9700 mdio-thunder mlxsw_switchib macvtap atlantic cdc_ether mcs7830 nicpf mdi peak_pci atl1e cdc_subset ipvtap btcoexist mt76x0-common veth slip iwldvm bcm7xxx vitesse netconsole epic100 myri10ge r8169 qede microchip_t1 liquidi bnx2x brcmutil mwifiex_sdi mlx5_core rtlwifi vmxnet3 nlmon hns3 hdlc_raw esd_usb2 atl2 mt76x2-common iwlwifi mdio-bcm-unimac national ath rtwpci rtw88 nfp rtl8821ae fjes thunderbolt-net 8139cp atl1 mscc vcan dp83848 dp83640 hdlc_fr e1000e ipheth net_failover aquantia rtl8192ee igbvf rocker intel-xway tg3" --omit "ramdisk network ifcfg qemu-net" --install "chmod" --nofscks
```

## 设置磁盘调度算法

本节介绍如何设置磁盘调度算法。

### 临时修改调度策略

例如将所有IO调度算法修改为mq-deadline，此修改重启后会失效。

```shell
echo mq-deadline > /sys/block/sd*/queue/scheduler
```

### 永久设置调度策略

可以通过在内核启动配置文件grub.cfg中的kernel行追加：elevator=mq-deadline，重启后生效。

```text
linux   /vmlinuz-5.10.0-153.12.0.89.oe2203SP4.x86_64 root=/dev/mapper/openeuler-root ro resume=/dev/mapper/openeuler-swap rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap quiet crashkernel=512M elevator=mq-deadline
```

---

---
url: /zh/docs/22.03_LTS_SP4/tools/security.md
---


---

---
url: /zh/docs/22.03_LTS_SP4/server/security/secharden/security_hardening_tool.md
---
# 安全加固工具

## 加固操作

### 概述

安全加固工具会根据usr-security.conf设置加固策略，使用加固工具设置加固策略需要用户修改usr-security.conf。本节介绍usr-security.conf的修改规则。用户可配置的加固项请参见[加固指导](./account_passwords.md)对应内容。

### 注意事项

* 修改配置后，需要重启安全加固服务使配置生效。重启方法请参见[加固生效](#加固生效)对应内容。
* 用户修改加固配置时，仅修改/etc/openEuler\_security/usr-security.conf文件，不建议修改/etc/openEuler\_security/security.conf。security.conf中为基本加固项，仅运行一次。
* 当重启安全加固服务使配置生效后，在usr-security.conf中删除对应加固项并重启安全加固服务并不能清除之前已生效的配置。
* 安全加固操作记录在日志文件/var/log/openEuler-security.log中。

### 配置格式

usr-security.conf中的每一行代表一项配置，根据配置内容的不同有不同配置格式，这里给出各类配置的格式说明。

> \[!NOTE]说明
>
> * 所有配置项以执行ID开头，执行ID仅为了方便用户识别配置内容，取值为正整数，由用户自行定义。
> * 配置项的各内容之间使用@作为分隔符。
> * 若实际配置内容中包含@，需要使用@@表示以和分隔符区分，例如实际内容为xxx@yyy，则配置为xxx@@yyy。目前不支持@位于配置内容的开头和结尾。

* d：注释。

  格式：执行ID@d@对象文件@匹配项。

  功能：将对象文件中以匹配项开头（行首可以有空格）的行注释（在行首添加#）。

  示例：执行ID为401，注释/etc/sudoers文件中以%wheel开头的行。

  ```sh
  401@d@/etc/sudoers@%wheel
  ```

* m：替换。

  格式：执行ID@m@对象文件@匹配项@替换目标值。

  功能：将对象文件中以匹配项开头（行首可以有空格）的行替换为“匹配项加替换目标值 ”。若匹配行开头有空格，替换后将删除这些空格。

  示例：执行ID为101，将/etc/ssh/sshd\_config文件中以Protocol 开头的行替换为Protocol 2。匹配和替换时也会考虑Protocol后的空格。

  ```sh
  101@m@/etc/ssh/sshd_config@Protocol @2
  ```

* sm：精确修改。

  格式：执行ID@sm@对象文件@匹配项@替换目标值。

  功能：将对象文件中以匹配项开头（行首可以有空格）的行替换为“匹配项加替换目标值 ”。若匹配行开头有空格，替换后将保留这些空格，这是sm和m的区别。

  示例：执行ID为201，将/etc/audit/hzqtest文件中以size开头的行替换为size 2048。

  ```h
  201@sm@/etc/audit/hzqtest@size@ 2048
  ```

* M：修改子项。

  格式：执行ID@M@对象文件@匹配项@匹配子项\[@匹配子项的值]。

  功能：匹配对象文件中以匹配项开头（行首可以有空格）的行，并将该行中以匹配子项开始的内容替换为“匹配子项和匹配子项的值”，其中匹配子项的值可选。

  示例：执行ID为101，找到file文件中以key开头的行，并将这些行中以key2开始的内容替换为key2value2。

  ```sh
  101@M@file@key@key2@value2
  ```

* systemctl：管理服务。

  格式：执行ID@systemctl@对象服务@具体操作。

  功能：使用systemctl管理对象服务，具体操作可取值为start、stop、restart、disable等systemctl所有可用的命令。

  示例：执行ID为218，停止cups.service服务，等同于systemctl stop cups.service的配置行。

  ```sh
  218@systemctl@cups.service@stop
  ```

* 其他命令。

  格式：执行ID@命令@对象文件。

  功能：执行对应命令，即执行命令行“命令 对象文件”。

  示例一：执行ID为402，使用rm -f命令删除文件/etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem。

  ```sh
  402@rm -f @/etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem
  ```

  示例二：执行ID为215，使用touch命令创建文件/etc/cron.allow。

  ```sh
  215@touch @/etc/cron.allow
  ```

  示例三：执行ID为214，使用chown命令将文件/etc/at.allow的属主改为root:root。

  ```sh
  214@chown root:root @/etc/at.allow
  ```

  示例四：执行ID为214，使用chmod命令去除文件/etc/at.allow属主所在群组及其他非属主用户的rwx权限。

  ```sh
  214@chmod og-rwx @/etc/at.allow
  ```

## 加固生效

完成修改usr-security.conf文件后，请运行如下命令使新添加的配置生效。

```sh
systemctl restart openEuler-security.service
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/security/cert_signature/secure_boot.md
---
# 安全启动

## 概述

安全启动（Secure Boot）就是利用公私钥对启动部件进行签名和验证。在启动过程中，前一个部件验证后一个部件的数字签名，如果能验证通过，则运行后一个部件；如果验证不通过，则暂停启动。通过安全启动可以保证系统启动过程中各个部件的完整性，防止没有经过认证的部件被加载运行，从而防止对系统及用户数据产生安全威胁。\
安全启动涉及的验证组件： BIOS->shim->grub->vmlinuz（依次验签通过并加载），其中vmlinuz是内核镜像。\
相关的EFI启动组件由openEuler签名平台采用signcode方式进行签名。公钥证书由BIOS集成到签名数据库DB中，启动过程中BIOS对shim进行验证，shim和grub组件从BIOS的签名数据库DB中获取公钥证书并对下一级组件进行验证。

## 背景和解决方案

前期openEuler版本中，安全启动相关组件没有签名，无法直接使用安全启动功能保障系统组件的完整性。\
从22.03-LTS-SP4版本开始，openEuler使用社区签名平台对OS侧的相关组件进行签名，包括shim、grub和vmlinuz组件，并将社区签名根证书内嵌于shim组件中。

## 使用方法

### openEuler证书获取

openEuler根证书获取地址：<https://www.openeuler.org/zh/security/security-bulletins/>，进入“证书中心”目录下载。\
网页上根证书识别名称为“openEuler Shim Default CA”，default-x509ca.cert。

### BIOS侧操作

将openEuler根证书导入BIOS的db证书库中，并在BIOS中开启安全启动开关，可实现安全启动功能。\
BIOS证书导入方法及安全启动开启方法可参考具体BIOS厂商提供的资料。

### OS侧操作

**查看db数据库中的证书信息**：mokutil –db
![](./figures/mokutil-db.png)
说明：证书信息较多，截图中只显示部分重要信息\
**查看系统安全启动状态**：mokutil --sb

* SecureBoot disabled：安全启动关闭

  ![](./figures/mokutil-sb-off.png)

* SecureBoot enabled：安全启动开启

  ![](./figures/mokutil-sb-on.png)

* not supported：系统不支持安全启动

  ![](./figures/mokutil-sb-unsupport.png)

## 约束限制

* **软件限制**：OS系统需要采用UEFI启动
* **架构限制**：ARM/X86
* **硬件约束**：需要BIOS支持安全启动相关校验功能

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/system_container/security_and_isolation.md
---
# 安全性和隔离性

## user-namespace多对多

### 功能描述

user namespace是将容器的root映射到主机的普通用户，使得容器中的进程和用户在容器里有特权，但是在主机上就是普通权限，防止容器中的进程逃逸到主机上，进行非法操作。更进一步，使用user namespace技术后，容器和主机使用不同的uid和gid，保证容器内部的用户资源和主机资源进行隔离，例如文件描述符等。

系统容器支持通过--user-remap接口参数将不同容器的user namespace映射到宿主机不同的user namespace，实现容器user namespace隔离。

### 参数说明

### 约束限制

* 如果系统容器指定了--user-remap，那么rootfs目录必须能够被--user-remap指定的uid/gid用户所访问，否则会导致容器user  namespace无法访问rootfs，容器启动失败。
* 容器内所有的id都应该能映射到主机rootfs，某些目录/文件可能是从主机mount到容器，比如/dev/pts目录下面的设备文件，如果offset值太小可能会导致mount失败。
* uid、gid和offset的值由上层调度平台控制，容器引擎只做合法性检查。
* \--user-remap只适用于系统容器。
* \--user-remap和--privileged不能共存，否则容器启动会报错。
* 如果uid或gid指定为0，则--user-remap参数不生效。
* 如果系统容器指定了--user-remap，用户需保证--user-remap指定的uid/gid用户能访问isulad元数据目录（/var/lib/isulad/、/var/lib/isulad/engines/、/var/lib/isulad/engines/lcr）。
* \--user-remap与--userns不能同时使用。

### 使用指导

> \[!NOTE]说明
> 指定--user-remap参数前，请先将rootfs下所有目录和文件的uid和gid做整体偏移，偏移量为--user-remap指定uid和gid的偏移量。\
> 例如将dev目录的uid和gid整体uid和gid偏移100000的参考命令为：\
> chown 100000:100000 dev

系统容器启动指定--user-remap参数：

```sh
[root@localhost ~]# chmod 751 /var/lib/isulad/
[root@localhost ~]# chmod 751 /var/lib/isulad/engines/
[root@localhost ~]# chmod 751 /var/lib/isulad/engines/lcr
[root@localhost ~]# isula run -tid --user-remap 100000:100000:65535 --system-container --external-rootfs /home/root-fs none /sbin/init
eb9605b3b56dfae9e0b696a729d5e1805af900af6ce24428fde63f3b0a443f4a
```

分别在宿主机和容器内查看/sbin/init进程信息：

```sh
[root@localhost ~]# isula exec eb ps aux | grep /sbin/init
root         1  0.6  0.0  21624  9624 ?        Ss   15:47   0:00 /sbin/init
[root@localhost ~]# ps aux | grep /sbin/init
100000    4861  0.5  0.0  21624  9624 ?        Ss   15:47   0:00 /sbin/init
root      4948  0.0  0.0 213032   808 pts/0    S+   15:48   0:00 grep --color=auto /sbin/init
```

可以看到/sbin/init进程在容器内的owner是root用户，但是在宿主机的owner是uid=100000这个用户。

在容器内创建一个文件，然后在宿主机上查看文件的owner：

```sh
[root@localhost ~]# isula exec -it eb bash
[root@localhost /]# echo test123 >> /test123
[root@localhost /]# exit
exit
[root@localhost ~]# ll /home/root-fs/test123
-rw-------. 1 100000 100000 8 Aug  2 15:52 /home/root-fs/test123
```

可以看到，在容器内生成了一个文件，它的owner是root，但是在宿主机上看到的owner是id=100000这个用户。

## 用户权限控制

### 功能描述

容器引擎支持通过TLS认证方式来认证用户的身份，并依此控制用户的权限，当前容器引擎可以对接authz插件实现权限控制。

### 接口说明

通过配置iSulad容器引擎启动参数来指定权限控制插件，daemon配置文件默认为/etc/isulad/daemon.json。

### 约束限制

* authz需要配置用户权限策略，策略文件默认为/var/lib/authz-broker/policy.json，该配置文件支持动态修改，修改完即时生效，不需要重启插件服务。
* 由于容器引擎为root用户启动，放开一般用户使用的一些命令可能会导致该用户不当获得过大权限，需谨慎配置。目前container\_attach、container\_create和container\_exec\_create动作可能会有风险。
* 对于某些复合操作，比如isula exec、isula attach等命令依赖isula inspect是否有权限，如果用户没有inspect权限会直接报错。
* 采用SSL/TLS加密通道在增加安全性的同时也会带来性能损耗，如增加延时，消耗较多的CPU资源，除了数据传输外，加解密需要更大吞吐量，因此在并发场景下，相比非TLS通信，其并发量有一定程度上的下降。经实测，在ARM服务器（Cortex-A72 64核）接近空载情况下，采用TLS并发起容器，其最大并发量在200~250范围内。
* 服务端指定--tlsverify时，认证文件默认配置路径为/etc/isulad。且默认文件名分别为ca.pem、cert.pem、key.pem。

### 使用示例

1. 确认宿主机安装了authz插件，如果需要安装，安装并启动authz插件服务命令如下：

   ```sh
   [root@localhost ~]# yum install authz
   [root@localhost ~]# systemctl start authz
   ```

2. 要启动该功能，首先需要配置容器引擎和用户的TLS证书。可以使用OPENSSL来生成需要的证书，具体步骤如下：

   ```sh
   #SERVERSIDE
    
   # Generate CA key
   openssl genrsa -aes256 -passout "pass:$PASSWORD" -out "ca-key.pem" 4096
   # Generate CA
   openssl req -new -x509 -days $VALIDITY -key "ca-key.pem" -sha256 -out "ca.pem" -passin "pass:$PASSWORD" -subj "/C=$COUNTRY/ST=$STATE/L=$CITY/O=$ORGANIZATION/OU=$ORGANIZATIONAL_UNIT/CN=$COMMON_NAME/emailAddress=$EMAIL"
   # Generate Server key
   openssl genrsa -out "server-key.pem" 4096
    
   # Generate Server Certs.
   openssl req -subj "/CN=$COMMON_NAME" -sha256 -new -key "server-key.pem" -out server.csr
    
   echo "subjectAltName = DNS:localhost,IP:127.0.0.1" > extfile.cnf
   echo "extendedKeyUsage = serverAuth" >> extfile.cnf
    
   openssl x509 -req -days $VALIDITY -sha256 -in server.csr -passin "pass:$PASSWORD" -CA "ca.pem" -CAkey "ca-key.pem" -CAcreateserial -out "server-cert.pem" -extfile extfile.cnf
    
   #CLIENTSIDE
    
   openssl genrsa -out "key.pem" 4096
   openssl req -subj "/CN=$CLIENT_NAME" -new -key "key.pem" -out client.csr
   echo "extendedKeyUsage = clientAuth" > extfile.cnf
   openssl x509 -req -days $VALIDITY -sha256 -in client.csr -passin "pass:$PASSWORD" -CA "ca.pem" -CAkey "ca-key.pem" -CAcreateserial -out "cert.pem" -extfile extfile.cnf
   ```

   若要直接使用以上过程作为脚本，需替换各变量为配置数值。生成CA时使用的参数若为空则写为“''”。PASSWORD、COMMON\_NAME、CLIENT\_NAME、VALIDITY为必选项。

3. 容器引擎启动时添加TLS相关参数和认证插件相关参数，并保证认证插件的运行。此外，为了使用TLS认证，容器引擎必须使用TCP侦听的方式启动，不能使用传统的unix socket的方式启动。容器daemon端配置如下：

   ```con
   {
       "tls": true,
       "tls-verify": true,
       "tls-config": {
              "CAFile": "/root/.iSulad/ca.pem",
              "CertFile": "/root/.iSulad/server-cert.pem",
              "KeyFile":"/root/.iSulad/server-key.pem"
       },
       "authorization-plugin": "authz-broker"
   }
   ```

4. 然后需要配置策略，对于基本授权流程，所有策略都位于一个配置文件下/var/lib/authz-broker/policy.json。该配置文件支持动态修改，更改时不需要重新启动插件，只需要向authz进程发送SIGHUP信号。文件格式是每行一个策略JSON对象。每行只有一个匹配。具体的策略配置示例如下：

   * 所有用户都可以运行所有iSulad命令：{"name":"policy\_0","users":\[""],"actions":\[""]}。
   * Alice可以运行所有iSulad命令：{"name":"policy\_1","users":\["alice"],"actions":\[""]}。
   * 空用户都可以运行所有iSulad命令： {"name":"policy\_2","users":\[""],"actions":\[""]}。
   * Alice和Bob可以创建新的容器：{"name":"policy\_3","users":\["alice","bob"],"actions":\["container\_create"]}。
   * service\_account可以读取日志并运行docker top：{"name":"policy\_4","users":\["service\_account"],"actions":\["container\_logs","container\_top"]}。
   * Alice可以执行任何container操作：{"name":"policy\_5","users":\["alice"],"actions":\["container"]}。
   * Alice可以执行任何container操作，但请求的种类只能是get：{"name":"policy\_5","users":\["alice"],"actions":\["container"], "readonly":true }。

   > \[!NOTE]说明
   >
   > * 配置中匹配action支持正则表达式。
   > * users不支持正则表达式。
   > * users不能有重复用户，即同一用户不能被多条规则匹配。

5. 配置并更新完之后，客户端配置TLS参数连接容器引擎，即是以受限的权限访问。

   ```sh
   [root@localhost ~]#  isula version --tlsverify --tlscacert=/root/.iSulad/ca.pem --tlscert=/root/.iSulad/cert.pem --tlskey=/root/.iSulad/key.pem  -H=tcp://127.0.0.1:2375
   ```

   如果想默认配置TLS认证进行客户端连接，可以将文件移动到~/.iSulad，并设置ISULAD\_HOST和ISULAD\_TLS\_VERIFY变量（而不是每次调用时传递 -H=tcp://$HOST:2375和--tlsverify）。

   ```sh
   [root@localhost ~]# mkdir -pv ~/.iSulad
   [root@localhost ~]# cp -v {ca,cert,key}.pem ~/.iSulad 
   [root@localhost ~]# export ISULAD_HOST=localhost:2375 ISULAD_TLS_VERIFY=1
   [root@localhost ~]# isula version
   ```

## proc文件系统隔离

### 场景描述

容器虚拟化带来轻量高效，快速部署的同时，也因其隔离性不够彻底，给用户带来一定程度的使用不便。由于Linux内核namespace本身还不够完善，因此容器在隔离性方面也存在一些缺陷。例如，在容器内部proc文件系统中可以看到宿主机上的proc信息（如meminfo,  cpuinfo，stat， uptime等）。利用lxcfs工具可以将容器内的看到宿主机/proc文件系统的内容，替换成本容器实例的相关/proc内容，以便容器内业务获取正确的资源数值。

### 接口说明

系统容器对外提供两个工具包：一个是lxcfs软件，另外一个是配合lxcfs一起使用的lxcfs-tools工具。其中lxcfs作为宿主机daemon进程常驻，lxcfs-tools通过hook机制将宿主机的lxcfs文件系统绑定挂载到容器。

lxcfs-tools命令行格式如下：

```sh
lxcfs-tools [OPTIONS] COMMAND [COMMAND_OPTIONS]
```

### 约束限制

* 当前只支持proc文件系统下的cpuinfo,  meminfo, stat, diskstats,  partitions，swaps和uptime文件，其他的文件和其他内核API文件系统（比如sysfs）未做隔离。
* 安装rpm包后会在/var/lib/isulad/hooks/hookspec.json生成样例json文件，用户如果需要增加日志功能，需要在定制时加入--log配置。
* diskstats只能显示支持cfq调度的磁盘信息，无法显示分区信息。容器内设备会被显示为/dev目录下的名字。若不存在则为空。此外，容器根目录所在设备会被显示为sda。
* 挂载lxcfs时必须使用slave参数。若使用shared参数，可能会导致容器内挂载点泄露到主机，影响主机运行。
* lxcfs支持服务优雅降级使用，若lxcfs服务crash或者不可用，容器内查看到的cpuinfo,  meminfo, stat, diskstats, partitions, swaps和uptime均为host信息，容器其他业务功能不受影响。
* lxcfs底层依赖fuse内核模块以及libfuse库，因此需要内核支持fuse。
* lxcfs当前仅支持容器内运行64位的app，如果容器内运行32位的app可能会导致app读取到的cpuinfo信息不符合预期。
* lxcfs只是对容器cgroup进行资源视图模拟，对于容器内的系统调用（例如sysconf）获取到的仍然是主机的信息，lxcfs无法做到内核隔离。
* lxcfs使用隔离后的cpuinfo显示的cpu信息具有如下特征：
  * processor：从0开始依次递增。
  * physical  id：从0开始依次递增。
  * sibliing：固定为1。
  * core  id：固定为0。
  * cpu  cores：固定为1。

### 使用示例

1. 首先需要安装lxcfs和lxcfs-tools这两个包，并启动lxcfs服务。

   ```sh
   [root@localhost ~]# yum install lxcfs lxcfs-tools 
   [root@localhost ~]# systemctl start lxcfs
   ```

2. 容器启动完成之后查看容器内是否存在lxcfs挂载点。

   ```sh
   [root@localhost ~]# isula run -tid -v /var/lib/lxc:/var/lib/lxc --hook-spec /var/lib/isulad/hooks/hookspec.json --system-container --external-rootfs /home/root-fs none init
   a8acea9fea1337d9fd8270f41c1a3de5bceb77966e03751346576716eefa9782
   [root@localhost ~]# isula exec a8 mount | grep lxcfs
   lxcfs on /var/lib/lxc/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/cpuinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/diskstats type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/meminfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/partitions type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/stat type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/swaps type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   lxcfs on /proc/uptime type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
   ```

3. 执行update命令更新容器的cpu和mem资源配置，然后查看容器资源。根据如下回显可知，容器资源视图显示的是容器真实资源数据而不是宿主机的数据。

   ```sh
   [root@localhost ~]# isula update --cpuset-cpus 0-1 --memory 1G a8
   a8
   [root@localhost ~]# isula exec a8 cat /proc/cpuinfo
   processor       : 0
   BogoMIPS        : 100.00
   cpu MHz         : 2400.000
   Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
   CPU implementer : 0x41
   CPU architecture: 8
   CPU variant     : 0x0
   CPU part        : 0xd08
   CPU revision    : 2

   processor       : 1
   BogoMIPS        : 100.00
   cpu MHz         : 2400.000
   Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
   CPU implementer : 0x41
   CPU architecture: 8
   CPU variant     : 0x0
   CPU part        : 0xd08
   CPU revision    : 2

   [root@localhost ~]# isula exec a8 free -m
                 total        used        free      shared  buff/cache   available
   Mem:           1024          17         997           7           8        1006
   Swap:          4095           0        4095
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/security_features.md
---
# 安全特性

## seccomp安全配置场景

### 场景说明

seccomp（**secure computing** **mode**）是linux kernel从2.6.23版本开始引入的一种简洁的sandboxing机制。在一些特定场景下，用户需要在容器中执行一些“特权”操作，但又不想启动特权容器，用户经常会在run时添加--cap-add来获得一些“小范围”的权限。对于安全要求比较严格的容器实例，上述的CAP粒度不一定能够满足安全需要，可使用一些办法精细化控制权限范围。

* 举例

  普通容器场景中，用户使用-v将宿主机某目录（包含某普通用户无法执行的二进制），映射到容器中。

  在容器中，可以将二进制修改权限chmod 4777加入S标志位。这样在宿主机上，原先不能运行二进制的普通用户（或者运行此二进制受限），可以在S标志位的添加动作后，在运行此二进制的时候，获取到二进制自身的权限（比如root），从而提权运行或者访问其他文件。

  这个场景，如果在严格安全要求下，需要使用seccomp裁剪chmod、fchmod、fchmodat系统调用。

### 使用限制

* 不要禁用iSulad的seccomp特性

  默认的iSulad有一个seccomp的配置，配置中使用的是白名单，不在配置的sys\_call会被seccomp禁掉，使用接口--security-opt 'seccomp:unconfined'可以禁止使用seccomp特性。如果禁用seccomp或使用自定义seccomp配置但过滤名单不全，都会增加容器对内核的攻击面。

* 默认的Seccomp配置是白名单，不在白名单的syscall默认会返回SCMP\_ACT\_ERRNO，同时会根据不同的Cap开放不同的系统调用，不在白名单上面的权限，iSulad默认不会将权限给到容器。

### 使用指导

通过--security-opt将配置文件传给要过滤系统调用的容器。

```bash
isula run -itd --security-opt seccomp=/path/to/seccomp/profile.json rnd-dockerhub.huawei.com/official/busybox
```

> \[!NOTE]说明
>
> * 创建容器时通过--security-opt将配置文件传给容器时，采用默认配置文件（/etc/isulad/seccomp\_default.json）。
> * 创建容器时--security-opt设置为unconfined时，对容器不过滤系统调用。
> * “/path/to/seccomp/profile.json”需要是绝对路径。
> * \--security-opt采用“=”号进行分割，不支持使用“:”号分割。

#### 获取普通容器的默认seccomp配置

* 启动一个普通容器（或者是带--cap-add的容器），并查看默认权限配置：

  ```bash
  cat /etc/isulad/seccomp_default.json | python -m json.tool > profile.json
  ```

  可以看到"seccomp"字段中，有很多的"syscalls"，在此基础上，仅提取syscalls的部分，参考定制seccomp配置文件，进行定制化操作。

  ```conf
  "defaultAction": "SCMP_ACT_ERRNO",
  "syscalls": [
  {
  "action": "SCMP_ACT_ALLOW",
  "name": "accept"
  },
  {
  "action": "SCMP_ACT_ALLOW",
  "name": "accept4"
  },
  {
  "action": "SCMP_ACT_ALLOW",
  "name": "access"
  },
  {
  "action": "SCMP_ACT_ALLOW",
  "name": "alarm"
  },
  {
  "action": "SCMP_ACT_ALLOW",
  "name": "bind"
  },
  ]...
  ```

* 查看转换为lxc可识别的seccomp配置

  ```bash
  cat /var/lib/isulad/engines/lcr/74353e38021c29314188e29ba8c1830a4677ffe5c4decda77a1e0853ec8197cd/seccomp
  ```

  ```text
  ...
  waitpid allow
  write allow
  writev allow
  ptrace allow
  personality allow [0,0,SCMP_CMP_EQ,0]
  personality allow [0,8,SCMP_CMP_EQ,0]
  personality allow [0,131072,SCMP_CMP_EQ,0]
  personality allow [0,131080,SCMP_CMP_EQ,0]
  personality allow [0,4294967295,SCMP_CMP_EQ,0]
  ...
  ```

#### 定制seccomp配置文件

在启动容器的时候使用--security-opt引入seccomp配置文件，容器实例会按照配置文件规则进行限制系统API的运行。首先获取普通容器的默认seccomp，得到完整模板，然后按照本节定制配置文件，启动容器：

```bash
isula run --rm -it --security-opt seccomp:/path/to/seccomp/profile.json rnd-dockerhub.huawei.com/official/busybox
```

配置文件模板：

```conf
{
"defaultAction": "SCMP_ACT_ALLOW",
"syscalls": [
{
"name": "syscall-name",
"action": "SCMP_ACT_ERRNO",
"args": null
}
]
}
```

> \[!TIP]须知
>
> * defaultAction、syscalls：对应的action的类型是一样的，但其值是不能一样的，目的就是让所有的syscall都有一个默认的action，并且如果syscalls数组中有明确的定义，就以syscalls中的为准，由于defaultAction、action的值不一样，就能保证action不会有冲突。当前支持的action有：
>   * "SCMP\_ACT\_ERRNO"：禁止，并打印错误信息。
>   * "SCMP\_ACT\_ALLOW"：允许。
> * syscalls： 数组，可以只有一个syscall，也可以有多个，可以带args，也可以不带。
> * name：要过滤的syscall。
> * args：数组，里面的每个object的定义如下：
>
>   ```conf
>   type Arg struct {  
>   Index    uint     `json:"index"`     //参数的序号，如open(fd, buf, len),fd 对应的就是0，buf为1  
>   Value    uint64   `json:"value"`     //跟参数进行比较的值  
>   ValueTwo uint64   `json:"value_two"` //仅当Op=MaskEqualTo时起作用，用户传入值跟Value按位与操作后，跟ValueTwo进行比较，若相等则执行action。  
>   Op       Operator `json:"op"`  
>   }  
>   ```
>
> args中的Op，其取值可以下页面的任意一种：\
> "SCMP\_CMP\_NE":  NotEqualTo\
> "SCMP\_CMP\_LT":  LessThan\
> "SCMP\_CMP\_LE":  LessThanOrEqualTo\
> "SCMP\_CMP\_EQ":  EqualTo\
> "SCMP\_CMP\_GE":  GreaterThanOrEqualTo\
> "SCMP\_CMP\_GT":  GreaterThan\
> "SCMP\_CMP\_MASKED\_EQ": MaskEqualTo

## capabilities安全配置场景

### 场景说明

capabilities机制是linux kernel 2.2之后引入的安全特性，用更小的粒度控制超级管理员权限,可以避免使用 root 权限，将root用户的权限细分为不同的领域，可以分别启用或禁用。capabilities详细信息可通过Linux Programmer's Manual进行查看（[capabilities(7) - Linux man page](http://man7.org/linux/man-pages/man7/capabilities.7.html)）：

```bash
man capabilities
```

### 使用限制

* isulad默认Capabilities（白名单）配置如下，普通容器进程将默认携带：

  ```conf
  "CAP_CHOWN",
  "CAP_DAC_OVERRIDE",
  "CAP_FSETID",
  "CAP_FOWNER",
  "CAP_MKNOD",
  "CAP_NET_RAW",
  "CAP_SETGID",
  "CAP_SETUID",
  "CAP_SETFCAP",
  "CAP_SETPCAP",
  "CAP_NET_BIND_SERVICE",
  "CAP_SYS_CHROOT",
  "CAP_KILL",
  "CAP_AUDIT_WRITE"
  ```

* 默认的权能配置，包含了CAP\_SETUID和CAP\_FSETID，如host和容器共享目录，容器可对共享目录的二进制文件进行文件权限设置，host上的普通用户可能使用该特性进行提权攻击。CAP\_AUDIT\_WRITE，容器可以对host写入，存在一定的风险，如果使用场景不需要，推荐在启动容器的时候使用--cap-drop将其删除。

* 增加Capabilities意味着容器进程具备更大的能力，同时也会开放更多的系统调用接口。

### 使用指导

iSulad使用--cap-add/--cap-drop给容器增加/删去特定的权限，在非必要情况下，不要给容器增加额外的权限，推荐将容器默认但非必要的权限也去掉。

```bash
isula run --rm -it --cap-add all --cap-drop SYS_ADMIN rnd-dockerhub.huawei.com/official/busybox
```

## SELinux安全配置场景

### 场景说明

SELinux(Security-Enhanced Linux)是一个Linux内核的安全模块，提供了访问控制安全策略机制，iSulad将采用MCS（多级分类安全）实现对容器内进程打上标签限制容器访问资源的方式，减少提权攻击的风险，防止造成更为重要的危害。

### 使用限制

* 确保宿主机已使能SELinux，且daemon端已打开SELinux使能开发（/etc/isulad/daemon.json中“selinux-enabled”字段为true, 或者命令行参数添加--selinux-enabled）
* 确保宿主机上已配置合适的selinux策略，推荐使用container-selinux进行配置
* 引入SELinux会影响性能，设置SELinux之前需要对场景进行评估，确定必要时打开daemon端SELinux开关并设置容器SELinux配置
* 对挂载卷进行标签配置时，源目录不允许为/、/usr、/etc、/tmp、/home、/run、/var、/root以及/usr的子目录。

> \[!NOTE]说明
>
> * 目前iSulad不支持对容器的文件系统打标签，确保容器文件系统及配置目录打上容器可访问标签，需使用chcon命令对其打上标签。
> * 若iSulad启用SELinux访问控制，建议daemon启动前对/var/lib/isulad目录打上标签，容器容器创建时目录下生产的文件及文件夹将默认继承其标签，例如：
>
>   ```bash
>   chcon -R system_u:object_r:container_file_t:s0 /var/lib/isulad  
>   ```

### 使用指导

* daemon端使能selinux：

  ```bash
  isulad --selinux-enabled
  ```

* 启动容器时配置selinux标签安全上下文

  \--security-opt="label=user:USER" 配置安全上下文用户

  \--security-opt="label=role:ROLE" 配置安全上下文角色

  \--security-opt="label=type:TYPE" 配置安全上下文类型

  \--security-opt="label=level:LEVEL" 配置安全上下文域

  \--security-opt="label=disable" 容器禁用SELinux配置

  ```bash
  $ isula run -itd --security-opt label=type:container_t --security-opt label=level:s0:c1,c2 rnd-dockerhub.huawei.com/official/centos
  9be82878a67e36c826b67f5c7261c881ff926a352f92998b654bc8e1c6eec370
  ```

* 为挂载卷打selinux标签('z'为共享模式)

  ```bash
  $ isula run -itd -v /test:/test:z rnd-dockerhub.huawei.com/official/centos
  9be82878a67e36c826b67f5c7261c881ff926a352f92998b654bc8e1c6eec370

  $ls -Z /test
  system_u:object_r:container_file_t:s0 file
  ```

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/image_builder/isula_build/isula_build.md
---
# 安装

## 环境准备

为了确保isula-build成功安装，需满足以下软件硬件要求。

* 支持的机器架构：x86\_64 和 AArch64
* 支持的操作系统：openEuler
* 用户具有root权限

### 安装isula-build

使用isula-build构建容器镜像，需要先安装以下软件包。

#### （推荐）方法一：使用yum安装

1. 配置openEuler yum源。

2. 使用root权限，登录目标服务器，安装isula-build。

   ```sh
   sudo yum install -y isula-build
   ```

#### 方法二：使用rpm包安装

1. 从openEuler yum源中获取isula-build对应安装包isula-build-\*.rpm。例如isula-build-0.9.6-4.oe1.x86\_64.rpm。

2. 将获取的rpm软件包上传至目标服务器的任一目录，例如 /home/。

3. 使用root权限，登录目标服务器，参考如下命令安装isula-build。

   ```sh
   sudo rpm -ivh /home/isula-build-*.rpm
   ```

> \[!NOTE]说明
>
> 安装完成后，需要手工启动isula-build服务。启动请参见[管理服务](#管理服务)。

# 配置与管理服务

## 配置服务

在安装完 isula-build 软件包之后，systemd 管理服务会以 isula-build 软件包自带的 isula-build 服务端默认配置启动 isula-build 服务。如果 isula-build 服务端的默认配置文件不能满足用户的需求，可以参考如下介绍进行定制化配置。需要注意的是，修改完默认配置之后，需要重启 isula-build 服务端使新配置生效，具体操作可参考下一章节。

目前 isula-build 服务端包含如下配置文件：

* /etc/isula-build/configuration.toml：isula-builder 总体配置文件，用于设置 isula-builder 日志级别、持久化目录和运行时目录、OCI runtime等。其中各参数含义如下：

| 配置项    | 是否可选 | 配置项含义                        | 配置项取值                                      |
| --------- | -------- | --------------------------------- | ----------------------------------------------- |
| debug     | 可选     | 设置是否打开debug日志             | true：打开debug日志false：关闭debug日志    |
| loglevel  | 可选     | 设置日志级别                      | debuginfowarnerror               |
| run\_root  | 必选     | 设置运行时数据根目录              | 运行时数据根目录路径，例如/var/run/isula-build/ |
| data\_root | 必选     | 设置本地持久化目录                | 本地持久化目录路径，例如/var/lib/isula-build/   |
| runtime   | 可选     | 设置runtime种类，目前仅支持runc   | runc                                            |
| group     | 可选     | 设置本地套接字isula\_build.sock文件属组使得加入该组的非特权用户可以操作isula-build | isula |
| experimental | 可选     | 设置是否开启实验特性                                         | true：开启实验特性；false：关闭实验特性         |

* /etc/isula-build/storage.toml：本地持久化存储的配置文件，包含所使用的存储驱动的配置。

| 配置项 | 是否可选 | 配置项含义                     |
| ------ | -------- | ------------------------------ |
| driver | 可选     | 存储驱动类型，目前支持overlay2 |

更多设置可参考 [containers-storage.conf.5](https://github.com/containers/storage/blob/main/docs/containers-storage.conf.5.md)。

* /etc/isula-build/registries.toml : 针对各个镜像仓库的配置文件。

| 配置项              | 是否可选 | 配置项含义                                                   |
| ------------------- | -------- | ------------------------------------------------------------ |
| registries.search   | 可选     | 镜像仓库搜索域，在此list的镜像仓库可以被感知，不在此列的不被感知。 |
| registries.insecure | 可选     | 可访问的不安全镜像仓库地址，在此列表中的镜像仓库将不会通过鉴权，不推荐使用。 |

更多设置可参考 [containers-registries.conf.5](https://github.com/containers/image/blob/main/docs/containers-registries.conf.5.md)。

* /etc/isula-build/policy.json：镜像pull/push策略文件。当前不支持对其进行配置。

> \[!NOTE]说明
>
> * isula-build 支持最大 1MiB 的上述配置文件。
> * isula-build 不支持将持久化工作目录 dataroot 配置在内存盘上，比如 tmpfs。
> * isula-build 目前仅支持使用overlay2为底层 graphdriver。
> * 在设置--group参数前，需保证本地OS已经创建了对应的用户组，且非特权用户已经加入该组。重启isula-builder之后即可使该非特权用户使用isula-build功能。同时，为了保持权限一致性，isula-build的配置文件目录/etc/isula-build的属组也会被设置为--group指定的组。

## 管理服务

目前 openEuler 采用 systemd 管理软件服务，isula-build 软件包已经自带了 systemd 的服务文件，用户安装完 isula-build 软件包之后，可以直接通过 systemd 工具对它进行服务启停等操作。用户同样可以手动启动 isula-build 服务端软件。需要注意的是，同一个节点上不可以同时启动多个 isula-build 服务端软件。

> \[!NOTE]说明
>
> 同一个节点上不可以同时启动多个 isula-build 服务端软件。

### 通过 systemd 管理（推荐方式）

用户可以通过如下 systemd 的标准指令控制 isula-build 服务的启动、停止、重启等动作：

* 启动 isula-build 服务：

  ```sh
  sudo systemctl start isula-build.service
  ```

* 停止 isula-build 服务：

  ```sh
  sudo systemctl stop isula-build.service
  ```

* 重启 isula-build 服务：

  ```sh
  sudo systemctl restart isula-build.service
  ```

isula-build 软件包安装的 systemd 服务文件保存在 `/usr/lib/systemd/system/isula-build.service`。如果用户需要修改 isula-build 服务的 systemd 配置，可以修改该文件，执行如下命令使配置生效，之后再根据上面提到的 systemd 管理指令重启 isula-build 服务

```sh
sudo systemctl daemon-reload
```

### 直接运行 isula-build 服务端

您也可以通过执行 isula-build 服务端命令（ isula-builder）的方式启动服务。其中，服务端启动配置，可通过isula-builder命令支持的 flags 设置。isula-build 服务端目前支持的 flags 如下：

* -D, --debug: 是否开启调测模式。
* \--log-level: 日志级别，支持 “debug”, “info”, “warn” or “error”，默认为 “info”。
* \--dataroot: 本地持久化路径，默认为”/var/lib/isula-build/“。
* \--runroot: 运行时路径，默认为”/var/run/isula-build/“。
* \--storage-driver：底层存储驱动类型。
* \--storage-opt: 底层存储驱动配置。
* \--group: 设置本地套接字isula\_build.sock文件属组使得加入该组的非特权用户可以操作isula-build，默认为“isula”。
* \--experimental: 是否开启实验特性，默认为false。

> \[!NOTE]说明
>
> 当命令行启动参数中传递了与配置文件相同的配置选项时，优先使用命令行参数启动。

启动 isula-build 服务。例如指定本地持久化路径/var/lib/isula-build，且不开启调试的参考命令如下：

```sh
sudo isula-builder --dataroot "/var/lib/isula-build" --debug=false
```

# 使用指南

## 前提条件

isula-build 构建 Dockerfile 内的 RUN 指令时依赖可执行文件 runc ，需要 isula-build 的运行环境上预装好 runc。安装方式视用户使用场景而定，如果用户不需要使用完整的 docker-engine 工具链，则可以仅安装 docker-runc rpm包：

```sh
sudo yum install -y docker-runc
```

如果用户需要使用完整的 docker-engine 工具链，则可以安装 docker-engine rpm包，默认包含可执行文件 runc ：

```sh
sudo yum install -y docker-engine
```

> \[!NOTE]说明
>
> 用户需保证OCI runtime（runc）可执行文件的安全性，避免被恶意替换。

## 总体说明

isula-build 客户端提供了一系列命令用于构建和管理容器镜像，当前 isula-build 包含的命令行指令如下：

* ctr-img，容器镜像管理。ctr-img又包含如下子命令：
  * build，根据给定dockerfile构建出容器镜像。
  * images，列出本地容器镜像。
  * import，导入容器基础镜像。
  * load，导入层叠镜像。
  * rm，删除本地容器镜像。
  * save，导出层叠镜像至本地磁盘。
  * tag，给本地容器镜像打tag。
  * pull，拉取镜像到本地。
  * push，推送本地镜像到远程仓库。
* info，查看isula-build的运行环境和系统信息。
* login，登录远端容器镜像仓库。
* logout，退出远端容器镜像仓库。
* version，查看isula-build和isula-builder的版本号。
* manifest（实验特性），管理manifest列表。

> \[!NOTE]说明
>
> * isula-build completion 和 isula-builder completion 命令用于生成bash命令补全脚本。该命令为命令行框架隐式提供，不会显示在help信息中。
> * isula-build客户端不包含配置文件，当用户需要使用isula-build实验特性时，需要在客户端通过命令`export ISULABUILD_CLI_EXPERIMENTAL=enabled`配置环境变量ISULABUILD\_CLI\_EXPERIMENTAL来开启实验特性。

以下按照上述维度依次详细介绍这些命令行指令的使用。

## ctr-img: 容器镜像管理

isula-build 将所有容器镜像管理相关命令划分在子命令 `ctr-img` 下，命令原型为：

```sh
isula-build ctr-img [command]
```

### build: 容器镜像构建

ctr-img 的子命令 build 用于构建容器镜像，命令原型为：

```sh
isula-build ctr-img build [flags]
```

其中 build 包含如下 flags：

* \--build-arg：string列表，构建过程中需要用到的变量。
* \--build-static：KeyValue值，构建二进制一致性。目前包含如下Key值：
  * build-time：string，使用固定时间戳来构建容器镜像；时间戳格式为“YYYY-MM-DD HH-MM-SS”。
* -f, --filename：string，Dockerfile的路径，不指定则是使用当前路径的Dockerfile文件。
* \--format： string，设置构建镜像的镜像格式：oci | docker（需开启实验特性选项）。
* \--iidfile：string，输出 image ID 到本地文件。
* -o, --output：string，镜像导出的方式和路径。
* \--proxy：布尔值，继承主机侧环境的proxy环境变量（默认为true）。
* \--tag：string，设置构建成功的镜像的tag值。
* \--cap-add：string列表，构建过程中RUN指令所需要的权限。

**以下为各个 flags 的详解。**

**--build-arg**

从命令行接受参数作为Dockerfile中的参数，用法：

```sh
$ echo "This is bar file" > bar.txt
$ cat Dockerfile_arg
FROM busybox
ARG foo
ADD ${foo}.txt .
RUN cat ${foo}.txt
$ sudo isula-build ctr-img build --build-arg foo=bar -f Dockerfile_arg
STEP  1: FROM busybox
Getting image source signatures
Copying blob sha256:8f52abd3da461b2c0c11fda7a1b53413f1a92320eb96525ddf92c0b5cde781ad
Copying config sha256:e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b
Writing manifest to image destination
Storing signatures
STEP  2: ARG foo
STEP  3: ADD ${foo}.txt .
STEP  4: RUN cat ${foo}.txt
This is bar file
Getting image source signatures
Copying blob sha256:6194458b07fcf01f1483d96cd6c34302ffff7f382bb151a6d023c4e80ba3050a
Copying blob sha256:6bb56e4a46f563b20542171b998cb4556af4745efc9516820eabee7a08b7b869
Copying config sha256:39b62a3342eed40b41a1bcd9cd455d77466550dfa0f0109af7a708c3e895f9a2
Writing manifest to image destination
Storing signatures
Build success with image id: 39b62a3342eed40b41a1bcd9cd455d77466550dfa0f0109af7a708c3e895f9a2
```

**--build-static**

指定为静态构建，即使用isula-build构建容器镜像时消除所有时间戳和其他构建因素（例如容器ID、hostname等）的差异。最终构建出满足静态要求的容器镜像。

在使用isula-build进行容器镜像构建时，假如给 build 子命令一个固定的时间戳，并在限定如下条件的时候：

* 构建环境前后保持一致。
* 构建Dockerfile前后保持一致。
* 构建产生的中间数据前后保持一致。
* 构建命令相同。
* 第三方库版本一致。

对于容器镜像构建，isula-build支持相同的Dockerfile。如果构建环境相同，则多次构建生成的镜像内容和镜像ID相同。

\--build-static接受k=v形式的键值对选项，当前支持的选项有：

* build-time：字符串类型。构建静态镜像的固定时间戳，格式为“YYYY-MM-DD HH-MM-SS”。时间戳影响diff层创建修改时间的文件属性。

  使用示例如下：

  ```sh
  sudo isula-build ctr-img build -f Dockerfile --build-static='build-time=2020-05-23 10:55:33' .
  ```

  以此方式，同一环境多次构建出来的容器镜像和镜像ID均会保持一致。

**--format**

开始实验特性后该选项可用，默认为OCI镜像格式。可以手动指定镜像格式进行构建，例如，下面分别为构建OCI镜像格式以及Docker镜像格式镜像的命令。

```sh
export ISULABUILD_CLI_EXPERIMENTAL=enabled; sudo isula-build ctr-img build -f Dockerfile --format oci .
```

```sh
export ISULABUILD_CLI_EXPERIMENTAL=enabled; sudo isula-build ctr-img build -f Dockerfile --format docker .
```

**--iidfile**

将构建的镜像ID输出到文件，用法：

```sh
isula-build ctr-img build --iidfile filename
```

例如，将容器镜像ID输出到testfile的参考命令如下：

```sh
sudo isula-build ctr-img build -f Dockerfile_arg --iidfile testfile
```

查看testfile中的容器镜像ID：

```sh
$ cat testfile
76cbeed38a8e716e22b68988a76410eaf83327963c3b29ff648296d5cd15ce7b
```

**-o, --output**

目前 -o, --output 支持如下形式：

* `isulad:image:tag`：将构建成功的镜像直接推送到 iSulad。比如：`-o isulad:busybox:latest`。同时需要注意如下约束：

  * isula-build 和 iSulad 必须在同一个节点上
  * tag必须配置
  * isula-build client端需要将构建成功的镜像暂存成 `/var/tmp/isula-build-tmp-%v.tar` 再导入至 iSulad，用户需要保证 `/var/tmp/` 目录有足够磁盘空间

* `docker-daemon:image:tag`：将构建成功的镜像直接推送到 Docker daemon。比如：`-o docker-daemon:busybox:latest`。同时需要注意如下约束：
  * isula-build 和 docker 必须在同一个节点上
  * tag必须配置

* `docker://registry.example.com/repository:tag`：将构建成功的镜像以Docker镜像格式直接推送到远端镜像仓库。比如：`-o docker://localhost:5000/library/busybox:latest`。

* `docker-archive:<path>/<filename>:image:tag`：将构建成功的镜像以Docker镜像格式保存至本地。比如：`-o docker-archive:/root/image.tar:busybox:latest`。

打开实验特性之后，可以启用相应OCI镜像的构建：

* `oci://registry.example.com/repository:tag`：将构建成功的镜像以OCI镜像格式直接推送到远端镜像仓库（远程镜像仓库须支持OCI镜像格式）。比如：`-o oci://localhost:5000/library/busybox:latest`。

* `oci-archive:<path>/<filename>:image:tag`：将构建成功的镜像以OCI镜像的格式保存至本地。比如：`-o oci-archive:/root/image.tar:busybox:latest`。

除去各个flags之外，build子命令的命令行最后还会接收一个argument，该argument类型是string，意义为context，即该Dockerfile构建环境的上下文。该参数缺省值为isula-build被执行的当前路径。该路径会影响 .dockerignore 和 Dockerfile的ADD/COPY指令所检索的路径。

**--proxy**

选择构建时RUN指令启动的容器是否从环境上继承proxy相关环境变量“http\_proxy”,“https\_proxy”,“ftp\_proxy”,“no\_proxy”,“HTTP\_PROXY”,“HTTPS\_PROXY”,“FTP\_PROXY”,“NO\_PROXY”，默认为true。

当用户在Dockerfile配置proxy相关ARG或ENV，将覆盖所继承的环境变量。

注意：若client与daemon不在同一个终端运行，所能继承的环境变量为daemon所在终端的环境变量。

**--tag**

设置镜像构建成功之后，该镜像在本地磁盘存储时的tag。

**--cap-add**

添加构建过程中RUN指令所需权限，用法：

```sh
isula-build ctr-img build --cap-add ${CAP}
```

使用举例：

```sh
sudo isula-build ctr-img build --cap-add CAP_SYS_ADMIN --cap-add CAP_SYS_PTRACE -f Dockerfile
```

> \[!NOTE]说明
>
> * isula-build 最大支持并发构建100个容器镜像。
> * isula-build 支持Dockerfile最大为1MiB。
> * isula-build 支持 .dockerignore 最大为 1MiB。
> * 用户需保证Dockerfile文件的权限为仅当前用户可读写，避免别的用户进行篡改。
> * 构建时，RUN指令会启动容器在容器内进行构建，目前 isula-build 仅支持使用主机网络。
> * isula-build 导出的镜像压缩格式，目前仅支持tar格式。
> * isula-build 在每一个镜像构建stage完成后做一次提交，而不是每执行 Dockerfile的一行就提交一次。
> * isula-build 暂不支持构建缓存。
> * isula-build 仅在构建RUN指令时会启动构建容器。
> * 目前不支持docker镜像格式的history功能。
> * isula-build 的stage name支持以数字开头。
> * isula-build 的stage name最长可为64个字符。
> * isula-build 暂不支持对单次Dockerfile的构建进行资源限制。如有资源限制需求，可通过对 isula-builder 服务端配置资源限额的方式进行限制。
> * isula-build 目前不支持Dockerfile里的ADD指令提供的数据来源是远端url。
> * isula-build 使用docker-archive以及oci-archive类型导出的本地tar包未经压缩。如有需求，用户可以手动进行压缩。

### image: 查看本地持久化构建镜像

可通过images命令查看当前本地持久化存储的镜像：

```sh
$ sudo isula-build ctr-img images
---------------------------------------  -----------  -----------------  ------------------------  ------------
REPOSITORY                               TAG          IMAGE ID           CREATED                   SIZE
---------------------------------------  -----------  -----------------  ------------------------  ------------
localhost:5000/library/alpine            latest       a24bb4013296       2022-01-17 10:02:19       5.85 MB
<none>                                   <none>       39b62a3342ee       2022-01-17 10:01:12       1.45 MB
---------------------------------------  -----------  -----------------  ------------------------  ------------
```

> \[!NOTE]说明
>
> 通过`isula-build ctr-img images`查看的镜像大小与`docker images`的显示上有一定差异。这是因为统计镜像大小时，isula-build是直接计算每层tar包大小之和，而docker是通过解压tar遍历diff目录计算文件大小之和，因此存在统计上的差异。

### import: 导入容器基础镜像

可以通过`ctr-img import`指令将rootfs形式的tar文件导入到isula-build中。

命令原型如下：

```sh
isula-build ctr-img import [flags]
```

使用举例：

```sh
$ sudo isula-build ctr-img import busybox.tar mybusybox:latest
Getting image source signatures
Copying blob sha256:7b8667757578df68ec57bfc9fb7754801ec87df7de389a24a26a7bf2ebc04d8d
Copying config sha256:173b3cf612f8e1dc34e78772fcf190559533a3b04743287a32d549e3c7d1c1d1
Writing manifest to image destination
Storing signatures
Import success with image id: "173b3cf612f8e1dc34e78772fcf190559533a3b04743287a32d549e3c7d1c1d1"
$ sudo isula-build ctr-img images
---------------------------------------  -----------  -----------------  ------------------------  ------------
REPOSITORY                               TAG          IMAGE ID           CREATED                   SIZE
---------------------------------------  -----------  -----------------  ------------------------  ------------
mybusybox                                latest       173b3cf612f8       2022-01-12 16:02:31       1.47 MB
---------------------------------------  -----------  -----------------  ------------------------  ------------
```

> \[!NOTE]说明
>
> isula-build 支持导入最大1GiB的容器基础镜像。

### load: 导入层叠镜像

层叠镜像指的是通过 docker save 或 isula-build ctr-img save 等指令，将一个构建完成的镜像保存至本地之后，镜像压缩包内是一层一层 layer.tar 的镜像包。可以通过 ctr-img load 指令将它导入至 isula-build。

命令原型如下：

```sh
isula-build ctr-img load [flags]
```

目前支持的 flags 为：

* -i, --input：本地tar包的路径

使用举例如下：

```sh
$ sudo isula-build ctr-img load -i ubuntu.tar
Getting image source signatures
Copying blob sha256:cf612f747e0fbcc1674f88712b7bc1cd8b91cf0be8f9e9771235169f139d507c
Copying blob sha256:f934e33a54a60630267df295a5c232ceb15b2938ebb0476364192b1537449093
Copying blob sha256:943edb549a8300092a714190dfe633341c0ffb483784c4fdfe884b9019f6a0b4
Copying blob sha256:e7ebc6e16708285bee3917ae12bf8d172ee0d7684a7830751ab9a1c070e7a125
Copying blob sha256:bf6751561805be7d07d66f6acb2a33e99cf0cc0a20f5fd5d94a3c7f8ae55c2a1
Copying blob sha256:c1bd37d01c89de343d68867518b1155cb297d8e03942066ecb44ae8f46b608a3
Copying blob sha256:a84e57b779297b72428fc7308e63d13b4df99140f78565be92fc9dbe03fc6e69
Copying blob sha256:14dd68f4c7e23d6a2363c2320747ab88986dfd43ba0489d139eeac3ac75323b2
Copying blob sha256:a2092d776649ea2301f60265f378a02405539a2a68093b2612792cc65d00d161
Copying blob sha256:879119e879f682c04d0784c9ae7bc6f421e206b95d20b32ce1cb8a49bfdef202
Copying blob sha256:e615448af51b848ecec00caeaffd1e30e8bf5cffd464747d159f80e346b7a150
Copying blob sha256:f610bd1e9ac6aa9326d61713d552eeefef47d2bd49fc16140aa9bf3db38c30a4
Copying blob sha256:bfe0a1336d031bf5ff3ce381e354be7b2bf310574cc0cd1949ad94dda020cd27
Copying blob sha256:f0f15db85788c1260c6aa8ad225823f45c89700781c4c793361ac5fa58d204c7
Copying config sha256:c07ddb44daa97e9e8d2d68316b296cc9343ab5f3d2babc5e6e03b80cd580478e
Writing manifest to image destination
Storing signatures
Loaded image as c07ddb44daa97e9e8d2d68316b296cc9343ab5f3d2babc5e6e03b80cd580478e
```

> \[!NOTE]说明
>
> * isula-build 支持导入最大50G的容器层叠镜像。
> * isula-build 会自动识别容器层叠镜像的格式并进行导入。

### rm: 删除本地持久化镜像

可通过rm命令删除当前本地持久化存储的镜像。命令原型为：

```sh
isula-build ctr-img rm IMAGE [IMAGE...] [FLAGS]
```

目前支持的 flags 为：

* -a, --all：删除所有本地持久化存储的镜像。
* -p, --prune：删除所有没有tag的本地持久化存储的镜像。

使用示例如下：

```sh
$ sudo isula-build ctr-img rm -p
Deleted: sha256:78731c1dde25361f539555edaf8f0b24132085b7cab6ecb90de63d72fa00c01d
Deleted: sha256:eeba1bfe9fca569a894d525ed291bdaef389d28a88c288914c1a9db7261ad12c
```

### save: 导出层叠镜像

可通过save命令导出层叠镜像到本地磁盘。命令原型如下：

```sh
isula-build ctr-img save [REPOSITORY:TAG]|imageID -o xx.tar
```

目前支持的 flags 为：

* -f, --format：导出层叠镜像的镜像格式：oci | docker（需开启实验特性选项）
* -o, --output：本地tar包路径

以下示例通过 `image/tag` 的形式将镜像进行导出：

```sh
$ sudo isula-build ctr-img save busybox:latest -o busybox.tar
Getting image source signatures
Copying blob sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a
Copying blob sha256:824082a6864774d5527bda0d3c7ebd5ddc349daadf2aa8f5f305b7a2e439806f
Copying blob sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef
Copying config sha256:21c3e96ac411242a0e876af269c0cbe9d071626bdfb7cc79bfa2ddb9f7a82db6
Writing manifest to image destination
Storing signatures
Save success with image: busybox:latest
```

以下示例通过 `ImageID` 的形式将镜像进行导出：

```sh
$ sudo isula-build ctr-img save 21c3e96ac411 -o busybox.tar
Getting image source signatures
Copying blob sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a
Copying blob sha256:824082a6864774d5527bda0d3c7ebd5ddc349daadf2aa8f5f305b7a2e439806f
Copying blob sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef
Copying config sha256:21c3e96ac411242a0e876af269c0cbe9d071626bdfb7cc79bfa2ddb9f7a82db6
Writing manifest to image destination
Storing signatures
Save success with image: 21c3e96ac411
```

以下示例导出多个镜像到同一个tar包：

```sh
$ sudo isula-build ctr-img save busybox:latest nginx:latest -o all.tar
Getting image source signatures
Copying blob sha256:eb78099fbf7fdc70c65f286f4edc6659fcda510b3d1cfe1caa6452cc671427bf
Copying blob sha256:29f11c413898c5aad8ed89ad5446e89e439e8cfa217cbb404ef2dbd6e1e8d6a5
Copying blob sha256:af5bd3938f60ece203cd76358d8bde91968e56491daf3030f6415f103de26820
Copying config sha256:b8efb18f159bd948486f18bd8940b56fd2298b438229f5bd2bcf4cedcf037448
Writing manifest to image destination
Storing signatures
Getting image source signatures
Copying blob sha256:e2d6930974a28887b15367769d9666116027c411b7e6c4025f7c850df1e45038
Copying config sha256:a33de3c85292c9e65681c2e19b8298d12087749b71a504a23c576090891eedd6
Writing manifest to image destination
Storing signatures
Save success with image: [busybox:latest nginx:latest]
```

> \[!NOTE]说明
>
> * save 导出的镜像默认格式为未压缩的tar格式，如有需求，用户可以再save之后手动压缩。
> * 在使用镜像名导出镜像时，需要给出完整的镜像名格式：REPOSITORY:TAG。

### tag: 给本地持久化镜像打标签

可使用tag命令给本地持久化的容器镜像打tag。命令原型如下：

```sh
isula-build ctr-img tag <imageID>/<imageName> busybox:latest
```

使用举例：

```sh
$ sudo isula-build ctr-img images
---------------------------------------  -----------  -----------------  --------------------------  ------------
REPOSITORY                               TAG          IMAGE ID           CREATED                     SIZE
---------------------------------------  -----------  -----------------  --------------------------  ------------
alpine                                   latest       a24bb4013296       2020-05-29 21:19:46         5.85 MB
---------------------------------------  -----------  -----------------  --------------------------  ------------
$ sudo isula-build ctr-img tag a24bb4013296 alpine:v1
$ sudo isula-build ctr-img images
---------------------------------------  -----------  -----------------  ------------------------  ------------
REPOSITORY                               TAG          IMAGE ID           CREATED                   SIZE
---------------------------------------  -----------  -----------------  ------------------------  ------------
alpine                                   latest       a24bb4013296       2020-05-29 21:19:46       5.85 MB
alpine                                   v1           a24bb4013296       2020-05-29 21:19:46       5.85 MB
---------------------------------------  -----------  -----------------  ------------------------  ------------
```

### pull: 拉取镜像到本地

可通过pull命令拉取远程镜像仓库中的镜像到本地。命令原型如下：

```sh
isula-build ctr-img pull REPOSITORY[:TAG]
```

使用示例：

```sh
$ sudo isula-build ctr-img pull example-registry/library/alpine:latest
Getting image source signatures
Copying blob sha256:8f52abd3da461b2c0c11fda7a1b53413f1a92320eb96525ddf92c0b5cde781ad
Copying config sha256:e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b
Writing manifest to image destination
Storing signatures
Pull success with image: example-registry/library/alpine:latest
```

### push: 将本地镜像推送到远程仓库

可通过push命令将本地镜像推送到远程仓库。命令原型如下：

```sh
isula-build ctr-img push REPOSITORY[:TAG]
```

目前支持的 flags 为：

* -f, --format：推送的镜像格式：oci｜docker（需开启实验特性选项）

使用示例：

```sh
$ sudo isula-build ctr-img push example-registry/library/mybusybox:latest
Getting image source signatures
Copying blob sha256:d2421964bad195c959ba147ad21626ccddc73a4f2638664ad1c07bd9df48a675
Copying config sha256:f0b02e9d092d905d0d87a8455a1ae3e9bb47b4aa3dc125125ca5cd10d6441c9f
Writing manifest to image destination
Storing signatures
Push success with image: example-registry/library/mybusybox:latest
```

> \[!NOTE]说明
>
> 推送镜像时，需要先登录对应的镜像仓库

## info: 查看运行环境与系统信息

可以通过“isula-build info”指令查看 isula-build 目前的运行环境与系统信息。命令原型如下：

```sh
isula-build info [flags]
```

支持如下Flags：

* -H, --human-readable 布尔值，以常用内存表示格式打印内存信息，使用1000次幂
* -V, --verbose 布尔值，显示运行时内存占用信息

使用示例：

```sh
$ sudo isula-build info -HV
   General:
     MemTotal:     7.63 GB
     MemFree:      757 MB
     SwapTotal:    8.3 GB
     SwapFree:     8.25 GB
     OCI Runtime:  runc
     DataRoot:     /var/lib/isula-build/
     RunRoot:      /var/run/isula-build/
     Builders:     0
     Goroutines:   12
   Store:
     Storage Driver:     overlay
     Backing Filesystem: extfs
   Registry:
     Search Registries:
       oepkgs.net
     Insecure Registries:
       localhost:5000
       oepkgs.net
   Runtime:
  MemSys:           68.4 MB
     HeapSys:          63.3 MB
     HeapAlloc:        7.41 MB
     MemHeapInUse:     8.98 MB
     MemHeapIdle:      54.4 MB
     MemHeapReleased:  52.1 MB
```

## login: 登录远端镜像仓库

用户可以运行 login 命令来登录远程镜像仓库。命令原型如下：

```sh
 isula-build login SERVER [FLAGS]
```

目前支持的flag有：

```Conf
 Flags:
   -p, --password-stdin    Read password from stdin
   -u, --username string   Username to access registry
```

通过stdin输入密码。以下示例通过通过管道将creds.txt里的密码传给isula-build的stdin进行输入：

```sh
 $ cat creds.txt | sudo isula-build login -u cooper -p mydockerhub.io
 Login Succeeded
```

通过交互式输入密码：

```sh
 $ sudo isula-build login mydockerhub.io -u cooper
 Password:
 Login Succeeded
```

## logout: 退出远端镜像仓库

用户可以运行 logout 命令来登出远程镜像仓库。命令原型如下：

```sh
isula-build logout [SERVER] [FLAGS]
```

目前支持的flag有：

```sh
 Flags:
   -a, --all   Logout all registries
```

使用示例如下：

```sh
$ sudo isula-build logout -a
  Removed authentications
```

## version: 版本查询

可通过version命令查看当前版本信息：

```sh
$ sudo isula-build version
Client:
  Version:       0.9.6-4
  Go Version:    go1.15.7
  Git Commit:    83274e0
  Built:         Wed Jan 12 15:32:55 2022
  OS/Arch:       linux/amd64

Server:
  Version:       0.9.6-4
  Go Version:    go1.15.7
  Git Commit:    83274e0
  Built:         Wed Jan 12 15:32:55 2022
  OS/Arch:       linux/amd64
```

## manifest: manifest列表管理

manifest列表包含不同系统架构对应的镜像信息，通过使用manifest列表，用户可以在不同的架构中使用相同的manifest（例如openeuler:latest）获取对应架构的镜像，manifest包含create、annotate、inspect和push子命令。

> \[!NOTE]说明
>
> manifest为实验特性，使用时需开启客户端和服务端的实验选项，方式详见客户端总体说明和配置服务章节。

### create: manifest列表创建

manifest的子命令create用于创建manifest列表，命令原型为：

```sh
isula-build manifest create MANIFEST_LIST MANIFEST [MANIFEST...]
```

用户可以指定manifest列表的名称以及需要加入到列表中的远程镜像，若不指定任何远程镜像，则会创建一个空的manifest列表。

使用示例如下：

```sh
sudo isula-build manifest create openeuler localhost:5000/openeuler_x86:latest localhost:5000/openeuler_aarch64:latest
```

### annotate: manifest列表更新

manifest的子命令annotate用于更新manifest列表，命令原型为：

```sh
isula-build manifest annotate MANIFEST_LIST MANIFEST [flags]
```

用户可以指定需要更新的manifest列表以及其中的镜像，通过flags指定需要更新的选项，此命令也可用于添加新的镜像到列表中。

其中annotate包含如下flags：

* \--arch： string，重写镜像适用架构
* \--os： string，重写镜像适用系统
* \--os-features： string列表，指定镜像需要的OS特性，很少使用
* \--variant： string，指定列表中记录镜像的变量

使用示例如下：

```sh
sudo isula-build manifest annotate --os linux --arch arm64 openeuler:latest localhost:5000/openeuler_aarch64:latest
```

### inspect: manifest列表查询

manifest子命令inspect用于查询manifest列表信息，命令原型为：

```sh
isula-build manifest inspect MANIFEST_LIST
```

使用示例如下：

```sh
$ sudo isula-build manifest inspect openeuler:latest
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 527,
            "digest": "sha256:bf510723d2cd2d4e3f5ce7e93bf1e52c8fd76831995ac3bd3f90ecc866643aff",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        },
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 527,
            "digest": "sha256:f814888b4bb6149bd39ba8375a1932fb15071b4dbffc7f76c7b602b06abbb820",
            "platform": {
                "architecture": "arm64",
                "os": "linux"
            }
        }
    ]
}
```

### push: 将manifest列表推送到远程仓库

manifest子命令push用于将manifest列表推送到远程仓库，命令原型为：

```sh
isula-build manifest push MANIFEST_LIST DESTINATION
```

使用示例如下：

```sh
sudo isula-build manifest push openeuler:latest localhost:5000/openeuler:latest
```

# 直接集成容器引擎

isula-build可以与iSulad和docker集成，将构建好的容器镜像导入到容器引擎的本地存储中。

## 与iSulad集成

支持将构建成功的镜像直接导出到iSulad。

命令行举例：

```sh
sudo isula-build ctr-img build -f Dockerfile -o isulad:busybox:2.0
```

通过在-o参数中指定iSulad，将构建好的容器镜像导出到iSulad，可以通过isula images查询：

```sh
$ sudo isula images
isula images
REPOSITORY                     TAG        IMAGE ID             CREATED              SIZE
busybox                        2.0        2d414a5cad6d         2020-08-01 06:41:36  5.577 MB
```

> \[!NOTE]说明
>
> * 要求isula-build和iSulad在同一节点。
> * 直接导出镜像到iSulad时，isula-build client端需要将构建成功的镜像暂存成 `/var/lib/isula-build/tmp/[buildid]/isula-build-tmp-%v.tar` 再导入至 iSulad，用户需要保证 /var/lib/isula-build/tmp/ 目录有足够磁盘空间；同时如果在导出过程中 isula-build client进程被KILL或Ctrl+C终止，需要依赖用户手动清理 `/var/lib/isula-build/tmp/[buildid]/isula-build-tmp-%v.tar` 文件。

## 与Docker集成

支持将构建成功的镜像直接导出到Docker daemon。

命令行举例：

```sh
sudo isula-build ctr-img build -f Dockerfile -o docker-daemon:busybox:2.0
```

通过在-o参数中指定docker-daemon，将构建好的容器镜像导出到docker, 可以通过docker images查询。

```sh
$ sudo docker images
REPOSITORY                                          TAG                 IMAGE ID            CREATED             SIZE
busybox                                             2.0                 2d414a5cad6d        2 months ago        5.22MB
```

> \[!NOTE]说明
>
> 要求isula-build和Docker在同一节点。

# 使用注意事项

本章节主要介绍在使用isula-build构建镜像时相关的约束和限制，以及与docker build的差异。

## 约束和限制

1. 当导出镜像到[`iSulad`](https://atomgit.com/openeuler/iSulad/blob/master/README.md/)时，镜像必须指明tag。
2. 因为isula-builder运行`RUN`指令时，需要调用系统中的oci 运行时（如`runc`），用户需要保证该运行时的安全性，不受篡改。
3. `DataRoot`不能设置在内存盘上（tmpfs）。
4. `Overlay2`是目前isula-builder唯一支持的存储驱动。
5. `Docker`镜像是目前唯一支持的镜像格式，未来即将支持`oci`格式镜像。
6. `Dockerfile`文件权限强烈建议设置为**0600**以防止恶意篡改。
7. `RUN`命令中目前只支持主机侧网络（host network）。
8. 当导出镜像到本地tar包时，目前只支持保存为`tar`格式。
9. 当使用`import`功能导入基础镜像时，最大支持**1G**。

## 与“docker build”差异

`isula-build`兼容[Docker镜像格式规范](https://docs.docker.com/reference/dockerfile/)，但仍然和`docker build`存在一些差异：

1. 支持镜像压缩，即对每个`stage`进行提交而非每一行。
2. 目前不支持构建缓存。
3. 只有`RUN`指令会运行容器进行构建。
4. 目前不支持查询镜像构建历史。
5. `Stage`名称可以用数字开头。
6. `Stage`名称最大长度为64。
7. `ADD`命令不支持远端URL格式。
8. 暂不支持对单次构建进行资源限额，可采取对isula-builder配置资源限额的方式进行限制。
9. 统计镜像大小时，isula-build是直接计算每层tar包大小之和，而docker是通过解压tar遍历diff目录计算文件大小之和，因此通过`isula-build ctr-img images`查看的镜像大小与`docker images`的显示上有一定差异。
10. 操作时的镜像名称需要明确，格式为IMAGE\_NAME:IMAGE\_TAG。例如 busybox:latest, 其中latest不可省略。

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/installing_etcd.md
---
# 安装 etcd

## 准备环境

使能 etcd 使用的端口：

```bash
firewall-cmd --zone=public --add-port=2379/tcp
firewall-cmd --zone=public --add-port=2380/tcp
```

## 安装 etcd 二进制

当前是通过 rpm 包安装

```bash
rpm -ivh etcd*.rpm
```

准备目录

```bash
mkdir -p /etc/etcd /var/lib/etcd
cp ca.pem /etc/etcd/
cp kubernetes-key.pem /etc/etcd/
cp kubernetes.pem /etc/etcd/
# 关闭selinux
setenforce 0
# 禁用/etc/etcd/etcd.conf文件的默认配置
# 注释掉即可，例如：ETCD_LISTEN_CLIENT_URLS="http://localhost:2379"
```

## 编写 etcd.service 文件

以 `k8smaster0`机器为例：

```bash
$ cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "ETCD_UNSUPPORTED_ARCH=arm64 /usr/bin/etcd --name=k8smaster0 --cert-file=/etc/etcd/kubernetes.pem --key-file=/etc/etcd/kubernetes-key.pem --peer-cert-file=/etc/etcd/kubernetes.pem --peer-key-file=/etc/etcd/kubernetes-key.pem --trusted-ca-file=/etc/etcd/ca.pem --peer-trusted-ca-file=/etc/etcd/ca.pem --peer-client-cert-auth --client-cert-auth --initial-advertise-peer-urls https://192.168.122.154:2380 --listen-peer-urls https://192.168.122.154:2380 --listen-client-urls https://192.168.122.154:2379,https://127.0.0.1:2379 --advertise-client-urls https://192.168.122.154:2379 --initial-cluster-token etcd-cluster-0 --initial-cluster k8smaster0=https://192.168.122.154:2380,k8smaster1=https://192.168.122.155:2380,k8smaster2=https://192.168.122.156:2380 --initial-cluster-state new --data-dir /var/lib/etcd"

Restart=always
RestartSec=10s
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
```

**注意:**

* arm64上面需要增加启动设置`ETCD_UNSUPPORTED_ARCH=arm64`；
* 由于本文把etcd和k8s control部署在相同机器，所以使用了`kubernetes.pem`和`kubernetes-key.pem`证书来启动；
* ca证书，在整个部署流程里面使用了一个，etcd可以生成自己的ca，然后用自己的ca签名其他证书，但是需要在apiserver访问etcd的client用该ca签名的证书；
* `initial-cluster`需要把所有部署etcd的配置加上；
* 为了提高etcd的存储效率，可以使用ssd硬盘的目录，作为`data-dir`；

启动服务

```bash
$ systemctl enable etcd
$ systemctl start etcd
```

然后，依次部署其他机器即可。

## 验证基本功能

```bash
$  ETCDCTL_API=3 etcdctl -w table endpoint status --endpoints=https://192.168.122.155:2379,https://192.168.122.156:2379,https://192.168.122.154:2379   --cacert=/etc/etcd/ca.pem   --cert=/etc/etcd/kubernetes.pem   --key=/etc/etcd/kubernetes-key.pem
+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|           ENDPOINT           |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFTAPPLIED INDEX | ERRORS |
+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://192.168.122.155:2379 | b50ec873e253ebaa |  3.4.14 |  262 kB |     false |      false |       819 |         21 |           21 |        |
| https://192.168.122.156:2379 | e2b0d126774c6d02 |  3.4.14 |  262 kB |      true |      false |       819 |         21 |           21 |        |
| https://192.168.122.154:2379 | f93b3808e944c379 |  3.4.14 |  328 kB |     false |      false |       819 |         21 |           21 |        |
+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/installing_the_kubernetes_software_package.md
---
# 安装 Kubernetes 软件包

通过dnf安装k8s所需要的依赖工具包。

```bash
dnf install -y docker conntrack-tools socat
```

配置EPOL源之后，可以直接通过 dnf 安装 K8S。

```bash
dnf install kubernetes
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/security/secdetector/install_secdetector.md
---
# 安装 secDetector

## 软硬件要求

### 硬件要求

* 当前仅支持 x86\_64、aarch64 架构处理器。
* secDetector磁盘使用需求：配额1GB及以上。
* secDetector内存使用需求：配额100MB及以上。

### 操作系统要求

openEuler 22.03 LTS SP4 及以上。

### 环境准备

安装 openEuler 系统，安装方法参考《[安装指南](./../../installation_upgrade/installation/installation_guide.md)》。

## 安装 secDetector

1. 配置openEuler yum源：openEuler 22.03-LTS 发布版本上已默认配置完成yum源，无需额外操作。特殊情况下请参考openEuler官方文档配置在线yum源或通过ISO挂载配置本地yum源。

2. 安装secDetector。

   ```shell
   #安装secDetector
   sudo yum install secDetector
   ```

> \[!NOTE]说明
>
> 安装secDetector后在指定目录下可获得部署secDetector所需的相关文件：

```shell
#secDetector的kerneldriver的核心框架
/lib/modules/%{kernel_version}/extra/secDetector/secDetector_core.ko

#secDetector的kerneldriver的功能组件
/lib/modules/%{kernel_version}/extra/secDetector/secDetector_xxx.ko

#secDetector的守护者进程文件
/usr/bin/secDetectord

#secDetector的SDK库文件
/usr/lib64/secDetector/libsecDetectorsdk.so
/usr/include/secDetector/secDetector_sdk.h
/usr/include/secDetector/secDetector_topic.h
```

## 部署 secDetector

secDetector的主体secDetectord是以系统服务的形式部署在系统中的，前台业务系统可以通过集成SDK来与之通信。由于secDetector的部分能力必须构建在内核之中，因此secDetectord的功能全集还依赖于其后台驱动的部署。

### 部署 kernel driver

1. 插入 kernel driver 的基础框架：secDetector\_core.ko 是 kernel driver 的基础框架，要优先于其他内核模块进行部署。找到安装后的 secDetector\_core.ko 目录，将其插入内核。参考命令如下：

   ```shell
   sudo insmod secDetector_core.ko
   ```

   secDetector\_core 支持一个命令行参数ringbuf\_size。用户可以通过指定该参数的值来控制 kernel driver 与 用户态secDetectord之间数据通道的缓存空间尺寸。该参数可以被指定为4~1024中的一个整数，单位是MB。默认值是4，必须为2的幂。参考命令如下：

   ```shell
   sudo insmod secDetector_core.ko ringbuf_size=128
   ```

2. 插入 kernel driver 的功能模块：secDetector的 kernel driver 采用模块化部署方式。用户可以选择基于框架部署满足需要的功能模块，也可以选择部署全部模块。参考命令如下：

   ```shell
   sudo insmod secDetector_kmodule_baseline.ko

   sudo insmod secDetector_memory_corruption.ko

   sudo insmod secDetector_program_action.ko

   sudo insmod secDetector_xxx.ko
   ```

   * secDetector\_kmodule\_baseline.ko 提供了内核模块列表检测的能力，属于内存修改类探针；
   * secDetector\_memory\_corruption.ko 提供了内存修改检测的能力，属于内存修改类探针；
   * secDetector\_program\_action.ko 提供了程序行为检测的能力，属于程序行为类探针。

### 部署 usr driver 和 observer\_agent

当前用户态驱动 usr driver 和服务 observer\_agent 已经都被集成到secDetectord中，参考命令如下：

```shell
sudo ./secDetectord &
```

usr driver当前包含了文件操作类探针和进程管理类探针的能力。

secDetectord支持如下一些配置选项：

```shell
用法：secDetectord [选项]
secDetectord 默认会在后台运行，从探针中取得数据并转发给订阅者。
选项：
  -d         进入调试模式，进入前台运行，并且在控制台打印探针数据。
  -s <size>  配置eBPF缓冲区大小，单位为Mb，默认为4； size可选范围为4~1024，且必须为2的幂次方。当前拥有2个独立的缓冲区。
  -t <topic> 支持配置订阅的事件，默认为所有事件。topic 是位图格式。例如 -t 0x60 同时订阅进程创建和进程退出事件。详细请查阅 include/secDetector_topic.h。
```

### 部署SDK

SDK的库文件默认已经被部署到系统库目录中，用户需要在自己的程序中引用SDK的头文件即可使用。

---

---
url: /zh/docs/22.03_LTS_SP4/server/security/secgear/secgear_installation.md
---
# 安装 secGear

## ARM环境

### 环境要求

#### 硬件环境

| 项目   | 版本                                                |
| ------ | --------------------------------------------------- |
| 服务器 | TaiShan 200服务器（型号2280）                       |
| 主板   | 鲲鹏主板                                            |
| BMC    | 1711单板（型号BC82SMMAB），固件版本不低于3.01.12.49 |
| CPU    | 鲲鹏920处理器（型号7260、5250、5220）               |
| 机箱   | 不限，建议8盘或12盘                                 |

> \[!NOTE]说明
>
> * 要求服务器已经预置TrustZone特性套件，即预装TEE OS、TEE OS启动密钥、BMC、BIOS和License许可证。
> * 普通服务器无法仅通过升级BMC、BIOS、TEE OS固件实现TrustZone特性使能。
> * 带TrustZone特性的服务器出厂默认特性关闭，请参考BIOS设置使能服务器TrustZone特性。

#### 操作系统

openEuler 22.03 LTS SP4及以上

openEuler 22.09

openEuler 22.03 LTS及以上

### 环境准备

参考鲲鹏官网[环境要求](https://www.hikunpeng.com/document/detail/zh/kunpengcctrustzone/overview/kunpengcctrustzone.html)和[搭建步骤](https://www.hikunpeng.com/document/detail/zh/kunpengcctrustzone/overview/kunpengcctrustzone.html)。

### 安装操作

1. 配置openEuler yum源，在线yum源或通过ISO挂载配置本地yum源，配置在线源如下（仅以22.03-LTS-SP4举例，其他版本需要使用版本对应的yum源）。

   ```shell
   vi openEuler.repo 
   [osrepo]
   name=osrepo
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/aarch64/
   enabled=1
   gpgcheck=1
   gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/aarch64/RPM-GPG-KEY-openEuler
   ```

2. 安装secGear

   ```shell
   #安装编译工具
   yum install cmake ocaml-dune

   #安装secGear
   yum install secGear-devel

   #检查是否安装成功。命令和回显如下表示安装成功。
   rpm -qa | grep -E 'secGear|itrustee|ocaml-dune'
   itrustee_sdk-xxx
   itrustee_sdk-devel-xxx
   secGear-xxx
   secGear-devel-xxx
   ocaml-dune-xxx
   ```

## X86环境

### 环境要求

#### 硬件环境

支持Intel SGX（Intel Software Guard Extensions） 特性的处理器。

#### 操作系统

openEuler 22.03 LTS SP4及以上

openEuler 22.09

openEuler 22.03 LTS及以上

### 环境准备

购买支持Intel SGX特性设备，参考对应设备BIOS配置手册，开启SGX特性。

### 安装操作

1. 配置openEuler yum源，在线yum源或通过ISO挂载配置本地yum源，配置在线源如下（仅以22.03-LTS-SP4举例，其他版本需要使用版本对应的yum源）。

   ```shell
   vi openEuler.repo 
   [osrepo]
   name=osrepo
   baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
   enabled=1
   gpgcheck=1
   gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/RPM-GPG-KEY-openEuler
   ```

2. 安装secGear

   ```shell
   #安装编译工具
   yum install cmake ocaml-dune

   #安装secGear
   yum install secGear-devel

   #检查是否安装成功。命令和回显如下表示安装成功。
   rpm -qa | grep -E 'secGear|ocaml-dune|sgx'
   secGear-xxx
   secGear-devel-xxx
   ocaml-dune-xxx
   libsgx-epid-xxx
   libsgx-enclave-common-xxx
   libsgx-quote-ex-xxx
   libsgx-aesm-launch-plugin-xxx
   libsgx-uae-service-xxx
   libsgx-ae-le-xxx
   libsgx-urts-xxx
   sgxsdk-xxx
   sgx-aesm-service-xxx
   linux-sgx-driver-xxx
   libsgx-launch-xxx
   ```

---

---
url: /zh/docs/22.03_LTS_SP4/server/development/application_dev/installing_obs.md
---
# 安装obs工具

## 介绍

OBS（Open Build Service）是通用编译工具，用于将源码包构建为 RPM 软件包或 Linux 镜像。
obs-server 是 OBS 对应的软件包。

## 架构适配

支持在多种架构平台（x86\_64、AArch64 等）上使用。

## 安装

此处以 openEuler 22.03 LTS SP4版本的 AArch64 架构为例，介绍 obs-server 多版本分支软件包的安装操作。

1. 确认设备已安装操作系统 openEuler 22.03 LTS SP4。

   ```shell
   $ cat /etc/openEuler-release
   openEuler release 22.03 LTS SP4
   ```

2. 配置 yum 源。yum 源配置内容参考如下：

   ```shell
   [everything]
   name=everything
   baseurl=https://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/aarch64/
   enabled=1
   gpgcheck=0
   ```

   执行下述命令拷贝上述文本到 repo 源中：

   ```shell
   sudo vi /etc/yum.repos.d/xxx.repo
   ```

3. 使 yum 源生效。

   ```shell
   sudo yum clean all
   sudo yum makecache
   ```

4. 检查是否已安装其他版本的软件包。

   ```shell
   sudo rpm -qa obs-server obs-common obs-api mod_passenger obs-api-deps obs-bundled-gems passenger ruby ruby-help ruby-irb rubygem-bundler rubygem-io-console rubygem-json rubygem-openssl rubygem-psych rubygem-rake rubygem-rdoc rubygems rubygem-bigdecimal rubygem-did_you_mean
   ```

5. （可选）如果有其他版本的软件包，请卸载软件包防止安装冲突。

   ```shell
   sudo yum remove -y obs-server obs-common obs-api mod_passenger obs-api-deps obs-bundled-gems passenger ruby ruby-help ruby-irb rubygem-bundler rubygem-io-console rubygem-json rubygem-openssl rubygem-psych rubygem-rake rubygem-rdoc rubygems rubygem-bigdecimal rubygem-did_you_mean
   ```

   > **注意事项：**
   >
   > * 该分支作为 openEuler 22.03 LTS SP4版本发布的 obs-server 多版本分支。
   > * 安装依赖包可能由于存在多个版本，导致依赖冲突安装失败。所以安装前先卸载上述软件包。

6. 安装 obs-server 相关软件包。

   ```shell
   sudo yum install -y obs-api obs-server
   ```

7. 检查 obs-server 软件包安装成功。

   ```shell
   $ rpm -qa | grep obs-server
   obs-server-2.10.15-7.oe2203SP4.noarch
   $ rpm -qa | grep obs-api
   obs-api-2.10.15-7.oe2203SP4.noarch
   ```

## 部署

1. 获取部署脚本，具体请参见获取链接：
   <https://atomgit.com/openeuler/infrastructure/tree/master/obs/tf/startup>

2. 执行 restart\_service.sh 脚本部署 OBS 工具。

## 使用说明

可以通过 OBS 网页和osc命令的方式构建RPM软件包。详细内容可参考[《构建RPM包》](https://docs.openeuler.org/zh/docs/22.03_LTS_SP4/docs/ApplicationDev/%E6%9E%84%E5%BB%BARPM%E5%8C%85.html)。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/install_stratovirt.md
---
# 安装StratoVirt

## 软硬件要求

### 最低硬件要求

* 处理器架构：仅支持AArch64和x86\_64处理器架构。AArch64需要ARMv8及更高版本且支持虚拟化扩展；x86\_64支持VT-x。

* 2核CPU

* 4GiB内存

* 16GiB可用磁盘空间

### 软件要求

操作系统：openEuler 22.03 LTS SP4

## 安装组件

使用StratoVirt虚拟化，需要安装StratoVirt。安装前，请确保已经配置了openEuler yum源。

1. 使用root权限，安装StratoVirt组件，参考命令如下：

   ```sh
   # yum install stratovirt
   ```

2. 查看是否安装成功。

   ```sh
   $ stratovirt -version
   StratoVirt 2.1.0
   ```

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/syscare/installing_syscare.md
---
# 安装SysCare

本章介绍在openEuler中安装SysCare的方法

## 安装SysCare核心组件

### 最低硬件要求

* 2 CPU (x86\_64 / aarch64)
* 4GB RAM
* 100GB 硬盘

### 前提条件

1. 安装openEuler 22.03 LTS SP4版本；
2. 补丁制作需要root权限。

### 源码编译安装SysCare

SysCare源码已经归档至代码仓：<https://atomgit.com/openeuler/syscare.git>。
用户可自行到代码仓下载并编译安装。

示例如下：

```shell
dnf install -y kernel-source-`uname -r` kernel-debuginfo-`uname -r` kernel-devel-`uname -r`
dnf install -y elfutils-libelf-devel openssl-devel dwarves flex python3-devel rpm-build bison cmake make gcc g++ rust cargo bpftool clang libbpf libbpf-devel llvm libbpf-static

git clone https://atomgit.com/openeuler/syscare.git
cd syscare
mkdir build_tmp
cd build_tmp
cmake -DCMAKE_INSTALL_PREFIX=/usr ..
make
make install
```

### repo安装SysCare

SysCare已集成在openEuler 22.03 LTS SP4的软件源中，可以通过dnf或yum命令直接进行下载、安装。

```shell
dnf install syscare syscare-build
```

### 卸载SysCare

```shell
dnf remove syscare*
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/maintenance/syssentry/installation_and_usage.md
---
# 安装sysSentry

## 操作系统

openEuler-22.03-LTS-SP4

## 环境准备

* 安装openEuler-22.03-LTS-SP4系统，安装方法参考 《[安装指南](./../../installation_upgrade/installation/installation_guide.md)》。

* 安装sysSentry需要使用root权限。

## 配置yum源

在/etc/yum.repos.d/创建.repo文件（eg. /etc/yum.repos.d/openEuler.repo），使用编辑器打开.repo文件添加如下内容：

**aarch64架构：**

```ini
[openEuler-22.03-LTS-SP4]
name=openEuler-22.03-LTS-SP4
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/aarch64/
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/aarch64/RPM-GPG-KEY-openEuler

[openEuler-22.03-LTS-SP4-update]
name=openEuler-22.03-LTS-SP4-update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/update/aarch64/
enabled=1
gpgcheck=0
```

**x86\_64架构：**

```ini
[openEuler-22.03-LTS-SP4]
name=openEuler-22.03-LTS-SP4
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/
enabled=1
gpgcheck=1
gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/everything/x86_64/RPM-GPG-KEY-openEuler

[openEuler-22.03-LTS-SP4-update]
name=openEuler-22.03-LTS-SP4-update
baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/update/x86_64/
enabled=1
gpgcheck=0
```

## 安装sysSentry

```sh
yum install sysSentry libxalarm -y
```

# 使用sysSentry

## 启动巡检框架

```sh
systemctl start sysSentry
systemctl start xalarmd
# 执行成功后可通过status命令查看状态为running
systemctl status sysSentry
systemctl status xalarmd
```

## 配置巡检任务

sysSentry巡检框架以巡检模块（module）的形式管理巡检任务，每个巡检模块由/etc/sysSentry/tasks/目录下对应的.mod配置文件定义，巡检模块的名称即为.mod文件的名称，.mod配置文件用于配置巡检任务的运行参数。

## 管理巡检任务

* 启动指定巡检任务

```sh
sentryctl start <module_name>
```

* 终止指定巡检任务

```sh
sentryctl stop <module_name>
```

* 列出所有已加载的巡检任务及状态

```sh
sentryctl list
```

例如：

![输入图片说明](figures/sentryctl_list.png)

* 查询指定巡检任务的状态

```sh
sentryctl status <module_name>
```

巡检状态回显信息：

| 状态 | 描述 |
| --- | --- |
| RUNNING | 巡检任务正在运行  |
| WAITING | 仅period类型巡检任务可设置此状态，表示period巡检任务等待下一次被调度执行  |
| EXITED | 巡检任务尚未执行，或者oneshot类型的巡检任务执行结束处于此状态 |
| FAILED | 巡检任务未拉起成功，或者巡检任务未正常退出 |

* 重载指定任务配置

```sh
sentryctl reload <module_name>
```

* 查询指定巡检任务的巡检结果

```sh
sentryctl get_result <module_name>
```

回显信息格式为json格式，内容格式如下：

```json
{
    "result": "xxx",
    "start_time": "YY-mm-DD HH:MM:SS",
    "end_time": "YY-mm-DD HH:MM:SS",
    "error_msg" : "xxx",
    "details":{}  # detail信息因巡检任务不同而不同，具体信息由巡检模块提供
}
```

"result"和"error\_msg"对应关系如下：

| result | 对应error\_msg信息  |
| --- | --- |
| PASS | "" |
| SKIP | "not supported.maybe some rpm package not be installed." |
| FAIL | "FAILED. config may be incorrect or the command may be invalid/killed!" |
| MINOR\_ALM |"the command output shows that the status is 'INFO' or 'GENERAL\_WARN'."  |
| MAJOR\_ALM | "the command output shows that the status is 'WARN' or 'IMPORTANT\_WARN'." |
| CRITICAL\_ALM | "the command output shows that the status is 'FAIL' or 'EMERGENCY\_WARN'." |

# 常见问题Q\&A

* 短时间内频繁启动并关闭sysSentry服务，查看服务启动日志会出现`RuntimeError: reentrant call inside <_io.BufferedWriter name='</var/run/sysSentry/sysSentry.pid>'>` 错误信息，这对功能是否受影响？

  启动sysSentry服务又短时间内停止sysSentry服务导致sysSentry初始化流程未结束就接收到终止信号，可能会触发两次sig\_handler流程。如果第一次流程正好执行到close()的时候触发了第二次sig\_handler流程，这就会引发RuntimeError。该问题不影响正常功能，建议用户启动sysSentry服务后不要立即关闭sysSentry服务。

---

---
url: /zh/docs/22.03_LTS_SP4/server/performance/powerapi/installation_usage.md
---
# 安装与使用

## 安装

powerapi 已经引入到 openEuler22.03-LTS-SP4 版本，可以使用 yum 执行以下命令进行安装：

```sh
yum install powerapi
```

## 使用

powerapi 安装后，会自动运行 pwrapis，为 eagle 和第三方系统提供功耗管理相关服务。\
powerapi 暂不支持以命令行的方式使用，对于单个函数的调用，需要安装 powerapi-devel，在源代码中包含头文件后调用相关函数。详细使用方法可见下一章[《使用 powerapi 进行开发》](./development_using_powerapi.md)。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/hybrid_deployment/rubik/installation_and_deployment.md
---
# 安装与部署

## 概述

本章节主要介绍 rubik 组件的安装以及部署方式，以openEuler 24.03-LTS-SP1版本为例进行部署。

## 软硬件要求

### 硬件要求

* 当前仅支持 x86、aarch64 架构。
* rubik 磁盘使用需求：配额 1GB 及以上。
* rubik 内存使用需求：配额 100MB 及以上。

### 软件要求

* 操作系统：openEuler 24.03-LTS-SP1。
* 内核：openEuler 24.03-LTS-SP1 版本内核。

### 环境准备

* 安装 openEuler 系统。
* 安装并部署 kubernetes。
* 安装 docker 或 containerd 容器引擎。

## 安装 rubik

rubik 以`DaemonSet`形式部署在 k8s 的每一个节点上，故需要在每一个节点上使用以下步骤安装 rubik rpm 包。

1. 配置 yum 源：rubik组件位于openEuler EPOL源中，以openEuler 24.03-LTS-SP1版本为例，参考如下：

   ```sh
   # openEuler 24.03-LTS-SP1 官方发布源
   name=openEuler24.03-LTS-SP1
   baseurl=https://repo.openeuler.org/openEuler-24.03-LTS-SP1/everything/$basearch/ 
   enabled=1
   gpgcheck=1
   gpgkey=https://repo.openeuler.org/openEuler-24.03-LTS-SP1/everything/$basearch/RPM-GPG-KEY-openEuler
   ```

   ```sh
   # openEuler 24.03-LTS-SP1:Epol 官方发布源
   name=openEuler24.03-LTS-SP1-Epol
   baseurl=https://repo.openeuler.org/openEuler-24.03-LTS-SP1/EPOL/$basearch/
   enabled=1
   gpgcheck=1
   gpgkey=https://repo.openeuler.org/openEuler-24.03-LTS-SP1/everything/$basearch/RPM-GPG-KEY-openEuler
   ```

2. 使用 root 权限安装 rubik：

   ```shell
   sudo yum install -y rubik
   ```

> \[!NOTE]说明
>
> rubik 工具相关文件会安装在/var/lib/rubik 目录下。

## 部署 rubik

rubik 以容器形式运行在混合部署场景下的 k8s 集群中，用于对不同优先级业务进行资源隔离和限制，避免离线业务对在线业务产生干扰，在提高资源总体利用率的同时保障在线业务的服务质量。当前 rubik 支持对 CPU、内存资源进行隔离和限制等特性，需配合 openEuler 24.03-LTS-SP1 版本的内核使用。若用户想要开启内存优先级特性（即针对不同优先级业务实现内存资源的分级），需要通过设置/proc/sys/vm/memcg\_qos\_enable 开关，有效值为 0 和 1，其中 0 为默认值表示关闭特性，1 表示开启特性。

```bash
sudo echo 1 > /proc/sys/vm/memcg_qos_enable
```

### 部署 rubik daemonset

1. 构建rubik镜像：使用`/var/lib/rubik/build_rubik_image.sh`脚本自动构建或者直接使用 docker容器引擎构建 rubik 镜像。由于 rubik 以 daemonSet 形式部署，故每一个节点都需要 rubik 镜像。用户可以在一个节点构建镜像后使用 docker save/load 功能将 rubik 镜像 load 到 k8s 的每一个节点，也可以在各节点上都构建一遍 rubik 镜像。以 docker 为例，其构建命令如下：

   ```sh
   docker build -f /var/lib/rubik/Dockerfile -t rubik:2.0.1-2 .
   ```

2. 在 k8s master 节点，修改`/var/lib/rubik/rubik-daemonset.yaml`文件中的 rubik 镜像名，与上一步构建出来的镜像名保持一致。

   ```yaml
   ...
   containers:
   - name: rubik-agent
     image: rubik_image_name_and_tag  # 此处镜像名需与上一步构建的 rubik 镜像名一致
     imagePullPolicy: IfNotPresent
   ...
   ```

3. 在 k8s master 节点，使用 kubectl 命令部署 rubik daemonset，rubik 会自动被部署在 k8s 的所有节点：

   ```sh
   kubectl apply -f /var/lib/rubik/rubik-daemonset.yaml
   ```

4. 使用`kubectl get pods -A`命令查看 rubik 是否已部署到集群每一个节点上（rubik-agent 数量与节点数量相同且均为 Running 状态）：

   ```sh
   [root@localhost rubik]# kubectl get pods -A | grep rubik
   NAMESPACE     NAME                                            READY   STATUS    RESTARTS   AGE
   ...
   kube-system   rubik-agent-76ft6                               1/1     Running   0          4s
   ...
   ```

## 常用配置说明

通过以上方式部署的 rubik 将以默认配置启动，用户可以根据实际需要修改 rubik 配置，可通过修改 rubik-daemonset.yaml 文件中的 config.json 段落内容后重新部署 rubik daemonset 实现。以下介绍几个常见配置，其他配置详见 [配置文档](./configuration.md)。

### Pod 绝对抢占特性

用户在开启了 rubik 绝对抢占特性后，仅需在部署业务 Pod 时在 yaml 中通过 annotation 指定其优先级。部署后 rubik 会自动感知当前节点 Pod 的创建与更新，并根据用户配置的优先级设置 Pod 优先级。对于已经启动的或者更改注解的Pod， rubik 会自动更正Pod的优先级配置。

```yaml
...
  "agent": {
    "enabledFeatures": [
      "preemption"
    ]
  },
  "preemption": {
    "resource": [
      "cpu",
      "memory"
    ]
  }
...
```

> \[!NOTE]说明
>
> 优先级配置仅支持Pod由在线切换为离线，不允许由离线切换为在线。

## 在/离线业务配置示例

rubik 部署成功后，用户在部署实际业务时，可以根据以下配置示例对业务 yaml 文件进行修改，指定业务的在离线类型，rubik 即可在业务部署后对其优先级进行配置，从而达到资源隔离的目的。

以下为部署一个 nginx 在线业务的示例：

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  namespace: qosexample
  annotations:
    volcano.sh/preemptable: "false"   # volcano.sh/preemptable 为 true 代表业务为离线业务，false 代表业务为在线业务，默认为 false
spec:
  containers:
  - name: nginx
    image: nginx
    resources:
      limits:
        memory: "200Mi"
        cpu: "1"
      requests:
        memory: "200Mi"
        cpu: "1"
```

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/kubeos/kubeos/installation_and_deployment.md
---
# 安装与部署

本章介绍如何安装和部署容器 OS 升级工具。

## 软硬件要求

### 硬件要求

* 当前仅支持 x86和 AArch64 架构

### 软件要求

* 操作系统：openEuler 24.03-LTS-SP1

### 环境准备

* 安装 openEuler 系统，安装方法参考《[安装指南](../../../server/installation_upgrade/installation/installation_preparations.md)》
* 安装 qemu-img，bc，parted，tar，yum，docker，dosfstools

## 安装容器OS升级工具

安装容器 OS 升级工具的操作步骤如下：

1. 配置 openEuler 24.03-LTS-SP1 yum 源：

   ```sh
   [openEuler24.03-LTS-SP1] # openEuler 24.03-LTS-SP1 官方发布源
   name=openEuler24.03-LTS-SP1
   baseurl=http://repo.openeuler.org/openEuler-24.03-LTS-SP1/everything/$basearch/ 
   enabled=1
   gpgcheck=1
   gpgkey=http://repo.openeuler.org/openEuler-24.03-LTS-SP1/everything/$basearch/RPM-GPG-KEY-openEuler
   ```

2. 使用 root 帐户安装容器 OS 升级工具：

   ```shell
   yum install KubeOS KubeOS-scripts -y
   ```

> \[!NOTE]说明
>
> 容器 OS 升级工具会安装在 /opt/kubeOS 目录下，包括os-operator，os-proxy，os-agent二进制，制作容器 OS 工具及相应配置文件 。

## 部署容器OS升级工具

容器OS升级工具安装完成后，需要对此进行配置部署，本章介绍如何配置和部署容器OS升级工具。

### 制作os-operator和os-proxy镜像

#### 环境准备

使用 Docker 制作容器镜像，请先确保 Docker 已经安装和配置完成。

#### 操作步骤

1. 进入工作目录。

   ```shell
   cd /opt/kubeOS
   ```

2. 指定 proxy 的镜像仓库、镜像名及版本。

   ```shell
   export IMG_PROXY=your_imageRepository/os-proxy_imageName:version
   ```

3. 指定 operator 的镜像仓库、镜像名及版本。

   ```shell
   export IMG_OPERATOR=your_imageRepository/os-operator_imageName:version
   ```

4. 请用户自行编写Dockerfile来构建镜像 ，Dockfile编写请注意以下几项：

   * os-operator和os-proxy镜像需要基于baseimage进行构建，请用户保证baseimage的安全性。
   * 需将os-operator和os-proxy二进制文件分别拷贝到对应的镜像中。
   * 请确保os-proxy镜像中os-proxy二进制文件件属主和属组为root，文件权限为500。
   * 请确保os-operator镜像中os-operator二进制文件属主和属组为容器内运行os-operator进程的用户，文件权限为500。
   * os-operator和os-proxy的二进制文件在镜像内的位置和容器启动时运行的命令需与部署的yaml中指定的字段相对应。

   Dockerfile示例如下

   ```dockerfile
   FROM your_baseimage
   COPY ./bin/proxy /proxy
   ENTRYPOINT ["/proxy"]
   ```

   ```dockerfile
   FROM your_baseimage
   COPY --chown=6552:6552 ./bin/operator /operator
   ENTRYPOINT ["/operator"]
   ```

   Dockerfile也可以使用多阶段构建。

5. 构建容器镜像（os-operator 和 os-proxy 镜像）。

   ```shell
   # 指定proxy的Dockerfile地址
   export DOCKERFILE_PROXY=your_dockerfile_proxy
   # 指定operator的Dockerfile路径
   export DOCKERFILE_OPERATOR=your_dockerfile_operator
   # 镜像构建
   docker build -t ${IMG_OPERATOR} -f ${DOCKERFILE_OPERATOR} .
   docker build -t ${IMG_PROXY} -f ${DOCKERFILE_PROXY} .
   ```

6. 将容器镜像 push 到镜像仓库。

   ```shell
   docker push ${IMG_OPERATOR}
   docker push ${IMG_PROXY}
   ```

### 制作容器OS虚拟机镜像

#### 注意事项

* 以虚拟机镜像为例，如需进行物理机的镜像制作请见《[容器OS镜像制作指导](./kubeos_image_creation.md)》。
* 制作容器OS 镜像需要使用 root 权限。
* 容器OS 镜像制作工具的 rpm 包源为 openEuler 具体版本的 everything 仓库和 EPOL 仓库。制作镜像时提供的 repo 文件中，yum 源建议同时配置 openEuler 具体版本的 everything 仓库和 EPOL 仓库。
* 使用默认 rpmlist 制作的容器OS虚拟机镜像，默认保存在调用`kbimg`路径下的`scripts-auto`文件夹内，该分区至少有 25GiB 的剩余磁盘空间。
* 制作容器 OS 镜像时，不支持用户自定义配置挂载文件。

#### 操作步骤

制作容器OS 虚拟机镜像使用 kbimg，命令详情请见《[容器OS镜像制作指导](./kubeos_image_creation.md)》。

制作容器OS 虚拟机镜像的步骤如下：

1. 进入执行目录：

   ```shell
   cd /opt/kubeOS/scripts
   ```

2. 执行 kbming 制作容器OS，参考命令如下：

   ```shell
   ./kbimg create -f ./kbimg.toml vm-img
   ```

   容器 OS 镜像制作完成后，会在 /opt/kubeOS/scripts/scripts-auto 目录下生成：

   * raw格式的系统镜像system.img，system.img大小默认为20G，支持的根文件系统分区大小<2560MiB，持久化分区<15GB。
   * qcow2 格式的系统镜像 system.qcow2。
   * 可用于升级的根文件系统 kubeos.tar。

   制作出来的容器 OS 虚拟机镜像目前只能用于 CPU 架构为 x86 和 AArch64 的虚拟机场景，不支持 x86 架构的虚拟机使用 legacy 启动模式启动。

### 部署CRD,operator和proxy

#### 注意事项

* 请先部署 Kubernetes 集群，部署方法参考[《openEuler 24.03-LTS-SP1 Kubernetes 集群部署指南》](../../cluster_deployment/kubernetes/overview.md)。

* 集群中准备进行升级的 Worker 节点的 OS 需要为使用上一节方式制作出来的容器 OS，如不是，请用 system.qcow2重新部署虚拟机，虚拟机部署请见[《openEuler 24.03-LTS-SP1 虚拟化用户指南》](../../../virtualization/virtualization_platform/virtualization/introduction_to_virtualization.md)，Master节点目前不支持容器 OS 升级，请用openEuler 24.03-LTS-SP1部署Master节点。

* 部署 OS 的 CRD（CustomResourceDefinition），os-operator，os-proxy 以及 RBAC (Role-based access control) 机制的 YAML 需要用户自行编写。

* operator 和 proxy 部署在 kubernetes 集群中，operator 应部署为 deployment，proxy 应部署为daemonset。

* 尽量部署好 kubernetes 的安全措施，如 rbac 机制，pod 的 service account 和 security policy 配置等。

#### 操作步骤

1. 准备 YAML 文件，包括用于部署 OS 的CRD、RBAC 机制、os- operator 和os- proxy 的 YAML 文件，可参考[yaml-example](https://atomgit.com/openeuler/KubeOS/tree/master/docs/example/config)。假设分别为 crd.yaml、rbac.yaml、manager.yaml 。

2. 部署 CRD、RBAC、os-operator 和 os-proxy。假设 crd.yaml、rbac.yaml、manager.yaml 文件分别存放在当前目录的 config/crd、config/rbac、config/manager 目录下 ，参考命令如下：

   ```shell
   kubectl apply -f config/crd
   kubectl apply -f config/rbac 
   kubectl apply -f config/manager
   ```

3. 部署完成后，执行以下命令，确认各个组件是否正常启动。如果所有组件的 STATUS 为 Running，说明组件已经正常启动。

   ```shell
   kubectl get pods -A
   ```

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/nestos/nestos/installation_and_deployment.md
---
# 安装与部署

## 在 VMware 上部署 NestOS

本指南展示了如何在VMware虚拟机管理程序上配置最新的 NestOS。

目前NestOS仅支持x86\_64架构。

### 开始之前

​在开始部署 NestOS 之前,需要做如下准备工作：

* 下载 NestOS ISO
* 准备 config.bu 文件
* 配置 butane 工具(Linux环境/win10环境)
* 安装有VMware的宿主机

### 初步安装与启动

#### 启动 NestOS

初次启动 NestOS ，ignition 尚未安装，可根据系统提示使用 nestos-installer 组件进行ignition的安装。

### 配置 ignition 文件

#### 获取 Butane

可以通过 Butane 将 bu 文件转化为 ignition 文件。ignition 配置文件被设计为可读但不可写，是为了阻止用户尝试手动编写配置。Butane 提供了多种环境的支持，可以在 linux/windows 宿主机中或容器环境中进行配置。

```shell
docker pull quay.io/coreos/butane:release
```

#### 生成登录密码

在宿主机执行如下命令，并输入你的密码。

```shell
# openssl passwd -1 -salt yoursalt
Password:
$1$yoursalt$1QskegeyhtMG2tdh0ldQN0
```

#### 生成ssh-key

在宿主机执行如下命令，获取公钥和私钥以供后续 ssh 登录。

```shell
# ssh-keygen -N '' -f ./id_rsa
Generating public/private rsa key pair.
Your identification has been saved in ./id_rsa
Your public key has been saved in ./id_rsa.pub
The key fingerprint is:
SHA256:4fFpDDyGHOYEd2fPaprKvvqst3T1xBQuk3mbdon+0Xs root@host-12-0-0-141
```

```shell
The key's randomart image is:
+---[RSA 3072]----+
|   ..= . o .   |
|    * = o * .  |
|     + B = *   |
|     o B O + . |
|      S O B o  |
|       * = . . |
|      . +o . . |
|     +.o . .E  |
|    o*Oo   ... |
+----[SHA256]-----+
```

可以在当前目录查看id\_rsa.pub公钥：

```shell
# cat id_rsa.pub
ssh-rsa
AAAAB3NzaC1yc2...
```

#### 编写bu文件

进行最简单的初始配置，如需更多详细的配置，参考后面的 ignition 详解。
如下为最简单的 config.bu 文件：

```shell
variant: fcos
version: 1.1.0
passwd:
  users:
    - name: nest
      password_hash: "$1$yoursalt$1QskegeyhtMG2tdh0ldQN0"
      ssh_authorized_keys:
        - "ssh-rsa
  AAAAB3NzaC1yc2EAAA..."
```

#### 生成ignition文件

将 config.bu 通过 Butane 工具转换为 config.ign 文件，如下为在容器环境下进行转换。

```shell
# docker run --interactive --rm quay.io/coreos/butane:release \
--pretty --strict < your_config.bu > transpiled_config.ign
```

### 安装 NestOS

将宿主机生成的config.ign文件通过scp拷贝到前面初步启动的 NestOS 中，该OS目前运行在内存中,
并没有安装到硬盘。

```shell
sudo -i
scp root@your_ipAddress:/root/config.ign /root
```

根据系统所给提示，执行如下指令完成安装。

```shell
nestos-installer install /dev/sda --ignition-file config.ign
```

安装完成后重启 NestOS 。

```shell
systemctl reboot
```

---

---
url: /zh/docs/22.03_LTS_SP4/edge_computing/ros/installation_and_deployment.md
---
# 安装与部署

## 软件要求

* 操作系统：openEuler 22.03 LTS SP4

## 硬件要求

* x86\_64架构、AArch64架构

## 环境准备

* 安装openEuler 22.03 LTS SP4系统，安装方法参考 [《openEuler 22.03 LTS SP4 安装指南》](./../../server/installation_upgrade/installation/installation_guide.md)。

## 1. ROS2

### 1. ros-humble

#### 1. 安装ros-humble

1. 安装ros-humble 软件包

   ```shell
   [root@openEuler ~]# yum install openeuler-ros
   [root@openEuler ~]# yum install ros-humble-ros-base ros-humble-xxx 例如安装小乌龟ros-humble-turtlesim
   ```

2. 执行如下命令，查看安装是否成功。如果回显有对应软件包，表示安装成功。

   ```shell
   [root@openEuler ~]# rpm -q ros-humble
   ```

#### 2.  测试 ros-humble

##### 运行小乌龟

1. 启动小乌龟

   ```shell
   [root@openEuler ~]# source /opt/ros/humble/setup.bash
   [root@openEuler ~]# ros2 run turtlesim turtlesim_node
   ```

2. 打开小乌龟控制终端

   ```shell
   [root@openEuler ~]# source /opt/ros/humble/setup.bash
   [root@openEuler ~]# ros2 run turtlesim turtle_teleop_key
   ```

3. 在乌龟控制终端中使用方向键控制乌龟运动

![ros-humble](./figures/ros-humble.png)

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/administration/sysmaster/devmaster_install_deploy.md
---
# 安装与部署

`devmaster`目前可应用于以 `sysmaster`为1号进程的虚拟机环境，本章节介绍安装部署的规格约束以及操作流程。

## 软件要求

* 操作系统：`openEuler 22.03 LTS SP4`

## 硬件要求

* `x86_64`架构、`aarch64`架构

## 安装部署流程

1. 执行如下命令，使用`yum`工具安装`devmaster`包：

   ```shell
   # yum install devmaster
   ```

2. `devmaster`包自带`sysmaster`服务配置文件，安装后会覆盖`udev`的相关服务，并在启动阶段自动被`sysmaster`拉起。卸载`devmaster`后，会恢复`udev`的相关服务。

3. 安装`devmaster`后，重启系统生效。

4. 查看 `/run/devmaster/data/`目录，如果生成设备数据库文件，则表示部署成功：

   ```shell
   # ll /run/devmaster/data/
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/administration/sysmaster/sysmaster_install_deploy.md
---
# 安装与部署

`sysmaster`可应用于容器和虚拟机，本文档将以 `aarch64`系统为例说明如何在各场景下进行安装与部署。

## 软件要求

* 操作系统：`openEuler 22.03 LTS SP4`

## 硬件要求

* `x86_64`架构、`aarch64`架构

## 容器场景安装与部署

1. 安装 docker。

   ```bash
   yum install -y docker
   systemctl restart docker
   ```

2. 加载基础容器镜像。

   下载容器镜像：

   ```bash
   wget https://repo.openeuler.org/openEuler-22.03-LTS-SP4/docker_img/aarch64/openEuler-docker.aarch64.tar.xz
   xz -d openEuler-docker.aarch64.tar.xz
   ```

   加载容器镜像：

   ```bash
   docker load --input openEuler-docker.aarch64.tar
   ```

3. 构建容器。

   基于 `docker images`命令查询得到的镜像名创建 `Dockerfile`，以 `openEuler-22.03-LTS-SP4`为例。

   ```bash
   cat << EOF > Dockerfile
   FROM openEuler-22.03-LTS-SP4
   RUN yum install -y sysmaster
   CMD ["/usr/lib/sysmaster/init"]
   EOF
   ```

   构建容器：

   ```bash
   docker build -t openEuler-22.03-LTS-SP4:latest .
   ```

4. 启动并进入容器。

   启动容器：

   ```bash
   docker run -itd --privileged openEuler-22.03-LTS-SP4:latest
   ```

   获取`CONTAINERID`：

   ```bash
   docker ps
   ```

   使用上一步获取到 `CONTAINERID`进入容器：

   ```bash
   docker exec -it CONTAINERID /bin/bash
   ```

## 虚拟机场景安装与部署

1. `initramfs`镜像制作。

   为了避免 `initrd`阶段 `systemd`的影响，需要制作一个剔除 `systemd`的 `initramfs`镜像，并以该镜像进入 `initrd`流程。使用如下命令：

   ```bash
   dracut -f --omit "systemd systemd-initrd systemd-networkd dracut-systemd" /boot/initrd_withoutsd.img
   ```

2. 新增启动项。

   在 `grub.cfg`中增加新的启动项，`aarch64`下的路径为 `/boot/efi/EFI/openEuler/grub.cfg`，`x86_64`下的路径为 `/boot/grub2/grub.cfg`，拷贝一份原有启动项，并做以下几处修改：

   * `menuentry` 设置启动项名称为 `openEuler sysmaster`
   * `linux` 项内核启动参数修改 `root=/dev/mapper/openeuler-root ro` 为 `root=/dev/mapper/openeuler-root rw`
   * `linux` 项内核启动参数修改 `plymouth`，如果环境上安装了 `plymouth`, 需要添加 `plymouth.enable=0` 禁用 `plymouth`
   * `linux` 项内核启动参数增加 `init=/usr/lib/sysmaster/init`
   * `initrd` 项修改为 `/initrd_withoutsd.img`

3. 安装 sysmaster。

   ```bash
   yum install sysmaster
   ```

4. 重启后出现 `openEuler sysmaster`启动项表示已成功配置，选择此启动项进入虚拟机。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/development/fangtian/fangtian_environment_configuration.md
---
# 安装与部署

本章介绍在 openEuler 中安装 FangTian 的方法。

## 软硬件要求

### 硬件要求

当前仅支持 x86和 AArch64 架构。

### 软件要求

操作系统：openEuler 22.03 LTS SP4

### 环境准备

安装 openEuler 系统，安装方法参考《[openEuler 安装指南](../../installation_upgrade/installation/installation_guide.md)》。

### FangTian 软件包安装

````
x86架构下：

```shell
sudo dnf install ft_multimedia ft_mmi ft_flutter ft_engine arkui-linux ft_utils
sudo dnf install ft_multimedia-devel ft_mmi-devel ft_flutter-devel ft_engine-devel
```

AArch64架构下：

```shell
sudo dnf install ft_multimedia ft_mmi ft_flutter ft_engine ft_utils
sudo dnf install ft_multimedia-devel ft_mmi-devel ft_flutter-devel ft_engine-devel
```
````

## 启动引擎

* 系统服务samgr的启动

  预设置：安装 binder，ashmem 等。

  ```shell
  sudo /usr/share/sa/pre_oneshot_samgr 
  ```

  可以直接拉起samgr。

  ```shell
  mkdir -p ~/tmp
  sudo samgr > ~/tmp/samgr.log 2>&1 &
  ```

  或者，自行配置为 service 再启动服务。

  ```shell
  sudo systemctl restart samgr
  ```

* 引擎sa的启动

  ```shell
  sa_main /system/profile/ft/ft.xml > ~/tmp/ftsa.log 2>&1 &
  ```

  > 说明
  >
  > * sa 代表一种系统能力，一个进程可包含多个 sa。ft.xml 配置了多个 sa，共同组成进程 ft。关于 samgr 及 sa 概念可以参考 OpenHarmony。
  > * 关于 sa 的配置 xml 及 sa\_main samgr 均已在软件包安装时自动部署。

## 基于 FangTian 做简单 GUI 应用开发运行

C++ GUI 简单应用《[示例参考](https://atomgit.com/openeuler/ft_engine/blob/master/samples/)》。

运行：

```shell
desktop &
```

结果如下：

![](./figures/desktop_simple_apps.png)

```
> 说明
>
> 开发者可以查看[FT接口](https://atomgit.com/openeuler/ft_engine/wikis/1.0-alpha%E6%8E%A5%E5%8F%A3/1.0-alpha%20Interface%20Overview)进行应用的开发。
```

---

---
url: /zh/docs/22.03_LTS_SP4/server/development/gcc/pin_user_guide.md
---
# 安装与部署

## 软件要求

* 操作系统：openEuler-22.03-LTS-SP4

## 硬件要求

* x86\_64架构
* AArch64架构

## 环境准备

* 安装openEuler系统，安装方法参考[《openEuler 安装指南》](./../../../server/installation_upgrade/installation/installation_guide.md)。

### 安装依赖软件

#### 安装插件框架GCC客户端依赖软件

```shell
yum install -y git
yum install -y make
yum install -y cmake
yum install -y grpc
yum install -y grpc-devel
yum install -y grpc-plugins
yum install -y protobuf-devel
yum install -y jsoncpp
yum install -y jsoncpp-devel
yum install -y gcc-plugin-devel
yum install -y llvm-mlir
yum install -y llvm-mlir-devel
yum install -y llvm-devel
```

#### 安装插件框架服务端依赖软件

```shell
yum install -y git
yum install -y make
yum install -y cmake
yum install -y grpc
yum install -y grpc-devel
yum install -y grpc-plugins
yum install -y protobuf-devel
yum install -y jsoncpp
yum install -y jsoncpp-devel
yum install -y llvm-mlir
yum install -y llvm-mlir-devel
yum install -y llvm-devel
```

## 安装Pin

### 安装

#### yum安装插件框架GCC客户端

```shell
yum install -y pin-gcc-client
```

#### yum安装插件框架服务端

```shell
yum install -y pin-server
```

### 编译构建

#### 构建插件框架GCC客户端

```shell
git clone https://atomgit.com/openeuler/pin-gcc-client.git
cd pin-gcc-client
mkdir build
cd build
cmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH}
make
```

#### 构建插件框架服务端

```shell
git clone https://atomgit.com/openeuler/pin-server.git
cd pin-server
mkdir build
cd build
cmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH}
make
```

# 使用方法

用户可以通过`-fplugin`和`-fplugin-arg-libpin_xxx`使能插件工具。
命令如下：

```shell
$(TARGET): $(OBJS)
    $(CXX) -fplugin=${CLIENT_PATH}/libpin_gcc_client.so \
    -fplugin-arg-libpin_gcc_client-server_path=${SERVER_PATH}/pin_server \
    -fplugin-arg-libpin_gcc_client-log_level="1" \
    -fplugin-arg-libpin_gcc_client-arg1="xxx"
```

为了方便用户使用，可以通过`${INSTALL_PATH}/bin/pin-gcc-client.json`文件，进行插件配置。配置选项如下：

`path`：配置插件框架服务端可执行文件路径。

`sha256file`：配置插件工具的校验文件`xxx.sha256`路径。

`timeout`：配置跨进程通信超时时间，单位`ms`。

编译选项：

`-fplugin`：指定插件客户端.so所在路径。

`-fplugin-arg-libpin_gcc_client-server_path`：指定插件服务端可执行程序所在路径。

`-fplugin-arg-libpin_gcc_client-log_level`：指定日志系统默认记录等级，取值`0~3`。默认为`1`。

`-fplugin-arg-libpin_gcc_client-argN`：用户可以根据插件工具要求，指定其他参数。argN代指插件工具要求的参数字段。

# 兼容性说明

此节主要列出当前一些特殊场景下的兼容性问题。本项目持续迭代中，会尽快进行修复，也欢迎广大开发者加入。

* 插件框架在`-flto`阶段使能时，不支持使用`make -j`多进程编译。建议改用`make -j1`进行编译。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/maintenance/kernel_live_upgrade/installation_and_deployment.md
---
# 安装与部署

本章介绍如何安装和部署内核热升级工具。

## 软硬件要求

### 硬件要求

* 当前仅支持arm64架构

### 软件要求

* 操作系统：openEuler 22.03-LTS-SP4

## 环境准备

* 安装openEuler系统，安装方法参考 《[openEuler 22.03-LTS-SP4 安装指南](./../../installation_upgrade/installation/installation_guide.md)》。

* 安装内核热升级工具需要使用root权限。

## 安装内核热升级工具

本章介绍内核热升级工具的安装方法。

安装内核热升级工具的操作步骤如下：

1. 挂载openEuler的iso文件。

   ```shell
   # mount openEuler-22.03-LTS-SP4-aarch64-dvd.iso /mnt
   ```

2. 配置本地yum源。

   ```shell
   # vi /etc/yum.repos.d/local.repo
   ```

   配置内容如下所示：

   ```shell
   [local]
   name=local
   baseurl=file:///mnt
   gpgcheck=1
   enabled=1
   ```

3. 将RPM数字签名的GPG公钥导入系统。

   ```shell
   # rpm --import /mnt/RPM-GPG-KEY-openEuler
   ```

4. 安装内核热升级工具。

   ```shell
   # yum install nvwa -y
   ```

5. 验证是否安装成功，命令和回显如下，则表示安装成功。

   ```shell
   # rpm -qa | grep nvwa
   nvwa-xxx
   ```

## 部署内核热升级工具

本章介绍内核热升级工具的配置部署：

### 配置介绍

内核热升级工具的配置文件位于/etc/nvwa，配置文件包括：

* nvwa-restore.yaml

  该配置文件用于指导内核热升级工具在内核热升级过程中如何保存和恢复现场，具体配置如下：

  * pids

    pids用于指明nvwa热升级过程中需要保留和恢复的进程，此处的进程通过进程号(pid)进行标识，需要注意的是，nvwa管理的进程在nvwa服务启动后，会被自动恢复。

  * services

    services用于指明nvwa热升级过程中需要保留和恢复的服务。与pids的区别在于，内核热升级工具可以直接保存和恢复进程的状态，对于服务，内核热升级工具则需要依赖systemd进行相关操作。此处的服务名称，应该使用systemd中使用的服务名称。需要注意的是，对于nvwa管理的服务，是否要在nvwa启动时自动恢复，取决于systemd中有没有使能该服务，且当前支持的服务类型只有notify和oneshot。

  * restore\_net

    restore\_net用于指明是否需要内核热升级工具保存和恢复网络配置，如果网络配置有误，有可能导致恢复后网络不可用，默认关闭。

  * enable\_quick\_kexec

    enable\_quick\_kexec用于指明是否需要使能quick kexec特性，quick kexec是nvwa社区推出的，加速内核重启过程的一个特性。使用该特性，需要在cmdline中，加入"quickkexec=128M"。128指分配给quick kexec特性的内存大小，该内存将用于在升级过程中加载kernel和initramfs，因此大小需要大于升级过程中涉及到的kernel，initramfs大小之和。该特性默认关闭。

  * enable\_pin\_memory

    enable\_pin\_memory用于指明是否需要使能pin memory特性，pin memory是nvwa社区推出的，加速进程保存恢复过程的一个特性。pin\_memory特性暂不支持多进程应用备份恢复，使用该特性，需要在cmdline中，加入"max\_pin\_pid\_num=10  redirect\_space\_size=2M pinmemory=200M@0x640000000"。

    其中，max\_pin\_pid\_num代表支持pin memory恢复的最大进程数目，redirect\_space\_size代表pin memory过程中重定向物理页所需要的预留内存空间，建议配置为pin memory总预留内存的1/100，pinmemory指明这段内存的起点和大小。从0x640000000开始的200M空间，是pin memory使用的全部内存空间，这段空间不应该被其他程序使用。

* nvwa-restore.yaml的配置示例

  ```yaml
  pids:
    - 14109
  services:
    - redis
  restore_net: false
  enable_quick_kexec: true
  enable_pin_memory: true
  ```

* nvwa-server.yaml

  该文件包含了内核热升级工具运行过程中，需要使用到的配置信息，具体如下：

  * criu\_dir

    用于指明内核热升级工具在保存现场过程中，存储产生的信息文件夹路径。需要注意的是，这些信息可能会占用较大的磁盘空间。

  * criu\_exe

    用于指明内核热升级工具使用的criu可执行文件路径，除非是对criu进行调测，一般不建议修改。

  * kexec\_exe

    用于指明内核热升级工具使用的kexec可执行文件路径，除非是对kexec进行调测，一般不建议修改。

  * systemd\_etc

    用于指明覆盖systemd配置过程中，使用到的文件夹路径。该路径由systemd决定，一般不需要修改。

  * log\_dir

    存放内核热升级工具产生的log信息，log模块当前未启用。内核热升级工具日志信息的查看，参考[使用方法-产生的日志信息](./usage_guide.md#产生的日志信息)。

* nvwa-server.yaml的配置示例

  ```yaml
  criu_dir: /var/nvwa/running/
  criu_exe: /usr/sbin/criu
  kexec_exe: /usr/sbin/kexec
  systemd_etc: /etc/systemd/system/
  log_dir: /etc/nvwa/log/
  ```

## 使能内核热升级工具

内核热升级工具的运行依赖配置文件，配置文件修改后应该重新运行内核热升级工具程序。

安装成功后，可以通过systemd的相关命令来操作内核热升级工具。

* 使能nvwa。

  ```shell
    systemctl enable nvwa
  ```

* 启动nvwa。

  ```shell
    systemctl start nvwa
  ```

* 查看nvwa服务状态及相关信息。

  ```shell
    service nvwa status
  ```

* 更多用法参考systemd用法。

---

---
url: /zh/docs/22.03_LTS_SP4/server/performance/atune/installation_and_deployment.md
---
# 安装与部署

本章介绍如何安装和部署A-Tune。

## 软硬件要求

### 硬件要求

* 鲲鹏920处理器

### 软件要求

* 操作系统：openEuler 22.03 LTS SP4

## 环境准备

* 安装openEuler系统，安装方法参考 [《openEuler 22.03 LTS SP4安装指南》](./../../installation_upgrade/installation/installation_guide.md)。

* 安装A-Tune需要使用root权限。

## 安装A-Tune

本节介绍A-Tune的安装模式和安装方法。

### 安装模式介绍

A-Tune支持单机模式和分布式模式安装：

* 单机模式

  client和server安装到同一台机器上。

* 分布式模式

  client和server分别安装在不同的机器上。

两种安装模式的简单图示如下：

![](./figures/zh-cn_image_0231122163.png)

### 安装操作

安装A-Tune的操作步骤如下：

1. 挂载openEuler的iso文件。

   ```shell
   # mount openEuler-22.03-LTS-SP4-aarch64-dvd.iso /mnt
   ```

2. 配置本地yum源。

   ```shell
   # vim /etc/yum.repos.d/local.repo
   ```

   配置内容如下所示：

   ```shell
   [local]
   name=local
   baseurl=file:///mnt
   gpgcheck=1
   enabled=1
   ```

3. 将RPM数字签名的GPG公钥导入系统。

   ```shell
   # rpm --import /mnt/RPM-GPG-KEY-openEuler
   ```

4. 安装A-Tune服务端。

   > \[!NOTE]说明\
   > 本步骤会同时安装服务端和客户端软件包，对于单机部署模式，请跳过**步骤5**。

   ```shell
   # yum install atune -y
   # yum install atune-engine -y
   ```

5. 若为分布式部署，请安装A-Tune客户端。

   ```shell
   # yum install atune-client -y
   ```

6. 验证是否安装成功。命令和回显如下表示安装成功。

   ```shell
   # rpm -qa | grep atune
   atune-client-xxx
   atune-db-xxx
   atune-xxx
   atune-engine-xxx
   ```

## 部署A-Tune

本节介绍A-Tune的配置部署。

### 配置介绍

A-Tune配置文件/etc/atuned/atuned.cnf的配置项说明如下：

* A-Tune服务启动配置

  可根据需要进行修改。

  * protocol：系统gRPC服务使用的协议，unix或tcp，unix为本地socket通信方式，tcp为socket监听端口方式。默认为unix。
  * address：系统gRPC服务的侦听地址，默认为unix socket，若为分布式部署，需修改为侦听的ip地址。
  * port：系统gRPC服务的侦听端口，范围为0~65535未使用的端口。如果protocol配置是unix，则不需要配置。
  * connect：若为集群部署时，A-Tune所在节点的ip列表，ip地址以逗号分隔。
  * rest\_host：系统rest service的侦听地址，默认为localhost。
  * rest\_port：系统rest service的侦听端口，范围为0~65535未使用的端口，默认为8383。
  * engine\_host：与系统atune engine service链接的地址。
  * engine\_port：与系统atune engine service链接的端口。
  * sample\_num：系统执行analysis流程时采集样本的数量，默认为20。
  * interval：系统执行analysis流程时采集样本的间隔时间，默认为5s。
  * grpc\_tls：系统gRPC的SSL/TLS证书校验开关，默认不开启。开启grpc\_tls后，atune-adm命令在使用前需要设置以下环境变量方可与服务端进行通讯：
    * export ATUNE\_TLS=yes
    * export ATUNED\_CACERT=<客户端CA证书路径>
    * export ATUNED\_CLIENTCERT=<客户端证书路径>
    * export ATUNED\_CLIENTKEY=<客户端密钥路径>
    * export ATUNED\_SERVERCN=server
  * tlsservercafile：gRPC服务端CA证书路径。
  * tlsservercertfile：gRPC服务端证书路径。
  * tlsserverkeyfile：gRPC服务端密钥路径。
  * rest\_tls：系统rest service的SSL/TLS证书校验开关，默认开启。
  * tlsrestcacertfile：系统rest service的服务端CA证书路径。
  * tlsrestservercertfile：系统rest service的服务端证书路径
  * tlsrestserverkeyfile：系统rest service的服务端密钥路径。
  * engine\_tls：系统atune engine service的SSL/TLS证书校验开关，默认开启。
  * tlsenginecacertfile：系统atune engine service的客户端CA证书路径。
  * tlsengineclientcertfile：系统atune engine service的客户端证书路径
  * tlsengineclientkeyfile：系统atune engine service的客户端密钥路径。

* system信息

  system为系统执行相关的优化需要用到的参数信息，必须根据系统实际情况进行修改。

  * disk：执行analysis流程时需要采集的对应磁盘的信息或执行磁盘相关优化时需要指定的磁盘。
  * network：执行analysis时需要采集的对应的网卡的信息或执行网卡相关优化时需要指定的网卡。
  * user：执行ulimit相关优化时用到的用户名。目前只支持root用户。

* 日志信息

  根据情况修改日志的级别，默认为info级别，日志信息打印在/var/log/messages中。

* monitor信息

  为系统启动时默认采集的系统硬件信息。

* tuning信息

  tuning为系统进行离线调优时需要用到的参数信息。

  * noise：高斯噪声的评估值。
  * sel\_feature：控制离线调优参数重要性排名输出的开关，默认关闭。

### 配置示例

```shell
#################################### server ############################### 
 # atuned config 
 [server] 
 # the protocol grpc server running on 
 # ranges: unix or tcp 
 protocol = unix 

 # the address that the grpc server to bind to
 # default is unix socket /var/run/atuned/atuned.sock
 # ranges: /var/run/atuned/atuned.sock or ip address
 address = /var/run/atuned/atuned.sock 

 # the atune nodes in cluster mode, separated by commas
 # it is valid when protocol is tcp
 # connect = ip01,ip02,ip03

 # the atuned grpc listening port
 # the port can be set between 0 to 65535 which not be used
 # port = 60001

 # the rest service listening port, default is 8383
 # the port can be set between 0 to 65535 which not be used
 rest_host = localhost
 rest_port = 8383

 # the tuning optimizer host and port, start by engine.service
 # if engine_host is same as rest_host, two ports cannot be same
 # the port can be set between 0 to 65535 which not be used
 engine_host = localhost
 engine_port = 3838

 # when run analysis command, the numbers of collected data.
 # default is 20
 sample_num = 20

 # interval for collecting data, default is 5s
 interval = 5

 # enable gRPC authentication SSL/TLS
 # default is false
 # grpc_tls = false
 # tlsservercafile = /etc/atuned/grpc_certs/ca.crt
 # tlsservercertfile = /etc/atuned/grpc_certs/server.crt
 # tlsserverkeyfile = /etc/atuned/grpc_certs/server.key

 # enable rest server authentication SSL/TLS
 # default is true
 rest_tls = true
 tlsrestcacertfile = /etc/atuned/rest_certs/ca.crt
 tlsrestservercertfile = /etc/atuned/rest_certs/server.crt
 tlsrestserverkeyfile = /etc/atuned/rest_certs/server.key

 # enable engine server authentication SSL/TLS
 # default is true
 engine_tls = true
 tlsenginecacertfile = /etc/atuned/engine_certs/ca.crt
 tlsengineclientcertfile = /etc/atuned/engine_certs/client.crt
 tlsengineclientkeyfile = /etc/atuned/engine_certs/client.key


 #################################### log ############################### 
 [log]
 # either "debug", "info", "warn", "error", "critical", default is "info"
 level = info

 #################################### monitor ############################### 
 [monitor]
 # with the module and format of the MPI, the format is {module}_{purpose}
 # the module is Either "mem", "net", "cpu", "storage"
 # the purpose is "topo"
 module = mem_topo, cpu_topo

 #################################### system ############################### 
 # you can add arbitrary key-value here, just like key = value
 # you can use the key in the profile
 [system]
 # the disk to be analysis
 disk = sda

 # the network to be analysis
 network = enp189s0f0

 user = root
 
 #################################### tuning ###############################
 # tuning configs
 [tuning]
 noise = 0.000000001
 sel_feature = false
```

A-Tune engine配置文件/etc/atuned/engine.cnf的配置项说明如下：

* A-Tune engine服务启动配置

  可根据需要进行修改。

  * engine\_host：系统atune engine service的监听地址，默认为localhost。
  * engine\_port：系统atune engine service的监听端口，范围为0~65535未使用的端口，默认为3838。
  * engine\_tls：系统atune engine service的SSL/TLS证书校验开关，默认开启。
  * tlsenginecacertfile：系统atune engine service的服务端CA证书路径。
  * tlsengineservercertfile：系统atune engine service的服务端证书路径
  * tlsengineserverkeyfile：系统atune engine service的服务端密钥路径。

* 日志信息

  根据情况修改日志的级别，默认为info级别，日志信息打印在/var/log/messages中。

### 配置示例

```shell
 #################################### engine ###############################
 [server]
 # the tuning optimizer host and port, start by engine.service
 # if engine_host is same as rest_host, two ports cannot be same
 # the port can be set between 0 to 65535 which not be used
 engine_host = localhost
 engine_port = 3838

 # enable engine server authentication SSL/TLS
 # default is true
 engine_tls = true
 tlsenginecacertfile = /etc/atuned/engine_certs/ca.crt
 tlsengineservercertfile = /etc/atuned/engine_certs/server.crt
 tlsengineserverkeyfile = /etc/atuned/engine_certs/server.key

 #################################### log ###############################
 [log]
 # either "debug", "info", "warn", "error", "critical", default is "info"
 level = info
```

## 启动A-Tune

A-Tune安装完成后，需要启动A-Tune服务才能使用。

* 启动atuned服务：

  ```shell
  # systemctl start atuned
  ```

* 查询atuned服务状态：

  ```shell
  # systemctl status atuned
  ```

  若回显为如下，则服务启动成功。

  ![](./figures/zh-cn_image_0214540398.png)

## 启动A-Tune engine

若需要使用AI相关的功能，需要启动A-Tune engine服务才能使用。

* 启动atune-engine服务：

  ```shell
  # systemctl start atune-engine
  ```

* 查询atune-engine服务状态：

  ```shell
  # systemctl status atune-engine
  ```

  若回显为如下，则服务启动成功。

  ![](./figures/zh-cn_image_0245342444.png)

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/installation_configuration.md
---
# 安装与配置

本章介绍iSulad的安装、安装后配置，以及升级和卸载的方法。

> \[!NOTE]说明
>
> iSulad的安装、升级、卸载均需要使用root权限。

## 安装方法

iSulad可以通过yum或rpm命令两种方式安装，由于yum会自动安装依赖，而rpm命令需要手动安装所有依赖，所以推荐使用yum安装。

这里给出两种安装方式的操作方法。

* （推荐）使用yum安装iSulad，参考命令如下：

  ```bash
  # sudo yum install -y iSulad
  ```

* 使用rpm安装iSulad，需要下载iSulad及其所有依赖库的RPM包，然后手动安装。安装单个iSulad的RPM包（依赖包安装方式相同），参考命令如下：

  ```bash
  # sudo rpm -ihv iSulad-xx.xx.xx-xx.xxx.aarch64.rpm
  ```

## 配置方法

iSulad 安装完成后，可以根据需要进行相关配置。

### 配置方式

轻量级容器引擎（iSulad）服务端daemon为isulad，isulad可以通过配置文件进行配置，也可以通过命令行的方式进行配置，例如：isulad --xxx，优先级从高到低是：命令行方式>配置文件>代码中默认配置。

> \[!NOTE]说明
>
> 如果采用systemd管理iSulad进程，修改/etc/sysconfig/iSulad文件中的OPTIONS字段，等同于命令行方式进行配置。

* **命令行方式**

  在启动服务的时候，直接通过命令行进行配置。其配置选项可通过以下命令查阅：

  ```bash
  # isulad --help
  isulad

  lightweight container runtime daemon

  Usage:  isulad [global options]

  GLOBAL OPTIONS:

        --authorization-plugin               Use authorization plugin
        --cgroup-parent                      Set parent cgroup for all containers
        --cni-bin-dir                        The full path of the directory in which to search for CNI plugin binaries. Default: /opt/cni/bin
        --cni-conf-dir                       The full path of the directory in which to search for CNI config files. Default: /etc/cni/net.d
        --container-log-driver               Set default container log driver, such as: json-file
        --container-log-opts                 Set default container log driver options, such as: max-file=7 to set max number of container log files
        --default-ulimit                     Default ulimits for containers (default [])
    -e, --engine                             Select backend engine
    -g, --graph                              Root directory of the iSulad runtime
    -G, --group                              Group for the unix socket(default is isulad)
        --help                               Show help
        --hook-spec                          Default hook spec file applied to all containers
    -H, --host                               The socket name used to create gRPC server
        --image-layer-check                  Check layer integrity when needed
        --insecure-registry                  Disable TLS verification for the given registry
        --insecure-skip-verify-enforce       Force to skip the insecure verify(default false)
        --log-driver                         Set daemon log driver, such as: file
    -l, --log-level                          Set log level, the levels can be: FATAL ALERT CRIT ERROR WARN NOTICE INFO DEBUG TRACE
        --log-opt                            Set daemon log driver options, such as: log-path=/tmp/logs/ to set directory where to store daemon logs
        --native.umask                       Default file mode creation mask (umask) for containers
        --network-plugin                     Set network plugin, default is null, support null and cni
    -p, --pidfile                            Save pid into this file
        --pod-sandbox-image                  The image whose network/ipc namespaces containers in each pod will use. (default "pause-${machine}:3.0")
        --registry-mirrors                   Registry to be prepended when pulling unqualified images, can be specified multiple times
        --selinux-enabled                    Enable selinux support
        --start-timeout                      timeout duration for waiting on a container to start before it is killed
    -S, --state                              Root directory for execution state files
        --storage-driver                     Storage driver to use(default overlay2)
    -s, --storage-opt                        Storage driver options
        --tls                                Use TLS; implied by --tlsverify
        --tlscacert                          Trust certs signed only by this CA (default "/root/.iSulad/ca.pem")
        --tlscert                            Path to TLS certificate file (default "/root/.iSulad/cert.pem")
        --tlskey                             Path to TLS key file (default "/root/.iSulad/key.pem")
        --tlsverify                          Use TLS and verify the remote
        --use-decrypted-key                  Use decrypted private key by default(default true)
        --userns-remap                       User/Group setting for user namespaces
    -V, --version                            Print the version
        --websocket-server-listening-port    CRI websocket streaming service listening port (default 10350)
  ```

  示例： 启动isulad，并将日志级别调整成DEBUG

  ```bash
  # isulad -l DEBUG
  ```

* **配置文件方式**

  isulad配置文件为/etc/isulad/daemon.json和/etc/isulad/daemon\_constants.json，各配置字段说明如下：

  配置文件/etc/isulad/daemon.json

  配置文件/etc/isulad/daemon\_constants.json

  示例：

  ```bash
  # cat /etc/isulad/daemon.json
  {
      "group": "isulad",
      "default-runtime": "lcr",
      "graph": "/var/lib/isulad",
      "state": "/var/run/isulad",
      "engine": "lcr",
      "log-level": "ERROR",
      "pidfile": "/var/run/isulad.pid",
      "log-opts": {
          "log-file-mode": "0600",
          "log-path": "/var/lib/isulad",
          "max-file": "1",
          "max-size": "30KB"
      },
      "log-driver": "stdout",
      "hook-spec": "/etc/default/isulad/hooks/default.json",
      "start-timeout": "2m",
      "storage-driver": "overlay2",
      "storage-opts": [
          "overlay2.override_kernel_check=true"
      ],
      "registry-mirrors": [
          "docker.io"
      ],
      "insecure-registries": [
          "rnd-dockerhub.huawei.com"
      ],
      "pod-sandbox-image": "",
      "native.umask": "secure",
      "network-plugin": "",
      "cni-bin-dir": "",
      "cni-conf-dir": "",
      "image-layer-check": false,
      "use-decrypted-key": true,
      "insecure-skip-verify-enforce": false,
      "cri-runtime": {
          "kata": "io.containerd.kata.v2"
      }
  }

  # cat /etc/isulad/daemon.json
  {
      "default-host": "docker.io",
      "registry-transformation":{
          "docker.io": "registry-1.docker.io",
          "index.docker.io": "registry-1.docker.io"
      }
  }

  ```

  > \[!TIP]须知
  >
  > 默认配置文件/etc/isulad/daemon.json仅供参考，请根据实际需要进行配置。

### 存储说明

### 约束限制

* 高并发场景（并发启动200容器）下，glibc的内存管理机制会导致内存空洞以及虚拟内存较大（例如10GB）的问题。该问题是高并发场景下glibc内存管理机制的限制，而不是内存泄露，不会导致内存消耗无限增大。可以通过设置MALLOC\_ARENA\_MAX环境变量来减少虚拟内存的问题，而且可以增大减少物理内存的概率。但是这个环境变量会导致iSulad的并发性能下降，需要用户根据实际情况做配置。

  参考实践情况，平衡性能和内存，可以设置MALLOC\_ARENA\_MAX为4。（在arm64服务器上面对iSulad的性能影响在10%以内）

  配置方法：

  1. 手动启动iSulad的场景，可以直接export MALLOC\_ARENA\_MAX=4，然后再启动iSulad即可。
  2. systemd管理iSulad的场景，可以修改/etc/sysconfig/iSulad，增加一条MALLOC\_ARENA\_MAX=4即可。

* 为daemon指定各种运行目录时的注意事项

  以--root为例，当使用/new/path/作为daemon新的Root Dir时，如果/new/path/下已经存在文件，且目录或文件名与isulad需要使用的目录或文件名冲突（例如：engines、mnt等目录）时，isulad可能会更新原有目录或文件的属性，包括属主、权限等为自己的属主和权限。

  所以，用户需要明白重新指定各种运行目录和文件，会对冲突目录、文件属性的影响。建议用户指定的新目录或文件为isulad专用，避免冲突导致的文件属性变化以及带来的安全问题。

* 日志文件管理：

  > \[!TIP]须知
  >
  > 日志功能对接： iSulad由systemd管理，日志也由systemd管理，然后传输给rsyslogd。rsyslog默认会对写日志速度有限制，可以通过修改/etc/rsyslog.conf文件，增加"$imjournalRatelimitInterval 0"配置项，然后重启rsyslogd的服务即可。

* 命令行参数解析限制

  使用iSulad命令行接口时，其参数解析方式与docker略有不同，对于命令行中带参数的flag，不管使用长flag还是短flag，只会将该flag后第一个空格或与flag直接相连接的'='后的字符串作为flag的参数，具体如下：

  1. 使用短flag时，与“-”连接的字符串中的每个字符都被当作短flag（当有=号时，=号后的字符串当成=号前的短flag的参数）。

     isula run -du=root busybox 等价于 isula run -du root busybox 或 isula run -d -u=root busybox 或 isula run -d -u root busybox ，当使用isula run -du:root时，由于-:不是有效的短flag，因此会报错。前述的命令行也等价于isula run -ud root busybox，但不推荐这种使用方式，可能带来语义困扰。

  2. 使用长flag时，与“--”连接的字符串作为一个整体当成长flag，若包含=号，则=号前的字符串为长flag，=号后的为参数。

     ```bash
     isula run --user=root busybox
     ```

     等价于

     ```bash
     isula run --user root busybox
     ```

* 启动一个isulad容器，不能够以非root用户进行isula run -i/-t/-ti以及isula attach/exec操作。

* isulad的临时文件存放路径默认为/var/lib/isulad/isulad\_tmpdir，如果更换了isulad的根目录，则路径为`$isulad_root/isulad_tmpdir`。如果需要修改isulad存放临时文件的目录，可以通过在启动isulad前配置ISULAD\_TMPDIR环境变量来实现，isulad启动时会进行检查，如果发现配置了ISULAD\_TMPDIR环境变量，则会将`$ISULAD_TMPDIR/isulad_tmpdir`目录作为临时文件存放的路径，注意`$ISULAD_TMPDIR`下面的不要存放名称为isulad\_tmpdir的文件/文件夹等，因为isulad启动时会递归删除`$ISULAD_TMPDIR/isulad_tmpdir`目录来防止数据残留。同时请确保`$ISULAD_TMPDIR`路径只有root用户可以访问，防止其他用户操作导致的安全问题。

### DAEMON多端口的绑定

## 描述

daemon端可以绑定多个unix socket或者tcp端口，并在这些端口上侦听，客户端可以通过这些端口和daemon端进行交互。

## 接口

用户可以在/etc/isulad/daemon.json文件的hosts字段配置一个或者多个端口。当然用户也可以不指定hosts。

```conf
{
    "hosts": [
        "unix:///var/run/isulad.sock",
        "tcp://localhost:5678",
        "tcp://127.0.0.1:6789"
    ]
}
```

用户也可以在/etc/sysconfig/iSulad中通过-H或者--host配置端口。用户同样可以不指定hosts。

```conf
OPTIONS='-H unix:///var/run/isulad.sock --host tcp://127.0.0.1:6789'
```

如果用户在daemon.json文件及iSulad中均未指定hosts，则daemon在启动之后将默认侦听unix:///var/run/isulad.sock。

## 限制

* 用户不可以在/etc/isulad/daemon.json和/etc/sysconfig/iSulad两个文件中同时指定hosts，如果这样做将会出现错误，isulad无法正常启动；

  ```text
  unable to configure the isulad with file /etc/isulad/daemon.json: the following directives are specified both as a flag and in the configuration file: hosts: (from flag: [unix:///var/run/isulad.sock tcp://127.0.0.1:6789], from file: [unix:///var/run/isulad.sock tcp://localhost:5678 tcp://127.0.0.1:6789])
  ```

* 若指定的host是unix socket，则必须是合法的unix socket，需要以"unix://"开头，后跟合法的socket绝对路径；

* 若指定的host是tcp端口，则必须是合法的tcp端口，需要以"tcp://"开头，后跟合法的IP地址和端口，IP地址可以为localhost；

* 可以指定至多10个有效的端口，超过10个则会出现错误，isulad无法正常启动。

### 配置TLS认证与开启远程访问

#### 描述

iSulad采用C/S模式进行设计，在默认情况，iSulad守护进程isulad只侦听本地/var/run/isulad.sock，因此只能在本地通过客户端isula执行相关命令操作容器。为了能使isula可以远程访问容器，isulad守护进程需要通过tcp:ip的方式侦听远程访问的端口。然而，仅通过简单配置tcp ip:port进行侦听，这样会导致所有的ip都可以通过调用`isula -H tcp://<remote ip="" server="">:port`与isulad通信，容易导致安全问题，因此推荐使用较安全版本的TLS(**Transport Layer Security - 安全传输层协议**）方式进行远程访问。

#### 生成TLS证书

* 明文私钥和证书生成方法示例

  ```bash
  #!/bin/bash
  set -e
  echo -n "Enter pass phrase:"
  read password
  echo -n "Enter public network ip:"
  read publicip
  echo -n "Enter host:"
  read HOST

  echo " => Using hostname: $publicip, You MUST connect to iSulad using this host!"

  mkdir -p $HOME/.iSulad
  cd $HOME/.iSulad
  rm -rf $HOME/.iSulad/*

  echo " => Generating CA key"
  openssl genrsa -passout pass:$password -aes256 -out ca-key.pem 4096
  echo " => Generating CA certificate"
  openssl req -passin pass:$password -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem -subj "/C=CN/ST=zhejiang/L=hangzhou/O=Huawei/OU=iSulad/CN=iSulad@huawei.com"
  echo " => Generating server key"
  openssl genrsa -passout pass:$password -out server-key.pem 4096
  echo " => Generating server CSR"
  openssl req -passin pass:$password -subj /CN=$HOST -sha256 -new -key server-key.pem -out server.csr
  echo subjectAltName = DNS:$HOST,IP:$publicip,IP:127.0.0.1 >> extfile.cnf
  echo extendedKeyUsage = serverAuth >> extfile.cnf
  echo " => Signing server CSR with CA"
  openssl x509 -req -passin pass:$password -days 365 -sha256 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile extfile.cnf
  echo " => Generating client key"
  openssl genrsa -passout pass:$password -out key.pem 4096
  echo " => Generating client CSR"
  openssl req -passin pass:$password -subj '/CN=client' -new -key key.pem -out client.csr
  echo " => Creating extended key usage"
  echo extendedKeyUsage = clientAuth > extfile-client.cnf
  echo " => Signing client CSR with CA"
  openssl x509 -req -passin pass:$password -days 365 -sha256 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cert.pem -extfile extfile-client.cnf
  rm -v client.csr server.csr extfile.cnf extfile-client.cnf
  chmod -v 0400 ca-key.pem key.pem server-key.pem
  chmod -v 0444 ca.pem server-cert.pem cert.pem
  ```

* 加密私钥和证书请求文件生成方法示例

  ```bash
  #!/bin/bash

  echo -n "Enter public network ip:"
  read publicip
  echo -n "Enter pass phrase:"
  read password

  # remove certificates from previous execution.
  rm -f *.pem *.srl *.csr *.cnf


  # generate CA private and public keys
  echo 01 > ca.srl
  openssl genrsa -aes256 -out ca-key.pem -passout pass:$password 2048
  openssl req -subj '/C=CN/ST=zhejiang/L=hangzhou/O=Huawei/OU=iSulad/CN=iSulad@huawei.com' -new -x509 -days $DAYS -passin pass:$password -key ca-key.pem -out ca.pem

  # create a server key and certificate signing request (CSR)
  openssl genrsa -aes256 -out server-key.pem -passout pass:$PASS 2048
  openssl req -new -key server-key.pem -out server.csr -passin pass:$password -subj '/CN=iSulad'

  echo subjectAltName = DNS:iSulad,IP:${publicip},IP:127.0.0.1 > extfile.cnf
  echo extendedKeyUsage = serverAuth >> extfile.cnf
  # sign the server key with our CA
  openssl x509 -req -days $DAYS -passin pass:$password -in server.csr -CA ca.pem -CAkey ca-key.pem -out server-cert.pem -extfile extfile.cnf

  # create a client key and certificate signing request (CSR)
  openssl genrsa -aes256 -out key.pem -passout pass:$password 2048
  openssl req -subj '/CN=client' -new -key key.pem -out client.csr -passin pass:$password

  # create an extensions config file and sign
  echo extendedKeyUsage = clientAuth > extfile.cnf
  openssl x509 -req -days 365 -passin pass:$password -in client.csr -CA ca.pem -CAkey ca-key.pem -out cert.pem -extfile extfile.cnf

  # remove the passphrase from the client and server key
  openssl rsa -in server-key.pem -out server-key.pem -passin pass:$password
  openssl rsa -in key.pem -out key.pem -passin pass:$password

  # remove generated files that are no longer required
  rm -f ca-key.pem ca.srl client.csr extfile.cnf server.csr
  ```

#### 接口

```conf
{
    "tls": true,
    "tls-verify": true,
    "tls-config": {
  "CAFile": "/root/.iSulad/ca.pem",
  "CertFile": "/root/.iSulad/server-cert.pem",
  "KeyFile":"/root/.iSulad/server-key.pem"
    }
}
```

#### 限制

服务端支持的模式如下：

* 模式1（验证客户端）：tlsverify, tlscacert, tlscert, tlskey。
* 模式2（不验证客户端）：tls, tlscert, tlskey。

客户端支持的模式如下:

* 模式1(使用客户端证书进行身份验证，并根据给定的CA验证服务器)：tlsverify, tlscacert, tlscert, tlskey。
* 模式2(验证服务器)：tlsverify, tlscacert。

如果需要采用双向认证方式进行通讯，则服务端采用模式1，客户端采用模式1；

如果需要采用单向认证方式进行通讯，则服务端采用模式2，客户端采用模式2。

> \[!TIP]须知
>
> * 采用RPM安装方式时，服务端配置可通过/etc/isulad/daemon.json以及/etc/sysconfig/iSulad配置修改。
> * 相比非认证或者单向认证方式，双向认证具备更高的安全性，推荐使用双向认证的方式进行通讯。
> * GRPC开源组件日志不由iSulad进行接管，如果需要查看GRPC相关日志，请按需设置GRPC\_VERBOSITY和GRPC\_TRACE环境变量。

#### 示例

服务端：

```bash
 isulad -H=tcp://0.0.0.0:2376 --tlsverify --tlscacert ~/.iSulad/ca.pem --tlscert ~/.iSulad/server-cert.pem --tlskey ~/.iSulad/server-key.pem
```

客户端：

```bash
 isula version -H=tcp://$HOSTIP:2376 --tlsverify --tlscacert ~/.iSulad/ca.pem --tlscert ~/.iSulad/cert.pem --tlskey ~/.iSulad/key.pem
```

### 配置devicemapper存储驱动

使用devicemapper存储驱动需要先配置一个thinpool设备，而配置thinpool需要一个独立的块设备，且该设备需要有足够的空闲空间用于创建thinpool，请用户根据实际需求确定。这里假设独立块设备为/dev/xvdf，具体的配置方法如下：

**一、配置thinpool**

1. 停止isulad服务。

   ```bash
   # systemctl stop isulad
   ```

2. 基于块设备创建一个lvm卷。

   ```bash
   # pvcreate /dev/xvdf
   ```

3. 使用刚才创建的物理卷创建一个卷组。

   ```bash
   # vgcreate isula /dev/xvdf
   Volume group "isula" successfully created：
   ```

4. 创建名为thinpool和thinpoolmeta的两个逻辑卷。

   ```bash
   # lvcreate --wipesignatures y -n thinpool isula -l 95%VG
   Logical volume "thinpool" created.
   ```

   ```bash
   # lvcreate --wipesignatures y -n thinpoolmeta isula -l 1%VG
   Logical volume "thinpoolmeta" created.
   ```

5. 将新创建的两个逻辑卷转换成thinpool以及thinpool所使用的metadata，这样就完成了thinpool配置。

   ```bash
   # lvconvert -y --zero n -c 512K --thinpool isula/thinpool --poolmetadata isula/thinpoolmeta

   WARNING: Converting logical volume isula/thinpool and isula/thinpoolmeta to
   thin pool's data and metadata volumes with metadata wiping.
   THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
   Converted isula/thinpool to thin pool.
   ```

**二、修改isulad配置文件**

1. 如果环境之前运行过isulad，请先备份之前的数据。

   ```bash
   # mkdir /var/lib/isulad.bk
   # mv /var/lib/isulad/* /var/lib/isulad.bk
   ```

2. 修改配置文件

   这里提供了两种配置方式，用户可根据实际情况的选择合适的方式。

   * 编辑/etc/isulad/daemon.json，配置storage-driver字段值为devicemapper，并配置storage-opts字段的相关参数，支持参数请参见[参数说明](#zh-cn_topic_0222861454_section1712923715282)。配置参考如下所示：

     ```conf
     {
         "storage-driver": "devicemapper"
         "storage-opts": [
          "dm.thinpooldev=/dev/mapper/isula-thinpool",
          "dm.fs=ext4",
                 "dm.min_free_space=10%"
         ]
     }
     ```

   * 或者也可以通过编辑/etc/sysconfig/iSulad，在isulad启动参数里显式指定，支持参数请参见[参数说明](#zh-cn_topic_0222861454_section1712923715282)。配置参考如下所示：

     ```conf
     OPTIONS="--storage-driver=devicemapper --storage-opt dm.thinpooldev=/dev/mapper/isula-thinpool --storage-opt dm.fs=ext4 --storage-opt dm.min_free_space=10%"
     ```

3. 启动isulad，使配置生效。

   ```bash
   # systemctl start isulad
   ```

#### 参数说明

storage-opts 支持的参数请参见[表1](#zh-cn_topic_0222861454_table3191161993812)。

**表 1**  storage-opts字段参数说明

#### 注意事项

* 配置devicemapper时，如果系统上没有足够的空间给thinpool做自动扩容，请禁止自动扩容功能。

  禁止自动扩容的方法是把/etc/lvm/profile/isula-thinpool.profile中thin\_pool\_autoextend\_threshold和thin\_pool\_autoextend\_percent两个值都改成100，如下所示：

  ```conf
  activation {   
    thin_pool_autoextend_threshold=100   
    thin_pool_autoextend_percent=100 
  }
  ```

* 使用devicemapper时，容器文件系统必须配置为ext4，需要在isulad的配置参数中加上--storage-opt dm.fs=ext4。

* 当graphdriver为devicemapper时，如果metadata文件损坏且不可恢复，需要人工介入恢复。禁止直接操作或篡改daemon存储devicemapper的元数据。

* 使用devicemapper lvm时，异常掉电导致的devicemapper thinpool损坏，无法保证thinpool损坏后可以修复，也不能保证数据的完整性，需重建thinpool。

**iSula开启了user namespace特性，切换devicemapper存储池时的注意事项**

* 一般启动容器时，deviceset-metadata文件为：/var/lib/isulad/devicemapper/metadata/deviceset-metadata。
* 使用了user namespace场景下，deviceset-metadata文件使用的是：/var/lib/isulad/{userNSUID.GID}/devicemapper/metadata/deviceset-metadata。
* 使用devicemapper存储驱动，容器在user namespace场景和普通场景之间切换时，需要将对应deviceset-metadata文件中的BaseDeviceUUID内容清空；针对thinpool扩容或者重建的场景下，也同样的需要将对应deviceset-metadata文件中的BaseDeviceUUID内容清空，否则isulad服务会重启失败。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/secure_container/installation_and_deployment_2.md
---
# 安装与配置

## 安装方法

### 前提条件

* 安全容器的安装需要使用root权限。
* 为了获取更好的性能体验，安全容器需要运行在裸金属服务器上，**暂不支持安全容器运行在虚拟机内**。
* 安全容器运行依赖以下组件，请确保环境上已安装所需版本的依赖组件。以下组件来自配套的openEuler版本。如果使用iSula容器引擎，请参考iSula容器引擎的[安装与配置](../../container_engine/isula_container_engine/installation_configuration.md)章节安装iSulad。
  * docker-engine
  * qemu

### 安装操作

安全容器发布组件集成在同一个kata-containers-<*version*>.rpm包中，使用rpm命令可以直接安装对应的软件，其中version为版本号。

```sh
rpm -ivh kata-containers-<version>.rpm
```

## 配置方法

### docker-engine容器引擎的配置

为了让docker-engine容器引擎支持新的容器运行时kata-runtime，需要通过以下步骤对docker-engine容器引擎进行配置：

1. 请保证环境上所有的软件包（docker-engine、kata-containers）都已经安装完毕。

2. 停止docker-engine。

   ```sh
   systemctl stop docker
   ```

3. 修改docker-engine的配置文件/etc/docker/daemon.json，并新增如下配置：

   ```conf
   {
     "runtimes": {
       "kata-runtime": {
         "path": "/usr/bin/kata-runtime",
         "runtimeArgs": [
             "--kata-config",
             "/usr/share/defaults/kata-containers/configuration.toml"
           ]
       }
     }
   }
   ```

4. 重新启动docker-engine。

   ```sh
   systemctl start docker
   ```

### iSula容器引擎的配置

与docker-engine容器引擎类似，为了让iSula容器引擎支持新的容器运行时kata-runtime，需要通过以下步骤对iSula容器引擎进行配置：

1. 请保证环境上所有的软件包（iSulad、kata-containers）都已经安装完毕。

2. 停止isulad。

   ```sh
   systemctl stop isulad
   ```

3. 修改iSula容器引擎的配置文件/etc/isulad/daemon.json，并新增如下配置：

   ```sh
   {
     "runtimes": {
       "kata-runtime": {
         "path": "/usr/bin/kata-runtime",
         "runtime-args": [
             "--kata-config",
             "/usr/share/defaults/kata-containers/configuration.toml"
           ]
       }
     }
   }
   ```

4. 重新启动isulad。

   ```sh
   systemctl start isulad
   ```

### 安全容器全局配置文件configuration-toml

安全容器提供全局配置文件configuration.toml进行配置开关，用户也可以定制安全容器配置文件路径与配置选项。

在docker-engine的runtimeArges字段可以利用--kata-config指定私有文件，默认的配置文件路径为/usr/share/defaults/kata-containers/configuration.toml。

常用配置文件字段如下，详细的配置文件选项参见“安全容器 > 附录 > configuration.toml配置说明”。

1. hypervisor.qemu
   * path ：指定虚拟化qemu执行路径。
   * kernel ：指定guest kernel执行路径。
   * initrd ：指定guest initrd执行路径。
   * machin\_type ：指定模拟芯片类型，其中arm为virt，x86架构为pc。
   * kernel\_params ：指定guest内核运行参数。

2. proxy.kata
   * path ：指定kata-proxy运行路径。
   * enable\_debug ：kata-proxy进程debug开关。

3. agent.kata
   * enable\_blk\_mount ：开启block设备guest挂载。
   * enable\_debug ：kata-agent进程debug开关。

4. runtime
   * enable\_cpu\_memory\_hotplug：CPU和内存热插拔开关。
   * enable\_debug：kata-runtime进程debug开关。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_preparations.md
---
# 安装准备

介绍安装前需要考虑软硬件兼容性状况，以及相关的配置和准备工作。

## 获取安装源

在安装开始前，您需要获取openEuler的发布包和校验文件。

请按以下步骤获取openEuler的发布包和校验文件：

1. 登录[openEuler社区](https://openeuler.org)网站。
2. 单击“下载”。
3. 单击“社区发行版”，显示版本列表。
4. 在版本列表的“openEuler 22.03 LTS SP4”版本处单击“前往下载”按钮，进入openEuler 22.03\_LTS\_SP4版本下载列表。
5. 根据实际待安装环境的架构和场景选择需要下载的 openEuler 的发布包和校验文件。
   1. 若为AArch64架构。
      1. 单击“AArch64”。
      2. 若选择本地安装，选择“Offline Standard ISO”或者“Offline Everything ISO”对应的“立即下载”将发布包 “openEuler-22.03-LTS-SP4-aarch64-dvd.iso”下载到本地。
      3. 若选择网络安装，选择“Network Install ISO”将发布包 “openEuler-22.03-LTS-SP4-netinst-aarch64-dvd.iso”下载到本地。
   2. 若为x86\_64架构。
      1. 单击“x86\_64”。
      2. 若选择本地安装，选择“Offline Standard ISO”或者“Offline Everything ISO”对应的“立即下载”将发布包 “openEuler-22.03-LTS-SP4-x86\_64-dvd.iso”下载到本地。
      3. 若选择网络安装，选择“Network Install ISO”将发布包 “openEuler-22.03-LTS-SP4-netinst-x86\_64-dvd.iso ”下载到本地。

> \[!NOTE]说明
>
> * 网络安装方式的 ISO 发布包较小，在有网络的安装环境可以选择网络安装方式。
> * AArch64架构的发布包支持UEFI模式，x86\_64架构的发布包支持UEFI模式和Legacy模式。

## 发布包完整性校验

> \[!NOTE]说明\
> 本章节以AArch64架构的发布包完整性校验为例，x86\_64架构的发布包完整性校验的操作方法相同。

### 简介

为了确认软件包在传输过程中由于网络原因或者存储设备原因是否出现下载不完整的问题，在获取到软件包后，需要对软件包的完整性进行校验，通过了校验的软件包才能被部署。

这里通过对比校验文件中记录的校验值和手动方式计算的iso文件校验值，判断软件包是否完整。若两个值相同，说明iso文件完整，否则，iso完整性被破坏，请重新获取iso发布包。

### 前提条件

在校验发布包完整性之前，需要准备如下文件：

iso文件：openEuler-22.03-LTS-SP4-aarch64-dvd.iso。

校验文件：ISO对应完整性校验值，复制保存对应的ISO值。

### 操作指导

文件完整性校验操作步骤如下：

1. 计算文件的sha256校验值。执行命令如下：

   ```shell
   $ sha256sum openEuler-22.03-LTS-SP4-aarch64-dvd.iso
   ```

   命令执行完成后，输出校验值。

2. 判断步骤1计算的校验值与刚刚复制的SHA256的值是否一致。

   如果校验值一致说明iso文件完整性没有破坏，如果校验值不一致则可以确认文件完整性已被破坏，需要重新获取。

## 物理机的安装要求

若需要在物理机环境上安装openEuler操作系统，则物理机需要满足如下的硬件兼容性和最小硬件要求。

### 硬件兼容支持

openEuler安装时，应注意硬件兼容性方面的问题，当前已支持的服务器类型请参考[兼容性列表](https://www.openeuler.org/zh/compatibility/)。

### 最小硬件要求

openEuler所需的最小硬件要求如[表2](#tff48b99c9bf24b84bb602c53229e2541)所示。

**表 2**  最小硬件要求

|  部件名称   | 最小硬件要求  |
|  :----  | :----  |
| 架构  | AArch64或x86\_64 |
| CPU  | 2\*CPU (1 core) |
| 内存  | 不小于4GB（为了获得更好的应用体验，建议不小于8GB） |
| 硬盘  | 不小于32GB（为了获得更好的应用体验，建议不小于120GB） |

## 虚拟机的安装要求

若需要在虚拟机环境上安装openEuler操作系统，则虚拟机需要满足如下的虚拟化平台兼容性和最小虚拟化要求。

### 虚拟化平台兼容性

openEuler安装时，应注意虚拟化平台兼容性的问题，当前已支持的虚拟化平台为：

* openEuler自有的虚拟化组件（HostOS为openEuler，虚拟化组件为发布包中的qemu、KVM）创建的虚拟化平台。
* 华为公有云的x86虚拟化平台。

### 最小虚拟化空间要求

openEuler所需的最小虚拟化空间要求如[表3](#tedf1812a7c43420faa8a4ed0424c18ef)所示。

**表 3**  最小虚拟化空间要求

|  部件名称   | 最小虚拟化空间要求  |
|  :----  | :----  |
| 架构  | AArch64或x86\_64 |
| CPU  | 1个CPU |
| 内存  | 不小于4GB（为了获得更好的应用体验，建议不小于8GB） |
| 硬盘  | 不小于32GB（为了获得更好的应用体验，建议不小于120GB） |

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_preparations_1.md
---
# 安装准备

介绍安装前需要考虑软硬件兼容性状况，以及相关的配置和准备工作。

## 获取安装源

在安装开始前，您需要获取 openEuler 发布的树莓派镜像及其校验文件。

1. 登录[openEuler Repo](https://repo.openeuler.org/)网站。
2. 单击选择 openEuler 22.03 LTS SP4 版本。
3. 单击“raspi\_img”，进入树莓派镜像的下载列表。
4. 单击“openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz”，将 openEuler 发布的树莓派镜像下载到本地。
5. 单击“openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz.sha256sum”，将 openEuler 发布的树莓派镜像的校验文件下载到本地。

## 镜像完整性校验

### 简介

为了防止软件包在传输过程中由于网络原因或者存储设备原因出现下载不完整的问题，在获取到软件包后，需要对软件包的完整性进行校验，通过了校验的软件包才能部署。

这里通过对比校验文件中记录的校验值和手动方式计算的文件校验值，判断软件包是否完整。若两个值相同，说明下载的文件完整，否则，下载的文件完整性被破坏，请重新获取软件包。

### 前提条件

在校验镜像文件的完整性之前，需要准备如下文件：

镜像文件：openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz

校验文件：openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz.sha256sum

### 操作指导

文件完整性校验操作步骤如下：

1. 获取校验文件中的校验值。执行命令如下：

   ```shell
   $ cat openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz.sha256sum
   ```

2. 计算文件的 sha256 校验值。执行命令如下：

   ```shell
   $ sha256sum openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz
   ```

   命令执行完成后，输出校验值。

3. 对比步骤 1 和步骤 2 计算的校验值是否一致。

   如果校验值一致说明下载的文件完整性没有破坏，如果校验值不一致则可以确认文件完整性已被破坏，需要重新获取。

## 安装要求

在树莓派环境上安装 openEuler 操作系统，则树莓派需要满足如下的硬件兼容性和最小硬件要求。

### 硬件兼容支持

openEuler 树莓派版本镜像目前支持树莓派 3B/3B+/4B/400。

### 最小硬件要求

openEuler 树莓派版本镜像所需的最小硬件要求如[表1](#tff48b99c9bf24b84bb602c53229e2542)所示。

**表 1**  最小硬件要求

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/system_container/installation_guideline.md
---
# 安装指导

> \[!NOTE]说明
> 系统容器的安装需要使用root权限。

1. 首先需要安装iSulad容器引擎。

   ```sh
   # yum install iSulad
   ```

2. 安装系统容器依赖包。

   ```sh
   # yum install syscontainer-tools authz lxcfs-tools lxcfs
   ```

3. 查看iSulad是否已经启动。

   ```sh
   # systemctl status isulad
   ```

4. 开启lxcfs和authz服务。

   ```sh
   # systemctl start lxcfs
   # systemctl start authz
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_guide.md
---
# 安装指导

本章以光盘安装为例介绍安装openEuler，其他安装方式除在启动安装时的引导方式不同外，待启动安装后安装流程相同，在此不再说明。

## 启动安装

### 使用光盘引导安装

在服务器的光驱中加载openEuler安装镜像，重启服务器，具体步骤如下。

> \[!NOTE]说明
> 在安装开始前，需要保证服务器启动选项为光驱优先。安装步骤以BMC挂载虚拟光驱进行光盘安装的操作举例。通过物理光驱安装的操作简单，启动安装后的流程相同，在此不再说明。

1. 在虚拟界面工具栏中，单击虚拟光驱工具如下图所示。

   **图 1**  光驱图标\
   ![](./figures/CD-ROM_drive_icon.png)

   弹出镜像对话框，如下图所示。

   **图 2**  镜像对话框\
   ![](./figures/Image_dialog_box.png)

2. 在镜像对话框中，选择“镜像文件”， 并单击“...”。弹出“打开”对话框。

3. 选择镜像文件，单击“打开”。然后在镜像对话框中，单击“连接”。当“连接”显示为“断开”后，表示虚拟光驱已连接到服务器。

4. 在工具栏中，单击重启工具重启设备，如下图所示。

   **图 3**  重启图标\
   ![](./figures/restarticon.png)

### 安装引导界面

系统使用引导介质完成引导后会显示引导菜单。该引导菜单除启动安装程序外还提供一些选项。安装系统时，默认采用“Test this media & install openEuler 22.03-LTS-SP4”方式进行安装。如果要选择默认选项之外的选项，请使用键盘中的“↑”和“↓”方向键进行选择，并在选项为高亮状态时按“Enter”。

> \[!NOTE]说明
>
> * 如果60秒内未按任何键，系统将从默认选项“Test this media & install openEuler 22.03-LTS-SP4”自动进入安装界面。
> * 安装物理机时，如果使用键盘上下键无法选择启动选项，按“Enter”键无响应，可以单击BMC界面上的鼠标控制图标“![](./figures/zh-cn_image_0229420473.png)”，设置“键鼠复位”。

**图 4**  安装引导界面\
![](figures/Installation_wizard.png)

安装引导选项说明如下：

* Install openEuler 22.03-LTS-SP4 —— 在您的服务器上使用图形用户界面模式安装。

* Test this media & install openEuler 22.03-LTS-SP4 —— 默认选项，在您的服务器上使用图形用户界面模式安装，但在启动安装程序前会进行安装介质的完整性检查。

* Troubleshooting —— 问题定位模式，系统无法正常安装时使用。进入问题定位模式后，有如下两个选项。
  * Install openEuler 22.03-LTS-SP4 in basic graphics mode —— 简单图形安装模式，该模式下在系统启动并运行之前不启动视频驱动程序。
  * Rescue the openEuler system —— 救援模式，用于修复系统。该模式下输出定向到 VNC（Virtual Network Computing）或 BMC（Baseboard Management Controller）端，串口不可用。
  * Run a memory test（仅legacy安装模式下存在） —— 用于测试内存是否正常工作，以确定是否因内存问题导致的系统问题。
  * Boot from local drive（仅legacy安装模式下存在） —— 该模式下可以选择本地硬盘启动系统。

> \[!NOTE]说明
>
> * UEFI启动模式下，按“e”进入参数编辑界面；按“c”进入命令行模式。
> * Legacy启动模式下，按“Tab”进入参数编辑界面；Legacy安装使用syslinux，不走grub，该模式下无法切换到命令行模式。

### 图形化模式安装

在“安装引导界面”中选择“Test this media & install openEuler 22.03-LTS-SP4”进入图形化模式安装。

可以通过键盘操作图形化安装程序。

* “Tab”、“shift Tab”：界面控件（按钮、区域框、复选框等）间的移动。
* “↑”、“↓”方向键：列表里的移动。
* “←”、“→”方向键：水平工具条和表条间移动。
* “空格”、“Enter”：选择或删除高亮显示的选项、展开或折叠下拉菜单。
* “Alt”+“快捷键”：选择快捷键所在的控件，其中快捷键可通过按住Alt高亮（加下划线）显示。

## 设置安装程序语言

启动安装后，在进入安装程序主界面之前，系统会提示用户设置安装过程中使用的语言。当前默认为英语，用户可根据实际情况进行调整，如[图5](#zh-cn_topic_0186390093_zh-cn_topic_0122145864_fig144630179151)所示，选择“中文”。

**图 5**  选择语言

![](figures/selectlanguage.png)

完成设置后，单击“继续”，进入安装设置主界面。

如果您想退出安装，可以单击“退出”并在弹出的“您确定要退出安装程序吗？”对话框中单击“是”重新进入“安装引导界面”。

## 进入安装界面

系统安装程序正常启动后，会进入[图6](#zh-cn_topic_0186390094_zh-cn_topic_0122145883_fig5969171592212)所示的安装设置主界面。用户可以进行时间、语言、安装源、网络、安装位置等相关设置。

部分配置项会有告警符号，用户完成该选项配置后，告警符号消失。当界面上不存在告警符号时，用户才能单击“开始安装”进行系统安装。

如果您想退出安装，可以单击“退出”并在弹出的“您确定要退出安装程序吗？”对话框中单击“是”重新进入“安装引导界面”。

**图 6**  安装信息摘要\
![](figures/installation_overview.png)

## 设置键盘

在“安装信息摘要”页面中选择“键盘”，用户可以在系统中添加或者删除多个键盘布局。

* 要查看键盘布局，请在左侧选框中单击选中该键盘布局，然后单击下面的“键盘”按钮。
* 要测试键盘布局，请在左侧选框中添加键盘布局，然后在右上角键盘图标处进行点击切换为目标键盘，单击右侧文本框内部，输入文本以确认所选键盘布局可正常工作。

**图 7**  键盘布局\
![](figures/Keyboard_layout.png)

设置完成后，请单击左上角“完成”返回“安装信息摘要”页面。

## 设置系统语言

在“安装信息摘要”页面中选择“语言支持”，设置系统的语言。如[图8](#zh-cn_topic_0186390098_zh-cn_topic_0122145772_fig187301927172619)所示，用户也可根据实际情况进行调整，选择“中文”。

> \[!NOTE]说明
> 若选择“中文”，系统安装完成后，使用VNC登录不支持中文显示，使用串口登录支持中文显示，使用SSH登录时是否支持中文显示与使用的SSH客户端有关。若选择“English”，则无影响。

**图 8**  语言支持\
![](figures/languagesupport.png)

设置完成后，请单击左上角“完成”返回“安装信息摘要”页面。

## 设置时间和日期

在“安装信息摘要”页面中选择“时间和日期”，设置系统的时区、日期、时间等。

设置时区时，用户可通过页面顶部的“地区”和“城市”下拉菜单中进行选择，如[图9](#zh-cn_topic_0186390096_zh-cn_topic_0122145900_fig1260162652312)所示。

如果您所在城市没有出现在下拉菜单中，请选择同一时区中离您最近的城市。

> \[!NOTE]说明
>
> * 手动设置时区时，请先关闭右上角“网络时间”同步开关。
> * 如需使用网络时间，请保证网络能连通远程NTP服务器，设置网络具体请参见“[设置网络和主机名](#设置网络和主机名)”。

**图 9**  日期和时间\
![](figures/dateandtime.png)

设置完成后，请单击左上角“完成”返回“安装信息摘要”页面。

## 设置安装源

在“安装信息摘要”页面中选择“安装源”，指定安装源的位置。

* 当使用完整光盘安装时，安装程序会自动探测并显示安装源信息，用户直接使用默认配置即可，不需要进行设置，如[图10](#zh-cn_topic_0186390100_zh-cn_topic_0144427079_fig93633295132)所示。

  **图 10**  安装源

  ![](figures/Installation_source.png)

* 当使用网络源进行安装的时候，需设置网络源的 URL。

  * http 或 https 方式

    http 或 https 方式的安装源如下图所示。 输入框内容以实际版本发布的安装源地址为准，如<https://repo.openeuler.org/openEuler-22.03-LTS-SP4/OS/x86_64/> ，其中openEuler-22.03-LTS-SP4 为版本号，x86\_64 为CPU 架构，可根据实际情况输入。

    ![](figures/installsource.png)

    > \[!NOTE]说明
    >
    > * 如果https服务器使用的是私有证书，则需要在安装引导界面按“e”（UEFI模式下）或“Tab”（Legacy模式下）进入已选选项的参数编辑界面，在参数中增加inst.noverifyssl参数。UEFI模式下inst.noverifyssl参数应加在linux开头的行。

  * ftp 方式

    ftp 方式的安装源如下图所示，输入框内容根据 ftp 地址输入。

    ![](figures/sourceftp.png)

    ftp服务器需要用户自己搭建，将iso镜像进行挂载，挂载出的文件拷贝到ftp的共享目录中。

  * nfs 方式

    nfs 方式的安装源如下图所示，输入框内容根据 nfs 地址输入。

    ![](figures/sourcenfs.png)

    nfs服务器需要用户自己搭建，将iso镜像进行挂载，挂载出的文件拷贝到nfs的共享目录中。

安装过程中，如果“设置安装源”有疑问，可参考“[选择安装源出现异常](https://docs.openeuler.openatom.cn/zh/docs/common/faq/server/installation_faq1.html#%E9%97%AE%E9%A2%984-%E9%80%89%E6%8B%A9%E5%AE%89%E8%A3%85%E6%BA%90%E5%87%BA%E7%8E%B0%E5%BC%82%E5%B8%B8)”。

设置完成后，请单击左上角“完成”返回“安装信息摘要”页面。

## 选择安装软件

在“安装信息摘要”页面中选择“软件选择”，指定需要安装的软件包。

用户需要根据实际的业务需求，在左侧选择一个“最小安装”，在右侧选择安装环境的附加选项，如[图11](#zh-cn_topic_0186390261_zh-cn_topic_0122145865_fig03031519101414)所示。

**图 11**  软件选择

![](figures/choosesoftware.png)

> \[!NOTE]说明
>
> * 在最小安装的环境下，并非安装源中所有的包都会安装。如果用户需要使用的包未安装，可将安装源挂载到本地制作repo源，通过DNF工具单独安装。
> * 选择“虚拟化主机”时会默认安装虚拟化组件qemu、libvirt、edk2，且可在附加选项处选择是否安装ovs等组件。

设置完成后，请单击左上角“完成”返回“安装信息摘要”页面。

## 设置安装目的地

在“安装信息摘要”页面中选择“安装目的地”，设置操作系统的安装磁盘及分区。

在[图12](#fig1195417125015)所示的页面中您可以看到计算机中的本地可用存储设备。

**图 12**  安装目标位置\
![](figures/Target_installation_position.png)

### 存储配置

在“安装目标位置”界面，您需要进行存储配置以便对系统分区。您可以手动配置分区，也可以选择让安装程序自动分区。

> \[!NOTE]说明
>
> * 在进行分区时，出于系统性能和安全的考虑，建议您划分如下单独分区：/boot、/var、/var/log 、/var/log/audit、/home、/tmp。
> * 系统如果配置了swap分区，当系统的物理内存不够用时，会使用swap分区。虽然 swap分区可以增大物理内存大小的限制，但是如果由于内存不足使用到swap分区，会增加系统的响应时间，性能变差。因此在物理内存充足或者性能敏感的系统中，不建议配置swap分区。
> * 如果需要拆分逻辑卷组则需要选择“自定义”进行手动分区，并在“手动分区”界面单击“卷组”区域中的“修改”按钮重新配置卷组。

**自动**

如果是在未使用过的存储设备中执行全新安装，或者不需要保留该存储设备中的任何数据，建议选择“自动”进行自动分区。设置完成后，请单击“完成”返回“安装信息摘要”页面。

**自定义**

若用户需进行手动分区，选择“自定义”按钮，并单击左上角“完成”，出现手动分区界面。

在“手动分区”界面可以通过如下两种方式进行分区，分区完成后如[图13](#fig1277151815248)所示。

* 自动创建：在界面单击“点击这里自动创建它们”，系统会根据可用的存储空间，自动分区。

* 手动创建：单击“![](./figures/zh-cn_image_0229291243.png)”添加新挂载点，建议每个挂载点的期望容量不超过可用空间。

  > \[!NOTE]说明
  > 若设置的挂载点期望容量超过了可用空间，系统将剩余的可用空间全部分配给该挂载点。

**图 13**  手动分区\
![](figures/Manual_partitioning.png)

> \[!NOTE]说明
> 如果选择非 UEFI 引导，则不需要 /boot/efi 分区。若选择 UEFI 引导，则必须有 /boot/efi 分区。

设置完成后，请单击左上角”完成“按钮，弹出“更改摘要”对话框，提示更改产生的变更信息。

点击“接受更改”，返回“安装信息摘要”页面。

## 设置网络和主机名

在“安装信息摘要”页面中选择“网络和主机名”，设置系统的网络功能。

安装程序会自动探测可本地访问的接口。探测到的接口列在左侧方框中，右侧显示相应的接口详情，如[图14](#zh-cn_topic_0186390264_zh-cn_topic_0122145831_fig123700157297)所示。用户可以通过页面右上角的开关，来开启或者关闭网络接口。开关默认是关闭状态，若设置安装源选择的是在网络上安装，需要开启开关。用户还可以单击“配置”以配置选中的接口。勾选“自动以优先级连接”选项，即可将该网卡设置为开机自启动，如[图15](#zh-cn_topic_0186390264_zh-cn_topic_0122145831_fig6)所示。

用户可在页面下方“主机名”字段输入主机名。主机名可以是完全限定域名（FQDN），其格式为hostname.domainname；也可以是简要主机名，其格式为hostname。

**图 14**  网络和主机名\
![](figures/NetworkandHostName.png)

**图 15**  配置网络

![](figures/confignetwork1.png)

设置完成后，请单击左上角“完成”返回“安装信息摘要”页面。

## 设置root密码

在“安装信息摘要”页面中选择“根密码”，弹出设置“ROOT密码”界面，如[图16](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1323165793018)所示，根据[密码复杂度](#密码复杂度)输入密码并再次输入密码进行确认。

> \[!NOTE]说明
>
> * root帐户是用来执行关键系统管理任务，不建议您在日常工作及系统访问时使用root帐户。
>
> * 在“ROOT密码”界面若选择“锁定root帐户”则root帐户将禁用。

**图 16**  root密码\
![](figures/root_password.png)

### 密码复杂度

用户设置的root用户密码或新创建用户的密码均需要满足密码复杂度要求，否则会导致密码设置或用户创建失败。设置密码的复杂度的要求如下：

1. 口令长度至少8个字符。
2. 口令至少包含大写字母、小写字母、数字和特殊字符中的任意3种。
3. 口令不能和帐号一样。
4. 口令不能使用字典词汇。

   > \[!NOTE]说明
   > 在已装好的openEuler环境中，可以通过`cracklib-unpacker /usr/share/cracklib/pw_dict > dictionary.txt`命令导出字典库文件dictionary.txt，用户可以查询密码是否在该字典中。

完成设置后，单击左上角的“完成”返回“安装信息摘要”页面。

## 创建用户

在“安装信息摘要”页面中选择“创建用户”，弹出“创建用户”的界面如[图17](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319)所示。输入用户名，并设置密码。另外您还可以通过“高级”选项设置用户主目录、用户组等，如[图18](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig128716531312)所示。

**图 17**  创建用户\
![](figures/createuser.png)

**图 18**  高级用户配置\
![](figures/Advanced_User_Configuration.png)

### 高级设置要求

创建用户，默认会创建同名用户组，可以手动设置用户以及用户组ID，高级选项中用户组名请留空（如果填写同名用户组，会导致不能创建用户）；如果用户组和用户不同名，请正常填写需要创建的用户组。

完成设置后，单击左上角的“完成”返回“安装信息摘要”页面。

## 开始安装

在安装界面上完成所有必填选项的配置后，界面上的警告会消失。此时，用户可以单击“开始安装”进行系统安装。

## 安装过程

开始安装后会出现进度页面，显示安装进度及所选软件包写入系统的进度，如[图19](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1590863119306)所示。

> ![](./figures/zh-cn_image_0229291229.png)
> 若系统安装过程中，单击“退出”，或复位、下电服务器，则安装过程被中断，系统将不可用，需要重新进行安装。

**图 19**  安装过程\
![](figures/installation_procedure.png)

## 安装完成

安装过程执行完成后，openEuler完成安装，单击“重启系统”后，系统将重新启动。

> \[!NOTE]说明
>
> * 如果当前使用物理光盘安装操作系统，且在重启过程中安装光盘没有自动弹出，请手动取出光盘，则可以直接进入openEuler命令行登录界面。
> * 如果当前使用虚拟光驱安装操作系统，则需要修改服务器的启动项为“硬盘”，然后重启服务器，则可以直接进入openEuler命令行登录界面。

## 文本模式安装

> \[!NOTE]说明
>
> * UEFI启动模式下，选择安装模式，按“e”进入参数编辑界面，在linux启动参数行末尾添加inst.text，按“ctrl+x”，进入文本安装。
> * Legacy启动模式下，选择安装模式，按“Tab”进入参数编辑界面，添加启动参数inst.text，按回车键，进入文本安装。
>   可以通过界面提示操作键盘进行文本程序安装。如【图1】所示。

**图 1** 文本菜单
![](figures/zh-cn_image_text_menu.png)

> \[!NOTE]说明
> \[x]表示此项已配置（或者系统默认已配置），可以根据需要进行设置；\[!]表示此项未配置，在启动安装前必须完成配置。

## 设置系统语言

在文本主界面【图1】输入“1”，然后回车，设置系统的语言。用户可根据实际情况进行调整，如[图2](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig4782177787260)所示。

**图 2** 文本菜单 &#x20;

> ![](figures/zh-cn_image_text_language.png)

设置完成后，在界面输入“c”，返回安装配置主页面。

## 设置时间和日期

在文本主界面【图1】输入“2”并回车，进入时间设置，如[图3](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig4615555918615)所示。

**图 3** 时间设置 &#x20;

> ![](figures/zh-cn_image_text_timedate.png)

* 时区设置

在[图3](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig4615555918615)中，输入“1”并回车，进入系统时区设置。用户可根据实际情况进行调整，设置系统的时区及城市，如【图4】所示。

**图 4** 时区设置

> ![](figures/zh-cn_image_text_timeZone.png)

选择城市时，因为城市数量多需要分页显示，出现“Press ENTER to continue”提示，请点击回车以显示全部的城市信息，如[图5](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig9289965048109)所示。

**图 5** 城市设置 &#x20;

> ![](figures/zh-cn_image_text_timeZoneCity.png)

设置完成后，在界面输入“c”，返回安装配置主页面。

* NTP设置

在[图3](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig4615555918615)中，输入“2”并回车，进入NTP设置，如【图6】所示。

**图 6** NTP设置

> ![](figures/zh-cn_image_text_timeNTP.png)

设置完成后，在界面输入“c”，返回安装配置主页面。

## 设置安装源

在文本主界面【图1】输入“3”并回车，指定安装源的位置，如[图7](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig8095563954238)所示。您可以选择可本地访问的安装源，也可以选择网络位置。

**图 7** 安装源设置 &#x20;

![](figures/zh-cn_image_text_installSource.png)

设置完成后，在界面输入“c”，返回安装配置主页面。

> \[!NOTE]说明
>
> * CD/DVD：从挂载光驱获取安装源。
> * local ISO file：从本地磁盘中获取iso文件。
> * Network：通过http、https、ftp、nfs获取网路服务器上的安装源。

## 选择安装软件

在文本主界面【图1】输入“4”并回车，指定需要安装的软件包。如[图8](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig2423583527003)所示。
Minimal Install：最小环境。提供openEuler基本功能，默认为最小环境安装。用户选择安装环境，在界面输入“c”；然后选择已选环境的附加选项“Add-ons”，选择需要安装的软件包。

**图 8** 安装源设置 &#x20;

![](figures/zh-cn_image_text_installSofware.png)

不同安装环境的可选附加包会有差异：

1. Standard：标准。openEuler标准安装。
2. Container Management：容器管理。用于管理 Linux 容器的工具。
3. Development Tools：开发工具。基本开发环境。
4. Headless Management：无图形终端系统管理工具。用于管理无图像终端系统的工具。
5. Legacy UNIX Compatibility：传统UNIX兼容性。用于从继承UNIX环境中迁移或者可用于该环境的兼容程序。
6. Network Servers：网络服务器。这些软件包包括基于网络的服务器，例如DHCP、Kerberos和NIS。
7. Scientific Support：科学计数法支持。用于数学和科学计算以及平行计算的工具。
8. Security Tools：安全性工具。用于完整性和可信验证的安全性工具。
9. System Tools：系统工具。这组软件包是各类系统工具的集合，如：连接SMB共享的客户；监控网络交通的工具。
10. Smart Card Support：智能卡支持。支持使用智能卡验证。

设置完成后，在界面输入“c”，返回安装配置主页面。

## 设置安装目的地

在文本主界面【图1】输入“5”并回车，设置操作系统的安装位置。如[图9](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig3928430235799)所示。安装程序会自动探测并显示可用安装位置信息，通常情况下，用户直接使用默认配置即可。

**图 9** 安装目的地设置 &#x20;

![](figures/zh-cn_image_text_installdest.png)

* 分区选项

  * Replace Existing Linux system（s）：替换已有系统。只使用已有系统占用的空间，已有系统数据将会被覆盖掉。

  * Use All Space：使用所有空间。清理系统盘所有数据，安装的系统会占用所有的系统盘空间。

  * Use Free Space：使用剩余空间。仅在系统盘剩余空间上安装系统，不会清理已有系统的数据。

  * Manually assign mount points：手动分配挂载点。用户可以手动分配挂载点，但是该功能仅在实验阶段，实际上并不能实现手动分配挂载点的功能。

* 分区方案

  * Standard Partition：标准分区。标准分区可包含文件系统或者 swap 空间，也可为软件 RAID 或者 LVM 物理卷提供容器。

  * LVM：逻辑卷。逻辑卷管理（LVM）显示一个基本物理存储空间（比如硬盘或者 LUN）的简单裸机视图。可将物理存储中视为物理卷的分区分组成为卷组。可将每个卷组分成多个逻辑卷，每个逻辑卷模拟一个标准磁盘分区。因此，LVM 逻辑卷可作为包含多个物理磁盘的分区使用。

  * LVM Thin Provisioning：逻辑卷精简配置。使用精简配置，您可以管理可用空间的存储池，也称精简池，可在程序需要时将其分配给任意数量的设备。可在需要时动态扩展精简池以便有效分配存储空间。

设置完成后，在界面输入“c”，返回安装配置主页面。

> \[!NOTE]说明

* 在进行分区时，出于系统性能和安全的考虑，建议您划分如下单独分区：/boot、/var、/var/log 、/var/log/audit、/home、/tmp。

* 系统如果配置了swap分区，当系统的物理内存不够用时，会使用swap分区。虽然 swap分区可以增大物理内存大小的限制，但是如果由于内存不足使用到swap分区，会增加系统的响应时间，性能变差。因此在物理内存充足或者性能敏感的系统中，不建议配置swap分区。同时也需要注意，swap分区的卸载对内存有一定要求，系统的空闲内存（包括当前系统内的可回收内存）大小应该大于当前的swap分区中的所使用内存，否则swap分区的卸载将会失败。

* 系统默认从第一块磁盘引导。若本地有两块或更多可用存储设备，建议选择第一块磁盘作为操作系统安装位置，否则可能造成操作系统无法正常启动，详见4.2 安装 openEuler 时选择第二盘位为安装目标，操作系统无法启动。

## 设置网络和主机名

在文本主界面【图1】输入“6”并回车，设置系统的网络功能。如[图10](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig1795569513661)所示。用户可以设置主机名，以及配置网络设备。安装阶段可以不用配置，系统启动之后再进行配置。

**图 10** 网络设置 &#x20;

![](figures/zh-cn_image_text_net.png)

* 设置主机名

选择“1”，输入主机名localhost，然后按回车键，配置网卡。
由于内核和systemd不接受hostname以“.”符号结尾，请在设置hostname时确保其末尾不是“.”符号，否则会造成设置失败。

* 配置网卡

选择“2”，配置网卡信息。如[图11](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig8855555707338)所示。

**图 11** 网卡配置 &#x20;

![](figures/zh-cn_image_text_networkConfig.png)

1. IPv4 address or "dhcp" for DHCP：配置IPv4的地址，默认值为dhcp，有dhcp服务器分配ip地址。
2. IPv4 netmask：配置IPv4的掩码。
3. IPv4 gateway：配置IPv4的网关。
4. IPv6 address\[/prefix] or "auto" for automatic, "dhcp" for DHCP, "ignore" to turn off：配置IPv6地址，默认为auto。
5. IPv6 default gateway：配置IPv6默认网关。
6. Nameservers (comma separated)：DNS服务器。
7. Connect automatically after reboot：重启之后自动连接网络。
8. Apply configuration in installer： 在安装程序中应用该配置。

设置完成后，在界面输入“c”，返回安装配置主页面。

## 设置root密码

在文本主界面【图1】输入“7”并回车，设置root密码。如[图12](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig3845102911461)所示。

**图 12** root密码设置 &#x20;

> ![](figures/zh-cn_image_text_rootset.png)

选择“1”，选择SM3加密方式，然后按“c”键，输入密码。也可以默认跳过不选择SM3加密，按“c”键，输入密码。如[图13](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig3633420080130)所示。

**图 13** 输入root密码 &#x20;

> ![](figures/zh-cn_image_text_rootpassword.png)

设置完成后，在界面输入“c”，返回安装配置主页面。

> \[!NOTE]说明
>
> * 强制用户在安装系统时设置root密码，否则无法安装通过。
>
> * root帐户用于执行关键系统管理任务，不建议您在日常工作及系统访问时使用root帐户。
>
> * 默认密码加密算法yescrypt，在不支持yescrypt时会调整为SHA512。
>
> * 用户在设置root用户密码或者新创建用户密码时，强烈建议按照密码复杂度要求设置。设置弱密码（不符合下文所述复杂度要求的密码）时系统会发出告警并询问是否仍然使用，输入“yes”可以强制设置弱密码，但是弱密码存在较大安全风险，请用户根据实际情况谨慎决策。建议密码的复杂度的要求如下：
>
>   1. 口令长度至少8个字符。
>   2. 口令至少包含大写字母、小写字母、数字和特殊字符中的任意3种。
>   3. 口令不能和帐号一样。
>   4. 口令不能使用字典词汇。

## 创建用户

在文本主界面【图1】输入“8”并回车，按照界面提示，创建用户、设置用户全名/用户名、用户密码、是否管理员、属组等信息。如[图14](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig7344535785884)所示。

**图 14** 创建用户 &#x20;

![](./figures/zh-cn_image_text_createUsr.png)

> \[!NOTE]说明
>
> * 默认密码加密算法yescrypt，在不支持yescrypt时会调整为SHA512。
> * 用户在新创建用户密码时，需要满足密码复杂度要求，否则会设置失败。设置密码的复杂度的要求如下：
>   1. 口令长度至少8个字符。
>   2. 口令至少包含大写字母、小写字母、数字和特殊字符中的任意3种。
>   3. 口令不能和帐号一样。
>   4. 口令不能使用字典词汇。

设置完成后，在界面输入“c”，返回安装配置主页面。

## 开始安装

在文本主界面【图1】上完成所有必填选项的配置后，界面上的警告\[!]会消失，如[图15](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig2269035885876)所示。此时，在界面输入“b”，进行系统安装。

**图 15** 配置完成 &#x20;

> ![](./figures/zh-cn_image_text_startInstall.png)

完成安装，如[图16](#zh-cn_topic_0186390268_zh-cn_topic_0122145909_fig2942971748186)所示。

**图 16** 安装完成 &#x20;

> ![](./figures/zh-cn_image_text_Installed.png)

安装完成后，点击回车会重启系统。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_guide_1.md
---
# 安装指导

本章介绍将“[树莓派镜像刷写入 SD 卡](./installation_modes_1.md)”后，启用树莓派的主要过程。

## 启动系统

将刷写镜像后的 SD 卡插入树莓派，通电启用。

树莓派硬件相关信息请参考[树莓派官网](https://www.raspberrypi.org/)。

## 登录系统

登录树莓派有以下两种方式：

1. 本地登录

   树莓派连接显示器（树莓派视频输出接口为 Micro HDMI）、键盘、鼠标后，启动树莓派，可以看到树莓派启动日志输出到显示器上。待树莓派启动成功，输入用户名（root）和密码（openeuler）登录。

2. ssh 远程登录

   树莓派默认采用 DHCP 的方式自动获取 IP。如果树莓派连接已知路由器，可登录路由器查看，新增的 IP 即为树莓派 IP。例如，树莓派对应 IP 为：192.168.31.109，使用命令 `ssh root@192.168.31.109` 后输入密码 `openeuler`，即可远程登录树莓派。

## 配置系统

### 扩展根目录分区

默认根目录分区空间比较小，在使用之前，需要对分区进行扩容。

请按照以下步骤扩展根目录分区：

1. 在 root 权限下执行 `fdisk -l` 命令查看磁盘分区信息。命令和回显如下：

   ```sh
   # fdisk -l
   Disk /dev/mmcblk0: 14.86 GiB, 15931539456 bytes, 31116288 sectors
   Units: sectors of 1 * 512 = 512 bytes
   Sector size (logical/physical): 512 bytes / 512 bytes
   I/O size (minimum/optimal): 512 bytes / 512 bytes
   Disklabel type: dos
   Disk identifier: 0xf2dc3842

   Device         Boot   Start     End Sectors  Size Id Type
   /dev/mmcblk0p1 *       8192  593919  585728  286M  c W95 FAT32 (LBA)
   /dev/mmcblk0p2       593920 1593343  999424  488M 82 Linux swap / Solaris
   /dev/mmcblk0p3      1593344 5044223 3450880  1.7G 83 Linux
   ```

   SD 卡对应盘符为 /dev/mmcblk0，包括 3 个分区，分别为：

   * /dev/mmcblk0p1：引导分区
   * /dev/mmcblk0p2：交换分区
   * /dev/mmcblk0p3：根目录分区

   这里我们需要将根目录分区 `/dev/mmcblk0p3` 进行扩容。

2. 在 root 权限下执行 `fdisk /dev/mmcblk0` 命令进入到交互式命令行界面，按照以下步骤扩展分区，如[图 1](#zh-cn_topic_0151920806_f6ff7658b349942ea87f4521c0256c315)所示。

   1. 输入 `p`，查看分区信息。

      记录分区 `/dev/mmcblk0p3` 的起始扇区号，即 `/dev/mmcblk0p3` 分区信息中 `Start` 列的值，示例中为 `1593344`。

   2. 输入 `d`，删除分区。

   3. 输入 `3` 或直接按 `Enter`，删除序号为 `3` 的分区，即 `/dev/mmcblk0p3` 分区。

   4. 输入 `n`，创建新的分区。

   5. 输入 `p` 或直接按 `Enter`，创建 `Primary` 类型的分区。

   6. 输入 `3` 或直接按 `Enter`，创建序号为 `3` 的分区，即 `/dev/mmcblk0p3` 分区。

   7. 输入新分区的起始扇区号，即第 `1` 步中记录的起始扇区号，示例中为 `1593344`。

      > \[!NOTE]说明\
      > 请勿直接按“Enter”或使用默认参数。

   8. 按 `Enter`，使用默认的最后一个扇区号作为新分区的终止扇区号。

   9. 输入 `N`，不修改扇区标记。

   10. 输入 `w`，保存分区设置并退出交互式命令行界面。

   **图 1**  分区扩容\
   ![](./figures/Partition_expansion.png)

3. 在 root 权限下执行 `fdisk -l` 命令查看磁盘分区信息，以确保磁盘分区正确。命令和回显如下：

   ```sh
   # fdisk -l
   Disk /dev/mmcblk0: 14.86 GiB, 15931539456 bytes, 31116288 sectors
   Units: sectors of 1 * 512 = 512 bytes
   Sector size (logical/physical): 512 bytes / 512 bytes
   I/O size (minimum/optimal): 512 bytes / 512 bytes
   Disklabel type: dos
   Disk identifier: 0xf2dc3842

   Device         Boot   Start      End  Sectors  Size Id Type
   /dev/mmcblk0p1 *       8192   593919   585728  286M  c W95 FAT32 (LBA)
   /dev/mmcblk0p2       593920  1593343   999424  488M 82 Linux swap / Solaris
   /dev/mmcblk0p3      1593344 31116287 29522944 14.1G 83 Linux
   ```

4. 在 root 权限下执行 `resize2fs /dev/mmcblk0p3`，增大未加载的文件系统大小。

5. 执行 `df -lh` 命令查看磁盘空间信息，以确保根目录分区已扩展。

   > \[!NOTE]说明
   > 如果根目录分区未扩展，可执行 `reboot` 命令重启树莓派之后再在 root 权限下执行 `resize2fs /dev/mmcblk0p3`。

### 连接 WIFI

请按照以下步骤连接 WIFI：

1. 查看 IP 和网卡信息。

   `ip a`

   获取无线网卡 wlan0 信息：

   ```sh
   1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
       inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
       inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
   2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
       link/ether dc:a6:32:50:de:57 brd ff:ff:ff:ff:ff:ff
       inet 192.168.31.109/24 brd 192.168.31.255 scope global dynamic noprefixroute eth0
       valid_lft 41570sec preferred_lft 41570sec
       inet6 fe80::cd39:a969:e647:3043/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
   3: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
       link/ether e2:e6:99:89:47:0c brd ff:ff:ff:ff:ff:ff
   ```

2. 扫描可以连接的 WIFI 信息。

   `nmcli dev wifi`

3. 连接 WIFI。

   在 root 权限下执行 `nmcli dev wifi connect SSID password PWD` 命令连接 WIFI。

   其中，`SSID` 为上一步扫描到的可供连接的 WIFI 的 SSID，`PWD` 为对应 WIFI 的密码。例如，`SSID` 为 `openEuler-wifi`，密码为 `12345678`，则连接该 WIFI 命令为：`nmcli dev wifi connect openEuler-wifi password 12345678`，连接成功：

   ```sh
   Device 'wlan0' successfully activated with '26becaab-4adc-4c8e-9bf0-1d63cf5fa3f1'.
   ```

4. 查看 IP 和无线网卡信息。

   `ip a`

   ```sh
   1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
       inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
       inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
   2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
       link/ether dc:a6:32:50:de:57 brd ff:ff:ff:ff:ff:ff
       inet 192.168.31.109/24 brd 192.168.31.255 scope global dynamic noprefixroute eth0
       valid_lft 41386sec preferred_lft 41386sec
       inet6 fe80::cd39:a969:e647:3043/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
   3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
       link/ether dc:a6:32:50:de:58 brd ff:ff:ff:ff:ff:ff
       inet 192.168.31.110/24 brd 192.168.31.255 scope global dynamic noprefixroute wlan0
       valid_lft 43094sec preferred_lft 43094sec
       inet6 fe80::394:d086:27fa:deba/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_modes.md
---
# 安装方式介绍

> \[!NOTE]说明
>
> * 硬件服务器仅支持Taishan 200服务器和FusionServer Pro 机架服务器，具体支持的服务器型号可参考“[硬件兼容支持](./installation_preparations.md#硬件兼容支持)”；虚拟化平台仅支持openEuler自有的虚拟化组件（HostOS为openEuler，虚拟化组件为发布包中的qemu、KVM）创建的虚拟化平台和华为公有云的x86虚拟化平台。
> * 安装方式当前仅支持光盘、USB盘安装、网络安装、qcow2镜像安装和私有镜像安装。其中仅华为公有云的x86虚拟化平台支持私有镜像安装。

## 通过光盘安装

本节介绍如何使用或者制作光盘安装源，并介绍相应的操作步骤，指导用户进行安装。

### 准备安装源

如果您获取的是系统安装光盘，那么可以直接使用光盘安装系统。如果您获取的是系统ISO镜像，可以通过刻录软件将系统的ISO镜像刻录到DVD中，使用刻录完成的DVD安装系统。

### 启动安装

根据以下步骤启动安装程序：

> \[!NOTE]说明\
> 您需要先设置您的系统优先从光盘进行启动引导。以BIOS为例，您需要将“Boot Type Order”中的“CD/DVD-ROM Drive”选项调整到首位。

1. 断开所有安装不需要的驱动器（非必须），比如USB。
2. 启动您的计算机系统。
3. 在计算机中插入安装光盘。
4. 重启计算机系统。

在短暂的延迟后会出现图形化引导界面，该界面包含不同引导选项。如果您在一分钟内未进行任何操作，安装程序将自动以默认选项开始运行。

## 通过USB盘安装

本节介绍如何制作USB盘安装源，并介绍基本的操作步骤，指导用户进行安装。

### 准备安装源

您需要注意USB盘容量的大小，它必须有足够的空间放下整个镜像，建议USB盘空间大于16G。

1. 将USB盘连接到该系统中，并执行 dmesg 命令查看相关的日志信息。在该日志的最后可以看到刚刚连接的USB盘所生成的一组信息，应类似如下：

   ```sh
   [ 170.171135] sd 5:0:0:0: [sdb] Attached SCSI removable disk
   ```

   > \[!NOTE]说明\
   > 连接的USB盘名称以sdb进行举例。

2. 切换为root用户。使用su命令，需要输入相应的密码。

   ```sh
   $ su - root
   ```

3. 确保USB盘没有被挂载。使用如下命令进行查询：

   ```sh
   # findmnt /dev/sdb
   ```

   * 如果执行此命令后无输出，表明未挂载，可以继续执行下一步。

   * 如果输出以下信息，表明USB盘已经自动挂载。

     ```sh
     # findmnt /dev/sdb
     TARGET   SOURCE   FSTYPE  OPTIONS
     /mnt/iso /dev/sdb iso9660 ro,relatime
     ```

     此时，您需要使用umount命令卸载该设备。

     ```sh
     # umount /mnt/iso
     ```

4. 使用dd命令将ISO安装镜像直接写入USB盘：

   > \[!NOTE]说明\
   > 如isolinux描述，由mkisofs命令创建的ISO 9660 文件系统会通过BIOS固件启动，但只能从CD、DVD和BD等介质启动。所以在使用dd命令制作x86的启动U盘前需要使用  isohybrid -u your.iso 对iso进行处理，然后正常使用dd命令将iso写入u盘即可。（该问题仅影响x86）。

   ```sh
   # dd if=/path/to/image.iso of=/dev/device bs=blocksize
   ```

   使用您下载的ISO镜像文件的完整路径替换 /path/to/image.iso，使用之前由 dmesg 命令给出的设备名称替换device，同时设置合理的块大小（例如：512k）替换 blocksize，这样可以加快写入进度。

   例如：如果该ISO镜像文件位于 /home/testuser/Downloads/openEuler-21.09-aarch64-dvd.iso，同时探测到的设备名称为sdb，则该命令如下：

   ```sh
   # dd if=/home/testuser/Downloads/openEuler-21.09-aarch64-dvd.iso of=/dev/sdb bs=512k
   ```

5. 等待镜像写入完成，安全退出USB盘并拔掉。
   镜像写入过程中不会有进度显示，当#号再次出现时，执行如下命令将数据同步写入磁盘。退出root帐户，拔掉USB盘。此时，您可以使用该USB盘作为系统的安装源。

   ```bash
   # sync
   ```

### 启动安装

请根据以下步骤启动安装程序：

> \[!NOTE]说明\
> 您需要先设置您的系统优先从USB进行启动引导。以BIOS为例，您需要将“Boot Type Order”中的USB选项调整到首位。

1. 断开所有安装不需要的驱动器。
2. 打开您的计算机系统。
3. 在计算机中插入USB盘。
4. 重启计算机系统。

在短暂的延迟后会出现图形化引导页面，该页面包含不同引导选项。如果您在一分钟内未进行任何操作，安装程序将自动开始安装。

## 使用PXE通过网络安装

要使用 PXE 引导，您需要正确配置服务器以及您的计算机需支持 PXE 的网络接口。

如果目标硬件安装有支持PXE的网络接口卡，我们可以配置它从其他网络系统的文件而不是本地介质（如光盘）来引导计算机并执行Anaconda安装程序。

对于PXE网络安装，客户机通过支持PXE的网卡，向网络发送请求DHCP信息的广播，请求IP地址等信息。DHCP服务器给客户机提供一个IP地址和其他网络信息如域名服务器、ftp服务器（它提供启动安装程序所必需的文件）的IP地址或主机名，以及服务器上文件的位置。

> \[!NOTE]说明\
> 此处不详细讨论tftp、DHCP、http等服务器配置，相关详细配置请参考“[全自动化安装指导](./using_kickstart_for_automatic_installation.md#全自动化安装指导)”。

## 通过qcow2镜像安装

本节介绍如何使用或者制作qcow2镜像，并介绍相应的操作步骤，指导用户进行安装。

### 制作qcow2镜像

1. 安装qemu-img软件包。

   ```sh
   # dnf install -y qemu-img
   ```

2. 使用qemu-img工具的create命令，创建镜像文件，命令格式为：

   ```sh
   $ qemu-img create -f <imgFormat> -o <fileOption> <fileName> <diskSize>
   ```

   其中，各参数含义如下：

   * *imgFormat*  ：镜像格式，取值为raw, qcow2等。
   * *fileOption*  ：文件选项，用于设置镜像文件的特性，如指定后端镜像文件、压缩、加密等特性。
   * *fileName*  ：文件名称。
   * *diskSize*  ：磁盘大小，用于指定块磁盘设备的大小，支持的单位有K、M、G、T，分别代表KiB、MiB、GiB、TiB。

   例如，创建一个磁盘设备大小为32GB、格式为qcow2的镜像文件openEuler-image.qcow2，命令和回显如下：

   ```sh
   $ qemu-img create -f qcow2 openEuler-image.qcow2 32G
   Formatting 'openEuler-image.qcow2', fmt=qcow2 size=34359738368 cluster_size=65536 lazy_refcounts=off refcount_bits=16
   ```

### 启动安装

根据以下步骤启动安装程序：

1. 准备qcow2镜像文件。
2. 准备虚拟机网络。
3. 准备UEFI引导工具集EDK II。
4. 准备虚拟机XML配置文件。
5. 创建虚拟机。
6. 启动虚拟机。

各步骤详细的操作请参考《[虚拟化用户指南](../../../virtualization/virtualization_platform/virtualization/introduction_to_virtualization.md)》。

## 通过私有镜像安装

本节介绍如何使用或者制作私有镜像，并介绍相应的操作步骤，指导用户进行安装。

### 制作私有镜像

制作私有镜像的方法请参见《[镜像服务用户指南](https://support.huaweicloud.com/usermanual-ims/zh-cn_topic_0013901628.html)》。

### 启动安装

华为公有云的x86虚拟化平台的启动请参见《[弹性云服务器 ECS的用户指南](https://support.huaweicloud.com/wtsnew-ecs/index.html)》。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/server/installation_upgrade/installation/installation_modes_1.md
---
# 安装方式介绍

> !\[!NOTE]说明
>
> * 硬件仅支持树莓派 3B/3B+/4B/400。
> * 采用刷写镜像到 SD 卡方式安装。本章节提供 Windows/Linux/Mac 上刷写镜像的操作方法。
> * 本章节使用的镜像是参考“[安装准备](./installation_preparations_1.md)”获取 openEuler 的树莓派版本镜像。

## Windows 下刷写镜像

本节以 Windows 10 为例，介绍如何在 Windows 环境下将镜像刷写到 SD 卡。

### 格式化 SD 卡

请按照以下步骤格式化 SD 卡：

1. 下载并安装格式化 SD 卡工具，以下操作以 SD Card Formatter 格式化工具为例。

2. 打开 SD Card Formatter，在 “Select card” 中选择需要格式化的 SD 卡的盘符。

   若 SD 卡之前未安装过镜像，盘符只有一个。在 “Select card” 中选择需要格式化的 SD 卡对应盘符。

   若 SD 卡之前安装过镜像，盘符会有一个或多个。例如，SD 卡对应三个盘符：E、G、H。在 “Select card” 中选择需要格式化的 SD 卡对应 boot 分区盘符 E。

3. 在 “Formatting options” 中选择格式化方式。默认为 “Quick format”。

4. 单击“Format”开始格式化。界面通过进度条显示格式化进度。

5. 格式化完成后会弹出 “Formatting was successfully completed” 的提示框，单击“OK”完成格式化。

### 写入 SD 卡

> \[!NOTE]说明\
> 如果获取的是压缩后的镜像文件“openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz”，需要先将压缩文件解压得到 “openEuler-22.03-LTS-SP4-raspi-aarch64.img”镜像文件。

请按照以下步骤将“openEuler-22.03-LTS-SP4-raspi-aarch64.img”镜像文件写入 SD 卡：

1. 下载并安装刷写镜像的工具，以下操作以 Win32 Disk Imager 工具为例。
2. 右键选择“以管理员身份运行”，打开 Win32 Disk Imager。
3. 在“映像文件”中选择 img 格式的镜像文件路径。
4. 在“设备”中选择待写入的 SD 卡盘符。
5. 单击“写入”。界面通过任务进度条显示写入 SD 卡的进度。
6. 写入完成后会弹出 “写入成功” 的提示框，单击“OK”完成写入。

## Linux 下刷写镜像

本节介绍如何在 Linux 环境下将镜像刷写到SD卡。

### 查看磁盘分区信息

在 root 权限下执行 `fdisk -l` 获取 SD 卡磁盘信息，例如 SD 卡对应磁盘为 /dev/sdb。

### 卸载 SD 卡挂载点

1. 执行 `df -lh` 命令查看当前已挂载的卷。
2. 如果 SD 卡对应的分区未挂载，则跳过该步骤；如果 SD 卡对应分区已挂载，如 SD 卡对应的两个分区 /dev/sdb1 和 /dev/sdb3 已挂载，则需要卸载对应分区，在 root 权限下执行以下命令：

   `umount /dev/sdb1`

   `umount /dev/sdb3`

### 写入 SD 卡

1. 如果获取的是压缩后的镜像，需要先执行 `xz -d openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz` 命令将压缩文件解压得到“openEuler-22.03-LTS-SP4-raspi-aarch64.img”镜像文件；否则，跳过该步骤。
2. 将镜像 `openEuler-22.03-LTS-SP4-raspi-aarch64.img` 刷写入 SD 卡，在 root 权限下执行以下命令：

   `dd bs=4M if=openEuler-22.03-LTS-SP4-raspi-aarch64.img of=/dev/sdb`

   > \[!NOTE]说明
   > 一般情况下，将块大小设置为 4M。如果写入失败或者写入的镜像无法使用，可以尝试将块大小设置为 1M 重新写入，但是设置为 1M 比较耗时。

## Mac 下刷写镜像

本节介绍如何在 Mac 环境下将镜像刷写到SD卡。

### 查看磁盘分区信息

在 root 权限下执行 `diskutil list` 获取 SD 卡磁盘信息，例如 SD 卡对应磁盘为 /dev/disk3。

### 卸载 SD 卡挂载点

1. 执行 `df -lh` 命令查看当前已挂载的卷。
2. 如果 SD 卡对应的分区未挂载，则跳过该步骤；如果 SD 卡对应分区已挂载，如 SD 卡对应的两个分区 /dev/disk3s1 和 /dev/disk3s3 已挂载，则需要卸载对应分区，在 root 权限下执行以下命令：

   `diskutil umount /dev/disk3s1`

   `diskutil umount /dev/disk3s3`

### 写入 SD 卡

1. 如果获取的是压缩后的镜像，需要先执行 `xz -d openEuler-22.03-LTS-SP4-raspi-aarch64.img.xz` 命令将压缩文件解压得到“openEuler-22.03-LTS-SP4-raspi-aarch64.img”镜像文件；否则，跳过该步骤。
2. 将镜像 `openEuler-22.03-LTS-SP4-raspi-aarch64.img` 刷入 SD 卡，在 root 权限下执行以下命令：

   `dd bs=4m if=openEuler-22.03-LTS-SP4-raspi-aarch64.img of=/dev/sdb`

   > \[!NOTE]说明
   > 一般情况下，将块大小设置为 4m。如果写入失败或者写入的镜像无法使用，可以尝试将块大小设置为 1m 重新写入，但是设置为 1m 比较耗时。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/virtualization_installation.md
---
# 安装虚拟化组件

本章介绍在openEuler中安装虚拟化组件的方法。

## 最低硬件要求

在openEuler系统中安装虚拟化组件，最低硬件要求：

* AArch64处理器架构：ARMv8以上并且支持虚拟化扩展
* x86\_64处理器架构：支持VT-x
* 2核CPU
* 4GB的内存
* 16GB可用磁盘空间

## 安装虚拟化核心组件

### 安装方法

#### 前提条件

* 已经配置yum源。配置方式请参见《openEuler 22.03 LTS SP4 管理员指南》。
* 安装操作需要root用户权限。

#### 安装步骤

1. 安装QEMU组件。

   ```shell
   # yum install -y qemu
   ```

2. 安装libvirt组件。

   ```shell
   # yum install -y libvirt
   ```

3. 启动libvirtd服务。

   ```shell
   # systemctl start libvirtd
   ```

> \[!NOTE]说明\
> KVM模块已经集成在openEuler内核中，因此不需要单独安装。

### 验证安装是否成功

1. 查看内核是否支持KVM虚拟化，即查看/dev/kvm和/sys/module/kvm文件是否存在，命令和回显如下：

   ```shell
   $ ls /dev/kvm
   /dev/kvm
   ```

   ```shell
   $ ls /sys/module/kvm
   parameters  uevent
   ```

   若上述文件存在，说明内核支持KVM虚拟化。若上述文件不存在，则说明系统内核编译时未开启KVM虚拟化，需要更换支持KVM虚拟化的Linux内核。

2. 确认QEMU是否安装成功。若安装成功则可以看到QEMU软件包信息，命令和回显如下：

   ```shell
   $ rpm -qi qemu
   Name        : qemu
   Epoch       : 10
   Version     : 6.2.0
   Release     : 76.oe2203SP4
   Architecture: aarch64
   Install Date: Tue 15 Aug 2023 09:04:47 PM CST
   Group       : Unspecified
   Size        : 26733299
   License     : GPLv2 and BSD and MIT and CC-BY-SA-4.0
   Signature   : RSA/SHA256, Tue 01 Aug 2023 09:28:19 PM CST, Key ID 007fb747fb37bc6f
   Source RPM  : qemu-6.2.0-76.oe2203SP4.src.rpm
   Build Date  : Tue 01 Aug 2023 09:24:00 PM CST
   Build Host  : localhost
   Relocations : (not relocatable)
   URL         : http://www.qemu.org
   Summary     : QEMU is a generic and open source machine emulator and virtualizer
   Description :
   QEMU is a generic and open source processor emulator which achieves a good
   emulation speed by using dynamic translation. QEMU has two operating modes:

    * Full system emulation. In this mode, QEMU emulates a full system (for
      example a PC), including a processor and various peripherals. It can be
      used to launch different Operating Systems without rebooting the PC or
      to debug system code.
    * User mode emulation. In this mode, QEMU can launch Linux processes compiled
      for one CPU on another CPU.

   As QEMU requires no host kernel patches to run, it is safe and easy to use.
   ```

3. 确认libvirt是否安装成功。若安装成功则可以看到libvirt软件包信息，命令和回显如下：

   ```shell
   $ rpm -qi libvirt
   Name        : libvirt
   Version     : 6.2.0
   Release     : 57.oe2203SP4
   Architecture: aarch64
   Install Date: Tue 30 Jul 2023 04:56:21 PM CST
   Group       : Unspecified
   Size        : 0
   License     : LGPLv2+
   Signature   : RSA/SHA256, Tue 01 Aug 2023 09:28:19 PM CST, Key ID 007fb747fb37bc6f
   Source RPM  : libvirt-6.2.0-57.oe2203SP4.src.rpm
   Build Date  : Tue 01 Aug 2023 09:24:00 PM CST
   Build Host  : 71e8c1ce149f
   Relocations : (not relocatable)
   URL         : https://libvirt.org/
   Summary     : Library providing a simple virtualization API
   Description :
   Libvirt is a C toolkit to interact with the virtualization capabilities
   of recent versions of Linux (and other OSes). The main package includes
   the libvirtd server exporting the virtualization support.
   ```

4. 查看libvirt服务是否启动成功。若服务处于“active”状态，说明服务启动成功，可以正常使用libvirt提供的virsh命令行工具，命令和回显如下：

   ```shell
   $ systemctl status libvirtd
   ● libvirtd.service - Virtualization daemon
      Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
      Active: active (running) since Tue 2019-08-06 09:36:01 CST; 5h 12min ago
        Docs: man:libvirtd(8)
              https://libvirt.org
    Main PID: 40754 (libvirtd)
       Tasks: 20 (limit: 32768)
      Memory: 198.6M
      CGroup: /system.slice/libvirtd.service
              ─40754 /usr/sbin/libvirtd

   ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/installation_and_configuration_3.md
---
# 安装配置

本章节主要介绍和开源容器Docker安装相关的重要配置。

## 注意事项

* Docker容器的安装需要使用root权限。
* docker-engine rpm包与containerd rpm包、runc rpm包、podman rpm包不能同时安装。因为docker-engine rpm包中已经包含Docker运行所需的所有组件，其中包括containerd、runc、docker二进制，且containerd、runc和podman rpm包也分别提供了对应的二进制，所以重复安装时会出现软件包冲突。

## 基本安装配置

### 配置daemon参数

可以通过在/etc/docker/daemon.json文件中添加配置项自定义配置参数，相关配置项以及如何使用可以通过dockerd --help查看。配置示例如下：

```sh
cat /etc/docker/daemon.json 
{        
    "debug": true,        
    "storage-driver": "overlay2",        
    "storage-opts": ["overlay2.override_kernel_check=true"] 
}
```

### daemon运行目录配置

用户需要明白重新指定各种运行目录和文件（包括--graph、--exec-root等），可能会存在目录冲突，或文件属性变换，对应用的正常使用造成影响。

> \[!TIP]须知
>
> 用户指定的目录或文件应为docker专用，避免冲突导致的文件属性变化带来安全问题。

* 以--graph为例，当我们使用/new/path/作为daemon新的Root Dir时，如果/new/path/下已经存在文件，且目录或文件名与docker需要使用的目录或文件名冲突（例如： containers、hooks、tmp等目录）时，docker可能会更新原有目录或文件的属性，包括属主、权限等为自己的属主和权限。

> \[!TIP]须知
>
> 从docker-17.05开始，--graph参数被标记为Deprecated，用新的参数--data-root替代。

### daemon自带网络配置

* Docker daemon使用--bip参数指定docker0网桥的网段之后，如果在下一次重启的时候去掉--bip参数，docker0网桥会沿用上一次的--bip配置，即使重启之前已经删除docker0网桥。原因是docker会保存网络配置并在下一次重启的时候默认恢复上一次配置。
* Docker network create 并发创建网络的时候，可以创建具有相同名字的两个网络。原因是docker network是通过id来区分的，name只是个便于识别的别名而已，不保证唯一性。
* Docker在桥接bridge网络模式下，Docker容器是通过宿主机上的NAT模式，建立与宿主机之外世界的通信。Docker Daemon在启动一个容器时，每在宿主机上映射一个端口都会启动一个docker-proxy进程来实现访问代理。建议用户在使用这种userland-proxy时，只映射必须的端口，减少docker-proxy进行端口映射所消耗的资源。

### daemon-umask配置

容器主进程和exec进程的默认umask为0022，为了满足安全性需求，避免容器受到攻击，修改runc的实现，将默认umask修改为0027。修改后others群组将无法访问新建文件或目录。

docker启动容器时的默认umask值为0027，可以在dockerd启动时，使用--exec-opt native.umask=normal参数将容器启动时的umask修改为0022。

> \[!TIP]须知
>
> 如果docker create/run也配置了native.umask参数，则以docker create/run中的配置为准。

详细的配置见[docker create](./container_management_2.md#create)和[docker run](./container_management_2.md#run)章节的参数说明。

### daemon启动时间

Docker服务由systemd管理，systemd对各个服务的启动时间有限制，如果指定时间内docker服务未能成功启动，则可能由以下原因导致：

* 如果使用devicemapper且为第一次启动，docker daemon需要对该设备做文件系统初始化操作，而该操作会进行大量磁盘IO操作，在磁盘性能不佳或存在大量IO竞争时，很可能会导致docker daemon启动超时。devicemapper设备只需要初始化一次，后续docker daemon启动时不再需要重复初始化。
* 如果当前系统资源占用太高，导致系统卡顿，系统所有的操作都会变慢，也可能会出现docker服务启动超时的情况。
* daemon重启过程中，需要遍历并读取docker工作目录下每一个容器的配置文件、容器init层和可写层的配置，如果当前系统存在过多容器（包含created和exited的容器），并且磁盘读写性能受限，也会出现daemon遍历文件过久导致docker服务启动超时的情况。

出现服务启动超时情况，建议对以下两种情况进行排查调整：

* 容器编排层定期清理不需要的容器，尤其是exited的容器。
* 结合解决方案的性能要求场景，调整编排层的清理周期和docker服务的启动时间。

### 关联组件journald

重启systemd-journald后需要重启docker daemon。journald通过pipe获取docker daemon的日志，如果journald服务重启，会导致该pipe被关闭，docker的日志写入操作便会触发SIGPIPE信号，该错误信号会导致docker daemon crash。由于忽略该信号影响严重，可能导致后续docker daemon的日志无法记录，因此建议用户在重启journald服务或者journald 异常后主动去重启docker daemon，保证docker日志能够被正常记录，避免daemon crash导致的状态异常。

### 关联组件firewalld

需要在重启或拉起firewalld之后重启docker服务，保证docker服务在firewalld之后启动。

* firewalld服务启动会清空当前系统的iptables规则，所以在启动docker daemon过程中，重启firewalld可能会导致docker服务插入iptables规则失败，从而导致docker服务启动失败。
* docker服务启动后重启firewalld服务，或者状态发生了变化（从启动到停止，或者从停止到启动），会导致docker的iptables规则被删除，创建带端口映射的容器失败。

### 关联组件iptables

docker使用--icc=false选项时，可以限制容器之间互通，但若os自带某些规则，可以造成限制容器之间互通失效，例如：

```text
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) 
... 
0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0 
... 
0     0 DROP       all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
...
```

在Chain FORWARD中，DROP上面多出了一条ACCEPT icmp的规则，造成加了--icc=false后，容器之间也能ping通，但容器之间如果使用udp/tcp协议，对端仍然是不可达的。

因此，在容器os中使用docker，如果需要使用--icc=false选项时，建议先在host上清理一下iptables相关的规则。

### 关联组件audit

docker支持配置audit，但不是强制的。例如：

```text
-w /var/lib/docker -k docker 
-w /etc/docker -k docker 
-w /usr/lib/systemd/system/docker.service -k docker 
-w /usr/lib/systemd/system/docker.socket -k docker 
-w /etc/sysconfig/docker -k docker 
-w /usr/bin/docker-containerd -k docker 
-w /usr/bin/docker-runc -k docker 
-w /etc/docker/daemon.json -k docker
```

配置docker的audit，好处在于可以记录更多信息便于审计，但从安全角度来看，它对防攻击并没有实质性的作用。另一方面，audit配置会导致严重的效率问题，可能导致系统卡顿，生产环境中请谨慎使用。

下面以“-w /var/lib/docker -k docker”为例，演示docker audit的配置：

```sh
[root@localhost signal]# cat /etc/audit/rules.d/audit.rules | grep docker -w /var/lib/docker/  -k docker 
[root@localhost signal]# auditctl -R /etc/audit/rules.d/audit.rules | grep docker 
[root@localhost signal]# auditctl -l | grep docker -w /var/lib/docker/ -p rwxa -k docker
```

> \[!NOTE]说明
>
> -p \[r|w|x|a] 和-w一起使用，观察用户对这个目录的读、写、执行或者属性变化（如时间戳变化）。这样的话，在/var/lib/docker目录下的任何文件、目录操作，都会打印日志到audit.log中，从而会有太多的日志往audit.log中记录，会严重地影响auditd， 比如内存、cpu占用等，进而影响os的运行。例如：每次执行"ls /var/lib/docker/containers"都会有类似如下日志记录到/var/log/audit/audit.log中。

```text
type=SYSCALL msg=audit(1517656451.457:8097): arch=c000003e syscall=257 success=yes exit=3 a0=ffffffffffffff9c a1=1b955b0 a2=90800 a3=0 items=1 ppid=17821 pid=1925 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts6 ses=4 comm="ls" exe="/usr/bin/ls" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="docker"type=CWD msg=audit(1517656451.457:8097):  cwd="/root"type=PATH msg=audit(1517656451.457:8097): item=0 name="/var/lib/docker/containers" inode=1049112 dev=fd:00 mode=040700 ouid=0 ogid=0 rdev=00:00 obj=unconfined_u:object_r:container_var_lib_t:s0 objtype=NORMAL 
```

### 安全配置seccomp

在做容器网络性能测试时发现，Docker相对于原生内核namespace性能有所下降，经分析开启seccomp后，系统调用（如：sendto）不会通过system\_call\_fastpath进行，而是调用tracesys，这会带来性能大幅下降。因此，建议在有高性能要求的业务的容器场景下关闭seccomp，示例如下：

```sh
docker run -itd --security-opt seccomp=unconfined busybox:latest
```

### 禁止修改docker-daemon的私有目录

不允许对Docker用的根目录（默认/var/lib/docker）和运行时目录（默认/run/docker）以及其文件作任何修改，包括在该目录下删除文件，添加文件，对目录或者文件做软/硬链接，修改文件的属性/权限，修改文件的内容等，如果确实需要做修改，后果自负。

### 普通用户大量部署容器场景下的配置注意事项

普通用户在OS主机上能创建的进程数的上限，例如：可以在系统中创建配置文件“/etc/security/limits.d/20-nproc.conf”限制；类似的，普通用户在容器里也能创建的进程数的上限，由容器镜像中“/etc/security/limits.d/20-nproc.conf”文件对应的值决定，如下所示：

```sh
cat /etc/security/limits.conf 
*       soft    nproc   4096
```

当普通用户大量部署容器，导致容器内进程过多资源不够出现报错时，需要把容器镜像“/etc/security/limits.d/20-nproc.conf”文件中如上所示的4096配置值加大。

可配置的最大值请参考内核的最大能力，如下：

```sh
[root@localhost ~]# sysctl -a | grep pid_max 
kernel.pid_max = 32768
```

## 存储驱动配置

本发行版docker支持overlay2和devicemapper两种存储驱动。由于overlay2较devicemapper而言，拥有更好的性能，建议用户在生产环境中优先考虑。

### 配置overlay2存储驱动

#### 配置方法

docker默认为使用overlay2存储驱动，也可以通过如下两种方式显式指定。

* 编辑/etc/docker/daemon.json，通过storage-driver字段显式指定。

  ```sh
  cat /etc/docker/daemon.json
  {
      "storage-driver": "overlay2"
  }
  ```

* 编辑/etc/sysconfig/docker-storage，通过docker daemon启动参数显式指定。

  ```sh
  cat /etc/sysconfig/docker-storage 
  DOCKER_STORAGE_OPTIONS="--storage-driver=overlay2"
  ```

#### 注意事项

* 部分容器生命周期管理的操作会报找不到相应的rootfs或者相关的可执行文件。

* 如果容器的健康检查配置的是执行容器内的可执行文件，也会报错，导致容器的健康检查失败。

* 如果将overlay2作为graphdriver，在容器中第一次修改镜像中的文件时，若该文件的大小大于系统剩余的空间，修改将会失败。因为即使修改很小，也要把这个文件完整的拷贝到上层，剩余空间不足导致失败。

* overlay2文件系统相比普通文件系统天然存在一些行为差异，归纳如下：
  * 内核版本

    overlay2只兼容原生4.0以上内核，建议配合使用ext4文件系统。

  * Copy-UP性能问题

    修改lower层文件会触发文件复制到upper层，其中数据块复制和fsync比较耗时。

  * rename目录问题
    * 只有源路径和目标路径都在merged层时，才允许rename系统调用，否则rename系统调用会报错-EXDEV。
    * 内核4.10引入了redirect dir特性来修复rename问题，对应内核选项为CONFIG\_OVERLAY\_FS\_REDIRECT\_DIR。

      在使用overlay2场景下，对文件系统目录进行重命名时，如果系统配置文件/sys/module/overlay/parameters/redirect\_dir中配置的特性开关为关闭状态，则会导致使用失败；如果用户要使用相关特性，需要用户手动设置/sys/module/overlay/parameters/redirect\_dir为“Y”。

  * Hard link break问题
    * 当lower层目录中有多个硬链接，在merged层写入数据会触发Copy-UP，导致硬链接断开。
    * 内核4.13引入了index feature来修复这个问题，对应内核选项为 CONFIG\_OVERLAY\_FS\_INDEX。注意这个选项没有前向兼容性，不支持热升级。

  * st\_dev和st\_ino变化

    触发Copy-UP之后，用户只能看到merged层中的新文件，inode会变化。虽然attr和xattr可以复制，但st\_dev和st\_ino具有唯一性，不可复制。这会导致stat和ls查看 到相应的变化。

  * fd变化

    Copy-UP之前，以只读模式打开文件得到描述符fd1，Copy-UP之后，打开同名文件得到文件描述符fd2， 二者实际指向不同的文件。向fd2写入的数据不会在fd1中体现。

#### 异常场景

容器使用配置了overlay2存储驱动的过程中，可能出现挂载点被覆盖的异常情况。例如

#### 异常场景-挂载点被覆盖

挂载关系：在问题容器的挂载点的下面，存在一个/var/lib/docker/overlay2的挂载点：

```sh
[root@localhost ~]# mount -l | grep overlay 
overlay on /var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/merged type overlay (rw,relatime,seclabel,lowerdir=/var/lib/docker/overlay2/l/JL5PZQLNDCIBU3ZOG3LPPDBHIJ:/var/lib/docker/overlay2/l/ELRPYU4JJG4FDPRLZJCZZE4UO6,upperdir=/var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/diff,workdir=/var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/work) 
/dev/mapper/dm-root on /var/lib/docker/overlay2 type ext4 (rw,relatime,seclabel,data=ordered)
```

执行部分docker命令会遇到错误，比如：

```sh
[root@localhost ~]# docker rm 1348136d32
docker rm: Error response from daemon: driver "overlay2" failed to remove root filesystem for 1348136d32: error while removing /var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785: invalid argument
```

此时，在主机侧可以发现对应容器的rootfs找不到，但这并不意味着rootfs丢失，只是被/var/lib/docker/overlay2挂载点覆盖，业务仍然可以正常运行，不受影响。修复方案可以参考如下：

* 修复方案一
  1. 确定当前docker所使用graphdriver：

     ```sh
     docker info | grep "Storage Driver"
     ```

  2. 查询当前的挂载点：

     ```text
     Devicemapper: mount -l | grep devicemapper 
     Overlay2: mount -l | grep overlay2
     ```

     输出格式为： A on B type C (D)

     其中，A：块设备名称或overlay，B：挂载点，C：文件系统类型，D：挂载属性。

  3. 从下往上逐一umount这些挂载点B。

  4. 然后全部docker restart这些容器，或者删除所有容器。

  5. 重启docker。

     ```sh
     systemctl restart docker
     ```

* 修复方案二
  1. 业务迁移
  2. 节点重启

### 配置devicemapper存储驱动

用户如果需要使用devicemapper存储驱动，可以通过如下两种方式显式指定。

* 编辑/etc/docker/daemon.json，通过storage-driver字段显式指定。

  ```sh
  cat /etc/docker/daemon.json
  {
      "storage-driver": "devicemapper"
  }
  ```

* 编辑/etc/sysconfig/docker-storage，通过docker daemon启动参数显式指定。

  ```sh
  cat /etc/sysconfig/docker-storage 
  DOCKER_STORAGE_OPTIONS="--storage-driver=devicemapper"
  ```

#### 注意事项

* 使用devicemapper必须使用devicemapper+direct-lvm的方式。

* 配置devicemapper时，如果系统上没有足够的空间给thinpool做自动扩容，请禁止自动扩容功能。

* 禁止把/etc/lvm/profile/docker-thinpool.profile中如下两个值都改成100。

  ```text
  activation {   
    thin_pool_autoextend_threshold=80   
    thin_pool_autoextend_percent=20 
  }
  ```

* 使用devicemapper时推荐加上--storage-opt dm.use\_deferred\_deletion=true --storage-opt dm.use\_deferred\_removal=true。

* 使用devicemapper时，容器文件系统推荐使用ext4，需要在docker daemon的配置参数中加 上--storage-opt dm.fs=ext4。

* 当graphdriver为devicemapper时，如果metadata文件损坏且不可恢复，需要人工介入恢复。禁止直接操作或篡改daemon存储devicemapper的元数据。

* 使用devicemapper lvm时，异常掉电导致的devicemapper thinpool损坏，无法保证thinpool损坏后可以修复，也不能保证数据的完整性，需重建thinpool。

docker daemon开启了user namespace特性，切换devicemapper存储池时的**注意事项**

* 一般启动容器时，deviceset-metadata文件为：/var/lib/docker/devicemapper/metadata/deviceset-metadata。
* 使用了user namespace场景下，deviceset-metadata文件使用的是：/var/lib/docker/{userNSUID.GID}/devicemapper/metadata/deviceset-metadata。
* 使用devicemapper存储驱动，容器在user namespace场景和普通场景之间切换时，需要将对应deviceset-metadata文件中的BaseDeviceUUID内容清空；针对thinpool扩容或者重建的场景下，也同样的需要将对应deviceset-metadata文件中的BaseDeviceUUID内容清空，否则docker服务会重启失败。

## 强制退出docker相关后台进程的影响

### 信号量残留

使用devicemapper作为graphdriver时，强制退出强制退出可能导致信号量残留。docker在操作dm的过程中会创建信号量，如果在释放信号量前，daemon被强制退出，可能导致该信号量无法释放，一次强制退出最多泄露一个信号量，泄露概率低。而linux系统有信号量上限限制，当信号量泄露次数达到上限值时将无法创建新的信号量，进而导致docker daemon启动失败。排查方法如下：

1. 首先查看系统上残留的信号量

   ```sh
   $ ipcs 
   ------ Message Queues -------- 
   key        msqid      owner      perms      used-bytes   messages 
   ------ Shared Memory Segments -------- 
   key        shmid      owner      perms      bytes      nattch     status 
   ------ Semaphore Arrays -------- 
   key        semid      owner      perms      nsems 
   0x0d4d3358 238977024  root       600        1 
   0x0d4d0ec9 270172161  root       600        1 
   0x0d4dc02e 281640962  root       600        1
   ```

2. 接着用dmsetup查看devicemapper创建的信号量，该信号量集合是上一步中查看到的系统信号量的子集

   ```sh
   dmsetup udevcookies 
   ```

3. 最后查看内核信号量设置上限，第四个值就是当前系统的信号量使用上限

   ```sh
   # cat /proc/sys/kernel/sem 
   250     32000   32      128
   ```

   如果步骤1中残留的信号量数量与步骤3中看到的信号量上限相等，则是达到上限，此时docker daemon无法正常启动。可以使用下述命令增加信号量使用上限值来让docker恢复启动

   ```sh
   echo 250 32000  32  1024 > /proc/sys/kernel/sem
   ```

   也可以手动清理devicemapper残留的信号量（下面是清理一分钟以前申请的dm相关信号量）

   ```sh
   # dmsetup udevcomplete_all 1 
   This operation will destroy all semaphores older than 1 minutes with keys that have a prefix 3405 (0xd4d). 
   Do you really want to continue? [y/n]: y 
   0 semaphores with keys prefixed by 3405 (0xd4d) destroyed. 0 skipped.
   ```

### 网卡残留

使用bridge模式启动容器的过程中，强制退出daemon可能导致网卡残留。使用bridge网络模式，当docker创建容器时，会先在host上创建一对veth，然后再把该网卡信息存到数据库中，如果在创建完成，存到docker的数据库之前，daemon被强制退出，那么该网卡无法被docker关联，下次启动也无法删除（docker本身会清理自己数据库中不用的网卡），从而造成网卡残留。

### 重启容器失败

容器hook耗时较长，且启动阶段遇到containerd被强制退出，再次执行容器start操作可能失败。容器启动阶段遇到containerd被强制退出，docker start操作直接返回错误；containerd被重新拉起后，上次启动可能仍处于runc create执行阶段（执行用户自定义hook，可能耗时较长），此时再次下发docker start命令启动该容器，可能提示以下错误：

```text
Error response from daemon: oci runtime error: container with id exists: xxxxxx
```

该错误是由runc create一个已经存在（创建中）的容器导致，等第一次start对应的runc操作结束后再次执行docker start便可以成功。

由于hook的执行不受docker控制，这种场景下尝试回收该容器有可能导致containerd进程启动卡死（执行未知hook程序），且问题的风险可控（短期影响当前容器的创建）：

* 问题出现后等待第一次操作结束可以再次成功启动该容器。
* 一般是在容器启动失败后创建新的容器，不复用已经失败的容器。

综上，该问题暂时作为场景约束。

### 服务无法正常重启

短时间内频繁重启docker服务导致该服务无法正常重启。docker系统服务由systemd负责监控，如果docker服务在10s内重启次数超过5次，systemd服务就会监控到该异常行为，因此会禁止docker服务启动。只有等到下一个10s周期开始后，docker服务才能响应重启命令正常重启。

## 系统掉电影响

主机意外掉电或系统panic等场景下，由于docker daemon的状态无法及时刷新到磁盘，导致重启后docker daemon状态不正常，可能出现的问题有（包括但不限于）：

* 掉电前创建的容器，重启后docker ps -a看不到，该问题是因为该容器的状态文件没有刷新到磁盘，从而导致重启后daemon无法获取到该容器的状态（镜像、卷、网络等也可能会有类似问题）。
* 掉电前某个文件正处于写入状态，尚未完全写入，重启后daemon重新加载该文件发现文件格式不正常或内容不完整，导致重启加载出错。
* 针对掉电时会破坏docker DB的情况，在重启节点时会清理data-root下面的db文件。因此重启前创建的如下信息在重启后会被删除：
  * network，用docker network创建的资源会在重启后清除。
  * volume，用 docker volume创建的资源会在重启后删除。
  * 构建缓存，构建缓存信息会在重启后删除。
  * containerd保存的元数据，由于启动容器会重建containerd元数据，重启节点会清理containerd中保存的元数据。

    > \[!NOTE]说明
    >
    > 用户若选择采用手动清理恢复环境的方式，可通过配置环境变量“DISABLE\_CRASH\_FILES\_DELETE=true”屏蔽daemon掉电重启时db文件清理功能。

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/kubeos/kubeos/overview.md
---
# 容器OS升级指南

本文档介绍基于openEuler系统的容器OS升级特性的安装部署和使用方法，容器OS升级是使OS可以通过标准扩展方式接入调度系统，通过调度系统管理集群内节点的OS的升级。

本文档适用于使用openEuler系统并希望了解和使用容器OS的社区开发者、开源爱好者以及相关合作伙伴。使用人员需要具备以下经验和技能：

* 熟悉Linux基本操作。
* 对kubernetes和docker有一定了解。

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/kubeos/kubeos/kubeos_image_creation.md
---
# 容器OS镜像制作指导

## 简介

kbimg是KubeOS部署和升级所需的镜像制作工具，可以使用kbimg制作KubeOS docker，虚拟机和物理机镜像。

## 命令介绍

### 命令格式

kbimg - CLI tool for generating various types of image for KubeOS

```text
Usage: kbimg [OPTIONS] <COMMAND> 

Commands:
  create  Create a new KubeOS image
  help    Print this message or the help of the given subcommand(s)

Options:
  -d, --debug    Enable debug mode, generate the scripts without execution
  -h, --help     Print help
  -V, --version  Print version
```

kbimg-create - Create a new KubeOS image

```text
Usage: kbimg create --file <FILE> <IMAGE_TYPE>

Arguments:
  <IMAGE_TYPE>  [possible values: vm-img, pxe-img, upgrade-img, admin-container]

Options:
  -f, --file <FILE>  Path to the toml configuration file
  -h, --help         Print help
```

### 配置文件说明

#### from\_repo

从 repo 创建升级容器镜像、虚拟机镜像或PXE物理机镜像

| 参数 | 描述 |
| --- | --- |
| agent\_path | os-agent 二进制的路径 |
| legacy\_bios | 目前仅支持设置为`false`，即UEFI引导 |
| repo\_path | repo 文件的路径，repo 文件中配置制作镜像所需要的 yum 源 |
| root\_passwd | root 用户密码，与/etc/shadow文件内密码格式一致，可使用`openssl passwd -6 -salt $(head -c18 /dev/urandom \| openssl base64)`命令生成 |
| version | KubeOS 镜像的版本，将写入/etc/os-release文件内作为OS标识 |
| rpmlist | 期望安装进镜像内的rpm包列表 |
| upgrade\_img | \[可选项]指定生成的升级容器镜像的镜像名(制作升级容器镜像必需) |

#### admin\_container

制作admin运维容器

| 参数 | 描述 |
| --- | --- |
| hostshell | hostshell二进制路径，可在项目根目录下通过`make hostshell`编译 |
| img\_name | 指定生成的容器镜像名 |

#### pxe\_config

在制作PXE物理机镜像时，配置该参数用于PXE安装。制作PXE物理机镜像时必需。

| 参数 | 描述 |
| --- | --- |
| server\_ip | 用于下载根文件系统 tar 包的 HTTP 服务器地址 |
| rootfs\_name | 放置于 HTTP 服务器的文件系统 tar 包名称 |
| disk | 安装 KubeOS 系统的目标磁盘名 |
| route\_ip | 配置目标机器网卡的路由 IP |
| dhcp | \[可选项] 是否启用 DHCP 模式配置网络，默认为 false |
| local\_ip | \[可选项] 配置目标机器网卡的 IP，dhcp 为 false 时必需 |
| net\_name | \[可选项] 配置目标机器网卡名，dhcp 为 false 时必需 |
| netmask | \[可选项] 配置目标机器网卡的子网掩码，dhcp 为 false 时必需 |

**注意**：`pxe_config`下的配置参数无法进行校验，需要用户自行确认其正确性。

#### users

\[可选项] 添加用户

| 参数 | 描述 |
| --- | --- |
| name | 用户名 |
| passwd | 密码 |
| primary\_groups | \[可选项] 用户主组(默认为用户同名组) |
| groups | \[可选项] 用户附加组 |

**注意**：添加用户会默认创建用户同名组，配置用户附加组时，若组不存在会报错失败。若有特殊配置需求，用户可通过[chroot\_script](#chroot_script)脚本自行实现。

#### copy\_files

\[可选项] 拷贝文件到rootfs内指定目录

| 参数 | 描述 |
| --- | --- |
| dst | 目标路径 |
| src | 源文件路径 |
| create\_dir | \[可选项]拷贝前创建文件夹 |

**注意**：拷贝文件无法保留权限，如果需要特殊权限，可借助[chroot\_script](#chroot_script)脚本自行实现。

#### grub

\[可选项] grub配置，配置dm-verity时必需

| 参数 | 描述 |
| --- | --- |
| passwd | grub 明文密码 |

#### systemd\_service

\[可选项] 配置 systemd 服务开机自启

| 参数 | 描述 |
| --- | --- |
| name | systemd 服务名 |

#### chroot\_script

\[可选项] 自定义 chroot 脚本

| 参数 | 描述 |
| --- | --- |
| path | 脚本路径 |
| rm | \[可选项]执行完毕后是否删除该脚本，配置`true`删除，`false`或空保留 |

#### disk\_partition

\[可选项] 自定义分区大小和镜像大小

| 参数 | 描述 |
| --- | --- |
| root | root分区大小, 单位为MiB，默认2560MiB |
| img\_size | \[可选项]镜像大小，单位为GB，默认20GB |

#### persist\_mkdir

\[可选项] persist 分区新建目录

| 参数 | 描述 |
| --- | --- |
| name | 目录名 |

#### dm\_verity

\[可选项] 制作启用dm-verity功能的虚拟机或升级镜像

| 参数 | 描述 |
| --- | --- |
| efi\_key | efi明文口令 |
| grub\_key | grub明文口令 |
| keys\_dir |\[可选项]可指定密钥文件夹，复用先前制作镜像创建的密钥  |

## 使用说明

### 注意事项

* kbimg 执行需要 root 权限。
* 当前仅支持 x86和 AArch64 架构使用。
* 不支持并发执行。如果使用脚本`&`连续执行可能会出现异常情况。制作过程中碰到异常掉电或中断后无法清理环境时，可参考[异常退出清理方法](#异常退出清理方法)清理后重新制作。
* 容器 OS 镜像制作工具的 rpm 包源为 openEuler 具体版本的 everything 仓库和 EPOL 仓库。制作镜像时提供的 repo 文件中，yum 源建议同时配置 openEuler 具体版本的 everything 仓库和 EPOL 仓库。
* dm-verity使用说明：
  * 仅支持虚拟机场景，暂不支持物理机环境。
  * 不支持通过 HTTP/HTTPS 服务器下载升级镜像进行系统升级。仅支持从容器镜像仓库下载升级镜像进行升级。
  * 启动虚拟机时，必须配置使用 virtio 类型设备。
  * 启用dm-verity功能的升级容器镜像不可用于升级未开启dm-verity的容器OS。同理，未启动dm-verity功能的升级容器镜像不可用于升级开启dm-verity功能的容器OS。在集群内，部分节点开启dm-verity功能，部分未开启，需要用户控制下发对应的升级镜像。
  * 制作升级容器镜像和虚拟机镜像时，推荐使用相同的密钥(配置`keys_dir`为先前制作镜像时创建的密钥文件路径。配置`efi_key`或`grub_key`一致不能保证密钥文件是一模一样的)。若密钥不一致，在切换备用分区时可能导致证书校验失败，从而无法启动系统。出现证书校验失败问题时，需要重新导入备用分区证书进行修复。

### KubeOS OCI 镜像制作

#### 注意事项

* 制作出的 OCI 镜像仅用于后续的虚拟机/物理机镜像升级使用，不支持启动容器。
* 使用默认 rpmlist 进行容器OS镜像制作时所需磁盘空间至少为6G，若使用自定义 rpmlist 可能会超过6G。

#### 使用示例

* 配置文件示例

```toml
[from_repo]
agent_path = "./bin/rust/release/os-agent"
legacy_bios = false
repo_path = "/etc/yum.repos.d/openEuler.repo"
root_passwd = "$1$xyz$RdLyKTL32WEvK3lg8CXID0" # default passwd: openEuler12#$
rpmlist = [
    "NetworkManager",
    "cloud-init",
    "conntrack-tools",
    "containerd",
    "containernetworking-plugins",
    "cri-tools",
    "dhcp",
    "ebtables",
    "ethtool",
    "iptables",
    "kernel",
    "kubernetes-kubeadm",
    "kubernetes-kubelet",
    "openssh-server",
    "passwd",
    "rsyslog",
    "socat",
    "tar",
    "vi",
]
upgrade_img = "kubeos-upgrade:v1"
version = "v1"
```

* 结果说明
  * 制作完成后，通过`docker images`查看制作出来的KubeOS容器镜像
  * update-boot.img/update-root.img/update-hash.img: 仅在dm-verity模式下生成，可忽略。

### KubeOS 虚拟机镜像制作

#### 注意事项

* 制作出来的容器 OS 虚拟机镜像目前只能用于 CPU 架构为 x86 和 AArch64 的虚拟机。
* 容器 OS 目前不支持 x86 架构的虚拟机使用 legacy 启动模式启动。
* 默认root密码为openEuler12#$
* 使用默认rpmlist进行容器OS镜像制作时所需磁盘空间至少为25G，若使用自定义rpmlist可能会超过25G。

#### 使用示例

* 配置文件示例

```toml
[from_repo]
agent_path = "./bin/rust/release/os-agent"
legacy_bios = false
repo_path = "/etc/yum.repos.d/openEuler.repo"
root_passwd = "$1$xyz$RdLyKTL32WEvK3lg8CXID0" # default passwd: openEuler12#$
rpmlist = [
    "NetworkManager",
    "cloud-init",
    "conntrack-tools",
    "containerd",
    "containernetworking-plugins",
    "cri-tools",
    "dhcp",
    "ebtables",
    "ethtool",
    "iptables",
    "kernel",
    "kubernetes-kubeadm",
    "kubernetes-kubelet",
    "openssh-server",
    "passwd",
    "rsyslog",
    "socat",
    "tar",
    "vi",
]
version = "v1"
```

* 结果说明
  容器 OS 镜像制作完成后，会在 ./scripts-auto 目录下生成
  * system.qcow2: 用于启动虚拟机的qcow2 格式的系统镜像，大小默认为 20GiB，支持的根文件系统分区大小 < 2560 MiB，持久化分区 < 15GB。
  * system.img: 用于启动虚拟机的img 格式的系统镜像，大小默认为 20GiB，支持的根文件系统分区大小 < 2560 MiB，持久化分区 < 15GB。
  * kubeos.tar: 用于升级的根文件系统tar包。
  * update-boot.img/update-root.img/update-hash.img: 仅在dm-verity模式下生成，可忽略。

### KubeOS 物理机安装所需镜像及文件制作

#### 注意事项

* 制作出来的容器 OS 物理安装所需的镜像目前只能用于 CPU 架构为 x86 和 AArch64 的物理机安装。
* `pxe_config`配置中指定的ip为安装时使用的临时ip,请在系统安装启动后请参考[《openEuler 24.03-LTS-SP1 管理员指南-配置网络》](../../../server/network/network_config/network_configuration.md)进行网络配置。
* 不支持多个磁盘都安装KubeOS，可能会造成启动失败或挂载紊乱。
* 容器OS 目前不支持 x86 架构的物理机使用 legacy 启动模式启动。
* 使用默认rpmlist进行镜像制作时所需磁盘空间至少为5G，如自已定义 rpmlist 可能会超过5G。
* PXE物理机镜像制作不支持dm-verity功能
* 在 PXE 安装阶段，需要从 HTTP 服务器的根目录下载根分区 tar 包（tar包名称为toml配置文件中配置的名称）。请确保机器拥有足够的内存空间以存储根分区 tar 包及临时中间文件。

#### 使用示例

* 首先需要修改`kbimg.toml`中`pxe_config`的配置，对相关参数进行配置，详细参数可见[参数说明](#pxe_config)，ip目前仅支持ipv4，配置示例如下

  ```toml
  [pxe_config]
  dhcp = false
  # rootfs file name
  rootfs_name = "kubeos.tar"
  # select the target disk to install kubeOS
  disk = "/dev/vda"
  # pxe server ip address where stores the rootfs on the http server
  server_ip = "192.168.122.50"
  # target machine ip
  local_ip = "192.168.122.100"
  # target machine route
  route_ip = "192.168.122.1"
  # target machine netmask
  netmask = "255.255.255.0"
  # target machine netDevice name
  net_name = "eth0"
  ```

* 如需进行DNS配置，请先自定义`resolv.conf`文件，并启用`copy_files`字段将配置文件拷贝到`/etc`目录

  ```toml
  [[copy_files]]
  dst = "/etc"
  src = "<path to resolv.conf>"
  ```

* KubeOS物理机安装所需镜像制作，及pxe\_config配置全示例

  ```toml
  [from_repo]
  agent_path = "./bin/rust/release/os-agent"
  legacy_bios = false
  repo_path = "/etc/yum.repos.d/openEuler.repo"
  root_passwd = "$1$xyz$RdLyKTL32WEvK3lg8CXID0" # default passwd: openEuler12#$
  rpmlist = [
      "NetworkManager",
      "cloud-init",
      "conntrack-tools",
      "containerd",
      "containernetworking-plugins",
      "cri-tools",
      "dhcp",
      "ebtables",
      "ethtool",
      "iptables",
      "kernel",
      "kubernetes-kubeadm",
      "kubernetes-kubelet",
      "openssh-server",
      "passwd",
      "rsyslog",
      "socat",
      "tar",
      "vi",
      "coreutils",
      "dosfstools",
      "dracut",
      "gawk",
      "hwinfo",
      "net-tools",
      "parted",
  ]
  version = "v1"

  [pxe_config]
  dhcp = true
  rootfs_name = "kubeos.tar"
  disk = "/dev/vda"
  server_ip = "192.168.122.50"
  route_ip = "192.168.122.1"
  #local_ip = "192.168.1.100"
  #netmask = "255.255.255.0"
  #net_name = "eth0"
  ```

* 结果说明
  * initramfs.img: 用于pxe启动用的 initramfs 镜像
  * kubeos.tar: pxe安装所用的根分区文件系统

## 附录

### 异常退出清理方法

1.若在使用`kbimg`制作镜像过程中，异常退出，无法清理环境，可使用如下方法进行清理：

```bash
function unmount_dir() {
  local dir=$1
  if [ -L "${dir}" ] || [ -f "${dir}" ]; then
    echo "${dir} is not a directory, please check it."
    return 1
  fi
  if [ ! -d "${dir}" ]; then
    return 0
  fi
  local real_dir=$(readlink -e "${dir}")
  local mnts=$(awk '{print $2}' < /proc/mounts | grep "^${real_dir}" | sort -r)
  for m in ${mnts}; do
    echo "Unmount ${m}"
    umount -f "${m}" || true
  done
  return 0
}
ls -l ./scripts-auto/test.lock && rm -rf ./scripts-auto/test.lock
unmount_dir ./scripts-auto/rootfs/proc
unmount_dir ./scripts-auto/rootfs/sys
unmount_dir ./scripts-auto/rootfs/dev/pts
unmount_dir ./scripts-auto/rootfs/dev
unmount_dir ./scripts-auto/mnt/boot/grub2
unmount_dir ./scripts-auto/mnt
rm -rf ./scripts-auto/rootfs ./scripts-auto/mnt
```

2.如果执行以上命令仍然无法删除目录，可尝试先调用如下命令，再重新执行第一步的命令。

```bash
fuser -kvm ./scripts-auto/rootfs
fuser -kvm ./scripts-auto/mnt
```

### 详细toml配置文件示例

请根据需求和[配置文件说明](#配置文件说明)，修改如下示例配置文件，生成所需镜像。

```toml
[from_repo]
agent_path = "./bin/rust/release/os-agent"
legacy_bios = false
repo_path = "/etc/yum.repos.d/openEuler.repo"
root_passwd = "$1$xyz$RdLyKTL32WEvK3lg8CXID0" # default passwd: openEuler12#$, use "openssl passwd -6 -salt $(head -c18 /dev/urandom | openssl base64)" to generate your passwd
rpmlist = [
    "NetworkManager",
    "cloud-init",
    "conntrack-tools",
    "containerd",
    "containernetworking-plugins",
    "cri-tools",
    "dhcp",
    "ebtables",
    "ethtool",
    "iptables",
    "kernel",
    "kubernetes-kubeadm",
    "kubernetes-kubelet",
    "openssh-server",
    "passwd",
    "rsyslog",
    "socat",
    "tar",
    "vi",
    # Below packages are required for pxe-image. Uncomment them if you want to generate pxe-image.
    # "coreutils",
    # "dosfstools",
    # "dracut",
    # "gawk",
    # "hwinfo",
    # "net-tools",
    # "parted",
]
upgrade_img = "kubeos-upgrade:v1"
version = "v1"

# [admin_container]
# img_name = "kubeos-admin-container:v1"
# hostshell = "./bin/hostshell"

# [pxe_config]
# dhcp = false
# disk = "/dev/vda"
# local_ip = "192.168.1.100"
# net_name = "eth0"
# netmask = "255.255.255.0"
# rootfs_name = "kubeos.tar"
# route_ip = "192.168.1.1"
# server_ip = "192.168.1.50"

# [[users]]
# groups = ["admin", "wheel"]
# name = "foo"
# passwd = "foo"
# primary_group = "foo"

# [[users]]
# groups = ["example"]
# name = "bar"
# passwd = "bar"

# [[copy_files]]
# create_dir = "/root/test"
# dst = "/root/test/foo.txt"
# src = "/root/KubeOS/foo.txt"

# [[copy_files]]
# dst = "/etc/bar.txt"
# src = "../bar.txt"

# [grub]
# passwd = "foo"

# [systemd_service]
# name = ["containerd", "kubelet"]

# [chroot_script]
# path = "./my_chroot.sh"
# rm = true

# [disk_partition]
# img_size = 30 # GB
# root = 3000  # MiB

# [persist_mkdir]
# name = ["bar", "foo"]

# [dm_verity]
# efi_key = "foo"
# grub_key = "bar"
# keys_dir = "./keys"
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/checking_the_container_health_status.md
---
# 容器健康状态检查

## 场景说明

在实际的生产环境中，开发者提供的应用程序或者平台提供的服务难免存在bug，因此，一套管理系统对运行的应用程序进行周期性的健康检查和修复就是不可或缺的。容器健康检查机制便添加了用户定义的对容器进行健康检查的功能。在容器创建时配置--health-cmd选项，在容器内部周期性地执行命令，通过命令的返回值来监测容器的健康状态。

## 配置方法

在容器启动时的配置：

```sh
isula run -itd --health-cmd "echo iSulad >> /tmp/health_check_file || exit 1" --health-interval 5m --health-timeout 3s --health-exit-on-unhealthy  busybox bash
```

可配置的选项：

* \--health-cmd，必选，在容器内执行的命令。返回值为0表示成功，非0表示失败。
* \--health-interval，默认 30s，最大为int64上限（纳秒），自定义配置最小值1s，相邻两次命令执行的间隔时间（注：入参0s时视为default）。
* \--health-timeout，默认 30s，最大为int64上限（纳秒），自定义配置最小值1s，单次检查命令执行的时间上限，超时则任务命令执行失败（注：入参0s时视为default）。
* \--health-start-period，默认 0s，最大为int64上限（纳秒），自定义配置最小值1s，容器初始化时间。
* \--health-retries，默认 3，最大为int32上限，健康检查失败最大的重试次数。
* \--health-exit-on-unhealthy，默认false，检测到容器非健康时是否杀死容器。

## 检查规则

1. 容器启动后，容器状态中显示health:starting。
2. 经过start-period时间后开始，以interval为间隔周期性在容器中执行CMD。即：当一次命令执行完毕后，经过interval时间，执行下一次命令。
3. 若CMD命令在timeout限制的时间内执行完毕，并且返回值为0，则视为一次检查成功。否则视为一次检查失败。检查成功后，容器状态变为health:healthy。
4. 若CMD命令连续retries次检查失败，则容器状态变为health:unhealthy。失败后容器也会继续进行健康检查。
5. 容器状态为health:unhealthy时，任意一次检查成功会使得容器状态变为health:healthy。
6. 设置--exit-on-unhealthy的情况下，如果容器因为非被杀死退出（退出返回值137）后，健康检查只有容器在重新启动后才会继续生效。
7. CMD执行完毕或超时时，iSulad daemon会将这次检查的起始时间、返回值和标准输出记录到容器的配置文件中。最多记录5条。此外，容器的配置文件中还存储着健康检查的相关参数。
8. 运行中的容器的健康检查状态也会被写入容器配置中。通过isula inspect可以看到。

```conf
"Health": {
    "Status": "healthy",
    "FailingStreak": 0,
    "Log": [
        {
            "Start": "2018-03-07T07:44:15.481414707-05:00",
            "End": "2018-03-07T07:44:15.556908311-05:00",
            "ExitCode": 0,
            "Output": ""
        },
        {
            "Start": "2018-03-07T07:44:18.557297462-05:00",
            "End": "2018-03-07T07:44:18.63035891-05:00",
            "ExitCode": 0,
            "Output": ""
        },
        ......
}
```

## 使用限制

* 容器内健康检查的状态信息最多保存5条。会保存最后得到的5条记录。
* 容器启动时若健康检查相关参数配置为0，则按照缺省值处理。
* 带有健康检查配置的容器启动后，若iSulad daemon退出，则健康检查不会执行。iSulad daemon再次启动后，正在运行且带有健康检查配置的容器其健康状态会变为starting。之后检查规则同上。
* 如果健康检查从第一次开始便一直失败，其状态保持与之前一致（starting），直到达到指定失败次数（--health-retries）后变为unhealthy，或者检查成功后变为healthy。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/system_container/reboot_or_shutdown_in_a_container.md
---
# 容器内reboot/shutdown

## 功能描述

系统容器支持在容器内执行reboot和shutdown命令。执行reboot命令效果同重启容器一致；执行shutdown命令效果同停止容器一致。

## 参数说明

## 约束限制

* shutdown功能，依赖于不同的OS，以实际容器运行环境对应OS为准。
* 执行“shutdown -h now”命令关闭系统时，不能多次占用console。例如“isula run -ti”命令打开一个console，在另一个host bash中isula attach该容器，会打开另一个console，此时执行shutdown会失败。

## 使用示例

* 容器启动时指定--restart on-reboot参数，示例如下：

  ```sh
  [root@localhost ~]# isula run -tid --restart on-reboot --system-container --external-rootfs /root/myrootfs none init
  106faae22a926e22c828a0f2b63cf5c46e5d5986ea8a5b26de81390d0ed9714f
  ```

* 进入容器执行reboot命令：

  ```sh
  [root@localhost ~]# isula exec -it 10 bash
  [root@localhost /]# reboot
  ```

  查看容器是否重启：

  ```sh
  [root@localhost ~]# isula exec -it 10 ps aux
  USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
  root         1  0.1  0.0  21588  9504 ?        Ss   12:11   0:00 init
  root        14  0.1  0.0  27024  9376 ?        Ss   12:11   0:00 /usr/lib/system
  root        17  0.0  0.0  18700  5876 ?        Ss   12:11   0:00 /usr/lib/system
  dbus        22  0.0  0.0   9048  3624 ?        Ss   12:11   0:00 /usr/bin/dbus-d
  root        26  0.0  0.0   8092  3012 ?        Rs+  12:13   0:00 ps aux
  ```

* 进入容器执行shutdown命令：

  ```sh
  [root@localhost ~]# isula exec -it 10 bash
  [root@localhost /]# shutdown -h now
  [root@localhost /]# [root@localhost ~]#
  ```

  检查容器是否停止：

  ```sh
  [root@localhost ~]# isula exec -it 10 bash
  Error response from daemon: Exec container error;Container is not running:106faae22a926e22c828a0f2b63cf5c46e5d5986ea8a5b26de81390d0ed9714f
  ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/container_engine.md
---
# 容器引擎

Docker daemon是一个常驻后台的系统进程，docker 子命令执行前先要启动docker  daemon。

如果是通过rpm包或者系统包管理工具安装的，就可以使用systemctl start docker来启动docker daemon。

docker命令支持多个参数选项，对于参数选项有以下约定：

1. 单个字符的选项可以合并在一起，如：

   ```bash
   docker run -t -i busybox /bin/sh
   ```

   可以写成

   ```bash
   docker run -ti busybox /bin/sh
   ```

2. 在命令帮助中看到的如--icc=true之类的bool命令选项，如果没有使用这个选项，则这个标志位的值就是在命令帮助中看到的缺省值，如果使用了这个选项则这个标志位的值就是命令帮助中看的值的相反值，如果启动docker  daemon没有加上使用--icc选项，则默认设置了--icc=true,如果使用了--icc选项则表示是--icc=false。

3. 在命令帮助中看到的--attach=\[]之类的选项，表示这类的选项可以多次设置，如：

   ```bash
   docker run --attach=stdin --attach=stdout -i -t busybox /bin/sh
   ```

4. 在命令帮助中看到的-a, --attach=\[]之类的选项，表示这种选项既可以用-a value指定也可以用--attach=value指定。如：

   ```bash
   docker run -a stdin --attach=stdout -i -t busybox /bin/sh
   ```

5. \--name=””之类的选项需要的是一个字符串，只能指定一次，-c=0之类的选项需要的是一个整数，只能指定一次。

**表 1**  docker daemon启动时指定参数详解

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/container_management_1.md
---
# 容器管理

## 创建容器

### 下载镜像

运行docker命令需要root权限，当你使用普通用户登录时，需要用sudo权限执行docker命令。

```bash
[root@localhost ~]# docker pull busybox
```

该命令行将在docker官方的镜像库中下载busybox:latest（命令行中没指定TAG，所以使用默认的TAG名latest），镜像在下载过程中将检测所依赖的层本地是否存在，如果存在就跳过。从私有镜像库下载镜像时，请带上registry描述，例如：假如建立了一个私有镜像库，地址为192.168.1.110:5000，里面有一些常用镜像。使用下面命令行从私有镜像库中下载镜像。

```bash
[root@localhost ~]# docker pull 192.168.1.110:5000/busybox
```

从私有镜像库中下载下来的image名字带有镜像库地址的信息名字比较长，可以用docker tag命令生成一个名字简单点的image。

```bash
[root@localhost ~]# docker tag 192.168.1.110:5000/busybox busybox
```

可以通过docker images命令查看本地镜像列表。

### 运行一个简单的应用

```bash
[root@localhost ~]# docker run busybox /bin/echo "Hello world"
Hello world
```

该命令行使用busybox:latest（命令行中没有指定tag，所以使用默认的tag名latest）镜像创建了一个容器，在容器内执行了echo "Hello world"。使用下面命令行可以查看刚才创建的这个容器。

```bash
[root@localhost ~]# docker ps -l
CONTAINER ID        IMAGE               COMMAND                   CREATED             STATUS                     PORTS               NAMES
d8c0a3315bc0        busybox "/bin/echo 'Hello wo…"   5 seconds ago       Exited (0) 3 seconds ago                       practical_franklin
```

### 创建一个交互式的容器

```bash
[root@localhost ~]# docker run -it busybox /bin/bash
root@bf22919af2cf:/# ls 
bin  boot  dev  etc  home  lib  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var 
root@bf22919af2cf:/# pwd 
/
```

-ti选项分配一个伪终端给容器并可以使用STDIN进行交互，可以看到这时可以在容器内执行一些命令。这时的容器看起来完全是一个独立的linux虚拟机。使用exit命令退出容器。

### 后台运行容器

执行下面命令行，-d指示这个容器在后台运行，--name=container1 指定容器的名字为container1。

```bash
[root@localhost ~]# docker run -d --name=container1 busybox /bin/sh -c "while true;do echo hello world;sleep 1;done"
7804d3e16d69b41aac5f9bf20d5f263e2da081b1de50044105b1e3f536b6db1c
```

命令行的执行结果是返回了这个容器的ID，没有返回命令的执行结果hello world，此时容器在后台运行，可以用docker ps命令查看正在运行的容器:

```bash
[root@localhost ~]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
7804d3e16d69        busybox "/bin/sh -c 'while tr"   11 seconds ago      Up 10 seconds                           container1
```

用docker logs查看容器运行的输出：

```bash
[root@localhost ~]# docker logs container1
hello world
hello world
hello world
...
```

### 容器网络连接

默认情况下，容器可以访问外部网络，而外部网络访问容器时需要通过端口映射，下面以在docker中运行私有镜像库服务registry为例。下面的命令行中-P使registry镜像中开放的端口暴露给主机。

```bash
[root@localhost ~]# docker run --name=container_registry -d -P registry 
cb883f6216c2b08a8c439b3957fb396c847a99079448ca741cc90724de4e4731 
```

container\_registry这个容器已经启动了，但是并不知道容器中的服务映射到主机的哪个端口，通过docker port查看端口映射。

```bash
[root@localhost ~]# docker port container_registry 
5000/tcp -> 0.0.0.0:49155 
```

从输出可以看出，容器内的5000端口映射到了主机的49155端口。通过主机IP:49155就可以访问registry服务了，在浏览器中输入<http://localhost:49155>就可以返回registry的版本信息。

在运行registry镜像的时候还可以直接指定端口映射如：

```bash
docker run --name=container_registry -d -p 5000:5000 registry 
```

通过-p 5000:5000指定容器的5000端口映射到主机的5000端口。

### 注意事项

* **启动容器不能单独加-a stdin**

  启动容器时，不能单独加-a stdin，必须要同时加上-a stdout或者-a stderr，否则会导致终端即使在容器退出后也会卡住。

* **避免使用已有容器的长id、短id作为新容器的name**

  创建容器时，避免使用已有容器A的长id或短id作为新容器B的name。若使用容器A的长id作为容器B的name，当使用容器B的name作为指定容器进行操作时，docker匹配到的是容器A。若使用容器A的短id作为容器B的name，当使用容器A的短id作为指定容器进行相关操作时，docker匹配到的是容器B。这是因为，docker在匹配容器时，先精确匹配所有容器的长id。若未匹配成功，再根据container\_name进行精确匹配；若还未匹配成功，直接对容器id进行模糊匹配。

* **使用sh/bash等依赖标准输入输出的容器应该使用\`-ti\`参数，避免出现异常**

  正常情况：不用\`-ti\`参数启动sh/bash等进程容器，容器会马上退出。

  出现这种问题的原因在于，docker会先创建一个匹配用于容器内业务的stdin，在不设置-ti等交互式参数时，docker会在容器启动后关闭该pipe，而业务容器进程sh/bash在检测到stdin被关闭后会直接退出。

  异常情况：如果在上述过程中的特定阶段（关闭该pipe之前）强制杀死docker daemon，会导致该pipe的daemon端没有被及时关闭，这样即使不带\`-ti\`的sh/bash进程也不会退出，导致异常场景，这种容器就需要手动清理。

  Daemon重启后会接管原有的容器stream，而不带\`-ti\`参数的容器可能就无法处理（因为正常情况下这些容器不存在stream需要接管）；真实业务下几乎不存在这种使用方式(不带 \`-ti\`的sh/bash没有任何作用)，为了避免这类问题发生，限制交互类容器应该使用 \`-ti\`参数。

* **容器存储卷**

  启动容器时如果通过\`-v\`参数将主机上的文件挂载到容器中，在主机或容器中使用vi或sed命令修改文件可能会使文件inode发生改变，从而导致主机和容器内的文件不同步。容器中挂载文件时应该尽量避免使用这种文件挂载的方式（或不与vi和sed同时使用），也可以通过挂载文件上层目录来避免该问题。在docker挂载卷时“nocopy”选项可以避免将容器内挂载点目录下原有的文件拷贝到主机源目录下，但是这个选项只能在挂载匿名卷时使用，不能在bind mount的场景下使用。

* **避免使用可能会对host造成影响的选项**

  \--privileged 选项会让容器获得所有权限，容器可以做挂载操作和修改/proc、/sys等目录，可能会对host造成影响，普通容器需要避免使用该选项。

  共享host的namespace，比如--pid host/--ipc host/--net host等选项可以让容器跟host共享命名空间，同样会导致容器影响host的结果，需要避免使用。

* **kernel memory cgroup不稳定，禁止使用**

  kernel memory cgroup在小于4.0版本的Linux内核上仍属于实验阶段，运行起来不稳定，虽然Docker的Warning说是小于4.0就可以，但是我们评估认为，kmemcg在高版本内核仍然不稳定，所以不管是低版本还是高版本，均禁止使用。

  当docker run --kernel-memory时，会产生如下告警：

  ```text
  WARNING: You specified a kernel memory limit on a kernel older than 4.0. Kernel memory limits are experimental on older kernels, it won't work as expected as expected and can cause your system to be unstable.
  ```

* **blkio-weight参数在支持blkio精确控制的内核下不可用**

  \--blkio-weight-device 可以实现容器内更为精确的blkio控制，该控制需要指定磁盘设备，可以通过docker --blkio-weight-device参数实现。同时在这种内核下docker不再提供--blkio-weight方式限制容器blkio，使用该参数创建容器将会报错:

  ```text
  docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:398: container init caused \"process_linux.go:369: setting cgroup config for ready process caused \\\"blkio.weight not supported, use weight_device instead\\\"\""
  ```

* **使用--blkio-weight-device需要磁盘支持CFQ调度策略**

  \--blkio-weight-device参数需要磁盘工作于完全公平队列调度（CFQ：Completely Fair Queuing）的策略时才能工作。

  通过查看磁盘scheduler文件（‘/sys/block/<磁盘>/queue/scheduler’）可以获知磁盘支持的策略以及当前所采用的策略，如查看sda：

  ```bash
  # cat /sys/block/sda/queue/scheduler noop [deadline] cfq 
  ```

  当前sda支持三种调度策略：noop, deadline, cfq，并且正在使用deadline策略。通过echo修改策略为cfq：

  ```bash
  # echo cfq > /sys/block/sda/queue/scheduler
  ```

* **容器基础镜像中systemd使用限制**

  通过基础镜像创建的容器在使用过程中，容器基础镜像中的systemd仅用于系统容器，普通容器不支持使用。

### 并发性能

* docker内部的消息缓冲有一个上限，超过这个上限就会将消息丢弃，因此在并发执行命令时建议不要超过1000条命令，否则有可能会造成docker内部消息丢失，从而造成容器无法启动等严重问题。
* 并发创建容器并对容器执行restart时会偶现“oci runtime error: container init still running”报错，这是因为containerd对事件等待队列进行了性能优化，容器stop过程中执行runc delete，尝试在1s内kill掉容器的init进程，如果1s内init进程还没有被kill掉的话runc会返回该错误。由于containerd的GC（垃圾回收机制）每隔10s会回收之前runc delete的残留资源， 所以并不影响下次对容器的操作，一般出现上述报错的话等待4~5s之后再次启动容器即可。

### 安全特性解读

1. docker默认的权能配置分析

   原生的docker默认配置如下，默认进程携带的Cap如下:

   ```conf
   "CAP_CHOWN", 
   "CAP_DAC_OVERRIDE", 
   "CAP_FSETID", 
   "CAP_FOWNER", 
   "CAP_MKNOD", 
   "CAP_NET_RAW", 
   "CAP_SETGID", 
   "CAP_SETUID", 
   "CAP_SETFCAP", 
   "CAP_SETPCAP", 
   "CAP_NET_BIND_SERVICE", 
   "CAP_SYS_CHROOT", 
   "CAP_KILL", 
   "CAP_AUDIT_WRITE",
   ```

   默认的seccomp配置是白名单，不在白名单的syscall默认会返回SCMP\_ACT\_ERRNO，根据给docker不同的Cap开放不同的系统调用，不在上面的权限，默认docker都不会给到容器。

2. CAP\_SYS\_MODULE

   CAP\_SYS\_MODULE这个Cap是让容器可以插入或移除ko，增加该Cap可以让容器逃逸，甚至破坏内核。因为容器最大的隔离是Namespace，在ko中只要把他的Namespace指向init\_nsproxy即可。

3. CAP\_SYS\_ADMIN

   sys\_admin权限给容器带来的能力有：

   * 文件系统（mount，umount，quotactl）
   * namespace设置相关的（setns，unshare，clone new namespace）
   * driver ioctl
   * 对pci的控制，pciconfig\_read, pciconfig\_write, pciconfig\_iobase
   * sethostname

4. CAP\_NET\_ADMIN

   容器中有访问网络接口的和sniff网络流量的权限，容器可以获取到所有容器包括host的网络流量，对网络隔离破坏极大。

5. CAP\_DAC\_READ\_SEARCH

   该权限开放了open\_by\_handle\_at和name\_to\_handle\_at两个系统调用，如果host上没有selinux保护，容器中可通过暴力搜索file\_handle结构的inode号，进而可以打开host上的任意文件，影响文件系统的隔离性。

6. CAP\_SYS\_RAWIO

   容器中可对host写入io端口，可造成host内核崩溃。

7. CAP\_SYS\_PTRACE

   容器中有ptrace权限，可对容器的进程进行ptrace调试。现runc已经修补该漏洞，但有些工具比如nsenter和docker-enter并没有改保护，容器中可对这些工具执行的进程进行调试，获取这些工具带入的资源信息（Namespace、fd等），另外， ptrace可以绕过seccomp，极大增加内核攻击面。

8. Docker Cap接口 --cap-add all

   \--cap-add all表示赋予容器所有的权能，包括本节提到的比较危险的权能，使得容器可以逃逸。

9. 不要禁用docker的seccomp特性

   默认的docker有一个seccomp的配置，配置中使用的是白名单，不在配置的sys\_call会被seccomp禁掉，使用接口--security-opt 'seccomp:unconfined'可以禁止使用seccomp特性。如果禁用seccomp或使用自定义seccomp配置但过滤名单不全，都会增加容器对内核的攻击面。

10. 不要配置/sys和/proc目录可写

    /sys和/proc目录包含了linux维护内核参数、设备管理的接口，容器中配置该目录可写可能会导致容器逃逸。

11. Docker开放Cap  --CAP\_AUDIT\_CONTROL

    容器可以通过控制系统audit系统，并且通过AUDIT\_TTY\_GET/AUDIT\_TTY\_SET等命令可以获取审计系统中记录的tty执行输入记录，包括root密码。

12. CAP\_BLOCK\_SUSPEND和CAP\_WAKE\_ALARM

    容器可拥有阻塞系统挂起(epoll)的能力。

13. CAP\_IPC\_LOCK

    容器拥有该权限后，可以突破ulimit中的max locked memory限制，任意mlock超大内存块，造成一定意义的DoS攻击。

14. CAP\_SYS\_LOG

    容器拥有该权限后，可以dmesg读取系统内核日志，突破内核kaslr防护。

15. CAP\_SYS\_NICE

    容器拥有该权限后，可以改变进程的调度策略和优先级，造成一定意义的DoS攻击。

16. CAP\_SYS\_RESOURCE

    容器可以绕过对其的一些资源限制，比如磁盘空间资源限制、keymaps数量限制、pipe-size-max限制等，造成一定意义的DoS攻击。

17. CAP\_SYS\_TIME

    容器可以改变host上的时间。

18. Docker默认Cap风险分析

    Docker默认的Cap，包含了CAP\_SETUID和CAP\_FSETID，如host和容器共享目录，容器可对共享目录的二进制文件进行+s设置，host上的普通用户可使用其进行提权CAP\_AUDIT\_WRITE，容器可以对host写入，容器可以对host写入日志，host需配置日志防爆措施。

19. Docker和host共享namespace参数，比如 --pid，--ipc, --uts

    该参数为容器和host共享namespace空间，容器和host的namespace隔离没有了，容器可对host进行攻击。比如，使用--pid 和host共享pid namespace，容器中可以看到host上的进程pid号，可以随意杀死host的进程。

20. \--device 把host的敏感目录或者设备，映射到容器中

    Docker管理面有接口可以把host上的目录或者设备映射到容器中，比如--device，-v等参数，不要把host上的敏感目录或者设备映射到容器中。

## 创建容器使用hook-spec

### 原理及使用场景

docker支持hook的扩展特性，hook应用与底层runc的执行过程中，遵循OCI标准：<https://github.com/opencontainers/runtime-spec/blob/main/config.md#hooks>  。

hook主要有三种类型：prestart，poststart，poststop。分别作用于容器内用户应用程序启动之前，容器应用程序启动之后，容器应用程序停止之后。

### 接口参考

当前为docker run和create命令增加了参数“--hook-spec”，后面接spec文件的绝对路径，可以指定容器启动时的需要添加的hook，这些hook会自动附加在docker自己动态创建的hook后面（当前docker只有一个libnetwork的prestart hook），随容器的启动/销毁过程执行用户指定的程序。

spec的结构体定义为：

```conf
// Hook specifies a command that is run at a particular event in the lifecycle of a container
type Hook struct{       
               Path    string   `json:"path"`    
               Args    []string `json:"args,omitempty"`    
               Env     []string `json:"env,omitempty"`      
               Timeout *int     `json:"timeout,omitempty"`
}
// Hooks for container setup and teardown
type  Hooks struct{
               // Prestart is a list of hooks to be run before the container process is executed.
               // On Linux, they are run after the container namespaces are created.         
               Prestart []Hook `json:"prestart,omitempty"`
               // Poststart is a list of hooks to be run after the container process is started.         
               Poststart []Hook `json:"poststart,omitempty"`
               // Poststop is a list of hooks to be run after the container process exits.         
               Poststop []Hook `json:"poststop,omitempty"`
}
```

* Spec文件的path、args、env 都是必填信息；
* Timeout选填(建议配置)，参数类型为int，不接受浮点数，范围为\[1, 120]。
* Spec内容应该是json格式的，格式不对会报错，示例参考前面。
* 使用的时候既可以\`docker run --hook-spec /tmp/hookspec.json xxx\`, 也可以 \`docker create --hook-spec /tmp/hookspec.json xxx && docker start xxx\`。

### 为容器定制特有的hook

以启动过程中添加一个网卡的过程来说明。下面是相应的hook spec文件内容：

```conf
{
    "prestart": [
         {
             "path": "/var/lib/docker/hooks/network-hook",             
             "args": ["network-hook", "tap0", "myTap"],             
             "env": [],
             "timeout": 5
         }
     ],
     "poststart":[],     
     "poststop":[]
}
```

指定prestart hook增加一个网络hook的执行。路径是/var/lib/docker/hooks/network-hook，args代表程序的参数，第一个参数一般是程序名字，第二个是程序接受的参数。对于network-hook这个hook程序，需要两个参数，第一个是主机上的网卡名字，第二个是在容器内的网卡重命名。

* 注意事项
  1. hook path必须为docker的graph目录（--graph）下的hooks文件夹下，默认一般为 /var/lib/docker/hooks，可以通过docker info命令查看root路径。

     ```bash
     [root@localhost ~]# docker info 
     ... 
     Docker Root Dir: /var/lib/docker 
     ...
     ```

     这个路径可能会跟随用户手动配置，以及user namespace的使用（daemon --userns-remap）而变化。 path进行软链接解析后，必须以Docker Root Dir/hooks开头（如本例中使用 /var/lib/docker/hooks开头），否则会直接报错。

  2. hooks path必须指定绝对路径，因为这个是由daemon处理，相对路径对daemon无意义。同时绝对路径也更满足安全要求。

  3. hook程序打印到stderr的输出会打印给客户端并对容器的声明周期产生影响（比如启动失败），而输出到stdout的打印信息会被直接忽略。

  4. 严禁在hook里反向调用docker的指令。

  5. 配置的hook执行文件必须要有可执行权限，否则hook执行会报错。

  6. 使用hook时，执行时间应尽量短。如果hook中的prestart时间过长（超过2分钟），则会导致容器启动超时失败，如果hook中的poststop时间过长（超过2分钟），也会导致容器异常。

     目前已知的异常如下：执行docker stop命令停止容器时，2分钟超时执行清理时，由于hook还没执行结束，因此会等待hook执行结束（该过程持有锁），从而导致和该容器相关的操作都会卡住，需要等到hook执行结束才能恢复。另外，由于docker stop命令的2分钟超时处理是异步的过程，因此即使docker stop命令返回了成功，容器的状态也依然是up状态，需要等到hook执行完后状态才会修改为exited。

* 使用建议
  1. 建议配置hook的Timeout超时时间阈值，超时时间最好在5s以内。
  2. 建议不要配置过多hook，每个容器建议prestart、poststart、poststop这三个hook都只配置一个，过多hook会导致启动时间长。
  3. 建议用户识别多个hook之间的依赖关系，如果存在依赖关系，在组合hook配置文件时要根据依赖关系灵活调整顺序，hook的执行顺序是按照配置的spec文件上的先后顺序。

### 多个hook-spec

当有多个hook配置文件，要运行多个hook时，用户必须自己手工将多个hook配置文件组合成一个配置文件，使用--hook-spec参数指定此合并后的配置文件，方可生效所有的hook；如果配置多个--hook-spec参数，则只有最后一个生效。

配置举例：

hook1.json内容如下：

```bash
# cat /var/lib/docker/hooks/hookspec.json 
{
    "prestart": [
        {
            "path": "/var/lib/docker/hooks/lxcfs-hook",             
            "args": ["lxcfs-hook", "--log", "/var/log/lxcfs-hook.log"],             
            "env": []
        }
     ],     
     "poststart":[],     
     "poststop":[]
}
```

hook2.json内容如下：

```bash
# cat /etc/isulad-tools/hookspec.json 
{
      "prestart": [
         {
               "path": "/docker-root/hooks/docker-hooks",             
               "args": ["docker-hooks", "--state", "prestart"],             
               "env": []
         }
       ],     
       "poststart":[],     
       "poststop":[
          {
               "path": "/docker-root/hooks/docker-hooks",             
               "args": ["docker-hooks", "--state", "poststop"],             
               "env": []
          }
        ]
}
```

手工合并后的json内容如下：

```conf
{
       "prestart":[
          {
                "path": "/var/lib/docker/hooks/lxcfs-hook",             
                "args": ["lxcfs-hook", "--log", "/var/log/lxcfs-hook.log"],             
                "env": []
           },         
           {
                "path": "/docker-root/hooks/docker-hooks",             
                "args": ["docker-hooks", "--state", "prestart"],             
                "env": []
           }
        ],     
        "poststart":[],     
        "poststop":[
            {
                "path": "/docker-root/hooks/docker-hooks",             
                "args": ["docker-hooks", "--state", "poststop"],             
                "env": []
            }
         ]
}
```

需要注意的是，docker daemon会按照数组顺序依次读取hook配置文件中prestart等action中的hook二进制，进行执行动作。用户需要识别多个hook之间的依赖关系，如果有依赖关系，在组合hook配置文件时要根据依赖关系灵活调整顺序。

### 为所有容器定制默认的hook

Docker daemon同样可以接收--hook-spec的参数，--hook-spec的语义与docker create/run的--hook-spec参数相同，这里不再复述。也可以在/etc/docker/daemon.json里添加hook配置：

```conf
{
     "hook-spec": "/tmp/hookspec.json"
}
```

容器在运行时，会首先执行daemon定义的--hook-spec中指定的hooks，然后再执行每个容器单独定制的hooks。

## 创建容器配置健康检查

Docker提供了用户定义的对容器进行健康检查的功能。在Dockerfile中配置HEALTHCHECK CMD选项，或在容器创建时配置--health-cmd选项，在容器内部周期性地执行命令，通过命令的返回值来监测容器的健康状态。

### 配置方法

* 在Dockerfile中添加配置，如：

  ```conf
  HEALTHCHECK --interval=5m --timeout=3s --health-exit-on-unhealthy=true \
     CMD curl -f http://localhost/ || exit 1
  ```

  可配置的选项：

  1. \--interval=DURATION，默认 30s，相邻两次命令执行的间隔时间。另外，容器启动后，经过interval时间进行第一次检查。
  2. \--timeout=DURATION，默认 30s，单次检查命令执行的时间上限，超时则任务命令执行失败。
  3. \--start-period=DURATION，默认 0s，容器初始化时间。初始化期间也会执行健康检查，健康检查失败不会计入最大重试次数。但是，如果在初始化期间运行状况检查成功，则认为容器已启动。之后所有连续的检查失败都将计入最大重试次数。
  4. \--retries=N，默认 3，健康检查失败最大的重试次数。
  5. \--health-exit-on-unhealthy=BOOLEAN，默认false，检测到容器非健康时是否杀死容器
  6. CMD，必选，在容器内执行的命令。返回值为0表示成功，非0表示失败。

     在配置了HEALTHCHECK后创建镜像，HEALTHCHECK相关配置会被写入镜像的配置中。通过docker inspect可以看到。如：

     ```conf
     "Healthcheck": {
         "Test": [
             "CMD-SHELL",
             "/test.sh"
         ]
     },
     ```

* 在容器创建时的配置：

  ```bash
  docker run -itd --health-cmd "curl -f http://localhost/ || exit 1" --health-interval 5m --health-timeout 3s --health-exit-on-unhealthy centos bash
  ```

  可配置的选项：

  1. \--health-cmd，必选，在容器内执行的命令。返回值为0表示成功，非0表示失败。
  2. \--health-interval，默认 30s，最大为int64上限（纳秒）相邻两次命令执行的间隔时间。
  3. \--health-timeout，默认 30s，最大为int64上限（纳秒），单次检查命令执行的时间上限，超时则任务命令执行失败。
  4. \--health-start-period，默认 0s，最大为int64上限（纳秒），容器初始化时间。
  5. \--health-retries，默认 3，最大为int32上限，健康检查失败最大的重试次数。
  6. \--health-exit-on-unhealthy，默认false，检测到容器非健康时是否杀死容器。

     容器启动后，HEALTHCHECK相关配置会被写入容器的配置中。通过docker inspect可以看到。如：

     ```conf
     "Healthcheck": {
         "Test": [
             "CMD-SHELL",
             "/test.sh"
         ]
     },
     ```

### 检查规则

1. 容器启动后，容器状态中显示health:starting。
2. 经过start-period时间后开始，以interval为间隔周期性在容器中执行CMD。即：当一次命令执行完毕后，经过interval时间，执行下一次命令。
3. 若CMD命令在timeout限制的时间内执行完毕，并且返回值为0，则视为一次检查成功，否则视为一次检查失败。检查成功后，容器状态变为health:healthy。
4. 若CMD命令连续retries次检查失败，则容器状态变为health:unhealthy。失败后容器也会继续进行健康检查。
5. 容器状态为health:unhealthy时，任意一次检查成功会使得容器状态变为health:healthy。
6. 设置--health-exit-on-unhealthy的情况下，如果容器因为非被杀死退出（退出返回值137）后，健康检查只有容器在重新启动后才会继续生效。
7. CMD执行完毕或超时时，docker daemon会将这次检查的起始时间、返回值和标准输出记录到容器的配置文件中。最多记录最新的5条数据。此外，容器的配置文件中还存储着健康检查的相关参数。

通过docker ps可以看到容器状态。

```bash
[root@bac shm]# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                         PORTS               NAMES
7de2228674a2        testimg             "bash"              About an hour ago   Up About an hour (unhealthy)                       cocky_davinci
```

运行中的容器的健康检查状态也会被写入容器配置中。通过docker inspect可以看到。

```conf
"Health": {
    "Status": "healthy",
    "FailingStreak": 0,
    "Log": [
        {
            "Start": "2018-03-07T07:44:15.481414707-05:00",
            "End": "2018-03-07T07:44:15.556908311-05:00",
            "ExitCode": 0,
            "Output": ""
        },
        {
            "Start": "2018-03-07T07:44:18.557297462-05:00",
            "End": "2018-03-07T07:44:18.63035891-05:00",
            "ExitCode": 0,
            "Output": ""
        },
        ......
}
```

> \[!NOTE]说明
>
> * 容器内健康检查的状态信息最多保存5条。会保存最后得到的5条记录。
> * 容器内健康检查相关配置同时最多只能有一条生效。Dockerfile中配置的靠后的条目会覆盖靠前的；容器创建时的配置会覆盖镜像中的。
> * 在Dockerfile中可以通过 HEALTHCHECK NONE来取消引用的镜像中的健康检查配置。在容器运行时可以通过配置--no-healthcheck来取消镜像中的健康检查配置。不允许在启动时同时配置健康检查相关选项与--no-healthcheck选项。
> * 带有健康检查配置的容器启动后，若docker daemon退出，则健康检查不会执行，一直等待。docker daemon再次启动后，容器健康状态会变为starting。之后检查规则同上。
> * 构建容器镜像时若健康检查相关参数配置为空，则按照缺省值处理。
> * 容器启动时若健康检查相关参数配置为0，则按照缺省值处理。

## 停止与删除容器

用docker stop停止名为container1的容器：

```bash
[root@localhost ~]# docker stop container1
```

也可以用docker kill来杀死容器达到停止容器的目的：

```bash
[root@localhost ~]# docker kill container1
```

当容器停止之后，可以使用docker rm删除容器：

```bash
[root@localhost ~]# docker rm container1
```

当然，使用docker rm -f 强制删除容器也是可以的：

```bash
[root@localhost ~]# docker rm -f container1
```

### 注意事项

* 禁止使用docker rm -f XXX 删除容器。如果使用强制删除，docker rm会忽略过程中的错误，可能导致容器相关元数据残留。如果使用普通删除，如果删除过程出错，则会删除失败，不会导致元数据残留。
* 避免使用docker kill命令。docker kill命令发送相关信号给容器内业务进程，依赖于容器内业务进程对信号的处理策略，可能导致业务进程的信号处理行为与指令的预期不符合的情况。
* docker stop处于restarting状态的容器可能容器不会马上停止。如果一个容器使用了重启规则，当容器处于restarting状态时，docker stop这个容器时有很低的概率会立即返回，容器仍然会在重启规则的作用下再次启动。
* 不能用docker restart重启加了--rm参数的容器。加了--rm参数的容器在退出时，容器会主动删除，如果重启一个加了--rm的参数的容器， 可能会导致一些异常情况，比如启动容器时，同时加了--rm与-ti参数，对容器执行restart操作，可能会概率性卡住无法退出。

### docker stop/restart 指定t参数且t<0时，请确保自己容器的应用会处理stop信号

Stop的原理：（Restart会调用Stop流程）

1. Stop会首先给容器发送Stop 信号（15）
2. 然后等待一定的时间（这个时间就是用户输入的 t）
3. 过了一定时间，如果容器还活着，那么就发送kill信号（9）使容器强制退出

输入参数t（单位s）的含义：

* t<0  :  表示死等，不管多久都等待程序优雅退出，既然用户这么输入了，表示对自己的应用比较放心，认为自己的程序有合理的stop信号的处理机制
* t=0 ： 表示不等，立即发送kill -9 到容器
* t>0 ： 表示等一定的时间，如果容器还未退出，就发送kill -9 到容器

所以如果用户使用`t<0` (比如t=-1)，请确保自己容器的应用会正确处理signal 15，如果容器忽略了该信号，会导致docker stop一直卡住。

### 如果容器处于Dead状态，可能底层文件系统处于busy状态，需要手动删除

Docker在执行容器删除时，先停止容器的相关进程，之后将容器状态更改为Dead，最后执行容器rootfs的删除操作。当文件系统或者device mapper处于忙碌状态时，最后一步rootfs的删除会失败。docker ps -a查看会发现容器处于Dead状态。Dead状态的容器不能再次启动，需要等待文件系统不繁忙时，手动再次执行docker rm进行删除。

### 共享pid namespace容器，子容器处于pause状态会使得父容器stop卡住，并影响docker run命令执行

使用--pid参数创建共享pid  namespace的父子容器，在执行docker stop父容器时，如果子容器中有进程无法退出（比如处于D状态、pause状态），会产生父容器docker stop命令等待的情况，需要手动恢复这些进程，才能正常执行命令。

遇到该问题的时候，请对pause状态的容器使用docker  inspect 命令查询 PidMode对应的父容器是否为需要docker stop的容器。如果是该容器，请使用docker  unpause将子容器解除pause状态，指令即可继续执行。

一般来说，导致该类问题的可能原因是容器对应的pid  namespace由于进程残留导致无法被销毁。如果上述方法无法解决问题，可以通过借助linux工具，获取容器内残留进程，确定pid  namespace中进程无法退出的原因，解决后容器就可以退出：

* 获取容器pid  namespace id

  ```bash
  docker inspect --format={{.State.Pid}} CONTAINERID | awk '{print  "/proc/"$1"/ns/pid"}' |xargs readlink
  ```

* 获取该namespace下的线程

  ```bash
   ls -l /proc/*/task/*/ns/pid |grep -F PIDNAMESPACE_ID |awk '{print $9}' |awk -F  \/ '{print $5}'
  ```

## 容器信息查询

在任何情况下，容器的状态都不应该以docker命令执行是否成功返回为判断标准。如想查看容器状态，建议使用：

```bash
docker inspect <NAME|ID>
```

## 修改操作

### docker exec进入容器启动多个进程的注意事项

docker exec进入容器执行的第一个命令为 bash 命令时，当退出 exec 时，要保证在这次exec启动的进程都退出了，再执行exit退出，否则会导致exit退出时终端卡住的情况。如果要在exit退出时，exec中启动的进程仍然在后台保持运行，要在启动进程时加上nohup。

### docker rename和docker stats \<container\_name>的使用冲突

如果使用`docker stats <container_name>` 实时监控容器，当使用docker rename重命名容器之后，docker stats中显示的名字将还是原来的名字，不是rename后的名字。

### docker rename操作restarting状态的容器可能会失败

对一个处于restarting状态的容器执行rename操作的时候，docker会同步修改容器网络的相关配置。由于restarting状态的容器可能还未真正启动起来，网络是不存在的，导致rename操作报错sandbox不存在。建议rename只操作非restarting的稳定状态的容器。

### docker cp

1. 使用docker cp向容器中拷贝文件时，docker ps以及所有对这个容器的操作都将等待docker cp结束之后才能进行。
2. 容器以非root用户运行，当使用docker cp命令复制主机上的一个非root权限的文件到容器时，文件在容器中的权限角色会变成root。docker cp与cp命令不同，docker cp会修改复制到容器中文件的uid和gid为root。

### docker login

执行docker login后，会将user/passwd经 aes（256位）加密后保存在/root/.docker/config.json，同时生成  *root*.docker/aeskey(权限0600)，用来解密/root/.docker/config.json中的 user/passwd。目前不能定时更新aeskey，只能由用户手动删除aeskey来更新。aeskey更新后，不管是否重启过docker daemon，都需要重新login，才可以push。例如：

```bash
root@hello:~/workspace/dockerfile# docker login 
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. 
Username: example Password: 
Login Succeeded 
root@hello:~/workspace/dockerfile# docker push example/empty 
The push refers to a repository [docker.io/example/empty] 
547b6288eb33: Layer already exists 
latest: digest: sha256:99d4fb4ce6c6f850f3b39f54f8eca0bbd9e92bd326761a61f106a10454b8900b size: 524 
root@hello:~/workspace/dockerfile# rm /root/.docker/aeskey 
root@hello:~/workspace/dockerfile# docker push example/empty 
WARNING: Error loading config file:/root/.docker/config.json - illegal base64 data at input byte 0 
The push refers to a repository [docker.io/example/empty] 
547b6288eb33: Layer already exists 
errors: 
denied: requested access to the resource is denied 
unauthorized: authentication required 
root@hello:~/workspace/dockerfile# docker login 
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. 
Username: example 
Password: 
Login Succeeded 
root@hello:~/workspace/dockerfile# docker push example/empty 
The push refers to a repository [docker.io/example/empty] 
547b6288eb33: Layer already exists 
latest: digest: sha256:99d4fb4ce6c6f850f3b39f54f8eca0bbd9e92bd326761a61f106a10454b8900b size: 524
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/docker_engine/container_management_2.md
---
# 容器管理

# 总体说明

当前docker支持的子命令，按照功能划分为以下几组：

其中有些子命令还有一些参数选项如docker run,通过docker COMMAND --help可以查看相应COMMAND命令的帮助，命令选项参考上文的命令选项约定。下面详细介绍每个命令的使用。

## attach

用法：**docker attach \[OPTIONS] CONTAINER**

功能：附加到一个运行着的容器

选项：

\--no-stdin=false    不附加STDIN

\--sig-proxy=true    代理所有到容器内部的信号，不代理SIGCHLD, SIGKILL, SIGSTOP

示例：

````sh
$ sudo docker attach attach_test
root@2988b8658669:/# ls bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
```sh

## commit

用法：**docker commit \[OPTIONS\] CONTAINER \[REPOSITORY\[:TAG\]\]**

功能：由一个容器创建一个新的image

选项：

-a, \--author=""    指定作者

-m, \--message=""  提交的信息

-p, \--pause=true   在提交过程中暂停容器

示例：

运行一个容器，然后将这个容器提交成一个新的image

```sh
$ sudo docker commit test busybox:test
sha256:be4672959e8bd8a4291fbdd9e99be932912fe80b062fba3c9b16ee83720c33e1

$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             latest              e02e811dd08f        2 years ago         1.09MB
````

## cp

用法：docker cp \[OPTIONS] CONTAINER:SRC\_PATH DEST\_PATH|-

docker cp \[OPTIONS] SRC\_PATH|- CONTAINER:DEST\_PATH

功能：从指定的容器内的一个路径复制文件或文件夹到主机的指定路径中，或者把主机的文件或者文件夹拷贝到容器内。

注意：docker cp不支持容器内/proc，/sys，/dev，/tmp等虚拟文件系统以及用户在容器内自行挂载的文件系统内的文件拷贝。

选项：

-a, --archive   将拷贝到容器的文件属主设置为容器运行用户（--user）

-L, --follow-link   解析并跟踪文件的符号链接

示例：

复制registry容器中/test目录到主机的/home/aaa目录中

```sh
$ sudo docker cp registry:/test /home/aaa
```

## create

用法：**docker create \[OPTIONS] IMAGE \[COMMAND] \[ARG...]**

功能：使用image创建一个新的容器，并将返回一个容器的ID，创建之后的容器用docker start命令启动，OPTIONS用于创建容器时对容器进行配置，有些选项将覆盖image中对容器的配置，COMMAND指定容器启动时执行的命令。

选项：

**表 2**  参数说明

示例：

创建了一个名为busybox的容器，创建之后的容器用docker start命令启动。

```sh
$ sudo docker create -ti --name=busybox busybox /bin/bash
```

## diff

用法：**docker diff CONTAINER**

功能：检视容器的差异，相比于容器刚创建时做了哪些改变

选项：无

示例：

```sh
$ sudo docker diff registry
C /root
A /root/.bash_history
A /test
```

## exec

### 接口原型

```sh
rpc Exec(ExecRequest) returns (ExecResponse) {}
```

### 接口描述

在容器中执行命令，采用的gRPC通讯方式从CRI服务端获取url，再通过获得的url与websocket服务端建立长连接，实现与容器的交互。

### 注意事项

执行一条单独的命令，也能打开终端与容器交互。stdin/stdout/stderr之一必须是真的。如果tty为真，stderr必须是假的。 不支持多路复用，在这种情况下，stdout和stderr的输出将合并为单流。

### 参数

### 返回值

## export

用法：**docker export CONTAINER**

功能：将一个容器的文件系统内容以tar包导出到STDOUT

选项：无

示例：

将名为busybox的容器的内容导出到busybox.tar包中：

```sh
$ sudo docker export busybox > busybox.tar
$ ls
busybox.tar 
```

## inspect

用法：**docker inspect \[OPTIONS] CONTAINER|IMAGE \[CONTAINER|IMAGE...]**

功能：返回一个容器或者镜像的底层信息

选项：

-f, --format=""    按照给定的格式输出信息

-s, --size    若查询类型为容器，显示该容器的总体文件大小

\--type    返回指定类型的JSON格式

-t, --time=120 超时时间的秒数，若在该时间内docker inspect未执行成功，则停止等待并立即报错。默认为120秒。

示例：

1. 返回一个容器的信息

   ```sh
   $ sudo docker inspect busybox_test
   [
       {
           "Id": "9fbb8649d5a8b6ae106bb0ac7686c40b3cbd67ec2fd1ab03e0c419a70d755577",
           "Created": "2019-08-28T07:43:51.27745746Z",
           "Path": "bash",
           "Args": [],
           "State": {
               "Status": "running",
               "Running": true,
               "Paused": false,
               "Restarting": false,
               "OOMKilled": false,
               "Dead": false,
               "Pid": 64177,
               "ExitCode": 0,
               "Error": "",
               "StartedAt": "2019-08-28T07:43:53.021226383Z",
               "FinishedAt": "0001-01-01T00:00:00Z"
           },
   ......
   ```

2. 按照给定格式返回一个容器的指定信息，下面的例子返回busybox\_test容器IP地址

   ```sh
   $ sudo docker inspect -f {{.NetworkSettings.IPAddress}} busybox_test
   172.17.0.91
   ```

## logs

用法：**docker logs \[OPTIONS] CONTAINER**

功能：抓取容器内的日志信息，容器可以是运行状态的也可以是停止状态的

选项：

-f, --follow=false        实时打印日志信息

-t, --timestamps=false     显示日志的时间戳

\--since     显示指定时间之后的日志

\--tail="all"              设置显示的行数，默认显示所有

示例：

1. 查看jaegertracing容器的日志信息，该容器上跑了一个jaegertracing服务

   ```sh
   $ sudo docker logs jaegertracing
   {"level":"info","ts":1566979103.3696961,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":14269,"status":"unavailable"}
   {"level":"info","ts":1566979103.3820567,"caller":"memory/factory.go:55","msg":"Memory storage configuration","configuration":{"MaxTraces":0}}
   {"level":"info","ts":1566979103.390773,"caller":"tchannel/builder.go:94","msg":"Enabling service discovery","service":"jaeger-collector"}
   {"level":"info","ts":1566979103.3908608,"caller":"peerlistmgr/peer_list_mgr.go:111","msg":"Registering active peer","peer":"127.0.0.1:14267"}
   {"level":"info","ts":1566979103.3922884,"caller":"all-in-one/main.go:186","msg":"Starting agent"}
   {"level":"info","ts":1566979103.4047635,"caller":"all-in-one/main.go:226","msg":"Starting jaeger-collector TChannel server","port":14267}
   {"level":"info","ts":1566979103.404901,"caller":"all-in-one/main.go:236","msg":"Starting jaeger-collector HTTP server","http-port":14268}
   {"level":"info","ts":1566979103.4577134,"caller":"all-in-one/main.go:256","msg":"Listening for Zipkin HTTP traffic","zipkin.http-port":9411}
   ```

2. 加上-f选项，实时打印jaegertracing容器的日志信息

   ```sh
   $ sudo docker logs -f jaegertracing
   {"level":"info","ts":1566979103.3696961,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":14269,"status":"unavailable"}
   {"level":"info","ts":1566979103.3820567,"caller":"memory/factory.go:55","msg":"Memory storage configuration","configuration":{"MaxTraces":0}}
   {"level":"info","ts":1566979103.390773,"caller":"tchannel/builder.go:94","msg":"Enabling service discovery","service":"jaeger-collector"}
   {"level":"info","ts":1566979103.3908608,"caller":"peerlistmgr/peer_list_mgr.go:111","msg":"Registering active peer","peer":"127.0.0.1:14267"}
   {"level":"info","ts":1566979103.3922884,"caller":"all-in-one/main.go:186","msg":"Starting agent"}
   ```

## pause-unpause

用法：**docker pause CONTAINER**

**docker unpause CONTAINER**

功能：这两个命令是配对使用的，docker pause暂停容器内的所有进程，docker unpause恢复暂停的进程

选项：无

示例：

本示例将演示一个跑了docker registry（docker镜像服务）服务的容器，当使用docker pause 命令暂停这个容器的进程后，使用curl命令访问该registry服务将阻塞，使用docker unpause命令将恢复registry服务，可以用curl命令访问。

1. 启动一个registry容器

   ```sh
   $ sudo docker run -d --name pause_test -p 5000:5000 registry
   ```

   此时可以用curl命令访问这个服务，请求状态码会返回200 OK。

   ```sh
   $ sudo curl -v 127.0.0.1:5000
   ```

2. 暂停这个容器内的进程

   ```sh
   $ sudo docker pause pause_test
   ```

   此时用curl命令访问这个服务将阻塞，等待服务开启。

3. 恢复运行这个容器内的进程

   ```sh
   $ sudo docker unpause pause_test
   ```

   此时步骤2中的curl访问将恢复运行，请求状态码返回200 OK。

## port

用法：**docker port CONTAINER \[PRIVATE\_PORT\[/PROTO]]**

功能：列出容器的端口映射，或者查找指定端口在主机的哪个端口

选项：无

示例：

1. 列出容器所有的端口映射

   ```sh
   $ sudo docker port registry
   5000/tcp -> 0.0.0.0.：5000
   ```

2. 查找容器指定端口的映射

   ```sh
   $ sudo docker port registry 5000
   0.0.0.0.：5000
   ```

## ps

用法：**docker ps \[OPTIONS]**

功能：根据不同的选项列出不同状态的容器，在不加任何选项的情况下，将列出正在运行的容器

选项：

-a, --all=false     显示所用的容器

-f, --filter=\[]      筛选值，可用的筛选值有：exited=\<int>容器的退出码status=(restarting|running|paused|exited)容器的状态码（e.g. -f status=running，列出正在运行的容器）

-l, --latest=false   列出最近创建的一个容器

-n=-1            列出最近n次创建的容器

\--no-trunc=false   将64位的容器ID全部显示出来，默认显示12位容器的ID

-q, --quiet=false   显示容器的ID

-s, --size=false    显示容器的大小

示例：

1. 列出正在运行的容器

   ```sh
   $ sudo docker ps
   ```

2. 列出所有的容器

   ```sh
   $ sudo docker ps -a
   ```

## rename

用法：**docker rename OLD\_NAME NEW\_NAME**

功能：重命名容器

示例：

示例中，用docker run创建并启动一个容器，使用docker rename对容器重命名，并查看容器名是否改变。

```sh
$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
b15976967abb        busybox:latest        "bash"              3 seconds ago       Up 2 seconds                            festive_morse
$ sudo docker rename pedantic_euler new_name
$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
b15976967abb        busybox:latest        "bash"              34 seconds ago      Up 33 seconds                           new_name
```

## restart

用法：**docker restart \[OPTIONS] CONTAINER \[CONTAINER...]**

功能：重启一个运行中的容器

选项：

-t, --time=10   在杀掉容器之前等待容器停止的秒数，如果容器已停止，就重启。默认为10秒。

示例：

```sh
$ sudo docker restart busybox
```

> !\[NOTE]说明\
> 容器在restart过程中，如果容器内存在D状态或Z状态的进程，可能会导致容器重启失败，这需要进一步分析导致容器内进程D状态或Z状态的原因，待容器内进程D状态或Z状态解除后，再进行容器restart操作。

## rm

用法：**docker rm \[OPTIONS] CONTAINER \[CONTAINER...]**

功能：删除一个或多个容器

选项：

-f, --force=false      强制删除运行中的容器

-l, --link=false       删除指定的链接，而不是底层容器

-v, --volumes=false    删除与容器关联的卷

示例：

1. 删除一个停止运行的容器

   ```sh
   $ sudo docker rm test
   ```

2. 删除一个正在运行的容器

   ```sh
   $ sudo docker rm -f rm_test
   ```

## run

用法：**docker run \[OPTIONS] IMAGE \[COMMAND] \[ARG...]**

功能：该命令将由指定的image（如果指定的IMAGE不存在，则从官方镜像库中下载一个镜像）创建一个容器，并启动这个容器，并在容器中执行指定的命令。该命令集成了docker create命令、docker start命令、docker exec命令。

选项：（该命令的选项与docker create命令的选项一样，请参考docker create命令选项，仅仅多了以下两个选项）

\--rm=false        设置容器退出时自动删除容器

-v 挂载本地目录或匿名卷到容器内。注意：当将本地目录以带有selinux的安全标签的方式挂载到容器内的同时，尽量不要同时做该本地目录的增删操作，否则该安全标签可能不生效

\--sig-proxy=true    发往进程信号的代理，SIGCHLD, SIGSTOP, SIGKILL不使用代理

示例：

使用busybox镜像运行一个容器，在容器启动后执行/bin/sh

```sh
$ sudo docker run -ti busybox /bin/sh
```

## start

用法：**docker start \[OPTIONS] CONTAINER \[CONTAINER...]**

功能：启动一个或多个未运行容器

选项：

-a, --attach=false     容器的标准输出和错误输出附加到host的STDOUT和STDERR上

-i, --interactive=false  容器的标准输入附加到host的STDIN上

示例：

启动一个名为busybox的容器，添加-i -a选项附加标准输入输出，容器启动后直接进入容器内部，输入exit可以退出容器。

如果启动容器时不加-i -a选项，容器将在后台启动。

```sh
$ sudo docker start -i -a busybox
```

## stats

用法：**docker stats \[OPTIONS] \[CONTAINER...]**

功能：持续监控并显示指定容器（若不指定，则默认全部容器）的资源占用情况

选项：

-a, --all        显示所有容器（默认仅显示运行状态的容器）

\--no-stream    只显示第一次的结果，不持续监控

示例：

示例中，用docker run创建并启动一个容器，docker stats将输出容器的资源占用情况。

```sh
$ sudo docker stats
CONTAINER ID        NAME                    CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
2e242bcdd682        jaeger                  0.00%               77.08MiB / 125.8GiB   0.06%               42B / 1.23kB        97.9MB / 0B         38
02a06be42b2c        relaxed_chandrasekhar   0.01%               8.609MiB / 125.8GiB   0.01%               0B / 0B             0B / 0B             10
deb9e49fdef1        hardcore_montalcini     0.01%               12.79MiB / 125.8GiB   0.01%               0B / 0B             0B / 0B             9
```

## stop

用法：**docker stop \[OPTIONS] CONTAINER \[CONTAINER...]**

功能：通过向容器发送一个SIGTERM信号并在一定的时间后发送一个SIGKILL信号停止容器

选项：

-t, --time=10   在杀掉容器之前等待容器退出的秒数，默认为10S

示例：

```sh
$ sudo docker stop -t=15 busybox
```

## top

用法：**docker top CONTAINER \[ps OPTIONS]**

功能：显示一个容器内运行的进程

选项：无

示例：

先运行了一个名为top\_test的容器，并在其中执行了top指令

```sh
$ sudo docker top top_test
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
root                70045               70028               0                   15:52               pts/0               00:00:00            bash
```

显示的PID是容器内的进程在主机中的PID号。

## update

用法：**docker update \[OPTIONS] CONTAINER \[CONTAINER...]**

功能：热变更一个或多个容器配置。

选项：

**表 3**  参数说明

示例：

变更一个容器名为busybox的cpu和mem配置，包括容器获得主机CPU的相对权重值为512，容器中进程允许运行的CPU核心为0,1,2,3，容器运行内存限制为512m。

```sh
$ sudo docker update  --cpu-shares 512  --cpuset-cpus=0,3 --memory 512m ubuntu 
```

## wait

用法：**docker wait CONTAINER \[CONTAINER...]**

功能：等待一个容器停止，并打印出容器的退出码

选项：无

示例：

先开启一个名为busybox的容器

```sh
$ sudo docker start -i -a busybox
```

执行docker wait

```sh
$ sudo docker wait busybox
0
```

将阻塞等待busybox容器的退出，退出busybox容器后将看到打印退出码“0”。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/container_management.md
---
# 容器管理

## 创建容器

### 描述

isula create 命令用于创建一个新的容器。容器引擎会使用指定的容器镜像创建容器读写层，或者使用指定的本地rootfs作为容器的运行环境。创建完成后，会将容器的ID输出到标准输出，后续可以使用isula start 命令启动该容器。新创建的容器状态为**inited**状态。

### 用法

```sh
isula create [OPTIONS] IMAGE [COMMAND] [ARG...]
```

### 参数

create命令支持参数参考下表。

**表 1**  create命令参数列表

### 约束限制

* 使用--user或--group-add参数，在容器启动阶段校验user或group时，容器如果使用的是OCI镜像，是从镜像的真实rootfs的etc/passwd和etc/group文件中校验，如果使用的是rootfs文件夹或块设备作为容器的rootfs，则校验的是host中的etc/passwd和etc/group文件；查找时使用的rootfs会忽略-v 和--mount等挂载参数，意味着使用这些参数尝试覆盖etc/passwd和etc/group两个文件时，在查找阶段不生效，只在容器真正启动时生效。生成的配置保存在"iSulad根目录/engine/容器ID/start\_generate\_config.json"，文件格式如下：

  ```conf
  {
      "uid": 0,
      "gid": 8,
      "additionalGids": [
          1234,
          8
      ]
  }
  ```

### 示例

创建一个新容器

```sh
# isula create busybox
fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
# isula ps -a
STATUS PID IMAGE   COMMAND EXIT_CODE RESTART_COUNT STARTAT FINISHAT RUNTIME ID           NAMES                
inited -   busybox "sh"    0         0             -       -        lcr     fd7376591a9c fd7376591a9c4521...
```

## 启动容器

### 描述

isula start命令用于启动一个或多个容器。

### 用法

```sh
isula start [OPTIONS] CONTAINER [CONTAINER...]
```

### 参数

start命令支持参数参考下表。

**表 1**  start命令参数列表

### 示例

启动一个新容器

```sh
# isula start fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
```

## 运行容器

### 描述

isula run命令命令用于创建一个新的容器。会使用指定的容器镜像创建容器读写层，并且为运行指定的命令做好准备。创建完成后，使用指定的命令启动该容器。run命令相当于create然后start容器。

### 用法

```sh
isula run [OPTIONS] ROOTFS|IMAGE [COMMAND] [ARG...]
```

### 参数

run命令支持参数参考下表。

**表 1**  run命令参数列表

### 约束限制

* 容器父进程进程退出时，则对应的容器也自动退出。

* 创建普通容器时父进程不能为init，因为普通容器的权限不够，导致容器可以创建成功，但是attach进去的时候会卡住。

* 运行容器时，不指定--net，默认hostname为**localhost**。

* 使用--files-limit参数传入一个很小的值，如1时，启动容器时，iSulad创建cgroup子组后先设置files.limit值，再将容器进程的PID写入该子组的cgroup.procs文件，此时容器进程已经打开超过1个句柄，因而写入报错导致启动失败启动容器会失败。

* \--mount参数和--volume参数同时存在时，如果目的路径有冲突，则--mount会在--volume之后挂载(即将--volume中的挂载点覆盖掉)。

  备注：轻量级容器的参数中type支持bind或squashfs，当type=squashfs时，src是镜像的路径；原生docker的参数type支持bind、volume、tmpfs。

* restart重启策略不支持unless-stopped。

* 以下三种情况与docker 返回值不一致，docker返回127,轻量级容器返回125

  \--device参数指定主机设备为不存在的设备

  \--hook-spec参数指定不存在的hook json文件

  \--entrypoint 参数指定不存在的入口参数

* 使用--volume参数时，由于容器启动时会对/dev/ptmx设备进行删除重建，因此请勿将/dev目录挂载至容器/dev目录，应使用--device对/dev下的设备在容器中进行挂载

* 使用-it参数时，由于容器启动时会对/dev/ptmx设备进行删除重建，因此请勿将/dev目录挂载至容器/dev目录，应使用--device对/dev下的设备再容器中进行挂载。

* 禁止使用echo的方式向run命令的stdin输入数据，会导致客户端卡死。应该直接将echo的值作为命令行参数传给容器

  ```sh
  # echo ls | isula run -i busybox /bin/sh


  ^C
  #
  ```

  上述命令出现客户端卡死现象，这是由于上述命令相当于往stdin输入ls，随后EOF被读取，客户端不再发送数据，等待服务端退出，但是服务端无法区分客户端是否需要继续发送数据，因而服务端卡在read数据上，最终导致双方均卡死。

  正确的执行方式为：

  ```sh
  # isula run -i busybox ls
  bin
  dev
  etc
  home
  proc
  root
  sys
  tmp
  usr
  var
  #
  ```

* 使用host的根目录（/）作为容器的文件系统，那么在挂载路径时，如果有如下情况

  **表 2**  挂载情况

  > !\[WARNING]注意\
  > 第一种情况，先挂载/home/test1，然后挂载/home/test2，这种情况会导致/home/test1的内容覆盖掉原来/mnt下面的内容，这样可能导致/mnt下面不存在abc目录，这样会导致挂载/home/test2到/mnt/abc失败。\
  > 第二种情况，先挂载/home/test2，然后挂载/home/test1。这种情况，第二次的挂载会把/mnt的内容替换为/home/test1的内容，这样第一次挂载的/home/test2到/mnt/abc的内容就看不到了。\
  > 因此，不支持第一种使用方式；第二种使用用户需要了解这种数据无法访问的风险

* 请谨慎配置/sys和/proc目录可写。/sys和/proc目录包含了linux维护内核参数、设备管理的接口，容器中配置该目录可写可能会导致容器逃逸。

* 请谨慎配置容器与host共享namespace。比如—pid，--ipc, --uts，--net配置该参数为容器和host共享namespace空间，容器和host的namespace隔离没有了，在容器中可对host进行攻击。比如，使用—pid 和host共享pid namespace，容器中可以看到host上的进程pid号，可以随意杀死host的进程。

* 请谨慎配置使用--device、-v 等可以挂载主机资源的参数，请勿将host的敏感目录或者设备，映射到容器中，以防止敏感信息泄漏。

* 请谨慎使用--privileged选项启动容器，--privileged选项会导致容器权限过大，影响宿主机配置。

  > !\[WARNING]注意
  >
  > * 高并发场景（并发启动200容器）下，glibc的内存管理机制会导致内存空洞以及虚拟内存较大（例如10GB）的问题。该问题是高并发场景下glibc内存管理机制的限制，而不是内存泄露，不会导致内存消耗无限增大。可以通过设置MALLOC\_ARENA\_MAX环境变量来减少虚拟内存的问题，而且可以增大减少物理内存的概率。但是这个环境变量会导致iSulad的并发性能下降，需要用户根据实际情况做配置。\
  >   参考实践情况，平衡性能和内存，可以设置MALLOC\_ARENA\_MAX为4。（在arm64服务器上面对iSulad的性能影响在10%以内）\
  >   配置方法：
  >
  > 1. 手动启动iSulad的场景，可以直接export MALLOC\_ARENA\_MAX=4，然后再启动iSulad即可。
  > 2. systemd管理iSulad的场景，可以修改/etc/sysconfig/iSulad，增加一条MALLOC\_ARENA\_MAX=4即可。

### 示例

运行一个新容器

```sh
# isula run -itd busybox
9c2c13b6c35f132f49fb7ffad24f9e673a07b7fe9918f97c0591f0d7014c713b
```

## 停止容器

### 描述

isula stop命令用于停止一个或多个运行中的容器。首先向容器中的首进程会发送**SIGTERM**信号，在指定时间（默认为10s）内容器未停止时，会发送**SIGKILL**。

### 用法

```sh
isula stop [OPTIONS] CONTAINER [CONTAINER...]
```

### 参数

stop命令支持参数参考下表。

**表 1**  stop命令参数列表

### 约束限制

* 指定t参数且t<0时，请确保自己容器的应用会处理stop信号。

  Stop的原理：Stop会首先给容器发送Stop 信号（SIGTERM），然后等待一定的时间（这个时间就是用户输入的 t），过了指定时间如果容器还仍处于运行状态，那么就发送kill信号（SIGKILL）使容器强制退出。

* 输入参数t的含义：

  t<0  : 表示一直等待，不管多久都等待程序优雅退出，既然用户这么输入了，表示对自己的应用比较放心，认为自己的程序有 合理的stop信号的处理机制。

  t=0 ： 表示不等，立即发送kill -9 到容器。

  t>0 ： 表示等一定的时间，如果容器还未退出，就发送kill -9 到容器。

  所以如果用户使用t<0 （比如t=-1），请确保自己容器的应用会正确处理SIGTERM. 如果容器忽略了该信号，会导致isula stop一直卡住。

### 示例

停止一个容器

```sh
# isula stop fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
```

## 强制停止容器

### 描述

isula kill命令用于强制停止一个或多个运行中的容器。

### 用法

```sh
isula kill [OPTIONS] CONTAINER [CONTAINER...]
```

### 参数

kill命令支持参数参考下表。

**表 1**  kill命令参数列表

### 示例

杀掉一个容器

```sh
# isula kill fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
```

## 删除容器

### 描述

isula rm命令用于删除一个或多个容器。

### 用法

```sh
isula rm [OPTIONS] CONTAINER [CONTAINER...]
```

### 参数

rm命令支持参数参考下表。

**表 1**  rm命令参数列表

### 约束限制

* 在IO正常情况，空环境（只有1个容器）删除一个running容器的时间为T1，200个容器的环境（容器无大量IO操作，host IO正常）删除一个running容器所需时间为T2。T2的规格为：T2 = max { T1 \* 3,  5 } 秒钟。

### 示例

删除一个停止状态的容器

```sh
# isula rm fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
```

## 接入容器

### 描述

isula attach命令用于将当前终端的标准输入、标准输出和标准错误连接到正在运行的容器。

### 用法

```sh
isula attach [OPTIONS] CONTAINER
```

### 参数

attach命令支持参数参考下表。

**表 1**  attach命令参数列表

### 约束限制

* 原生docker attach容器会直接进入容器，而isulad  attach容器后需要敲一个回车才进入。

### 示例

接入一个运行状态的容器

```sh
# isula attach fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1
/ #
/ #
```

## 重命名容器

### 描述

isula rename命令用于重命名容器。

### 用法

```sh
isula rename [OPTIONS] OLD_NAME NEW_NAME
```

### 参数

rename命令支持参数参考下表。

**表 1**  rename 命令参数列表

### 示例

重命名一个容器

```sh
# isula rename my_container my_new_container
```

## 在容器中执行新命令

### 描述

isula exec命令用于正在运行的容器中运行一个新命令。新执行的命令将在容器的默认目录中运行。如果基础镜像指定了自定义目录，则将使用该目录。

### 用法

```sh
isula exec [OPTIONS] CONTAINER COMMAND [ARG...]
```

### 参数

exec命令支持参数参考下表。

**表 1**  exec命令参数列表

### 约束限制

* isula exec 不指定任何参数时，会默认使用-it参数， 表示分配一个伪终端，以交互式的方式进入容器

* 当使用isula exec 执行脚本，在脚本中执行后台进程时，需使用nohup标志忽略SIGHUP信号。

  使用isula exec运行脚本,在脚本中运行后台进程需使用nohup标志。否则内核会在exec执行的进程（session首进程）退出时，向后台执行的进程发送SIGHUP信号，导致后台进程退出，出现僵尸进程。

* isula exec 进入容器进程后，不能执行后台程序，否则会出现卡死现象。

  isula exec执行后台进程的方式如下：

  1. 使用isula exec进入容器终端，isula exec container\_name bash
  2. 进入容器后，执行 script &
  3. 执行exit，导致终端卡死

  ```markdown
  isula exec 进入容器后，执行后台程序卡住的原因为isula exec进入容器运行后台while1程序，当bash退出时，while1程序并不会退出，变为孤儿进程由1号
  进程接管，while1程序是由容器的初始bash进程fork &exec执行的，while1进程复制了bash进程的文件句柄，导致bash退出时，句柄并未完全关闭，导致
  console进程收不到句柄关闭事件，epoll_wait卡住，进程不退出。
  ```

* isula exec 不能用后台方式执行，否则可能会出现卡死现象。

  isula exec后台执行的方式如下：

  使用\*\*isula exec 脚本 &\*\*的方式后台执行exec，如：isula exec container\_name script & ，isula exec 后台执行，执行的脚本中不断cat某一文件，正常时在当前终端有输出，如果在当前终端执行回车操作，客户端会因读IO失败而退出读stdout的动作，使终端不再输出，服务端由于进程仍然在cat文件，会继续往fifo的buffer里写入数据，当缓存写满时，容器内进程会卡死在write动作上。

* 轻量级容器使用exec执行带有管道操作的命令时，建议使用/bin/bash -c 方式执行该命令。

  典型应用场景：

  使用isula exec container\_name -it ls /test | grep "xx" | wc -l,用于统计test目录下xx的文件个数，因exec执行的为"ls /test",其输出通过管道进行grep、wc 处理。exec执行的为"ls /test"的输出会换行，再针对该输出进行处理时，结果有误。

  原因：使用exec 执行ls /test,输出带有换行，针对该输出进行“| grep "xx" | wc -l“，处理结果为2（两行）

  ```sh
  # isula exec  -it container ls /test
  xx    xx10  xx12  xx14  xx3   xx5   xx7   xx9
  xx1   xx11  xx13  xx2   xx4   xx6   xx8
  #
  ```

  建议处理方式：使用run/exec执行带有管道操作的命令时，使用/bin/bash -c 执行命令，在容器中执行管道操作。

  ```sh
  # isula exec  -it container  /bin/sh -c "ls /test | grep "xx" | wc -l"
  15
  #
  ```

* 禁止使用echo的方式向exec命令的stdin输入数据，会导致客户端卡死。应该直接将echo的值作为命令行参数传给容器

  ```sh
  # echo ls | isula exec 38 /bin/sh


  ^C
  #
  ```

  上述命令可能出现客户端卡死现象，这是由于上述命令相当于往stdin输入ls，随后EOF被读取，客户端不再发送数据，等待服务端退出，但是服务端无法区分客户端是否需要继续发送数据，因而服务端卡在read数据上，最终导致双方均卡死。

  正确的执行方式为：

  ```sh
  # isula exec 38 ls
  bin   dev   etc   home  proc  root  sys   tmp   usr   var
  ```

### 示例

在运行中的容器中，执行echo命令

```sh
# isula exec c75284634bee echo "hello,world"
hello,world
```

## 查询单个容器信息

### 描述

isula inspect提供了容器的详细信息。

### 用法

```sh
isula inspect [OPTIONS] CONTAINER|IMAGE [CONTAINER|IMAGE...]
```

### 参数

inspect命令支持参数参考下表。

**表 1**  inspect命令参数列表

### 示例

查询容器信息

```sh
# isula inspect -f '{{.State.Status} {{.State.Running}}}' c75284634bee
running
true


# isula inspect c75284634bee
[
    {
        "Id": "c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a",
        "Created": "2019-08-01T22:48:13.993304927-04:00",
        "Path": "sh",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "Pid": 21164,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2019-08-02T06:09:25.535049168-04:00",
            "FinishedAt": "2019-08-02T04:28:09.479766839-04:00",
            "Health": {
                "Status": "",
                "FailingStreak": 0,
                "Log": []
            }
        },
        "Image": "busybox",
        "ResolvConfPath": "",
        "HostnamePath": "",
        "HostsPath": "",
        "LogPath": "none",
        "Name": "c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a",
        "RestartCount": 0,
        "HostConfig": {
            "Binds": [],
            "NetworkMode": "",
            "GroupAdd": [],
            "IpcMode": "",
            "PidMode": "",
            "Privileged": false,
            "SystemContainer": false,
            "NsChangeFiles": [],
            "UserRemap": "",
            "ShmSize": 67108864,
            "AutoRemove": false,
            "AutoRemoveBak": false,
            "ReadonlyRootfs": false,
            "UTSMode": "",
            "UsernsMode": "",
            "Sysctls": {},
            "Runtime": "lcr",
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "CapAdd": [],
            "CapDrop": [],
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": [],
            "HookSpec": "",
            "CPUShares": 0,
            "Memory": 0,
            "OomScoreAdj": 0,
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "CPUPeriod": 0,
            "CPUQuota": 0,
            "CPURealtimePeriod": 0,
            "CPURealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "SecurityOpt": [],
            "StorageOpt": {},
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "FilesLimit": 0,
            "Ulimits": [],
            "Hugetlbs": [],
            "HostChannel": {
                "PathOnHost": "",
                "PathInContainer": "",
                "Permissions": "",
                "Size": 0
            },
            "EnvTargetFile": "",
            "ExternalRootfs": ""
        },
        "Mounts": [],
        "Config": {
            "Hostname": "localhost",
            "User": "",
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "TERM=xterm",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Tty": true,
            "Cmd": [
                "sh"
            ],
            "Entrypoint": [],
            "Labels": {},
            "Annotations": {
                "log.console.file": "none",
                "log.console.filerotate": "7",
                "log.console.filesize": "1MB",
                "rootfs.mount": "/var/lib/isulad/mnt/rootfs",
                "native.umask": "secure"
            },
            "HealthCheck": {
                "Test": [],
                "Interval": 0,
                "Timeout": 0,
                "StartPeriod": 0,
                "Retries": 0,
                "ExitOnUnhealthy": false
            }
        },
        "NetworkSettings": {
            "IPAddress": ""
        }
    }
]
```

## 查询所有容器信息

### 描述

isula ps 用于查询所有容器的信息。

### 用法

```sh
isula ps [OPTIONS]
```

### 参数

ps命令支持参数参考下表。

**表 1**  ps命令参数列表

### 示例

查询所有容器信息

```sh
# isula ps -a

ID           IMAGE                                     STATUS  PID    COMMAND EXIT_CODE RESTART_COUNT STARTAT        FINISHAT    RUNTIME NAMES
e84660aa059c rnd-dockerhub.huawei.com/official/busybox running 304765 "sh"    0         0             13 minutes ago -           lcr     e84660aa059cafb0a77a4002e65cc9186949132b8e57b7f4d76aa22f28fde016
# isula ps -a --format "table {{.ID}} {{.Image}}" --no-trunc
ID                                                               IMAGE
e84660aa059cafb0a77a4002e65cc9186949132b8e57b7f4d76aa22f28fde016 rnd-dockerhub.huawei.com/official/busybox

```

## 重启容器

### 描述

isula restart 用于重启一个或者多个容器。

### 用法

```sh
isula restart [OPTIONS] CONTAINER [CONTAINER...]
```

### 参数

restart命令支持参数参考下表。

**表 1**  restart 命令参数列表

### 约束限制

* 指定t参数且t<0时，请确保自己容器的应用会处理stop信号。

  restart会首先调用stop停止容器。stop会首先给容器发送stop信号（SIGTERM），然后等待一定的时间（这个时间就是用户输入的 t），过了一定时间如果容器仍处于运行状态，那么就发送kill信号（SIGKILL）使容器强制退出。

* 输入参数t的含义：

  t<0  : 表示一直等待，不管多久都等待程序优雅退出，既然用户这么输入了，表示对自己的应用比较放心，认为自己的程序有合理的stop信号的处理机制。

  t=0 ： 表示不等，立即发送kill -9 到容器。

  t>0 ： 表示等一定的时间，如果容器还未退出，就发送kill -9 到容器。

  所以如果用户使用t<0（比如t=-1），请确保自己容器的应用会正确处理SIGTERM. 如果容器忽略了该信号，会导致isula restart一直卡住。

### 示例

重启单个容器

```sh
# isula restart c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a
 c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a 
```

## 等待容器退出

### 描述

isula wait用于等待一个或者多个容器退出。仅支持runtime类型为lcr的容器。

### 用法

```sh
isula wait [OPTIONS] CONTAINER [CONTAINER...]
```

### 参数

wait命令支持参数参考下表。

**表 1**  wait命令参数列表

### 示例

等待单个容器退出

```sh
# isula wait c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a
 137 
```

## 查看容器中进程信息

### 描述

isula top用于查看容器中的进程信息。

### 用法

```sh
isula top [OPTIONS] container [ps options]
```

### 参数

top命令支持参数参考下表。

**表 1**  top命令参数列表

### 示例

查询容器中进程信息

```sh
# isula top 21fac8bb9ea8e0be4313c8acea765c8b4798b7d06e043bbab99fc20efa72629c
UID        PID  PPID  C STIME TTY          TIME CMD
root     22166 22163  0 23:04 pts/1    00:00:00 sh
```

## 查看容器使用的资源

### 描述

isula stats用于实时显示资源使用的统计信息。

### 用法

```sh
isula stats [OPTIONS] [CONTAINER...]
```

### 参数

stats命令支持参数参考下表。

**表 1**  stats命令参数列表

### 示例

显示资源使用的统计信息

```sh
# isula stats --no-stream 21fac8bb9ea8e0be4313c8acea765c8b4798b7d06e043bbab99fc20efa72629c                                                                                 CONTAINER        CPU %      MEM USAGE / LIMIT          MEM %      BLOCK I / O                PIDS      
21fac8bb9ea8     0.00       56.00 KiB / 7.45 GiB       0.00       0.00 B / 0.00 B            1  
```

## 获取容器日志

### 描述

isula logs用于获取容器的日志。

### 用法

```sh
isula logs [OPTIONS] [CONTAINER...]
```

### 参数

logs命令支持参数参考下表。

**表 1**  logs命令参数列表

### 约束限制

* 容器串口logs日志记录功能，默认为开启状态，需要关闭可以通过 isula create --log-opt disable-log=true 或 isula run --log-opt disable-log=true 关闭。

### 示例

获取容器日志

```sh
# isula logs 6a144695f5dae81e22700a8a78fac28b19f8bf40e8827568b3329c7d4f742406
hello, world
hello, world
hello, world
```

## 容器与主机之间数据拷贝

### 描述

isula cp 用于容器与主机之间的数据拷贝，仅支持runtime类型为lcr的容器。

### 用法

```sh
isula cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH
isula cp [OPTIONS] SRC_PATH CONTAINER:DEST_PATH
```

### 参数

cp命令支持参数参考下表。

**表 1**  cp命令参数列表

### 约束限制

* iSulad在执行拷贝时，不会挂载/etc/hostname， /etc/resolv.conf，/etc/hosts三个文件，也不会对--volume和--mount参数传入的参数挂载到host，所以对这些文件的拷贝使用的是镜像中的原始文件，而不是真实容器中的文件。

  ```sh
  # isula cp b330e9be717a:/etc/hostname /tmp/hostname
  # cat /tmp/hostname
  # 
  ```

* iSulad在解压文件时，不会对文件系统中即将被覆盖的文件或文件夹做类型判断，而是直接覆盖，所以在拷贝时，如果源为文件夹，同名的文件会被强制覆盖为文件夹；如果源为文件，同名的文件夹会被强制覆盖为文件。

  ```sh
  # rm -rf /tmp/test_file_to_dir && mkdir /tmp/test_file_to_dir
  # isula exec b330e9be717a /bin/sh -c "rm -rf /tmp/test_file_to_dir && touch /tmp/test_file_to_dir"
  # isula cp b330e9be717a:/tmp/test_file_to_dir /tmp
  # ls -al /tmp | grep test_file_to_dir
  -rw-r-----    1 root     root             0 Apr 26 09:59 test_file_to_dir
  ```

* cp命令仅供维护定位时使用，应避免在生产环境上线使用cp命令。

### 示例

将主机/test/host目录拷贝到容器21fac8bb9ea8的/test目录下。

```sh
isula cp /test/host 21fac8bb9ea8:/test
```

将容器21fac8bb9ea8的/www目录拷贝到主机的/tmp目录中。

```sh
isula cp 21fac8bb9ea8:/www /tmp/
```

## 暂停容器中所有的进程

### 描述

isula pause用于暂停一个或者多个容器中的所有进程。

### 用法

```sh
isula pause [OPTIONS] CONTAINER [CONTAINER...]
```

### 参数

### 约束限制

* 只有状态为running的容器可以被执行pause操作
* 当容器被pause后，无法执行其他生命周期管理操作（如restart/exec/attach/kill/stop/rm等）
* 当带有健康检查配置的容器被pause后，容器状态最终变为unhealthy状态

### 示例

暂停一个正在运行的容器，命令示例如下：

```sh
# isula pause 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac
 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac 
```

## 恢复容器容器中的所有进程

### 描述

isula unpause用于恢复一个或者多个容器中的所有进程, 为isula pause的逆过程。

### 用法

```sh
isula unpause [OPTIONS] CONTAINER [CONTAINER...]
```

### 参数

### 约束限制

* 只有状态为paused的容器可以被执行unpause操作

### 示例

恢复一个被暂停的容器，命令示例如下：

```sh
# isula unpause 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac
 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac 
```

## 从服务端实时获取事件消息

### 描述

isula events用于从服务端获取实时事件。

### 用法

```sh
isula events [OPTIONS]
```

### 参数

### 约束限制

* 支持容器相关事件为：create、start、restart、stop、exec\_create、exec\_die、attach、kill、top、rename、archive-path、extract-to-dir、update、pause、unpause、export、resize。
* 支持镜像相关事件为：load、remove、pull、login、logout。

### 示例

从服务端实时获取事件消息，命令示例如下：

```sh
# isula events 
```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_form/system_container/dynamically_managing_container_resources_syscontainer_tools.md
---
# 容器资源动态管理

普通容器无法支持对容器内的资源进行管理，例如添加一个块设备到容器、插入一块物理/虚拟网卡到容器。系统容器场景下，通过syscontainer-tools工具可以实现动态为容器挂载/卸载块设备，网络设备，路由和卷等资源。

要使用此功能，需要安装syscontainer-tools工具：

```sh
[root@localhost ~]# yum install syscontainer-tools
```

## 设备管理

### 功能描述

syscontainer-tools支持将宿主机上的块设备（比如磁盘、LVM）或字符设备（比如GPU、binner、fuse）添加到容器中。在容器中使用该设备，例如可以对磁盘进行fdisk格式化，写入fs等操作。在容器不需要设备时，syscontainer-tools可以将设备从容器中删除，归还宿主机。

### 命令格式

```sh
syscontainer-tools [COMMAND][OPTIONS] <container_id> [ARG...]
```

其中：

COMMAND：设备管理相关的命令。

OPTIONS：设备管理命令支持的选项。

container\_id：容器id。

ARG：命令对应的参数。

### 参数说明

### 约束限制

* 添加/删除设备的时机可以是容器实例非运行状态，完成操作后启动容器，容器内会有体现；也可以在容器运行时（running）动态添加。
* 不能在容器内和host上并发进行fdisk对磁盘的格式化写入，会影响容器磁盘使用。
* add-device将磁盘添加到容器的特定目录时，如果容器内的父目录为多级目录（比如/dev/a/b/c/d/e...）且目录层级不存在，则syscontainer-tools会自动在容器内创建对应目录；当删除时，不会将创建的父目录删除。如果用户下一次add-device到该父目录，则会提示已经存在无法添加成功。
* add-device添加磁盘、更新磁盘参数时，配置磁盘Qos；当配置磁盘Qos的read/write bps、read/write IOPS值时，不建议配置值过小，当设置过小时，会造成磁盘表现为不可读（实际原因是速度过慢），最终影响业务功能。
* 使用--blkio-weight-device来限制指定块设备的权重，如果当前块设备仅支持BFQ模式，可能会报错，提示用户检查当前OS环境是否支持BFQ块设备权重值设置。

### 使用示例

* 启动一个系统容器，指定hook spec为syscontainer hook执行配置脚本

  ```sh
  [root@localhost ~]# isula run -tid --hook-spec /etc/syscontainer-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init
  eed1096c8c7a0eca6d92b1b3bc3dd59a2a2adf4ce44f18f5372408ced88f8350
  ```

* 添加一个块设备到容器

  ```sh
  [root@localhost ~]# syscontainer-tools add-device ee /dev/sdb:/dev/sdb123
  Add device (/dev/sdb) to container(ee,/dev/sdb123) done.
  [root@localhost ~]# isula exec ee fdisk -l /dev/sdb123
  Disk /dev/sdb123: 50 GiB, 53687091200 bytes, 104857600 sectors
  Units: sectors of 1 * 512 = 512 bytes
  Sector size (logical/physical): 512 bytes / 512 bytes
  I/O size (minimum/optimal): 512 bytes / 512 bytes
  Disklabel type: dos
  Disk identifier: 0xda58a448

  Device        Boot Start       End   Sectors Size Id Type
  /dev/sdb123p1       2048 104857599 104855552  50G  5 Extended
  /dev/sdb123p5       4096 104857599 104853504  50G 83 Linux
  ```

* 更新设备信息

  ```sh
  [root@localhost ~]# syscontainer-tools update-device --device-read-bps /dev/sdb:10m ee
  Update read bps for device (/dev/sdb,10485760) done.
  ```

* 删除设备

  ```sh
  [root@localhost ~]# syscontainer-tools remove-device ee /dev/sdb:/dev/sdb123
  Remove device (/dev/sdb) from container(ee,/dev/sdb123) done.
  Remove read bps for device (/dev/sdb) done.
  ```

## 网卡管理

### 功能描述

syscontainer-tools支持将宿主机上的物理网卡或虚拟网卡插入到容器，在不使用网卡的时候从容器中删除归还给宿主机，并且可以动态修改网卡配置。插入物理网卡即把宿主机上一块网卡直接添加到容器中，插入虚拟网卡则需要先创建一对veth pair，之后将一端插入到容器中。

### 命令格式

```sh
syscontainer-tools [COMMAND][OPTIONS] <container_id>
```

其中：

COMMAND：网卡管理相关的命令。

OPTIONS：网卡管理命令支持的选项。

container\_id：容器id。

### 参数说明

### 约束限制

* 支持添加物理网卡（eth）和虚拟网卡（veth）两种类型。
* 在添加网卡时可以同时对网卡进行配置，参数包括--ip/--mac/--bridge/--mtu/--qlen。
* 支持最多添加8个物理网卡到容器。
* 使用syscontainer-tools add-nic向容器添加eth网卡后，如果不加hook，在容器退出前必须手工将nic删除，否则在host上的eth网卡的名字会被更改成容器内的名字。
* 对于物理网卡（1822 vf网卡除外），add-nic必须使用原mac地址，update-nic禁止修改mac地址，容器内也不允许修改mac地址。
* 使用syscontainer-tools add-nic时，设置mtu值，设置范围跟具体的网卡型号有关。
* 使用syscontainer-tools向容器添加网卡和路由时，建议先执行add-nic添加网卡，然后执行add-route添加路由；使用syscontainer-tools从容器删除网卡和路由时，建议先执行remove-route删除路由，然后执行remove-nic删除网卡。
* 使用syscontainer-tools添加网卡时，一块网卡只能添加到一个容器中。

### 使用示例

* 启动一个系统容器，指定hook spec为syscontainer hook执行配置脚本：

  ```sh
  [root@localhost ~]# isula run -tid --hook-spec /etc/syscontainer-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init
  2aaca5c1af7c872798dac1a468528a2ccbaf20b39b73fc0201636936a3c32aa8
  ```

* 添加一个虚拟网卡到容器

  ```sh
  [root@localhost ~]# syscontainer-tools add-nic --type "veth" --name abc2:bcd2 --ip 172.17.28.5/24 --mac 00:ff:48:13:xx:xx --bridge docker0 2aaca5c1af7c
  Add network interface to container 2aaca5c1af7c (bcd2,abc2) done  
  ```

* 添加一个物理网卡到容器

  ```sh
  [root@localhost ~]# syscontainer-tools add-nic --type "eth" --name eth3:eth1 --ip 172.17.28.6/24  --mtu 1300  --qlen 2100 2aaca5c1af7c
  Add network interface to container 2aaca5c1af7c (eth3,eth1) done
  ```

  > \[!NOTE]说明
  > 添加虚拟网卡或物理网卡时，请确保网卡处于空闲状态，添加正在使用的网卡会导致系统网络断开。

## 路由管理

### 功能描述

syscontainer-tools工具可以对系统容器进行动态添加/删除路由表。

### 命令格式

```sh
syscontainer-tools [COMMAND][OPTIONS] <container_id> [ARG...]
```

其中：

COMMAND：路由管理相关的命令。

OPTIONS：路由管理命令支持的选项。

container\_id：容器id。

ARG：命令对应的参数。

### 接口说明

### 约束限制

* 使用syscontainer-tools向容器添加网卡和路由时，建议先执行add-nic添加网卡，然后执行add-route添加路由；使用syscontainer-tools从容器删除网卡和路由时，建议先执行remove-route删除路由，然后执行remove-nic删除网卡。
* 向容器内添加路由规则时，需确保所添加的路由规则与容器内现有的路由规则不会产生冲突。

### 使用示例

* 启动一个系统容器，指定hook spec为syscontainer hook执行配置脚本：

  ```sh
  [root@localhost ~]# isula run -tid --hook-spec /etc/syscontainer-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init
  0d2d68b45aa0c1b8eaf890c06ab2d008eb8c5d91e78b1f8fe4d37b86fd2c190b
  ```

* syscontainer-tools向系统容器添加一块物理网卡：

  ```sh
  [root@localhost ~]# syscontainer-tools add-nic --type "eth" --name enp4s0:eth123 --ip 172.17.28.6/24  --mtu 1300  --qlen 2100 0d2d68b45aa0
  Add network interface (enp4s0) to container (0d2d68b45aa0,eth123) done
  ```

* syscontainer-tools添加一条路由规则到系统容器，注意格式需按照'\[{"dest":"default",  "gw":"192.168.10.1"},{"dest":"192.168.0.0/16","dev":"eth0","src":"192.168.1.2"}]'来配置。如果dest为空会自动填成default。

  ```sh
  [root@localhost ~]# syscontainer-tools add-route 0d2d68b45aa0 '[{"dest":"172.17.28.0/32", "gw":"172.17.28.5","dev":"eth123"}]'
  Add route to container 0d2d68b45aa0, route: {dest:172.17.28.0/32,src:,gw:172.17.28.5,dev:eth123} done
  ```

* 查看容器内是否新增一条路由规则：

  ```sh
  [root@localhost ~]# isula exec -it 0d2d68b45aa0 route
  Kernel IP routing table
  Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
  172.17.28.0     172.17.28.5     255.255.255.255 UGH   0      0        0 eth123
  172.17.28.0     0.0.0.0         255.255.255.0   U     0      0        0 eth123
  ```

## 挂卷管理

### 功能描述

普通容器仅支持在创建时指定--volume参数将宿主机的目录/卷挂载到容器实现资源共享，但是无法在容器运行时将挂载到容器中的目录/卷卸载掉，也不支持将宿主机的目录/卷挂载到容器。系统容器可以通过syscontainer-tools工具实现动态将宿主机的目录/卷挂载到容器，以及将容器中的目录/卷进行卸载。

### 命令格式

```sh
syscontainer-tools [COMMAND][OPTIONS] <container_id> [ARG...]
```

其中：

COMMAND：路由管理相关的命令。

OPTIONS：路由管理命令支持的选项。

container\_id：容器id。

ARG：命令对应的参数。

### 接口说明

### 约束限制

* 挂载目录（add-path）的时候必须要指定绝对路径。
* 挂载目录（add-path）会在主机上生成/.sharedpath挂载点。
* 最多可以向单个容器中添加128个volume，超过128后无法添加成功。
* add-path不能将主机目录覆盖容器中的根目录（/），否则会造成功能影响。

### 使用示例

* 启动一个系统容器，指定hook spec为syscontainer hook执行配置脚本：

  ```sh
  [root@localhost ~]# isula run -tid --hook-spec /etc/syscontainer-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init
  e45970a522d1ea0e9cfe382c2b868d92e7b6a55be1dd239947dda1ee55f3c7f7
  ```

* syscontainer-tools将宿主机某个目录挂载到容器，实现资源共享：

  ```sh
  [root@localhost ~]# syscontainer-tools add-path e45970a522d1 /home/test123:/home/test123
  Add path (/home/test123) to container(e45970a522d1,/home/test123) done.
  ```

* 宿主机目录/home/test123创建一个文件，然后在容器内查看文件是否可以访问：

  ```sh
  [root@localhost ~]# echo "hello world" > /home/test123/helloworld
  [root@localhost ~]# isula exec e45970a522d1 bash
  [root@localhost /]# cat /home/test123/helloworld
  hello world
  ```

* syscontainer-tools将挂载目录从容器内删除：

  ```sh
  [root@localhost ~]# syscontainer-tools remove-path e45970a522d1 /home/test123:/home/test123
  Remove path (/home/test123) from container(e45970a522d1,/home/test123) done
  [root@localhost ~]# isula exec e45970a522d1 bash
  [root@localhost /]# ls /home/test123/helloworld
  ls: cannot access '/home/test123/helloworld': No such file or directory
  ```

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/container_engine/isula_container_engine/container_resource_management.md
---
# 容器资源管理

## 描述

可以通过namespace和cgroup等功能实现对容器的资源管理。isula支持使用cgroup v1和cgroup v2实现对资源的限制，其中
cgroup v2属于实验特性，不支持商用。当系统配置为只支持cgroup v2并将cgroup v2挂载到/sys/fs/cgroup目录时，isula使用
cgroup v2来进行资源管理。无论是cgroup v1还是使用cgroup v2对容器资源进行管理，isula提供给用户实现资源限制的接口是
一致的。

## 资源共享

### 描述

容器间或者容器与host之间可以共享namespace信息，包括pid, net, ipc, uts。

> !\[NOTE]说明\
> 当使用与主机共享namespace信息时，即缺少了对应的namespace隔离机制，在容器中可以查询、操作主机上的信息，存在安全
> 隐患。比如使用--pid=host共享主机pid namespace时，即可以看到主机上其他进程信息，造成信息泄露，甚至直接kill杀死主机
> 进程。请在确保安全的场景下，谨慎使用共享主机host namespace功能。

### 用法

isula create/run时使用namespace相关的参数共享资源，具体参数见下方参数列表。

### 参数

create/run时可以指定下列参数。

### 示例

如果两个容器需要共享同一个pid namespace，在运行容器时，直接加上--pid container:\<containerID> 即可，如：

```sh
isula run -tid --name test_pid busybox sh
isula run -tid --name test --pid container:test_pid busybox sh
```

## 限制运行时的CPU资源

### 描述

可以通过参数限制容器的各项cpu资源值。

### 用法

isula create/run时使用cpu相关的参数限制容器的各项cpu资源值，具体参数及取值见下方参数列表。

### 参数

create/run时可以指定下列参数。

### 示例

如果需要限制容器只是用特定的cpu，在运行容器时，直接加上--cpuset-cpus number 即可，如：

```sh
isula run -tid --cpuset-cpus 0,2-3 busybox sh
```

> !\[NOTE]说明\
> 是否设置成功，请参见“查询单个容器信息”章节。

## 限制运行时的内存

### 描述

可以通过参数限制容器的各项内存值上限。

### 用法

isula create/run时使用内存相关的参数限制容器的各项内存使用上限，具体参数及取值见下方参数列表。

### 参数

create/run时可以指定下列参数。

### 示例

如果需要限制容器内内存的上限，在运行容器时，直接加上--memory \<number>\[\<unit>]即可，如：

```sh
isula run -tid --memory 1G busybox sh
```

## 限制运行时的IO资源

### 描述

可以通过参数限制容器中设备读写速度。

### 用法

isula create/run时使用--device-read-bps/--device-write-bps \<device-path>:\<number>\[\<unit>]来限制容器中设备的读写速度。

### 参数

create/run时指定--device-read/write-bps参数。

### 示例

如果需要限制容器内设备的读写速度，在运行容器时，直接加上--device-write-bps/--device-read-bps \<device-path>:\<number>\[\<unit>]即可，例如，限制容器busybox内设备/dev/sda的读速度为 1MB 每秒，则命令如下：

```sh
isula run -tid --device-write /dev/sda:1mb busybox sh
```

限制写速度的命令如下：

```sh
isula run -tid read-bps /dev/sda:1mb busybox sh
```

## 限制容器rootfs存储空间

### 描述

在ext4上使用overlay2时，可以设置单个容器的文件系统限额，比如设置A容器的限额为5G，B容器为10G。

该特性通过ext4文件系统的project quota功能来实现，在内核支持的前提下，通过系统调用SYS\_IOCTL设置某个目录的project ID，再通过系统调用SYS\_QUOTACTL设置相应的project ID的hard limit和solft limit值达到限额的目的。

### 用法

1. 环境准备

   文件系统支持Project ID和Project Quota属性，4.19版本内核已经支持，外围包e2fsprogs版本不低于1.43.4-2。

2. 在容器挂载overlayfs之前，需要对不同容器的upper目录和work目录设置不同的project id，同时设置继承选项，在容器挂载overlayfs之后不允许再修改project id和继承属性。

3. 配额的设置需要在容器外以特权用户进行。

4. daemon中增加如下配置

   ```sh
   -s overlay2 --storage-opt overlay2.override_kernel_check=true
   ```

5. daemon支持以下选项，用于为容器设置默认的限制，

   \--storage-opt overlay2.basesize=128M 指定默认限制的大小，若isula run时也指定 了--storage-opt size选项，则以run时指定来生效，若daemon跟isula run时都不指定大小，则表示不限制。

6. 需要开启文件系统Project ID和Project Quota属性。
   * 新格式化文件系统并mount

     ```sh
     # mkfs.ext4 -O quota,project /dev/sdb
     # mount -o prjquota /dev/sdb /var/lib/isulad
     ```

### 参数

create/run时指定--storage-opt参数。

### 示例

在isula run/create命令行上通过已有参数“--storage-opt size=\<value>”来设置限额。其中value是一个正数，单位可以是\[kKmMgGtTpP]?\[iI]?\[bB]?，在不带单位的时候默认单位是字节。

```sh
# isula run -ti --storage-opt size=10M busybox
/ # df -h
Filesystem                Size      Used Available Use% Mounted on
overlay                  10.0M     48.0K     10.0M   0% /
none                     64.0M         0     64.0M   0% /dev
none                     10.0M         0     10.0M   0% /sys/fs/cgroup
tmpfs                    64.0M         0     64.0M   0% /dev
shm                      64.0M         0     64.0M   0% /dev/shm
/dev/mapper/vg--data-ext41
                          9.8G     51.5M      9.2G   1% /etc/hostname
/dev/mapper/vg--data-ext41
                          9.8G     51.5M      9.2G   1% /etc/resolv.conf
/dev/mapper/vg--data-ext41
                          9.8G     51.5M      9.2G   1% /etc/hosts
tmpfs                     3.9G         0      3.9G   0% /proc/acpi
tmpfs                    64.0M         0     64.0M   0% /proc/kcore
tmpfs                    64.0M         0     64.0M   0% /proc/keys
tmpfs                    64.0M         0     64.0M   0% /proc/timer_list
tmpfs                    64.0M         0     64.0M   0% /proc/sched_debug
tmpfs                     3.9G         0      3.9G   0% /proc/scsi
tmpfs                    64.0M         0     64.0M   0% /proc/fdthreshold
tmpfs                    64.0M         0     64.0M   0% /proc/fdenable
tmpfs                     3.9G         0      3.9G   0% /sys/firmware
/ # 
/ # dd if=/dev/zero of=/home/img bs=1M count=12 && sync
dm-4: write failed, project block limit reached.
10+0 records in
9+0 records out
10432512 bytes (9.9MB) copied, 0.011782 seconds, 844.4MB/s
/ # df -h | grep overlay
overlay                  10.0M     10.0M         0 100% /
/ # 
```

### 约束

1. 限额只针对rw层。

   overlay2的限额是针对容器的rw层的，镜像的大小不计算在内。

2. 内核支持并使能。

   内核必须支持ext4的project quota功能，并在mkfs的时候要加上-O quota,project，挂载的时候要加上-o prjquota。任何一个不满足，在使用--storage-opt size=\<value>时都将报错。

   ```sh
   # isula run -it --storage-opt size=10Mb busybox df -h
   Error response from daemon: Failed to prepare rootfs with error: time="2019-04-09T05:13:52-04:00" level=fatal msg="error creating read-
   write layer with ID "a4c0e55e82c55e4ee4b0f4ee07f80cc2261cf31b2c2dfd628fa1fb00db97270f": --storage-opt is supported only for overlay over
   xfs or ext4 with 'pquota' mount option"
   ```

3. 限制额度的说明。
   1. 限制的额度大于isulad的root所在分区的size时，在容器内用df看到的文件系统的额度是isulad的root所在分区的size，而不是设置的限额。
   2. \--storage-opt size=0代表不限制，且设置值不能小于4096。size的精度为1个字节，如果指定精度含小数个字节，小数部分被忽略，如指定size=0.1实际等同于size=0不限制。（受计算机存储浮点数精度的限制，即0.999999999999999999999999999与1是等价的，具体的9的个数不同计算机可能存在差异，故设置4095.999999999999999999999999999与4096等价，其他情况类似），注意isula inspect显示原始命令行指定形式，如果含小数字节，需自行忽略小数部分。
   3. 限制的额度过小时，比如--storage-opt size=4k，可能会导致容器无法启动，因为启动容器本身需要创建一些文件。
   4. 上一次启动isulad时，isulad的root所在分区挂载时加了-o prjquota选项，这次启动时不加，那么上一次启动中创建的带quota的容器的设置值不生效。
   5. daemon端配额--storage-opt overlay2.basesize，其取值范围与--storage-opt size相同。

4. 指定storage-opt为4k时，轻量级容器启动与docker有差异

   使用选项 storage-opt size=4k 和镜像 rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest 运行容器。

   docker启动失败。

   ```sh
   # docker run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest
   docker: Error response from daemon: symlink /proc/mounts /var/lib/docker/overlay2/e6e12701db1a488636c881b44109a807e187b8db51a50015db34a131294fcf70-init/merged/etc/mtab: disk quota exceeded.
   See 'docker run --help'.
   ```

   轻量级容器不报错，正常启动

   ```sh
   # isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest
   636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728
   # isula ps
   STATUS  PID   IMAGE                                                 COMMAND   EXIT_CODE RESTART_COUNT STARTAT       FINISHAT RUNTIME ID           NAMES                                                            
   running 17609 rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest /bin/bash 0         0             2 seconds ago -        lcr     636480b1fc2c 636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728 
   ```

   在启动容器的过程中，如果需要在容器的rootfs路径下创建文件，若镜像本身占用的大小超过4k，且此时的quota设置为4k，则创建文件必定失败。

   docker在启动容器的过程中，会比isulad创建更多的挂载点，用于挂载host上的某些路径到容器中，如/proc/mounts, /dev/shm等，如果镜像内本身不存在这些文件，则会创建，根据上述原因该操作会导致文件创建失败，因而容器启动失败。

   轻量级容器在启动容器过程中，使用默认配置时，挂载点较少，如/proc，或/sys等路径不存在时，才会创建。用例中的镜像rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest本身含有/proc， /sys等，因此整个启动容器的过程中，都不会有新文件或路径生成，故轻量级容器启动过程不会报错。为验证这一过程，当把镜像替换为rnd-dockerhub.huawei.com/official/busybox-aarch64:latest时，由于该镜像内无/proc存在，轻量级容器启动一样会报错。

   ```sh
   # isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/busybox-aarch64:latest
   8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4
   Error response from daemon: Start container error: runtime error: 8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4:tools/lxc_start.c:main:404 starting container process caused "Failed to setup lxc,
   please check the config file."
   ```

5. 其他说明。

   使用限额功能的isulad切换数据盘时，需要保证被切换的数据盘使用\`prjquota\`选项挂载，且/var/lib/isulad/storage/overlay2目录的挂载方式与/var/lib/isulad相同。

   > !\[NOTE]说明\
   > 切换数据盘时需要保证/var/lib/isulad/storage/overlay2的挂载点被卸载。

## 限制容器内文件句柄数

### 描述

可以通过参数限制容器中可以打开的文件句柄数。

### 用法

isula create/run时使用--files-limit来限制容器中可以打开的文件句柄数。

### 参数

create/run时指定--files-limit参数。

### 示例

在运行容器时，直接加上--files-limit n 即可，如：

```sh
isula run -ti --files-limit 1024 busybox bash
```

### 约束

1. 使用--files-limit参数传入一个很小的值，如1，可能导致容器启动失败。

   ```sh
   # isula run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64
   004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4
   Error response from daemon: Start container error: Container is not running:004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4
   ```

   而docker会启动成功，其files.limit cgroup值为max。

   ```sh
   # docker run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64
   ef9694bf4d8e803a1c7de5c17f5d829db409e41a530a245edc2e5367708dbbab
   # docker exec -it ef96 cat /sys/fs/cgroup/files/files.limit
   max
   ```

   根因是lxc和runc启动过程的原理不一样，lxc创建cgroup子组后先设置files.limit值，再将容器进程的PID写入该子组的cgroup.procs文件，此时该进程已经打开超过1个句柄，因而写入报错导致启动失败。runc创建cgroup子组后先将容器进程的PID写入该子组的cgroup.procs文件，再设置files.limit值，此时由于该子组内的进程已经打开超过1个句柄，因而写入files.limit不会生效，内核也不会报错，容器启动成功。

## 限制容器内可以创建的进程/线程数

### 描述

可以通过参数限制容器中能够创建的进程/线程数。

### 用法

在容器create/run时，使用参数--pids-limit来限制容器中可以创建的进程/线程数。

### 参数

create/run时指定--pids-limit参数。

### 示例

在运行容器时，直接加上--pids-limit n 即可，如：

```sh
isula run -ti --pids-limit 1024 busybox bash
```

### 约束

由于创建容器的过程中会临时创建一些进程，所以此值不能设置的太小，不然容器可能起不来，建议大于10。

## 配置容器内的ulimit值

### 描述

可以通过参数控制执行程序的资源。

### 用法

在容器create/run时配置--ulimit参数，或通过daemon端配置，控制容器中执行程序的资源。

### 参数

通过两种方法配置ulimit

1. isula create/run时使用--ulimit \<type>=\<soft>\[:\<hard>]来控制shell执行程序的资源。

2. 通过daemon端参数或配置文件

   详见"(命令行参数说明"与"部署方式"的--default-ulimits相关选项。

   \--ulimit可以对以下类型的资源进行限制。

### 示例

在容器的创建或者运行时，加上--ulimit \<type>=\<soft>\[:\<hard>]即可，如：

```sh
isula create/run -tid --ulimit nofile=1024:2048 busybox sh
```

### 约束

不能在daemon.json和/etc/sysconfig/iSulad文件（或isulad命令行）中同时配置ulimit限制，否则isulad启动会报错。

---

---
url: /zh/docs/22.03_LTS_SP4/cloud/image_builder/isula_build/overview.md
---
# 容器镜像构建

isula-build是iSula容器团队推出的容器镜像构建工具，支持通过Dockerfile文件快速构建容器镜像。

isula-build采用服务端/客户端模式，其中，isula-build为客户端，提供了一组命令行工具，用于镜像构建及管理等；isula-builder为服务端，用于处理客户端管理请求，作为守护进程常驻后台。

![isula-build architecture](./figures/isula-build_arch.png)

> \[!NOTE]说明
>
> isula-build当前支持OCI镜像格式（[OCI Image Format Specification](https://github.com/opencontainers/image-spec/blob/main/spec.md/)）以及Docker镜像格式（[Image Manifest Version 2, Schema 2](https://docs.docker.com/registry/spec/manifest-v2-2/)）。通过命令`export ISULABUILD_CLI_EXPERIMENTAL=enabled`开启实验特性以支持OCI镜像格式。不开启实验特性时，isula-build默认采用Docker镜像格式；当开启实验特性后，将默认采用OCI镜像格式。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/interconnect_libvirt.md
---
# 对接 libvirt

## 概述

libvirt 是一个管理 hypervisor 的上层软件，它通过不同的驱动统一管理不同类型的 hypervisor ，并对外提供统一、稳定的应用程序接口。

在云场景中，业内广泛使用 libvirt 管理大规模的虚拟机。为了方便地对大规模的 StratoVirt 虚拟机进行部署、编排和管理，StratoVirt 标准虚拟化支持对接 libvirt，打通了 libvirt 北向接口。用户可以通过 libvirt 对应的 XML 文件描述一个虚拟机，包括虚拟机名称、CPU、磁盘等。

本章介绍 StratoVirt 平台支持的 XML 配置，以及如何使用 virsh 命令管理虚拟机。

## 前提条件

StratoVirt 对接 libvirt，主机需要满足如下条件：

* 已正确配置 yum 源
* 已正确安装并启动 libvirt
* 已正确安装 StratoVirt

## 虚拟机配置

libvirt 工具采用 XML 格式的文件描述一个虚拟机特征，包括虚拟机名称、CPU、内存、磁盘、网卡等信息。用户可以通过修改配置文件，对虚拟机进行管理。

StratoVirt 对接 libvirt 之前，需要先配置 XML 文件。本小节介绍 StratoVirt 对接 libvirt 时支持的 XML 配置项以及配置方式。

> \[!NOTE]说明
>
> 使用 libvirt 管理 StratoVirt 虚拟机前，应该注意到 StratoVirt 当前支持的特性、特性之间的互斥关系、特性的配置前提条件、规格等，详细信息请参见命令行方式的 "[虚拟机配置](./vm_configuration.md)”章节。

### 虚拟机描述

虚拟机 XML 文件必须包含描述虚拟机的最基本元素： domain 和 name 。

#### 元素介绍

* domain：虚拟机配置的根元素，用于配置运行 StratoVirt 虚拟机的 hypervisor 类型。

  属性 type：domain 的类型，在 StratoVirt 虚拟化中，该值为 kvm 。

* name：虚拟机名称。

  虚拟机名称是一个长度不超过 255 字符的字符串。同一个主机上的虚拟机名称不能重复，虚拟机名称必须由数字、字母、“\_”、“－”、“:” 组成。

#### 配置示例

假设配置虚拟机名称为 StratoVirt ，示例为：

```shell
<domain type='kvm'>
    <name>StratoVirt</name>
    ... 
</domain>
```

### 虚拟CPU和内存

本节介绍虚拟 CPU 和虚拟内存的配置。

#### 元素介绍

* vcpu：虚拟处理器的个数。

* memory：虚拟内存大小。

  属性 unit ：指定内存单位，属性值支持 KiB（210 字节）、MiB（220 字节）、GiB（230 字节）、TiB（240 字节）等。

  > \[!NOTE]说明
  >
  > StratoVirt 暂不支持 CPU 拓扑结构，请勿配置该项。

#### 配置示例

配置 8GiB 内存，4 个虚拟处理器的示例如下：

```xml
<domain type='kvm'>
    ... 
    <vcpu>4</vcpu>
    <memory unit='GiB'>8</memory>
    ... 
</domain>
```

### 虚拟机设备

本节介绍如何使用 XML 文件配置虚拟机设备：磁盘，网卡，rng，balloon，console，vsock 设备。

#### 磁盘

##### 元素介绍

* 属性 type ：指定后端存储介质类型，在 StratoVirt 虚拟化中，该值为 file 。

  属性 device：呈现给虚拟机的存储介质类型，在 StratoVirt 虚拟化中，该值为 disk 。

* driver：指定后端驱动的详细信息。

  属性 type ：磁盘的格式类型，在 StratoVirt 虚拟化中，该值为 raw 。StratoVirt 当前只支持 raw 格式的磁盘。

  属性 iothread：为磁盘配置 iothread ，取值为 iothread 编号。在配置磁盘的 iothread 之前，需使用 iothread 元素配置 iothread 的个数。

* source： 指定后端存储介质。

  属性 file：指定磁盘路径。

* target：指定后端驱动的详细信息。

  属性 dev：指定磁盘名称。

  属性 bus：指定磁盘设备的类型，在 StratoVirt 虚拟化中，该值为 virtio 。

* iotune： 指定磁盘 IO 特性。

  属性 total\_iops\_sec：设置磁盘 iops 的值。

* address：用于设置设备所要挂载的总线属性。

  属性 type：总线类型，在 StratoVirt 虚拟化中，该值为 pci 。

  属性 domain：虚拟机的域。

  属性 bus：设备将要挂载的 bus 号。

  属性 slot：设备将要挂载的 slot 号，取值范围为：\[0, 31] 。

  属性 function：设备将要挂载的 function 号，取值范围为：\[0, 7] 。

##### 配置示例

配置磁盘路径为：`/home/openEuler-22.03-LTS-SP4-stratovirt.img`，配置 1 个 iothread，并且磁盘 iothread 配置为 iothread1 ，iops 为 10000，并将其挂载在 bus 为 1、slot 为 0，function 为 0  的 PCI 总线上，示例为：

```xml
<domain type='kvm'>
    ...
    <iothreads>1</iothreads>
    <devices>
        <disk type='file' device='disk'>
     <driver name='qemu' type='raw' iothread="1"/>
     <source file='/home/openEuler-22.03-LTS-SP4-stratovirt.img'/>
     <target dev='hda' bus='virtio'/>
        <iotune>
         <total_iops_sec>10000</total_iops_sec>    
        </iotune>
     <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x00' />
 </disk>
    ...
    </devices>
</domain>
```

#### 网络设备

##### 元素介绍

* interface：网络接口

  属性 type：指定网络设备类型。

* mac：虚拟网卡地址

  属性 address：虚拟网卡地址。

* source： 指定后端网桥

  属性 bridge：指定网桥。

* target：指定后端网卡

  属性 dev：指定后端的 tap 设备。

* model： 虚拟网卡类型

  属性 type： 虚拟网卡类型，在 StratoVirt 虚拟化中，该值为 virtio。

* driver：用来指定是否开启 vhost 。

  属性 name：如果设置 name 为 qemu 则使用 virtio-net 设备，如果不配置 driver 或者 name 值为 vhost ，则使用 vhost-net 设备。

##### 配置示例

配置网络前请参考 [配置linux网桥](https://docs.openeuler.org/zh/docs/22.03_LTS_SP4/docs/Virtualization/%E5%87%86%E5%A4%87%E4%BD%BF%E7%94%A8%E7%8E%AF%E5%A2%83.html#%E5%87%86%E5%A4%87%E8%99%9A%E6%8B%9F%E6%9C%BA%E7%BD%91%E7%BB%9C)，配置好 Linux 网桥。配置 mac 地址为：`de:ad:be:ef:00:01`，网桥为配置好的 br0 ，使用 virtio-net 设备，并将其挂载在 bus 为 2、slot 为 0，function 为 0  的 PCI 总线上，示例为：

```xml
<domain type='kvm'>
    ...
    <devices>
        <interface type='bridge'>
            <mac address='de:ad:be:ef:00:01' />
            <source bridge='br0'/> 
            <model type='virtio'/>
            <driver name='qemu'/>
            <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x00' />
        </interface>
        ...
    </devices>
</domain>
```

#### balloon 设备

##### 元素介绍

* memballoon：balloon 设备类型

  属性 model ：指定 balloon 设备类型，在 StratoVirt 虚拟化中，该值为 virtio 。

* alias：balloon 设备的别名

  属性 name ：balloon 设备的 id 。

  属性 autodeflate ：设置 auto deflate（自动收缩）特性，可选值为：`on` 、`off` 。

##### 配置示例

配置 balloon 设备，开启 autodeflate 特性，并将其挂载在 bus 为 3、slot 为 0，function 为 0  的 PCI 总线上，示例为：

```xml
<domain type='kvm'>
    ...
    <devices>
        <memballoon model='virtio'>
            <alias name='balloon0' autodeflate='on'/>
            
            <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x00' />
        </memballoon>
        ...
    </devices>
</domain>
```

#### console 设备

由于 console 设备挂载在 virtio-serial 下的总线上，所以在创建 console 设备时，需要创建 virtio-serial 设备。

> \[!NOTE]说明
>
> StratoVirt 的 console 设备暂时不支持多端口特性，每个虚拟机只能配置一个 console 设备。

##### 元素介绍

* controller：控制器

  属性 type：控制器类型，此处值为 virtio-serial 。

* alias：别名

  属性 name：设备的 id。

* console：console 设备

  属性 type：指定 console 设备的重定向方式。支持的重定向方式有：pty ， file 和 unix 。

* target：配置 console 设备。

  属性 type：指定 console 设备类型，在 StratoVirt 虚拟化中，该值为 virtio 。

##### 配置示例

配置重定向方式为 pty ，并将其挂载在 bus 为 4、slot 为 0，function 为 0  的 PCI 总线上，示例为：

```xml
<domain type='kvm'>
    ...
    <devices>
        <controller type='virtio-serial'>
            <alias name='virt-serial0'/>
            <address type='pci' domain='0x000' bus='0x04' slot='0x00' function='0x00'/>
        </controller>
        <console type='pty'>
            <target type='virtio'/>
            <alias name='console0'/>
        </console>
        ...
    </devices>
</domain>
```

#### rng 设备

##### 元素介绍

* rng：rng 设备

  属性 model：指定 rng 设备类型，在 StratoVirt 虚拟化中，该值为 virtio 。

* rate：rng 设备产生随机数速率

  属性 period ：用于设置随机数产生周期，单位为毫秒，当前 StratoVirt 不支持设置周期值，默认值为 1000 毫秒，请将该值设置为 1000。

  属性 bytes ：周期内产生的最大字节数。

* backend：设置 rng 设备后端，值为 host 中 rng 设备的路径

  属性 model：用于指定后端设备类型，在 StratoVirt 虚拟化中，该值为 random 。

##### 配置示例

配置周期为 1000ms 内最多产生 1234 字节，rng 设备在 host 中路径为 `/dev/random` ，并将其挂载在 bus 为 5、slot 为 0，function 为 0  的 PCI 总线上，示例为：

```xml
<domain type='kvm'>
    ...
    <devices>
        <rng model='virtio'>
            <rate period='1000' bytes='1234'/>
            <backend model='random'>/dev/random</backend>
            <address type='pci' domain='0x000' bus='0x05' slot='0x00' function='0x00'/>
        </rng>
        ...
    </devices>
</domain>
```

#### vsock 设备

##### 元素介绍

* vsock：vsock 设备

  属性 model：指定 vsock 设备类型，在 StratoVirt 虚拟化中，该值为 virtio 。

* cid：设置 vsock 设备的 cid

  属性 address：用于设置 cid 的值

##### 配置示例

配置 cid 为 8，并将其挂载在 bus 为 6、slot 为 0，function 为 0  的 PCI 总线上，示例为：

```xml
<domain type='kvm'>
    ...
    <devices>
        <vsock model='virtio'>
            <cid auto='no' address='8'/>
            <address type='pci' domain='0x000' bus='0x06' slot='0x00' function='0x00'/>
        </vsock>
        ...
    </devices>
</domain>
```

### 体系架构相关配置

XML 中还有一些体系架构相关的配置，如 pflash、主板等。

#### 元素介绍

* os：定义虚拟机启动参数

  子元素 type ：指定虚拟机类型，属性 arch 表示架构，属性 machine 表示主板类型，在 StratoVirt 虚拟化中，AArch64 架构只支持 virt 主板，x86\_64 架构只支持 Q35 主板。

  子元素 kernel ：用于指定 kernel 路径。

  子元素 cmdline ：指定命令行启动参数。

  子元素 loader ：指定加载固件，属性 readonly 表示是否只读；属性 type 表示类型，在 StratoVirt 虚拟化中，该值为 pflash 。

* features：hypervisor 支持的以下特性

  子元素 acpi ：是否支持 ACPI，在 StratoVirt 虚拟化中使用了 ACPI 特性，所以该特性必须配置。

  子元素 gic ：ARM 处理器指定中断处理器，属性 version 表示 GIC 的版本，在 StratoVirt 虚拟化中，该值为 3 。

##### 配置示例

配置虚拟机 CPU 架构  ARM，主板为 virt ，启动命令行为：`console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw` 。pflash 路径为：`/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw`，属性为只读。 kernel 路径为：`/home/std-vmlinuxz`。示例为：

```xml
<domain type='kvm'>
    ...
    <os>
        <type arch='aarch64' machine='virt'>hvm</type>
        <kernel>/home/std-vmlinuxz</kernel>
        <cmdline>console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw</cmdline>
        <loader readonly='yes' type='pflash'>`/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw`</loader>
    </os>
    ...
</domain>
```

### 内存大页

#### 元素介绍

* memoryBacking：表示配置内存相关的信息

* hugepages：配置内存大页

* page：大页配置

  属性 size ：内存大页的大小

  属性 unit ：大页大小的单位

#### 配置示例

配置 2MiB 大页示例如下：

```xml
<domain type='kvm'>
    ...
    <memoryBacking>
    <hugepages>
        <page size='2' unit='MiB' />
    </hugepages>
    </memoryBacking>
    ...
</domain>
```

### 配置示例

#### x86 配置示例

配置一台名为 StratoVirt ，内存 8GiB ，配置 1GiB 单位的内存大页，4 个虚拟 CPU，架构为 x86\_64 ，主板类型为 Q35  ，对应 XML 文件的配置示例如下：

```xml
<domain type='kvm'>
    <name>StratoVirt</name>
    <memory unit='GiB'>8</memory>
    <!-- hugepages -->
    <memoryBacking>
    <hugepages>
        <page size='1' unit='GiB' />
    </hugepages>
    </memoryBacking>
    <vcpu>4</vcpu>
    <!-- iothread -->
    <iothreads>1</iothreads>
    <os>
    <type arch='x86_64' machine='q35'>hvm</type>
    <kernel>/path/to/standard_vm_kernel</kernel>
    <cmdline>console=hvc0 root=/dev/vda reboot=k panic=1 rw</cmdline>
    <loader readonly='yes' type='pflash'>/path/to/pflash</loader>
    <nvram>/path/to/OVMF_VARS</nvram>
    </os>
    <features>
        <acpi/>
    </features>
    <devices>
        <emulator>/path/to/StratoVirt_binary_file</emulator>
        <controller type='pci' index='0' model='pcie-root' />
        <!-- pcie-root-port -->
        <controller type='pci' index='1' model='pcie-root-port' />
        <controller type='pci' index='2' model='pcie-root-port' />
        <controller type='pci' index='3' model='pcie-root-port' />
        <controller type='pci' index='4' model='pcie-root-port' />
        <controller type='pci' index='5' model='pcie-root-port' />
        <controller type='pci' index='6' model='pcie-root-port' />
        <controller type='pci' index='7' model='pcie-root-port' />
        <!-- block -->
        <disk type='file' device='disk'>
            <driver name='qemu'  type='raw' iothread='1'/>
            <source file='/path/to/rootfs'/>
            <target dev='hda' bus='virtio'/>
            <iotune>
                <total_iops_sec>1000</total_iops_sec>
            </iotune>
            <address type='pci' domain='0x000' bus='0x01' slot='0x00' function='0x00'/>
        </disk>
        <!-- net -->
        <interface type='bridge'>
            <mac address='de:ad:be:ef:00:01'/>
            <source bridge='qbr0'/>
            <model type='virtio'/>
            <address type='pci' domain='0x000' bus='0x02' slot='0x00' function='0x00'/>
        </interface>
        <!-- virtio-serial -->
        <controller type='virtio-serial' index='0'>
            <alias name='virt-serial0'/>
            <address type='pci' domain='0x000' bus='0x03' slot='0x00' function='0x00'/>
        </controller>
        <console type='pty'>
            <target type='virtio' port='0'/>
            <alias name='console0'/>
        </console>
        <!-- balloon -->
        <memballoon model='virtio'>
            <alias name='balloon0'/>
            <address type='pci' domain='0x000' bus='0x04' slot='0x00' function='0x00'/>
        </memballoon>
        <!-- rng -->
        <rng model='virtio'>
            <rate period='1000' bytes='1234'/>
            <backend model='random'>/path/to/random_file</backend>
            <address type='pci' domain='0x000' bus='0x05' slot='0x00' function='0x00'/>
        </rng>
        <!-- vsock -->
        <vsock model='virtio'>
            <cid auto='no' address='3'/>
            <address type='pci' domain='0x000' bus='0x06' slot='0x00' function='0x00'/>
        </vsock>
    </devices>
</domain>
```

#### ARM 配置示例

如果想要配置一台名为 StratoVirt ，内存 8GiB，配置 1GiB 单位大页，4 个虚拟 CPU，架构为 aarch64 ，主板类型为 virt ，对应 XML 文件的配置示例如下：

```xml
<domain type='kvm'>
    <name>StratoVirt</name>
    <memory unit='GiB'>8</memory>
    <!-- hugepages -->
    <memoryBacking>
    <hugepages>
        <page size='1' unit='GiB' />
    </hugepages>
    </memoryBacking>
    <vcpu>4</vcpu>
    <!-- iothread -->
    <iothreads>1</iothreads>
    <os>
    <type arch='aarch64' machine='virt'>hvm</type>
    <kernel>/path/to/standard_vm_kernel</kernel>
    <cmdline>console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw</cmdline>
    <loader readonly='yes' type='pflash'>/path/to/pflash</loader>
    </os>
    <features>
        <acpi/>
        <gic version='3'/>
    </features>
    <devices>
        <emulator>/path/to/StratoVirt_binary_file</emulator>
        <controller type='pci' index='0' model='pcie-root'/>
        <!-- block -->
        <disk type='file' device='disk'>
            <driver name='qemu'  type='raw' iothread='1'/>
            <source file='/path/to/rootfs'/>
            <target dev='hda' bus='virtio'/>
            <address type='pci' domain='0x000' bus='0x00' slot='0x01' function='0x00'/>
            <iotune>
                <total_iops_sec>1000</total_iops_sec>
            </iotune>
        </disk>
        <!-- net -->
        <interface type='bridge'>
            <mac address='de:ad:be:ef:00:01'/>
            <source bridge='qbr0'/>
            <model type='virtio'/>
            <address type='pci' domain='0x000' bus='0x00' slot='0x02' function='0x00'/>
        </interface>
        <!-- virtio-serial -->
        <controller type='virtio-serial' index='0'>
            <alias name='virt-serial0'/>
            <address type='pci' domain='0x000' bus='0x00' slot='0x03' function='0x00'/>
        </controller>
        <console type='pty'>
            <target type='virtio' port='0'/>
            <alias name='console0'/>
        </console>
        <!-- balloon -->
        <memballoon model='virtio'>
            <alias name='balloon0'/>
            <address type='pci' domain='0x000' bus='0x00' slot='0x04' function='0x00'/>
        </memballoon>
        <!-- rng -->
        <rng model='virtio'>
            <rate period='1000' bytes='1234'/>
            <backend model='random'>/path/to/random_file</backend>
            <address type='pci' domain='0x000' bus='0x00' slot='0x05' function='0x00'/>
        </rng>
        <!-- vsock -->
        <vsock model='virtio'>
            <cid auto='no' address='3'/>
            <address type='pci' domain='0x000' bus='0x00' slot='0x06' function='0x00'/>
        </vsock>
    </devices>
</domain>
```

## 管理虚拟机

libvirt 使用 virsh 命令来管理虚拟机，当 StratoVirt 平台和 libvirt 对接时，仅支持以下与 StratoVirt 交互的命令：

* create：创建虚拟机

* suspend：挂起虚拟机

* resume：恢复虚拟机

* destroy：销毁虚拟机

* console：通过 console 登录虚拟机

> \[!NOTE]说明
>
> StratoVirt 暂不支持虚拟机重启、虚拟机关机等命令。

### 管理虚拟机生命周期

假设用户已经按照需要完成一个名为 StratoVirt 的虚拟机配置文件 st.xml ，则对应生命周期管理的命令如下：

* 创建虚拟机

  ```shell
  virsh create st.xml
  ```

  虚拟机创建完成后，可以通过 **virsh list** 命令查看，会存在一个名为 StratoVirt 的虚拟机。

* 挂起虚拟机

  ```shell
  virsh suspend StratoVirt
  ```

  虚拟机挂起后，虚拟机暂停运行。可以通过 **virsh list** 命令查看，虚拟机 StratoVirt 的状态为 paused 。

* 恢复虚拟机

  ```sh
  virsh resume StratoVirt
  ```

  虚拟机恢复后，可以通过 **virsh list** 命令查看，虚拟机 StratoVirt 的状态为 running 。

* 销毁虚拟机

  ```sh
  virsh destroy StratoVirt
  ```

  虚拟机销毁后，使用 **virsh list** 查看虚拟机，发现虚拟机 StratoVirt 不存在。

### 登录虚拟机

虚拟机创建完成后，可以通过 **virsh console** 登录虚拟机内部操作虚拟机。假设虚拟机名称为 StratoVirt，参考命令如下：

```sh
virsh console StratoVirt
```

> \[!NOTE]说明
>
> 为了可以正常使用 virsh console 命令，需要在 XML 中配置 console 设备的重定向类型为  pty 。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/stratovirt/interconnect_isula.md
---
# 对接iSula安全容器

## 概述

为了给容器提供更好的隔离环境，提高系统安全性，可以使用 iSula 安全容器，即通过 StratoVirt 对接 iSula 安全容器。

## 对接iSula安全容器

### **前提条件**

已安装 iSulad 和 kata-containers，并确保 iSulad 支持 containerd-kata-shim-v2 容器运行时和 devicemapper 存储驱动。

此处给出安装 iSulad 和 kata-containers 并进行相应配置的参考方法。

1. 配置 yum 源，使用 root 权限安装 iSulad 和 kata-containers ：

   ```shell
   # yum install iSulad
   # yum install kata-containers
   ```

2. 制作并配置存储 Storage：

   需要用户规划好磁盘如/dev/sdxx,该磁盘会被格式化。

   ```shell
    # pvcreate /dev/sdxx
    # vgcreate isulaVG0 /dev/sdxx
    # lvcreate --wipesignatures y -n thinpool isulaVG0 -l 95%VG
    # lvcreate --wipesignatures y -n thinpoolmeta isulaVG0 -l 1%VG
    # lvconvert -y --zero n -c 512K --thinpool isulaVG0/thinpool --poolmetadata isulaVG0/thinpoolmeta
   ```

   在配置文件 /etc/lvm/profile/isulaVG0-thinpool.profile 中添加如下：

   ```conf
   activation {
       thin_pool_autoextend_threshold=80
       thin_pool_autoextend_percent=20
   }
   ```

   更改配置文件/etc/isulad/daemon.json中的storage-driver 和 storage-opts 如下：将默认存储驱动类型 overlay 配置成 devicemapper 。

   ```conf
   "storage-driver": "devicemapper",
   "storage-opts": [
    "dm.thinpooldev=/dev/mapper/isulaVG0-thinpool",
    "dm.fs=ext4",
    "dm.min_free_space=10%"
   ],
   ```

3. 重启 isulad ：

   ```shell
    # systemctl daemon-reload
    # systemctl restart isulad
   ```

4. 确认 iSula 存储驱动是否配置成功：

   ```shell
    # isula info
   ```

   若回显有如下信息，说明配置成功。

   ```text
   Storage Driver: devicemapper
   ```

### **对接指导**

StratoVirt 通过对接 kata-containers来接入 isula 容器生态，此处给出对接 kata-containers 的操作指导。

#### 对接轻量虚拟机

1. 修改 kata 配置文件（默认路径为 /usr/share/defaults/kata-containers/configuration.toml，也可以参考同一目录下的configuration-stratovirt.toml进行配置） 。将安全容器的 hypervisor 类型修改为 stratovirt，kernel 修改为 kata-containers 的 kernel 镜像绝对路径，initrd 修改为 kata-containers 的 initrd 镜像文件（使用 yum 安装 kata-containers 时，默认会下载这两个镜像文件并存放在 /var/lib/kata/ 目录，配置时也可以使用其他镜像 ）。

   替换的配置内容参考如下：

   ```shell
   [hypervisor.stratovirt]
   path = "/usr/bin/stratovirt"
   kernel = "/var/lib/kata/kernel"
   initrd = "/var/lib/kata/kata-containers-initrd.img"
   machine_type = "microvm"
   block_device_driver = "virtio-mmio"
   use_vsock = true
   enable_netmon = true
   internetworking_model="tcfilter"
   sandbox_cgroup_with_emulator = false
   disable_new_netns = false
   disable_block_device_use = false
   disable_vhost_net = true
   ```

2. 使用 root 权限和 **isula** 命令运行 busybox 安全容器，完成 StratoVirt 和 安全容器的对接。

   ```shell
   # isula run -tid --runtime "io.containerd.kata.v2" --net=none --name test busybox:latest sh
   ```

3. 使用 **isula ps** 确认安全容器 test 正常运行，然后通过以下命令进入 test 容器。

   ```shell
   # isula exec -ti test sh
   ```

4. 通过虚拟机快照加速安全容器的启动速度，降低虚拟机内存开销。

   修改 kata 配置文件 configuration.toml，将配置项 enable\_template 设置为 true，即允许虚拟机通过制作快照方式进行启动：

   ```shell
   [factory]
   # VM templating support. Once enabled, new VMs are created from template
   # using vm cloning. They will share the same initial kernel, initramfs and
   # agent memory by mapping it readonly. It helps speeding up new container
   # creation and saves a lot of memory if there are many kata containers running
   # on the same host.
   #
   # When disabled, new VMs are created from scratch.
   #
   # Note: Requires "initrd=" to be set ("image=" is not supported).
   #
   # Default false
   enable_template = true
   ```

   配置项 enable\_template 设置为 true 后，kata-containers 创建安全容器时，将会检查默认路径（/run/vc/vm/template）下是否存在快照文件，如果存在，直接以该快照文件启动虚拟机，如果不存在，则会创建虚拟机快照，创建完成后，以该快照文件启动虚拟机。

5. 通过安全组件 ozone 进一步增强安全容器的隔离性。

   修改 kata 配置文件 configuration.toml，将配置项 ozone\_path 设置为 ozone 可执行文件的路径（如果使用 yum 安装 stratovirt，ozone 可执行文件默认在 /usr/bin 目录下）。配置该项后，将打开 ozone 安全沙箱功能，作为虚拟化层隔离被攻击者突破后的保险，进一步增强 StratoVirt 安全容器的隔离性:

   ```shell
   # Path for the ozone specific to stratovirt
   # If the ozone path is set, stratovirt will be launched in
   # ozone secure environment. It is disabled by default.
   ozone_path = "/usr/bin/ozone"
   ```

至此，可以在 test 容器内运行容器命令。

#### 对接标准虚拟机

使用 StratoVirt 标准虚拟机作为安全容器的 sandbox，需要额外修改少量配置。具体步骤如下：

1. 配置参考如下：

   ```shell
   [hypervisor.stratovirt]
   path = "/usr/bin/stratovirt"
   kernel = "/var/lib/kata/kernel"
   initrd = "/var/lib/kata/kata-containers-initrd.img"
   # x86_64 架构
   machine_type = "q35"
   # aarch64 架构
   machine_type = "virt"
   block_device_driver = "virtio-blk"
   pcie_root_port = 2
   use_vsock = true
   enable_netmon = true
   internetworking_model = "tcfilter"
   sandbox_cgroup_with_emulator = false
   disable_new_netns = false
   disable_block_device_use = false
   disable_vhost_net = true
   ```

   上述配置中，需要根据主机架构，修改对应的虚拟机机型。需要将 block\_device\_driver 驱动类型改为 virtio-blk。另外，StratoVirt 只支持将设备热插到root port，根据需要热插的设备数量，合理设置 pcie\_root\_port 值。

2. 安装启动标准虚拟机需要的固件

   x86\_64 架构：

   ```shell
   # yum install -y edk2-ovmf
   ```

   aarch64 架构：

   ```shell
   # yum install -y edk2-aarch64
   ```

3. 编译替换为 kata-containers 2.x 版本二进制

   当前只为 kata-containers 2.x 版本（对应 kata-containers 源码仓的 openEuler-21.09 分支）适配了 StratoVirt 标准虚拟机作为 sandbox。因此，需要手动下载 kata-containers 源码，编译并替换 `/usr/bin` 目录下的 containerd-shim-kata-v2 二进制文件。

   ```shell
   # mkdir -p /root/go/src/github.com/
   # cd /root/go/src/github.com/
   # git clone https://atomgit.com/src-openeuler/kata-containers.git
   # cd kata-containers
   # git checkout openEuler-21.09
   # ./apply-patches
   # cd src/runtime
   # make
   ```

   编译出的二进制位 containerd-shim-kata-v2，需要将默认 `/usr/bin/` 目录的 kata 二进制备份后替换：

   ```shell
   # cp /usr/bin/containerd-shim-kata-v2 /usr/bin/containerd-shim-kata-v2.bk
   # cp containerd-shim-kata-v2 /usr/bin/containerd-shim-kata-v2
   ```

4. 使用 root 权限 和 **isula** 命令运行 busybox 安全容器，完成 StratoVirt 和 安全容器的对接。

   ```shell
   # isula run -tid --runtime "io.containerd.kata.v2" --net=none --name test busybox:latest sh
   ```

5. 使用 **isula ps** 确认安全容器 test 正常运行，然后通过以下命令进入 test 容器。

   ```shell
   # isula exec -ti test sh
   ```

---

---
url: /zh/docs/22.03_LTS_SP4/embedded.md
---


---

---
url: >-
  /zh/docs/22.03_LTS_SP4/cloud/cluster_deployment/kubernetes/eggo_tool_introduction.md
---
# 工具介绍

本章介绍自动化部署工具的相关内容，建议用户在部署前阅读。

## 部署方式

openEuler 提供的 Kubernetes 集群自动化部署工具使用命令行方式进行集群的一键部署。它提供了如下几种部署方式：

* 离线部署：本地准备好所有需要用到的 RPM 软件包、二进制文件、插件、容器镜像，并将它们按照一定的格式打包成一个 tar.gz 文件，然后完成对应 YAML 配置文件的编写，即可执行命令实现一键部署。当虚拟机无法访问外部网络时，可以采用该部署方式。
* 在线部署：只需要完成对应 YAML 配置文件的编写，所需的RPM 软件包、二进制文件、插件、容器镜像，都在安装部署阶段连接互联网自动下载。该方式需要虚拟机能够访问软件源、集群依赖的镜像仓库，例如 Docker Hub 。

## 配置介绍

使用工具自动化部署 Kubernetes 集群时，使用 YAML 配置文件描述集群部署的信息，此处介绍各配置项含义以及配置示例。

### 配置项介绍

* cluster-id：集群名称，请遵循 DNS 域名的命名规范。例如 k8s-cluster

* username：需要部署 k8s 集群的机器的 ssh 登录用户名，所有机器都需要使用同一个用户名。

* private-key-path：ssh 免密登录的密钥存储文件的路径。private-key-path 和 password 只需要配置其中一项，如果两者都进行了配置，优先使用 private-key-path

* masters：master 节点列表，建议每个 master 节点同时作为 worker 节点。每个 master 节点包含如下配置子项，多个 master 节点配置多组子项内容：
  * name：master 节点名称，为 k8s 集群看到的该节点名称
  * ip：master 节点的 IP 地址
  * port：ssh 登录该节点的端口，默认为 22
  * arch：master 节点的 CPU 架构，例如 x86\_64 取值为 amd64

* workers：worker 节点列表。每个 worker 节点包含如下配置子项，多个 worker 节点配置多个子项内容：
  * name：worker 节点名称，为 k8s 集群看到的该节点名称
  * ip：worker 节点的 IP 地址
  * port：ssh 登录该节点的端口，默认为 22
  * arch：worker 节点的 CPU 架构，例如 x86\_64 取值为 amd64

* etcds：etcd 节点的列表。如果该项为空，则会为每个 master 节点部署一个 etcd，否则只会部署配置的 etcd 节点。每个 etcd 节点包含如下配置子项，多个 etcd 节点配置多组子项内容：
  * name：etcd 节点的名称，为 k8s 集群看到的该节点的名称
  * ip：etcd 节点的 IP 地址
  * port：ssh 登录的端口
  * arch：etcd 节点的 CPU 架构，例如 x86\_64 取值为 amd64

* loadbalance：loadbalance 节点列表。每个 loadbalance 节点包含如下配置子项，多个 loadbalance 节点配置多组子项内容：
  * name：loadbalance 节点的名称，为 k8s 集群看到的该节点的名称
  * ip：loadbalance 节点的 IP 地址
  * port：ssh 登录的端口
  * arch：loadbalance 节点的 CPU 架构，例如 x86\_64 取值为 amd64
  * bind-port：负载均衡服务的侦听端口

* external-ca：是否使用外部 CA 证书，使用则配置为 true，反之，配置为 false

* external-ca-path：外部 CA 证书文件的路径 。仅  external-ca 为 true 时有效

* service：k8s 创建的 service 信息。service 配置包含如下配置子项：
  * cidr：k8s 创建的 service 的 IP 地址网段
  * dnsaddr：k8s 创建的 service 的 DNS 地址
  * gateway：k8s创建的 service 的网关地址
  * dns：k8s 创建的 coredns 的配置。dns 配置包含如下配置子项：
    * corednstype：k8s 创建的 coredns 的部署类型，支持 pod 和 binary
    * imageversion：pod 部署类型的 coredns 镜像版本
    * replicas：pod 部署类型的 coredns 副本数量

* network：k8s 集群网络配置。network 配置包含如下配置子项：
  * podcidr：k8s 集群网络的 IP 地址网段
  * plugin：k8s 集群部署的网络插件
  * plugin-args：k8s 集群网络的网络插件的配置文件路径。例如 : {"NetworkYamlPath": "/etc/kubernetes/addons/calico.yaml"}

* apiserver-endpoint：进群外部可访问的 APISERVER 服务的地址或域名，如果配置了 loadbalances 则填loadbalance 地址，否则填写第 1 个 master 节点地址。

* apiserver-cert-sans：apiserver 相关证书中需要额外配置的 IP 和域名。它包含如下子配置项
  * dnsnames：apiserver 相关证书中需要额外配置的域名数组列表。
  * ips：apiserver 相关证书中需要额外配置的 IP 地址数组列表。

* apiserver-timeout：apiserver 响应超时时间

* etcd-token：etcd 集群名称

* dns-vip：dns 的虚拟 IP 地址

* dns-domain：DNS 域名后缀

* pause-image：pause 容器的完整镜像名称

* network-plugin：网络插件类型。仅支持配置 cni ，配置为空时使用 k8s 默认网络。

* cni-bin-dir：网络插件地址，多个地址使用 "," 分隔，例如：/usr/libexec/cni,/opt/cni/bin

* runtime：指定容器运行时类型，目前支持 docker 和 iSulad

* runtime-endpoint：容器运行时 endpoint，当 runtime 为 docker 时，可以不指定

* registry-mirrors：下载容器镜像时，使用的镜像仓库的 mirror 站点地址

* insecure-registries：下载容器镜像时，使用 http 协议下载镜像的镜像仓库地址

* config-extra-args：各个组件（例如 kube-apiserver、etcd）服务启动配置的额外参数。它包含如下子配置项：
  * name：组件名称，支持 etcd、kube-apiserver、kube-controller-manager、kube-scheduler、kube-proxy、kubelet

  * extra-args：组件的拓展参数，格式为 key: value 格式，注意 key 对应的组件参数前需要加上 "-" 或者 "--" 。

  * open-ports：配置需要额外打开的端口，k8s 自身所需端口不需要进行配置，k8s 以外的插件端口需要进行额外配置。
    * worker | master | etcd | loadbalance：指定打开端口的节点类型，每项配置包含一个多或者多个 port 和 protocol 子配置项。
      * port：端口地址
      * protocol：端口类型，可选值为 tcp 或者 udp

  * install：配置各种类型节点上需要安装的安装包或者二进制文件的详细信息，注意将对应文件放到在 tar.gz 安装包中。以下为全量配置说明，具体配置请根据实际情况选择。
    * package-source：配置安装包的详细信息
      * type：安装包的压缩类型，目前只支持 tar.gz 类型的安装包
      * dstpath：安装包在对端机器上的路径，必须是可用的绝对路径
      * srcpath：不同架构安装包的存放路径，架构必须与机器架构相对应，必须是可用的绝对路径
        * arm64：arm64 架构安装包的路径，配置的机器中存在 arm64 机器场景下需要配置
        * amd64：amd64 类型安装包的路径，配置的机器中存在 x86\_64 机器场景下需要配置
    > \[!NOTE]说明
    >
    > install 配置中 etcd、kubernetes-master、kubernetes-worker、network、loadbalance、container、image、dns 中的子配置项相同，都是 name、type、dst，schedule、TimeOut 。其中 dst，schedule、TimeOut 为可选项，用户根据安装的文件决定是否配置。下述仅以 etcd 和 kubernetes-master 节点的配置为例说明。
    * etcd：etcd 类型节点需要安装的包或二进制文件列表
      * name：需要安装的软件包或二进制文件的名称，如果是安装包则只写名称，不填写具体的版本号，安装时会使用 `$name*` 识别，例如 etcd 。如果为多个软件包，各名称使用 ，分隔 。
      * type：配置项类型，可选值为 pkg、repo、bin、file、dir、image、yaml、shell 。如果配置为 repo ，请在对应节点上配置 repo 源
      * dst：目的文件夹路径，type 为 bin、file、dir 类型时需要配置。表示将文件/文件夹放到节点的哪个目录下，为了防止用户误配置路径，导致 cleanup 时删除重要文件，此配置必须配置为白名单中的路径。详见 “白名单说明”
    * kubernetes-master：k8s master 类型节点需要安装的包或二进制文件列表
    * kubernetes-worker：k8s worker 类型节点需要安装的包或二进制文件列表
    * network：网络需要安装的包或二进制文件列表
    * loadbalance：loadbalance 类型节点需要安装的包或二进制文件列表
    * container：容器需要安装的包或二进制文件列表
    * image：容器镜像 tar 包
    * dns：k8s coredns 安装包。如果 corednstype 配置为 pod，此处无需配置
    * addition：额外的安装包或二进制文件列表
      * master：以下配置会安装在所有 master 节点
        * name：需要安装的软件包或二进制文件的名称
        * type：配置项类型，可选值为 pkg、repo、bin、file、dir、image、yaml、shell 。如果配置为 repo ，请在对应节点上配置 repo 源
        * schedule：仅在 type 为 shell 时有效，代表用户想要执行脚本的时机，支持 prejoin（节点加入前）、postjoin（节点加入后）、precleanup（节点退出前）、postcleanup（节点退出后）。
        * TimeOut：脚本执行超时时间，超时时该进程被强制终止运行。未配置默认为 30s
      * worker：配置会安装在所有 worker 节点，具体配置格式和 addition 下的 master 相同

### 白名单介绍

install 配置中 dst 项的值必须符合白名单规则，配置为白名单对应路径及其子目录。当前白名单如下：

* /usr/bin
* /usr/local/bin
* /opt/cni/bin
* /usr/libexec/cni
* /etc/kubernetes
* /usr/lib/systemd/system
* /etc/systemd/system
* /tmp

### 配置示例

此处给出一个 YAML 文件配置示例。从示例可知，同一台机器，可以部署多个类型的节点，但是不同节点的配置必须一致，例如 test0 机器部署了 master 和 worker 类型。

```yaml
cluster-id: k8s-cluster
username: root
private-key-path: /root/.ssh/private.key
masters:
- name: test0
  ip: 192.168.0.1
  port: 22
  arch: arm64
workers:
- name: test0
  ip: 192.168.0.1
  port: 22
  arch: arm64
- name: test1
  ip: 192.168.0.3
  port: 22
  arch: arm64
etcds:
- name: etcd-0
  ip: 192.168.0.4
  port: 22
  arch: amd64
loadbalance:
  name: k8s-loadbalance
  ip: 192.168.0.5
  port: 22
  arch: amd64
  bind-port: 8443
external-ca: false
external-ca-path: /opt/externalca 
service:
  cidr: 10.32.0.0/16
  dnsaddr: 10.32.0.10
  gateway: 10.32.0.1
  dns:  
    corednstype: pod
    imageversion: 1.8.4
    replicas: 2
network:
  podcidr: 10.244.0.0/16
  plugin: calico
  plugin-args: {"NetworkYamlPath": "/etc/kubernetes/addons/calico.yaml"}  
apiserver-endpoint: 192.168.122.222:6443      
apiserver-cert-sans:
  dnsnames: []
  ips: []
apiserver-timeout: 120s
etcd-external: false
etcd-token: etcd-cluster
dns-vip: 10.32.0.10 
dns-domain: cluster.local
pause-image: k8s.gcr.io/pause:3.2
network-plugin: cni 
cni-bin-dir: /usr/libexec/cni,/opt/cni/bin
runtime: docker
runtime-endpoint: unix:///var/run/docker.sock 
registry-mirrors: []
insecure-registries: []
config-extra-args:
  - name: kubelet
    extra-args:
      "--cgroup-driver": systemd
open-ports:
  worker:
  - port: 111
    protocol: tcp
  - port: 179
    protocol: tcp
install:
  package-source:
    type: tar.gz
    dstpath: ""
    srcpath: 
      arm64: /root/rpms/packages-arm64.tar.gz 
      amd64: /root/rpms/packages-x86.tar.gz
  etcd:
  - name: etcd
    type: pkg
    dst: ""
  kubernetes-master:
  - name: kubernetes-client,kubernetes-master
    type: pkg
  kubernetes-worker:
  - name: docker-engine,kubernetes-client,kubernetes-node,kubernetes-kubelet
    type: pkg
    dst: ""
  - name: conntrack-tools,socat
    type: pkg
    dst: ""
  network:
  - name: containernetworking-plugins
    type: pkg
    dst: ""
  loadbalance:
  - name: gd,gperftools-libs,libunwind,libwebp,libxslt
    type: pkg
    dst: ""
  - name: nginx,nginx-all-modules,nginx-filesystem,nginx-mod-http-image-filter,nginx-mod-http-perl,nginx-mod-http-xslt-filter,nginx-mod-mail,nginx-mod-stream
    type: pkg
    dst: ""
  container:
  - name: emacs-filesystem,gflags,gpm-libs,re2,rsync,vim-filesystem,vim-common,vim-enhanced,zlib-devel
    type: pkg
    dst: ""
  - name: libwebsockets,protobuf,protobuf-devel,grpc,libcgroup
    type: pkg
    dst: ""
  - name: yajl,lxc,lxc-libs,lcr,clibcni,iSulad
    type: pkg
    dst: ""
  image:  
  - name: pause.tar
    type: image
    dst: ""
  dns:    
  - name: coredns
    type: pkg
    dst: ""
  addition:
    master:
    - name: prejoin.sh
      type: shell
      schedule: "prejoin"
      TimeOut:  "30s"
    - name: calico.yaml
      type: yaml
      dst: ""
    worker:
    - name: docker.service
      type: file
      dst: /usr/lib/systemd/system/
    - name: postjoin.sh
      type: shell
      schedule: "postjoin"
```

### 安装包结构

如果是离线部署，需要准备 Kubernetes 以及相关的离线安装包，并遵循特定目录结构存放离线安装包。需要遵循的目录结构如下：

```shell
package
├── bin
├── dir
├── file
├── image
├── pkg
└── packages_notes.md
```

上述各目录的含义如下：

* 离线部署包的目录结构与集群配置 config 中的 package 的类型对应，package 类型有 pkg、repo、bin、file、dir、image、yaml、shell 八种。

* bin 目录存放二进制文件，对应 package 类型 bin 。

* dir 目录存放需要拷贝到目标机器的目录，需要配置 dst 目的地路径，对应 package 类型 dir 。

* file 目录存放 file、yaml、shell 三种类型的文件。其中 file 类型代表需要拷贝到目标机器的文件，同时需要配置 dst 目的地路径；yaml 类型代表用户自定义的 YAML 文件，会在集群部署完成后 apply 该 YAML 文件；shell 类型代表用户想要执行的脚本，同时需要配置 schedule 执行时机，执行时机包括 prejoin（节点加入前）、postjoin（节点加入后）、precleanup（节点退出前）、postcleanup（节点退出后）四个阶段。

* image 目录存放需要导入的容器镜像。这些容器镜像必须兼容 docker 的 tar 包格式（例如由 docker 或 isula-build 导出镜像）。

* pkg 目录下存放需要安装的 rpm/deb 包，对应 package 类型 pkg 。建议使用二进制文件，便于跨发行版本的部署。

### 命令参考

openEuler 提供的集群部署工具，使用命令行 eggo 进行集群部署。

#### 部署 k8s 集群

通过指定的 YAML 配置部署 k8s 集群：

**eggo deploy** \[ **-d** ]  **-f** *deploy.yaml*

| 参数          | 是否必选 | 参数含义                          |
| ------------- | -------- | --------------------------------- |
| --debug | -d | 否       | 打印调试信息                      |
| --file | -f  | 是       | 指定部署 k8s 集群的 YAML 文件路径 |

#### 加入单节点

将指定的单节点加入到 k8s 集群中：

**eggo** **join** \[ **-d** ]  **--id** *k8s-cluster*  \[ **--type** *master,worker* ] **--arch** *arm64* **--port** *22* \[ **--name** *master1*] *IP*

| 参数          | 是否必选 | 参数含义                                                     |
| ------------- | -------- | ------------------------------------------------------------ |
| --debug | -d | 否       | 打印调试信息                                                 |
| --id          | 是       | 指定将要加入 k8s 集群名称                                    |
| --type | -t  | 否       | 指定加入节点的类型，支持 master、worker 。多个类型使用 “,” 隔开，默认值为 worker 。 |
| --arch | -a  | 是       | 指定加入节点的 CPU 架构                                      |
| --port | -p  | 是       | 指定 ssh 登录所加入节点的端口号                              |
| --name | -n  | 否       | 指定加入节点的名称                                           |
| *IP*          | 是       | 加入节点的实际 IP 地址                                       |

#### 加入多节点

将指定的多个节点加入到 k8s 集群：

**eggo** **join**  \[ **-d** ]  **--id** *k8s-cluster*  **-f** *nodes.yaml*

| 参数          | 是否必选 | 参数含义                         |
| ------------- | -------- | -------------------------------- |
| --debug | -d | 否       | 打印调试信息                     |
| --id          | 是       | 指定将要加入 k8s 集群名称        |
| --file | -f  | 是       | 指定加入节点的 YAML 配置文件路径 |

#### 删除节点

删除 k8s 集群中的一个或者多个节点：

**eggo delete**  \[ **-d** ]  **--id** *k8s-cluster* *node* \[*node...*]

| 参数          | 是否必选 | 参数含义                                     |
| ------------- | -------- | -------------------------------------------- |
| --debug | -d | 否       | 打印调试信息                                 |
| --id          | 是       | 指定将要删除的节点所在的集群名称             |
| *node*        | 是       | 要删除的单个或多个节点的 IP 地址或者节点名称 |

#### 删除集群

删除整个 k8s 集群：

**eggo cleanup**  \[ **-d** ]  **--id** *k8s-cluster* \[ **-f** *deploy.yaml* ]

| 参数          | 是否必选 | 参数含义                                                     |
| ------------- | -------- | ------------------------------------------------------------ |
| --debug | -d | 否       | 打印调试信息                                                 |
| --id          | 是       | 指定将要清除的 k8s 集群名称                                  |
| --file | -f  | 否       | 指定清除 k8s 集群的 YAML 文件路径。不指定时，默认使用部署集群时缓存的集群配置。正常情况下，建议不配置该选项，仅异常情况下配置。 |

> \[!NOTE]说明
>
> * 建议使用部署集群时缓存的集群配置删除集群，即正常情况下，不建议配置 --file | -f 参数。当异常导致缓存配置破坏或者丢失时，才配置该参数。

#### 查询集群

查询当前所有通过 eggo 部署的 k8s 集群：

**eggo list** \[ **-d** ]

| 参数          | 是否必选 | 参数含义     |
| ------------- | -------- | ------------ |
| --debug | -d | 否       | 打印调试信息 |

#### 生成集群配置文件

快速生成部署 k8s 集群所需的 YAML 配置文件：

**eggo template**  **-d**  **-f** *template.yaml* **-n** *k8s-cluster* **-u** *username* **-p** *password* **--etcd** \[*192.168.0.1,192.168.0.2*]  **--masters** \[*192.168.0.1,192.168.0.2*] **--workers** *192.168.0.3* **--loadbalance** *192.168.0.4*

| 参数                | 是否必选 | 参数含义                        |
| ------------------- | -------- | ------------------------------- |
| --debug | -d       | 否       | 打印调试信息                    |
| --file | -f        | 否       | 指定生成的 YAML 文件的路径      |
| --name | -n        | 否       | 指定 k8s 集群的名称             |
| --username | -u    | 否       | 指定 ssh 登录所配置节点的用户名 |
| --password | -p    | 否       | 指定 ssh 登录所配置节点的密码   |
| --etcd              | 否       | 指定 etcd 节点的 IP 列表        |
| --masters           | 否       | 指定 master 节点的 IP 列表      |
| --workers           | 否       | 指定 worker 节点的 IP 列表      |
| --loadbalance | -l | 否       | 指定 loadbalance 节点的 IP      |

#### 查询帮助信息

查询 eggo 命令的帮助信息：

**eggo help**

#### 查询子命令帮助信息

查询 eggo 子命令的帮助信息：

**eggo deploy | join | delete | cleanup | list | template -h**

| 参数        | 是否必选 | 参数含义     |
| ----------- | -------- | ------------ |
| --help| -h | 是       | 打印帮助信息 |

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/tool_guide.md
---
# 工具使用指南

为了方便用户更好地使用虚拟化，openEuler 提供了一系列工具，包括 vmtop、LibcarePlus 等。本章介绍这些工具的安装和使用指导。

---

---
url: >-
  /zh/docs/22.03_LTS_SP4/virtualization/virtualization_platform/virtualization/vmtop.md
---
# 工具使用指南

* [vmtop使用指南](#vmtop使用指南)

## vmtop使用指南

### 概述

vmtop 是运行在宿主机host上的用户态工具。使用vmtop可以实时动态地查看虚拟机资源的使用情况，例如CPU占用率、内存占用率、vCPU陷入陷出次数等。因此，可以使用vmtop作为虚拟化问题定位和性能调优的工具。

#### 多架构支持

当前vmtop支持AArch64和x86\_64处理器架构。

#### 显示项说明

不同处理器架构的操作系统，vmtop的显示项存在差异，这里给出各个显示项的含义及其是否在对应架构呈现。
说明：以下采样差是指指定时间间隔内获取的两次数据的差值。

##### **AArch64和x86\_64架构公共显示项**

* VM/task-name: 虚拟机/进程名称
* DID: 虚拟机id
* PID: 虚拟机qemu进程的pid
* %CPU: 进程的CPU占用率
* EXTsum: kvm exit总次数（采样差）
* S: 进程状态
* P: 进程所占用的物理CPU号
* %ST: 被抢占时间与CPU运行时间的比
* %GUE: 虚拟机内部占用时间与CPU运行时间的比
* %HYP: 虚拟化开销占比

##### 仅AArch64架构的显示项

* EXThvc: hvc-exit次数（采样差）
* EXTwfe: wfe-exit次数（采样差）
* EXTwfi: wfi-exit次数（采样差）
* EXTmmioU: mmioU-exit次数（采样差）
* EXTmmioK: mmioK-exit次数（采样差）
* EXTfp: fp-exit次数（采样差）
* EXTirq: irq-exit次数（采样差）
* EXTsys64: sys64 exit次数（采样差）
* EXTmabt: mem abort exit次数（采样差）

##### 仅x86\_64架构的显示项

* PFfix: 缺页次数（采样差）
* PFgu: 向guest OS注入缺页次数（采样差）
* INvlpg: 冲刷tlb某项次数(tlb其中一项，并不固定)
* EXTio: io VM-exit次数（采样差）
* EXTmmio: mmio VM-exit次数（采样差）
* EXThalt: halt VM-exit次数（采样差）
* EXTsig: 信号处理引起的VM-exit次数（采样差）
* EXTirq: 中断引起的VM-exit次数（采样差）
* EXTnmiW: 处理不可屏蔽中断引起的VM-exit次数（采样差）
* EXTirqW: interruptwindow机制，开启中断使能时exit，以便注入中断（采样差）
* IrqIn: 注入irq中断次数（采样差）
* NmiIn: 注入nmi中断次数（采样差）
* TLBfl: 冲刷整个tlb次数（采样差）
* HostReL: 重载主机状态次数（采样差）
* Hyperv: 模拟Guest操作系统辅助虚拟化调用hypercall的处理次数（采样差）
* EXTcr: 访问CR寄存器退出次数（采样差）
* EXTrmsr: 读msr退出次数（采样差）
* EXTwmsr: 写msr退出次数（采样差）
* EXTapic: 写apic次数（采样差）
* EXTeptv: Ept缺页退出次数（采样差）
* EXTeptm: Ept错误退出次数（采样差）
* EXTpau: vCPU暂停退出次数（采样差）

### 使用方法

vmtop是一款命令行工具，直接以命令行的方式运行 vmtop 即可。
另外，vmtop还提供了不同可选选项，用于查询不同信息。

#### 语法格式

```sh
vmtop [选项]
```

#### 选项说明

* -d: 设置显示刷新的时间间隔，单位：秒。
* -H: 显示虚拟机的线程信息。
* -n: 设置显示刷新的次数，刷新完成后退出。
* -b: Batch模式显示，可以用于重定向到文件。
* -h: 显示帮助信息。
* -v: 显示版本。
* -p: 监控指定id的虚拟机。

#### 快捷键

在vmtop运行状态下使用的快捷键。

* H: 显示或关闭虚拟机线程信息，默认显示该信息。
* up/down: 向上/向下移动显示的虚拟机列表。
* left/right: 向左/向右移动显示的信息，从而显示因屏幕宽度被隐藏的列。
* f: 进入监控项编辑模式，选择要开启的监控项。
* q: 退出vmtop进程。

### 示例

在host上直接以命令行的方式运行vmtop：

```sh
vmtop
```

输出如下:

```sh
vmtop - 2020-09-14 09:54:48 - 1.0
Domains: 1 running

  DID  VM/task-name     PID  %CPU    EXThvc    EXTwfe    EXTwfi  EXTmmioU  EXTmmioK     EXTfp    EXTirq  EXTsys64   EXTmabt    EXTsum    S    P   %ST  %GUE  %HYP
    2       example 4054916  13.0         0         0      1206        10         0       144        62       174         0      1452    S  106   0.0  99.7  16.0
```

可以看到，host上只有一台名称为“example”的虚拟机，ID为2，CPU占用率是13.0%，在1秒内的陷入陷出总次数是1452，虚拟机进程占用的物理CPU为106号CPU，虚拟机内部占用时间与CPU运行时间的比是99.7%。

1.显示虚拟机线程信息
按下‘H’后可以显示线程信息：

```sh
vmtop - 2020-09-14 10:11:27 - 1.0
Domains: 1 running

  DID  VM/task-name     PID  %CPU    EXThvc    EXTwfe    EXTwfi  EXTmmioU  EXTmmioK     EXTfp    EXTirq  EXTsys64   EXTmabt    EXTsum    S    P   %ST  %GUE  %HYP
    2       example 4054916  13.0         0         0      1191        17         4       120        76       147         0      1435    S  119   0.0 123.7   4.0
   |_      qemu-kvm 4054916   0.0         0         0         0         0         0         0         0         0         0         0    S  119   0.0   0.0   0.0
   |_      qemu-kvm 4054928   0.0         0         0         0         0         0         0         0         0         0         0    S  119   0.0   0.0   0.0
   |_  signalfd_com 4054929   0.0         0         0         0         0         0         0         0         0         0         0    S  120   0.0   0.0   0.0
   |_  IO mon_iothr 4054932   0.0         0         0         0         0         0         0         0         0         0         0    S  117   0.0   0.0   0.0
   |_     CPU 0/KVM 4054933   3.0         0         0       280         6         4        28        19        41         0       350    S  105   0.0  27.9   0.0
   |_     CPU 1/KVM 4054934   3.0         0         0       260         0         0        16        12        36         0       308    S   31   0.0  20.0   0.0
   |_     CPU 2/KVM 4054935   3.0         0         0       341         0         0        44        20        26         0       387    R  108   0.0  27.9   4.0
   |_     CPU 3/KVM 4054936   5.0         0         0       310        11         0        32        25        44         0       390    S  103   0.0  47.9   0.0
   |_   memory_lock 4054940   0.0         0         0         0         0         0         0         0         0         0         0    S  126   0.0   0.0   0.0
   |_    vnc_worker 4054944   0.0         0         0         0         0         0         0         0         0         0         0    S  118   0.0   0.0   0.0
   |_        worker 4143738   0.0         0         0         0         0         0         0         0         0         0         0    S  120   0.0   0.0   0.0
```

example虚拟机有11个线程，其中包括vCPU线程、vnc\_worker、IO mon\_iotreads等等，每个线程同样会显示详细CPU占用、陷入陷出等信息。

2.选择监控项
按下‘f’进入监控项编辑模式：

```sh
field filter - select which field to be showed
Use up/down to navigate, use space to set whether chosen filed to be showed
'q' to quit to normal display

- DID
- VM/task-name
- PID
- %CPU
- EXThvc
- EXTwfe
- EXTwfi
- EXTmmioU
- EXTmmioK
- EXTfp
- EXTirq
- EXTsys64
- EXTmabt
- EXTsum
- S
- P
- %ST
- %GUE
- %HYP
```

当前所有监控项都默认显示，通过up/down键选择，用space键来设置对应显示项是否显示/隐藏，按‘q’键退出。
将%ST、%GUE、%HYP设置为隐藏后，输出如下:

```sh
vmtop - 2020-09-14 10:23:25 - 1.0
Domains: 1 running

  DID  VM/task-name     PID  %CPU    EXThvc    EXTwfe    EXTwfi  EXTmmioU  EXTmmioK     EXTfp    EXTirq  EXTsys64   EXTmabt    EXTsum    S    P
    2       example 4054916  12.0         0         0      1213        14         1       144        68       168         0      1464    S  125
   |_      qemu-kvm 4054916   0.0         0         0         0         0         0         0         0         0         0         0    S  125
   |_      qemu-kvm 4054928   0.0         0         0         0         0         0         0         0         0         0         0    S  119
   |_  signalfd_com 4054929   0.0         0         0         0         0         0         0         0         0         0         0    S  120
   |_  IO mon_iothr 4054932   0.0         0         0         0         0         0         0         0         0         0         0    S  117
   |_     CPU 0/KVM 4054933   2.0         0         0       303         6         0        29        10        35         0       354    S   98
   |_     CPU 1/KVM 4054934   4.0         0         0       279         0         0        39        17        49         0       345    S    1
   |_     CPU 2/KVM 4054935   3.0         0         0       283         0         0        33        20        40         0       343    S  122
   |_     CPU 3/KVM 4054936   3.0         0         0       348         8         1        43        21        44         0       422    S  110
   |_   memory_lock 4054940   0.0         0         0         0         0         0         0         0         0         0         0    S  126
   |_    vnc_worker 4054944   0.0         0         0         0         0         0         0         0         0         0         0    S  118
   |_        worker    1794   0.0         0         0         0         0         0         0         0         0         0         0    S  126
```

%ST、%GUE、%HYP将不会出现在显示界面上。

---

---
url: /zh/docs/22.03_LTS_SP4/server/releasenotes/resolved_issues.md
---
# 已修复问题

完整问题清单请参见[完整问题清单](https://gitee.com/organizations/src-openeuler/issues)。

完整的内核提交记录请参见[提交记录](https://atomgit.com/openeuler/kernel/commits/openEuler-22.03-LTS-SP4)。

已修复问题请参见[表1](#table2204014971491143)。

**表 1**  修复问题列表&#x20;

|ISSUE ID|ISSUE 路径|问题描述|关联仓库|
|-|-|-|-|
| I9RF7L | <https://gitee.com/open_euler/dashboard?issue_id=I9RF7L> | 【Eulermaker】octave在工程openEuler-22.03-LTS-SP4:epol构建失败  | octave |
| I9S7JR | <https://gitee.com/open_euler/dashboard?issue_id=I9S7JR> | 【Eulermaker】dde-network-core在工程openEuler-22.03-LTS-SP4:epol构建失败  | dde-network-core |
| I9S7JY | <https://gitee.com/open_euler/dashboard?issue_id=I9S7JY> | 【Eulermaker】glib2在工程openEuler-22.03-LTS-SP4:everything构建问题  | glib2 |
| I9SJPV | <https://gitee.com/open_euler/dashboard?issue_id=I9SJPV> | 【22.03-LTS-SP4-rc1】dpu-utilities 包在22.03-SP4中相比22.03-SP3 version版本降级  | dpu-utilities |
| I9SJPX | <https://gitee.com/open_euler/dashboard?issue_id=I9SJPX> | 【22.03-LTS-SP4-rc1】oemaker 包在22.03-SP4中相比22.03-SP3 version版本降级  | oemaker |
| I9SJPZ | <https://gitee.com/open_euler/dashboard?issue_id=I9SJPZ> | 【22.03-LTS-SP4-rc1】llvm-bolt 包在22.03-SP4中相比22.03-SP3 version版本降级  | llvm-bolt |
| I9SJQ0 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ0> | 【22.03-LTS-SP4-rc1】hadoop 包在22.03-SP4中相比22.03-SP3 version版本降级  | hadoop |
| I9SJQ2 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ2> | 【22.03-LTS-SP4-rc1】openEuler-release 包在22.03-SP4中相比22.03-SP3 version版本降级  | openEuler-release |
| I9SJQ3 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ3> | 【22.03-LTS-SP4-rc1】dde-session-shell 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-session-shell |
| I9SJQ4 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ4> | 【22.03-LTS-SP4-rc1】dde 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde |
| I9SJQ5 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ5> | 【22.03-LTS-SP4-rc1】dtkgui 包在22.03-SP4中相比22.03-SP3 version版本降级  | dtkgui |
| I9SJQ6 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ6> | 【22.03-LTS-SP4-rc1】dtkcore 包在22.03-SP4中相比22.03-SP3 version版本降级  | dtkcore |
| I9SJQ9 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQ9> | 【22.03-LTS-SP4-rc1】deepin-anything 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-anything |
| I9SJQB | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQB> | 【22.03-LTS-SP4-rc1】dde-kwin 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-kwin |
| I9SJQD | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQD> | 【22.03-LTS-SP4-rc1】libmetal 包在22.03-SP4中相比22.03-SP3 version版本降级  | libmetal |
| I9SJQE | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQE> | 【22.03-LTS-SP4-rc1】dde-qt-dbus-factory 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-qt-dbus-factory |
| I9SJQF | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQF> | 【22.03-LTS-SP4-rc1】deepin-compressor 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-compressor |
| I9SJQH | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQH> | 【22.03-LTS-SP4-rc1】dde-clipboard 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-clipboard |
| I9SJQI | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQI> | 【22.03-LTS-SP4-rc1】deepin-image-editor 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-image-editor |
| I9SJQL | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQL> | 【22.03-LTS-SP4-rc1】dde-control-center 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-control-center |
| I9SJQN | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQN> | 【22.03-LTS-SP4-rc1】deepin-wallpapers 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-wallpapers |
| I9SJQO | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQO> | 【22.03-LTS-SP4-rc1】dde-session-ui 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-session-ui |
| I9SJQP | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQP> | 【22.03-LTS-SP4-rc1】deepin-gtk-theme 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-gtk-theme |
| I9SJQQ | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQQ> | 【22.03-LTS-SP4-rc1】deepin-log-viewer 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-log-viewer |
| I9SJQS | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQS> | 【22.03-LTS-SP4-rc1】deepin-devicemanager 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-devicemanager |
| I9SJQU | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQU> | 【22.03-LTS-SP4-rc1】deepin-system-monitor 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-system-monitor |
| I9SJQW | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQW> | 【22.03-LTS-SP4-rc1】deepin-screen-recorder 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-screen-recorder |
| I9SJQY | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQY> | 【22.03-LTS-SP4-rc1】dtkcommon 包在22.03-SP4中相比22.03-SP3 version版本降级  | dtkcommon |
| I9SJQZ | <https://gitee.com/open_euler/dashboard?issue_id=I9SJQZ> | 【22.03-LTS-SP4-rc1】deepin-default-settings 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-default-settings |
| I9SJR1 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR1> | 【22.03-LTS-SP4-rc1】dde-api 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-api |
| I9SJR2 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR2> | 【22.03-LTS-SP4-rc1】deepin-gettext-tools 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-gettext-tools |
| I9SJR3 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR3> | 【22.03-LTS-SP4-rc1】startdde 包在22.03-SP4中相比22.03-SP3 version版本降级  | startdde |
| I9SJR5 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR5> | 【22.03-LTS-SP4-rc1】deepin-icon-theme 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-icon-theme |
| I9SJR6 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR6> | 【22.03-LTS-SP4-rc1】deepin-terminal 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-terminal |
| I9SJR8 | <https://gitee.com/open_euler/dashboard?issue_id=I9SJR8> | 【22.03-LTS-SP4-rc1】dde-daemon 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-daemon |
| I9SJRA | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRA> | 【22.03-LTS-SP4-rc1】deepin-desktop-schemas 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-desktop-schemas |
| I9SJRB | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRB> | 【22.03-LTS-SP4-rc1】dde-file-manager 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-file-manager |
| I9SJRC | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRC> | 【22.03-LTS-SP4-rc1】deepin-editor 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-editor |
| I9SJRD | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRD> | 【22.03-LTS-SP4-rc1】dde-launcher 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-launcher |
| I9SJRF | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRF> | 【22.03-LTS-SP4-rc1】deepin-menu 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-menu |
| I9SJRG | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRG> | 【22.03-LTS-SP4-rc1】dde-polkit-agent 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-polkit-agent |
| I9SJRH | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRH> | 【22.03-LTS-SP4-rc1】dde-dock 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-dock |
| I9SJRJ | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRJ> | 【22.03-LTS-SP4-rc1】deepin-pw-check 包在22.03-SP4中相比22.03-SP3 version版本降级  | deepin-pw-check |
| I9SJRL | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRL> | 【22.03-LTS-SP4-rc1】dtkwidget 包在22.03-SP4中相比22.03-SP3 version版本降级  | dtkwidget |
| I9SJRM | <https://gitee.com/open_euler/dashboard?issue_id=I9SJRM> | 【22.03-LTS-SP4-rc1】dde-calendar 包在22.03-SP4中相比22.03-SP3 version版本降级  | dde-calendar |
| I9SJZK | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZK> | 【22.03-LTS-SP4-rc1】ctags 包在22.03-SP4中相比22.03-SP3 release版本降级  | ctags |
| I9SJZN | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZN> | 【22.03-LTS-SP4-rc1】protobuf2 包在22.03-SP4中相比22.03-SP3 release版本降级  | protobuf2 |
| I9SJZP | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZP> | 【22.03-LTS-SP4-rc1】exempi 包在22.03-SP4中相比22.03-SP3 release版本降级  | exempi |
| I9SJZR | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZR> | 【22.03-LTS-SP4-rc1】virt-manager 包在22.03-SP4中相比22.03-SP3 release版本降级  | virt-manager |
| I9SJZS | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZS> | 【22.03-LTS-SP4-rc1】python-mako 包在22.03-SP4中相比22.03-SP3 release版本降级  | python-mako |
| I9SJZT | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZT> | 【22.03-LTS-SP4-rc1】gcc 包在22.03-SP4中相比22.03-SP3 release版本降级  | gcc |
| I9SJZU | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZU> | 【22.03-LTS-SP4-rc1】python-bottle 包在22.03-SP4中相比22.03-SP3 release版本降级  | python-bottle |
| I9SJZV | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZV> | 【22.03-LTS-SP4-rc1】dbus 包在22.03-SP4中相比22.03-SP3 release版本降级  | dbus |
| I9SJZW | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZW> | 【22.03-LTS-SP4-rc1】libcxxabi 包在22.03-SP4中相比22.03-SP3 release版本降级  | libcxxabi |
| I9SJZX | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZX> | 【22.03-LTS-SP4-rc1】sqlite 包在22.03-SP4中相比22.03-SP3 release版本降级  | sqlite |
| I9SJZY | <https://gitee.com/open_euler/dashboard?issue_id=I9SJZY> | 【22.03-LTS-SP4-rc1】linuxdoc-tools 包在22.03-SP4中相比22.03-SP3 release版本降级  | linuxdoc-tools |
| I9SK02 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK02> | 【22.03-LTS-SP4-rc1】groovy 包在22.03-SP4中相比22.03-SP3 release版本降级  | groovy |
| I9SK03 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK03> | 【22.03-LTS-SP4-rc1】libwd 包在22.03-SP4中相比22.03-SP3 release版本降级  | libwd |
| I9SK04 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK04> | 【22.03-LTS-SP4-rc1】python-imagesize 包在22.03-SP4中相比22.03-SP3 release版本降级  | python-imagesize |
| I9SK05 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK05> | 【22.03-LTS-SP4-rc1】storm 包在22.03-SP4中相比22.03-SP3 release版本降级  | storm |
| I9SK06 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK06> | 【22.03-LTS-SP4-rc1】perl 包在22.03-SP4中相比22.03-SP3 release版本降级  | perl |
| I9SK08 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK08> | 【22.03-LTS-SP4-rc1】scap-security-guide 包在22.03-SP4中相比22.03-SP3 release版本降级  | scap-security-guide |
| I9SK0A | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0A> | 【22.03-LTS-SP4-rc1】redis6 包在22.03-SP4中相比22.03-SP3 release版本降级  | redis6 |
| I9SK0B | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0B> | 【22.03-LTS-SP4-rc1】gradle 包在22.03-SP4中相比22.03-SP3 release版本降级  | gradle |
| I9SK0D | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0D> | 【22.03-LTS-SP4-rc1】libkae 包在22.03-SP4中相比22.03-SP3 release版本降级  | libkae |
| I9SK0E | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0E> | 【22.03-LTS-SP4-rc1】sysget 包在22.03-SP4中相比22.03-SP3 release版本降级  | sysget |
| I9SK0G | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0G> | 【22.03-LTS-SP4-rc1】libsrtp 包在22.03-SP4中相比22.03-SP3 release版本降级  | libsrtp |
| I9SK0I | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0I> | 【22.03-LTS-SP4-rc1】python-beaker 包在22.03-SP4中相比22.03-SP3 release版本降级  | python-beaker |
| I9SK0J | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0J> | 【22.03-LTS-SP4-rc1】unixODBC 包在22.03-SP4中相比22.03-SP3 release版本降级  | unixODBC |
| I9SK0K | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0K> | 【22.03-LTS-SP4-rc1】pytorch 包在22.03-SP4中相比22.03-SP3 release版本降级  | pytorch |
| I9SK0O | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0O> | 【22.03-LTS-SP4-rc1】groovy18 包在22.03-SP4中相比22.03-SP3 release版本降级  | groovy18 |
| I9SK0Q | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0Q> | 【22.03-LTS-SP4-rc1】llvm-libunwind 包在22.03-SP4中相比22.03-SP3 release版本降级  | llvm-libunwind |
| I9SK0S | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0S> | 【22.03-LTS-SP4-rc1】criu 包在22.03-SP4中相比22.03-SP3 release版本降级  | criu |
| I9SK0T | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0T> | 【22.03-LTS-SP4-rc1】sysmaster 包在22.03-SP4中相比22.03-SP3 release版本降级  | sysmaster |
| I9SK0U | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0U> | 【22.03-LTS-SP4-rc1】python-flask-restful 包在22.03-SP4中相比22.03-SP3 release版本降级  | python-flask-restful |
| I9SK0V | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0V> | 【22.03-LTS-SP4-rc1】man-db 包在22.03-SP4中相比22.03-SP3 release版本降级  | man-db |
| I9SK0Y | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0Y> | 【22.03-LTS-SP4-rc1】python-sphinx-theme-alabaster 包在22.03-SP4中相比22.03-SP3 release版本降级  | python-sphinx-theme-alabaster |
| I9SK0Z | <https://gitee.com/open_euler/dashboard?issue_id=I9SK0Z> | 【22.03-LTS-SP4-rc1】osinfo-db 包在22.03-SP4中相比22.03-SP3 release版本降级  | osinfo-db |
| I9SK11 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK11> | 【22.03-LTS-SP4-rc1】python-configshell 包在22.03-SP4中相比22.03-SP3 release版本降级  | python-configshell |
| I9SK13 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK13> | 【22.03-LTS-SP4-rc1】libcxx 包在22.03-SP4中相比22.03-SP3 release版本降级  | libcxx |
| I9SK14 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK14> | 【22.03-LTS-SP4-rc1】secGear 包在22.03-SP4中相比22.03-SP3 release版本降级  | secGear |
| I9SK15 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK15> | 【22.03-LTS-SP4-rc1】redis5 包在22.03-SP4中相比22.03-SP3 release版本降级  | redis5 |
| I9SK17 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK17> | 【22.03-LTS-SP4-rc1】openEuler-repos 包在22.03-SP4中相比22.03-SP3 release版本降级  | openEuler-repos |
| I9SK18 | <https://gitee.com/open_euler/dashboard?issue_id=I9SK18> | 【22.03-LTS-SP4-rc1】ft\_utils 包在22.03-SP4中相比22.03-SP3 release版本降级  | ft\_utils |
| I9SK1A | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1A> | 【22.03-LTS-SP4-rc1】dde-introduction 包在22.03-SP4中相比22.03-SP3 release版本降级  | dde-introduction |
| I9SK1C | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1C> | 【22.03-LTS-SP4-rc1】deepin-clone 包在22.03-SP4中相比22.03-SP3 release版本降级  | deepin-clone |
| I9SK1D | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1D> | 【22.03-LTS-SP4-rc1】shotwell 包在22.03-SP4中相比22.03-SP3 release版本降级  | shotwell |
| I9SK1E | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1E> | 【22.03-LTS-SP4-rc1】communication\_ipc 包在22.03-SP4中相比22.03-SP3 release版本降级  | communication\_ipc |
| I9SK1F | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1F> | 【22.03-LTS-SP4-rc1】ft\_wl\_fwk 包在22.03-SP4中相比22.03-SP3 release版本降级  | ft\_wl\_fwk |
| I9SK1I | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1I> | 【22.03-LTS-SP4-rc1】ft\_mmi 包在22.03-SP4中相比22.03-SP3 release版本降级  | ft\_mmi |
| I9SK1J | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1J> | 【22.03-LTS-SP4-rc1】k3s-containerd 包在22.03-SP4中相比22.03-SP3 release版本降级  | k3s-containerd |
| I9SK1K | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1K> | 【22.03-LTS-SP4-rc1】redshift 包在22.03-SP4中相比22.03-SP3 release版本降级  | redshift |
| I9SK1M | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1M> | 【22.03-LTS-SP4-rc1】ft\_multimedia 包在22.03-SP4中相比22.03-SP3 release版本降级  | ft\_multimedia |
| I9SK1N | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1N> | 【22.03-LTS-SP4-rc1】k3s 包在22.03-SP4中相比22.03-SP3 release版本降级  | k3s |
| I9SK1O | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1O> | 【22.03-LTS-SP4-rc1】deepin-desktop-base 包在22.03-SP4中相比22.03-SP3 release版本降级  | deepin-desktop-base |
| I9SK1P | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1P> | 【22.03-LTS-SP4-rc1】ukui-settings-daemon 包在22.03-SP4中相比22.03-SP3 release版本降级  | ukui-settings-daemon |
| I9SK1S | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1S> | 【22.03-LTS-SP4-rc1】systemabilitymgr\_safwk 包在22.03-SP4中相比22.03-SP3 release版本降级  | systemabilitymgr\_safwk |
| I9SK1U | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1U> | 【22.03-LTS-SP4-rc1】arkui-linux 包在22.03-SP4中相比22.03-SP3 release版本降级  | arkui-linux |
| I9SK1V | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1V> | 【22.03-LTS-SP4-rc1】filemanagement\_dfs\_service 包在22.03-SP4中相比22.03-SP3 release版本降级  | filemanagement\_dfs\_service |
| I9SK1W | <https://gitee.com/open_euler/dashboard?issue_id=I9SK1W> | 【22.03-LTS-SP4-rc1】migration-tools 包在22.03-SP4中相比22.03-SP3 release版本降级  | migration-tools |
| I9SKC6 | <https://gitee.com/open_euler/dashboard?issue_id=I9SKC6> | 【22.03-LTS-SP4-rc1】pyflakes 包在22.03-SP4中相比22.03-SP3 release版本降级  | pyflakes |
| I9SKC7 | <https://gitee.com/open_euler/dashboard?issue_id=I9SKC7> | 【22.03-LTS-SP4-rc1】openjdk-latest 包在22.03-SP4中相比22.03-SP3 version版本降级  | openjdk-latest |
| I9SKC9 | <https://gitee.com/open_euler/dashboard?issue_id=I9SKC9> | 【22.03-LTS-SP4-rc1】qt5integration 包在22.03-SP4中相比22.03-SP3 version版本降级  | qt5integration |
| I9SKCB | <https://gitee.com/open_euler/dashboard?issue_id=I9SKCB> | 【22.03-LTS-SP4-rc1】aops-diana 包在22.03-SP4中相比22.03-SP3 release版本降级  | aops-diana |
| I9SKCC | <https://gitee.com/open_euler/dashboard?issue_id=I9SKCC> | 【22.03-LTS-SP4-rc1】qt5dxcb-plugin 包在22.03-SP4中相比22.03-SP3 version版本降级  | qt5dxcb-plugin |
| I9SL18 | <https://gitee.com/open_euler/dashboard?issue_id=I9SL18> | 【22.03-LTS-SP4-rc1】oec-hardware 包在22.03-SP4中相比22.03-SP3 version版本降级  | oec-hardware |
| I9SL90 | <https://gitee.com/open_euler/dashboard?issue_id=I9SL90> | 【22.03-LTS-SP4-rc1】【arm/x86】PWR\_SYS\_GetRtPowerInfo接口未开放  | powerapi |
| I9SMBB | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBB> | 【22.03-LTS-SP4-rc1】aops-ceres 包在22.03-SP4中相比22.03-SP3 release版本降级  | aops-ceres |
| I9SMBC | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBC> | 【22.03-LTS-SP4-rc1】busybox 包在22.03-SP4中相比22.03-SP3 release版本降级  | busybox |
| I9SMBD | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBD> | 【22.03-LTS-SP4-rc1】containers-common 包在22.03-SP4中相比22.03-SP3 release版本降级  | containers-common |
| I9SMBF | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBF> | 【22.03-LTS-SP4-rc1】e2fsprogs 包在22.03-SP4中相比22.03-SP3 release版本降级  | e2fsprogs |
| I9SMBH | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBH> | 【22.03-LTS-SP4-rc1】ebtables 包在22.03-SP4中相比22.03-SP3 release版本降级  | ebtables |
| I9SMBI | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBI> | 【22.03-LTS-SP4-rc1】firewalld 包在22.03-SP4中相比22.03-SP3 release版本降级  | firewalld |
| I9SMBJ | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBJ> | 【22.03-LTS-SP4-rc1】gcc-cross 包在22.03-SP4中相比22.03-SP3 release版本降级  | gcc-cross |
| I9SMBK | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBK> | 【22.03-LTS-SP4-rc1】gdk-pixbuf2 包在22.03-SP4中相比22.03-SP3 release版本降级  | gdk-pixbuf2 |
| I9SMBL | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBL> | 【22.03-LTS-SP4-rc1】imageTailor 包在22.03-SP4中相比22.03-SP3 version版本降级  | imageTailor |
| I9SMBM | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBM> | 【22.03-LTS-SP4-rc1】iproute 包在22.03-SP4中相比22.03-SP3 release版本降级  | iproute |
| I9SMBO | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBO> | 【22.03-LTS-SP4-rc1】iSulad 包在22.03-SP4中相比22.03-SP3 release版本降级  | iSulad |
| I9SMBP | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBP> | 【22.03-LTS-SP4-rc1】KubeOS 包在22.03-SP4中相比22.03-SP3 version版本降级  | KubeOS |
| I9SMBQ | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBQ> | 【22.03-LTS-SP4-rc1】libsndfile 包在22.03-SP4中相比22.03-SP3 release版本降级  | libsndfile |
| I9SMBR | <https://gitee.com/open_euler/dashboard?issue_id=I9SMBR> | 【22.03-LTS-SP4-rc1】openstack-releases 包在22.03-SP4中相比22.03-SP3 release版本降级  | openstack-releases |
| I9SQWH | <https://gitee.com/open_euler/dashboard?issue_id=I9SQWH> | 【22.03-LTS-SP4-rc1】【arm/x86】调用PWR\_CreateDcTask接口创建不支持的数据类型的任务创建成功  | powerapi |
| I9SRCC | <https://gitee.com/open_euler/dashboard?issue_id=I9SRCC> | 【22.03\_SP4\_RC1\_everything】【arm\x86】mandoc安装过程有报错信息  | mandoc |
| I9STHL | <https://gitee.com/open_euler/dashboard?issue_id=I9STHL> | 【Eulermaker】python-beaker在工程openEuler-22.03-LTS-SP4:everything构建失败  | python-beaker |
| I9SURK | <https://gitee.com/open_euler/dashboard?issue_id=I9SURK> | 【22.03\_SP4\_RC1\_epol】【arm\x86】afterburn卸载过程有报错信息  | afterburn |
| I9SUVR | <https://gitee.com/open_euler/dashboard?issue_id=I9SUVR> | 【openEuler-22.03-LTS-SP4】storm-nimbus.service服务stop失败  | storm |
| I9SUXW | <https://gitee.com/open_euler/dashboard?issue_id=I9SUXW> | 【22.03-LTS-SP4-rc1】【编译告警检查】【arm/x86】执行编译出现告警信息  | powerapi |
| I9SV5C | <https://gitee.com/open_euler/dashboard?issue_id=I9SV5C> | 【22.03\_SP4\_RC1\_everything】【arm\x86】devmaster安装过程有报错信息  | sysmaster |
| I9SV9F | <https://gitee.com/open_euler/dashboard?issue_id=I9SV9F> | 【openEuler-22.03-LTS-SP4】storm-supervisor.service服务stop失败  | storm |
| I9T040 | <https://gitee.com/open_euler/dashboard?issue_id=I9T040> | 【22.03-LTS-SP4-rc1】【x86/arm】luarocks源码包本地自编译失败，缺少编译依赖openresty、openresty-openssl111-devel包  | luarocks |
| I9T0OH | <https://gitee.com/open_euler/dashboard?issue_id=I9T0OH> | 【22.03-LTS-SP4-rc1】【x86/arm】perl-Compress-Raw-Zlib源码包本地自编译失败，check阶段失败  | perl-Compress-Raw-Zlib |
| I9T5DF | <https://gitee.com/open_euler/dashboard?issue_id=I9T5DF> | 【22.03-LTS-SP4-rc1】【arm/x86】调用PWR\_PROC\_QueryProcs接口根据关键字查找任务不符合预期  | powerapi |
| I9T5Q8 | <https://gitee.com/open_euler/dashboard?issue_id=I9T5Q8> | 【22.03-LTS-SP4-rc1】【x86】netdata -D报核心已转储  | netdata |
| I9T5TN | <https://gitee.com/open_euler/dashboard?issue_id=I9T5TN> | 【22.03\_SP4\_RC1\_epol】【arm\x86】migration-tools-server升级过程有异常信息  | migration-tools |
| I9T6FB | <https://gitee.com/open_euler/dashboard?issue_id=I9T6FB> | 【22.03\_SP4\_RC1\_everthing】【arm\x86】obs-server升级到sp4版本后卸载失败  | obs-server |
| I9T7M8 | <https://gitee.com/open_euler/dashboard?issue_id=I9T7M8> | 【22.03-LTS-SP4-rc1】【arm/x86】首次调用PWR\_PROC\_GetWattState接口失败，返回错误码13  | powerapi |
| I9TNRP | <https://gitee.com/open_euler/dashboard?issue_id=I9TNRP> | 【22.03-LTS-SP4-rc1】【地址消毒测试】【arm/x86】调用PWR\_CreateDcTask接口创建数据类型为2的任务，触发任务回调函数导致服务挂掉  | powerapi |
| I9U0YX | <https://gitee.com/open_euler/dashboard?issue_id=I9U0YX> | 【openEuler-22.03-LTS-SP4 rc1】 sysmonitor.service服务启动后有报错信息  | sysmonitor |
| I9U163 | <https://gitee.com/open_euler/dashboard?issue_id=I9U163> | 【22.03-LTS-SP4-rc1】【x86/arm】glassfish-jsp源码包本地自编译失败  | glassfish-jsp |
| I9ULPM | <https://gitee.com/open_euler/dashboard?issue_id=I9ULPM> | 【22.03\_SP4\_RC2\_everthing】【arm\x86】obs-server降级过程存在异常提示  | obs-server |
| I9UPLT | <https://gitee.com/open_euler/dashboard?issue_id=I9UPLT> | 【22.03-LTS-SP4-rc2】【地址消毒测试】【arm/x86】调用TEST\_PWR\_PROC\_QueryProcs接口，关键字为空字符串，num=100（构造5000个进程循环进行打印 ），接口调用超时，返回错误码1  | powerapi |
| I9UQHL | <https://gitee.com/open_euler/dashboard?issue_id=I9UQHL> | 【22.03-LTS-SP4 RC2】\[deja]aarch64-sve-acle.exp的部分用例执行失败  | gcc |
| I9UVJP | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJP> | 【22.03-LTS-SP4-rc2】abseil-cpp 包在22.03-SP4中相比22.03-SP3 release版本降级  | abseil-cpp |
| I9UVJQ | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJQ> | 【22.03-LTS-SP4-rc2】shim 包在22.03-SP4中相比22.03-SP3 release版本降级  | shim |
| I9UVJR | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJR> | 【22.03-LTS-SP4-rc2】spdk 包在22.03-SP4中相比22.03-SP3 release版本降级  | spdk |
| I9UVJS | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJS> | 【22.03-LTS-SP4-rc2】syscare 包在22.03-SP4中相比22.03-SP3 release版本降级  | syscare |
| I9UVJT | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJT> | 【22.03-LTS-SP4-rc2】libsepol 包在22.03-SP4中相比22.03-SP3 release版本降级  | libsepol |
| I9UVJU | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJU> | 【22.03-LTS-SP4-rc2】libxml2 包在22.03-SP4中相比22.03-SP3 release版本降级  | libxml2 |
| I9UVJV | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJV> | 【22.03-LTS-SP4-rc2】kernel 包在22.03-SP4中相比22.03-SP3 release版本降级  | kernel |
| I9UVJW | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJW> | 【22.03-LTS-SP4-rc2】openssh 包在22.03-SP4中相比22.03-SP3 release版本降级  | openssh |
| I9UVJY | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJY> | 【22.03-LTS-SP4-rc2】ghostscript 包在22.03-SP4中相比22.03-SP3 release版本降级  | ghostscript |
| I9UVJZ | <https://gitee.com/open_euler/dashboard?issue_id=I9UVJZ> | 【22.03-LTS-SP4-rc2】libvirt 包在22.03-SP4中相比22.03-SP3 release版本降级  | libvirt |
| I9UVK0 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK0> | 【22.03-LTS-SP4-rc2】skopeo 包在22.03-SP4中相比22.03-SP3 release版本降级  | skopeo |
| I9UVK2 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK2> | 【22.03-LTS-SP4-rc2】dwarves 包在22.03-SP4中相比22.03-SP3 release版本降级  | dwarves |
| I9UVK3 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK3> | 【22.03-LTS-SP4-rc2】openjdk-11 包在22.03-SP4中相比22.03-SP3 release版本降级  | openjdk-11 |
| I9UVK4 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK4> | 【22.03-LTS-SP4-rc2】bind 包在22.03-SP4中相比22.03-SP3 release版本降级  | bind |
| I9UVK5 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK5> | 【22.03-LTS-SP4-rc2】pcs 包在22.03-SP4中相比22.03-SP3 release版本降级  | pcs |
| I9UVK6 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK6> | 【22.03-LTS-SP4-rc2】bash 包在22.03-SP4中相比22.03-SP3 release版本降级  | bash |
| I9UVK8 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK8> | 【22.03-LTS-SP4-rc2】lwip 包在22.03-SP4中相比22.03-SP3 release版本降级  | lwip |
| I9UVK9 | <https://gitee.com/open_euler/dashboard?issue_id=I9UVK9> | 【22.03-LTS-SP4-rc2】lsof 包在22.03-SP4中相比22.03-SP3 release版本降级  | lsof |
| I9UVKA | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKA> | 【22.03-LTS-SP4-rc2】nautilus 包在22.03-SP4中相比22.03-SP3 release版本降级  | nautilus |
| I9UVKB | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKB> | 【22.03-LTS-SP4-rc2】util-linux 包在22.03-SP4中相比22.03-SP3 release版本降级  | util-linux |
| I9UVKC | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKC> | 【22.03-LTS-SP4-rc2】python-jinja2 包在22.03-SP4中相比22.03-SP3 release版本降级  | python-jinja2 |
| I9UVKD | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKD> | 【22.03-LTS-SP4-rc2】deepin-turbo 包在22.03-SP4中相比22.03-SP3 release版本降级  | deepin-turbo |
| I9UVKE | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKE> | 【22.03-LTS-SP4-rc2】ovirt-engine 包在22.03-SP4中相比22.03-SP3 release版本降级  | ovirt-engine |
| I9UVKF | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKF> | 【22.03-LTS-SP4-rc2】deepin-graphics-driver-manager 包在22.03-SP4中相比22.03-SP3 release版本降级  | deepin-graphics-driver-manager |
| I9UVKG | <https://gitee.com/open_euler/dashboard?issue_id=I9UVKG> | 【22.03-LTS-SP4-rc2】ignition 包在22.03-SP4中相比22.03-SP3 release版本降级  | ignition |
| I9UVMX | <https://gitee.com/open_euler/dashboard?issue_id=I9UVMX> | 【22.03-LTS-SP4-rc2】docker 包在22.03-SP4中相比22.03-SP3 release版本降级  | docker |
| I9UVOU | <https://gitee.com/open_euler/dashboard?issue_id=I9UVOU> | 【22.03-LTS-SP4-rc2】poissonsearch-oss 包在22.03-SP4中相比22.03-SP3 version版本降级  | poissonsearch-oss |
| I9UVOV | <https://gitee.com/open_euler/dashboard?issue_id=I9UVOV> | 【22.03-LTS-SP4-rc2】openjdk-17 包在22.03-SP4中相比22.03-SP3 version版本降级  | openjdk-17 |
| I9UWOZ | <https://gitee.com/open_euler/dashboard?issue_id=I9UWOZ> | 【22.03-LTS-SP4-rc2】gazelle 包在22.03-SP4中相比22.03-SP3 release版本降级  | gazelle |
| I9UWPT | <https://gitee.com/open_euler/dashboard?issue_id=I9UWPT> | 【22.03-LTS-SP4-rc2】openjdk-1.8.0 包在22.03-SP4中相比22.03-SP3 version版本降级  | openjdk-1.8.0 |
| I9UWQK | <https://gitee.com/open_euler/dashboard?issue_id=I9UWQK> | 【22.03-LTS-SP4 RC2】auto-bolt结合NPB应用编译ICE:during GIMPLE pass: unswitch：internal compiler error: Segmentation fault  | gcc |
| I9V325 | <https://gitee.com/open_euler/dashboard?issue_id=I9V325> | 【22.03-LTS-SP4-rc2】【arm/x86】调用PWR\_PROC\_SetWattAttrs接口，设置domainMask=1，查询结果为10  | powerapi |
| I9V34B | <https://gitee.com/open_euler/dashboard?issue_id=I9V34B> | 【22.03-LTS-SP4 RC2】\[codedb]-O2 -fllc-allocate 选项编译postgresql-11.3报internal compiler error: Aborted : during GIMPLE pass: llc\_allocate  | gcc |
| I9V3TI | <https://gitee.com/open_euler/dashboard?issue_id=I9V3TI> | 【22.03-LTS-SP4 RC2】-O3 -fwhole-program -fllc-allocate 报Segmentation fault : during GIMPLE pass: llc\_allocate  | gcc |
| I9V738 | <https://gitee.com/open_euler/dashboard?issue_id=I9V738> | 【22.03\_SP4\_RC2\_everthing】【arm\x86】obs-api版本冲突导致无法升级  | obs-server |
| I9V75L | <https://gitee.com/open_euler/dashboard?issue_id=I9V75L> | 【22.03-LTS-SP4-rc2】注册主机模板缺少ssh\_pkey字段  | aops-ceres |
| I9V7BM | <https://gitee.com/open_euler/dashboard?issue_id=I9V7BM> | 【22.03-LTS-SP4-rc2】【arm/x86】调用PWR\_PROC\_SetSmartGridState接口，返回错误码502  | powerapi |
| I9VAXG | <https://gitee.com/open_euler/dashboard?issue_id=I9VAXG> | 【22.03-LTS-SP4-rc2】【arm/x86】调用PWR\_PROC\_AddWattProcs添加不存在的进程，执行成功，查询到的进程为随机值  | powerapi |
| I9VPAX | <https://gitee.com/open_euler/dashboard?issue_id=I9VPAX> | 【22.03-LTS-SP4-rc2】【x86】确认libomp-test二进制包是否需要删除  | libomp |
| I9VPPE | <https://gitee.com/open_euler/dashboard?issue_id=I9VPPE> | 【22.03-LTS-SP4-rc2】【arm/x86】repo源中缺少arkui-linux二进制包，且只有arm架构的arkui-linux-devel二进制包，跟EBS不一致  | arkui-linux |
| I9VR7L | <https://gitee.com/open_euler/dashboard?issue_id=I9VR7L> | 【22.03-LTS-SP4-rc2】创建热补丁移除任务，提示语错误  | aops-hermes |
| I9VVTI | <https://gitee.com/open_euler/dashboard?issue_id=I9VVTI> | 【22.03-LTS-SP4-rc2】创建REPO设置任务，提示语错误  | aops-hermes |
| I9W0J1 | <https://gitee.com/open_euler/dashboard?issue_id=I9W0J1> | 【22.03-LTS-SP4-rc2】【arm/x86】eagle服务启动之后，日志中有报错：/etc/eagle/plugin/libidle\_service.so: No such file or directory  | eagle |
| IA4DBK | <https://gitee.com/open_euler/dashboard?issue_id=IA4DBK> | 【22.03-LTS-SP4-rc3】考虑是否将utshell的软件包版本命名跟openEuler社区其他包保持一致  | utshell |
| IA4MIA | <https://gitee.com/open_euler/dashboard?issue_id=IA4MIA> | 【22.03-LTS-SP4-rc3】【x86/arm】luarocks命令执行失败  | luarocks |
| IA4MKD | <https://gitee.com/open_euler/dashboard?issue_id=IA4MKD> | 【22.03-LTS-SP4-rc3】【arm/x86】安装软件包，执行oeawarectl --help，报错：error while loading shared libraries: libyaml-cpp.so.0.6: cannot open shared object file: No such file or directory  | oeAware-manager |
| IA4QCT | <https://gitee.com/open_euler/dashboard?issue_id=IA4QCT> | 【22.03-LTS-SP4-rc3】【arm/x86】安装软件包，使能存在依赖的插件实例，然后禁用其中一个依赖的实例，命令执行成功，但是当前插件实例并未禁用成功  | oeAware-manager |
| IA4R47 | <https://gitee.com/open_euler/dashboard?issue_id=IA4R47> | 【22.03-LTS-SP4-rc3】【arm/x86】安装软件包，配置enable\_list，name为不存在的插件，instances为存在的实例，重启服务，该插件使能成功  | oeAware-manager |
| IA55IU | <https://gitee.com/open_euler/dashboard?issue_i