openEuler Copilot System FAQ (Network Environment)

1. HuggingFace Connection Issue

text

File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.eanconectionError: <urlib3.comnection.Hipscomnection object at exfblab6490>: Failed to establish a new conmection: [Errno 101] Network is unreachable

Solution

bash

pip3 install -U huggingface_hub
export HF_ENDPOINT=https://hf-mirror.com

2. Querying the RAG Service API

Enter the pod for RAG.

bash

curl  -k -X POST "http://localhost:8005/kb/get_answer"  -H "Content-Type: application/json"  -d '{ \
      "question": "", \
      "kb_sn": "default_test", \
      "fetch_source": true  }'

3. Helm Upgrade Cluster Errors

text

Error: INSTALLATI0N FAILED: Kubernetes cluster unreachable: Get "http:/localhost:880/version": dial tcp [:1:8089: connect: connection refused

text

Error: UPGRADE FAILED: Kubernetes cluster unreachable: the server could not find the requested resource

Solution

bash

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

4. Pod Log Access Issues

text

[root@localhost euler-copilot]# kubectl logs rag-deployservice65c75c48d8-44vcp-n euler-copilotDefaulted container "rag" out of: rag.rag-copy secret (init)Error from server: Get "https://172.21.31.11:10250/containerlogs/euler copilot/rag deploy"service 65c75c48d8-44vcp/rag": Forbidden

Solution

Ensure the local Kubernetes node IP address is excluded from proxy settings:

bash

cat /etc/systemd/system/k3s.service.env
http_proxy="http://172.21.60.51:3128"
https_proxy="http://172.21.60.51:3128"
no_proxy=172.21.31.10 # Add the node IP address.

5. Streaming Response Issues in LLM API in the GPU environment

Curl requests fail when "stream": true but succeed with "stream": false.

bash

curl http://localhost:30000/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer sk-123456"   -d '{
"model": "qwen1.5",
"messages": [
{
"role": "system",
"content": "You are an sentiment analysis expert..."
},
{
"role": "user",
"content": "Hello"
}
],
"stream": true,
"n": 1,
"max_tokens": 32768
}'

Solution:

bash

pip install Pydantic=1.10.13

6. SGLang Deployment

bash

# 1. Activate the Conda environment (assuming the name is "myenv").
conda activate myenv

# 2. Install sglang[all] and flashinfer.
pip install sglang[all]==0.3.0
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/

# 3. Launch the server.
python -m sglang.launch_server --served-model-name Qwen2.5-32B --model-path Qwen2.5-32B-Instruct-AWQ --host 0.0.0.0 --port 8001 --api-key sk-12345 --mem-fraction-static 0.5 --tp 8

Verification:

bash

pip show sglang
pip show flashinfer

Note:

API key: Ensure the --api-key value is valid.
Model path: Verify the --model-path points to an existing model directory.
CUDA version: flashinfer requires CUDA 12.1 and PyTorch 2.4.
Thread pool size: Adjust --tp (for example, --tp 8 for 8 GPUs) based on available resources.

7. Embedding API Request

bash

curl -k -X POST http://$IP:8001/embedding \
     -H "Content-Type: application/json" \
     -d '{"texts": ["sample text 1", "sample text 2"]}'
# Replace $IP with the internal network address of the vectorize embedding service.

8. Certificate Generation

bash

# Download mkcert: https://github.com/FiloSottile/mkcert/releases
# x86_64
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64
# arm64
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-arm64
# 2. Generate keys.
mkcert -install
mkcert example.com # Supports domains or IP addresses.
# 3. Copy certificates and keys to /home/euler-copilot-framework_openeuler/euler-copilot-helm/chart_ssl/traefik-secret.yaml, then apply:
kubectl apply -f traefik-secret.yaml

View source on Gitee

Bug

openEuler Copilot System FAQ (Network Environment) ​

1. HuggingFace Connection Issue ​

2. Querying the RAG Service API ​

3. Helm Upgrade Cluster Errors ​

4. Pod Log Access Issues ​

5. Streaming Response Issues in LLM API in the GPU environment ​

6. SGLang Deployment ​

7. Embedding API Request ​

8. Certificate Generation ​

openEuler Copilot System FAQ (Network Environment)

1. HuggingFace Connection Issue

2. Querying the RAG Service API

3. Helm Upgrade Cluster Errors

4. Pod Log Access Issues

5. Streaming Response Issues in LLM API in the GPU environment

6. SGLang Deployment

7. Embedding API Request

8. Certificate Generation