Frequently Asked Questions

openEuler Copilot System FAQ (Network Environment)

1. HuggingFace Connection Issue

text
File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.eanconectionError: <urlib3.comnection.Hipscomnection object at exfblab6490>: Failed to establish a new conmection: [Errno 101] Network is unreachable
  • Solution
bash
pip3 install -U huggingface_hub
export HF_ENDPOINT=https://hf-mirror.com

2. Querying the RAG Service API

  • Enter the pod for RAG.
bash
curl  -k -X POST "http://localhost:8005/kb/get_answer"  -H "Content-Type: application/json"  -d '{ \
      "question": "", \
      "kb_sn": "default_test", \
      "fetch_source": true  }'

3. Helm Upgrade Cluster Errors

text
Error: INSTALLATI0N FAILED: Kubernetes cluster unreachable: Get "http:/localhost:880/version": dial tcp [:1:8089: connect: connection refused

or

text
Error: UPGRADE FAILED: Kubernetes cluster unreachable: the server could not find the requested resource
  • Solution
bash
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

4. Pod Log Access Issues

text
[root@localhost euler-copilot]# kubectl logs rag-deployservice65c75c48d8-44vcp-n euler-copilotDefaulted container "rag" out of: rag.rag-copy secret (init)Error from server: Get "https://172.21.31.11:10250/containerlogs/euler copilot/rag deploy"service 65c75c48d8-44vcp/rag": Forbidden
  • Solution

Ensure the local Kubernetes node IP address is excluded from proxy settings:

bash
cat /etc/systemd/system/k3s.service.env
http_proxy="http://172.21.60.51:3128"
https_proxy="http://172.21.60.51:3128"
no_proxy=172.21.31.10 # Add the node IP address.

5. Streaming Response Issues in LLM API in the GPU environment

Curl requests fail when "stream": true but succeed with "stream": false.

bash
curl http://localhost:30000/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer sk-123456"   -d '{
"model": "qwen1.5",
"messages": [
{
"role": "system",
"content": "You are an sentiment analysis expert..."
},
{
"role": "user",
"content": "Hello"
}
],
"stream": true,
"n": 1,
"max_tokens": 32768
}'
  • Solution:
bash
pip install Pydantic=1.10.13

6. SGLang Deployment

bash
# 1. Activate the Conda environment (assuming the name is "myenv").
conda activate myenv

# 2. Install sglang[all] and flashinfer.
pip install sglang[all]==0.3.0
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/

# 3. Launch the server.
python -m sglang.launch_server --served-model-name Qwen2.5-32B --model-path Qwen2.5-32B-Instruct-AWQ --host 0.0.0.0 --port 8001 --api-key sk-12345 --mem-fraction-static 0.5 --tp 8
  • Verification:
bash
pip show sglang
pip show flashinfer
  • Note:
  1. API key: Ensure the --api-key value is valid.
  2. Model path: Verify the --model-path points to an existing model directory.
  3. CUDA version: flashinfer requires CUDA 12.1 and PyTorch 2.4.
  4. Thread pool size: Adjust --tp (for example, --tp 8 for 8 GPUs) based on available resources.

7. Embedding API Request

bash
curl -k -X POST http://$IP:8001/embedding \
     -H "Content-Type: application/json" \
     -d '{"texts": ["sample text 1", "sample text 2"]}'
# Replace $IP with the internal network address of the vectorize embedding service.

8. Certificate Generation

bash
# Download mkcert: https://github.com/FiloSottile/mkcert/releases
# x86_64
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64
# arm64
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-arm64
# 2. Generate keys.
mkcert -install
mkcert example.com # Supports domains or IP addresses.
# 3. Copy certificates and keys to /home/euler-copilot-framework_openeuler/euler-copilot-helm/chart_ssl/traefik-secret.yaml, then apply:
kubectl apply -f traefik-secret.yaml