openEuler Copilot System FAQ (Network Environment)
1. HuggingFace Connection Issue
text
File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.eanconectionError: <urlib3.comnection.Hipscomnection object at exfblab6490>: Failed to establish a new conmection: [Errno 101] Network is unreachable
- Solution
bash
pip3 install -U huggingface_hub
export HF_ENDPOINT=https://hf-mirror.com
2. Querying the RAG Service API
- Enter the pod for RAG.
bash
curl -k -X POST "http://localhost:8005/kb/get_answer" -H "Content-Type: application/json" -d '{ \
"question": "", \
"kb_sn": "default_test", \
"fetch_source": true }'
3. Helm Upgrade Cluster Errors
text
Error: INSTALLATI0N FAILED: Kubernetes cluster unreachable: Get "http:/localhost:880/version": dial tcp [:1:8089: connect: connection refused
or
text
Error: UPGRADE FAILED: Kubernetes cluster unreachable: the server could not find the requested resource
- Solution
bash
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
4. Pod Log Access Issues
text
[root@localhost euler-copilot]# kubectl logs rag-deployservice65c75c48d8-44vcp-n euler-copilotDefaulted container "rag" out of: rag.rag-copy secret (init)Error from server: Get "https://172.21.31.11:10250/containerlogs/euler copilot/rag deploy"service 65c75c48d8-44vcp/rag": Forbidden
- Solution
Ensure the local Kubernetes node IP address is excluded from proxy settings:
bash
cat /etc/systemd/system/k3s.service.env
http_proxy="http://172.21.60.51:3128"
https_proxy="http://172.21.60.51:3128"
no_proxy=172.21.31.10 # Add the node IP address.
5. Streaming Response Issues in LLM API in the GPU environment
Curl requests fail when "stream": true
but succeed with "stream": false
.
bash
curl http://localhost:30000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer sk-123456" -d '{
"model": "qwen1.5",
"messages": [
{
"role": "system",
"content": "You are an sentiment analysis expert..."
},
{
"role": "user",
"content": "Hello"
}
],
"stream": true,
"n": 1,
"max_tokens": 32768
}'
- Solution:
bash
pip install Pydantic=1.10.13
6. SGLang Deployment
bash
# 1. Activate the Conda environment (assuming the name is "myenv").
conda activate myenv
# 2. Install sglang[all] and flashinfer.
pip install sglang[all]==0.3.0
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/
# 3. Launch the server.
python -m sglang.launch_server --served-model-name Qwen2.5-32B --model-path Qwen2.5-32B-Instruct-AWQ --host 0.0.0.0 --port 8001 --api-key sk-12345 --mem-fraction-static 0.5 --tp 8
- Verification:
bash
pip show sglang
pip show flashinfer
- Note:
- API key: Ensure the
--api-key
value is valid. - Model path: Verify the
--model-path
points to an existing model directory. - CUDA version:
flashinfer
requires CUDA 12.1 and PyTorch 2.4. - Thread pool size: Adjust
--tp
(for example,--tp 8
for 8 GPUs) based on available resources.
7. Embedding API Request
bash
curl -k -X POST http://$IP:8001/embedding \
-H "Content-Type: application/json" \
-d '{"texts": ["sample text 1", "sample text 2"]}'
# Replace $IP with the internal network address of the vectorize embedding service.
8. Certificate Generation
bash
# Download mkcert: https://github.com/FiloSottile/mkcert/releases
# x86_64
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64
# arm64
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-arm64
# 2. Generate keys.
mkcert -install
mkcert example.com # Supports domains or IP addresses.
# 3. Copy certificates and keys to /home/euler-copilot-framework_openeuler/euler-copilot-helm/chart_ssl/traefik-secret.yaml, then apply:
kubectl apply -f traefik-secret.yaml
Bug