iSulad Support for NRI ​

Overview ​

NRI (Node Resource Interface) is a public interface designed to manage node resources and serves as a general framework for extending CRI-compatible container runtime plugins.

It offers extension plugins the ability to track container states and make limited configuration adjustments. This enables users to integrate custom logic into OCI-compatible runtimes, allowing controlled modifications to containers or executing additional operations beyond the OCI scope at specific lifecycle stages. For instance, it can enhance the allocation and management of devices and other container resources.

iSulad currently supports NRI API version 0.6.1.

Configuring iSulad for NRI Support ​

Note: This feature is enabled by default only in the openeuler 24.09 release.

Required Packages ​

Install the isula-rust-extensions package.

Configuration ​

First, enable CRI v1 support, as NRI functionality is exclusive to v1:

"enable-cri-v1": true,

To use the default configuration, simply enable the following option:

"nri-support": true,

All other settings use default values.

For advanced configurations:

# add support for NRI plugin.
"nri-support": true,
# Allow connections from externally launched NRI plugins.
"disable-connections": true,
# plugin-config-path is the directory to search for plugin-specific configuration.
"plugin-config-path": "/etc/nri/conf.d"
# plugin-path is the directory to search for plugins to launch on startup.
"plugin-path": "/opt/nri/plugins"
# plugin-registration-timeout is the timeout for a plugin to register after connection.
"plugin-registration-timeout": 5
# plugin-request-timeout is the timeout for a plugin to handle an event/request.
"plugin-request-timeout": 2

Usage Example ​

A basic NRI plugin example is available in the NRI repository. Below demonstrates the logger plugin:

Clone the NRI repository and compile the plugin binary:

git clone https://github.com/containerd/nri.git
cd plugins/logger
go build .

Pre-started NRI Plugin Example ​

1. To automatically load the logger plugin during iSulad startup, place its binary in the plugin_path directory specified in the iSulad configuration and the plugin configuration in the plugin_config_path directory.

   sh

   [root@openEuler logger]# mkdir -p /opt/nri/plugins
   [root@openEuler logger]# cp 01-logger /opt/nri/plugins
   [root@openEuler logger]# ls -l /opt/nri/plugins
   total 16896
   -rwxr-xr-x. 1 root root 17300525 Aug 19 15:26 01-logger
   [root@openEuler logger]# mkdir -p /etc/nri/conf.d/
   [root@openEuler logger]# vim /etc/nri/conf.d/01-logger.conf
   [root@openEuler logger]# cat /etc/nri/conf.d/01-logger.conf
   AddAnnotation: test_nri_isula
   [root@openEuler logger]#

2. After restarting iSulad, logs are printed to the iSulad log by default if no storage location is configured. If a storage path is specified in 01-logger.conf, logs will be saved there. Example logs:

   log

   INFO   [0026] RemovePodSandbox: pod:                       
   INFO   [0026] RemovePodSandbox:    annotations:            
   INFO   [0026] RemovePodSandbox:      cri.sandbox.isulad.checkpoint: '{"version":"v1","name":"test-nri-sandbox2","ns":"testns","data":{"host_network":true},"checksum":"c86a8542e7380049831cca636355345cc5921beebc41cd93e80467491c27a8d9"}' 
   INFO   [0026] RemovePodSandbox:      cri.sandbox.network.setup.v2: "true" 
   INFO   [0026] RemovePodSandbox:      devices.nri.io/container.c0: | 
   INFO   [0026] RemovePodSandbox:        - path: /dev/nri-null 
   INFO   [0026] RemovePodSandbox:          type: c           
   INFO   [0026] RemovePodSandbox:          major: 1          
   INFO   [0026] RemovePodSandbox:          minor: 3          
   INFO   [0026] RemovePodSandbox:      io.kubernetes.cri.container-type: sandbox 
   INFO   [0026] RemovePodSandbox:      io.kubernetes.cri.sandbox-attempt: "1" 
   INFO   [0026] RemovePodSandbox:      io.kubernetes.cri.sandbox-name: test-nri-sandbox2 
   INFO   [0026] RemovePodSandbox:      io.kubernetes.cri.sandbox-namespace: testns 
   INFO   [0026] RemovePodSandbox:      io.kubernetes.cri.sandbox-uid: b49ef5ee-ee30-11ed-a05b-0242ac120003 
   INFO   [0026] RemovePodSandbox:      ulimits.nri.containerd.io/container.c0: | 
   INFO   [0026] RemovePodSandbox:        - type: RLIMIT_NOFILE 
   INFO   [0026] RemovePodSandbox:          hard: 1048576     
   INFO   [0026] RemovePodSandbox:          soft: 1048576     
   INFO   [0026] RemovePodSandbox:    id: 475daee4ca64e1d35894a5c27771bee32e55b22753f3ba80f57869e9b294a62b 
   INFO   [0026] RemovePodSandbox:    labels:                 
   INFO   [0026] RemovePodSandbox:      cri.isulad.type: podsandbox 
   INFO   [0026] RemovePodSandbox:      io.kubernetes.container.name: POD 
   INFO   [0026] RemovePodSandbox:    linux:                  
   INFO   [0026] RemovePodSandbox:      pod_resources:        
   INFO   [0026] RemovePodSandbox:        cpu:                
   INFO   [0026] RemovePodSandbox:          period: {}        
   INFO   [0026] RemovePodSandbox:          quota: {}         
   INFO   [0026] RemovePodSandbox:          shares:           
   INFO   [0026] RemovePodSandbox:            value: 2        
   INFO   [0026] RemovePodSandbox:        memory:             
   INFO   [0026] RemovePodSandbox:          limit: {}         
   INFO   [0026] RemovePodSandbox:    name: k8s_POD_test-nri-sandbox2_testns_b49ef5ee-ee30-11ed-a05b-0242ac120003_1 
   INFO   [0026] RemovePodSandbox:    namespace: testns       
   INFO   [0026] RemovePodSandbox:    uid: b49ef5ee-ee30-11ed-a05b-0242ac120003 
             iSula 20240829135021.168 - Event: {Object: CRI, Type: Removed Pod: 47}

External NRI Plugin Registration ​

To register the logger plugin after iSulad has started, execute the plugin binary:

➜  logger git:(v0.6.1) ✗ sudo ./02-logger
INFO   [0000] Created plugin 02-logger (02-logger, handles RunPodSandbox,StopPodSandbox,RemovePodSandbox,CreateContainer,PostCreateContainer,StartContainer,PostStartContainer,UpdateContainer,PostUpdateContainer,StopContainer,RemoveContainer) 
INFO   [0000] Registering plugin 02-logger...              
INFO   [0000] Configuring plugin 02-logger for runtime v2/2.0.0-beta.2+unknown... 
INFO   [0000] got configuration data: "" from runtime v2 2.0.0-beta.2+unknown 
INFO   [0000] Subscribing plugin 02-logger (02-logger) for events RunPodSandbox,StopPodSandbox,RemovePodSandbox,CreateContainer,PostCreateContainer,StartContainer,PostStartContainer,UpdateContainer,PostUpdateContainer,StopContainer,RemoveContainer 
INFO   [0000] Started plugin 02-logger...                  
INFO   [0000] Synchronize: pods:                           
INFO   [0000] Synchronize:    null                         
INFO   [0000] Synchronize: containers:                     
INFO   [0000] Synchronize:    null

Restrictions ​

Interface Parameter Restrictions ​

Certain parameters cannot be modified:

// Container to evict (IOW unsolicitedly stop).
ContainerEviction evict;

// ref:https://github.com/containerd/containerd/pull/5490
// - User defines blockio classes, for example: ThrottledIO and LowLatency. Class names are not restricted, and the number of classes is not limited.
// iSulad not support
OptionalString blockio_class;

// iSulad now not support the following hook types
message Hooks {
  repeated Hook create_runtime = 2;
  repeated Hook create_container = 3;
  repeated Hook start_container = 4;
}

Usage Restrictions ​

1. iSulad currently supports NRI only through CRI V1.
2. The supported NRI API version is 0.6.1.
3. If a plugin crashes, iSulad logs an error only during the next invocation.
4. Externally registered plugins are not forcefully terminated if iSulad exits. Their lifecycle remains independent of iSulad.