1 - Installing Helm 3

This section describes how to install Helm 3.

For details, see https://helm.sh/docs/intro/install/ .

Prerequisites

Ensure that the master node in the Kubernetes cluster can access the Internet.

Procedure

  1. Run the following command to download the Helm 3 installation script.

    curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
    
  2. Run the following command to modify the permission on the Helm 3 installation script.

    chmod 700 get_helm.sh
    
  3. Determine the Helm version to be installed based on the version mapping between Helm and Kubernetes. For details about the version mapping, see Helm Version Support Policy . Then run the following command to change the DESIRED_VERSION environment variable to the Helm version to be installed and run the installation command.

    DESIRED_VERSION=v3.9.0 ./get_helm.sh
    
  4. Run the following command to check whether Helm 3 of the specified version is successfully installed.

    helm version
    

    If the following information is displayed, the installation is successful.

    version.BuildInfo{Version:"v3.9.0", GitCommit:"7ceeda6c585217a19a1131663d8cd1f7d641b2a7", GitTreeState:"clean", GoVersion:"go1.17.5"}
    

2 - Collecting Information

2.1 - Obtaining the CSI Version

This section describes how to view the CSI version.

Procedure

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Run the following command to query information about the node where huawei-csi-node resides.

    kubectl get pod -A  -owide | grep huawei-csi-node
    

    The following is an example of the command output.

    NAMESPACE     NAME                                       READY   STATUS    RESTARTS        AGE     IP               NODE            NOMINATED NODE   READINESS GATES
    huawei-csi    huawei-csi-node-87mss                      3/3     Running   0               6m41s   192.168.129.155      node-1          <none>           <none>
    huawei-csi    huawei-csi-node-xp8cc                      3/3     Running   0               6m41s   192.168.129.156      node-2          <none>           <none
    
  3. Use a remote access tool, such as PuTTY, to log in to any node where huawei-csi-node resides through the node IP address.

  4. Run the following command to view the CSI version.

    cat /var/lib/kubelet/plugins/csi.huawei.com/version
    

    The version information is displayed as follows:

    4.9.0
    

2.2 - Viewing Huawei CSI Logs

Viewing Logs of the huawei-csi-controller Service

  1. Run the following command to obtain the node where huawei-csi-controller is located.

    kubectl get pod -A -o wide | grep huawei
    

    The following is an example of the command output, where IP indicates the node IP address and NODE indicates the node name.

    NAME                                    READY   STATUS    RESTARTS   AGE     IP             NODE         NOMINATED NODE   READINESS GATES
    huawei-csi-controller-695b84b4d8-tg64l  9/9     Running  0          14s     <host1-ip>     <host1-name> <none>           <none>
    
  2. Use a remote access tool, such as PuTTY, to log in to the node where the huawei-csi-controller service resides in the Kubernetes cluster through the management IP address.

  3. Go to the log directory.

    cd /var/log/huawei
    
  4. Run the following command to view the customized output logs of the container.

    vi huawei-csi-controller
    
  5. Go to the container directory.

    cd /var/log/containers
    
  6. Run the following command to view the standard output logs of the container.

    vi huawei-csi-controller-<name>_huawei-csi_huawei-csi-driver-<container-id>.log
    

Viewing Logs of the huawei-csi-node Service

  1. Run the following command to obtain the node where huawei-csi-node is located.

    kubectl get pod -A -o wide | grep huawei
    

    The following is an example of the command output, where IP indicates the node IP address and NODE indicates the node name.

    NAME                     READY   STATUS    RESTARTS   AGE     IP             NODE         NOMINATED NODE   READINESS GATES
    huawei-csi-node-g6f7z    3/3     Running  0          14s     <host2-ip>     <host2-name> <none>           <none>
    
  2. Use a remote access tool, such as PuTTY, to log in to the node where the huawei-csi-node service resides in the Kubernetes cluster through the management IP address.

  3. Go to the log directory.

    cd /var/log/huawei
    
  4. Run the following command to view the customized output logs of the container.

    vi huawei-csi-node
    
  5. Go to the container directory.

    cd /var/log/containers
    
  6. Run the following command to view the standard output logs of the container.

    vi huawei-csi-node-<name>_huawei-csi_huawei-csi-driver-<container-id>.log
    

2.3 - Collecting Logs

Performing Check Before Collection

  1. Use a remote access tool, such as PuTTY, to log in to the node where the oceanctl tool is installed in the Kubernetes cluster through the management IP address.

  2. Run the following command. The displayed version is v4.9.0.

    oceanctl version
    

    The following is an example of the command output.

    Oceanctl Version: v4.9.0
    
  3. Run the oceanctl collect logs –help command. The following information is displayed.

    $ oceanctl collect logs --help
    Collect logs of one or more nodes in specified namespace in Kubernetes
    
    Usage:
      oceanctl collect logs [flags]
    
    Examples:
      # Collect logs of all nodes in specified namespace
      oceanctl collect logs -n <namespace>
    
      # Collect logs of specified node in specified namespace
      oceanctl collect logs -n <namespace> -N <node>
    
      # Collect logs of all nodes in specified namespace
      oceanctl collect logs -n <namespace> -a
    
      # Collect logs of all nodes in specified namespace with a maximum of 50 nodes collected at the same time
      oceanctl collect logs -n <namespace> -a --threads-max=50
    
      # Collect logs of specified node in specified namespace
      oceanctl collect logs -n <namespace> -N <node> -a
    
    Flags:
      -a, --all                Collect all nodes messages
      -h, --help               help for logs
      -n, --namespace string   namespace of resources
      -N, --nodename string    Specify the node for which information is to be collected.
          --threads-max int    set maximum number[1~1000] of threads for nodes to be collected. (default 50)
    
    Global Flags:
          --log-dir string   Specify the directory for printing log files. (default "/var/log/huawei")
    
  4. Run the following command to check whether a Pod is started properly. In the command, huawei-csi indicates the namespace for installing CSI.

    kubectl get deployment -n huawei-csi
    

    The following is an example of the command output.

    NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
    huawei-csi-controller   1/1     1            1           21h
    

Collecting All Logs in the CSI Namespace Using oceanctl

  1. Use a remote access tool, such as PuTTY, to log in to the node checked in Performing Check Before Collection through the management IP address.

  2. Run the oceanctl collect logs -n <namespace> -a –threads-max=<max_node_processing_num> command to collect CSI logs of all nodes where CSI containers reside in the cluster. In the command, threads-max indicates the maximum number of nodes for which logs can be collected at the same time. The default value is 50. You can set the value based on the host performance and load.

    oceanctl collect logs -n huawei-csi -a --threads-max=10
    
  3. Check the log package generated in the /tmp directory. You can run the unzip <zip_name> -d collect_logs command to decompress the log package. In the preceding command, <zip_name> indicates the package name.

    # date
    Wed Sep 20 02:49:24 EDT 2023
    
    # ls
    huawei-csi-2023-09-20-02:48:22-all.zip
    

Collecting the Log of a Single CSI Node Using oceanctl

  1. Use a remote access tool, such as PuTTY, to log in to the node checked in Performing Check Before Collection through the management IP address.

  2. Run the **oceanctl collect logs -n <namespace> -N **<nodeName> command to collect CSI logs of all nodes where CSI containers reside in the cluster.

    oceanctl collect logs -n huawei-csi -N node-1
    
  3. Check the log package generated in the /tmp directory. You can run the unzip <zip_name> -d collect_logs command to decompress the log package. In the preceding command, <zip_name> indicates the package name.

    # date
    Thu Sep 21 04:08:47 EDT 2023
    
    # ls
    huawei-csi-2023-09-21-04:05:15-node-1.zip
    

3 - Downloading a Container Image

Downloading a Container Image Using containerd

  1. Run the following command to download an image to a local path. In the command, image:tag indicates the image to be pulled and its tag.

    ctr image pull <image>:<tag>
    
  2. Run the following command to export the image to a file. In the command, image:tag indicates the image to be exported, and file indicates the name of the exported image file.

    ctr image export <file>.tar <image>:<tag>
    

Downloading a Container Image Using Docker

  1. Run the following command to download an image to a local path. In the command, image:tag indicates the image to be pulled.

    docker pull <image>:<tag>
    
  2. Run the following command to export the image to a file. In the command, image:tag indicates the image to be exported, and file indicates the name of the exported image file.

    docker save <image>:<tag> -o <file>.tar
    

Downloading a Container Image Using Podman

  1. Run the following command to download an image to a local path. In the command, image:tag indicates the image to be pulled.

    podman pull <image>:<tag>
    
  2. Run the following command to export the image to a file. In the command, image:tag indicates the image to be exported, and file indicates the name of the exported image file.

    podman save <image>:<tag> -o <file>.tar 
    

4 - Updating the huawei-csi-controller or huawei-csi-node Service

Perform this operation when you need to update the huawei-csi-controller or huawei-csi-node service, for example, changing the number of copies for the huawei-csi-controller service.

Procedure

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Go to the /helm/esdk directory and run the following command to obtain the original service configuration file. helm-huawei-csi indicates the Helm chart name specified during the installation of the earlier version, and huawei-csi indicates the Helm chart namespace specified during the installation of the earlier version. For details about the component package path, see Table 1 .

    helm get values helm-huawei-csi -n huawei-csi -a > ./update-values.yaml
    
  3. Run the vi update-values.yaml command to open the file obtained in 2 and modify the configuration items by referring to Parameters in the values.yaml File of Helm . After the modification, press Esc and enter :wq! to save the modification.

  4. Run the following command to update Huawei CSI services.

    helm upgrade helm-huawei-csi ./ -n huawei-csi  -f ./update-values.yaml
    

5 - Modifying the Log Output Mode

huawei-csi supports two log output modes: file and console. file indicates that logs are output to the fixed directory (/var/log/huawei), and console indicates that logs are output to the standard directory of the container. You can set the log output mode as required. The default mode is file.

Procedure

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Go to the /helm/esdk directory and run the following command to obtain the original service configuration file. helm-huawei-csi indicates the Helm chart name specified during the installation of the earlier version, and huawei-csi indicates the Helm chart namespace specified during the installation of the earlier version. For details about the component package path, see Table 1 .

    helm get values helm-huawei-csi -n huawei-csi -a > ./update-values.yaml
    
  3. Run the vi update-values.yaml command to open the file obtained in 2 and modify the configuration items. After the modification, press Esc and enter :wq! to save the modification.

    # The CSI driver parameter configuration
    csiDriver:
      # Driver name, it is strongly recommended not to modify this parameter
      # The CCE platform needs to modify this parameter, e.g. csi.oceanstor.com
      driverName: csi.huawei.com
      # Endpoint, it is strongly recommended not to modify this parameter
      endpoint: /csi/csi.sock
      # DR Endpoint, it is strongly recommended not to modify this parameter
      drEndpoint: /csi/dr-csi.sock
      # Maximum number of concurrent disk scans or detaches, support 1~10
      connectorThreads: 4
      # Flag to enable or disable volume multipath access, support [true, false]
      volumeUseMultipath: true
      # Multipath software used by fc/iscsi. support [DM-multipath, HW-UltraPath, HW-UltraPath-NVMe]
      scsiMultipathType: DM-multipath
      # Multipath software used by roce/fc-nvme. only support [HW-UltraPath-NVMe]
      nvmeMultipathType: HW-UltraPath-NVMe
      # Timeout interval for waiting for multipath aggregation when DM-multipath is used on the host. support 1~600
      scanVolumeTimeout: 3
      # Timeout interval for running command on the host. support 1~600
      execCommandTimeout: 30
      # check the number of paths for multipath aggregation
      # Allowed values:
      #   true: the number of paths aggregated by DM-multipath is equal to the number of online paths
      #   false: the number of paths aggregated by DM-multipath is not checked.
      # Default value: false
      allPathOnline: false
      # Interval for updating backend capabilities. support 60~600
      backendUpdateInterval: 60
      # Huawei-csi-controller log configuration
      controllerLogging:
        # Log record type, support [file, console]
        module: file
        # Log Level, support [debug, info, warning, error, fatal]
        level: info
        # Directory for storing logs
        fileDir: /var/log/huawei
        # Size of a single log file
        fileSize: 20M
        # Maximum number of log files that can be backed up.
        maxBackups: 9
      # Huawei-csi-node log configuration
      nodeLogging:
        # Log record type, support [file, console]
        module: file
        # Log Level, support [debug, info, warning, error, fatal]
        level: info
        # Directory for storing logs
        fileDir: /var/log/huawei
        # Size of a single log file
        fileSize: 20M
        # Maximum number of log files that can be backed up.
        maxBackups: 9
    
  4. Run the following command to update the log configuration.

    helm upgrade helm-huawei-csi ./ -n huawei-csi  -f ./update-values.yaml
    

6 - Enabling the ReadWriteOncePod Feature Gate

The ReadWriteOnce access mode is the fourth access mode introduced by Kubernetes v1.22 for PVs and PVCs. If you create a Pod using a PVC in ReadWriteOncePod access mode, Kubernetes ensures that the Pod is the only Pod in the cluster that can read or write the PVC.

If the Kubernetes version is earlier than v1.29, you need to enable the ReadWriteOncePod feature in feature-gates of kube-apiserver, kube-scheduler, and kubelet.


Currently, the ReadWriteOncePod feature gate cannot be enabled on the CCE or CCE Agile platform.

Procedure

  1. Enable the ReadWriteOncePod feature gate for kube-apiserver.

    1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

    2. Run the vi /etc/kubernetes/manifests/kube-apiserver.yaml command, press I or Insert to enter the insert mode, and add –feature-gates=ReadWriteOncePod=true to the kube-apiserver container. After the modification is complete, press Esc and enter :wq! to save the modification.

      ...
      spec:
        containers:
        - command:
          - kube-apiserver
          - --feature-gates=ReadWriteOncePod=true
          ...
      


      After the editing is complete, Kubernetes will automatically apply the updates.

  2. Enable the ReadWriteOncePod feature gate for kube-scheduler.

    1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

    2. Run the vi /etc/kubernetes/manifests/kube-scheduler.yaml command, press I or Insert to enter the insert mode, and add –feature-gates=ReadWriteOncePod=true to the kube-scheduler container. After the modification is complete, press Esc and enter :wq! to save the modification.

      ...
      spec:
        containers:
        - command:
          - kube-scheduler
          - --feature-gates=ReadWriteOncePod=true
          ...
      


      After the editing is complete, Kubernetes will automatically apply the updates.

  3. Enable the ReadWriteOncePod feature gate for kubelet.


    The dynamic Kubelet configuration function is not used since v1.22 and deleted in v1.24. Therefore, you need to perform the following operations on kubelet on each worker node in the cluster.

    1. Use a remote access tool, such as PuTTY, to log in to any worker node in the Kubernetes cluster through the management IP address.

    2. Run the vi /var/lib/kubelet/config.yaml command, press I or Insert to enter the editing state, and add ReadWriteOncePod: true to the featureGates field of the KubeletConfiguration object. If the featureGates field does not exist, add it at the same time. After the modification is complete, press Esc and enter :wq! to save the modification.

      apiVersion: kubelet.config.k8s.io/v1beta1
      featureGates:
        ReadWriteOncePod: true
        ...
      


      The default path of the kubelet configuration file is /var/lib/kubelet/config.yaml. Enter the path based on site requirements.

    3. After the configuration is complete, run the systemctl restart kubelet command to restart kubelet.

7 - Configuring Access to the Kubernetes Cluster as a Non-root User

Procedure

  1. Copy the authentication file of the Kubernetes cluster and modify /etc/kubernetes/admin.conf to be the actual authentication file.

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    
  2. Change the user and user group of the authentication file.

    sudo chown $(id -u):$(id -g) $HOME/.kube/config
    
  3. Configure the KUBECONFIG environment variable of the current user. The following uses Ubuntu 20.04 as an example.

    echo "export KUBECONFIG=$HOME/.kube/config" >> ~/.bashrc
    source ~/.bashrc
    

8 - Configuring IPv6

This section describes how to use Huawei CSI to configure IPv6.

8.1 - Configuring IPv6 During the Initial Installation of Huawei CSI

This section describes how to specify IPv6 during the initial installation of Huawei CSI.

Prerequisites

  • IPv6 can be configured only when the storage backend type is oceanstor-nas, oceanstor-san, or oceanstor-dtree.
  • The host environment and Kubernetes cluster environment support IPv6. For details, see IPv4/IPv6 dual-stack of Kubernetes .

Procedure

  1. Install the CSI-dependent components by following the instructions in 1 to 7 in Installation Procedure .

  2. Run the vi values.yaml command to open the configuration file and modify the service.ipFamilyPolicy and service.ipFamilies configuration items. For details about the parameters, see Table 5 .

    The following is an example:

    service:
      ipFamilyPolicy: SingleStack
      ipFamilies:
        - IPv6
    
  3. Install and deploy Huawei CSI by following the instructions in 8 to 9 in Installation Procedure .

  4. Run the following command to check the status of the service in the huawei-csi namespace.

    kubectl get service -n huawei-csi
    

    The following is an example of the command output.

    NAME                    TYPE        CLUSTER-IP         EXTERNAL-IP   PORT(S)    AGE
    huawei-csi-controller   ClusterIP   fd00:10:96::8136   <none>        4433/TCP   19m
    
  5. Run the following command to check the value of ipFamilies for the huawei-csi-controller service.

    kubectl get svc -n huawei-csi huawei-csi-controller -o=jsonpath='{.spec.ipFamilies}'
    

    The following is an example of the command output.

    ["IPv6"]
    
  6. Run the following command to check the value of ipFamilyPolicy for the huawei-csi-controller service.

    kubectl get svc -n huawei-csi huawei-csi-controller -o=jsonpath='{.spec.ipFamilyPolicy}'
    

    The following is an example of the command output.

    SingleStack
    

8.2 - Switching from IPv4 to IPv6 for Huawei CSI

This section describes how to switch from IPv4 to IPv6 when updating Huawei CSI.

Prerequisites

  • The host environment and Kubernetes cluster environment support IPv6. For details, see IPv4/IPv6 dual-stack of Kubernetes .
  • Huawei CSI has been installed and services are running properly.

Procedure

  1. Run the following command to obtain the original service configuration file. helm-huawei-csi indicates the Helm chart name specified during the installation of the earlier version, and huawei-csi indicates the Helm chart namespace specified during the installation of the earlier version.

    helm get values helm-huawei-csi -n huawei-csi -a > ./update-values.yaml
    
  2. Run the vi update-values.yaml command to open the file obtained in 1 and modify the service.ipFamilyPolicy and service.ipFamilies configuration items. For details about the parameters, see Table 5 .

    The following is an example:

    service:
      ipFamilyPolicy: SingleStack
      ipFamilies:
        - IPv6
    
  3. Before switching to IPv6, run the following command to delete the service in the huawei-csi namespace.

    kubectl delete service -n huawei-csi --all
    
  4. Run the following command to upgrade Huawei CSI. In the following command, helm-huawei-csi indicates the specified Helm chart name, huawei-csi indicates the specified Helm chart namespace, and update-values.yaml indicates the file obtained in 1 .

    helm upgrade helm-huawei-csi ./ -n huawei-csi -f ./values.yaml -f ./update-values.yaml
    
  5. Run the following command to query the ipFamilies parameter of the huawei-csi-controller service after the update.

    kubectl get svc -n huawei-csi huawei-csi-controller -o=jsonpath='{.spec.ipFamilies}'
    

    The expected output is as follows:

    ["IPv6"]
    

9 - Configuring Storage Topology Awareness

In the Kubernetes cluster, resources can be scheduled and provisioned based on the topology labels of nodes and the topology capabilities supported by storage backends.

Prerequisites

You need to configure topology labels on worker nodes in the cluster. The method is as follows:

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Run the following command to view information about worker nodes in the current cluster.

    kubectl get node
    

    The following is an example of the command output.

    NAME     STATUS   ROLES                      AGE   VERSION
    node01   Ready    controlplane,etcd,worker   42d   v1.22.3
    node02   Ready    worker                     42d   v1.22.3
    node03   Ready    worker                     42d   v1.22.3
    
  3. Run the following command to configure a topology label for a worker node. In the preceding command, nodename indicates the name of a worker node. For details about the key and value parameters, see Table 1 .

    kubectl label node <nodename> <key>=<value>
    

    Table 1 Description of topology label parameters

    Parameter

    Description

    Remarks

    <key>

    Unique identifier of a topology label.

    The value can be zone, region, or protocol.<protocol>.

    <protocol> can be set to iscsi, nfs, fc, or roce.

    <value>

    Value of a topology label.

    If key is set to zone or region, value is a user-defined parameter.

    If key is set to protocol.<protocol>, value is fixed at csi.huawei.com.

    • A topology label must start with topology.kubernetes.io. Topology label examples:
    • Example 1: topology.kubernetes.io/region=China-west
    • Example 2: topology.kubernetes.io/zone=ChengDu
    • Example 3: topology.kubernetes.io/protocol.iscsi=csi.huawei.com
    • Example 4: topology.kubernetes.io/protocol.fc=csi.huawei.com
    • A key in a topology label on a node can have only one value.
    • If multiple protocols are configured in a topology label on a node, when you configure a StorageClass, the StorageClass needs to meet only one of the protocols.
    • If both the region and the zone are configured in a topology label on a node, when you configure a StorageClass, the StorageClass must meet all filter criteria.
  4. Run the following command to view the label information about all worker nodes in the current cluster.

    kubectl get nodes -o=jsonpath='{range .items[*]}[{.metadata.name}, {.metadata.labels}]{"\n"}{end}' | grep --color "topology.kubernetes.io"
    

    The following is an example of the command output.

    [node01,"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"node01","kubernetes.io/os":"linux","node-role.kubernetes.io/controlplane":"true","node-role.kubernetes.io/etcd":"true","node-role.kubernetes.io/worker":"true","topology.kubernetes.io/zone":"ChengDu"}]
    

Procedure

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Go to the directory where the Helm project is located. If the previous Helm project cannot be found, copy the helm directory in the component package to any directory on the master node. For details about the component package path, see Table 1 .

  3. Go to the backend service configuration directory /examples/backend/ and back up the backend.yaml file.

    cp backend.yaml backend.yaml.bak
    
  4. Run the vi backend.yaml command to open the file and configure topology awareness as required. The following is an example. After the modification is complete, press Esc and enter :wq! to save the modification.

    storage: "oceanstor-san"
    name: "dorado-iscsi-155"
    namespace: "huawei-csi"
    urls:
      - "https://192.168.129.155:8088"
    pools:
      - "StoragePool001"
    parameters:
      protocol: "iscsi"
      portals:
        - "10.10.30.20"
        - "10.10.30.21"
    supportedTopologies:
      - { "topology.kubernetes.io/region": "China-west", "topology.kubernetes.io/zone": "ChengDu" }
      - { "topology.kubernetes.io/region": "China-south","topology.kubernetes.io/zone": "ShenZhen" }
    maxClientThreads: "30"
    
  5. Run the following command to delete the storage backend to be modified. In the command, dorado-iscsi-155 indicates the storage backend name.

    oceanctl delete backend dorado-iscsi-155 -n huawei-csi
    
  6. Run the following command to create a storage backend.

    oceanctl create backend -f ../examples/backend/backend.yaml -i yaml
    

    Enter the storage user name and password as prompted.

    Please enter this backend user name:admin
    Please enter this backend password:
    
  7. Run the vi StorageClass.yaml command to modify the .yaml file. Press I or Insert to enter the insert mode and add related parameters in the .yaml file. For details about the parameters, see Table 2 . After the modification is complete, press Esc and enter :wq! to save the modification.

    Add the following configuration items to the StorageClass.yaml file.

    • Example 1: Configure zone and region information in the StorageClass.

      kind: StorageClass
      apiVersion: storage.k8s.io/v1
      metadata:
        name: example-storageclass
      provisioner: csi.huawei.com
      parameters:
        volumeType: lun
        allocType: thin
      volumeBindingMode: WaitForFirstConsumer
      allowedTopologies:
      - matchLabelExpressions:
        - key: topology.kubernetes.io/zone
          values:
          - ChengDu
        - key: topology.kubernetes.io/region
          values:
          - China-west
      
    • Example 2: Configure protocol information in the StorageClass.

      kind: StorageClass
      apiVersion: storage.k8s.io/v1
      metadata:
        name: protocol-example-storageclass
      provisioner: csi.huawei.com
      parameters:
        volumeType: lun
        allocType: thin
      volumeBindingMode: WaitForFirstConsumer
      allowedTopologies:
      - matchLabelExpressions:
        - key: topology.kubernetes.io/protocol.iscsi
          values:
          - csi.huawei.com
      

    Table 2 StorageClass parameter description

    Parameter

    Description

    Remarks

    volumeBindingMode

    PersistentVolume binding mode, used to control the time when PersistentVolume resources are dynamically allocated and bound.

    You can set this parameter to WaitForFirstConsumer or Immediate.

    WaitForFirstConsumer: indicates that the binding and allocation of the PersistentVolume are delayed until a Pod that uses the PVC is created.

    Immediate: The PersistentVolume is bound and allocated immediately after a PVC is created.

    allowedTopologies.matchLabelExpressions

    Topology information label, which is used to filter CSI backends and Kubernetes nodes. If the matching fails, PVCs or Pods cannot be created.

    Both key and value must be configured in a fixed format.

    key: This parameter can be set to topology.kubernetes.io/zone or topology.kubernetes.io/region.

    topology.kubernetes.io/protocol.<protocol>: <protocol> indicates the protocol type and can be iscsi, fc, or nfs.

    value:

    If key is topology.kubernetes.io/zone or topology.kubernetes.io/region, value must be the same as that in Prerequisites.

    If key is topology.kubernetes.io/protocol.<protocol>, value is fixed at csi.huawei.com.

  8. Run the following command to create a StorageClass based on the .yaml file.

    kubectl create -f StorgeClass.yaml
    
  9. Use the StorageClass to create a PVC with the topology capability. For details, see Table 1 .

10 - Configuring ALUA

Asymmetric Logical Unit Access (ALUA) is a model that supports access to multiple target ports. In the multipathing state, ALUA presents active/passive volumes to the host and provides a port access status switchover interface to switch over the working controllers for volumes. For example, when a volume of a controller fails, you can set the status of ports on the controller to Unavailable. After the host multipathing software that supports ALUA detects the status, it switches subsequent I/Os from the failed controller to the peer controller.

10.1 - Configuring ALUA Parameters for a Huawei Enterprise Storage Backend

For details about how to configure ALUA for Huawei enterprise storage, see the host connectivity guide of the corresponding product.

The ALUA configuration may vary according to the OS. Visit Huawei Technical Support , enter Host Connectivity Guide in the search box, and click the search button. In the search result, select the host connectivity guide for the desired OS. Configure ALUA according to the actual situation and the description in the guide. Huawei CSI will apply the configuration items you set to the initiator of the host on Huawei storage.


A node with a Pod provisioned does not proactively change ALUA information. The host ALUA configuration changes only after a Pod is provisioned again to the node.

ALUA Parameters for OceanStor V5 Series

Table 1 lists the ALUA parameters supported by Huawei CSI for OceanStor V5 series.

Table 1 ALUA parameters supported by Huawei CSI for OceanStor V5 series

Parameter

Description

Remarks

HostName

Host name rule. This parameter is mandatory. You can use a regular expression.

The host name can be obtained by running the cat /etc/hostname command. It can be matched by using regular expressions. When HostName is set to *, the configuration takes effect on hosts with any name. For details, see Regular expression.

If the host name of a compute node matches multiple ALUA configuration options, they will be sorted based on the matching accuracy and the first ALUA configuration option will be used. For details about the sorting rules, see Rules for Matching ALUA Configuration Items with Host Names.

MULTIPATHTYPE

Multipathing type. This parameter is mandatory. The value can be:

  • 0: Third-party multipathing is not used.
  • 1: Third-party multipathing is used.

--

FAILOVERMODE

Initiator switchover mode. This parameter is conditionally mandatory. The value can be:

  • 0: early-version ALUA
  • 1: common ALUA
  • 2: ALUA not used
  • 3: special ALUA

This parameter needs to be specified only when third-party multipathing is used. Configure the initiator switchover mode by referring to the connectivity guide.

SPECIALMODETYPE

Special mode type of the initiator. This parameter is conditionally mandatory. The value can be:

  • 0: special mode 0
  • 1: special mode 1
  • 2: special mode 2
  • 3: special mode 3

This parameter needs to be specified only when the initiator switchover mode is special ALUA. Configure the special mode type of the initiator by referring to the connectivity guide.

PATHTYPE

Initiator path type. This parameter is conditionally mandatory. The value can be:

  • 0: preferred path
  • 1: non-preferred path

This parameter needs to be specified only when third-party multipathing is used. Configure the initiator path type by referring to the connectivity guide.

The following uses OceanStor 18500 V5 as an example to describe how to connect to Red Hat. For details about the host connectivity guide, see Huawei SAN Storage Host Connectivity Guide for Red Hat.

The following ALUA configuration example is recommended in the OceanStor 18500 V5 host connectivity guide for Red Hat in non-HyperMetro storage scenarios. In this example, the OS on compute node myhost01 in the Kubernetes cluster is RHEL 5.x, and that on other compute nodes is RHEL 7.x. According to the recommendation, the switchover mode of RHEL 5.x should be “ALUA not used”, and that of RHEL 7.x should be “common ALUA”.

storage: oceanstor-san
name: oceanstor-iscsi-155
urls:
  - https://192.168.129.155:8088
  - https://192.168.129.156:8088
pools:
  - StoragePool001
parameters:
  protocol: iscsi
  portals:
    - 192.168.128.120
    - 192.168.128.121
  ALUA:
    ^myhost01$:
      MULTIPATHTYPE: 1
      FAILOVERMODE: 2
      PATHTYPE: 0
    "*":
      MULTIPATHTYPE: 1
      FAILOVERMODE: 1
      PATHTYPE: 0

ALUA Parameters for OceanStor and OceanStor Dorado Series

Table 2 lists the ALUA parameters supported by Huawei CSI for OceanStor and OceanStor Dorado series.


By default, the initiator host access mode of OceanStor and OceanStor Dorado series storage is “balanced mode”. Therefore, you are advised not to configure ALUA parameters for OceanStor and OceanStor Dorado series storage.

Table 2 ALUA parameters for OceanStor and OceanStor Dorado series

Parameter

Description

Remarks

HostName

Host name rule. This parameter is mandatory. You can use a regular expression.

The host name can be obtained by running the cat /etc/hostname command. It can be matched by using regular expressions. When HostName is set to *, the configuration takes effect on hosts with any name. For details, see Regular expression.

If the host name of a compute node matches multiple ALUA configuration options, they will be sorted based on the matching accuracy and the first ALUA configuration option will be used. For details about the sorting rules, see Rules for Matching ALUA Configuration Items with Host Names.

accessMode

Host access mode. This parameter is mandatory. The value can be:

  • 0: balanced mode
  • 1: asymmetric mode

The balanced mode is recommended in non-HyperMetro scenarios. Currently, Huawei CSI does not support SAN HyperMetro scenarios. Exercise caution when using the asymmetric mode.

hyperMetroPathOptimized

Whether the path of the host on the current storage array is preferred in HyperMetro scenarios. The value can be:

  • 1: yes
  • 0: no

This parameter needs to be specified only when the host access mode is set to asymmetric.

Currently, Huawei CSI does not support SAN HyperMetro scenarios. Exercise caution when using the asymmetric mode.

The following uses OceanStor Dorado 18000 as an example to describe how to connect to Red Hat. For details about the host connectivity guide, see OceanStor Dorado and OceanStor Host Connectivity Guide for Red Hat.

The following ALUA configuration example is recommended in the OceanStor Dorado 18000 host connectivity guide for Red Hat in non-HyperMetro storage scenarios.

storage: "oceanstor-san"
name: "dorado-iscsi-155"
urls:
  - "https://192.168.129.155:8088"
  - "https://192.168.129.156:8088"
pools:
  - "StoragePool001"
parameters:
  protocol: "iscsi"
  portals:
    - "192.168.128.120"
    - "192.168.128.121"
  ALUA:
    "*":
      accessMode: 0

Rules for Matching ALUA Configuration Items with Host Names

  • If the configured host name rule exactly matches the host name of the service node, the ALUA configuration item corresponding to the host name rule is used.

    For example, the host name rule in configuration item 1 is * and that in configuration item 2 is ^myhost01$. If the host name of a compute node is myhost01, it exactly matches configuration item 2. In this case, Huawei CSI will apply the configuration information in configuration item 2 to the storage side.

  • If the configured host name rule does not exactly match the host name of the service node, the first ALUA configuration item matched by regular expressions is used.

    For example, the host name rule in configuration item 1 is myhost0[0-9] and that in configuration item 2 is myhost0[5-9]. In this case, configuration item 1 has a higher priority than configuration item 2. If the host name of a compute node is myhost06, both configuration items can be matched. In this case, Huawei CSI will apply the configuration information in configuration item 1 to the storage side.

10.2 - Configuring ALUA Parameters for a Distributed Storage Backend

For details about how to configure ALUA for Huawei distributed storage, see the host connectivity guide of the corresponding product.

The ALUA configuration may vary according to the OS. Visit Huawei Technical Support , enter Host Connectivity Guide in the search box, and click the search button. In the search result, select the host connectivity guide for the desired OS. Configure ALUA according to the actual situation and the description in the guide. Huawei CSI will apply the configuration items you set to the initiator of the host on Huawei storage.


A node with a Pod provisioned does not proactively change ALUA information. The host ALUA configuration changes only after a Pod is provisioned again to the node. In non-HyperMetro scenarios of distributed storage, you are advised to set the switchover mode to “disable ALUA” (default value). This is because the storage system is in active/active mode and “enables ALUA” is meaningless. Therefore, you are advised not to configure ALUA parameters for distributed storage.

Table 1 lists the ALUA parameters supported by Huawei CSI for distributed storage.

Table 1 ALUA parameters for distributed storage

Parameter

Description

Remarks

HostName

The value of HostName is the host name of a worker node, for example, HostName1 and HostName2.

The host name can be obtained by running the cat /etc/hostname command. It can be matched by using regular expressions. When HostName is set to *, the configuration takes effect on hosts with any name. For details, see Regular expression.

If the host name of a compute node matches multiple ALUA configuration options, they will be sorted based on the matching accuracy and the first ALUA configuration option will be used. For details about the sorting rules, see Rules for Matching ALUA Configuration Items with Host Names.

switchoverMode

Switchover mode. This parameter is mandatory. The value can be:

  • Disable_alua: disables ALUA.
  • Enable_alua: enables ALUA.

In non-HyperMetro scenario, you are advised to set the switchover mode to "disable ALUA". This is because the storage system is in active/active mode and "enables ALUA" is meaningless. Currently, Huawei CSI does not support SAN HyperMetro scenarios. Exercise caution when enabling ALUA.

pathType

Path type. This parameter is conditionally mandatory. The value can be:

  • optimal_path: preferred path
  • non_optimal_path: non-preferred path

This parameter is mandatory when the switchover mode is set to "enables ALUA".

Rules for Matching ALUA Configuration Items with Host Names

  • If the configured host name rule exactly matches the host name of the service node, the ALUA configuration item corresponding to the host name rule is used.

    For example, the host name rule in configuration item 1 is * and that in configuration item 2 is ^myhost01$. If the host name of a compute node is myhost01, it exactly matches configuration item 2. In this case, Huawei CSI will apply the configuration information in configuration item 2 to the storage side.

  • If the configured host name rule does not exactly match the host name of the service node, the first ALUA configuration item matched by regular expressions is used.

    For example, the host name rule in configuration item 1 is myhost0[0-9] and that in configuration item 2 is myhost0[5-9]. In this case, configuration item 1 has a higher priority than configuration item 2. If the host name of a compute node is myhost06, both configuration items can be matched. In this case, Huawei CSI will apply the configuration information in configuration item 1 to the storage side.

10.3 - Example ALUA Configuration Policy of OceanStor V5 Series

Example 1: The configuration file content is as follows:

parameters:
  ALUA:
    "*":
      MULTIPATHTYPE: 1
      FAILOVERMODE: 3
      SPECIALMODETYPE: 0
      PATHTYPE: 0
    node1:
      MULTIPATHTYPE: 1
      FAILOVERMODE: 3
      SPECIALMODETYPE: 0
      PATHTYPE: 1

If the host name is node1, both of the preceding ALUA configuration sections can be used to configure initiators. According to the configuration policy rules in Configuring ALUA Parameters for a Huawei Enterprise Storage Backend , the priority of the second configuration section (where HostName is node1) is higher than that of the first configuration section (where HostName is *).

Example 2: The configuration file content is as follows:

parameters:
  ALUA:
    node[0-9]:
      MULTIPATHTYPE: 1
      FAILOVERMODE: 3
      SPECIALMODETYPE: 0
      PATHTYPE: 0
    node[5-7]:
      MULTIPATHTYPE: 1
      FAILOVERMODE: 3
      SPECIALMODETYPE: 0
      PATHTYPE: 1

If the host name is node6, both of the preceding ALUA configuration sections can be used to configure initiators. According to the configuration policy rules in Configuring ALUA Parameters for a Huawei Enterprise Storage Backend , select the first ALUA configuration section to configure initiators.

Example 3: The configuration file content is as follows:

parameters:
  ALUA:
   node$:
      MULTIPATHTYPE: 1
      FAILOVERMODE: 3
      SPECIALMODETYPE: 0
      PATHTYPE: 0
   node10$:
      MULTIPATHTYPE: 1
      FAILOVERMODE: 3
      SPECIALMODETYPE: 0
      PATHTYPE: 1

According to the configuration policy rules in Configuring ALUA Parameters for a Huawei Enterprise Storage Backend : For host node1, select the first ALUA configuration section to configure initiators. For host node10, select the second ALUA configuration section to configure initiators. ^ matches the beginning of a character string, and $ matches the end of a character string.

10.4 - Example ALUA Configuration Policy of OceanStor Dorado

Example 1: The configuration file content is as follows:

parameters:
  ALUA:
    "*":
     accessMode: 1
     hyperMetroPathOptimized: 1
   node1:
      accessMode: 1
      hyperMetroPathOptimized: 0

If the host name is node1, both of the preceding ALUA configuration sections can be used to configure initiators. According to the configuration policy rules in Configuring ALUA Parameters for a Huawei Enterprise Storage Backend , the priority of the second configuration section (where HostName is node1) is higher than that of the first configuration section (where HostName is *).

Example 2: The configuration file content is as follows:

parameters:
  ALUA:
   node[0-9]:
     accessMode: 1
     hyperMetroPathOptimized: 1
   node[5-7]:
     accessMode: 1
     hyperMetroPathOptimized: 0

If the host name is node6, both of the preceding ALUA configuration sections can be used to configure initiators. According to the configuration policy rules in Configuring ALUA Parameters for a Huawei Enterprise Storage Backend , select the first ALUA configuration section to configure initiators.

Example 3: The configuration file content is as follows:

parameters:
  node1$:
    node[0-9]:
    accessMode: 1
    hyperMetroPathOptimized: 1
  node10$:
    accessMode: 1
    hyperMetroPathOptimized: 0

According to the configuration policy rules in Configuring ALUA Parameters for a Huawei Enterprise Storage Backend : For host node1, select the first ALUA configuration section to configure initiators. For host node10, select the second ALUA configuration section to configure initiators. ^ matches the beginning of a character string, and $ matches the end of a character string.

10.5 - Example ALUA Configuration Policy of Distributed Storage

Example 1: The configuration file content is as follows:

parameters:
  ALUA:
    "*":
      switchoverMode: Enable_alua
      pathType: optimal_path
    node1:
      switchoverMode: Enable_alua
      pathType: non_optimal_path

If the host name is node1, both of the preceding ALUA configuration sections can be used to configure initiators. According to the configuration policy rules in Configuring ALUA Parameters for a Distributed Storage Backend , the priority of the second configuration section (where HostName is node1) is higher than that of the first configuration section (where HostName is *).

Example 2: The configuration file content is as follows:

parameters:
  ALUA:
    node[0-9]:
      switchoverMode: Enable_alua
      pathType: optimal_path
    node[5-7]:
      switchoverMode: Enable_alua
      pathType: non_optimal_path

If the host name is node6, both of the preceding ALUA configuration sections can be used to configure initiators. According to the configuration policy rules in Configuring ALUA Parameters for a Distributed Storage Backend , select the first ALUA configuration section to configure initiators.

Example 3: The configuration file content is as follows:

parameters:
  ALUA:
    node1$:
      switchoverMode: Enable_alua
      pathType: optimal_path
    node10$:
      switchoverMode: Enable_alua
      pathType: non_optimal_path

According to the configuration policy rules in Configuring ALUA Parameters for a Distributed Storage Backend : For host node1, select the first ALUA configuration section to configure initiators. For host node10, select the second ALUA configuration section to configure initiators. ^ matches the beginning of a character string, and $ matches the end of a character string.