This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Configuring Storage Topology Awareness

In the Kubernetes cluster, resources can be scheduled and provisioned based on the topology labels of nodes and the topology capabilities supported by storage backends.

Prerequisites

You need to configure topology labels on worker nodes in the cluster. The method is as follows:

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Run the following command to view information about worker nodes in the current cluster.

    kubectl get node
    

    The following is an example of the command output.

    NAME     STATUS   ROLES                      AGE   VERSION
    node01   Ready    controlplane,etcd,worker   42d   v1.22.3
    node02   Ready    worker                     42d   v1.22.3
    node03   Ready    worker                     42d   v1.22.3
    
  3. Run the following command to configure a topology label for a worker node. In the preceding command, nodename indicates the name of a worker node. For details about the key and value parameters, see Table 1.

    kubectl label node <nodename> <key>=<value>
    

    Table 1 Parameter description

    Parameter

    Description

    Remarks

    <key>

    Unique identifier of a topology label.

    The value can be zone, region, or protocol.<protocol>.

    <protocol> can be set to iscsi, nfs, fc, or roce.

    <value>

    Value of a topology label.

    If key is set to zone or region, value is a user-defined parameter.

    If key is set to protocol.<protocol>, value is fixed at csi.huawei.com.

    • A topology label must start with topology.kubernetes.io. Topology label examples:
    • Example 1: topology.kubernetes.io/region=China-west
    • Example 2: topology.kubernetes.io/zone=ChengDu
    • Example 3: topology.kubernetes.io/protocol.iscsi=csi.huawei.com
    • Example 4: topology.kubernetes.io/protocol.fc=csi.huawei.com
    • A key in a topology label on a node can have only one value.
    • If multiple protocols are configured in a topology label on a node, when you configure a StorageClass, the StorageClass needs to meet only one of the protocols.
    • If both the region and the zone are configured in a topology label on a node, when you configure a StorageClass, the StorageClass must meet all filter criteria.
  4. Run the following command to view the label information about all worker nodes in the current cluster.

    kubectl get nodes -o=jsonpath='{range .items[*]}[{.metadata.name}, {.metadata.labels}]{"\n"}{end}' | grep --color "topology.kubernetes.io"
    

    The following is an example of the command output.

    [node01,"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"node01","kubernetes.io/os":"linux","node-role.kubernetes.io/controlplane":"true","node-role.kubernetes.io/etcd":"true","node-role.kubernetes.io/worker":"true","topology.kubernetes.io/zone":"ChengDu"}]
    

1 - Configuring Storage Topology Awareness Using Helm

Procedure

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Go to the directory where the Helm project is located. If the previous Helm project cannot be found, copy the helm directory in the component package to any directory on the master node. For details about the component package path, see Table 1.

  3. Go to the backend service configuration directory /examples/backend/ and back up the backend.yaml file.

    cp backend.yaml backend.yaml.bak
    
  4. Run the **vi **backend.yaml command to open the file and configure topology awareness as required. The following is an example. After the modification is complete, press Esc and enter :wq! to save the modification.

    storage: "oceanstor-san"
    name: "dorado-iscsi-155"
    namespace: "huawei-csi"
    urls:
      - "https://192.168.129.155:8088"
    pools:
      - "StoragePool001"
    parameters:
      protocol: "iscsi"
      portals:
        - "10.10.30.20"
        - "10.10.30.21"
    supportedTopologies:
      - { "topology.kubernetes.io/region": "China-west", "topology.kubernetes.io/zone": "ChengDu" }
      - { "topology.kubernetes.io/region": "China-south","topology.kubernetes.io/zone": "ShenZhen" }
    maxClientThreads: "30"
    
  5. Run the following command to delete the storage backend to be modified. In the command, dorado-iscsi-155 indicates the storage backend name.

    oceanctl delete backend dorado-iscsi-155 -n huawei-csi
    
  6. Run the following command to create a storage backend.

    oceanctl create backend -f ../examples/backend/backend.yaml -i yaml
    

    Enter the storage user name and password as prompted.

    Please enter this backend user name:admin
    Please enter this backend password:
    
  7. Run the vi StorageClass.yaml command to modify the .yaml file. Press I or Insert to enter the insert mode and add related parameters in the .yaml file. For details about the parameters, see Table 1. After the modification is complete, press Esc and enter :wq! to save the modification.

    Add the following configuration items to the StorageClass.yaml file.

    • Example 1: Configure zone and region information in the StorageClass.

      kind: StorageClass
      apiVersion: storage.k8s.io/v1
      metadata:
        name: example-storageclass
      provisioner: csi.huawei.com
      parameters:
        volumeType: lun
        allocType: thin
      volumeBindingMode: WaitForFirstConsumer
      allowedTopologies:
      - matchLabelExpressions:
        - key: topology.kubernetes.io/zone
          values:
          - ChengDu
        - key: topology.kubernetes.io/region
          values:
          - China-west
      
    • Example 2: Configure protocol information in the StorageClass.

      kind: StorageClass
      apiVersion: storage.k8s.io/v1
      metadata:
        name: protocol-example-storageclass
      provisioner: csi.huawei.com
      parameters:
        volumeType: lun
        allocType: thin
      volumeBindingMode: WaitForFirstConsumer
      allowedTopologies:
      - matchLabelExpressions:
        - key: topology.kubernetes.io/protocol.iscsi
          values:
          - csi.huawei.com
      

    Table 1 Parameter description

    Parameter

    Description

    Remarks

    volumeBindingMode

    PersistentVolume binding mode, used to control the time when PersistentVolume resources are dynamically allocated and bound.

    You can set this parameter to WaitForFirstConsumer or Immediate.

    WaitForFirstConsumer: indicates that the binding and allocation of the PersistentVolume are delayed until a Pod that uses the PVC is created.

    Immediate: The PersistentVolume is bound and allocated immediately after a PVC is created.

    allowedTopologies.matchLabelExpressions

    Topology information label, which is used to filter CSI backends and Kubernetes nodes. If the matching fails, PVCs or Pods cannot be created.

    Both key and value must be configured in a fixed format.

    key: This parameter can be set to topology.kubernetes.io/zone or topology.kubernetes.io/region.

    topology.kubernetes.io/protocol.<protocol>: <protocol> indicates the protocol type and can be iscsi, fc, or nfs.

    value:

    If key is topology.kubernetes.io/zone or topology.kubernetes.io/region, value must be the same as the topology label set in the prerequisites.

    If key is topology.kubernetes.io/protocol.<protocol>, value is fixed at csi.huawei.com.

  8. Run the following command to create a StorageClass based on the .yaml file.

    kubectl create -f StorgeClass.yaml
    
  9. Use the StorageClass to create a PVC with the topology capability. For details, see PVC Parameters for Dynamic Volume Provisioning.