This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Persistent Volume Management

Based on service requirements, files in containers need to be persistently stored on disks. When the containers are re-built or re-allocated to new nodes, the persistent data can still be used.

To persistently store data on storage devices, you need to use the PersistentVolume (PV) and PersistentVolumeClaim (PVC) when provisioning containers.

  • PV: a piece of storage in the Kubernetes cluster that has been provisioned by an administrator or dynamically provisioned using a StorageClass .
  • PVC: a request for storage by a user. A PVC consumes PV resources. A PVC can request specific size and access modes. For example, a PV can be mounted in ReadWriteOnce, ReadOnlyMany, or ReadWriteMany mode. For details, see Access Modes .

This section describes how to use Huawei CSI to create, clone, and expand the capacity of a PV and PVC.

1 - Configuring PVs

Huawei CSI allows storage resources (LUNs or file systems) to be created on Huawei storage and provided for containers based on user settings. For details about the supported features, see Compatibility and Features of the storage device.

PV configurations can be classified into configuring dynamic PVs or static PVs, and managing PVs.

  • Configuring dynamic PV does not require a PV to be created in advance. Huawei CSI automatically creates resources required by a PV on storage devices based on a StorageClass. In addition, you can create a PV when creating a PVC.
  • Configuring a static PV requires the administrator to create required resources on a storage device in advance and use existing resources by creating a PV. In addition, you can specify the associated PV when creating a PVC.
  • Managing PVs does not require a PV to be created in advance. You can specify the StorageClass and resource information on the storage device in a PVC. Create the PVC and PV at the same time, and manage all existing storage resources in the cluster.

1.1 - Configuring Dynamic PVs

Dynamic volume provisioning allows storage volumes to be created on demand. Dynamic volume provisioning depends on the StorageClass objects. The cluster administrator can define multiple StorageClass objects as required and specify a StorageClass that meets service requirements when declaring a PV or PVC. When applying for resources from Huawei storage devices, Huawei CSI creates storage resources that meet service requirements based on the preset StorageClass.

Configuration Description

Perform the following steps to configure and use dynamic PVs:

Preparation

Before configuring dynamic PVs, configure StorageClass by referring to Configuring a StorageClass .

Configuring a PVC

  1. Prepare the PVC configuration file mypv.yaml. The following is an example. For details about other parameters, see Table 1 .

    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: mypvc
    spec:
      accessModes:
        - ReadWriteOnce
      volumeMode: Filesystem
      storageClassName: mysc
      resources:
        requests:
          storage: 100Gi
    
  2. Run the following command to create a PVC using the configuration file.

    kubectl create -f mypvc.yaml
    
  3. After a period of time, run the following command to view the information about the created PVC.

    kubectl get pvc mypvc
    

    The following is an example of the command output. If the PVC status is Bound, the PVC has been created and can be used by a Pod.

    NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    mypvc       Bound    pvc-840054d3-1d5b-4153-b73f-826f980abf9e   100Gi      RWO            mysc           12s
    

Table 1 PVC parameters for configuring a dynamic PV

Parameter

Description

Mandatory

Default Value

Remarks

metadata.name

User-defined name of a PVC object.

Yes

-

Take Kubernetes v1.22.1 as an example. The value can contain digits, lowercase letters, hyphens (-), and periods (.), and must start and end with a letter or digit.

spec.volumeMode

Volume mode. This parameter is optional. When LUN volumes are used, the following types are supported:

  • Filesystem: local file system.
  • Block: raw device.

No

Filesystem

This parameter takes effect when a PV is mounted. The default value is Filesystem.

  • Filesystem indicates that a container accesses a PV using a local file system. The local file system type is specified by the fsType field in the specified StorageClass. Storage of the Dtree type also uses this parameter.
  • Block indicates that a PV is accessed in raw volume mode.

spec.storageClassName

Name of the StorageClass object.

Yes

-

Name of the StorageClass object required by services.

spec.resources.requests.storage

Size of the volume to be created. The format is ***Gi and the unit is GiB.

Yes

10Gi

The PVC capacity depends on storage specifications and host specifications. For example, OceanStor Dorado 6.1.2 or OceanStor Pacific series 8.1.0 is connected to CentOS 7. If ext4 file systems are used, see Table 2. If XFS file systems are used, see Table 3. If NFS or raw devices are used, the capacity must meet the specifications of the used Huawei storage device model and version.

If the PVC capacity does not meet the specifications, a PVC or Pod may fail to be created due to the limitations of storage specifications or host file system specifications.

spec.accessModes

Access mode of the volume.

  • RWO (ReadWriteOnce): A volume can be mounted to a node in read/write mode. This mode also allows multiple Pods running on the same node to access the volume.
  • ROX (ReadOnlyMany): A volume can be mounted to multiple nodes in read-only mode.
  • RWX (ReadWriteMany): A volume can be mounted to multiple nodes in read/write mode.
  • RWOP (ReadWriteOncePod): A volume can only be mounted to a single Pod in read/write mode. Kubernetes 1.22 and later versions support this feature.

Yes

ReadWriteOnce

  • RWO/ROX/RWOP: supported by all types of volumes. RWOP is supported only by Kubernetes 1.22 and later versions. For versions earlier than Kubernetes 1.29, you need to enable this feature by following the instructions in Enabling the ReadWriteOncePod Feature Gate.
  • The support for RWX is as follows:
    • NAS storage: supported by all volumes
    • SAN storage: supported only by volumes whose volumeMode is set to Block

Table 2 ext4 capacity specifications

Storage Type

Storage Specifications

ext4 Specifications

CSI Specifications

OceanStor Dorado

512 Ki to 256 Ti

50 Ti

512 Ki to 50 Ti

OceanStor Pacific series

64 Mi to 512 Ti

50 Ti

64 Mi to 50 Ti

OceanDisk

512 Ki to 256 Ti

50 Ti

512 Ki to 50 Ti

Table 3 XFS capacity specifications

Storage Type

Storage Specifications

XFS Specifications

CSI Specifications

OceanStor Dorado

512 Ki to 256 Ti

500 Ti

512 Ki to 256 Ti

OceanStor Pacific series

64 Mi to 512 Ti

500 Ti

64 Mi to 500 Ti

OceanDisk

512 Ki to 256 Ti

500 Ti

512 Ki to 256 Ti

Using a PVC

After a PVC is created, you can use the PVC to create a Pod. The following is a simple example of using a PVC. In this example, the created Pod uses the newly created mypvc.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers: 
      - image: nginx:alpine
        name: container-0 
        volumeMounts: 
        - mountPath: /tmp
          name: pvc-mypvc 
      restartPolicy: Always 
      volumes: 
      - name: pvc-mypvc 
        persistentVolumeClaim: 
          claimName:  mypvc  # name of PVC


If Pods are batch created using PVCs, the Pods are in the ContainerCreating status for a long time, and the huawei-csi-node service is in the OOMKilled status, the memory of the huawei-csi-node service is insufficient. Increase the memory limit of huawei-csi-node by referring to Table 1 .

1.2 - Configuring a Static PV

Static volume provisioning allows administrators to use a resource created on the storage side as a PV for containers in the cluster.

Configuration Description

Perform the following steps to configure a static PV:

Preparation

A storage resource, such as a LUN or file system, required by the PV to be created exists on the storage device. If the storage resource is a file system, you also need to create the share and client information of the file system.

Configuring a PV

  1. Prepare the PV configuration file mypv.yaml. The following is an example. For details about other parameters, see Table 1 .

    kind: PersistentVolume
    apiVersion: v1
    metadata:
      name: mypv
    spec:
      volumeMode: Filesystem
      storageClassName: "" # The value must be to "".
      accessModes:
        - ReadWriteOnce
      csi:
        driver: csi.huawei.com # Enter the CSI driver name.
        volumeHandle: iscsi-dorado-181.lun0001 # Enter the volume name.
        fsType: xfs # Set the file system type.
      capacity:
        storage: 100Gi
    


    In the configuration file for static volume provisioning, storageClassName must be set to "". Otherwise, Kubernetes will use the default StorageClass.

  2. Run the following command to create a PV based on the prepared .yaml file.

    kubectl create -f mypv.yaml
    
  3. After a period of time, run the following command to view the information about the created PV.

    kubectl get pv
    

    The following is an example of the command output. If the PV status is Available, the PV is successfully created.

    NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM               STORAGECLASS   REASON   AGE
    mypv       100Gi      RWO            Retain           Available                                               4s
    

Table 1 Parameters for configuring a static PV

Parameter

Description

Mandatory

Default Value

Remarks

metadata.name

User-defined name of a PV object.

Yes

-

Take Kubernetes v1.22.1 as an example. The value can contain digits, lowercase letters, hyphens (-), and periods (.), and must start and end with a letter or digit.

spec.volumeMode

Volume mode. This parameter is optional. When LUN volumes are used, the following types are supported:

  • Filesystem: local file system.
  • Block: raw device.

No

Filesystem

This parameter takes effect when a PV is mounted. The default value is Filesystem.

  • Filesystem indicates that a container accesses a PV using a local file system. The local file system type is specified by the fsType field in the specified StorageClass.
  • Block indicates that a PV is accessed in raw volume mode.

spec.storageClassName

Name of the StorageClass object. This parameter is mandatory.

Yes

-

Set the parameter to an empty string, that is, enter "".

spec.accessModes

Access mode of the volume.

  • RWO (ReadWriteOnce): A volume can be mounted to a node in read/write mode. This mode also allows multiple Pods running on the same node to access the volume.
  • ROX (ReadOnlyMany): A volume can be mounted to multiple nodes in read-only mode.
  • RWX (ReadWriteMany): A volume can be mounted to multiple nodes in read/write mode.
  • RWOP (ReadWriteOncePod): A volume can only be mounted to a single Pod in read/write mode. Kubernetes 1.22 and later versions support this feature.

Yes

ReadWriteOnce

  • RWO/ROX/RWOP: supported by all types of volumes. RWOP is supported only by Kubernetes 1.22 and later versions. Check whether this feature is enabled for your Kubernetes cluster by referring to Enabling the ReadWriteOncePod Feature Gate.
  • The support for RWX is as follows:
    • NAS storage: supported by all volumes
    • SAN storage: supported only by volumes whose volumeMode is set to Block

spec.csi.driver

CSI driver name.

Yes

csi.huawei.com

Set this parameter to the driver name set during Huawei CSI installation.

spec.csi.volumeHandle

Unique identifier of a storage resource. This parameter is mandatory.

Format: <backendName>.<volume-name>

Yes

-

The value of this parameter consists of the following parts:

  • <backendName>: indicates the name of the backend where the volume resides. You can run the following command to obtain the configured backend information.

    oceanctl get backend

  • <volume-name>: indicates the name of a resource (LUN/file system) on the storage. You can obtain the value from DeviceManager.

spec.csi.fsType

Type of a host file system. This parameter is optional. The supported types are:

  • ext2
  • ext3
  • ext4
  • xfs

No

-

If this parameter is not set, the default value ext4 is used. This parameter is available only when volumeMode is set to Filesystem.

spec.csi.volumeAttributes.dTreeParentName

Name of the parent file system when the volume resource type is dtree.

Conditionally mandatory

-

This parameter is mandatory when the managed object is a dtree resource and the parentname parameter is not configured in the storage backend.

If dTreeParentName is configured only in the PV but parentname is not configured in the corresponding storage backend, set CSIDriverObject.attachRequired to true during CSI installation according to Table 5.

spec.capacity.storage

Volume size.

Yes

100Gi

Ensure that the size is the same as that of the corresponding resource on the storage. Kubernetes will not invoke CSI to check whether the value of this parameter is correct. Therefore, the PV can be successfully created even if its capacity is inconsistent with that of the corresponding resource on the storage.

spec.mountOptions.nfsvers

NFS mount option on the host. The following mount option is supported:

nfsvers: protocol version for NFS mounting. The value can be 3, 4, 4.0, 4.1, or 4.2.

No

-

This parameter is optional after the -o parameter when the mount command is executed on the host. The value is in list format.

If the NFS version is specified for mounting, NFS 3, 4.0, 4.1, and 4.2 protocols are supported (the protocol must be supported and enabled on storage devices). If nfsvers is set to 4, the latest protocol version NFS 4 may be used for mounting due to different OS configurations, for example, 4.2. If the 4.0 protocol is required, you are advised to set nfsvers to 4.0.

spec.mountOptions.acl

The DPC namespace supports the ACL function. The DPC client supports POSIX ACL, NFSv4 ACL, and NT ACL authentication.

No

-

The descriptions of acl, aclonlyposix, cnflush, and cflush are for reference only. For details about the parameters, see OceanStor Pacific Series Product Documentation and choose Configuration > Basic Service Configuration Guide for File > Configuring Basic Services (DPC Scenario) > Accessing a DPC Share on a Client > Step 2.

spec.mountOptions.aclonlyposix

The DPC namespace supports POSIX ACL, and the DPC client supports POSIX ACL authentication.

The following protocols support POSIX ACL: DPC, NFSv3, and HDFS. If NFSv4 ACL or NT ACL is used, the DPC client cannot identify the ACL of this type. As a result, the ACL of this type does not take effect.

No

-

If aclonlyposix and acl are used together, only acl takes effect. That is, the namespace supports the ACL function.

spec.mountOptions.cnflush

Asynchronous disk flushing mode. That is, data is not flushed to disks immediately when files in the namespace are closed.

No

-

Asynchronous flushing mode: When a file is closed, data in the cache is not flushed to storage media in synchronous mode. Instead, data is written from the cache to the storage media in asynchronous flushing mode. After the write service is complete, data is flushed from the cache to disks periodically based on the flushing period. In a multi-client scenario, if concurrent operations are performed on the same file, the file size update is affected by the disk flushing period. That is, the file size is updated only after the disk flushing is complete. Generally, the update is completed within several seconds. Synchronous I/Os are not affected by the disk flushing period.

spec.mountOptions.cflush

Synchronous disk flushing mode. That is, data is flushed to disks immediately when files in the namespace are closed.

No

-

By default, the synchronous disk flushing mode is used.

Configuring a PVC

After a PV is created in static volume provisioning mode, you can create a PVC based on the PV for containers.

  1. Prepare the PVC configuration file mypv.yaml. The following is an example. For details about other parameters, see Table 2 .

    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: mypvc
    spec:
      storageClassName: ""
      accessModes:
        - ReadWriteOnce
      volumeMode: Filesystem
      resources:
        requests:
          storage: 100Gi
      volumeName: mypv # Enter the name of the corresponding PV.
    
  2. Run the following command to create a PVC based on the configured .yaml file.

    kubectl create -f mypvc.yaml
    
  3. After a period of time, run the following command to view the information about the created PVC.

    kubectl get pvc
    

    The following is an example of the command output. If the PVC status is Bound, the PVC is successfully created.

    NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    mypvc       Bound    pvc-840054d3-1d5b-4153-b73f-826f980abf9e   100Gi      RWO                           12s
    

Table 2 PVC parameters for configuring a static PV

Parameter

Description

Mandatory

Default Value

Remarks

metadata.name

User-defined name of a PVC object.

Yes

-

Take Kubernetes v1.22.1 as an example. The value can contain digits, lowercase letters, hyphens (-), and periods (.), and must start and end with a letter or digit.

spec.accessModes

Access mode of the volume.

  • RWO (ReadWriteOnce): A volume can be mounted to a node in read/write mode. This mode also allows multiple Pods running on the same node to access the volume.
  • ROX (ReadOnlyMany): A volume can be mounted to multiple nodes in read-only mode.
  • RWX (ReadWriteMany): A volume can be mounted to multiple nodes in read/write mode.
  • RWOP (ReadWriteOncePod): A volume can only be mounted to a single Pod in read/write mode. Kubernetes 1.22 and later versions support this feature.

Yes

ReadWriteOnce

  • RWO/ROX/RWOP: supported by all types of volumes. RWOP is supported only by Kubernetes 1.22 and later versions. For versions earlier than Kubernetes 1.29, you need to enable this feature by following the instructions in Enabling the ReadWriteOncePod Feature Gate.
  • The support for RWX is as follows:
    • NAS storage: supported by all volumes
    • SAN storage: supported only by volumes whose volumeMode is set to Block

spec.volumeMode

Volume mode.

No

Filesystem

This parameter is optional. The value can be Filesystem or Block. The default value is Filesystem. This parameter takes effect when a Pod is created. Filesystem indicates that a file system is created on a PVC to access the storage. Block indicates that a raw volume is used to access the storage.

spec.resources.requests.storage

Size of the volume to be created.

Yes

-

Size of the volume to be created. The format is ***Gi and the unit is GiB.

The PVC capacity depends on storage specifications and host specifications. For example, OceanStor Dorado 6.1.2 or OceanStor Pacific series 8.1.0 is connected to CentOS 7. If ext4 file systems are used, see Table 2. If XFS file systems are used, see Table 3. If NFS or raw devices are used, the capacity must meet the specifications of the used Huawei storage device model and version.

If the PVC capacity does not meet the specifications, a PVC or Pod may fail to be created due to the limitations of storage specifications or host file system specifications.

When a PVC is created using a static PV and the PVC capacity is smaller than the capacity of the bound PV, the PVC capacity is set to the capacity of the bound PV. If the PVC capacity is greater than the capacity of the bound PV, the PVC cannot be created.

spec.volumeName

Name of the PV object.

Yes

-

This parameter is mandatory when a PVC is created statically.

spec.storageClassName

Name of the StorageClass object.

Yes

-

When a PVC is created, an empty character string is transferred. If this parameter is not set, the default StorageClass object name will be used.

Using a PVC

The use method is the same as that for dynamic volume provisioning in Using a PVC .

1.3 - Managing PVs

Manage Volume Provisioning allows administrators to use resources created on storage as PVs and supports features of dynamic volumes, such as capacity expansion, snapshot, and clone. This is a custom capability of Huawei CSI. This feature applies to the following scenarios:

  • In the reconstruction containerized applications, existing storage volumes need to be used.
  • The Kubernetes cluster is rebuilt.
  • Storage data is migrated in disaster recovery (DR) scenarios.


Manage Volume Provisioning allows existing storage resources to be managed by Kubernetes. You are not allowed to manage a storage resource for multiple times and concurrently delete or create a storage resource. When a storage resource is managed by multiple clusters, operations on the managed volume in a single cluster take effect only in the cluster and will not be synchronized to other clusters. Instead, you need to perform these operations on the managed volume in other clusters. For example, when you expand the capacity of a PVC in a cluster, the capacity of the corresponding PVC in other clusters will not be automatically expanded. In this case, you need to manually expand the capacity in other clusters by running the expansion commands in Expanding the Capacity of a PV .

Configuration Description

Perform the following steps to manage and use PVs:

Preparation

  • You have registered the storage where the volume to be managed resides with CSI.
  • You have logged in to the storage device to obtain the name and capacity of the volume to be managed.
  • The StorageClass has been configured. For details, see Configuring a StorageClass (pay attention to the Whether the Volume Management Takes Effect field in the table).

Configuring a PVC

  1. Prepare the PVC configuration file mypv.yaml. The following is an example. For details about other parameters, see Table 1 .

    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: mypvc
      annotations:
        csi.huawei.com/manageVolumeName: "*"  # Enter the storage resource name.
        csi.huawei.com/manageBackendName: "*" # Enter the storage backend name.
      labels:
        provisioner: csi.huawei.com
    spec:
      accessModes:
        - ReadWriteOnce
      volumeMode: Filesystem
      storageClassName: mysc
      resources:
        requests:
          storage: 100Gi
    
  2. Run the following command to create a PVC using the configuration file.

    kubectl create -f mypvc.yaml
    
  3. After a period of time, run the following command to view the information about the created PVC.

    kubectl get pvc mypvc
    

    The following is an example of the command output. If the PVC status is Bound, the PVC has been created and can be used by a Pod.

    NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    mypvc       Bound    pvc-840054d3-1d5b-4153-b73f-826f980abf9e   100Gi      RWO            mysc           12s
    

Table 1 PVC parameters for managing PVs

Parameter

Description

Mandatory

Default Value

Remarks

metadata.annotations

PVC object annotations. Set the following parameters:

  • Driver name/manageVolumeName: volume name on the storage.
  • Driver name/manageBackendName: name of the backend to which the volume belongs.

Yes

csi.huawei.com/manageVolumeName: * csi.huawei.com/manageBackendName: *

  • For details about how to obtain Driver name, see Table 4.
  • Driver name/manageVolumeName: name of an existing volume on the storage. Only English characters are supported.
  • Driver name/manageBackendName: name of the storage backend in CSI.

You can run the oceanctl get backend -n huawei-csi command to obtain the backend name.

metadata.labels

PVC object labels.

No

-

Format: provisioner: Driver name specified during installation

Example: provisioner: csi.huawei.com

This parameter takes effect when a PVC is created. It is used to listen to PVC resources and obtain information about metadata.annotations.

metadata.name

User-defined name of a PVC object.

Yes

-

Take Kubernetes v1.22.1 as an example. The value can contain digits, lowercase letters, hyphens (-), and periods (.), and must start and end with a letter or digit.

spec.volumeMode

Volume mode. This parameter is optional. When LUN volumes are used, the following types are supported:

  • Filesystem: local file system.
  • Block: raw device.
NOTE:

This parameter takes effect when a PV is mounted. The use method of this parameter must be the same as that of the managed volume.

  • If a volume is used as a raw volume before being managed, volumeMode must be set to Block.
  • If a volume is used in ext2, ext3, or ext4 mode before being managed, volumeMode must be set to Filesystem and fsType in the StorageClass must be set to ext2, ext3, or ext4.
  • If a volume is used in XFS mode before being managed, volumeMode must be set to Filesystem and fsType in the StorageClass must be set to xfs.

No

Filesystem

This parameter takes effect when a PV is mounted.

  • Filesystem indicates that a container accesses a PV using a local file system. The local file system type is specified by the fsType field in the specified StorageClass.
  • Block indicates that a PV is accessed in raw volume mode.

spec.storageClassName

Name of the StorageClass object.

Yes

-

The configuration of the StorageClass must be the same as that of the managed volume.

spec.resources.requests.storage

Size of the volume to be created. The format is ***Gi and the unit is GiB.

Yes

-

The PVC capacity depends on storage specifications and host specifications. For example, OceanStor Dorado 6.1.2 or OceanStor Pacific series 8.1.0 is connected to CentOS 7. If ext4 file systems are used, see Table 2. If XFS file systems are used, see Table 3. If NFS or raw devices are used, the capacity must meet the specifications of the used Huawei storage device model and version.

If the PVC capacity does not meet the specifications, a PVC or Pod may fail to be created due to the limitations of storage specifications or host file system specifications.

spec.accessModes

Access mode of the volume.

  • RWO (ReadWriteOnce): A volume can be mounted to a node in read/write mode. This mode also allows multiple Pods running on the same node to access the volume.
  • ROX (ReadOnlyMany): A volume can be mounted to multiple nodes in read-only mode.
  • RWX (ReadWriteMany): A volume can be mounted to multiple nodes in read/write mode.
  • RWOP (ReadWriteOncePod): A volume can only be mounted to a single Pod in read/write mode. Kubernetes 1.22 and later versions support this feature.

Yes

ReadWriteOnce

  • RWO/ROX/RWOP: supported by all types of volumes. RWOP is supported only by Kubernetes 1.22 and later versions. For versions earlier than Kubernetes 1.29, you need to enable this feature by following the instructions in Enabling the ReadWriteOncePod Feature Gate.
  • The support for RWX is as follows:
    • NAS storage: supported by all volumes
    • SAN storage: supported only by volumes whose volumeMode is set to Block

Using a PVC

The procedure is the same as that for Using a PVC for dynamic volume provisioning.

2 - Managing PVs

2.1 - Expanding the Capacity of a PV


For OceanStor V700R001C10 as well as OceanStor Dorado V700R001C10 and later versions, the minimum capacity of a file system after capacity expansion is limited. For details, see the product documentation of the corresponding storage device.

When the capacity of a PVC used by a container is insufficient, you need to expand the capacity of the PVC.

Prerequisites

  • A PVC has been created, the backend to which it resides exists and supports capacity expansion.

  • Compatibility and Features lists the storage that supports capacity expansion, and Kubernetes Feature Matrix lists the Kubernetes versions that support capacity expansion.

  • The csi-resizer service is enabled for huawei-csi-controller.

    kubectl describe deploy huawei-csi-controller -n huawei-csi | grep csi-resizer
    

    If the following information is displayed, the csi-resizer service is enabled.

       csi-resizer:
        Image:      k8s.gcr.io/sig-storage/csi-resizer:v1.9.0
    

Procedure

  1. Run the following command to check whether the StorageClass supports capacity expansion. In the preceding command, mysc indicates the name of the StorageClass to be queried.

    kubectl get sc mysc
    

    The following is an example of the command output.

    NAME              PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
    mysc              csi.huawei.com   Delete          Immediate           true                  172m
    

    If the value of ALLOWVOLUMEEXPANSION is true, the current StorageClass supports capacity expansion. In this case, go to 3 .

  2. Run the following command to change the value of allowVolumeExpansion to true. In the preceding command, mysc indicates the name of the StorageClass to be modified.

    kubectl patch sc mysc --patch '{"allowVolumeExpansion":true}'
    
  3. Run the following command to query the StorageClass name of the PVC. In the preceding command, mypvc indicates the name of the PVC to be expanded.

    kubectl get pvc mypvc
    

    The following is an example of the command output.

    NAME               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
    mypvc              Bound    pvc-3383be36-537c-4cb1-8f32-a415fa6ba384   2Gi        RW0            mysc              145m
    
  4. Run the following command to expand the capacity.

    kubectl patch pvc mypvc -p '{"spec":{"resources":{"requests":{"storage":"120Gi"}}}}'
    

    In the preceding command, mypvc indicates the name of the PVC to be expanded, and 120Gi indicates the capacity after expansion. Change the values based on the site requirements.

    • The PVC capacity depends on storage specifications and host specifications. For example, OceanStor Dorado 6.1.2 or OceanStor Pacific series 8.1.0 is connected to CentOS 7. If ext4 file systems are used, see Table 2 . If XFS file systems are used, see Table 3 . If NFS or raw devices are used, the capacity must meet the specifications of the used Huawei storage device model and version.
    • If the PVC capacity does not meet the specifications, a PVC or Pod may fail to be created due to the limitations of storage specifications or host file system specifications.
    • If the capacity expansion fails because the target capacity exceeds the storage pool capacity, see Failed to Expand the PVC Capacity Because the Target Capacity Exceeds the Storage Pool Capacity .
  5. Run the following command to check whether the capacity modification takes effect.

    kubectl get pvc
    

    The following is an example of the command output. If the value of CAPACITY is changed to the specified capacity, the capacity expansion is successful.

    NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    mypvc       Bound    pvc-3383be36-537c-4cb1-8f32-a415fa6ba384   120Gi       RWO            mysc           24s
    

2.2 - Cloning a PV

This section describes how to clone a PVC.

When cloning a PVC, you need to specify the data source. The following is an example of cloning a PVC. In this example, mypvc is used as the data source and a PVC named myclone is created.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myclone
spec:
  storageClassName: mysc
  dataSource:
    name: mypvc
    kind: PersistentVolumeClaim
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

  • The specified storageClassName must be the same as the StorageClass of the source volume in dataSource.
  • The capacity of the clone volume must be greater than or equal to that of the source volume. Equal capacity is recommended.

Prerequisites

The source PVC already exists in the system, and the backend where the source PVC resides supports cloning. Compatibility and Features lists the storage that supports cloning, and Kubernetes Feature Matrix lists the Kubernetes versions that support cloning.

Procedure

  1. Run the following command to create a PVC based on the configuration file of the clone volume.

    kubectl create -f myclone.yaml
    

2.3 - Changing a PV

2.3.1 - Enabling the PVC Change Feature

The PVC change feature is disabled by default during Huawei CSI installation. To use this feature, perform the following steps.

2.3.1.1 - Enabling the PVC Change Feature Using Helm

Prerequisites

You have installed Huawei CSI using Helm.

Procedure

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Run the following command to check whether the PVC change feature is enabled.

    helm-huawei-csi indicates the Helm chart name specified during installation, and huawei-csi indicates the Helm chart namespace specified during installation. For details about the component package path, see Table 1 .

    helm get values helm-huawei-csi -n huawei-csi -a | grep volumeModify -A 1
    

    The following is an example of the command output.

    • If enabled: true is displayed in the command output, the feature is enabled. In this case, skip the following steps.
    • If enabled: false is displayed in the command output, perform the following steps to enable the PVC change feature.
    volumeModify:
      enabled: false
    
  3. Go to the /helm/esdk directory and run the following command to configure the volume change CRD.

    # kubectl apply -f ./crds/volume-modify/
    customresourcedefinition.apiextensions.k8s.io/volumemodifyclaims.xuanwu.huawei.io configured
    customresourcedefinition.apiextensions.k8s.io/volumemodifycontents.xuanwu.huawei.io configured
    


    If the command output contains Warning: resource customresourcedefinitions/volumemodifycontents.xuanwu.huawei.io is missing the kubectl.kubernetes.io/last-applied-configuration…, you can ignore it. This message is displayed because the kubectl create command instead of the kubectl apply command is used for installation by Helm.

  4. Run the following command to obtain the original service configuration file.

    helm get values helm-huawei-csi -n huawei-csi -a > ./update-values.yaml
    
  5. Run the vi update-values.yaml command to open the file obtained in 4 and modify the following configuration. After the modification is complete, press Esc and enter :wq! to save the modification.

    csiExtender:
      volumeModify:    
        enabled: true
    
  6. Run the following command to update Huawei CSI services.

    helm upgrade helm-huawei-csi ./ -n huawei-csi  -f ./update-values.yaml
    
  7. Run the following command to check whether the services are started.

    kubectl get pod -n huawei-csi
    

    The following is an example of the command output. In the preceding command, huawei-csi indicates the namespace for deploying Huawei CSI.

    NAME                                     READY     STATUS    RESTARTS   AGE
    huawei-csi-controller-6dfcc4b79f-9vjtq   10/10     Running   0          24m
    huawei-csi-node-tqs87                    3/3       Running   0          20m
    

2.3.1.2 - Enabling the PVC Change Feature Manually

Prerequisites

Huawei CSI has been manually installed.

Procedure

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Go to the manual/esdk working directory and run the following command to configure the volume change CRD.

    kubectl apply -f ./crds/volume-modify/
    
  3. Run the following command. For details about the component package path, see Table 1 .

    kubectl apply -f ./deploy/huawei-csi-controller-extender.yaml
    
  4. Run the following command to check whether the services are started.

    kubectl get pod -n huawei-csi
    

    The following is an example of the command output. In the preceding command, huawei-csi indicates the namespace for deploying Huawei CSI.

    NAME                                     READY     STATUS    RESTARTS   AGE
    huawei-csi-controller-6dfcc4b79f-9vjtq   10/10     Running   0          24m
    huawei-csi-node-tqs87                    3/3       Running   0          24m
    

2.3.2 - Configuring PVC Changes

The PVC change feature is implemented using CRD. Related resources are described as follows.

Table 1 Resource description

NAME

APIVERSION

NAMESPACED

KIND

volumemodifyclaims

xuanwu.huawei.io/v1

false

VolumeModifyClaim

volumemodifycontents

xuanwu.huawei.io/v1

false

VolumeModifyContent

  • VolumeModifyClaim resources can be created, deleted, and queried, but cannot be updated.
  • VolumeModifyContent resources can only be queried and are used to display the change details of a single PVC. Do not manually create, delete, or modify the resources.
  • VolumeModifyContent resources are managed by VolumeModifyClaim. Do not manually manage VolumeModifyContent resources.

2.3.2.1 - Creating a PVC Change

Prerequisites

The storage backends associated with the PVC to be changed are HyperMetro storage backends. If they are not HyperMetro storage backends, configure them by following the instructions in Updating a Storage Backend .

PVC Change File Description

The sample template of the PVC change file is /examples/volumemodifyclaim.yaml. The following table lists the configuration items.

Table 1 Parameter description

Parameter

Description

Mandatory

Default Value

Remarks

apiVersion

API group, which is of the string type.

Yes

xuanwu.huawei.io/v1

The value is fixed at xuanwu.huawei.io/v1.

kind

Resource type, which is of the string type.

Yes

VolumeModifyClaim

The value is fixed at VolumeModifyClaim.

metadata.name

Name of a cluster resource object, which is of the string type.

Yes

-

The name must comply with the naming rules of a DNS subdomain name. The value can contain a maximum of 63 characters, including digits, lowercase letters, hyphens (-), and periods (.). It must start and end with a lowercase letter or digit.

Note: During a PVC change, the original StorageClass is backed up. The name of the backup StorageClass is <Original StorageClass name><VolumeModifyClaim name>, and must comply with the StorageClass naming rules.

spec.source.kind

Data source type, which is of the string type.

Yes

StorageClass

This parameter can only be set to StorageClass.

spec.source.name

Data source name, which is of the string type.

Yes

-

Only a StorageClass name can be configured.

spec.parameters.hyperMetro

Whether to change a common volume to a HyperMetro volume. Currently, the value can only be "true".

Yes

-

Only common storage volumes at the primary site can be changed to HyperMetro storage volumes.

spec.parameters.metroPairSyncSpeed

Data synchronization speed of a HyperMetro pair. The value ranges from 1 to 4.

The value can be:

  • 1: low
  • 2: medium
  • 3: high
  • 4: highest

No

-

This parameter is available only when spec.parameters.hyperMetro is set to "true".

Note:

  • If this parameter is not configured, the storage speed of the HyperMetro pair is determined by the storage device.
  • The highest synchronization speed may increase the host latency.

  • The spec.source.kind and spec.source.name parameters are used to specify the volume change scope. For example, if they are set to a StorageClass and the corresponding name respectively, all PVCs in the Bound state provisioned using the target StorageClass will be changed.
  • After all associated PVCs are changed, Huawei CSI will replace the original StorageClass and add the spec.parameters parameter of the VolumeModifyClaim so that the PVCs meet the StorageClass definition.

For details about the configuration in typical scenarios, see the following example:

Changing a Common Volume to a HyperMetro Volume

The following is an example of changing a common volume to a HyperMetro volume:

apiVersion: xuanwu.huawei.io/v1
kind: VolumeModifyClaim
metadata:
  name: myvmc
spec:
  source:
    kind: StorageClass
    name: mysc
  parameters:
    hyperMetro: "true"

Creating a PVC Change Resource

  • The changed HyperMetro volumes must be in HyperMetro AA mode.
  • When a common volume is changed to a HyperMetro volume, only the storage volume at the primary site can be changed.
  • Do not use Huawei CSI to manage a PVC during PVC change resource creation.
  • Multiple VolumeModifyClaim resources cannot be created for the same PVC. If the target PVC needs to be changed for multiple times, perform the changes one by one.

To create a PVC change resource using a PVC change file, perform the following steps:

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Run the following command to create a PVC change.

    kubectl create -f volumemodifyclaim.yaml 
    
  3. Query the creation result by following the instructions in Querying a PVC Change .

2.3.2.2 - Querying a PVC Change

This section describes how to use Kubectl to query the PVC change status. Currently, Huawei CSI provides the following APIs through CRD.

Querying a VolumeModifyClaim

To query a VolumeModifyClaim using kubectl, perform the following steps.

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Run the following command to query a PVC change. In the command, vmc-name indicates the name of the VolumeModifyClaim resource.

    kubectl get volumemodifyclaims <vmc-name> -owide
    

    The following is an example of the command output.

    NAME    STATUS      READY   SOURCEKIND     SOURCENAME   STARTEDAT              COMPLETEDAT            AGE
    myvmc   Completed   1/1     StorageClass   mysc         2024-06-06T03:19:13Z   2024-06-06T03:19:16Z   2m2s
    

    Table 1 Command output description

    Parameter

    Description

    NAME

    VolumeModifyClaim resource name.

    STATUS

    VolumeModifyClaim resource status. The value can be:

    • Pending: initial status.
    • Creating: The VolumeModifyClaim has completed basic verification and the server has received the change task, but the task has not been completed.
    • Completed: All associated PVCs are changed.
    • Rollback: When associated PVCs are partially changed, a user deletes PVCs.
    • Deleting: When all associated PVCs are changed, a user deletes PVCs.

    READY

    Ratio of the number of changed PVCs to the total number of PVCs that need to be changed.

    SOURCEKIND

    Data source type, for example, StorageClass.

    SOURCENAME

    Data source name, for example, StorageClass name.

    STARTEDAT

    Change start time, that is, the timestamp when the server receives the task and starts to process the task.

    COMPLETEDAT

    Change completion time, that is, the timestamp when the changes of all associated PVCs are complete. This parameter exists only when STATUS is Completed.

    AGE

    Lifetime of a VolumeModifyClaim from the time when it is created to the current time.


You can use kubectl to view the Events information of a VolumeModifyClaim. If a VolumeModifyClaim cannot meet the creation requirements or an error occurs during the creation, the server will record the Events information. The following command is used as an example:

kubectl describe volumemodifyclaims local-to-hypermetro 

Querying a VolumeModifyContent

A VolumeModifyContent is created using a VolumeModifyClaim and records the change details of a single PVC. To query a VolumeModifyContent using kubectl, perform the following steps:

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Run the following command to query a PVC change. In the command, myvmc-uid indicates the VolumeModifyContent resource name.

    kubectl get volumemodifycontents myvmc-uid  -owide
    

    The following is an example of the command output.

    NAME         STATUS      MODIFYCLAIMNAME     SOURCEVOLUME   STARTEDAT              COMPLETEDAT            AGE
    myvmc-uid    Completed   myvmc               default/mypvc  2024-06-06T03:19:07Z   2024-06-06T03:19:09Z   36m
    

    Table 2 Command output description

    Parameter

    Description

    NAME

    VolumeModifyContent resource name. The format is VolumeModifyClaim name-UID of the associated PVC.

    STATUS

    VolumeModifyContent resource status. The value can be:

    • Pending: initial status.
    • Creating: The VolumeModifyContent has completed basic verification and the server has received the change task, but the task has not been completed.
    • Completed: The associated PVC is changed.
    • Rollback: The PVC change is being rolled back.

    MODIFYCLAIMNAME

    Name of the associated VolumeModifyClaim.

    SOURCEVOLUME

    Information about the associated PVC. The format is Namespace name/PVC name.

    STARTEDAT

    PVC change start time, that is, the timestamp when the server receives the task and starts to process the task.

    COMPLETEDAT

    PVC change completion time, that is, the timestamp when the changes of all associated PVCs are complete. This parameter exists only when STATUS is Completed.

    AGE

    Lifetime of a VolumeModifyContent from the time when it is created to the current time.


You can use kubectl to view the Events information of a VolumeModifyContent. If a VolumeModifyContent cannot meet the creation requirements or an error occurs during the PVC change, the server will record the Events information. The following command is used as an example:

kubectl describe volumemodifycontents myvmc-uid

2.3.2.3 - Deleting a PVC Change

  • If STATUS of a VolumeModifyClaim is Creating, deleting the VolumeModifyClaim resource will delete the created resource on the storage side and then remove the cluster resource. After the deletion, if you continue to use the original StorageClass for PVC management, you need to restore the associated storage backend to a non-HyperMetro storage backend.
  • If STATUS of a VolumeModifyClaim is Pending or Completed, deleting the VolumeModifyClaim resource will only remove the cluster resource and will not delete the created resource on the storage side (that is, there is not interaction with the storage side).
  • VolumeModifyContent resources are managed by VolumeModifyClaim. Do not manually manage VolumeModifyContent resources.
  • If some PVCs among the PVCs to be changed meet the change requirements and the batch change fails, all PVC changes will be removed. As a result, the PVCs that meet the change requirements will not meet the change requirements.
  • If a PVC to be changed has been manually managed on the storage side, the change may fail. Do not manually manage storage volumes when using the change feature.

This section describes how to use kubectl to delete a PVC change. The procedure is as follows.

Procedure

  1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

  2. Run the following command to delete a PVC change. In the command, vmc-name indicates the name of the VolumeModifyClaim resource.

    kubectl delete volumemodifyclaims <vmc-name>
    
  3. Query the creation result by following the instructions in Querying a PVC Change .