Manage Volume Provisioning

1: StorageClass Configuration Examples in Typical Manage Volume Provisioning Scenarios
2: StorageClass Parameters for Manage Volume Provisioning
3: PVC Parameters for Manage Volume Provisioning

Manage Volume Provisioning allows administrators to use resources created on storage as PVs and supports features of dynamic volumes, such as capacity expansion, snapshot, and clone. This is a custom capability of Huawei CSI. This feature applies to the following scenarios:

In the reconstruction containerized applications, existing storage volumes need to be used.
The Kubernetes cluster is rebuilt.
Storage data is migrated in disaster recovery (DR) scenarios.

In scenarios where multiple Kubernetes clusters are deployed, when Manage Volume Provisioning is used to manage the same storage resource, management operations performed on the PVC corresponding to the resource in any cluster will not be synchronized to other clusters. For example, when you expand the capacity of a PVC in a cluster, the capacity of the corresponding PVC in other clusters will not be automatically expanded. In this case, you need to manually expand the capacity in other clusters by running the expansion commands in Expanding the Capacity of a PVC.

Prerequisites

You have registered the storage where the volume to be managed resides with CSI.
You have logged in to the storage device to obtain the name and capacity of the volume to be managed.

Configuring a StorageClass

Create a StorageClass configuration file, for example, mysc.yaml, based on service requirements by referring to StorageClass Configuration Examples in Typical Manage Volume Provisioning Scenarios and StorageClass Parameters for Manage Volume Provisioning.
Run the following command to create a StorageClass using the configuration file.
```
kubectl apply -f mysc.yaml
```

Run the following command to view the information about the created StorageClass.

kubectl get sc mysc

The following is an example of the command output.

NAME   PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
mysc   csi.huawei.com   Delete          Immediate           true                   8s

Configuring a PVC

Based on service requirements, modify specific parameters by referring to the description in this section and the PVC configuration file example to generate the PVC configuration file to be created, for example, the mypvc.yaml file in this example.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: mypvc
  annotations:
    csi.huawei.com/manageVolumeName: "*"  # Enter the storage resource name.
    csi.huawei.com/manageBackendName: "*" # Enter the storage backend name.
  labels:
    provisioner: csi.huawei.com
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem
  storageClassName: mysc
  resources:
    requests:
      storage: 100Gi

Run the following command to create a PVC using the configuration file.
```
kubectl create -f mypvc.yaml
```
After a period of time, run the following command to view the information about the created PVC.
```
kubectl get pvc mypvc
```
The following is an example of the command output. If the PVC status is Bound, the PVC has been created and can be used by a Pod.
```
NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
mypvc       Bound    pvc-840054d3-1d5b-4153-b73f-826f980abf9e   100Gi      RWO            mysc           12s
```
- After the PVC is created, if the PVC is in the Pending state after a long time (for example, one minute), refer to When a PVC Is Created, the PVC Is in the Pending State.
- You are advised to create or delete a maximum of 100 PVCs in a batch.

Using a PVC

The use method is the same as that for dynamic volume provisioning in Using a PVC.

1 - StorageClass Configuration Examples in Typical Manage Volume Provisioning Scenarios

For details about how to configure a StorageClass in typical Manage Volume Provisioning scenarios, see the following examples:

Setting the Backend and Storage Pool in a StorageClass

If multiple Huawei backends are configured in a Kubernetes cluster or a Huawei backend provides multiple storage pools, you are advised to configure the specified backend and storage pool information in the StorageClass. This prevents Huawei CSI from randomly selecting backends and storage pools and ensures that the storage device where the volume resides complies with the plan.

For details about how to set the backend and storage pool for SAN storage, see the following configuration example.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: mysc
provisioner: csi.huawei.com
allowVolumeExpansion: true
parameters:
  backend: "iscsi-san-181"
  pool: "pool001"
  volumeType: lun
  allocType: thin

For details about how to set the backend and storage pool for NAS storage, see the following configuration example.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: mysc
provisioner: csi.huawei.com
allowVolumeExpansion: true
parameters:
  backend: "iscsi-nas-181"
  pool: "pool001"
  volumeType: fs
  allocType: thin
  authClient: "*"

Setting the NFS Access Mode in a StorageClass

When a container uses an NFS file system as a storage resource, refer to the following configuration example. In this example, NFS version 4.1 is specified for mounting.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: mysc
provisioner: csi.huawei.com
parameters:
  backend: nfs-nas-181
  pool: pool001
  volumeType: fs
  allocType: thin
mountOptions:
  - nfsvers=4.1 # Specify the version 4.1 for NFS mounting.

Setting the Local File System Access Mode in a StorageClass

If a container uses a LUN of enterprise storage or distributed storage as a storage resource and a file system needs to be formatted as a local file system, refer to the following example.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: mysc
provisioner: csi.huawei.com
parameters:
  backend: iscsi-lun-181
  pool: pool001
  volumeType: lun
  allocType: thin
  fsType: xfs

Setting the DPC Access Mode in a StorageClass

If a container uses OceanStor Pacific series storage and the storage supports DPC-based access, you can configure mounting parameters for DPC-based access in the StorageClass. In this example, acl is used as the authentication parameter for mounting, and cnflush is used to set the asynchronous disk flushing mode.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: mysc
provisioner: csi.huawei.com
parameters:
  backend: nfs-dpc-101
  pool: pool001
  volumeType: fs
  allocType: thin
  authClient: "*"
mountOptions:
  - acl # Set the authentication parameter.
  - cnflush # Set the asynchronous disk flushing mode.

Setting the Permission on a Mount Directory in a StorageClass

To modify the permission on a mount directory in a container, you can configure the directory permission in a StorageClass. The following is a configuration example.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: mysc
provisioner: csi.huawei.com
allowVolumeExpansion: true
parameters:
  volumeType: fs
  allocType: thin
  authClient: "*"
  fsPermission: "777" # Set the directory permission.

After the StorageClass configuration is complete, perform the following steps to create a StorageClass.

Run the following command to create a StorageClass based on the .yaml file.
```
kubectl create -f mysc.yaml
```
Run the following command to view the information about the created StorageClass.
```
kubectl get sc
```
The following is an example of the command output.
```
NAME   PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
mysc   csi.huawei.com   Delete          Immediate           false                  34s
```
After creating a StorageClass, you can use the StorageClass to create a PV or PVC.

In the Manage Volume Provisioning mode, pay attention to the following when using a StorageClass:
Modifications to a StorageClass do not take effect on existing PVs. You need to delete these PVs and create them again using the modified StorageClass to apply the modified parameters.

2 - StorageClass Parameters for Manage Volume Provisioning

A StorageClass provides administrators with methods to describe a storage “class”. Different types may map to a different group of capability definitions. Kubernetes cluster users can dynamically provision volumes based on a StorageClass.

A StorageClass supports the following parameters.

If SAN storage is used, refer to example file /examples/sc-lun.yaml. If NAS storage is used, refer to example file /examples/sc-fs.yaml.

Table 1 StorageClass configuration parameters

Parameter	Description	Mandatory	Default Value	Remarks
metadata.name	User-defined name of a StorageClass object.	Yes	-	Take Kubernetes v1.22.1 as an example. The value can contain digits, lowercase letters, hyphens (-), and periods (.), and must start and end with a letter or digit.
provisioner	Name of the provisioner.	Yes	csi.huawei.com	Set this parameter to the driver name set during Huawei CSI installation. The value is the same as that of driverName in the values.yaml file.
reclaimPolicy	Reclamation policy. The following types are supported: Delete: Resources are automatically reclaimed. Retain: Resources are manually reclaimed.	Yes	-	Delete: When a PV/PVC is deleted, resources on the storage device are also deleted. Retain: When a PV/PVC is deleted, resources on the storage device are not deleted.
allowVolumeExpansion	Whether to allow volume expansion. If this parameter is set to true, the capacity of the PV that uses the StorageClass can be expanded.	No	false	This function can only be used to expand PV capacity but cannot be used to reduce PV capacity. The PV capacity expansion function is supported in Kubernetes 1.14 (alpha) and later versions.
parameters.backend	Name of the backend where the resource to be created is located.	No	-	If this parameter is not set, Huawei CSI will randomly select a backend that meets the capacity requirements to create resources. You are advised to specify a backend to ensure that the created resource is located on the expected backend.
parameters.volumeType	Type of the volume to be created. The following types are supported: lun: A LUN is provisioned on the storage side. fs: A file system is provisioned on the storage side.	Yes	-	If NAS storage is used, this parameter must be set to fs. If SAN storage is used, this parameter must be set to lun.
parameters.fsType	Type of a host file system. The supported types are: ext2 ext3 ext4 xfs	No	ext4	This parameter is valid only when volumeType of a StorageClass is set to lun and volumeMode of a PVC is set to Filesystem.
parameters.applicationType	Application type name for creating a LUN or NAS when the backend is OceanStor Dorado. NOTE: If an application type has been configured before a volume is managed, the value of applicationType must be the same as the configured application type.	No	-	If the value of volumeType is lun, log in to DeviceManager and choose Services > Block Service > LUN Groups > LUNs > Create to obtain the application type name. If the value of volumeType is fs, log in to DeviceManager and choose Services > File Service > File Systems > Create to obtain the application type name.
parameters.fsPermission	Permission on the directory mounted to a container.	No	-	For details about the configuration format, refer to the Linux permission settings, for example, 777 and 755. This parameter is available when volumeType is set to lun.
mountOptions.nfsvers	NFS mount option on the host. The following mount option is supported: nfsvers: protocol version for NFS mounting. The value can be 3, 4, 4.0, 4.1, or 4.2.	No	-	This parameter is optional after the -o parameter when the mount command is executed on the host. The value is in list format. If the NFS version is specified for mounting, NFS 3, 4.0, 4.1, and 4.2 protocols are supported (the protocol must be supported and enabled on storage devices). If nfsvers is set to 4, the latest protocol version NFS 4 may be used for mounting due to different OS configurations, for example, 4.2. If the 4.0 protocol is required, you are advised to set nfsver:ws to 4.0.
mountOptions.acl	The DPC namespace supports the ACL function. The DPC client supports POSIX ACL, NFSv4 ACL, and NT ACL authentication.	No	-	The descriptions of acl, aclonlyposix, cnflush, and cflush are for reference only. For details about the parameters, see OceanStor Pacific Series Product Documentation and choose Configuration > Basic Service Configuration Guide for File > Configuring Basic Services (DPC Scenario) > Accessing a DPC Share on a Client > Step 2.
mountOptions.aclonlyposix	The DPC namespace supports POSIX ACL, and the DPC client supports POSIX ACL authentication. The following protocols support POSIX ACL: DPC, NFSv3, and HDFS. If NFSv4 ACL or NT ACL is used, the DPC client cannot identify the ACL of this type. As a result, the ACL of this type does not take effect.	No	-	If aclonlyposix and acl are used together, only acl takes effect. That is, the namespace supports the ACL function.
mountOptions.cnflush	Asynchronous disk flushing mode. That is, data is not flushed to disks immediately when files in the namespace are closed.	No	-	Asynchronous flushing mode: When a file is closed, data in the cache is not flushed to storage media in synchronous mode. Instead, data is written from the cache to the storage media in asynchronous flushing mode. After the write service is complete, data is flushed from the cache to disks periodically based on the flushing period. In a multi-client scenario, if concurrent operations are performed on the same file, the file size update is affected by the disk flushing period. That is, the file size is updated only after the disk flushing is complete. Generally, the update is completed within several seconds. Synchronous I/Os are not affected by the disk flushing period.
mountOptions.cflush	Synchronous disk flushing mode. That is, data is flushed to disks immediately when files in the namespace are closed.	No	-	By default, the synchronous disk flushing mode is used.
mountOptions.sec	Kerberos 5 protocol for mounting NFS file systems.	No	-	If Kerberos 5 is used, set this parameter to krb5. If Kerberos 5i is used, set this parameter to krb5i. If Kerberos 5p is used, set this parameter to krb5p. Kerberos supports only NFSv4.0 or NFSv4.1.
mountOptions.proto	Transmission protocol used for NFS mounting. The value can be rdma.	No	-	Ensure that NFS over RDMA is enabled on the storage system. NAS storage of OceanStor Dorado 6.1.7 or later is supported.
mountOptions.port	Protocol port used for NFS mounting.	Conditionally mandatory	-	If the transmission protocol is rdma, set this parameter to 20049.
mountOptions.discard	Automatically triggers the Trim or Discard operation when a file system is mounted. This operation instructs a block device to release unused blocks.	No	-	The xfs and ext4 file systems are supported.

3 - PVC Parameters for Manage Volume Provisioning

After configuring a StorageClass, you can use the StorageClass to configure a PVC. For details about the PVC configuration template, see example file pvc-manager.yaml in the examples directory in Huawei CSI software package.

Table 1 Parameters in the pvc-manager.yaml file

Parameter	Description	Mandatory	Default Value	Remarks
metadata.annotations	PVC object annotations. Set the following parameters: Driver name/manageVolumeName: volume name on the storage. Driver name/manageBackendName: name of the backend to which the volume belongs.	Yes	csi.huawei.com/manageVolumeName: * csi.huawei.com/manageBackendName: *	For details about how to obtain Driver name, see Table 4. Driver name/manageVolumeName: name of an existing volume on the storage. Only English characters are supported. Driver name/manageBackendName: name of the storage backend in CSI. You can run the oceanctl get backend -n huawei-csi command to obtain the backend name.
metadata.labels	PVC object labels.	No	-	Format: provisioner: Driver name specified during installation Example: provisioner: csi.huawei.com This parameter takes effect when a PVC is created. It is used to listen to PVC resources and obtain information about metadata.annotations.
metadata.name	User-defined name of a PVC object.	Yes	-	Take Kubernetes v1.22.1 as an example. The value can contain digits, lowercase letters, hyphens (-), and periods (.), and must start and end with a letter or digit.
spec.volumeMode	Volume mode. This parameter is optional. When LUN volumes are used, the following types are supported: Filesystem: local file system. Block: raw device. NOTE: This parameter takes effect when a PV is mounted. The use method of this parameter must be the same as that of the managed volume. If a volume is used as a raw volume before being managed, volumeMode must be set to Block. If a volume is used in ext2, ext3, or ext4 mode before being managed, volumeMode must be set to Filesystem and fsType in the StorageClass must be set to ext2, ext3, or ext4. If a volume is used in XFS mode before being managed, volumeMode must be set to Filesystem and fsType in the StorageClass must be set to xfs.	No	Filesystem	This parameter takes effect when a PV is mounted. Filesystem indicates that a container accesses a PV using a local file system. The local file system type is specified by the fsType field in the specified StorageClass. Block indicates that a PV is accessed in raw volume mode.
spec.storageClassName	Name of the StorageClass object.	Yes	-	The configuration of the StorageClass must be the same as that of the managed volume.
spec.resources.requests.storage	Size of the volume to be created. The format is ***Gi and the unit is GiB. The size must be an integer multiple of 512 bytes.	Yes	-	The PVC capacity depends on storage specifications and host specifications. For example, OceanStor Dorado 6.1.2 or OceanStor Pacific series 8.1.0 is connected to CentOS 7. If ext4 file systems are used, see Table 2. If XFS file systems are used, see Table 3. If NFS or raw devices are used, the capacity must meet the specifications of the used Huawei storage device model and version. If the PVC capacity does not meet the specifications, a PVC or Pod may fail to be created due to the limitations of storage specifications or host file system specifications.
spec.accessModes	Access mode of the volume. RWO (ReadWriteOnce): A volume can be mounted to a node in read/write mode. This mode also allows multiple Pods running on the same node to access the volume. ROX (ReadOnlyMany): A volume can be mounted to multiple nodes in read-only mode. RWX (ReadWriteMany): A volume can be mounted to multiple nodes in read/write mode. RWOP (ReadWriteOncePod): A volume can only be mounted to a single Pod in read/write mode. Kubernetes 1.22 and later versions support this feature.	Yes	ReadWriteOnce	RWO/ROX/RWOP: supported by all types of volumes. RWOP is supported only by Kubernetes 1.22 and later versions. Check whether this feature is enabled for your Kubernetes cluster by referring to Enabling the ReadWriteOncePod Feature Gate. The support for RWX is as follows: NAS storage: supported by all volumes SAN storage: supported only by volumes whose volumeMode is set to Block

Table 2 ext4 capacity specifications

Storage Type	Storage Specifications	ext4 Specifications	CSI Specifications
OceanStor Dorado 6.1.2	512 Ki to 256 Ti	50 Ti	512 Ki to 50 Ti
OceanStor Pacific series 8.1.0	64 Mi to 512 Ti	50 Ti	64 Mi to 50 Ti

Table 3 XFS capacity specifications

Storage Type	Storage Specifications	XFS Specifications	CSI Specifications
OceanStor Dorado 6.1.2	512 Ki to 256 Ti	500 Ti	512 Ki to 500 Ti
OceanStor Pacific series 8.1.0	64 Mi to 512 Ti	500 Ti	64 Mi to 500 Ti