Storage Backend Management
Backend is an abstract concept of Huawei storage resources. Each Huawei storage device can abstract multiple backend resources using features such as tenants, storage pools, and protocols. Each backend exists independently and defines Huawei storage information required for providing persistent volumes for Kubernetes clusters.
This chapter describes how to use the oceanctl tool to manage storage backends, including creating, querying, updating, and deleting backends.
Description of the oceanctl Tool
- You have obtained the oceanctl tool, copied the oceanctl tool to the environment directory, for example, /usr/local/bin, and obtained the execute permission. The oceanctl tool is stored in /bin/oceanctl of the software package.
- The oceanctl tool depends on kubectl (for the Kubernetes platform) or oc (for the OpenShift platform) commands. Therefore, you need to run the tool on a node where kubectl or oc commands can be executed.
- By default, the user who runs oceanctl commands must have the read and write permissions on the /var/log directory. If you do not have the permissions on the directory, run the –log-dir=/path/to/custom command to specify a directory on which you have the permissions as the log file directory.
- huawei-csi is the default namespace used by oceanctl to create a backend.
- For details about oceanctl commands, see Description of oceanctl Commands.
1 - Managing Storage Backends
This section describes how to create a storage backend. Currently, you can create a backend based on the configured backend yaml file or the exported configmap.json file.
If you create a backend by adding a backend yaml file, configure the backend file by referring to Examples of Storage Backend Configuration Files in Typical Scenarios.
If the exported configmap.json file exists, create a storage backend by referring to Creating a Storage Backend.
1.1.1 - Examples of Storage Backend Configuration Files in Typical Scenarios
For details about the backend configuration in typical scenarios, see the following examples. For details about the parameter configuration, see Storage Backend Parameters.
Configuring a Storage Backend of the iSCSI Type
If you want to use the iSCSI protocol, ensure that the iSCSI client has been installed on the host before installing Huawei CSI. You can check whether the client has been installed on the host by referring to Checking the Status of Host-Dependent Software. If the iSCSI client is not installed, restart the huawei-csi-node service after installing the iSCSI client. During the restart, do not use Huawei CSI to create new resources or mount or unmount an existing PVC. The following command is used as an example:
kubectl delete pods -n huawei-csi -l app=huawei-csi-node
The following is an example of the backend configuration file of the iSCSI type for enterprise storage:
storage: "oceanstor-san"
name: "dorado-iscsi-155"
namespace: "huawei-csi"
urls:
- "https://192.168.129.155:8088"
- "https://192.168.129.156:8088"
pools:
- "StoragePool001"
parameters:
protocol: "iscsi"
portals:
- "192.168.128.120"
- "192.168.128.121"
maxClientThreads: "30"
The following is an example of the backend configuration file of the iSCSI type for distributed storage:
storage: "fusionstorage-san"
name: "pacific-iscsi-125"
namespace: "huawei-csi"
urls:
- "https://192.168.129.125:8088"
- "https://192.168.129.126:8088"
pools:
- "StoragePool001"
parameters:
protocol: "iscsi"
portals:
- "192.168.128.122"
- "192.168.128.123"
maxClientThreads: "30"
Configuring a Storage Backend of the FC Type
If you want to use the FC protocol, ensure that the FC network between the host and the storage device is connected before installing Huawei CSI. If the FC network is not connected, connect the FC network and then restart the huawei-csi-node service. During the restart, do not use Huawei CSI to create new resources or mount or unmount an existing PVC. The following command is used as an example:
kubectl delete pods -n huawei-csi -l app=huawei-csi-node
The following is an example of the backend configuration file of the FC type for enterprise storage:
storage: "oceanstor-san"
name: "fc-155"
namespace: "huawei-csi"
urls:
- "https://192.168.129.155:8088"
- "https://192.168.129.156:8088"
pools:
- "StoragePool001"
parameters:
protocol: "fc"
maxClientThreads: "30"
Configuring a Storage Backend of the NVMe over RoCE Type
If you want to use the NVMe over RoCE protocol, ensure that the NVMe over RoCE network between the host and the storage device is connected before installing Huawei CSI. If the NVMe over RoCE network is not connected, connect the NVMe over RoCE network and then restart the huawei-csi-node service. During the restart, do not use Huawei CSI to create new resources or mount or unmount an existing PVC. The following command is used as an example:
kubectl delete pods -n huawei-csi -l app=huawei-csi-node
The following is an example of the backend configuration file of the NVMe over RoCE type for enterprise storage:
storage: "oceanstor-san"
name: "roce-155"
namespace: "huawei-csi"
urls:
- "https://192.168.129.155:8088"
- "https://192.168.129.156:8088"
pools:
- "StoragePool001"
parameters:
protocol: "roce"
portals:
- "192.168.128.120"
- "192.168.128.121"
maxClientThreads: "30"
Configuring a Storage Backend of the NVMe over FC Type
The following is an example of the backend configuration file of the NVMe over FC type for enterprise storage:
storage: "oceanstor-san"
name: "fc-nvme-155"
namespace: "huawei-csi"
urls:
- "https://192.168.129.155:8088"
- "https://192.168.129.156:8088"
pools:
- "StoragePool001"
parameters:
protocol: "fc-nvme"
maxClientThreads: "30"
Configuring a Storage Backend of the NFS Type
The following is an example of the backend configuration file of the NFS type for enterprise storage:
storage: "oceanstor-nas"
name: "nfs-155"
namespace: "huawei-csi"
urls:
- "https://192.168.129.155:8088"
- "https://192.168.129.156:8088"
pools:
- "StoragePool001"
parameters:
protocol: "nfs"
portals:
- "192.168.128.155"
maxClientThreads: "30"
The following is an example of the backend configuration file of the NFS type for distributed storage:
storage: "fusionstorage-nas"
name: "nfs-126"
namespace: "huawei-csi"
urls:
- "https://192.168.129.125:8088"
- "https://192.168.129.126:8088"
pools:
- "StoragePool001"
parameters:
protocol: "nfs"
portals:
- "192.168.128.123"
maxClientThreads: "30"
Configuring a Storage Backend of the SCSI Type
The following is an example of the backend configuration file of the SCSI type for distributed storage:
storage: "fusionstorage-san"
name: "scsi-155"
namespace: "huawei-csi"
urls:
- "https://192.168.129.155:8088"
pools:
- "StoragePool001"
parameters:
protocol: "scsi"
portals:
- {"hostname01": "192.168.125.21","hostname02": "192.168.125.22"}
maxClientThreads: "30"
Configuring a Storage Backend of the DPC Type
The following is an example of the backend configuration file of the DPC type for distributed storage:
storage: "fusionstorage-nas"
name: "dpc-155"
namespace: "huawei-csi"
urls:
- "https://192.168.129.155:8088"
- "https://192.168.129.156:8088"
pools:
- "StoragePool001"
parameters:
protocol: "dpc"
maxClientThreads: "30"
Configuring Storage Backends of the Dtree Type
The following is an example of the backend configuration file of the Dtree type for enterprise storage:
storage: "oceanstor-dtree"
name: "nfs-dtree"
namespace: "huawei-csi"
urls:
- "https://192.168.129.155:8088"
parameters:
protocol: "nfs"
parentname: "parent-filesystem"
portals:
- "192.168.128.155"
maxClientThreads: "30"
Configuring Storage Backends of the HyperMetro Type
- Before configuring NAS HyperMetro, you need to configure the HyperMetro relationship between two storage devices, including the remote device, HyperMetro domain, and the like. The HyperMetro domain of the file system can only work in HyperMetro active-active (AA) mode. For details about the configuration operation, see the product documentation of the corresponding storage model.
- The accounts for connecting to NAS HyperMetro backends must be the administrator accounts of the storage vStores.
- Except NAS HyperMetro backends, the management URLs of other backends cannot be the URL of a logical management port of a vStore that has established the HyperMetro relationship.
- When a HyperMetro storage backend is used, do not provision common file systems. Otherwise, services may be interrupted in logical port failover scenarios.
CSI allows you to connect to OceanStor or OceanStor Dorado and provision HyperMetro volumes of the NFS type on the storage side. You need to configure storage backends that work in HyperMetro mode. The procedure is as follows: Create two configuration files and create backends one by one.
This example shows how to configure backends of the HyperMetro type for Huawei OceanStor or OceanStor Dorado. First, create local storage backend configuration file nfs-hypermetro-155.yaml.
storage: "oceanstor-nas"
name: "nfs-hypermetro-155"
namespace: "huawei-csi"
urls:
- "https://192.168.129.155:8088"
- "https://192.168.129.156:8088"
pools:
- "StoragePool001"
metrovStorePairID: "f09838237b93c000"
metroBackend: "nfs-hypermetro-157"
parameters:
protocol: "nfs"
portals:
- "192.168.129.155"
maxClientThreads: "30"
After the local backend is created, create remote storage backend configuration file nfs-hypermetro-157.yaml.
storage: "oceanstor-nas"
name: "nfs-hypermetro-157"
namespace: "huawei-csi"
urls:
- "https://192.168.129.157:8088"
- "https://192.168.129.158:8088"
pools:
- "StoragePool001"
metrovStorePairID: "f09838237b93c000"
metroBackend: "nfs-hypermetro-155"
parameters:
protocol: "nfs"
portals:
- "192.168.129.157"
maxClientThreads: "30"
1.1.2 - Storage Backend Parameters
An example template of the backend configuration file is /examples/backend/backend.yaml. The following table lists the parameters.
Table 1 backend parameters
Parameter | Description | Mandatory | Default Value | Remarks |
|---|
storage | Storage service type. - If enterprise storage provides SAN, set this parameter to oceanstor-san.
- If enterprise storage provides NAS, set this parameter to oceanstor-nas.
- If enterprise storage provides NAS of the Dtree type, set this parameter to oceanstor-dtree.
- If distributed storage provides SAN, set this parameter to fusionstorage-san.
- If distributed storage provides NAS, set this parameter to fusionstorage-nas.
| Yes | oceanstor-nas | One backend can provide only one storage service. If a single Huawei storage system can provide both SAN and NAS storage services, you can configure multiple backends and use different storage service types for each backend. |
name | Storage backend name. The value can contain a maximum of 63 characters, including lowercase letters, digits, and hyphens (-). It must start with a letter or digit. | Yes | - | Ensure that the storage backend name is unique. |
namespace | Namespace. | No | - | The storage backend must be in the same namespace as Huawei CSI. |
vstoreName | vStore name on the storage side. This parameter needs to be specified when the connected backend is OceanStor V5 and resources need to be provisioned under a specified vStore. | Conditionally mandatory | - | This parameter needs to be specified only when the backend is OceanStor V5 and vStores need to be supported. |
accountName | Account name on the storage side. This parameter is mandatory when OceanStor Pacific series NAS is connected and NAS resources need to be provisioned under a specified account. | Conditionally mandatory | - | This parameter needs to be specified only when the backend is OceanStor Pacific series NAS and accounts need to be supported. |
urls | Management URLs of storage device. The value format is a list. The value can be a domain name or an IP address + port number. Only IPv4 addresses are supported. | Yes | - | If the connected backend is OceanStor or OceanStor Dorado storage and resources need to be provisioned under a specified vStore, set this parameter to the URL of the logical management port of the vStore. |
pools | Storage pools of storage devices. The value format is a list. | Conditionally mandatory | - | This parameter is optional when storage is set to oceanstor-dtree. |
parameters.protocol | Storage protocol. The value is a character string. - iscsi
- fc
- roce
- fc-nvme
- nfs
- dpc
- scsi
| Yes | - | - If the value is set to iscsi, ensure that an iSCSI client has been installed on the connected compute node.
- If the value is set to nfs, ensure that an NFS client tool has been installed on the connected compute node.
- If the value is set to fc-nvme or roce, ensure that the nvme-cli tool has been installed on the connected compute node. The tool version must be 1.x and not earlier than 1.9.
- If the value is set to dpc, ensure that DPC has been installed on the connected compute node and the node has been added as a DPC compute node on the storage device to be connected.
- If the value is set to scsi, ensure that a distributed storage VBS client has been installed on the connected compute node.
|
parameters.portals | Service access port. Nodes will use this port to read and write storage resources. The value format is a list. Multiple ports can be configured if the protocol is iscsi or roce. Only one port can be configured if the protocol is nfs. Service ports do not need to be configured if the protocol is fc, fc-nvme, or dpc. If the protocol is scsi, the port is in dictionary format where the key indicates the host name and the value indicates the IP address (only IPv4 addresses are supported). | Conditionally mandatory | - | - If a vStore or account is used to connect to a backend, portals must be set to the logical port information of the vStore or account.
- If nfs is used, the value can be a domain name.
|
parameters.ALUA | ALUA configuration of the storage backend. If the worker node uses the native multipathing software provided by the OS and ALUA is enabled, you need to configure this parameter. | Conditionally mandatory | - | If ALUA is enabled for the host multipathing software, ensure that the backend ALUA configuration is the same as that of the host ALUA configuration. For details about the ALUA configuration, see Configuring ALUA Using Helm. |
parameters.parentname | Name of a file system on the current storage device. Dtree is created in the file system. This parameter is mandatory when storage is set to oceanstor-dtree. | Conditionally mandatory | - | Query the name on the File Systems page of DeviceManager. |
metrovStorePairID | HyperMetro vStore pair ID. This parameter is mandatory when a PV to be created on the storage side needs to support the NAS HyperMetro feature. In this case, you need to enter the ID of the HyperMetro vStore pair to which the PV to be created belongs. | Conditionally mandatory | - | You can query the HyperMetro vStore pair ID on DeviceManager. |
metroBackend | Backend name of the HyperMetro peer. The value is a character string. This parameter is mandatory when a PV to be created on the storage side needs to support the NAS HyperMetro feature. In this case, you need to enter the name of the other backend to form a HyperMetro pair with the current backend. | Conditionally mandatory | - | The names of the two backends in the pair must be entered. After the two backends form a HyperMetro relationship, they cannot form a HyperMetro relationship with other backends. |
supportedTopologies | Storage topology awareness configuration. The parameter format is JSON of the list type. | Conditionally mandatory | - | This parameter is mandatory if storage topology awareness is enabled. For details, see Configuring Storage Topology Awareness Using Helm. |
maxClientThreads | Maximum number of concurrent connections to a storage backend. | No | 30 | If this parameter is not specified, the default maximum number of connections is 30. |
1.2 - Querying a Storage Backend
Run the oceanctl commands in Querying a Storage Backend to query the storage backend information.
1.4 - Deleting a Storage Backend
Do not delete a storage backend when a volume management operation is being performed on it.
Example of Deleting a Backend
Run the following command to obtain information about a storage backend.
The following is an example of the command output.
NAMESPACE NAME PROTOCOL STORAGETYPE SN STATUS ONLINE URL
huawei-csi backend-1 roce oceanstor-san xxxxxxxxxxxxxxxxxxxx Bound true https://192.168.129.157:8088
huawei-csi backend-2 roce oceanstor-san xxxxxxxxxxxxxxxxxxxx Bound true https://192.168.129.158:8088
Run the following command to delete the specified storage backend.
oceanctl delete backend backend-1
Run the following command to check the deletion result.
oceanctl get backend backend-1
The following is an example of the command output. If not found is displayed, the deletion is successful.
Error from server (NotFound): backend "backend-1" not found
2 - (Optional) Adding a Certificate to a Storage Backend
This section describes how to create a certificate for a storage backend. If certificate verification is required for logging in to the storage, you can add a certificate by referring to this section. Currently, you can create a certificate for a storage backend based on the specified .crt or .pem file.
Before creating a certificate for a storage backend, import the prepared certificate to the storage array.
2.1 - Creating a Certificate for a Storage Backend
Prerequisites
A certificate has been created. Take OceanStor Dorado as an example. For details about how to create a certificate, click here.
Example of Creating a Certificate
Prepare a certificate file in advance, for example, cert.crt.
Run the following command to obtain information about a storage backend.
The following is an example of the command output.
NAMESPACE NAME PROTOCOL STORAGETYPE SN STATUS ONLINE URL
huawei-csi backend-1 roce oceanstor-san xxxxxxxxxxxxxxxxxxxx Bound true https://192.168.129.157:8088
huawei-csi backend-2 roce oceanstor-san xxxxxxxxxxxxxxxxxxxx Bound true https://192.168.129.158:8088
Run the following command to create a certificate for the specified storage backend.
oceanctl create cert cert-1 -b backend-1 -f /path/to/cert.crt
Check the certificate creation result.
oceanctl get cert -b backend-1
The following is an example of the command output.
NAMESPACE NAME BOUNDBACKEND
huawei-csi cert-1 backend-1
2.2 - Querying a Storage Backend Certificate
Query storage backend certificates using the commands in Querying a Storage Backend Certificate.
2.3 - Updating a Storage Backend Certificate
Before updating a certificate, prepare a new certificate file and update the storage backend certificate by following the instructions provided in this section. If the certificate is no longer used, delete the certificate from the storage backend by referring to Deleting a Storage Backend Certificate.
Procedure
Run the following command to obtain information about a storage backend.
The following is an example of the command output.
NAMESPACE NAME PROTOCOL STORAGETYPE SN STATUS ONLINE URL
huawei-csi backend-1 roce oceanstor-san xxxxxxxxxxxxxxxxxxxx Bound true https://192.168.129.157:8088
huawei-csi backend-2 roce oceanstor-san xxxxxxxxxxxxxxxxxxxx Bound true https://192.168.129.158:8088
Run the following command to check whether the specified storage backend has a certificate.
oceanctl get cert -b backend-1
The following is an example of the command output.
NAMESPACE NAME BOUNDBACKEND
huawei-csi cert-1 backend-1
Run the following command to update the certificate of the specified storage backend.
oceanctl update cert -b backend-1 -f /path/to/cert.crt