Huawei Container Storage Suite

• 1: About This Document
• 2: Overview
• 3: Quick Start
• 4: Compatibility and Features
• 4.1: Kubernetes and OS Compatibility
• 4.2: Kubernetes Feature Matrix
• 4.3: Compatibility with Huawei Enterprise Storage
• 4.4: Compatibility with Huawei Distributed Storage
• 5: Installation and Deployment
• 5.1: Installation Preparations
• 5.1.1: Downloading the Huawei CSI Software Package
• 5.1.2: Uploading a Huawei CSI Image
• 5.1.3: Checking User Configurations on Huawei Storage
• 5.1.4: Checking Volume Snapshot-Dependent Components
• 5.1.5: Checking the Host Multipathing Configuration
• 5.1.6: Checking the Status of Host-Dependent Software
• 5.1.7: Checking the Images on Which CSI Depends
• 5.2: Installing Huawei CSI
• 5.2.1: Installing Huawei CSI Using Helm
• 5.2.1.1: Installing Huawei CSI on Kubernetes, OpenShift, and Tanzu
• 5.2.1.2: Installing Huawei CSI on the CCE or CCE Agile Platform
• 5.2.1.3: Parameters in the values.yaml File of Helm
• 5.2.2: Manually Installing Huawei CSI
• 5.3: Uninstalling Huawei CSI
• 5.3.1: Uninstalling Huawei CSI Using Helm
• 5.3.1.1: Uninstalling Huawei CSI on Kubernetes, OpenShift, and Tanzu
• 5.3.1.2: Uninstalling Huawei CSI on CCE or CCE Agile
• 5.3.1.3: Uninstalling CSI-Dependent Component Services
• 5.3.2: Manually Uninstalling Huawei CSI
• 5.4: Upgrading or Rolling Back Huawei CSI
• 5.4.1: Upgrading or Rolling Back Huawei CSI Using Helm
• 5.4.1.1: Rolling Back Huawei CSI on Kubernetes, OpenShift, and Tanzu
• 5.4.1.2: Upgrading Huawei CSI
• 5.4.1.2.1: Upgrading from 2.x or 3.x to 4.x
• 5.4.1.2.2: Upgrading Huawei CSI on Kubernetes, OpenShift, and Tanzu
• 5.4.1.2.3: Upgrading Huawei CSI on CCE or CCE Agile
• 5.4.1.3: Rolling Back Huawei CSI on CCE or CCE Agile
• 5.4.2: Manual Upgrade/Rollback
• 5.4.2.1: Upgrading Huawei CSI
• 5.4.2.2: Rolling Back Huawei CSI
• 6: Storage Backend Management
• 6.1: Managing Storage Backends
• 6.1.1: Creating a Storage Backend
• 6.1.1.1: Examples of Storage Backend Configuration Files in Typical Scenarios
• 6.1.1.2: Storage Backend Parameters
• 6.1.2: Querying a Storage Backend
• 6.1.3: Updating a Storage Backend
• 6.1.3.1: Updating the Password of a Storage Backend Using oceanctl
• 6.1.3.2: Manually Updating a Storage Backend
• 6.1.4: Deleting a Storage Backend
• 6.2: (Optional) Adding a Certificate to a Storage Backend
• 6.2.1: Creating a Certificate for a Storage Backend
• 6.2.2: Querying a Storage Backend Certificate
• 6.2.3: Updating a Storage Backend Certificate
• 6.2.4: Deleting a Storage Backend Certificate
• 6.3: Description of oceanctl Commands
• 7: Using Huawei CSI
• 7.1: Managing a PVC
• 7.1.1: Creating a PVC
• 7.1.1.1: Dynamic Volume Provisioning
• 7.1.1.1.1: StorageClass Configuration Examples in Typical Dynamic Volume Provisioning Scenarios
• 7.1.1.1.2: StorageClass Parameters for Dynamic Volume Provisioning
• 7.1.1.1.3: PVC Parameters for Dynamic Volume Provisioning
• 7.1.1.2: Manage Volume Provisioning
• 7.1.1.2.1: StorageClass Configuration Examples in Typical Manage Volume Provisioning Scenarios
• 7.1.1.2.2: StorageClass Parameters for Manage Volume Provisioning
• 7.1.1.2.3: PVC Parameters for Manage Volume Provisioning
• 7.1.1.3: Static Volume Provisioning
• 7.1.1.3.1: PV Parameters for Static Volume Provisioning
• 7.1.1.3.2: PVC Parameters for Static Volume Provisioning
• 7.1.2: Expanding the Capacity of a PVC
• 7.1.3: Cloning a PVC
• 7.1.4: Creating a PVC Using a Snapshot
• 7.2: Creating a VolumeSnapshot
• 7.2.1: Checking Information About Volume Snapshot-dependent Components
• 7.2.2: Configuring a VolumeSnapshotClass
• 7.2.3: Configuring a VolumeSnapshot
• 8: Advanced Features
• 8.1: Configuring ALUA
• 8.1.1: Configuring ALUA Using Helm
• 8.1.1.1: Configuring ALUA Parameters for a Huawei Enterprise Storage Backend
• 8.1.1.2: Configuring ALUA Parameters for a Distributed Storage Backend
• 8.2: Configuring Storage Topology Awareness
• 8.2.1: Configuring Storage Topology Awareness Using Helm
• 8.3: PVC Change
• 8.3.1: Enabling the PVC Change Feature
• 8.3.1.1: Enabling the PVC Change Feature Using Helm
• 8.3.1.2: Enabling the PVC Change Feature Manually
• 8.3.2: Configuring PVC Changes
• 8.3.2.1: Creating a PVC Change
• 8.3.2.1.1: Preparing a PVC Change File
• 8.3.2.1.2: Creating a PVC Change Resource
• 8.3.2.2: Querying a PVC Change
• 8.3.2.3: Deleting a PVC Change
• 9: Common Operations
• 9.1: Installing Helm 3
• 9.2: Collecting Information
• 9.2.1: Obtaining the CSI Version
• 9.2.2: Viewing Huawei CSI Logs
• 9.2.3: Collecting Logs
• 9.3: Downloading a Container Image
• 9.4: Updating the huawei-csi-controller or huawei-csi-node Service
• 9.5: Modifying the Log Output Mode
• 9.6: Enabling the ReadWriteOncePod Feature Gate
• 9.7: Configuring Access to the Kubernetes Cluster as a Non-root User
• 10: Troubleshooting
• 10.1: Huawei CSI Service Issues
• 10.1.1: Failed to Start the huawei-csi-node Service with Error Message /var/lib/iscsi is not a directory Reported
• 10.1.2: Huawei CSI Services Fail to Be Started and Error Message '/etc/localtime is not a file' Is Displayed
• 10.1.3: Failed to Start huawei-csi Services with the Status Displayed as InvalidImageName
• 10.2: Storage Backend Issues
• 10.2.1: A webhook Fails to Be Called When the oceanctl Tool Is Used to Manage Backends
• 10.2.2: A Backend Fails to Be Created Using the oceanctl Tool and Error Message `context deadline exceeded` Is Displayed
• 10.2.3: An Account Is Locked After the Password Is Updated on the Storage Device
• 10.3: PVC Issues
• 10.3.1: When a PVC Is Created, the PVC Is in the Pending State
• 10.3.2: Before a PVC Is Deleted, the PVC Is in the Pending State
• 10.3.3: Failed to Expand the Capacity of a Generic Ephemeral Volume
• 10.3.4: Failed to Expand the PVC Capacity Because the Target Capacity Exceeds the Storage Pool Capacity
• 10.4: Pod Issues
• 10.4.1: After a Worker Node in the Cluster Breaks Down and Recovers, Pod Failover Is Complete but the Source Host Where the Pod Resides Has Residual Drive Letters
• 10.4.2: When a Pod Is Created, the Pod Is in the ContainerCreating State
• 10.4.3: A Pod Is in the ContainerCreating State for a Long Time When It Is Being Created
• 10.4.4: A Pod Fails to Be Created and the Log Shows That the Execution of the mount Command Times Out
• 10.4.5: A Pod Fails to Be Created and the Log Shows That the mount Command Fails to Be Executed
• 10.4.6: A Pod Fails to Be Created and Message publishInfo doesn't exist Is Displayed in the Events Log
• 10.4.7: After a Pod Fails to Be Created or kubelet Is Restarted, Logs Show That the Mount Point Already Exists
• 10.4.8: I/O error Is Displayed When a Volume Directory Is Mounted to a Pod
• 10.4.9: Failed to Create a Pod Because the iscsi tcp Service Is Not Started Properly When the Kubernetes Platform Is Set Up for the First Time
• 10.5: Common Problems and Solutions for Interconnecting with the Tanzu Kubernetes Cluster
• 10.5.1: A Pod Cannot Be Created Because the PSP Permission Is Not Created
• 10.5.2: Changing the Mount Point of a Host
• 10.5.3: Changing the Default Port of the livenessprobe Container
• 10.5.4: Failed to Create an Ephemeral Volume
• 11: Appendix
• 11.1: Example ALUA Configuration Policy of OceanStor V5 and OceanStor Dorado V3
• 11.2: Example ALUA Configuration Policy of OceanStor Dorado
• 11.3: Example ALUA Configuration Policy of Distributed Storage
• 11.4: Communication Matrix
• 11.5: Configuring Custom Permissions
• 11.6: Huawei CSI Resource Management

1 - About This Document

Intended Audience

This document is intended for:

• Technical support engineers
• O&M engineers
• Engineers with basic knowledge of storage and Kubernetes

Symbol Conventions

The symbols that may be found in this document are defined as follows.

2 - Overview

Container Storage Interface (CSI) is an industry standard used to expose block and file storage systems to container workloads on container orchestration systems (COs) such as Kubernetes. Huawei CSI plug-in is used to communicate with Huawei enterprise storage and distributed storage products and provide storage services for Kubernetes container workloads. It is a mandatory plug-in used by Huawei enterprise storage and distributed storage in the Kubernetes environment.

Kubernetes uses a series of officially maintained sidecar components to register and listen to Kubernetes object resources and call CSI Driver through gRPC when necessary. Huawei CSI Driver implements the call initiated by sidecar on Huawei storage, for example, creating a Persistent Volume (PV) is to create a LUN or file system on Huawei storage. The following figure shows the overall structure of Kubernetes, Huawei CSI, and Huawei storage.

Figure 1 CSI overall architecture

Huawei CSI consists of two components: huawei-csi-controller and huawei-csi-node.

• huawei-csi-controller: one or more Pods (including Controller Service and Identity Service) running in Deployment mode. It is used to interact with Huawei storage using RESTful. Therefore, the node running the huawei-csi-controller component must be connected to the management plane network of the storage.
• huawei-csi-node: a Pod (including Node Service and Identity Service) that runs on Kubernetes worker nodes in DaemonSet mode. It is used to mount and unmount a LUN/file system provided by Huawei storage on worker nodes. Therefore, the node running the huawei-csi-node component must be connected to the service plane network of the storage.

The following figure shows the deployment model of Huawei CSI.

Figure 2 CSI deployment model

This document describes how to install, deploy, and use the Huawei CSI V4.5.0 plug-in.

3 - Quick Start

This chapter describes how to quickly install and use Huawei CSI to manage Persistent Volume Claims (PVCs).

Huawei CSI Use Process

Figure 1 CSI installation and use process

Compatibility and Features

Before using this plug-in, learn about its compatibility with Huawei storage, container platforms, and host operating systems (OSs), as well as supported features.

Compatibility and Features

Installation Preparations

Before installing Huawei CSI, you need to prepare the configurations of related environments such as container platforms and hosts.

Installation Preparations

Installation and Deployment

Huawei CSI provides two installation modes: installation using Helm and manual installation, which are suitable for different container platforms such as Kubernetes and OpenShift.

Installation and Deployment

Creating a Storage Backend

Before using Huawei CSI, you need to create storage backend resources.

Creating a Storage Backend

Using Huawei CSI

Now, you can use Huawei CSI to manage PVCs.

Using Huawei CSI

4 - Compatibility and Features

This chapter describes the container management platforms, operating systems (OSs), and multipathing software supported by Huawei CSI plug-in, as well as the features and functions provided by the CSI plug-in when working with Huawei storage.

4.1 - Kubernetes and OS Compatibility

Huawei CSI plug-in supports the following container management platforms.

Table 1 Supported container management platforms

• The connection between Huawei CSI and Tanzu Kubernetes supports only the centralized storage NAS scenario. For the related FAQ, see Common Problems and Solutions for Interconnecting with the Tanzu Kubernetes Cluster.
• The connection between Huawei CSI and CCE or CCE Agile supports only centralized storage.

The following table lists the OSs and multipathing software supported by the Huawei CSI plug-in.

Table 2 Supported host OSs and multipathing software versions

Note 1: Only Kylin x86_64 V10 SP2 supports UltraPath 31.2.0.

Note 2: Only Kylin ARM V10 SP3 supports UltraPath 31.3.0.

Note 3: Anolis OS supports only OceanStor Pacific storage.

For DM-Multipath 0.7, some virtual devices may not be displayed in the command output after the multipathd show maps command is executed. Therefore, you are advised to use version 0.8 or later. You can query the DM-Multipath version in either of the following ways:

• If the rpm package is used, run the rpm -qa | grep multipath or rpm -qa | grep device-mapper command.
• If the deb package is used, run the dpkg -l | grep multipath command.

4.2 - Kubernetes Feature Matrix

This section describes the features of different Kubernetes versions supported by Huawei CSI.

Table 1 Kubernetes versions and supported features

• Note 1: Manage Provisioning is a volume management feature customized by Huawei CSI. This feature allows existing storage resources to be managed by Kubernetes. You are not allowed to manage a storage resource for multiple times and concurrently delete or create a storage resource. When a storage resource is managed by multiple clusters, operations on the managed volume in a single cluster take effect only in the cluster and will not be synchronized to other clusters. Instead, you need to perform these operations on the managed volume in other clusters.
• Note 2: Modify Volume is a PVC change feature customized by Huawei CSI. This feature allows a common volume to be changed to a HyperMetro volume. To use this feature, ensure that the connected storage supports the volume HyperMetro feature.

4.3 - Compatibility with Huawei Enterprise Storage

Huawei CSI plug-in is compatible with Huawei OceanStor series all-flash storage and hybrid flash storage. The following table lists the supported storage versions.

Table 1 Supported Huawei enterprise storage

Huawei CSI plug-in supports the following features for Huawei enterprise storage.

Table 2 Features supported by Huawei enterprise storage and constraints

• Note 1: Manage Provisioning is a volume management feature customized by Huawei CSI. This feature allows existing storage resources to be managed by Kubernetes. You are not allowed to manage a storage resource for multiple times and concurrently delete or create a storage resource.
• Note 2: If the user’s container platform is deployed in a virtualization environment, only iSCSI networking is supported.
• Note 3: If NVMe over RoCE or NVMe over FC is used, the version of the nvme-cli tool on worker nodes must be 1.9 or later. To query the version, run the nvme version command.
• Note 4: Only OceanStor Dorado 6.1.0 and later versions support NFS. Only OceanStor Dorado 6.1.3 and later versions support NFS 4.1. OceanStor Dorado 6.1.7 and later versions support NFS over RDMA. Only OceanStor Dorado 6.1.8 and later versions support NFS 4.2.
• Note 5: The provisioned PVC whose volumeType is lun and accessModes is ReadOnlyMany does not support capacity expansion.
• Note 6: Only system users can configure QoS.
• Note 7: Only the active-active (AA) mode is supported.
• Note 8: Only OceanStor Dorado 6.1.3 and later versions support multi-tenant.

Huawei CSI plug-in supports the following Dtree features for Huawei enterprise storage.

Table 3 Features supported by Dtree

Table 4 Huawei storage versions supported by Dtree

4.4 - Compatibility with Huawei Distributed Storage

Huawei CSI plug-in is compatible with Huawei OceanStor series distributed storage systems. The following table lists the supported storage versions.

Table 1 Supported Huawei distributed storage

Huawei CSI plug-in supports the following features for Huawei distributed storage.

Table 2 Features supported by Huawei distributed storage and constraints

• Note 1: Manage Provisioning is a volume management feature customized by Huawei CSI. This feature allows existing storage resources to be managed by Kubernetes. You are not allowed to manage a storage resource for multiple times and concurrently delete or create a storage resource.
• Note 2: Only OceanStor Pacific series 8.1.2 and later versions support DPC. For details about whether the OSs supported by Huawei CSI support DPC, see the compatibility document of the corresponding product version.
• Note 3: Only OceanStor Pacific series 8.1.2 and later versions support NFS 4.1.
• Note 4: The provisioned PVC whose volumeType is lun and accessModes is ReadOnlyMany does not support capacity expansion.

5 - Installation and Deployment

5.1 - Installation Preparations

This chapter describes the preparations for the installation.

Prerequisites

Before performing the operations described in this chapter, ensure that the following conditions are met:

• A container management platform has been deployed and is running properly, and its compatibility meets the requirements described in Kubernetes and OS Compatibility.
• (Mandatory for enterprise storage) Initial configuration for interconnecting with Huawei enterprise storage has been completed, including storage pool division and port configuration. The version of the storage product meets the requirements in Compatibility with Huawei Enterprise Storage.
• (Mandatory for distributed storage) Initial configuration for interconnecting with Huawei distributed storage has been completed, including storage pool division and port configuration. The version of the storage product meets the requirements in Compatibility with Huawei Distributed Storage.
• The connectivity between Huawei storage and the container platform host has been configured. For example, the worker node running huawei-csi-controller communicates properly with the management IP address of the storage device to be connected, and the worker node running huawei-csi-node communicates properly with the service IP address of the storage device to be connected. In iSCSI scenarios, the ping command can be used to verify the connectivity.
• Ensure that the language of the operating system is English.
• Ensure that storage resource names, such as storage pool names and tenant names, are in English.

5.1.1 - Downloading the Huawei CSI Software Package

This section describes how to download the software package and the component structure of the software package.

1. Open a browser and enter https://github.com/Huawei/eSDK_K8S_Plugin/releases in the address box.

2. Download the software package of the 4.5.0 version based on the CPU architecture.

   Software package naming rule: Plug-in name (eSDK_Huawei_Storage_Kubernetes_CSI_Plugin) + Version number + CPU architecture

3. Decompress the downloaded software package. The following table shows the component structure of the software package.

   Table 1 Component description

   Component	Description
   image/huawei-csi-v4.5.0-arch.tar	huawei-csi-driver image. arch is X86 or ARM.
   image/storage-backend-controller-v4.5.0-arch.tar	Back-end management controller image. arch is X86 or ARM.
   image/storage-backend-sidecar-v4.5.0-arch.tar	Back-end management sidecar image. arch is X86 or ARM.
   image/huawei-csi-extender-v4.5.0-arch.tar	huawei-csi-extender image. arch is X86 or ARM.
   bin/	Binary file used by an image provided by Huawei.
   bin/oceanctl	Command line tool provided by Huawei, which can be used to manage storage backends.
   helm/	Helm project used to deploy Huawei CSI.
   manual/	Used to manually install and deploy Huawei CSI.
   examples/	.yaml sample file used during CSI use.
   examples/backend	.yaml sample file used to create a storage backend.

5.1.2 - Uploading a Huawei CSI Image

Huawei provides the huawei-csi image for users. For details about how to obtain the image file, see Downloading the Huawei CSI Software Package.

To use the CSI image on the container management platform, you need to import the CSI image to the cluster in advance using either of the following methods:

• (Recommended) Use Docker to upload the CSI image to the image repository.
• Manually import the CSI image to all nodes where Huawei CSI needs to be deployed.

Uploading an Image to the Image Repository

The installation of Huawei CSI depends on the following image files provided by Huawei. Import and upload the image files in sequence. For details about how to obtain the image files, see Downloading the Huawei CSI Software Package.

• huawei-csi-v4.5.0-arch.tar
• storage-backend-controller-v4.5.0-arch.tar
• storage-backend-sidecar-v4.5.0-arch.tar
• huawei-csi-extender-v4.5.0-arch.tar

Prerequisites

A Linux host with Docker installed is available, and the host can access the image repository.

Procedure

1. Run the docker load -i huawei-csi-v4.5.0-arch.tar command to import the CSI image to the current node.

   docker load -i huawei-csi-v4.5.0-arch.tar 
   

2. Run the docker tag huawei-csi:4.5.0 repo.huawei.com/huawei-csi:4.5.0 command to add the image repository address to the image tag. repo.huawei.com indicates the image repository address.

   docker tag huawei-csi:4.5.0 repo.huawei.com/huawei-csi:4.5.0
   

3. Run the docker push repo.huawei.com/huawei-csi:4.5.0 command to upload the CSI image to the image repository. repo.huawei.com indicates the image repository address.

   docker push repo.huawei.com/huawei-csi:4.5.0
   

• You can also use containerd to import and upload the images.
• For details about how to import and upload images to the CCE or CCE Agile platform, see the user manual of the platform.

Uploading an Image to a Local Node

If the image has been uploaded to the image repository, skip this section.

Prerequisites

• The node has the corresponding Huawei CSI image file. For details about how to obtain the image file, see Downloading the Huawei CSI Software Package.
• Docker or another container engine has been installed on the node.

Procedure

1. Use a remote access tool, such as PuTTY, to log in to the node where the image is to be imported through the management IP address.

2. Copy the image directory in the Kubernetes CSI component package to any directory on the current node.

3. Run the cd image command to go to the image working directory. For details about the tool path, see Table 1.

4. Run the following commands in sequence to import all Huawei CSI images in the image directory to the local node. In the commands, name indicates the name of a .tar image package.

   Run the following command using the Docker container engine:

   docker load -i <name>.tar
   

   Run the following command using the containerd container engine:

   ctr -n k8s.io image import <name>.tar
   

   Run the following command using the Podman container engine:

   podman load -i <name>.tar
   

   If another container engine is installed on the node, use the image import command for the corresponding container engine.

5.1.3 - Checking User Configurations on Huawei Storage

After Huawei storage is connected to the container platform, Huawei CSI needs to manage storage resources on Huawei storage based on service requirements, such as creating and mapping volumes. In this case, Huawei CSI needs to use the users created on Huawei storage to communicate with Huawei storage. The following table lists the user information required for different storage devices.

Table 1 User requirements for connecting storage to CSI

• Note 1: If a user-defined role is used, you need to configure permissions for the role. For details about how to configure the minimum permissions, see Configuring Custom Permissions.

You are advised not to use the users of the super administrator role.

5.1.4 - Checking Volume Snapshot-Dependent Components

This section describes how to check the volume snapshot-dependent components in the cluster.

Kubernetes earlier than v1.17.0 does not support the snapshot function. If the snapshot CRD is deployed, the cluster may be faulty. Therefore, if Huawei CSI is deployed on Kubernetes earlier than v1.17.0, perform the check according to Kubernetes Earlier Than v1.17.0.

Kubernetes Earlier Than v1.17.0

If the Kubernetes version is earlier than v1.17.0, the cluster may be faulty during snapshot deployment. Perform the following steps to delete the snapshot CRD installation file.

1. Run the following command to check the Kubernetes version. In the following example, the Kubernetes version is v1.16.0.

   kubectl get node
   

   The following is an example of the command output.

   NAME          STATUS       ROLES     AGE    VERSION
   test-master   Ready        master    311d   v1.16.0
   test-node     Ready        <none>    311d   v1.16.0
   

2. Go to the /helm/esdk/crds/snapshot-crds directory and run the following command to delete the snapshot CRD installation file. For details about the component package path, see Table 1.

   rm -rf ./huawei-csi-snapshot-crd-v1.yaml
   

5.1.5 - Checking the Host Multipathing Configuration

If you plan to use the FC/iSCSI/NVMe over RoCE/NVMe over FC protocol to access Huawei storage in a container environment, you are advised to use host multipathing software to enhance the link redundancy and performance of the host and storage. If you do not want to use the software, skip this section.

For details about the OSs and multipathing software supported by Huawei CSI, see Table 2.

• If you want to use the FC/iSCSI protocol to connect to Huawei storage, you are advised to use native DM-Multipath provided by the OS.
• If you want to use the NVMe over RoCE/NVMe over FC protocol to connect to Huawei storage, you are advised to use Huawei-developed UltraPath-NVMe.
• If you want to use the SCSI protocol to connect to Huawei storage, disable DM-Multipath provided by the OS.

Prerequisites

Multipathing software has been correctly installed on a host.

• If you use native DM-Multipath provided by the OS, contact your host or OS provider to obtain the documents and software packages required for the installation.
• If you use Huawei-developed UltraPath or UltraPath-NVMe, contact Huawei engineers to obtain the UltraPath or UltraPath-NVMe documents and software packages. For details about the software package versions, see Table 1.

Procedure

1. If you use the iSCSI/FC protocol to connect to Huawei enterprise storage, configure and check host multipathing by referring to OceanStor Dorado and OceanStor Host Connectivity Guide for Red Hat.

2. If you use the NVMe over RoCE/NVMe over FC protocol to connect to Huawei enterprise storage, configure and check host multipathing by referring to OceanStor Dorado and OceanStor Host Connectivity Guide for Red Hat.

3. If you use iSCSI to connect to Huawei distributed storage, configure and check host multipathing by referring to Configuring Multipathing for an Application Server in FusionStorage 8.0.1 Block Storage Basic Service Configuration Guide.

4. If you use the native multipathing software provided by the OS, check whether the /etc/multipath.conf file contains the following configuration item.

   defaults {
           user_friendly_names yes
           find_multipaths no
   }
   

   If the configuration item does not exist, add it to the beginning of the /etc/multipath.conf file.

   For details about the functions of the user_friendly_names and find_multipaths parameters, see dm_multipath/config_file_defaults.

5.1.6 - Checking the Status of Host-Dependent Software

This section describes how to check whether the status of host-dependent software on worker nodes in a cluster is normal. In this example, the host OS is CentOS 7.9 x86_64.

• Check the status of the iSCSI client.

  systemctl status iscsi iscsid 
  

• Check the status of the NFS client.

  systemctl status rpcbind 
  

• Check the status of DM-Multipath.

  systemctl status multipathd.socket multipathd
  

• Check the status of UltraPath.

  systemctl status nxup
  

• Check the status of UltraPath-NVMe.

  systemctl status upudev upService_plus
  

5.1.7 - Checking the Images on Which CSI Depends

The installation of Huawei CSI depends on the images listed in the following table. If all worker nodes in the cluster have been connected to the Internet and can pull images online, you can skip this section. If nodes in the cluster cannot connect to the Internet, download the corresponding image file based on the Kubernetes version and upload it to the image repository or import it to all worker nodes in the Kubernetes cluster.

The huawei-csi-controller service depends on the following sidecar images: livenessprobe, csi-provisioner, csi-attacher, csi-resizer, csi-snapshotter, snapshot-controller, storage-backend-controller, storage-backend-sidecar, huawei-csi-driver, and huawei-csi-extender. The huawei-csi-node service depends on the following sidecar images: livenessprobe, csi-node-driver-registrar, and huawei-csi-driver.

For details about the functions and details of each image, see the following table.

Table 1 Images on which Huawei CSI depends

If the cluster is not connected to the Internet, manually download the container images and upload them to the cluster. For details, see Downloading a Container Image.

5.2 - Installing Huawei CSI

This section describes how to install Huawei CSI.

In the current version, resource requests and limits are added to Huawei CSI. For details, see Huawei CSI Resource Management.

Prerequisites

• Operations described in Installation Preparations have been completed.
• All worker nodes of the cluster communicate properly with the service network of the storage device to be connected. In iSCSI scenarios, the ping command can be used to verify the connectivity.
• Software clients required by the corresponding protocol, such as iSCSI and NFS clients, have been installed on all worker nodes of the cluster.

5.2.1 - Installing Huawei CSI Using Helm

Helm Installation Description

This section describes how to install Huawei CSI using Helm 3.

• Huawei CSI can be installed as the root user or a non-root user. When installing Huawei CSI as a non-root user, ensure that the current user can access the API Server of the Kubernetes cluster. For details about how to configure access to the Kubernetes cluster as a non-root user, see Configuring Access to the Kubernetes Cluster as a Non-root User.
• Huawei CSI must be run as the root user.

Helm is a software package management tool in the Kubernetes ecosystem. Similar to Ubuntu APT, CentOS YUM, or Python pip, Helm manages Kubernetes application resources.

You can use Helm to package, distribute, install, upgrade, and roll back Kubernetes applications in a unified manner.

• For details about how to obtain and install Helm, see https://helm.sh/docs/intro/install/.
• For details about the mapping between Helm and Kubernetes versions, see https://helm.sh/docs/topics/version_skew/.

When installing huawei-csi-controller, Helm deploys the following components in the workloads of the Deployment type in the specified namespace:

• huawei-csi-driver: Huawei CSI driver.
• storage-backend-controller: Huawei backend management controller, used to manage storageBackendClaim resources.
• storage-backend-sidecar: used to manage storageBackendContent resources.
• Kubernetes External Provisioner: used to provide or delete volumes.
• Kubernetes External Attacher: used to attach or detach volumes.
• Kubernetes External Resizer: used to expand the capacity of volumes.
• Kubernetes External liveness-probe: used to determine the health status of a Pod.
• (Optional) huawei-csi-extender: Huawei CSI extender.
• (Optional) Kubernetes External Snapshotter: used to provide snapshot support (installed as CRD).
• (Optional) Kubernetes External Snapshot Controller: used to control volume snapshots.

When installing huawei-csi-node, Helm deploys the following components in the workloads of the DaemonSet type in the specified namespace:

• huawei-csi-driver: Huawei CSI driver.
• Kubernetes Node Registrar: used to process driver registration.
• liveness-probe: used to determine the health status of a Pod.

5.2.1.1 - Installing Huawei CSI on Kubernetes, OpenShift, and Tanzu

Installation Procedure

1. Use a remote access tool, such as PuTTY, to log in to any master node in the cluster through the management IP address.

2. Copy the helm directory in the Kubernetes CSI component package to any directory on the master node. For details about the Helm tool path, see Table 1.

3. Go to the helm/esdk working directory.

   cd helm/esdk
   

4. Prepare the values.yaml file. Huawei CSI provides the values.yaml template file in the helm/esdk directory of the software package. You can also modify parameters according to Parameters in the values.yaml File of Helm to customize Huawei CSI.

5. Perform the following configuration before the installation:

   • If the container platform is Kubernetes, skip this step.
   • If the container platform is OpenShift, perform the configuration in Installation and Configuration on the OpenShift Platform.
   • If the container platform is Tanzu, perform the configuration in Installation and Configuration on the Tanzu Platform.

6. Run the following command to update the storage backend CRD.

   kubectl apply -f ./crds/backend/
   

7. (Optional) Check snapshot-dependent components by following the instructions provided in Checking Volume Snapshot-Dependent Components. After confirming that the components are correct, run the following command to update the snapshot CRD. If controller.snapshot.enabled is set to false or the Kubernetes version is earlier than v1.17, you can skip this step. For details, see Table 2.

   kubectl apply -f ./crds/snapshot-crds/ --validate=false
   

8. Run the following command to install Huawei CSI. In the preceding command, helm-huawei-csi indicates the custom Helm chart name, ./ indicates that the Helm project in the current directory is used, and huawei-csi indicates the custom Helm chart namespace.

   helm install helm-huawei-csi ./ -n huawei-csi --create-namespace
   

   The following is an example of the command output.

   NAME: helm-huawei-csi
   LAST DEPLOYED: Wed Jun  8 11:50:28 2022
   NAMESPACE: huawei-csi
   STATUS: deployed
   REVISION: 1
   TEST SUITE: None
   

9. After the huawei-csi service is deployed, run the following command to check whether the service is started.

   kubectl get pod -n huawei-csi
   

   The following is an example of the command output. If the Pod status is Running, the installation is successful.

   NAME                                     READY   STATUS    RESTARTS   AGE
   huawei-csi-controller-6dfcc4b79f-9vjtq   9/9     Running   0          24m
   huawei-csi-controller-6dfcc4b79f-csphc   9/9     Running   0          24m
   huawei-csi-node-g6f4k                    3/3     Running   0          20m
   huawei-csi-node-tqs87                    3/3     Running   0          20m
   

Installation and Configuration on the OpenShift Platform

For the OpenShift platform, run the following commands to create the SecurityContextConstraints resource.

1. Run the following command to edit the helm_scc.yaml file.

   vi helm_scc.yaml
   

2. Modify the helm_scc.yaml file. In the following command output, huawei-csi indicates the created namespace. Replace it based on the actual situation.

   apiVersion: security.openshift.io/v1
   kind: SecurityContextConstraints
   metadata:
     name: helm-scc
   allowHostDirVolumePlugin: true
   allowHostIPC: true
   allowHostNetwork: true
   allowHostPID: true
   allowHostPorts: true
   allowPrivilegeEscalation: true
   allowPrivilegedContainer: true
   
   defaultAddCapabilities:
   - SYS_ADMIN
   runAsUser:
     type: RunAsAny
   seLinuxContext:
     type: RunAsAny
   fsGroup:
     type: RunAsAny
   users:
   - system:serviceaccount:huawei-csi:huawei-csi-controller
   - system:serviceaccount:huawei-csi:huawei-csi-node
   

3. Run the following command to create a SecurityContextConstraints file.

   oc create -f helm_scc.yaml
   

Installation and Configuration on the Tanzu Platform

On the Tanzu platform, run the following command to configure the kubelet installation directory.

1. Go to the helm/esdk directory in the installation package, run the following command to open the configuration file, modify the file, and save the file. For details about the installation package directory, see Table 1.

   vi values.yaml
   

2. Modify the kubeletConfigDir parameter as follows:

   # Specify kubelet config dir path.
   # kubernetes and openshift is usually /var/lib/kubelet
   # Tanzu is usually /var/vcap/data/kubelet
   # CCE is usually /mnt/paas/kubernetes/kubelet
   kubeletConfigDir: /var/vcap/data/kubelet
   

For TKGI 1.16 or earlier of the Tanzu platform, run the following commands to configure the RBAC permission.

1. Run the following command to create a file named rbac.yaml.

   vi rbac.yaml
   

2. Copy the following content to the rbac.yaml file, save the file, and exit.

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: huawei-csi-psp-role
   rules:
   - apiGroups: ['policy']
     resources: ['podsecuritypolicies']
     verbs: ['use']
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: huawei-csi-psp-role-cfg
   roleRef:
     kind: ClusterRole
     name: huawei-csi-psp-role
     apiGroup: rbac.authorization.k8s.io
   subjects:
   - kind: Group
     apiGroup: rbac.authorization.k8s.io
     name: system:serviceaccounts:huawei-csi
   - kind: Group
     apiGroup: rbac.authorization.k8s.io
     name: system:serviceaccounts:default
   

3. Run the following command to create the RBAC permission.

   kubectl create -f rbac.yaml
   

5.2.1.2 - Installing Huawei CSI on the CCE or CCE Agile Platform

This section describes how to install Huawei CSI on the CCE or CCE Agile platform.

Creating a Helm Installation Package

The CCE or CCE Agile platform cannot directly install Huawei CSI using Helm. You need to manually create a Helm installation package and upload it to the chart list on the platform for installation.

1. Use a remote access tool, such as PuTTY, to log in to any node where Helm is deployed through the management IP address.

2. Copy the helm directory in the Huawei CSI component package to any directory on the node. For details about the Helm tool path, see Table 1.

3. Go to the helm working directory.

   cd helm/
   

4. Modify the kubeletConfigDir and csiDriver.driverName parameters in the helm/esdk/values.yaml file.

   vi ./esdk/values.yaml
   

   Modify the following parameters:

   # Specify kubelet config dir path.
   # kubernetes and openshift is usually /var/lib/kubelet
   # Tanzu is usually /var/vcap/data/kubelet
   # CCE is usually /mnt/paas/kubernetes/kubelet
   kubeletConfigDir: /mnt/paas/kubernetes/kubelet
   
   # The CSI driver parameter configuration
   csiDriver:
     # Driver name, it is strongly recommended not to modify this parameter
     # The CCE platform needs to modify this parameter, e.g. csi.oceanstor.com
     driverName: csi.oceanstor.com
   

5. Run the following command to create a Helm installation package. This command will generate the installation package to the current path.

   helm package ./esdk/ -d ./
   

Installing Huawei CSI

1. Use a remote access tool, such as PuTTY, to log in to any master node where the CCE Agile platform is deployed through the management IP address.

2. Run the following command to create a namespace for deploying Huawei CSI. huawei-csi indicates the custom namespace.

   kubectl create namespace huawei-csi
   

3. Export the Helm installation package. For details, see Creating a Helm Installation Package.

4. On the home page, choose Charts > My Charts > Upload Chart. The Upload Chart dialog box is displayed. Import the exported Helm installation package to the CCE Agile platform.

   

5. After the installation package is uploaded, choose Charts > My Charts. On the My Charts page that is displayed, choose Install > Submit. The chart release name can be customized.

   

6. On the home page, choose Charts > Releases and select the project specified during installation (for example, default in the following figure). After the installation is successful, Installed is displayed in the Status column.

   

5.2.1.3 - Parameters in the values.yaml File of Helm

When using Helm to install CSI, you need to prepare the values.yaml file of the Helm project based on the features required during deployment. Huawei CSI provides the values.yaml template file in the helm/esdk directory of the software package.

This section describes the configuration items in the values.yaml file and backend configuration examples in typical scenarios.

images Parameters

The images parameters in the values.yaml file are used to configure the component image information on which Huawei CSI depends during running. Set the following parameters:

Table 1 images parameters

• For details about the values of huaweiCSIService, storageBackendSidecar, storageBackendController, and huaweiCSIExtender, see Uploading a Huawei CSI Image. Use the name and version of the finally generated image.
• For details about other sidecar image parameters, see Checking the Images on Which CSI Depends. Use the name and version of the finally uploaded image.

controller Parameters

The controller parameters are used to configure the huawei-csi-controller component.

Table 2 controller parameters

If controller.snapshot.enabled is set to true, you need to install the volume snapshot CRD resource in the helm/crd/snapshot-crds directory.

node Parameters

The node parameters are used to configure the huawei-csi-node component.

Table 3 node parameters

csiDriver Parameters

The csiDriver parameters include the basic configurations for running Huawei CSI, such as Huawei driver name and multipathing type.

Table 4 csiDriver parameters

If Huawei CSI has been deployed in your container environment, ensure that the value of csiDriver.driverName is the same as that configured during previous deployment. Otherwise, existing volumes or snapshots provisioned by Huawei CSI in the system cannot be managed by the newly deployed Huawei CSI.

Other Parameters

Other parameters include some features of the CSI plug-in or the policies for obtaining images.

Table 5 Other parameters

Ensure that the namespace entered in kubernetes.namespace exists on Kubernetes. If the namespace does not exist, run the following command to create it. In this example, the namespace for running Huawei CSI is huawei-csi.

kubectl create namespace huawei-csi

5.2.2 - Manually Installing Huawei CSI

This section describes how to manually install Huawei CSI.

Currently, only the Kubernetes platform supports manual installation of Huawei CSI.

Procedure

1. Use a remote access tool, such as PuTTY, to log in to any master node in the cluster through the management IP address.

2. Copy the manual directory in the Kubernetes CSI component package to any directory on the master node.

3. Run the following command to create a namespace.

   kubectl create ns huawei-csi
   

4. Go to the manual/esdk working directory. For details about the path, see Table 1.

   cd manual/esdk
   

5. Run the following command to update the storage backend CRD.

   kubectl apply -f ./crds/backend/
   

6. (Optional) Check snapshot-dependent components by following the instructions provided in Checking Volume Snapshot-Dependent Components. After confirming that the components are correct, run the following command to update the snapshot CRD. If the Kubernetes version is earlier than v1.17, skip this step.

   kubectl apply -f ./crds/snapshot-crds/ --validate=false
   

7. (Optional) Run the following command to install CSIDriver. If the CSIDriver feature is not used, you can skip this step. For details, see the CSIDriver feature.

   kubectl apply -f ./deploy/csidriver.yaml 
   

8. Run the following command to install the huawei-csi-controller service.

   If the Kubernetes version is earlier than v1.17, modify the ./deploy/huawei-csi-controller.yaml file as follows:

   • If the Kubernetes version is earlier than v1.17, the snapshot feature is not supported. In this case, delete the snapshot-related container configurations items csi-snapshotter and snapshot-controller.
   • If the Kubernetes version is earlier than v1.17, the csi-provisioner sidecar image provided by the Kubernetes community does not support the –leader-election parameter. Therefore, the leader-election parameter of the csi-provisioner container is deleted and only single-copy deployment is supported.
   • Modify the dependent image version based on the version requirements in Checking the Images on Which CSI Depends.

   kubectl apply -f ./deploy/huawei-csi-controller.yaml
   

9. Run the following command to install the huawei-csi-node service.

   kubectl apply -f ./deploy/huawei-csi-node.yaml 
   

10. Run the following command to check whether the services are started.

    kubectl get pod -n huawei-csi
    

    The following is an example of the command output. If the Pod status is Running, the installation is successful.

    NAME                                     READY   STATUS    RESTARTS   AGE
    huawei-csi-controller-68745d489c-v5xkj   9/9     Running   0          13m
    huawei-csi-node-4hbqp                    3/3     Running   0          13m
    huawei-csi-node-f7dkf                    3/3     Running   0          13m
    huawei-csi-node-xrntc                    3/3     Running   0          13m
    

In the multi-copy controller deployment scenario, you can modify the spec.replica field of the Deployment resource in the ./deploy/huawei-csi-controller.yaml file to specify the number of copies. After the modification, run the following command for the modification to take effect.

kubectl apply -f ./deploy/huawei-csi-controller.yaml

5.3 - Uninstalling Huawei CSI

This chapter describes how to uninstall Huawei CSI. The uninstallation method varies according to the installation mode.

If you do not uninstall Huawei CSI for the purpose of an upgrade, ensure that all resources (such as PV, PVC, snapshot, and storage backend resources) provisioned by Huawei CSI have been cleared on your container platform before uninstalling Huawei CSI. Otherwise, once you uninstall Huawei CSI, these resources cannot be automatically scheduled, managed, or cleared.

5.3.1 - Uninstalling Huawei CSI Using Helm

5.3.1.1 - Uninstalling Huawei CSI on Kubernetes, OpenShift, and Tanzu

This section describes how to uninstall Huawei CSI on the Kubernetes, OpenShift, and Tanzu platforms.

Procedure

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Run the following command to uninstall Huawei CSI. In the command, helm-huawei-csi indicates the custom Helm chart name and huawei-csi indicates the namespace where the Helm chart resides. This command will uninstall the huawei-csi-controller, huawei-csi-node, and RBAC resources of Huawei CSI.

   helm uninstall helm-huawei-csi -n huawei-csi
   

   After the uninstallation command is executed, you need to check whether the uninstallation is successful. In the preceding command, huawei-csi indicates the namespace where the chart is located.

   helm list -n huawei-csi
   

   The following is an example of the command output. If the command output is empty, the service is successfully uninstalled.

   NAME    NAMESPACE       REVISION        UPDATED STATUS  CHART   APP VERSION
   

3. Uninstall the huawei-csi-host-info object. For details, see Uninstalling the huawei-csi-host-info Object.

4. Uninstall the webhook resource. For details, see Uninstalling a Webhook Resource.

5. (Optional) Uninstall the snapshot-dependent component service. For details, see Uninstalling the Snapshot-Dependent Component Service.

6. (Optional) Uninstall the Lease resource. For details, see Uninstalling a Lease Resource.

5.3.1.2 - Uninstalling Huawei CSI on CCE or CCE Agile

This section describes how to uninstall Huawei CSI on the CCE or CCE Agile platform. The following uses CCE Agile v22.3.2 as an example.

Procedure

1. Log in to the CCE Agile platform.

2. On the home page, choose Charts > Releases. The Releases page is displayed.

3. Select a Huawei CSI release and click Uninstall. In the displayed dialog box, click OK.

   

4. Uninstall the huawei-csi-host-info object. For details, see Uninstalling the huawei-csi-host-info Object.

5. Uninstall the webhook resource. For details, see Uninstalling a Webhook Resource.

6. (Optional) Uninstall the snapshot-dependent component service. For details, see Uninstalling the Snapshot-Dependent Component Service.

5.3.1.3 - Uninstalling CSI-Dependent Component Services

This section describes how to uninstall the CSI-dependent component services.

Uninstalling the huawei-csi-host-info Object

Secret object huawei-csi-host-info stores the initiator information about each node in the cluster, for example, iSCSI initiators. When you run the helm uninstall command, the resource will not be uninstalled. To uninstall the resource, perform the following steps:

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Run the following command to delete the Secret object. huawei-csi-host-info is the name of the Secret object, and huawei-csi is the namespace where the Secret object is located.

   kubectl delete secret huawei-csi-host-info -n huawei-csi
   

3. Run the following command to check whether the Secret object is successfully uninstalled.

   kubectl get secret huawei-csi-host-info -n huawei-csi 
   

   The following is an example of the command output. If NotFound is displayed in the command output, the huawei-csi-host-info object is successfully uninstalled.

   Error from server (NotFound): secrets "huawei-csi-host-info" not found
   

Uninstalling a Webhook Resource

The webhook resource named storage-backend-controller.xuanwu.huawei.io is used to verify the backend key information and connectivity with the storage. When you run the helm uninstall command, the resource will not be uninstalled. To uninstall the resource, perform the following steps:

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Run the following command to query the webhook-dependent component service.

   kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io storage-backend-controller.xuanwu.huawei.io
   

   The following is an example of the command output.

   NAME                                          WEBHOOKS   AGE
   storage-backend-controller.xuanwu.huawei.io   1          12d
   

3. Run the following command to uninstall the webhook-dependent component service.

   kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io storage-backend-controller.xuanwu.huawei.io
   

4. Run the following command to check whether the service is successfully uninstalled. If the command output is empty, the uninstallation is successful.

   kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io storage-backend-controller.xuanwu.huawei.io
   

Uninstalling the Snapshot-Dependent Component Service

• Do not uninstall the snapshot-dependent component service when snapshots exist. Otherwise, Kubernetes will automatically delete all user snapshots and they cannot be restored. Exercise caution when performing this operation. For details, see Delete a CustomResourceDefinition.
• Do not uninstall the snapshot-dependent component service during the CSI upgrade.

Scenario Description

• Currently, Huawei CSI uses the snapshot feature.
• Currently, only Huawei CSI is available in the Kubernetes cluster, and Huawei CSI is no longer used.
• Before the uninstallation, ensure that no VolumeSnapshot resource managed by Huawei CSI exists in the Kubernetes cluster.

Procedure

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Run the following command to uninstall the snapshot-dependent component service.

   kubectl delete crd volumesnapshotclasses.snapshot.storage.k8s.io volumesnapshotcontents.snapshot.storage.k8s.io volumesnapshots.snapshot.storage.k8s.io
   

3. Run the following command to check whether the service is successfully uninstalled. If the command output is empty, the uninstallation is successful.

   kubectl get crd | grep snapshot.storage.k8s.io
   

Uninstalling a Lease Resource

If the value of the controller.controllerCount configuration item in the values.yaml file is greater than 1, huawei-csi-controller will be deployed in multi-copy mode. The multiple copies of huawei-csi-controller are implemented using the LeaderElection mechanism of Kubernetes. This mechanism creates a Lease object to store the current Holder information. When you run the helm uninstall command, the resource will not be uninstalled. To uninstall the resource, perform the following steps. If the value of controller.controllerCount is 1, you can skip the following steps. For details about the configuration item, see Table 2.

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Run the following command to query the Lease information.

   kubectl get lease -n huawei-csi
   

   The following is an example of the command output.

   NAME                                         HOLDER    AGE
   csi-huawei-com                               node-1    24d
   external-attacher-leader-csi-huawei-com      node-1    24d
   external-resizer-csi-huawei-com              node-1    24d
   external-snapshotter-leader-csi-huawei-com   node-1    24d
   snapshot-controller-leader                   node-1    24d
   storage-backend-controller                   node-1    24d
   huawei-csi-extender                          node-1    24d
   

3. Run the following command to uninstall the Lease resource.

   kubectl delete lease -n huawei-csi csi-huawei-com external-attacher-leader-csi-huawei-com external-resizer-csi-huawei-com external-snapshotter-leader-csi-
   

4. Run the following command to check whether the uninstallation is successful.

   kubectl get lease -n huawei-csi
   

   The following is an example of the command output. If the command output is empty, the uninstallation is successful.

   No resources found in huawei-csi namespace.
   

5.3.2 - Manually Uninstalling Huawei CSI

This section describes how to manually uninstall Huawei CSI.

If you do not uninstall Huawei CSI for the purpose of an upgrade, ensure that all resources (such as PV, PVC, snapshot, and storage backend resources) provisioned by Huawei CSI have been cleared on your container platform before uninstalling Huawei CSI. Otherwise, once you uninstall Huawei CSI, these resources cannot be automatically scheduled, managed, or cleared.

Uninstalling the huawei-csi-node Service

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Run the following command to uninstall the huawei-csi-node service. Replace huawei-csi with the namespace where Huawei CSI is located.

   kubectl delete daemonset huawei-csi-node -n huawei-csi
   

3. Run the following command to check whether the service is successfully uninstalled. If NotFound is displayed, the service is successfully uninstalled.

   kubectl get daemonset huawei-csi-node -n huawei-csi
   

Uninstalling the huawei-csi-controller Service

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Run the following command to uninstall the huawei-csi-controller service. Replace huawei-csi with the namespace where Huawei CSI is located.

   kubectl delete deployment huawei-csi-controller -n huawei-csi
   

3. Run the following command to check whether the service is successfully uninstalled. If NotFound is displayed, the service is successfully uninstalled.

   kubectl get deployment huawei-csi-controller -n huawei-csi
   

Uninstalling the csidriver Object

If the CSIDriver feature is not used during installation, you can skip the following steps.

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Run the following command to uninstall the csidriver object.

   kubectl delete csidriver csi.huawei.com
   

3. Run the following command to check whether the service is successfully uninstalled. If NotFound is displayed, the service is successfully uninstalled.

   kubectl get csidriver csi.huawei.com
   

Deleting the RBAC Permission

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Delete the RBAC permission.

   kubectl -n huawei-csi -l provisioner=csi.huawei.com delete ServiceAccount,Service,role,rolebinding,ClusterRole,ClusterRoleBinding
   

Uninstalling Other Resources

1. Uninstall the huawei-csi-host-info object. For details, see Uninstalling the huawei-csi-host-info Object.
2. Uninstall the webhook resource. For details, see Uninstalling a Webhook Resource.
3. (Optional) Uninstall the snapshot-dependent component service. For details, see Uninstalling the Snapshot-Dependent Component Service.
4. (Optional) Uninstall the Lease resource. For details, see Uninstalling a Lease Resource.

5.4 - Upgrading or Rolling Back Huawei CSI

This section describes how to upgrade or roll back Huawei CSI.

In the current version, resource requests and limits are added to Huawei CSI. For details, see Huawei CSI Resource Management.

5.4.1 - Upgrading or Rolling Back Huawei CSI Using Helm

To upgrade Huawei CSI from 2.x to 4.5.0, uninstall it by referring to the user guide of the earlier version and install Huawei CSI by referring to Installing Huawei CSI Using Helm.

To upgrade Huawei CSI from 2.x or 3.x to 4.5.0, see Upgrading from 2.x or 3.x to 4.x.

To upgrade Huawei CSI from 4.x to 4.5.0, see Upgrading Huawei CSI on Kubernetes, OpenShift, and Tanzu.

5.4.1.1 - Rolling Back Huawei CSI on Kubernetes, OpenShift, and Tanzu

Prerequisites

• CSI has been updated using Helm 3.

Procedure

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Go to the helm/esdk working directory. For the directory path, see Table 1.

   cd helm/esdk
   

3. Run the following command to query the historical versions of the CSI services deployed using Helm.

   helm history helm-huawei-csi -n huawei-csi 
   

   The following is an example of the command output.

   REVISION        UPDATED                         STATUS          CHART           APP VERSION     DESCRIPTION     
   1       	Mon Jan  8 04:15:40 2024	superseded	esdk-4.4.0	4.4.0      	Install complete
   2       	Mon Jan  8 04:16:12 2024	deployed  	esdk-4.5.0	4.5.0      	Upgrade complete
   

4. Run the following command to roll back the CSI services to the specified version.

   In the preceding command, revision-number indicates a version number queried in 3. For example, the version is 1.

   helm rollback helm-huawei-csi -n huawei-csi 1
   

   The following is an example of the command output. If Rollback was a success is displayed in the command output, the CSI services are successfully rolled back to the specified version.

   Rollback was a success! Happy Helming!
   

5.4.1.2 - Upgrading Huawei CSI

This section describes how to upgrade Huawei CSI.

During the upgrade or rollback, the existing resources such as PVCs, snapshots, and Pods will run properly and will not affect your service access.

• Some CSI 2.x versions are unavailable now. If the upgrade fails, CSI may fail to be rolled back to a version which is unavailable now.
• After an upgrade from 2.x, 3.x, or 4.x to 4.5.0, a Pod that has been provisioned in the source version may fail to be mounted again. For details, see Upgrading from 2.x or 3.x to 4.x.
• During the upgrade or rollback, you cannot use Huawei CSI to create new resources or mount or unmount an existing PVC.
• During the upgrade or rollback, do not uninstall the snapshot-dependent component service.

5.4.1.2.1 - Upgrading from 2.x or 3.x to 4.x

In CSI 2.x or 3.x, when block storage is used, the mapping with storage is set up in the huawei-csi-node service. Therefore, the huawei-csi-node service needs to communicate with the storage management network. Because the huawei-csi-node service is deployed as a DaemonSet, the huawei-csi-node service is deployed on each node in the cluster. As a result, in a large-scale cluster, each huawei-csi-node service sends requests to the storage and the number of storage connections may be fully occupied. Accordingly, huawei-csi-node cannot provide services properly. In CSI 4.x, the deployment model is optimized. The setup of the mapping with storage is migrated to the huawei-csi-controller service and the huawei-csi-node service does not need to communicate with the storage management network. This reduces the networking complexity of Huawei CSI. In addition, the huawei-csi-controller service is deployed as a Deployment. The number of copies is set based on the customer’s reliability requirements. Generally, the number of copies ranges from 1 to 3. Therefore, the number of connections between Huawei CSI and storage is greatly reduced, so that Huawei CSI can connect to a large-scale cluster. This change may cause a problem. That is, if a new mount process is generated after CSI is upgraded to 4.x but with workloads provisioned using 2.x or 3.x and the Container Orchestration (CO) system does not invoke the huawei-csi-controller service provided by Huawei CSI, the mounting will fail. For details, see A Pod Fails to Be Created and Message “publishInfo doesn’t exist” Is Displayed in the Events Log.

Backing Up Storage Backend Configurations

If you have evaluated the risks mentioned in the preceding notice and need to upgrade CSI from 2.x or 3.x to 4.5.0, perform the following steps to back up storage backend configurations:

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Run the following command to back up the backend information to the configmap.json file. For the OpenShift platform, replace kubectl with oc.

   kubectl get cm huawei-csi-configmap -n huawei-csi -o json > configmap.json
   

Upgrading Huawei CSI

Perform the upgrade according to the procedure described in Upgrading Huawei CSI.

Configuring the Storage Backend

Configure the storage backend by following the instructions in Managing Storage Backends according to the backend information backed up in Backing Up Storage Backend Configurations. After the storage backend is successfully configured, perform operations according to the risk handling methods described in the preceding notice to prevent problems during Pod failover.

5.4.1.2.2 - Upgrading Huawei CSI on Kubernetes, OpenShift, and Tanzu

Prerequisites

• Huawei CSI of an earlier version is installed using Helm.
• A Huawei CSI image of a new version has been created and uploaded to the image repository or imported to all nodes by following the instructions provided in Uploading a Huawei CSI Image.

Upgrading Huawei CSI

If CSI of an earlier version is deployed using Helm, perform the following steps to upgrade Huawei CSI.

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Copy the CSI component package of the target version to any directory on the master node.

3. Go to the helm/esdk working directory. For the directory path, see Table 1.

   cd helm/esdk
   

4. Run the kubectl apply -f ./crds/backend/ command to update the storage backend CRD.

   kubectl apply -f ./crds/backend/
   

5. (Optional) Check snapshot-dependent components by following the instructions provided in Checking Volume Snapshot-Dependent Components. After confirming that the components are correct, run the kubectl apply -f ./crds/snapshot-crds/ –validate=false command to update the snapshot CRD. If controller.snapshot.enabled is set to false or the Kubernetes version is earlier than v1.17, you can skip this step. For details, see Table 2.

   kubectl apply -f ./crds/snapshot-crds/ --validate=false
   

6. Run the following command to obtain the original service configuration file. helm-huawei-csi indicates the Helm chart name specified during the installation of the earlier version, and huawei-csi indicates the Helm chart namespace specified during the installation of the earlier version.

   helm get values helm-huawei-csi -n huawei-csi -a > ./update-values.yaml
   

7. Run the vi update-values.yaml command to open the file obtained in 6, modify the images configuration items, and update the image to the latest version. For details about the parameters to be modified, see Table 1.

   Table 1 images configuration items

   Parameter	Description	New Value
   images.huaweiCSIService	huawei-csi image.	huawei-csi:4.5.0
   images.storageBackendSidecar	Image used by Huawei backends to manage storageBackendContent resources.	storage-backend-sidecar:4.5.0
   images.storageBackendController	Image used by Huawei backends to manage storageBackendClaim resources.	storage-backend-controller:4.5.0
   images.huaweiCSIExtender	huawei-csi-extender image.	huawei-csi-extender:4.5.0
   images.sidecar.livenessProbe	livenessprobe sidecar image.	k8s.gcr.io/sig-storage/livenessprobe:v2.5.0
   images.sidecar.provisioner	csi-provisioner sidecar image.	k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0
   images.sidecar.attacher	csi-attacher sidecar image.	k8s.gcr.io/sig-storage/csi-attacher:v3.4.0
   images.sidecar.resizer	csi-resizer sidecar image.	k8s.gcr.io/sig-storage/csi-resizer:v1.4.0
   images.sidecar.snapshotter	csi-snapshotter sidecar image.	k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1
   images.sidecar.snapshotController	snapshot-controller sidecar image.	k8s.gcr.io/sig-storage/snapshot-controller:v4.2.1
   images.sidecar.registrar	csi-node-driver-registrar sidecar image.	k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0

8. (Optional) If you need to update configuration items or add configuration information during the upgrade, modify the configuration information in the update-values.yaml file by referring to Parameters in the values.yaml File of Helm.

   During the upgrade, if the update-values.yaml and values.yaml configuration files contain the same configuration item, the configuration in the update-values.yaml file takes effect preferentially.

9. Run the following command to upgrade Huawei CSI. In the following command, helm-huawei-csi indicates the specified Helm chart name, huawei-csi indicates the specified Helm chart namespace, and update-values.yaml indicates the file obtained in 6.

   helm upgrade helm-huawei-csi ./ -n huawei-csi -f ./values.yaml -f ./update-values.yaml
   

10. After the huawei-csi service is deployed, run the following command to check whether the service is started.

    kubectl get pod -n huawei-csi
    

    The following is an example of the command output. If the Pod status is Running, the service is started successfully.

    NAME                                     READY   STATUS    RESTARTS   AGE
    huawei-csi-controller-6dfcc4b79f-9vjtq   9/9     Running   0          24m
    huawei-csi-controller-6dfcc4b79f-csphc   9/9     Running   0          24m
    huawei-csi-node-g6f4k                    3/3     Running   0          20m
    huawei-csi-node-tqs87                    3/3     Running   0          20m
    

5.4.1.2.3 - Upgrading Huawei CSI on CCE or CCE Agile

Prerequisites

You have downloaded the CSI software package of a new version.

Procedure

1. Uninstall CSI. For details, see Uninstalling Huawei CSI on CCE or CCE Agile.
2. Install CSI of the new version. For details, see Installing Huawei CSI on the CCE or CCE Agile Platform.

5.4.1.3 - Rolling Back Huawei CSI on CCE or CCE Agile

• During the upgrade or rollback, the existing resources such as PVCs, snapshots, and Pods will run properly and will not affect your service access.
• During the upgrade or rollback, you cannot use Huawei CSI to create new resources or mount or unmount an existing PVC.
• During the upgrade or rollback, do not uninstall the snapshot-dependent component service.

Prerequisites

You have downloaded the CSI software package of the source version.

Procedure

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.
2. Uninstall CSI. For details, see Procedure.
3. Reinstall CSI of the source version. For details, see Installing Huawei CSI on the CCE or CCE Agile Platform.

5.4.2 - Manual Upgrade/Rollback

5.4.2.1 - Upgrading Huawei CSI

This section describes how to manually upgrade Huawei CSI.

During the upgrade or rollback, the existing resources such as PVCs, snapshots, and Pods will run properly and will not affect your service access.

• Some CSI 2.x versions are unavailable now. If the upgrade fails, CSI may fail to be rolled back to a version which is unavailable now.
• During the upgrade or rollback, you cannot use Huawei CSI to create new resources or mount or unmount an existing PVC.
• During the upgrade or rollback, do not uninstall the snapshot-dependent component service.

Upgrading CSI from 2.x or 3.x to 4.5.0

To upgrade CSI from 2.x or 3.x to 4.5.0, perform the following operations:

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.

2. Run the following command to back up the backend information to the configmap.json file. For the OpenShift platform, replace kubectl with oc.

   kubectl get cm huawei-csi-configmap -n huawei-csi -o json > configmap.json
   

3. Uninstall CSI. For details, see Manually Uninstalling Huawei CSI.

4. Install CSI of the current version. For details, see Manually Installing Huawei CSI.

5. Install the backend information backed up in 2 according to Managing Storage Backends.

Upgrading CSI from 4.x to 4.5.0

To upgrade CSI from 4.x to 4.5.0, perform the following operations:

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.
2. Uninstall CSI. For details, see Manually Uninstalling Huawei CSI.
3. Install CSI of the current version. For details, see Manually Installing Huawei CSI.

5.4.2.2 - Rolling Back Huawei CSI

Uninstall CSI by referring to Manually Uninstalling Huawei CSI, and then download and install CSI of the source version.

• During the upgrade or rollback, the existing resources such as PVCs, snapshots, and Pods will run properly and will not affect your service access.
• During the upgrade or rollback, you cannot use Huawei CSI to create new resources or mount or unmount an existing PVC.
• During the upgrade or rollback, do not uninstall the snapshot-dependent component service.

Prerequisites

You have downloaded the CSI software package of the source version.

Procedure

1. Use a remote access tool, such as PuTTY, to log in to any master node in the Kubernetes cluster through the management IP address.
2. Uninstall CSI. For details, see Manually Uninstalling Huawei CSI.
3. Reinstall CSI of the source version. For details, see Manually Installing Huawei CSI.

6 - Storage Backend Management

Backend is an abstract concept of Huawei storage resources. Each Huawei storage device can abstract multiple backend resources using features such as tenants, storage pools, and protocols. Each backend exists independently and defines Huawei storage information required for providing persistent volumes for Kubernetes clusters.

This chapter describes how to use the oceanctl tool to manage storage backends, including creating, querying, updating, and deleting backends.

Description of the oceanctl Tool

• You have obtained the oceanctl tool, copied the oceanctl tool to the environment directory, for example, /usr/local/bin, and obtained the execute permission. The oceanctl tool is stored in /bin/oceanctl of the software package.
• The oceanctl tool depends on kubectl (for the Kubernetes platform) or oc (for the OpenShift platform) commands. Therefore, you need to run the tool on a node where kubectl or oc commands can be executed.
• By default, the user who runs oceanctl commands must have the read and write permissions on the /var/log directory. If you do not have the permissions on the directory, run the –log-dir=/path/to/custom command to specify a directory on which you have the permissions as the log file directory.
• huawei-csi is the default namespace used by oceanctl to create a backend.
• For details about oceanctl commands, see Description of oceanctl Commands.

6.1 - Managing Storage Backends

This section describes how to create a storage backend. Currently, you can create a backend based on the configured backend yaml file or the exported configmap.json file.

If you create a backend by adding a backend yaml file, configure the backend file by referring to Examples of Storage Backend Configuration Files in Typical Scenarios.

If the exported configmap.json file exists, create a storage backend by referring to Creating a Storage Backend.

6.1.1 - Creating a Storage Backend

1. When oceanctl is used to create a storage backend, the entered account and key information is stored in the Secret object. It is recommended that the customer container platform encrypt the Secret object based on the suggestions of the supplier or K8s community. For details about how to encrypt the Secret object in the K8s community, see Enable Encryption at Rest.
2. When a backend is created using a .json file, the backend name of an earlier version may contain uppercase letters or underscores (_). In this case, the old name is remapped to a new name. The mapping process automatically occurs and does not affect the original functions. For example, ABC_123 is mapped to abc-123-fd68e. The mapping rules are as follows:

• Uppercase letters are converted to lowercase letters.
• An underscore (_) is converted to a hyphen (-).
• A 5-digit hash code is added to the end.

3. If a storage backend is connected to a vStore, the vStore name cannot be changed after the storage backend is created.

Procedure

1. Prepare the backend configuration file, for example, backend.yaml. For details, see Examples of Storage Backend Configuration Files in Typical Scenarios. To create multiple backends, separate them with —.

   storage: "oceanstor-san"
   name: "backend-1"
   namespace: "huawei-csi"
   urls:
     - "https://192.168.129.157:8088"
   pools:
     - "StoragePool001"
   parameters:
     protocol: "roce"
     portals:
       - "10.10.30.20"
       - "10.10.30.21"
   maxClientThreads: "30"
   ---
   storage: "oceanstor-san"
   name: "backend-2"
   namespace: "huawei-csi"
   urls:
     - "https://192.168.129.158:8088"
   pools:
     - "StoragePool001"
   parameters:
     protocol: "roce"
     portals:
       - "10.10.30.20"
       - "10.10.30.21"
   maxClientThreads: "30"
   

2. Run the following command to create a storage backend.

   oceanctl create backend -f /path/to/backend.yaml -i yaml
   

   The following is an example of the command output.

   NUMBER  CONFIGURED    NAME        STORAGE              URLS                
   1       false         backend-1   oceanstor-san        https://192.168.129.157:8088 
   2       false         backend-2   oceanstor-san        https://192.168.129.158:8088 
   Please enter the backend number to configure (Enter 'exit' to exit):
   

3. Enter the serial number of the backend to be created and enter the account and password.

   Please enter the backend number to configure (Enter 'exit' to exit):1
   Please enter this backend user name:admin
   Please enter this backend password:
   
   Backend backend-1 is configured
   NUMBER  CONFIGURED    NAME         STORAGE              URLS               
   1       true          backend-1    oceanstor-san        https://192.168.129.157:8088 
   2       false         backend-2    oceanstor-san        https://192.168.129.158:8088 
   Please enter the backend number to configure (Enter 'exit' to exit):
   

4. Check the storage backend creation result.

   oceanctl get backend
   

   The following is an example of the command output. If the backend status is Bound, the creation is successful.

   NAMESPACE     NAME         PROTOCOL    STORAGETYPE      SN                    STATUS  ONLINE  URL                 
   huawei-csi    backend-1    roce        oceanstor-san    xxxxxxxxxxxxxxxxxxxx  Bound   true    https://192.168.129.157:8088   
   huawei-csi    backend-2    roce        oceanstor-san    xxxxxxxxxxxxxxxxxxxx  Bound   true    https://192.168.129.158:8088   
   

6.1.1.1 - Examples of Storage Backend Configuration Files in Typical Scenarios

For details about the backend configuration in typical scenarios, see the following examples. For details about the parameter configuration, see Storage Backend Parameters.

• Configuring a Storage Backend of the iSCSI Type
• Configuring a Storage Backend of the FC Type
• Configuring a Storage Backend of the NVMe over RoCE Type
• Configuring a Storage Backend of the NVMe over FC Type
• Configuring a Storage Backend of the NFS Type
• Configuring a Storage Backend of the SCSI Type
• Configuring a Storage Backend of the DPC Type
• Configuring Storage Backends of the Dtree Type
• Configuring Storage Backends of the HyperMetro Type

Configuring a Storage Backend of the iSCSI Type

If you want to use the iSCSI protocol, ensure that the iSCSI client has been installed on the host before installing Huawei CSI. You can check whether the client has been installed on the host by referring to Checking the Status of Host-Dependent Software. If the iSCSI client is not installed, restart the huawei-csi-node service after installing the iSCSI client. During the restart, do not use Huawei CSI to create new resources or mount or unmount an existing PVC. The following command is used as an example:

kubectl delete pods -n huawei-csi -l app=huawei-csi-node

The following is an example of the backend configuration file of the iSCSI type for enterprise storage:

storage: "oceanstor-san"
name: "dorado-iscsi-155"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.155:8088"
  - "https://192.168.129.156:8088"
pools:
  - "StoragePool001"
parameters:
  protocol: "iscsi"
  portals:
    - "192.168.128.120"                
    - "192.168.128.121"
maxClientThreads: "30"

The following is an example of the backend configuration file of the iSCSI type for distributed storage:

storage: "fusionstorage-san"
name: "pacific-iscsi-125"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.125:8088"
  - "https://192.168.129.126:8088"
pools:
  - "StoragePool001"
parameters:
  protocol: "iscsi"
  portals:
    - "192.168.128.122"
    - "192.168.128.123"
maxClientThreads: "30"

Configuring a Storage Backend of the FC Type

If you want to use the FC protocol, ensure that the FC network between the host and the storage device is connected before installing Huawei CSI. If the FC network is not connected, connect the FC network and then restart the huawei-csi-node service. During the restart, do not use Huawei CSI to create new resources or mount or unmount an existing PVC. The following command is used as an example:

kubectl delete pods -n huawei-csi -l app=huawei-csi-node

The following is an example of the backend configuration file of the FC type for enterprise storage:

storage: "oceanstor-san"
name: "fc-155"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.155:8088"
  - "https://192.168.129.156:8088"
pools:
  - "StoragePool001"
parameters:
  protocol: "fc"
maxClientThreads: "30"

Configuring a Storage Backend of the NVMe over RoCE Type

If you want to use the NVMe over RoCE protocol, ensure that the NVMe over RoCE network between the host and the storage device is connected before installing Huawei CSI. If the NVMe over RoCE network is not connected, connect the NVMe over RoCE network and then restart the huawei-csi-node service. During the restart, do not use Huawei CSI to create new resources or mount or unmount an existing PVC. The following command is used as an example:

kubectl delete pods -n huawei-csi -l app=huawei-csi-node

The following is an example of the backend configuration file of the NVMe over RoCE type for enterprise storage:

storage: "oceanstor-san"
name: "roce-155"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.155:8088"
  - "https://192.168.129.156:8088"
pools:
  - "StoragePool001"
parameters:
  protocol: "roce"
  portals:
    - "192.168.128.120"        
    - "192.168.128.121"
maxClientThreads: "30"

Configuring a Storage Backend of the NVMe over FC Type

The following is an example of the backend configuration file of the NVMe over FC type for enterprise storage:

storage: "oceanstor-san"
name: "fc-nvme-155"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.155:8088"
  - "https://192.168.129.156:8088"
pools:
  - "StoragePool001"
parameters:
  protocol: "fc-nvme"
maxClientThreads: "30"

Configuring a Storage Backend of the NFS Type

The following is an example of the backend configuration file of the NFS type for enterprise storage:

storage: "oceanstor-nas"
name: "nfs-155"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.155:8088"
  - "https://192.168.129.156:8088"
pools:
  - "StoragePool001"
parameters:
  protocol: "nfs"
  portals:        
    - "192.168.128.155"
maxClientThreads: "30"

The following is an example of the backend configuration file of the NFS type for distributed storage:

storage: "fusionstorage-nas"
name: "nfs-126"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.125:8088"
  - "https://192.168.129.126:8088"
pools:
  - "StoragePool001"
parameters:
  protocol: "nfs"
  portals:
    - "192.168.128.123"
maxClientThreads: "30"

Configuring a Storage Backend of the SCSI Type

The following is an example of the backend configuration file of the SCSI type for distributed storage:

storage: "fusionstorage-san"
name: "scsi-155"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.155:8088"
pools:
  - "StoragePool001"
parameters:
  protocol: "scsi"
  portals:
    - {"hostname01": "192.168.125.21","hostname02": "192.168.125.22"}
maxClientThreads: "30"

Configuring a Storage Backend of the DPC Type

The following is an example of the backend configuration file of the DPC type for distributed storage:

storage: "fusionstorage-nas"
name: "dpc-155"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.155:8088"
  - "https://192.168.129.156:8088"
pools:
  - "StoragePool001"
parameters:      
  protocol: "dpc"
maxClientThreads: "30"

Configuring Storage Backends of the Dtree Type

The following is an example of the backend configuration file of the Dtree type for enterprise storage:

storage: "oceanstor-dtree"
name: "nfs-dtree"
namespace: "huawei-csi"
urls:
- "https://192.168.129.155:8088"
parameters:
  protocol: "nfs"
  parentname: "parent-filesystem"  
  portals:
    - "192.168.128.155"
maxClientThreads: "30"

Configuring Storage Backends of the HyperMetro Type

• Before configuring NAS HyperMetro, you need to configure the HyperMetro relationship between two storage devices, including the remote device, HyperMetro domain, and the like. The HyperMetro domain of the file system can only work in HyperMetro active-active (AA) mode. For details about the configuration operation, see the product documentation of the corresponding storage model.
• The accounts for connecting to NAS HyperMetro backends must be the administrator accounts of the storage vStores.
• Except NAS HyperMetro backends, the management URLs of other backends cannot be the URL of a logical management port of a vStore that has established the HyperMetro relationship.
• When a HyperMetro storage backend is used, do not provision common file systems. Otherwise, services may be interrupted in logical port failover scenarios.

CSI allows you to connect to OceanStor or OceanStor Dorado and provision HyperMetro volumes of the NFS type on the storage side. You need to configure storage backends that work in HyperMetro mode. The procedure is as follows: Create two configuration files and create backends one by one.

This example shows how to configure backends of the HyperMetro type for Huawei OceanStor or OceanStor Dorado. First, create local storage backend configuration file nfs-hypermetro-155.yaml.

storage: "oceanstor-nas"
name: "nfs-hypermetro-155"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.155:8088"
  - "https://192.168.129.156:8088"
pools:
  - "StoragePool001"
metrovStorePairID: "f09838237b93c000"
metroBackend: "nfs-hypermetro-157"
parameters:
  protocol: "nfs"
  portals:
    - "192.168.129.155"
maxClientThreads: "30"

After the local backend is created, create remote storage backend configuration file nfs-hypermetro-157.yaml.

storage: "oceanstor-nas"
name: "nfs-hypermetro-157"
namespace: "huawei-csi"
urls:
  - "https://192.168.129.157:8088"
  - "https://192.168.129.158:8088"
pools:
  - "StoragePool001"
metrovStorePairID: "f09838237b93c000"
metroBackend: "nfs-hypermetro-155"
parameters:
  protocol: "nfs"
  portals:
    - "192.168.129.157"
maxClientThreads: "30"

6.1.1.2 - Storage Backend Parameters

An example template of the backend configuration file is /examples/backend/backend.yaml. The following table lists the parameters.

Table 1 backend parameters

6.1.2 - Querying a Storage Backend

Run the oceanctl commands in Querying a Storage Backend to query the storage backend information.

6.1.3 - Updating a Storage Backend

• When oceanctl is used to update storage backend information, only the storage backend password can be updated.
• If the backend account password is updated on the storage device, the CSI plug-in will retry due to login failures. As a result, the account may be locked. If the account is locked, change the password by referring to An Account Is Locked After the Password Is Updated on the Storage Device.

6.1.3.1 - Updating the Password of a Storage Backend Using oceanctl

Example of Updating a Backend

1. Run the following command to obtain the help information about updating a storage backend.

   oceanctl update backend -h
   

   The following is an example of the command output.

   Update a backend for Ocean Storage in Kubernetes
   
   Usage:
     oceanctl update backend <name> [flags]
   
   Examples:
     # Update backend account information in default(huawei-csi) namespace
     oceanctl update backend <name>  --password
   
     # Update backend account information in specified namespace
     oceanctl update backend <name> -n namespace --password
   
   Flags:
     -h, --help               help for backend
     -n, --namespace string   namespace of resources
         --password           Update account password
   

2. Run the following command to update a storage backend.

   oceanctl update backend backend-1 --password
   

   Enter the user name and new password as prompted:

   Please enter this backend user name:admin
   Please enter this backend password:
   
   backend/backend-1 updated
   

6.1.3.2 - Manually Updating a Storage Backend

• PVC provisioning must be based on a configured storage backend. Therefore, if a PVC has been provisioned on a storage backend, do not change the storage backend.
• The name uniquely identifies a storage backend. The name of a storage backend with a PVC provisioned cannot be changed.
• After a storage backend is modified, the new configuration applies only to volumes to be provisioned.
• Do not perform volume management operations during the modification of a storage backend.

Procedure

1. Delete the storage backend to be modified. For details, see Deleting a Storage Backend.
2. Create a storage backend with the same name. For details, see Creating a Storage Backend. The storage backend name cannot be changed.

6.1.4 - Deleting a Storage Backend

Do not delete a storage backend when a volume management operation is being performed on it.

Example of Deleting a Backend

1. Run the following command to obtain information about a storage backend.

   oceanctl get backend 
   

   The following is an example of the command output.

   NAMESPACE     NAME         PROTOCOL    STORAGETYPE      SN                    STATUS  ONLINE  URL                 
   huawei-csi    backend-1    roce        oceanstor-san    xxxxxxxxxxxxxxxxxxxx  Bound   true    https://192.168.129.157:8088   
   huawei-csi    backend-2    roce        oceanstor-san    xxxxxxxxxxxxxxxxxxxx  Bound   true    https://192.168.129.158:8088  
   

2. Run the following command to delete the specified storage backend.

   oceanctl delete backend backend-1
   

3. Run the following command to check the deletion result.

   oceanctl get backend backend-1
   

   The following is an example of the command output. If not found is displayed, the deletion is successful.

   Error from server (NotFound): backend "backend-1" not found
   

6.2 - (Optional) Adding a Certificate to a Storage Backend

This section describes how to create a certificate for a storage backend. If certificate verification is required for logging in to the storage, you can add a certificate by referring to this section. Currently, you can create a certificate for a storage backend based on the specified .crt or .pem file.

Before creating a certificate for a storage backend, import the prepared certificate to the storage array.

6.2.1 - Creating a Certificate for a Storage Backend

Prerequisites

A certificate has been created. Take OceanStor Dorado as an example. For details about how to create a certificate, click here.

Example of Creating a Certificate

1. Prepare a certificate file in advance, for example, cert.crt.

2. Run the following command to obtain information about a storage backend.

   oceanctl get backend  
   

   The following is an example of the command output.

   NAMESPACE     NAME         PROTOCOL    STORAGETYPE      SN                    STATUS  ONLINE  URL                 
   huawei-csi    backend-1    roce        oceanstor-san    xxxxxxxxxxxxxxxxxxxx  Bound   true    https://192.168.129.157:8088   
   huawei-csi    backend-2    roce        oceanstor-san    xxxxxxxxxxxxxxxxxxxx  Bound   true    https://192.168.129.158:8088  
   

3. Run the following command to create a certificate for the specified storage backend.

   oceanctl create cert cert-1 -b backend-1 -f /path/to/cert.crt
   

4. Check the certificate creation result.

   oceanctl get cert -b backend-1
   

   The following is an example of the command output.

   NAMESPACE    NAME    BOUNDBACKEND   
   huawei-csi   cert-1  backend-1 
   

6.2.2 - Querying a Storage Backend Certificate

Query storage backend certificates using the commands in Querying a Storage Backend Certificate.

6.2.3 - Updating a Storage Backend Certificate

Before updating a certificate, prepare a new certificate file and update the storage backend certificate by following the instructions provided in this section. If the certificate is no longer used, delete the certificate from the storage backend by referring to Deleting a Storage Backend Certificate.

Procedure

1. Run the following command to obtain information about a storage backend.

   oceanctl get backend 
   

   The following is an example of the command output.

   NAMESPACE     NAME         PROTOCOL    STORAGETYPE      SN                    STATUS  ONLINE  URL                 
   huawei-csi    backend-1    roce        oceanstor-san    xxxxxxxxxxxxxxxxxxxx  Bound   true    https://192.168.129.157:8088   
   huawei-csi    backend-2    roce        oceanstor-san    xxxxxxxxxxxxxxxxxxxx  Bound   true    https://192.168.129.158:8088  
   

2. Run the following command to check whether the specified storage backend has a certificate.

   oceanctl get cert -b backend-1
   

   The following is an example of the command output.

   NAMESPACE   NAME    BOUNDBACKEND    
   huawei-csi  cert-1  backend-1  
   

3. Run the following command to update the certificate of the specified storage backend.

   oceanctl update cert -b backend-1 -f /path/to/cert.crt
   

6.2.4 - Deleting a Storage Backend Certificate

Procedure

1. Run the following command to obtain information about a storage backend.

   oceanctl get backend  
   

   The following is an example of the command output.

   NAMESPACE     NAME         PROTOCOL    STORAGETYPE      SN                    STATUS  ONLINE  URL                 
   huawei-csi    backend-1    roce        oceanstor-san    xxxxxxxxxxxxxxxxxxxx  Bound   true    https://192.168.129.157:8088   
   huawei-csi    backend-2    roce        oceanstor-san    xxxxxxxxxxxxxxxxxxxx  Bound   true    https://192.168.129.158:8088  
   

2. Run the following command to obtain information about the certificate of the specified storage backend.

   oceanctl get cert -b backend-1
   

   The following is an example of the command output.

   NAMESPACE   NAME    BOUNDBACKEND    
   huawei-csi  cert-1  backend-1  
   

3. Run the following command to delete the certificate of the specified storage backend.

   oceanctl delete cert -b backend-1
   

4. Check the deletion result.

   oceanctl get cert -b backend-1
   

   The following is an example of the command output. If no cert found is displayed, the deletion is successful.

   Error from server (NotFound): no cert found on backend backend-1 in huawei-csi namespace
   

6.3 - Description of oceanctl Commands

Obtaining Help Information

• Obtain the oceanctl help information.

  oceanctl --help
  

• Check the oceanctl version.

  oceanctl version
  

• Specify the custom log file directory. The following example describes how to check the oceanctl version.

  oceanctl version --log-dir=/path/to/custom
  

Creating a Storage Backend

• Run the following command to obtain the help information about creating a backend.

  oceanctl create backend -h
  

• Run the following command to create a storage backend based on the specified yaml file.

  oceanctl create backend -f /path/to/backend.yaml -i yaml
  

• Run the following command to create a storage backend based on the specified json file. The huawei-csi-configmap file can be exported only in json format.

  oceanctl create backend -f /path/to/configmap.json -i json
  

• Run the following command to create a storage backend in the specified namespace.

  oceanctl create backend -f /path/to/backend.yaml -i yaml -n <namespace>
  

• Run the following command to create a storage backend and ignore the storage backend name verification, for example, uppercase letters and underscores (_). Do not run this command unless necessary.

  oceanctl create backend -f /path/to/backend.yaml -i yaml --not-validate-name
  

• Run the following command to create a storage backend and specify provisioner. csi.oceanstor.com is the driver name specified during installation. For details, see 4.

  This command is used only when a backend is created on the CCE or CCE Agile platform.

  oceanctl create backend -f /path/to/backend.yaml  -i yaml --provisioner=csi.oceanstor.com
  

Querying a Storage Backend

• Run the following command to obtain the help information about querying a backend.

  oceanctl get backend -h
  

• Run the following command to query a single storage backend in the default namespace.

  oceanctl get backend <backend-name>
  

• Run the following command to query all storage backends in the specified namespace.

  oceanctl get backend -n <namespace>
  

• Run the following command to format the output. Currently, json, yaml, and wide are supported.

  oceanctl get backend <backend-name> -o json
  

Updating a Storage Backend

• Run the following command to obtain the help information about updating a backend.

  oceanctl update backend -h
  

• Run the following command to update the specified storage backend in the default namespace.

  oceanctl update backend <backend-name> --password
  

• Run the following command to update a storage backend in the specified namespace.

  oceanctl update backend <backend-name> -n <namespace> --password
  

Deleting a Storage Backend

• Run the following command to obtain the help information about deleting a backend.

  oceanctl delete backend -h
  

• Run the following command to delete the specified storage backend in the default namespace.

  oceanctl delete backend <backend-name> 
  

• Run the following command to delete all storage backends in the default namespace.

  oceanctl delete backend  --all
  

• Run the following command to delete a storage backend in the specified namespace.

  oceanctl delete backend <backend-name...> -n <namespace>
  

Creating a Storage Backend Certificate

• Run the following command to obtain the help information about querying a certificate.

  oceanctl create cert -h
  

• Run the following command to create a certificate for a single storage backend in the default namespace based on the specified .crt certificate file.

  oceanctl create cert <name> -f /path/to/cert.crt -b <backend-name> 
  

• Run the following command to create a certificate for a single storage backend in the specified namespace based on the specified .crt certificate file.

  oceanctl create cert <name> -f /path/to/cert.crt -b <backend-name> -n <namespace>
  

• Run the following command to create a certificate for a single storage backend in the specified namespace based on the specified .pem certificate file.

  oceanctl create cert <name> -f /path/to/cert.pem -b <backend-name> -n <namespace>
  

Querying a Storage Backend Certificate

• Run the following command to obtain the help information about querying a certificate.

  oceanctl get cert -h
  

• Run the following command to query the certificate of a specified storage backend in the default namespace.

  oceanctl get cert -b <backend-name>
  

• Run the following command to query the certificate of a specified storage backend in the specified namespace.

  oceanctl get cert -b <backend-name> -n <namespace>
  

Updating a Storage Backend Certificate

• Run the following command to obtain the help information about updating a certificate.

  oceanctl update cert -h
  

• Run the following command to update a certificate for a specified storage backend in the default namespace based on the specified .crt certificate file.

  oceanctl update cert -b <backend-name> -f /path/to/cert.crt
  

• Run the following command to update a certificate for a specified storage backend in the specified namespace based on the specified .crt certificate file.

  oceanctl update cert -b <backend-name> -n <namespace> -f /path/to/cert.crt
  

• Run the following command to update a certificate for a specified storage backend in the specified namespace based on the specified .pem certificate file.

  oceanctl update cert -b <backend-name> -n <namespace> -f /path/to/cert.pem
  

Deleting a Storage Backend Certificate

• Run the following command to obtain the help information about deleting a certificate.

  oceanctl delete cert -h
  

• Run the following command to delete the certificate of a specified storage backend in the default namespace.

  oceanctl delete cert -b <backend-name> 
  

• Run the following command to delete the certificate of a specified storage backend in the specified namespace.

  oceanctl delete cert -b <backend-name> -n <namespace>