Installation Preparations
This chapter describes the preparations for the installation.
Prerequisites
Before performing the operations described in this chapter, ensure that the following conditions are met:
- A container management platform has been deployed and is running properly, and its compatibility meets the requirements described in Kubernetes and OS Compatibility.
- (Mandatory for enterprise storage) Initial configuration for interconnecting with Huawei enterprise storage has been completed, including storage pool division and port configuration. The version of the storage product meets the requirements in Compatibility with Huawei Enterprise Storage.
- (Mandatory for distributed storage) Initial configuration for interconnecting with Huawei distributed storage has been completed, including storage pool division and port configuration. The version of the storage product meets the requirements in Compatibility with Huawei Distributed Storage.
- The connectivity between Huawei storage and the container platform host has been configured. For example, the worker node running huawei-csi-controller communicates properly with the management IP address of the storage device to be connected, and the worker node running huawei-csi-node communicates properly with the service IP address of the storage device to be connected. In iSCSI scenarios, the ping command can be used to verify the connectivity.
- Ensure that the language of the operating system is English.
- Ensure that storage resource names, such as storage pool names and tenant names, are in English.
1 - Downloading the Huawei CSI Software Package
This section describes how to download the software package and the component structure of the software package.
Open a browser and enter https://github.com/Huawei/eSDK_K8S_Plugin/releases in the address box.
Download the software package of the 4.5.0 version based on the CPU architecture.
Software package naming rule: Plug-in name (eSDK_Huawei_Storage_Kubernetes_CSI_Plugin) + Version number + CPU architecture
Decompress the downloaded software package. The following table shows the component structure of the software package.
Table 1 Component description
Component | Description |
---|
image/huawei-csi-v4.5.0-arch.tar | huawei-csi-driver image. arch is X86 or ARM. |
image/storage-backend-controller-v4.5.0-arch.tar | Back-end management controller image. arch is X86 or ARM. |
image/storage-backend-sidecar-v4.5.0-arch.tar | Back-end management sidecar image. arch is X86 or ARM. |
image/huawei-csi-extender-v4.5.0-arch.tar | huawei-csi-extender image. arch is X86 or ARM. |
bin/ | Binary file used by an image provided by Huawei. |
bin/oceanctl | Command line tool provided by Huawei, which can be used to manage storage backends. |
helm/ | Helm project used to deploy Huawei CSI. |
manual/ | Used to manually install and deploy Huawei CSI. |
examples/ | .yaml sample file used during CSI use. |
examples/backend | .yaml sample file used to create a storage backend. |
2 - Uploading a Huawei CSI Image
Huawei provides the huawei-csi image for users. For details about how to obtain the image file, see Downloading the Huawei CSI Software Package.
To use the CSI image on the container management platform, you need to import the CSI image to the cluster in advance using either of the following methods:
- (Recommended) Use Docker to upload the CSI image to the image repository.
- Manually import the CSI image to all nodes where Huawei CSI needs to be deployed.
Uploading an Image to the Image Repository
The installation of Huawei CSI depends on the following image files provided by Huawei. Import and upload the image files in sequence. For details about how to obtain the image files, see Downloading the Huawei CSI Software Package.
- huawei-csi-v4.5.0-arch.tar
- storage-backend-controller-v4.5.0-arch.tar
- storage-backend-sidecar-v4.5.0-arch.tar
- huawei-csi-extender-v4.5.0-arch.tar
Prerequisites
A Linux host with Docker installed is available, and the host can access the image repository.
Procedure
Run the docker load -i huawei-csi-v4.5.0-arch.tar command to import the CSI image to the current node.
docker load -i huawei-csi-v4.5.0-arch.tar
Run the docker tag huawei-csi:4.5.0 repo.huawei.com/huawei-csi:4.5.0 command to add the image repository address to the image tag. repo.huawei.com indicates the image repository address.
docker tag huawei-csi:4.5.0 repo.huawei.com/huawei-csi:4.5.0
Run the docker push repo.huawei.com/huawei-csi:4.5.0 command to upload the CSI image to the image repository. repo.huawei.com indicates the image repository address.
docker push repo.huawei.com/huawei-csi:4.5.0
- You can also use containerd to import and upload the images.
- For details about how to import and upload images to the CCE or CCE Agile platform, see the user manual of the platform.
Uploading an Image to a Local Node
If the image has been uploaded to the image repository, skip this section.
Prerequisites
- The node has the corresponding Huawei CSI image file. For details about how to obtain the image file, see Downloading the Huawei CSI Software Package.
- Docker or another container engine has been installed on the node.
Procedure
Use a remote access tool, such as PuTTY, to log in to the node where the image is to be imported through the management IP address.
Copy the image directory in the Kubernetes CSI component package to any directory on the current node.
Run the cd image command to go to the image working directory. For details about the tool path, see Table 1.
Run the following commands in sequence to import all Huawei CSI images in the image directory to the local node. In the commands, name indicates the name of a .tar image package.
Run the following command using the Docker container engine:
docker load -i <name>.tar
Run the following command using the containerd container engine:
ctr -n k8s.io image import <name>.tar
Run the following command using the Podman container engine:
podman load -i <name>.tar
If another container engine is installed on the node, use the image import command for the corresponding container engine.
3 - Checking User Configurations on Huawei Storage
After Huawei storage is connected to the container platform, Huawei CSI needs to manage storage resources on Huawei storage based on service requirements, such as creating and mapping volumes. In this case, Huawei CSI needs to use the users created on Huawei storage to communicate with Huawei storage. The following table lists the user information required for different storage devices.
Table 1 User requirements for connecting storage to CSI
Storage Type | User Type | Role | Level | Type |
---|
OceanStor V5 | System user | Administrator | Administrator | Local user |
vStore user | vStore administrator | Administrator | Local user |
OceanStor Dorado V3 | System user | Administrator | Administrator | Local user |
OceanStor 6.1.3, 6.1.5, 6.1.6, 6.1.7, 6.1.8 | System user | Administrator/User-defined role1 | N/A | Local user |
OceanStor Dorado 6.1.0, 6.1.2, 6.1.3, 6.1.5, 6.1.6, 6.1.7, 6.1.8 | System user | Administrator/User-defined role1 | N/A | Local user |
vStore user | vStore administrator | N/A | Local user |
OceanStor Pacific series | System user | Administrator | N/A | Local user |
- Note 1: If a user-defined role is used, you need to configure permissions for the role. For details about how to configure the minimum permissions, see Configuring Custom Permissions.
You are advised not to use the users of the super administrator role.
4 - Checking Volume Snapshot-Dependent Components
This section describes how to check the volume snapshot-dependent components in the cluster.
Kubernetes earlier than v1.17.0 does not support the snapshot function. If the snapshot CRD is deployed, the cluster may be faulty. Therefore, if Huawei CSI is deployed on Kubernetes earlier than v1.17.0, perform the check according to Kubernetes Earlier Than v1.17.0.
Kubernetes Earlier Than v1.17.0
If the Kubernetes version is earlier than v1.17.0, the cluster may be faulty during snapshot deployment. Perform the following steps to delete the snapshot CRD installation file.
Run the following command to check the Kubernetes version. In the following example, the Kubernetes version is v1.16.0.
The following is an example of the command output.
NAME STATUS ROLES AGE VERSION
test-master Ready master 311d v1.16.0
test-node Ready <none> 311d v1.16.0
Go to the /helm/esdk/crds/snapshot-crds directory and run the following command to delete the snapshot CRD installation file. For details about the component package path, see Table 1.
rm -rf ./huawei-csi-snapshot-crd-v1.yaml
5 - Checking the Host Multipathing Configuration
If you plan to use the FC/iSCSI/NVMe over RoCE/NVMe over FC protocol to access Huawei storage in a container environment, you are advised to use host multipathing software to enhance the link redundancy and performance of the host and storage. If you do not want to use the software, skip this section.
For details about the OSs and multipathing software supported by Huawei CSI, see Table 2.
- If you want to use the FC/iSCSI protocol to connect to Huawei storage, you are advised to use native DM-Multipath provided by the OS.
- If you want to use the NVMe over RoCE/NVMe over FC protocol to connect to Huawei storage, you are advised to use Huawei-developed UltraPath-NVMe.
- If you want to use the SCSI protocol to connect to Huawei storage, disable DM-Multipath provided by the OS.
Prerequisites
Multipathing software has been correctly installed on a host.
- If you use native DM-Multipath provided by the OS, contact your host or OS provider to obtain the documents and software packages required for the installation.
- If you use Huawei-developed UltraPath or UltraPath-NVMe, contact Huawei engineers to obtain the UltraPath or UltraPath-NVMe documents and software packages. For details about the software package versions, see Table 1.
Procedure
If you use the iSCSI/FC protocol to connect to Huawei enterprise storage, configure and check host multipathing by referring to OceanStor Dorado and OceanStor Host Connectivity Guide for Red Hat.
If you use the NVMe over RoCE/NVMe over FC protocol to connect to Huawei enterprise storage, configure and check host multipathing by referring to OceanStor Dorado and OceanStor Host Connectivity Guide for Red Hat.
If you use iSCSI to connect to Huawei distributed storage, configure and check host multipathing by referring to Configuring Multipathing for an Application Server in FusionStorage 8.0.1 Block Storage Basic Service Configuration Guide.
If you use the native multipathing software provided by the OS, check whether the /etc/multipath.conf file contains the following configuration item.
defaults {
user_friendly_names yes
find_multipaths no
}
If the configuration item does not exist, add it to the beginning of the /etc/multipath.conf file.
For details about the functions of the user_friendly_names and find_multipaths parameters, see dm_multipath/config_file_defaults.
6 - Checking the Status of Host-Dependent Software
This section describes how to check whether the status of host-dependent software on worker nodes in a cluster is normal. In this example, the host OS is CentOS 7.9 x86_64.
Check the status of the iSCSI client.
systemctl status iscsi iscsid
Check the status of the NFS client.
Check the status of DM-Multipath.
systemctl status multipathd.socket multipathd
Check the status of UltraPath.
Check the status of UltraPath-NVMe.
systemctl status upudev upService_plus
7 - Checking the Images on Which CSI Depends
The installation of Huawei CSI depends on the images listed in the following table. If all worker nodes in the cluster have been connected to the Internet and can pull images online, you can skip this section. If nodes in the cluster cannot connect to the Internet, download the corresponding image file based on the Kubernetes version and upload it to the image repository or import it to all worker nodes in the Kubernetes cluster.
The huawei-csi-controller service depends on the following sidecar images: livenessprobe, csi-provisioner, csi-attacher, csi-resizer, csi-snapshotter, snapshot-controller, storage-backend-controller, storage-backend-sidecar, huawei-csi-driver, and huawei-csi-extender. The huawei-csi-node service depends on the following sidecar images: livenessprobe, csi-node-driver-registrar, and huawei-csi-driver.
For details about the functions and details of each image, see the following table.
Table 1 Images on which Huawei CSI depends
Container Name | Container Image | K8s Version Requirements | Feature Description |
---|
livenessprobe | k8s.gcr.io/sig-storage/livenessprobe:v2.5.0 | v1.16+ | This image is provided by the Kubernetes community, used to monitor the health status of CSI and report it to Kubernetes so that Kubernetes can automatically detect CSI program problems and restart the Pod to rectify the problems. |
csi-resizer | k8s.gcr.io/sig-storage/csi-resizer:v1.4.0 | v1.16+ | This image is provided by the Kubernetes community, used to call CSI to provide more storage space for a PVC when expanding the capacity of the PVC. |
csi-node-driver-registrar | k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.3.0 | v1.16+ | This image is provided by the Kubernetes community, used to obtain CSI information and register a node with kubelet using the plug-in registration mechanism of kubelet so that Kubernetes can detect the connection between the node and Huawei storage. |
csi-snapshotter | k8s.gcr.io/sig-storage/csi-snapshotter:v4.2.1 | v1.17+ | This image is provided by the Kubernetes community, used to call CSI to create or delete a snapshot on the storage system when creating or deleting a VolumeSnapshot. |
snapshot-controller | k8s.gcr.io/sig-storage/snapshot-controller:v4.2.1 | v1.17+ | This image is provided by the Kubernetes community, used to listen to the VolumeSnapshot and VolumeSnapshotContent objects in the Kubernetes API and trigger csi-snapshotter to create a snapshot on the storage system when creating or deleting a VolumeSnapshot. |
csi-provisioner
| k8s.gcr.io/sig-storage/csi-provisioner:v3.0.0 | v1.17+ | This image is provided by the Kubernetes community, used to create or delete PVCs. - Calls the huawei-csi-controller service to create a LUN or file system on the storage system as a PV when creating a PVC.
- Calls the huawei-csi-controller service to delete the LUN or file system corresponding to the PV when deleting a PVC.
|
quay.io/k8scsi/csi-provisioner:v1.4.0 | v1.16.x |
csi-attacher
| k8s.gcr.io/sig-storage/csi-attacher:v3.4.0 | v1.17+ | Calls the huawei-csi-controller service to perform the "Publish/Unpublish Volume" operation when creating or deleting a Pod.
|
quay.io/k8scsi/csi-attacher:v1.2.1 | v.1.16.x |
storage-backend-controller | storage-backend-controller:4.5.0 | v1.16+ | This image is provided by Huawei CSI software package, used to manage storageBackendClaim resources. |
storage-backend-sidecar | storage-backend-sidecar:4.5.0 | v1.16+ | This image is provided by Huawei CSI software package, used to manage storageBackendContent resources. |
huawei-csi-driver | huawei-csi:4.5.0 | v1.16+ | This image is provided by Huawei CSI software package, used to provide all features supported by Huawei CSI. |
huawei-csi-extender | huawei-csi-extender:4.5.0 | v1.16+ | This image is provided by Huawei CSI software package, used to provide extended features of Huawei CSI. |
If the cluster is not connected to the Internet, manually download the container images and upload them to the cluster. For details, see Downloading a Container Image.