OpenShift Container Platform can be provisioned with persistent storage by using local volumes. Local persistent volumes allow you to access local storage devices, such as a disk or partition, by using the standard PVC interface.

Local volumes can be used without manually scheduling Pods to nodes, because the system is aware of the volume node’s constraints. However, local volumes are still subject to the availability of the underlying node and are not suitable for all applications.

Local volumes can only be used as a statically created Persistent Volume.

Installing the Local Storage Operator

The Local Storage Operator is not installed in OpenShift Container Platform by default. Use the following procedure to install and configure this Operator to enable local volumes in your cluster.

Prerequisites
  • Access to the OpenShift Container Platform web console.

Procedure
  1. Create the local-storage project:

    $ oc new-project local-storage
  2. Install the Local Storage Operator from the web console.

    1. Log in to the OpenShift Container Platform web console.

    2. Navigate to OperatorsOperatorHub.

    3. Type Local Storage into the filter box to locate the Local Storage Operator.

    4. Click Install.

    5. On the Create Operator Subscription page, select A specific namespace on the cluster. Select local-storage from the drop-down menu.

    6. Adjust the values for the Update Channel and Approval Strategy to the desired values.

    7. Click Subscribe.

  3. Once finished, the Local Storage Operator will be listed in the Installed Operators section of the web console.

Provision the local volumes

Local volumes cannot be created by dynamic provisioning. Instead, PersistentVolumes must be created by the Local Storage Operator. This provisioner will look for any devices, both filesystem and block volumes, at the paths specified in defined resource.

Prerequisites
  • The Local Storage Operator is installed.

  • Local disks are attached to the OpenShift Container Platform nodes.

Procedure
  1. Create the local volume resource. This must define the nodes and paths to the local volumes.

    Do not use different StorageClass names for the same device. Doing so will create multiple PVs.

    Example: Filesystem
    apiVersion: "local.storage.openshift.io/v1"
    kind: "LocalVolume"
    metadata:
      name: "local-disks"
      namespace: "local-storage" (1)
    spec:
      nodeSelector: (2)
        nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
              - ip-10-0-140-183
              - ip-10-0-158-139
              - ip-10-0-164-33
      storageClassDevices:
        - storageClassName: "local-sc"
          volumeMode: Filesystem (3)
          fsType: xfs (4)
          devicePaths: (5)
            - /path/to/device
    1 The namespace where the Local Storage Operator is installed.
    2 Optional: A node selector containing a list of nodes where the local storage volumes are attached. This example uses the node host names, obtained from oc get node. If a value is not defined, then the Local Storage Operator will attempt to find matching disks on all available nodes.
    3 The volume mode, either Filesystem or Block, defining the type of the local volumes.
    4 The file system that is created when the local volume is mounted for the first time.
    5 The path to where the local volumes have been attached to the node.
    Example: Block
    apiVersion: "local.storage.openshift.io/v1"
    kind: "LocalVolume"
    metadata:
      name: "local-disks"
      namespace: "local-storage" (1)
    spec:
      nodeSelector: (2)
        nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
              - ip-10-0-136-143
              - ip-10-0-140-255
              - ip-10-0-144-180
      storageClassDevices:
        - storageClassName: "localblock-sc"
          volumeMode: Block  (3)
          devicePaths: (4)
            - /dev/xvdg
    1 The namespace where the Local Storage Operator is installed.
    2 Optional: A node selector containing a list of nodes where the local storage volumes are attached. This example uses the node host names, obtained from oc get node. If a value is not defined, then the Local Storage Operator will attempt to find matching disks on all available nodes.
    3 The volume mode, either Filesystem or Block, defining the type of the local volumes.
    4 The path to where the local volumes have been attached to the node.
  2. Create the local volume resource in your OpenShift Container Platform cluster, specifying the file you just created:

    $ oc create -f <local-volume>.yaml
  3. Verify the provisioner was created, and the corresponding DaemonSets were created:

    $ oc get all -n local-storage
    
    NAME                                          READY   STATUS    RESTARTS   AGE
    pod/local-disks-local-provisioner-h97hj       1/1     Running   0          46m
    pod/local-disks-local-provisioner-j4mnn       1/1     Running   0          46m
    pod/local-disks-local-provisioner-kbdnx       1/1     Running   0          46m
    pod/local-disks-local-diskmaker-ldldw         1/1     Running   0          46m
    pod/local-disks-local-diskmaker-lvrv4         1/1     Running   0          46m
    pod/local-disks-local-diskmaker-phxdq         1/1     Running   0          46m
    pod/local-storage-operator-54564d9988-vxvhx   1/1     Running   0          47m
    
    NAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
    service/local-storage-operator    ClusterIP   172.30.49.90     <none>        60000/TCP   47m
    
    NAME                                           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
    daemonset.apps/local-disks-local-provisioner   3         3         3       3            3           <none>          46m
    daemonset.apps/local-disks-local-diskmaker     3         3         3       3            3           <none>          46m
    
    NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.apps/local-storage-operator   1/1     1            1           47m
    
    NAME                                                DESIRED   CURRENT   READY   AGE
    replicaset.apps/local-storage-operator-54564d9988   1         1         1       47m

    Note the desired and current number of DaemonSet processes. If the desired count is 0, it indicates the label selectors were invalid.

  4. Verify that the PersistentVolumes were created:

    $ oc get pv
    
    NAME                CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
    local-pv-1cec77cf   100Gi      RWO            Delete           Available           local-sc                88m
    local-pv-2ef7cd2a   100Gi      RWO            Delete           Available           local-sc                82m
    local-pv-3fa1c73    100Gi      RWO            Delete           Available           local-sc                48m

Create the local volume PersistentVolumeClaim

Local volumes must be statically created as a PersistentVolumeClaim (PVC) to be accessed by the Pod.

Prerequisite
  • PersistentVolumes have been created using the local volume provisioner.

Procedure
  1. Create the PVC using the corresponding StorageClass:

    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: local-pvc-name (1)
    spec:
      accessModes:
      - ReadWriteOnce
      volumeMode: Filesystem (2)
      resources:
        requests:
          storage: 100Gi (3)
      storageClassName: local-sc (4)
    1 Name of the PVC.
    2 The type of the PVC. Defaults to Filesystem.
    3 The amount of storage available to the PVC.
    4 Name of the StorageClass required by the claim.
  2. Create the PVC in the OpenShift Container Platform cluster, specifying the file you just created:

    $ oc create -f <local-pvc>.yaml

Attach the local claim

After a local volume has been mapped to a PersistentVolumeClaim (PVC) it can be specified inside of a resource.

Prerequisites
  • A PVC exists in the same namespace.

Procedure
  1. Include the defined claim in the resource’s Spec. The following example declares the PVC inside a Pod:

    apiVersion: v1
    kind: Pod
    spec:
      ...
      containers:
        volumeMounts:
        - name: localpvc (1)
          mountPath: "/data" (2)
      volumes:
      - name: localpvc
        persistentVolumeClaim:
          claimName: localpvc (3)
    1 Name of the volume to mount.
    2 Path inside the Pod where the volume is mounted.
    3 Name of the existing PVC to use.
  2. Create the resource in the OpenShift Container Platform cluster, specifying the file you just created:

    $ oc create -f <local-pod>.yaml

Using tolerations with Local Storage Operator Pods

Taints can be applied to nodes to prevent them from running general workloads. To allow the Local Storage Operator to use tainted nodes, you must add tolerations to the Pod or DaemonSet definition. This allows the created resources to run on these tainted nodes.

You apply tolerations to the Local Storage Operator Pod through the LocalVolume resource and apply taints to a node through the node specification. A taint on a node instructs the node to repel all Pods that do not tolerate the taint. Using a specific taint that is not on other Pods ensures that the Local Storage Operator Pod can also run on that node.

Taints and tolerations consist of a key, value, and effect. As an argument, it is expressed as key=value:effect. An operator allows you to leave one of these parameters empty.

Prerequisites
  • The Local Storage Operator is installed.

  • Local disks are attached to OpenShift Container Platform nodes with a taint.

  • Tainted nodes are expected to provision local storage.

Procedure

To configure local volumes for scheduling on tainted nodes:

  1. Modify the YAML file that defines the Pod and add the LocalVolume spec, as shown in the following example:

      apiVersion: "local.storage.openshift.io/v1"
      kind: "LocalVolume"
      metadata:
        name: "local-disks"
        namespace: "local-storage"
      spec:
        tolerations:
          - key: localstorage (1)
            operator: Equal (2)
            value: "localstorage" (3)
    1 Specify the key that you added to the node.
    2 Specify the Equal operator to require the key/value parameters to match. If operator is 'Exists`, the system checks that the key exists and ignores the value. If operator is Equal, then the key and value must match.
    3 Specify the value local of the tainted node.

The defined tolerations will be passed to the resulting DaemonSets, allowing the diskmaker and provisioner Pods to be created for nodes that contain the specified taints.

Deleting the Local Storage Operator’s resources

Removing a local volume

Occasionally, local volumes must be deleted. While removing the entry in the LocalVolume resource and deleting the PersistentVolume is typically enough, if you want to re-use the same device path or have it managed by a different StorageClass, then additional steps are needed.

The following procedure involves accessing a node as the root user. Modifying the state of the node beyond the steps in this procedure could result in cluster instability.

Prerequisite
  • The PersistentVolume must be in a Released or Available state.

    Deleting a PersistentVolume that is still in use can result in data loss or corruption.

Procedure
  1. Edit the previously created LocalVolume to remove any unwanted disks.

    1. Edit the cluster resource:

      $ oc edit localvolume <name> -n local-storage
    2. Navigate to the lines under devicePaths, and delete any representing unwanted disks.

  2. Delete any PersistentVolumes created.

    $ oc delete pv <pv-name>
  3. Delete any symlinks on the node.

    1. Create a debug pod on the node:

      $ oc debug node/<node-name>
    2. Change your root directory to the host:

      $ chroot /host
    3. Navigate to the directory containing the local volume symlinks.

      $ cd /mnt/local-storage/<sc-name> (1)
      1 The name of the StorageClass used to create the local volumes.
    4. Delete the symlink belonging to the removed device.

      $ rm <symlink>

Uninstalling the Local Storage Operator

To uninstall the Local Storage Operator, you must remove the Operator and all created resources in the local-storage project.

Prerequisites
  • Access to the OpenShift Container Platform web console.

Procedure
  1. Uninstall the Local Storage Operator from the web console.

    1. Log in to the OpenShift Container Platform web console.

    2. Navigate to OperatorsInstalled Operators.

    3. Type Local Storage into the filter box to locate the Local Storage Operator.

    4. Click the Options menu kebab at the end of the Local Storage Operator.

    5. Click Uninstall Operator.

    6. Click Remove in the window that appears.

  2. Delete the Pods created by the Local Storage Operator:

    $ oc delete pods --all -n local-storage
  3. Delete the PersistentVolumes (PV) created by the Local Storage Operator. All of these PVs begin with local-pv.

    $ oc delete pv <pv-name>
  4. Delete the local-storage project:

    $ oc delete project local-storage