Expanding OpenShift Dedicated Persistent Volume Claims (PVCs)

Expanding PVCs based on volume types that need file system re-sizing, such as AWS EBS, is a two-step process. This process involves expanding volume objects in the cloud provider and then expanding the file system on the actual node. These steps occur automatically after the PVC object is edited and might require a Pod restart to take effect.

Expanding the file system on the node only happens when a new Pod is started with the volume.

Prerequisites
  • The controlling StorageClass must have allowVolumeExpansion set to true. This is the default configuration in OpenShift Dedicated.

    Decreasing the size of an Amazon Elastic Block Store (EBS) volume is not supported. However, you can create a smaller volume and then migrate your data to it by using a tool such as oc rsync. After modifying a volume, you must wait at least six hours before making additional modifications to the same volume.

Procedure
  1. Edit the PVC and request a new size by editing the spec.resources.requests.storage value. The following oc patch command will change the PVC’s size:

    $ oc patch pvc <pvc_name> -p '{"spec": {"resources": {"requests": {"storage": "8Gi"}}}}'
  2. Once the cloud provider object has finished re-sizing, the PVC might be set to FileSystemResizePending. The following command is used to check the condition:

    $ oc describe pvc mysql
    
    Name:          mysql
    Namespace:     my-project
    StorageClass:  gp2
    Status:        Bound
    Volume:        pvc-5fa3feb4-7115-4735-8652-8ebcfec91bb9
    Labels:        app=cakephp-mysql-persistent
                   template=cakephp-mysql-persistent
                   template.openshift.io/template-instance-owner=6c7f7c56-1037-4105-8c08-55a6261c39ca
    Annotations:   pv.kubernetes.io/bind-completed: yes
                   pv.kubernetes.io/bound-by-controller: yes
                   volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/aws-ebs
                   volume.kubernetes.io/selected-node: ip-10-0-128-221.us-east-2.compute.internal
                   volume.kubernetes.io/storage-resizer: kubernetes.io/aws-ebs
    Finalizers:    [kubernetes.io/pvc-protection]
    Capacity:      1Gi (1)
    Access Modes:  RWO
    VolumeMode:    Filesystem
    Conditions: (2)
      Type                     Status  LastProbeTime      LastTransitionTime  Reason  Message
      ----                     ------  -----------------  ------------------  ------  -------
      FileSystemResizePending  True    <Timestamp>        <Timestamp>                 Waiting for user to (re-)start a Pod to
                                                                                      finish file system resize of volume on node.
    Events:
      Type       Reason                 Age   From                         Message
      ----       ------                 ----  ----                         -------
      Normal     WaitForFirstConsumer   36m   persistentvolume-controller  waiting for first consumer to be created before binding
      Normal     ProvisioningSucceeded  36m   persistentvolume-controller  Successfully provisioned volume
                                                                           pvc-5fa3feb4-7115-4735-8652-8ebcfec91bb9 using
                                                                           kubernetes.io/aws-ebs
    Mounted By:  mysql-1-q4nz7 (3)
    1 The current capacity of the PVC.
    2 Any relevant conditions are displayed here.
    3 The Pod that is currently mounting this volume
  3. If the output of the previous command included a message to restart the Pod, delete the mounting Pod that it specified:

    $ oc delete pod mysql-1-q4nz7
  4. Once the Pod is running, the newly requested size is available and the FileSystemResizePending condition is removed from the PVC.