×

After deploying an installer-provisioned OpenShift Container Platform cluster, you can use the following procedures to expand the number of worker nodes. Ensure that each prospective worker node meets the prerequisites.

Expanding the cluster using RedFish Virtual Media involves meeting minimum firmware requirements. See Firmware requirements for installing with virtual media in the Prerequisites section for additional details when expanding the cluster using RedFish Virtual Media.

Preparing the bare metal node

To expand your cluster, you must provide the node with the relevant IP address. This can be done with a static configuration, or with a DHCP (Dynamic Host Configuration protocol) server. When expanding the cluster using a DHCP server, each node must have a DHCP reservation.

Reserving IP addresses so they become static IP addresses

Some administrators prefer to use static IP addresses so that each node’s IP address remains constant in the absence of a DHCP server. To configure static IP addresses with NMState, see "Optional: Configuring host network interfaces in the install-config.yaml file" in the "Setting up the environment for an OpenShift installation" section for additional details.

Preparing the bare metal node requires executing the following procedure from the provisioner node.

Procedure
  1. Get the oc binary:

    $ curl -s https://mirror.openshift.com/pub/openshift-v4/clients/ocp/$VERSION/openshift-client-linux-$VERSION.tar.gz | tar zxvf - oc
    $ sudo cp oc /usr/local/bin
  2. Power off the bare metal node by using the baseboard management controller (BMC), and ensure it is off.

  3. Retrieve the user name and password of the bare metal node’s baseboard management controller. Then, create base64 strings from the user name and password:

    $ echo -ne "root" | base64
    $ echo -ne "password" | base64
  4. Create a configuration file for the bare metal node. Depending on whether you are using a static configuration or a DHCP server, use one of the following example bmh.yaml files, replacing values in the YAML to match your environment:

    $ vim bmh.yaml
    • Static configuration bmh.yaml:

      ---
      apiVersion: v1 (1)
      kind: Secret
      metadata:
       name: openshift-worker-<num>-network-config-secret (2)
      type: Opaque
      stringData:
       nmstate: |
        hosts:
        - name: openshift-master-0
          role: master
          bmc:
            address: redfish+http://<out_of_band_ip>/redfish/v1/Systems/
            username: <user>
            password: <password>
            disableCertificateVerification: null
          bootMACAddress: <NIC1_mac_address>
          bootMode: UEFI
          rootDeviceHints:
            deviceName: "/dev/sda"
          networkConfig: (3)
            interfaces:
            - name: <nic1_name> (4)
              type: ethernet
              state: up
              ipv4:
                address:
                - ip: <ip_address> (4)
                  prefix-length: 24
                enabled: true
            dns-resolver:
              config:
                server:
                - <dns_ip_address> (4)
            routes:
              config:
              - destination: 0.0.0.0/0
                next-hop-address: <next_hop_ip_address> (4)
                next-hop-interface: <next_hop_nic1_name> (4)
      ---
      apiVersion: v1
      kind: Secret
      metadata:
        name: openshift-worker-<num>-bmc-secret (2)
        namespace: openshift-machine-api
      type: Opaque
      data:
        username: <base64_of_uid> (5)
        password: <base64_of_pwd> (5)
      ---
      apiVersion: metal3.io/v1alpha1
      kind: BareMetalHost
      metadata:
        name: openshift-worker-<num> (2)
        namespace: openshift-machine-api
      spec:
        online: True
        bootMACAddress: <nic1_mac_address> (6)
        bmc:
          address: <protocol>://<bmc_url> (7)
          credentialsName: openshift-worker-<num>-bmc-secret (2)
          disableCertificateVerification: True (8)
          username: <bmc_username> (9)
          password: <bmc_password> (9)
        rootDeviceHints:
          deviceName: <root_device_hint> (10)
        preprovisioningNetworkDataName: openshift-worker-<num>-network-config-secret (11)
      1 To configure the network interface for a newly created node, specify the name of the secret that contains the network configuration. Follow the nmstate syntax to define the network configuration for your node. See "Optional: Configuring host network interfaces in the install-config.yaml file" for details on configuring NMState syntax.
      2 Replace <num> for the worker number of the bare metal node in the name fields, the credentialsName field, and the preprovisioningNetworkDataName field.
      3 Add the NMState YAML syntax to configure the host interfaces.
      4 Replace <nic1_name>, <ip_address>, <dns_ip_address>, <next_hop_ip_address> and <next_hop_nic1_name> with appropriate values.
      5 Replace <base64_of_uid> and <base64_of_pwd> with the base64 string of the user name and password.
      6 Replace <nic1_mac_address> with the MAC address of the bare metal node’s first NIC. See the "BMC addressing" section for additional BMC configuration options.
      7 Replace <protocol> with the BMC protocol, such as IPMI, RedFish, or others. Replace <bmc_url> with the URL of the bare metal node’s baseboard management controller.
      8 To skip certificate validation, set disableCertificateVerification to true.
      9 Replace <bmc_username> and <bmc_password> with the string of the BMC user name and password.
      10 Optional: Replace <root_device_hint> with a device path if you specify a root device hint.
      11 Optional: If you have configured the network interface for the newly created node, provide the network configuration secret name in the preprovisioningNetworkDataName of the BareMetalHost CR.
    • DHCP configuration bmh.yaml:

      ---
      apiVersion: v1
      kind: Secret
      metadata:
        name: openshift-worker-<num>-bmc-secret (1)
        namespace: openshift-machine-api
      type: Opaque
      data:
        username: <base64_of_uid> (2)
        password: <base64_of_pwd> (2)
      ---
      apiVersion: metal3.io/v1alpha1
      kind: BareMetalHost
      metadata:
        name: openshift-worker-<num> (1)
        namespace: openshift-machine-api
      spec:
        online: True
        bootMACAddress: <nic1_mac_address> (3)
        bmc:
          address: <protocol>://<bmc_url> (4)
          credentialsName: openshift-worker-<num>-bmc-secret (1)
          disableCertificateVerification: True (5)
          username: <bmc_username> (6)
          password: <bmc_password> (6)
        rootDeviceHints:
          deviceName: <root_device_hint> (7)
        preprovisioningNetworkDataName: openshift-worker-<num>-network-config-secret (8)
      1 Replace <num> for the worker number of the bare metal node in the name fields, the credentialsName field, and the preprovisioningNetworkDataName field.
      2 Replace <base64_of_uid> and <base64_of_pwd> with the base64 string of the user name and password.
      3 Replace <nic1_mac_address> with the MAC address of the bare metal node’s first NIC. See the "BMC addressing" section for additional BMC configuration options.
      4 Replace <protocol> with the BMC protocol, such as IPMI, RedFish, or others. Replace <bmc_url> with the URL of the bare metal node’s baseboard management controller.
      5 To skip certificate validation, set disableCertificateVerification to true.
      6 Replace <bmc_username> and <bmc_password> with the string of the BMC user name and password.
      7 Optional: Replace <root_device_hint> with a device path if you specify a root device hint.
      8 Optional: If you have configured the network interface for the newly created node, provide the network configuration secret name in the preprovisioningNetworkDataName of the BareMetalHost CR.

    If the MAC address of an existing bare metal node matches the MAC address of a bare metal host that you are attempting to provision, then the Ironic installation will fail. If the host enrollment, inspection, cleaning, or other Ironic steps fail, the Bare Metal Operator retries the installation continuously. See "Diagnosing a host duplicate MAC address" for more information.

  5. Create the bare metal node:

    $ oc -n openshift-machine-api create -f bmh.yaml
    Example output
    secret/openshift-worker-<num>-network-config-secret created
    secret/openshift-worker-<num>-bmc-secret created
    baremetalhost.metal3.io/openshift-worker-<num> created

    Where <num> will be the worker number.

  6. Power up and inspect the bare metal node:

    $ oc -n openshift-machine-api get bmh openshift-worker-<num>

    Where <num> is the worker node number.

    Example output
    NAME                    STATE       CONSUMER   ONLINE   ERROR
    openshift-worker-<num>  available              true

    To allow the worker node to join the cluster, scale the machineset object to the number of the BareMetalHost objects. You can scale nodes either manually or automatically. To scale nodes automatically, use the metal3.io/autoscale-to-hosts annotation for machineset.

Additional resources

Replacing a bare-metal control plane node

Use the following procedure to replace an installer-provisioned OpenShift Container Platform control plane node.

If you reuse the BareMetalHost object definition from an existing control plane host, do not leave the externallyProvisioned field set to true.

Existing control plane BareMetalHost objects may have the externallyProvisioned flag set to true if they were provisioned by the OpenShift Container Platform installation program.

Prerequisites
  • You have access to the cluster as a user with the cluster-admin role.

  • You have taken an etcd backup.

    Take an etcd backup before performing this procedure so that you can restore your cluster if you encounter any issues. For more information about taking an etcd backup, see the Additional resources section.

Procedure
  1. Ensure that the Bare Metal Operator is available:

    $ oc get clusteroperator baremetal
    Example output
    NAME        VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
    baremetal   4.10.12   True        False         False      3d15h
  2. Remove the old BareMetalHost and Machine objects:

    $ oc delete bmh -n openshift-machine-api <host_name>
    $ oc delete machine -n openshift-machine-api <machine_name>

    Replace <host_name> with the name of the host and <machine_name> with the name of the machine. The machine name appears under the CONSUMER field.

    After you remove the BareMetalHost and Machine objects, then the machine controller automatically deletes the Node object.

  3. Create the new BareMetalHost object and the secret to store the BMC credentials:

    $ cat <<EOF | oc apply -f -
    apiVersion: v1
    kind: Secret
    metadata:
      name: control-plane-<num>-bmc-secret (1)
      namespace: openshift-machine-api
    data:
      username: <base64_of_uid> (2)
      password: <base64_of_pwd> (3)
    type: Opaque
    ---
    apiVersion: metal3