×

A node selector specifies a map of key-value pairs. The rules are defined using custom labels on nodes and selectors specified in pods. You can use node selectors to place specific pods on specific nodes, all pods in a project on specific nodes, or create a default node selector to schedule pods that do not have a defined node selector or project selector.

For the pod to be eligible to run on a node, the pod must have the indicated key-value pairs as the label on the node.

If you are using node affinity and node selectors in the same pod configuration, see the important considerations below.

Using node selectors to control pod placement

You can use node selector labels on pods to control where the pod is scheduled.

With node selectors, OpenShift Container Platform schedules the pods on nodes that contain matching labels.

You can add labels to a node or MachineConfig, but the labels will not persist if the node or machine goes down. Adding the label to the MachineSet ensures that new nodes or machines will have the label.

To add node selectors to an existing pod, add a node selector to the controlling object for that node, such as a ReplicaSet, Daemonset, or StatefulSet. Any existing pods under that controlling object are recreated on a node with a matching label. If you are creating a new pod, you can add the node selector directly to the pod spec.

You cannot add a node selector to an existing scheduled pod.

Prerequisites

If you want to add a node selector to existing pods, determine the controlling object for that pod. For exeample, the router-default-66d5cf9464-m2g75 pod is controlled by the router-default-66d5cf9464 ReplicaSet:

$ oc describe pod router-default-66d5cf9464-7pwkc

Name:               router-default-66d5cf9464-7pwkc
Namespace:          openshift-ingress

....

Controlled By:      ReplicaSet/router-default-66d5cf9464

The web console lists the controlling object under ownerReferences in the pod YAML:

  ownerReferences:
    - apiVersion: apps/v1
      kind: ReplicaSet
      name: router-default-66d5cf9464
      uid: d81dd094-da26-11e9-a48a-128e7edf0312
      controller: true
      blockOwnerDeletion: true
Procedure
  1. Add the desired label to your nodes:

    $ oc label <resource> <name> <key>=<value>

    For example, to label a node:

    $ oc label nodes ip-10-0-142-25.ec2.internal type=user-node region=east

    The label is applied to the node:

    kind: Node
    apiVersion: v1
    metadata:
      name: ip-10-0-131-14.ec2.internal
      selfLink: /api/v1/nodes/ip-10-0-131-14.ec2.internal
      uid: 7bc2580a-8b8e-11e9-8e01-021ab4174c74
      resourceVersion: '478704'
      creationTimestamp: '2019-06-10T14:46:08Z'
      labels:
        beta.kubernetes.io/os: linux
        failure-domain.beta.kubernetes.io/zone: us-east-1a
        node.openshift.io/os_version: '4.2'
        node-role.kubernetes.io/worker: ''
        failure-domain.beta.kubernetes.io/region: us-east-1
        node.openshift.io/os_id: rhcos
        beta.kubernetes.io/instance-type: m4.large
        kubernetes.io/hostname: ip-10-0-131-14
        region: east (1)
        beta.kubernetes.io/arch: amd64
        type: user-node (1)
    ....
    1 Specify the label(s) you will add to the node.

    Alternatively, you can add the label to a MachineSet:

    $ oc edit MachineSet abc612-msrtw-worker-us-east-1c
    apiVersion: machine.openshift.io/v1beta1
    kind: MachineSet
    
    ....
    
    spec:
      replicas: 2
      selector:
        matchLabels:
          machine.openshift.io/cluster-api-cluster: ci-ln-89dz2y2-d5d6b-4995x
          machine.openshift.io/cluster-api-machine-role: worker
          machine.openshift.io/cluster-api-machine-type: worker
          machine.openshift.io/cluster-api-machineset: ci-ln-89dz2y2-d5d6b-4995x-worker-us-east-1a
      template:
        metadata:
          creationTimestamp: null
          labels:
            machine.openshift.io/cluster-api-cluster: ci-ln-89dz2y2-d5d6b-4995x
            machine.openshift.io/cluster-api-machine-role: worker
            machine.openshift.io/cluster-api-machine-type: worker
            machine.openshift.io/cluster-api-machineset: ci-ln-89dz2y2-d5d6b-4995x-worker-us-east-1a
        spec:
          metadata:
            creationTimestamp: null
            labels:
              region: east (1)
              type: user-node (1)
    ....
    1 Specify the label(s) you will add to the node.
  2. Add the desired node selector a pod:

    • To add a node selector to existing and furture pods, add a node selector to the controlling object for the pods:

      For example:

      kind: ReplicaSet
      
      ....
      
      spec:
      
      ....
      
        template:
          metadata:
            creationTimestamp: null
            labels:
              ingresscontroller.operator.openshift.io/deployment-ingresscontroller: default
              pod-template-hash: 66d5cf9464
          spec:
            nodeSelector:
              beta.kubernetes.io/os: linux
              node-role.kubernetes.io/worker: ''
              type: user-node (1)
      1 Add the desired node selector.
    • For a new pod, you can add the selector to the pod specification directly:

      apiVersion: v1
      kind: Pod
      
      ...
      
      spec:
        nodeSelector:
          <key>: <value>
      
      ...

      For example:

      apiVersion: v1
      kind: Pod
      
      ....
      
      spec:
        nodeSelector:
          region: east
          type: user-node

If you are using node selectors and node affinity in the same pod configuration, note the following:

  • If you configure both nodeSelector and nodeAffinity, both conditions must be satisfied for the pod to be scheduled onto a candidate node.

  • If you specify multiple nodeSelectorTerms associated with nodeAffinity types, then the pod can be scheduled onto a node if one of the nodeSelectorTerms is satisfied.

  • If you specify multiple matchExpressions associated with nodeSelectorTerms, then the pod can be scheduled onto a node only if all matchExpressions are satisfied.

Creating default cluster-wide node selectors

You can use default node selectors on pods together with labels on nodes to constrain all pods created in a cluster to specific nodes.

With cluster node selectors, when you create a pod in that cluster, OpenShift Container Platform adds the appropriate <key>:<value> and schedules the pod on nodes with matching labels.

You can add additional <key>:<value> pairs for the pod. But you cannot add a different <value> for a default <key>.

For example, if the cluster node selector is region: east the following pod spec adds a new pair and is allowed:

spec:
  nodeSelector:
    region: east
    type: user-node

The following pod spec uses a different value for region and is not allowed:

spec:
  nodeSelector:
    region: west

If the project where you are creating the pod has a project node selector, that selector takes preference over a cluster node selector.

Procedure

To add a default cluster node selector:

  1. Edit the Scheduler Operator Custom Resource to add the cluster node selectors:

    $ oc edit scheduler cluster
    apiVersion: config.openshift.io/v1
    kind: Scheduler
    metadata:
      name: cluster
    ...
    
    spec:
      defaultNodeSelector: type=user-node,region=east (1)
      mastersSchedulable: false
      policy:
        name: ""
    1 Add a node selector with the appropriate <key>:<value> pairs.

    After making this change, wait for the pods in the openshift-kube-apiserver project to redeploy. This can take several minutes. The default cluster node selector does not take effect until the pods redeploy.

  2. Edit a node or MachineSet to add labels:

    $ oc label <resource> <name> <key>=<value>

    For example, to label a node:

    $ oc label nodes ip-10-0-142-25.ec2.internal type=user-node region=east

    To label a MachineSet:

    $ oc label MachineSet abc612-msrtw-worker-us-east-1c type=user-node region=east

When you create a pod, OpenShift Container Platform adds the appropriate <key>:<value> and schedules the pod on the labeled node.

For example:

spec:
  nodeSelector:
    region: east
    type: user-node

Creating project-wide node selectors

You can use node selectors on a project together with labels on nodes to constrain all pods created in a namespace to the labeled nodes.

With project node selectors, when you create a pod in the namespace, OpenShift Container Platform adds the appropriate <key>:<value> and schedules the pod on nodes with matching labels.

You can add labels to a node or MachineConfig, but the labels will not persist if the node or machine goes down. Adding the label to the MachineSet ensures that new nodes or machines will have the label.

You can add additional <key>:<value> pairs for the pod. But you cannot add a different <value> for a default <key>.

For example, if the project node selector is region: east the following pod spec adds a new pair and is allowed:

spec:
  nodeSelector:
    region: east
    type: user-node

The following pod spec uses a different value for region and is not allowed:

spec:
  nodeSelector:
    region: west

If there is a cluster-wide default node selector, a project node selector takes preference.

Procedure

To add a default project node selector:

  1. Create a namespace or edit an existing namespace associated with the project to add the openshift.io/node-selector parameter:

    $ oc edit namespace <name>
    apiVersion: v1
    kind: Namespace
    metadata:
      annotations:
        openshift.io/node-selector: "type=user-node,region=east" (1)
        openshift.io/sa.scc.mcs: s0:c17,c14
        openshift.io/sa.scc.supplemental-groups: 1000300000/10000
        openshift.io/sa.scc.uid-range: 1000300000/10000
      creationTimestamp: 2019-06-10T14:39:45Z
      labels:
        openshift.io/run-level: "0"
      name: demo
      resourceVersion: "401885"
      selfLink: /api/v1/namespaces/openshift-kube-apiserver
      uid: 96ecc54b-8b8d-11e9-9f54-0a9ae641edd0
    spec:
      finalizers:
      - kubernetes
    status:
      phase: Active
    1 Add openshift.io/node-selector` with the appropriate <key>:<value> pairs.
  2. Edit a node or MachineSet to add labels:

    $ oc label <resource> <name> <key>=<value>

    For example, to label a node:

    $ oc label nodes ip-10-0-142-25.ec2.internal type=user-node region=east

    To label a MachineSet:

    $ oc label MachineSet abc612-msrtw-worker-us-east-1c type=user-node region=east

When you create a pod in the namespace, OpenShift Container Platform adds the appropriate <key>:<value> and schedules the pod on the labeled node.

For example:

spec:
  nodeSelector:
    region: east
    type: user-node