"event":"ipAllocated","ip":"172.22.0.201","msg":"IP address assigned by controller
As a cluster administrator, you can add the MetalLB Operator to your cluster so that when a service of type LoadBalancer
is added to the cluster, MetalLB can add an external IP address for the service.
The external IP address is added to the host network for your cluster.
Using MetalLB is valuable when you have a bare-metal cluster, or an infrastructure that is like bare metal, and you want fault-tolerant access to an application through an external IP address.
You must configure your networking infrastructure to ensure that network traffic for the external IP address is routed from clients to the host network for the cluster.
After deploying MetalLB with the MetalLB Operator, when you add a service of type LoadBalancer
, MetalLB provides a platform-native load balancer.
The MetalLB Operator monitors its own namespace for the following custom resources:
MetalLB
When you add a MetalLB
custom resource to the cluster, the MetalLB Operator deploys MetalLB on the cluster.
The Operator only supports a single instance of the custom resource.
If the instance is deleted, the Operator removes MetalLB from the cluster.
IPAddressPool
MetalLB requires one or more pools of IP addresses that it can assign to a service when you add a service of type LoadBalancer
.
An IPAddressPool
includes a list of IP addresses.
The list can be a single IP address that is set using a range, such as 1.1.1.1-1.1.1.1, a range specified in CIDR notation, a range specified as a starting and ending address separated by a hyphen, or a combination of the three.
An IPAddressPool
requires a name.
The documentation uses names like doc-example
, doc-example-reserved
, and doc-example-ipv6
.
An IPAddressPool
assigns IP addresses from the pool.
L2Advertisement
and BGPAdvertisement
custom resources enable the advertisement of a given IP from a given pool.
A single |
BGPPeer
The BGP peer custom resource identifies the BGP router for MetalLB to communicate with, the AS number of the router, the AS number for MetalLB, and customizations for route advertisement. MetalLB advertises the routes for service load-balancer IP addresses to one or more BGP peers.
BFDProfile
The BFD profile custom resource configures Bidirectional Forwarding Detection (BFD) for a BGP peer. BFD provides faster path failure detection than BGP alone provides.
L2Advertisement
The L2Advertisement custom resource advertises an IP coming from an IPAddressPool
using the L2 protocol.
BGPAdvertisement
The BGPAdvertisement custom resource advertises an IP coming from an IPAddressPool
using the BGP protocol.
After you add the MetalLB
custom resource to the cluster and the Operator deploys MetalLB, the controller
and speaker
MetalLB software components begin running.
MetalLB validates all relevant custom resources.
When you install the MetalLB Operator, the metallb-operator-controller-manager
deployment starts a pod. The pod is the implementation of the Operator. The pod monitors for changes to all the relevant resources.
When the Operator starts an instance of MetalLB, it starts a controller
deployment and a speaker
daemon set.
controller
The Operator starts the deployment and a single pod. When you add a service of type LoadBalancer
, Kubernetes uses the controller
to allocate an IP address from an address pool.
In case of a service failure, verify you have the following entry in your controller
pod logs:
"event":"ipAllocated","ip":"172.22.0.201","msg":"IP address assigned by controller
speaker
The Operator starts a daemon set for speaker
pods. By default, a pod is started on each node in your cluster. You can limit the pods to specific nodes by specifying a node selector in the MetalLB
custom resource when you start MetalLB. If the controller
allocated the IP address to the service and service is still unavailable, read the speaker
pod logs. If the speaker
pod is unavailable, run the oc describe pod -n
command.
For layer 2 mode, after the controller
allocates an IP address for the service, the speaker
pods use an algorithm to determine which speaker
pod on which node will announce the load balancer IP address.
The algorithm involves hashing the node name and the load balancer IP address. For more information, see "MetalLB and external traffic policy".
The speaker
uses Address Resolution Protocol (ARP) to announce IPv4 addresses and Neighbor Discovery Protocol (NDP) to announce IPv6 addresses.
For Border Gateway Protocol (BGP) mode, after the controller
allocates an IP address for the service, each speaker
pod advertises the load balancer IP address with its BGP peers. You can configure which nodes start BGP sessions with BGP peers.
Requests for the load balancer IP address are routed to the node with the speaker
that announces the IP address. After the node receives the packets, the service proxy routes the packets to an endpoint for the service. The endpoint can be on the same node in the optimal case, or it can be on another node. The service proxy chooses an endpoint each time a connection is established.
With layer 2 mode, one node in your cluster receives all the traffic for the service IP address. With BGP mode, a router on the host network opens a connection to one of the nodes in the cluster for a new client connection. How your cluster handles the traffic after it enters the node is affected by the external traffic policy.
cluster
This is the default value for spec.externalTrafficPolicy
.
With the cluster
traffic policy, after the node receives the traffic, the service proxy distributes the traffic to all the pods in your service.
This policy provides uniform traffic distribution across the pods, but it obscures the client IP address and it can appear to the application in your pods that the traffic originates from the node rather than the client.
local
With the local
traffic policy, after the node receives the traffic, the service proxy only sends traffic to the pods on the same node.
For example, if the speaker
pod on node A announces the external service IP, then all traffic is sent to node A.
After the traffic enters node A, the service proxy only sends traffic to pods for the service that are also on node A.
Pods for the service that are on additional nodes do not receive any traffic from node A.
Pods for the service on additional nodes act as replicas in case failover is needed.
This policy does not affect the client IP address. Application pods can determine the client IP address from the incoming connections.
The following information is important when configuring the external traffic policy in BGP mode. Although MetalLB advertises the load balancer IP address from all the eligible nodes, the number of nodes loadbalancing the service can be limited by the capacity of the router to establish equal-cost multipath (ECMP) routes. If the number of nodes advertising the IP is greater than the ECMP group limit of the router, the router will use less nodes than the ones advertising the IP. For example, if the external traffic policy is set to |
In layer 2 mode, the speaker
pod on one node announces the external IP address for a service to the host network.
From a network perspective, the node appears to have multiple IP addresses assigned to a network interface.
Since layer 2 mode relies on ARP and NDP, the client must be on the same subnet of the nodes announcing the service in order for MetalLB to work. Additionally, the IP address assigned to the service must be on the same subnet of the network used by the client to reach the service. |
The speaker
pod responds to ARP requests for IPv4 services and NDP requests for IPv6.
In layer 2 mode, all traffic for a service IP address is routed through one node. After traffic enters the node, the service proxy for the CNI network provider distributes the traffic to all the pods for the service.
Because all traffic for a service enters through a single node in layer 2 mode, in a strict sense, MetalLB does not implement a load balancer for layer 2.
Rather, MetalLB implements a failover mechanism for layer 2 so that when a speaker
pod becomes unavailable, a speaker
pod on a different node can announce the service IP address.
When a node becomes unavailable, failover is automatic.
The speaker
pods on the other nodes detect that a node is unavailable and a new speaker
pod and node take ownership of the service IP address from the failed node.