$ oc create clusterrolebinding <any_valid_name> --clusterrole=sudoer --user=<username>
The authentication layer identifies the user associated with requests to the OpenShift Container Platform API. The authorization layer then uses information about the requesting user to determine if the request should be allowed.
As an administrator, you can configure authentication using a master configuration file.
A user in OpenShift Container Platform is an entity that can make requests to the OpenShift Container Platform API. Typically, this represents the account of a developer or administrator that is interacting with OpenShift Container Platform.
A user can be assigned to one or more groups, each of which represent a certain set of users. Groups are useful when managing authorization policies to grant permissions to multiple users at once, for example allowing access to objects within a project, versus granting them to users individually.
In addition to explicitly defined groups, there are also system groups, or virtual groups, that are automatically provisioned by OpenShift. These can be seen when viewing cluster bindings.
In the default set of virtual groups, note the following in particular:
Virtual Group | Description |
---|---|
system:authenticated |
Automatically associated with all authenticated users. |
system:authenticated:oauth |
Automatically associated with all users authenticated with an OAuth access token. |
system:unauthenticated |
Automatically associated with all unauthenticated users. |
Requests to the OpenShift Container Platform API are authenticated using the following methods:
Obtained from the OpenShift Container Platform OAuth server using the
<master>/oauth/authorize
and <master>/oauth/token
endpoints.
Sent as an Authorization: Bearer…
header
Sent as an access_token=…
query parameter for websocket requests prior to
OpenShift Container Platform server version 3.6.
Sent as a websocket subprotocol header in the form
base64url.bearer.authorization.k8s.io.<base64url-encoded-token>
for websocket
requests in OpenShift Container Platform server version 3.6 and later.
Requires a HTTPS connection to the API server.
Verified by the API server against a trusted certificate authority bundle.
The API server creates and distributes certificates to controllers to authenticate themselves.
Any request with an invalid access token or an invalid certificate is rejected by the authentication layer with a 401 error.
If no access token or certificate is presented, the authentication layer assigns
the system:anonymous
virtual user and the system:unauthenticated
virtual
group to the request. This allows the authorization layer to determine which
requests, if any, an anonymous user is allowed to make.
A request to the OpenShift Container Platform API can include an Impersonate-User header,
which indicates that the requester wants to have the request handled as though
it came from the specified user. You impersonate a user by adding
the --as=<user>
flag to requests.
Before User A can impersonate User B, User A is authenticated. Then, an authorization check occurs to ensure that User A is allowed to impersonate the user named User B. If User A is requesting to impersonate a service account, system:serviceaccount:namespace:name, OpenShift Container Platform confirms that User A can impersonate the serviceaccount named name in namespace. If the check fails, the request fails with a 403 (Forbidden) error code.
By default, project administrators and editors can impersonate
service accounts in their namespace.
The sudoers role allows a user to
impersonate system:admin, which in turn has cluster administrator permissions.
The ability to impersonate system:admin grants some protection against typos,
but not security, for someone
administering the cluster. For example, running oc delete nodes --all
fails,
but running oc delete nodes --all --as=system:admin
succeeds. You
can grant a user that permission by running this command:
$ oc create clusterrolebinding <any_valid_name> --clusterrole=sudoer --user=<username>
If you need to create a project request on behalf of a user, include the
--as=<user> --as-group=<group1> --as-group=<group2>
flags in your command.
Because system:authenticated:oauth
is the only bootstrap group that can
create project requests, you must impersonate that group, as shown in the following example:
$ oc new-project <project> --as=<user> \ --as-group=system:authenticated --as-group=system:authenticated:oauth
The OpenShift Container Platform master includes a built-in OAuth server. Users obtain OAuth access tokens to authenticate themselves to the API.
When a person requests a new OAuth token, the OAuth server uses the configured identity provider to determine the identity of the person making the request.
It then determines what user that identity maps to, creates an access token for that user, and returns the token for use.
Every request for an OAuth token must specify the OAuth client that will receive and use the token. The following OAuth clients are automatically created when starting the OpenShift Container Platform API:
OAuth Client | Usage |
---|---|
openshift-web-console |
Requests tokens for the web console. |
openshift-browser-client |
Requests tokens at |
openshift-challenging-client |
Requests tokens with a user-agent that can handle |
To register additional clients:
$ oc create -f <(echo ' kind: OAuthClient apiVersion: v1 metadata: name: demo (1) secret: "..." (2) redirectURIs: - "http://www.example.com/" (3) grantMethod: prompt (4) ')
1 | The name of the OAuth client is used as the client_id parameter when making requests to <master>/oauth/authorize and <master>/oauth/token . |
2 | The secret is used as the client_secret parameter when making requests to <master>/oauth/token . |
3 | The redirect_uri parameter specified in requests to <master>/oauth/authorize and <master>/oauth/token must be equal to (or prefixed by) one of the URIs in redirectURIs . |
4 | The grantMethod is used to determine what action to take when this client requests tokens and has not yet been granted access by the user. Uses the same values seen in Grant Options. |
A service account can be used as a constrained form of OAuth client. Service accounts can only request a subset of scopes that allow access to some basic user information and role-based power inside of the service account’s own namespace:
user:info
user:check-access
role:<any_role>:<serviceaccount_namespace>
role:<any_role>:<serviceaccount_namespace>:!
When using a service account as an OAuth client:
client_id
is system:serviceaccount:<serviceaccount_namespace>:<serviceaccount_name>
.
client_secret
can be any of the API tokens for that service account. For example:
$ oc sa get-token <serviceaccount_name>
To get WWW-Authenticate
challenges, set an
serviceaccounts.openshift.io/oauth-want-challenges
annotation on the service
account to true.
redirect_uri
must match an annotation on the service account.
Redirect URIs for Service Accounts as
OAuth Clients provides more information.
Annotation keys must have the prefix
serviceaccounts.openshift.io/oauth-redirecturi.
or
serviceaccounts.openshift.io/oauth-redirectreference.
such as:
serviceaccounts.openshift.io/oauth-redirecturi.<name>
In its simplest form, the annotation can be used to directly specify valid redirect URIs. For example:
"serviceaccounts.openshift.io/oauth-redirecturi.first": "https://example.com" "serviceaccounts.openshift.io/oauth-redirecturi.second": "https://other.com"
The first
and second
postfixes in the above example are used to separate the
two valid redirect URIs.
In more complex configurations, static redirect URIs may not be enough. For
example, perhaps you want all ingresses for a route to be considered valid. This
is where dynamic redirect URIs via the
serviceaccounts.openshift.io/oauth-redirectreference.
prefix come into play.
For example:
"serviceaccounts.openshift.io/oauth-redirectreference.first": "{\"kind\":\"OAuthRedirectReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"Route\",\"name\":\"jenkins\"}}"
Since the value for this annotation contains serialized JSON data, it is easier to see in an expanded format:
{ "kind": "OAuthRedirectReference", "apiVersion": "v1", "reference": { "kind": "Route", "name": "jenkins" } }
Now you can see that an OAuthRedirectReference
allows us to reference the
route named jenkins
. Thus, all ingresses for that route will now be considered
valid. The full specification for an OAuthRedirectReference
is:
{ "kind": "OAuthRedirectReference", "apiVersion": "v1", "reference": { "kind": ..., (1) "name": ..., (2) "group": ... (3) } }
1 | kind refers to the type of the object being referenced. Currently, only route is supported. |
2 | name refers to the name of the object. The object must be in the same namespace as the service account. |
3 | group refers to the group of the object. Leave this blank, as the group for a route is the empty string. |
Both annotation prefixes can be combined to override the data provided by the reference object. For example:
"serviceaccounts.openshift.io/oauth-redirecturi.first": "custompath" "serviceaccounts.openshift.io/oauth-redirectreference.first": "{\"kind\":\"OAuthRedirectReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"Route\",\"name\":\"jenkins\"}}"
The first
postfix is used to tie the annotations together. Assuming that the
jenkins
route had an ingress of https://example.com, now
https://example.com/custompath is considered valid, but
https://example.com is not. The format for partially supplying override
data is as follows:
Type | Syntax |
---|---|
Scheme |
"https://" |
Hostname |
"//website.com" |
Port |
"//:8000" |
Path |
"examplepath" |
Specifying a host name override will replace the host name data from the referenced object, which is not likely to be desired behavior. |
Any combination of the above syntax can be combined using the following format:
<scheme:>//<hostname><:port>/<path>
The same object can be referenced more than once for more flexibility:
"serviceaccounts.openshift.io/oauth-redirecturi.first": "custompath" "serviceaccounts.openshift.io/oauth-redirectreference.first": "{\"kind\":\"OAuthRedirectReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"Route\",\"name\":\"jenkins\"}}" "serviceaccounts.openshift.io/oauth-redirecturi.second": "//:8000" "serviceaccounts.openshift.io/oauth-redirectreference.second": "{\"kind\":\"OAuthRedirectReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"Route\",\"name\":\"jenkins\"}}"
Assuming that the route named jenkins
has an ingress of
https://example.com, then both https://example.com:8000 and
https://example.com/custompath are considered valid.
Static and dynamic annotations can be used at the same time to achieve the desired behavior:
"serviceaccounts.openshift.io/oauth-redirectreference.first": "{\"kind\":\"OAuthRedirectReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"Route\",\"name\":\"jenkins\"}}" "serviceaccounts.openshift.io/oauth-redirecturi.second": "https://other.com"
In some cases the API server returns an unexpected condition error message that is difficult to debug without direct access to the API master log. The underlying reason for the error is purposely obscured in order to avoid providing an unauthenticated user with information about the server’s state.
A subset of these errors is related to service account OAuth configuration issues.
These issues are captured in events that can be viewed by non-administrator users. When encountering
an unexpected condition server error during OAuth, run oc get events
to view these events under ServiceAccount
.
The following example warns of a service account that is missing a proper OAuth redirect URI:
$ oc get events | grep ServiceAccount 1m 1m 1 proxy ServiceAccount Warning NoSAOAuthRedirectURIs service-account-oauth-client-getter system:serviceaccount:myproject:proxy has no redirectURIs; set serviceaccounts.openshift.io/oauth-redirecturi.<some-value>=<redirect> or create a dynamic URI using serviceaccounts.openshift.io/oauth-redirectreference.<some-value>=<reference>
Running oc describe sa/<service-account-name>
reports any OAuth events associated with the given service account name.
$ oc describe sa/proxy | grep -A5 Events Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 3m 3m 1 service-account-oauth-client-getter Warning NoSAOAuthRedirectURIs system:serviceaccount:myproject:proxy has no redirectURIs; set serviceaccounts.openshift.io/oauth-redirecturi.<some-value>=<redirect> or create a dynamic URI using serviceaccounts.openshift.io/oauth-redirectreference.<some-value>=<reference>
The following is a a list of the possible event errors:
No redirect URI annotations or an invalid URI is specified
Reason Message NoSAOAuthRedirectURIs system:serviceaccount:myproject:proxy has no redirectURIs; set serviceaccounts.openshift.io/oauth-redirecturi.<some-value>=<redirect> or create a dynamic URI using serviceaccounts.openshift.io/oauth-redirectreference.<some-value>=<reference>
Invalid route specified
Reason Message NoSAOAuthRedirectURIs [routes.route.openshift.io "<name>" not found, system:serviceaccount:myproject:proxy has no redirectURIs; set serviceaccounts.openshift.io/oauth-redirecturi.<some-value>=<redirect> or create a dynamic URI using serviceaccounts.openshift.io/oauth-redirectreference.<some-value>=<reference>]
Invalid reference type specified
Reason Message NoSAOAuthRedirectURIs [no kind "<name>" is registered for version "v1", system:serviceaccount:myproject:proxy has no redirectURIs; set serviceaccounts.openshift.io/oauth-redirecturi.<some-value>=<redirect> or create a dynamic URI using serviceaccounts.openshift.io/oauth-redirectreference.<some-value>=<reference>]
Missing SA tokens
Reason Message NoSAOAuthTokens system:serviceaccount:myproject:proxy has no tokens
The following steps represent one way a user could get into a broken state and how to debug or fix the issue:
Create a project utilizing a service account as an OAuth client.
Create YAML for a proxy service account object and ensure it uses the route proxy
:
vi serviceaccount.yaml
Add the following sample code:
apiVersion: v1 kind: ServiceAccount metadata: name: proxy annotations: serviceaccounts.openshift.io/oauth-redirectreference.primary: '{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"proxy"}}'
Create YAML for a route object to create a secure connection to the proxy:
vi route.yaml
Add the following sample code:
apiVersion: route.openshift.io/v1 kind: Route metadata: name: proxy spec: to: name: proxy tls: termination: Reencrypt apiVersion: v1 kind: Service metadata: name: proxy annotations: service.alpha.openshift.io/serving-cert-secret-name: proxy-tls spec: ports: - name: proxy port: 443 targetPort: 8443 selector: app: proxy
Create a YAML for a deployment configuration to launch a proxy as a sidecar:
vi proxysidecar.yaml
Add the following sample code:
apiVersion: extensions/v1beta1 kind: Deployment metadata: name: proxy spec: replicas: 1 selector: matchLabels: app: proxy template: metadata: labels: app: proxy spec: serviceAccountName: proxy containers: - name: oauth-proxy image: openshift3/oauth-proxy imagePullPolicy: IfNotPresent ports: - containerPort: 8443 name: public args: - --https-address=:8443 - --provider=openshift - --openshift-service-account=proxy - --upstream=http://localhost:8080 - --tls-cert=/etc/tls/private/tls.crt - --tls-key=/etc/tls/private/tls.key - --cookie-secret=SECRET volumeMounts: - mountPath: /etc/tls/private name: proxy-tls - name: app image: openshift/hello-openshift:latest volumes: - name: proxy-tls secret: secretName: proxy-tls
Create the objects
oc create -f serviceaccount.yaml oc create -f route.yaml oc create -f proxysidecar.yaml
Run oc edit sa/proxy
to edit the service account and change the serviceaccounts.openshift.io/oauth-redirectreference
annotation to point to a Route that does not exist.
apiVersion: v1 imagePullSecrets: - name: proxy-dockercfg-08d5n kind: ServiceAccount metadata: annotations: serviceaccounts.openshift.io/oauth-redirectreference.primary: '{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"notexist"}}' ...
Review the OAuth log for the service to locate the server error:
The authorization server encountered an unexpected condition that prevented it from fulfilling the request.
Run oc get events
to view the ServiceAccount
event:
oc get events | grep ServiceAccount 23m 23m 1 proxy ServiceAccount Warning NoSAOAuthRedirectURIs service-account-oauth-client-getter [routes.route.openshift.io "notexist" not found, system:serviceaccount:myproject:proxy has no redirectURIs; set serviceaccounts.openshift.io/oauth-redirecturi.<some-value>=<redirect> or create a dynamic URI using serviceaccounts.openshift.io/oauth-redirectreference.<some-value>=<reference>]
All requests for OAuth tokens involve a request to <master>/oauth/authorize
.
Most authentication integrations place an authenticating proxy in front of this
endpoint, or configure OpenShift Container Platform to validate credentials against a backing
identity
provider.
Requests to <master>/oauth/authorize
can come from user-agents that cannot
display interactive login pages, such as the CLI. Therefore, OpenShift Container Platform
supports authenticating using a WWW-Authenticate
challenge in addition to
interactive login flows.
If an authenticating proxy is placed in front of the
<master>/oauth/authorize
endpoint, it should send unauthenticated,
non-browser user-agents WWW-Authenticate
challenges, rather than displaying an
interactive login page or redirecting to an interactive login flow.
To prevent cross-site request forgery (CSRF) attacks against browser clients, Basic authentication challenges
should only be sent if a If the authenticating proxy cannot support |
Applications running in OpenShift Container Platform may need to discover information about
the built-in OAuth server. For example, they may need to discover what the
address of the <master>
server is without manual configuration. To aid in
this, OpenShift Container Platform implements the IETF
OAuth 2.0 Authorization Server Metadata draft specification.
Thus, any application running inside the cluster can issue a GET
request to
https://openshift.default.svc/.well-known/oauth-authorization-server to fetch
the following information:
{ "issuer": "https://<master>", (1) "authorization_endpoint": "https://<master>/oauth/authorize", (2) "token_endpoint": "https://<master>/oauth/token", (3) "scopes_supported": [ (4) "user:full", "user:info", "user:check-access", "user:list-scoped-projects", "user:list-projects" ], "response_types_supported": [ (5) "code", "token" ], "grant_types_supported": [ (6) "authorization_code", "implicit" ], "code_challenge_methods_supported": [ (7) "plain", "S256" ] }
1 | The authorization server’s issuer identifier, which is a URL that uses the
https scheme and has no query or fragment components. This is the location
where .well-known RFC 5785 resources
containing information about the authorization server are published. |
2 | URL of the authorization server’s authorization endpoint. See RFC 6749. |
3 | URL of the authorization server’s token endpoint. See RFC 6749. |
4 | JSON array containing a list of the OAuth 2.0 RFC 6749 scope values that this authorization server supports. Note that not all supported scope values are advertised. |
5 | JSON array containing a list of the OAuth 2.0 response_type values that this
authorization server supports. The array values used are the same as those used
with the response_types parameter defined by "OAuth 2.0 Dynamic Client
Registration Protocol" in RFC 7591. |
6 | JSON array containing a list of the OAuth 2.0 grant type values that this
authorization server supports. The array values used are the same as those used
with the grant_types parameter defined by OAuth 2.0 Dynamic Client
Registration Protocol in RFC 7591. |
7 | JSON array containing a list of PKCE
RFC 7636 code challenge methods
supported by this authorization server. Code challenge method values are used in
the code_challenge_method parameter defined in
Section 4.3 of RFC 7636.
The valid code challenge method values are those registered in the IANA PKCE
Code Challenge Methods registry. See
IANA OAuth Parameters. |
The OAuth server supports standard authorization code grant and the implicit grant OAuth authorization flows.
Run the following command to request an OAuth token by using the authorization code grant method:
$ curl -H "X-Remote-User: <username>" \
--cacert /etc/origin/master/ca.crt \
--cert /etc/origin/master/admin.crt \
--key /etc/origin/master/admin.key \
-I https://<master-address>/oauth/authorize?response_type=token\&client_id=openshift-challenging-client | grep -oP "access_token=\K[^&]*"
When requesting an OAuth token using the implicit grant flow
(response_type=token
) with a client_id configured to request WWW-Authenticate
challenges
(like openshift-challenging-client
), these are the possible server
responses from /oauth/authorize
, and how they should be handled:
Status | Content | Client response |
---|---|---|
302 |
|
Use the |
302 |
|
Fail, optionally surfacing the |
302 |
Other |
Follow the redirect, and process the result using these rules |
401 |
|
Respond to challenge if type is recognized (e.g. |
401 |
|
No challenge authentication is possible. Fail and show response body (which might contain links or details on alternate methods to obtain an OAuth token) |
Other |
Other |
Fail, optionally surfacing response body to the user |
To request an OAuth token using the implicit grant flow:
$ curl -u <username>:<password>
'https://<master-address>:8443/oauth/authorize?client_id=openshift-challenging-client&response_type=token' -skv / (1)
/ -H "X-CSRF-Token: xxx" (2)
* Trying 10.64.33.43...
* Connected to 10.64.33.43 (10.64.33.43) port 8443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 592 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification SKIPPED
* server certificate status verification SKIPPED
* common name: 10.64.33.43 (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: CN=10.64.33.43
* start date: Thu, 09 Aug 2018 04:00:39 GMT
* expire date: Sat, 08 Aug 2020 04:00:40 GMT
* issuer: CN=openshift-signer@1531109367
* compression: NULL
* ALPN, server accepted to use http/1.1
* Server auth using Basic with user 'developer'
> GET /oauth/authorize?client_id=openshift-challenging-client&response_type=token HTTP/1.1
> Host: 10.64.33.43:8443
> Authorization: Basic ZGV2ZWxvcGVyOmRzc2Zkcw==
> User-Agent: curl/7.47.0
> Accept: */*
> X-CSRF-Token: xxx
>
< HTTP/1.1 302 Found
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Expires: Fri, 01 Jan 1990 00:00:00 GMT
< Location:
https://10.64.33.43:8443/oauth/token/implicit#access_token=gzTwOq_mVJ7ovHliHBTgRQEEXa1aCZD9lnj7lSw3ekQ&expires_in=86400&scope=user%3Afull&token_type=Bearer (3)
< Pragma: no-cache
< Set-Cookie: ssn=MTUzNTk0OTc1MnxIckVfNW5vNFlLSlF5MF9GWEF6Zm55Vl95bi1ZNE41S1NCbFJMYnN1TWVwR1hwZmlLMzFQRklzVXRkc0RnUGEzdnBEa0NZZndXV2ZUVzN1dmFPM2dHSUlzUmVXakQ3Q09rVXpxNlRoVmVkQU5DYmdLTE9SUWlyNkJJTm1mSDQ0N2pCV09La3gzMkMzckwxc1V1QXpybFlXT2ZYSmI2R2FTVEZsdDBzRjJ8vk6zrQPjQUmoJCqb8Dt5j5s0b4wZlITgKlho9wlKAZI=; Path=/; HttpOnly; Secure
< Date: Mon, 03 Sep 2018 04:42:32 GMT
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host 10.64.33.43 left intact
1 | client-id is set to openshift-challenging-client and response-type is
set to token . |
2 | Set X-CSRF-Token header to a non-empty value. |
3 | The token is returned in the Location header of the 302 response as
access_token=gzTwOq_mVJ7ovHliHBTgRQEEXa1aCZD9lnj7lSw3ekQ . |
To view only the OAuth token value, run the following command:
$ curl -u <username>:<password>
'https://<master-address>:8443/oauth/authorize?client_id=openshift-challenging-client&response_type=token' (1)
-skv -H "X-CSRF-Token: xxx" --stderr - | grep -oP "access_token=\K[^&]*" (2)
hvqxe5aMlAzvbqfM2WWw3D6tR0R2jCQGKx0viZBxwmc
1 | client-id is set to openshift-challenging-client and response-type is
set to token . |
2 | Set X-CSRF-Token header to a non-empty value. |
You can also use the Code Grant
method to request a token
OpenShift Container Platform captures the following Prometheus system metrics during authentication attempts:
openshift_auth_basic_password_count
counts the number of oc login
user name and password attempts.
openshift_auth_basic_password_count_result
counts the number of oc login
user name and password attempts by result (success or error).
openshift_auth_form_password_count
counts the number of web console login attempts.
openshift_auth_form_password_count_result
counts the number of web console login attempts by result (success or error).
openshift_auth_password_total
counts the total number of oc login
and web console login attempts.