Chapter 5. Troubleshoot MCP gateway auth
You can troubleshoot common issues with authentication and authorization when working with the MCP gateway.
5.1. Troubleshoot MCP gateway authentication issues Copy linkLink copied to clipboard!
You can troubleshoot authentication issues and solve problems when working with the MCP gateway.
5.2. Troubleshoot MCP gateway authentication issues Copy linkLink copied to clipboard!
Authentication errors can happen in a variety of ways, including silent failures, broken sessions, and tool-access denials, depending on your backend MCP server setup. You can troubleshoot common problems by checking your connections and custom resource (CR) configurations.
When your clients cannot discover OAuth configuration, discovery is not working. Use the following steps to troubleshoot this situation.
Prerequisites
- You installed MCP gateway.
- You installed Connectivity Link.
-
You installed the OpenShift CLI (
oc). -
You configured a
Gatewayobject. -
You configured an
HTTPRouteobject for the gateway. - You registered an MCP server.
-
You created a
SecretCR for authentication.
Procedure
Retrieve the JSON object that lists the security requirements for your backend MCP server by running the following command:
$ curl https://<mcp_hostname>/.well-known/oauth-protected-resourceReplace
<mcp_hostname>with the name of the host for your MCP server.Example output
{ "resource_name": "MCP Server", "resource": "https://mcp.example.com/mcp", "authorization_servers": [ "https://auth.provider.com/realms/mcp" ], "bearer_methods_supported": ["header"], "scopes_supported": [ "basic", "groups", "roles", "profile" ] }Check that your associated
HTTPRouteCR includes a path for/.well-known/oauth-protected-resourceby running the following command:$ oc get httproute <route_name> -n <mcp_system> -o json | jq -r '.spec.rules[].matches[].path.value | select(. == "/.well-known/oauth-protected-resource")'-
Replace
<route_name>with the associatedHTTPRouteCR. -
Replace
<mcp_system>with the name of your MCP gateway deployment.
-
Replace
-
If there is no output, your
HTTPRouteCR does not have a path for/.well-known/oauth-protected-resource. You must add one. Check the specific
AuthPolicyCR configuration by running the following command:$ oc describe authpolicy <authn_name> -n <namespace>-
Replace
<authn_name>with the name of yourAuthPolicyCR. -
Replace
<namespace>with the namespace where yourAuthPolicyCR is applied.
-
Replace
Check that you excluded all
/.well-known/paths from yourAuthPolicyCR by trying to reach the endpoint without any credentials by using the following command:$ curl -o /dev/null -s -w "%{https_code}\n" https://<mcp_hostname>/.well-known/oauth-protected-resourceReplace
<mcp_hostname>with the name of the host for your MCP server.NoteThe following codes are examples of possible outputs:
-
200: Means that the exclusion exists and matches. -
401: Means that yourAuthPolicyCR is still demanding a token for this path. The exclusion is either not present or not working. -
404: The exclusion might be present and working, but theHTTPRouteCR does not point to that path or to a valid backend.
-
Optional. Check all MCP broker component environment variables by running the following command:
$ oc get deployment <mcp_gateway> -n <mcp_system> -o yaml | grep -A 10 env-
Replace
<mcp_gateway>with the name of your MCP gateway deployment. -
Replace
<mcp_system>with the namespace where your MCP gateway deployment is applied.
-
Replace
- Verify that the MCP broker component pod restarted after any environment variable changes.
5.2.1. Troubleshoot MCP gateway empty 401 errors Copy linkLink copied to clipboard!
When your 401 Unauthorized responses do not include OAuth discovery information, the WWW-Authenticate header is missing. This usually means that your AuthPolicy CR is not properly configured. Use the following steps to troubleshoot this situation.
Prerequisites
- You installed MCP gateway.
- You installed Connectivity Link.
-
You installed the OpenShift CLI (
oc). -
You configured a
Gatewayobject. -
You configured an
HTTPRouteobject for the gateway. - You registered an MCP server.
-
You created a
SecretCR for authentication.
Procedure
Isolate the failure point by using verbose output which lists the TLS handshake, the HTTP headers, and the server response code by running the following command:
$ curl -v https://<mcp_hostname>/mcp \ -H "Content-Type: application/json" \ -d '{"jsonrpc": "2.0", "id": 1, "method": "tools/list"}'Replace
<mcp_hostname>with the hostname for your backend MCP server.-
Verify that your
AuthPolicyCRspec.response.unauthenticated.headers:list includesresponse.unauthenticated.headers.WWW-Authenticate. -
Check that your
AuthPolicyCRresponse.unauthenticated.headers.WWW-Authenticate.valueincludes the correctmetadata. -
Ensure that your
AuthPolicyCR is applied to the correctGatewayobject and listener.
5.2.2. Troubleshoot MCP gateway valid tokens rejected Copy linkLink copied to clipboard!
When your valid tokens are rejected with 401 errors, that means your JWT token validation has failed. Use the following steps to troubleshoot this situation.
Prerequisites
- You installed MCP gateway.
- You installed Connectivity Link.
-
You installed the OpenShift CLI (
oc). -
You configured a
Gatewayobject. -
You configured an
HTTPRouteobject for the gateway. - You registered an MCP server.
-
You created a
SecretCR for authentication.
Procedure
List the
AuthPolicyCRs by running the following command:$ oc get authpolicy -ACheck the specific
AuthPolicyCR configuration by running the following command:$ oc describe authpolicy <authn_name> -n <namespace>-
Replace
<authn_name>with the name of yourAuthPolicyCR. -
Replace
<namespace>with the namespace where yourAuthPolicyCR is applied.
-
Replace
-
Verify that the
issuerUrlin theAuthPolicyCR matches your identity provider’srealm. Check the Authorino Operator logs by running the following command:
$ oc logs -n <kuadrant_system> -l authorino-resource=authorinoReplace
<kuadrant_system>with the namespace where Connectivity Link is installed.Decode JWT to verify claims by running the following command:
$ echo "<example_token>" | cut -d. -f2 | base64 -d | jqReplace
<example_token>with your token.Ensure that your issuer URL is reachable from the cluster by running the following command:
$ oc exec -n <kuadrant_system> deployment/authorino -- curl -s https://auth.provider.com/realms/mcp/.well-known/openid-configurationReplace
<kuadrant_system>with the name of your Connectivity Link deployment.-
Check the token expiration time by examining the
expclaim. -
Verify the audience, if required, by using an
audclaim. -
Ensure that the token includes all required claims such as
groups,email, and so on.
5.3. Troubleshoot MCP gateway authorization issues Copy linkLink copied to clipboard!
Authorization errors can happen in a variety of ways, including an authenticated user getting 403 errors for all tool calls, authorization checks not enforced, or authorization failing with CEL evaluation errors.
You can troubleshoot these problems by checking your configurations and troubleshooting CEL.
Prerequisites
- You installed MCP gateway.
- You installed Connectivity Link.
-
You installed the OpenShift CLI (
oc). -
You configured a
Gatewayobject. -
You configured an
HTTPRouteobject for the gateway. - You registered an MCP server.
Procedure
Check your
AuthPolicycustom resource (CR) authorization rules by running the following command:$ oc get authpolicy <policy_name> -n <namespace> -o yaml | grep -A 20 authorization-
Replace
<policy_name>with the name of yourAuthPolicyCR. -
Replace
<namespace> `with the namespace where your `AuthPolicyCR is applied.
-
Replace
When your authorization checks are not enforced, first check your
AuthPolicyCR status by running the following command:$ oc describe authpolicy <policy_name> -n <namespace>-
Replace
<policy_name>with the name of yourAuthPolicyCR. -
Replace
<namespace> `with the namespace where your `AuthPolicyCR is applied.
-
Replace
Next, verify your
AuthPolicyCR targets the correct resource by running the following command:$ oc get authpolicy <policy_name> -n <namespace> -o yaml | grep -A 5 targetRef-
Replace
<policy_name>with the name of yourAuthPolicyCR. -
Replace
<namespace> `with the namespace where your `AuthPolicyCR is applied.
-
Replace
Ensure that your
AuthPolicyCRtargetRefmatches yourGatewayobject name and namespace by running the following command:$ echo "--- AuthPolicy Targets ---" && \ oc get authpolicy -n <mcp_system> -o jsonpath='{range .items[*]}{.metadata.name}{"\t targets -> \t"}{.spec.targetRef.kind}{"/"}{.spec.targetRef.name}{"\n"}{end}' && \ echo "--- Actual Gateways ---" && \ oc get gateway -n <mcp_system> -o custom-columns=NAME:.metadata.nameReplace
<mcp_system>with the name of your MCP gateway deployment.-
If your
AuthPolicyCR and yourGatewayobject are in different namespaces, you must either move theAuthPolicyCR into the same namespace as the Gateway object, or target theHTTPRouteCR instead. Check that your
AuthPolicyCRsectionNamematches yourGatewayobject listener name.$ oc describe authpolicy <auth_policy_name> -n <mcp_system>-
Replace
<auth_policy_name>with the name of yourAuthPolicyCR. -
Replace
<mcp_system>with the name of your MCP gateway deployment.
-
Replace
-
Examine the
Statusblock for an entry about your listener. If thesectionNameis wrong, the policy shows"Accepted", but the policy does not affect the intended traffic path.
5.3.1. Troubleshoot MCP gateway CEL issues Copy linkLink copied to clipboard!
When authorization fails with CEL evaluation errors, you can troubleshoot by checking logs, communication, and other configuration.
Ensure that there is not a mismatch between your Envoy proxy layer and upstream application layers in performing URL normalization.
Envoy serves raw request paths to the wasm-shim component unless normalize_path and merge_slashes are explicitly enabled in the Envoy HTTP Connection Manager (HCM) settings. This means that you can treat /admin and //admin as distinct paths in your CEL predicates, and so on.
However, your application receives the requests. That upstream service can strip redundant slashes or decode URL-encoded values. This means it bypasses your configuration, serving the protected route.
Make sure that you configure your upstream service so that you do not unintentionally bypass your own security. For more information, see "Customize your system on path normalization" in the Istio documentation.
Prerequisites
- You installed MCP gateway.
- You installed Connectivity Link.
-
You installed the OpenShift CLI (
oc). -
You configured a
Gatewayobject. -
You configured an
HTTPRouteobject for the gateway. - You registered an MCP server.
Procedure
Check the Authorino Operator logs for CEL evaluation by running the following command:
$ oc logs -n <kuadrant_system> -l authorino-resource=authorino | grep -i authzReplace
<kuadrant_system>with the namespace where Connectivity Link is installed.- Ensure that the Authorino Operator can communicate with your identity server.
-
Verify that your JWT token includes
resource_access[server-name].rolesclaims. -
Verify the CEL syntax in your authorization
rules. -
Check that referenced fields exist, such as
auth.identity.groups. - Verify that the JSON or data object coming back from the source matches the exact fields you referenced in your Authorino configuration.
-
Test
CELexpression syntax using online validation tools. -
Add persistent logging to understand the
CELevaluation context.
5.3.2. Check the Kuadrant Operator status Copy linkLink copied to clipboard!
When authorization errors happen because the Kuadrant Operator is not working, you can diagnose the situation by checking the Operator status.
Prerequisites
- You installed MCP gateway.
- You installed Connectivity Link.
-
You installed the OpenShift CLI (
oc). -
You configured a
Gatewayobject. -
You configured an
HTTPRouteobject for the gateway. - You registered an MCP server.
Procedure
Check that the Kuadrant Operator is working by running the following command:
$ oc get pods -n <kuadrant_system>Replace
<kuadrant_system>with the namespace where your Connectivity Link is installed.Example output
NAME READY STATUS RESTARTS AGE authorino-78c5679f94-abc12 1/1 Running 0 5d dns-operator-controller-manager-5d4789f6-x1y2z 1/1 Running 0 5d kuadrant-operator-controller-manager-8495bc4d-98765 1/1 Running 0 5d limitador-67f89bc5d4-z9w8v 1/1 Running 0 5d-
If the
authorino-*pod showsCrashLoopBackOff, it either cannot reach your OIDC issuer or has an invalid configuration. -
If the
kuadrant-operator-controller-manager-*pod is down, any changes you make to yourAuthPolicyCR cannot be applied to theGatewayobject because the controller pod does not reconcile yourAuthPolicyCR.
-
If the
- Remedy pod issues as required.