OpenShift Container Platform 3.7 Release Notes

OPENSHIFT_MASTER: A URL to the OpenShift API .

KUBERNETES_MASTER: A URL to the Kubernetes API exposed by OpenShift.

nonroot drops KILL, MKNOD, SETUID, and SETGID.

hostaccess drops KILL, MKNOD, SETUID, and SETGID.

hostmount-anyuid drops MKNOD.

The secret for the private browser OAuth client was not correctly initialized. Therefore, the request token endpoint did not work. This bug fix correctly initializes the browser OAuth client on server start. The request endpoint can now be used to request tokens. (BZ#1491193)

The LDAP sync/prune command did not take into account the use of groupUIDNameMapping with a whitelist. The sync/prune command would fail with "group not found" errors because it would query for the wrong group name. With this bug fix, the command was updated to take groupUIDNameMapping into account when using a whitelist. Now, the command queries for the correct group name when groupUIDNameMapping and a whitelist are used together. (BZ#1484831)

RoleBinding objects can now be created without first creating a PolicyBinding object. (BZ#1477956

ImageStream output references and their corresponding secrets were resolved during build creation time. If the output imagestream did not exist yet, no push secret would be be computed, resulting in a build failure during push. With this bug fix, the ImageStream output and push secret will be computed when preparing to run the build, under logic which will retry until the imagestream is available. Builds that are started before the output imagestream exists will no longer fail during the push phase. (BZ#1443163)

Build, delete, and watch events, and the current Jenkins job being canceled were not handled when a build was canceled in OpenShift. Various negative, inconsistent Jenkins job results occurred along with many exception stack traces in the Jenkins system log. With this bug fix, Jenkins jobs are halted as soon as the build watch event detects that a build was deleted as the result of a build cancel action taken within OpenShift. There is now consistent, sensible behavior for the Jenkins users when builds are canceled or deleted. (BZ#1473329)

Source-to-image was not closing stdin/out/err pipes correctly in some error cases, causing a hang to occur. This was causing some OpenShift builds to hang in running status. (BZ#1442875)

The openshift jenkins sync plug-in was updating Jenkins pipeline build status annotations every second, regardless of whether the status changed. The frequency of updates would put unnecessary stress on the etcd instance backing openshift master. Now, Jenkins pipeline build status annotations are only updated if the status actually changes, or 30 seconds have passed. (BZ#1475867)

Permissions on directories injected as a build input via the image source input mechanism have user-only access permissions. The resulting application image cannot access the content when run as a random user ID. The directories will now be injected with group permissions, which allows the container user to access the directories. The directories will now be accessible at runtime as desired. (BZ#1480312)

When no tag is explicitly set, docker pulls all images. Builds would pull more images than necessary and take longer than needed. With this bug fix, a default tag will be set when the user does not supply a tag. Only a single image will be pulled for the build. (BZ#1498178)

The BitBucket build trigger webhook did not handle older versions of the webhook payload. Builds could not be triggered by older versions of the BitBucket server. This bug fix adds support for the older payload format. Builds can now be triggered by older versions of BitBucket. (BZ#1500731)

A regression bug was reported whereby source-to-image builds would fail if the source repository file system contained a broken symlink (pointing to a non-existent item). This is now resolved. (BZ#1506173)

The oc binary for macOS is not signed. Some of the customer’s company policies do not allow users to install unsigned binaries. This bug fix signs the oc binary using a Red Hat certificate. The oc binary is now trusted by companies that restrict the installation of unsigned binaries. (BZ#1436093)

The git clone command was being run without a timeout. Therefore, the oc new-app command was timing out. With this bug fix, oc new-app now uses git ls-remote with a timeout and the oc new-app command will not timeout. (BZ#1488283)

The POOL_META_SIZE configuration item is now added. The thin pool metadata size was set to .1% of free space of volume group. POOL_META_SIZE allows the operator to customize the size of thin pool metadata volume size to meet their workload. (BZ#1451769)

Shortly after OpenShift starts, the caches might not yet be synchronised. Asa result, scaling the replication controllers might fail. Retry the scaling when there is a cache miss. With this bug fix, the replication controllers are scaled properly. (BZ#1427992)

A .NET jenkins slave image for performing .NET CI/CD flows is now offered. This makes it easier to build and test .NET code bases using Jenkins. A .NET slave image is provided and configured out of the box in the Jenkins master image. (BZ#1451403)

Jenkins now installs all plug-ins via one RPM, and the missing plug-in is now included. (BZ#1481010)

importPolicy.insecure is ignored in oc import-image <imagestream:tag> As a result, re-import from an insecure registry fails because it expects a valid SSL certificate. When the image stream tag exists, use its importPolicy.insecure. With this bug fix, re-import succeeds. (BZ#1494231)

Images younger than the threshold are not added to the dependency graph. A blob that is used by a young image and by a prunable image is deleted because it has no references in the graph. Add young images to the graph and mark them as non-prunable. With this bug fix, the blob has references and is not deleted. (BZ#1487408)

The image pruning algorithm would consider only managed images for pruning. As a result, mirrored blobs for not managed images could not be pruned. External images could not be removed using pruning. With this bug fix, the pruning algorithm evaluates all the images, not just managed images. External images and their blobs can now be pruned. (BZ#1441028)

Previously, a bug in a regulator of concurrent file system access could cause a routine to hang. This caused many builds to hang during the registry push.This bug fix corrects the regulator. As a result, concurrent pushes no longer hang. (BZ#1436841)

Previously, the oc adm prune images command would print confusing errors (such as operation timeout). This bug fix enables errors to be printed with hints. As a result, users are able to prune images, including images outside of the OpenShift cluster. (BZ#1469654)

The registry previously appended forwarded target ports to redirected location URLs. The client’s new request to the target location lacked credentials, and as a result, image push failed due to an authorization error. This bug fix rebased the registry to a newer version that fixes forwarding processing logic. As a result, clients can push images successfully to the exposed registry using arbitrary TLS-termination. (BZ#1471707)

Previously, imagestreamtags were not checked for dangling image references. This caused references to deleted images to be retained. This bug fix removes references to deleted images. As a result, deleting an image should allow references to the image to be deleted from imagestreamtags. (BZ#1386917)

Documentation and command help are now updated to include information on troubleshooting insecure connections to the secured registry. Error messages are now printed with hints, and new flags have been added to allow for insecure fall-back. As a result, users can now easily enforce both secure and insecure connections. (BZ#1448595)

Previously, the installation would fail when creating the Heketi secret because the key file did not copy on the first master host. This bug fix enables the installer to copy the SSH private key to the master node. (BZ#1477718)

The Ansible quick install would previously fail if the hostname was manually defined containing an uppercase letter. As a result, Kubernetes converted the names of the nodes to lowercase and did not recognize a node name with an uppercase letter. This bug fix ensures that hostnames for node objects are created with lowercase letters. (BZ#1396350)

Previously, the node service was not restarted when Open vSwitch was restarted, which could result in a misconfigured networking environment. This bug fix updates the services to ensure that the node service is restarted whenever Open vSwitch is restarted. (BZ#1453113)

Previously, Ansible facts added the svc domain to the NO_PROXY settings. As a result, users behind proxies were not able to push to registry by DNS. This bug fix adds the svc domain to the Ansible facts code. As a result, users behind a proxy can now push to registry by DNS. (BZ#1467776)

The flannel network was previously defined using the same subnet as the Kubernetes services subnet. This caused a conflict between services and SDN networks. The flannel network is now correctly defined by the osm_cluster_network_cidr variable. (BZ#1473858)

The necessary role for role binding in openshift_metrics was missing due to being processed out of order in the role. The role binding creation would fail and the role would fail to install. This bug fix updates the metrics to create the role immediately. As a result, role binding can be created during installation. (BZ#1476195)

The etcd scaleup playbook had an error where it attempted to run commands on hosts other than the host that was currently being scaled up resulting in an error if the other hosts did not yet have certain dependencies met. The playbooks now properly target only the host currently being scaled up. (BZ#1490739)

The stand-alone entry point for the openshift_storage_nfs task did not have the os_firewall role included. This resulted in the firewall not being properly installed and configured. The os_firewall has been added to the play. (BZ#1491657)

The etcd quota backend was set to 2GB by default. This resulted in a cluster going into a hold state, blocking all writes into the etcd storage. The default quota backend was increased to 4GB by default to encompass the storage needs of bigger clusters. (BZ#1492891)

When a company CA is added as a named certificates, the CA is added to ca-bundle.crt as well. This can cause client certificate popups when using IE,Safari or Chrome if the user has client certs configured via the browser. The code has been changed to not use the ca-bundle.crt and use the internal CA for client cert CA. (BZ#1493276)

As part of deprecating the use of openshift_hosted_{logging,metrics}_* variables, a default size for the storage volume wasn’t set for an NFS installation. As a result, the playbook would fail that the variable was not defined at runtime. The code was changed to use the default '10Gi' if not specified. The installer runs as expected. (BZ#1495203)

The disconnected installer did not have a way to specify a username/password to login to the docker repository to access downloaded images, requiring the user to disable authentication. The installation script now includes a mechanism for entering credentials. (BZ#1500642)

A new Docker option --signature-enabled that was introduced in a recent Docker release is set to False by default. The OpenShift Container Platform installation removes the parameter during the installation and Docker would get the default value of True. The Ansible scripts have been changed to include this option. (BZ#1502560)

Upgrading the logging component from 3.4.1 to 3.5.0 using Ansible failed with a No Elasticsearch pods found running error. The logging upgrade has been disabled as the EFK stack used for 3.4 and 3.5 is the same. The upgrade functionality is not necessary. (BZ#1435144)

When using ansible to configure the openID-connect provider for the OpenID and GitLab providers resulted in an error when setting challenge to true. This happens because of the validate function did not allowing this. The Ansible validate function was removed for OpenID and GitLab providers. The installation can complete successfully, and login succeeds. (BZ#1444367)

Docker 1.12.6-34 uses /etc/containers/registries.conf to define registries, but OpenShift Container Platform installer uses /etc/sysconfig/docker. As a result, system containers were reading registry information from the incorrect file. The code was changed to duplicate the registries in both locations to ensure additional/blocked/insecure registries are honored. (BZ#1460930)

A containerized installation with system containers enabled (use_system_containers=true) failed due to missing mounts. The code was updated so that the install performs as expected. (BZ#1463574)

The OpenShift Container Platform would correctly fail is the public host name was 64 characters or greater. However, the error message displayed did not report the source of the failure. The installer has been changed to report if the installation failed due to hostname length. (BZ#1467790)

When installing the service catalog, the template service broker (TSB) was not getting created. As a result, the TSB had to be created manually. The code has been changed so that the TSB is created automatically. (BZ#1470623)

Input for include_granted_scopes, which was expected to become a single quoted boolean string, was instead being interpreted and written to the file incorrectly. The resulting configuration file could have the wrong value for include_granted_scopes and removal of a code block attempted to interpret the input for include_granted_scopes. Input that is expected to land via include_granted_scopes now passes to the master-config.yml as expected. (BZ#1488505)

Because the Docker image availability health check does not support authenticated registries, checks failed when running against an authenticated registry. The code was changed to allow Docker to health check authenticated registries. (BZ#1488833)

Running the redeploy-router-certificates.yml playbook caused the router pod to fail (CrashLoopBackOff). The code was changed so that after running the redeploy-router-certificates.yml playbook, the router pod runs as expected. (BZ#1490186)

With Ansible 2.3, warnings are issued when using Jinja delimiters in 'when' conditions. The delimiters have been removed from the code base to avoid these warnings. (BZ#1490268)

Due to an earlier code change, the installation failed when giving a wildcard certificate to the installer. The code has been changed to properly copy a wildcard certificate during installation. (BZ#1492786)

Because of internal refactoring, the list of hostnames in the NO_PROXY file was empty. The facts have been restored The list of NO_PROXY names is correctly defined. (BZ#1495142)

When openshift_docker_use_system_container was set to false, the installer was incorrectly attempting to start the container engine, resulting in the installation failing. The installer code was changed and the installation proceeds as expected. (BZ#1496725)

The installer can now use an inventory specified as a directory rather than just a single file. This adds a parameter INVENTORY_DIR to the openshift-ansible image such that the user can indicate that ansible-playbook should use a mounted inventory directory. (BZ#1498908)

The logic for selecting the Enterprise registry was moved to a location that which was never read when installing system containers. Enterprise installs using system containers would fail as the openshift-ansible image could not be found in the Docker hub registry. Moved the enterprise registry logic into a high level playbook so that it is set for all runtime set ups. The enterprise images can be found and installation works. (BZ#1503860)

Due to recent simplification and refactoring there was a possibility of /etc/atomic.conf not being updated with proxy values before the first Atomic command was executed. Proxy use with the Atomic command did not work during the install. A new openshift_atomic role has been created for atomic specific tasks. The first task added is proxy which handles updating /etc/atomic.conf to ensure the proper proxy configuration is configured. This task file is then included (via include_role) in system container related task files. The atomic command always is able to use the properly defined proxy settings. (BZ#1503903)

An undefined variable was used in a task. The undefined variable caused a jinja template evaluation error which would crash the installation. The undefined variable has been removed and replaced with more informative error text. The playbook does not error out for external NFS storage class installations. (BZ#1504535)

The OpenShift Health Checker was not part of an Installer Phase and was not reported after playbook execution. The OpenShift Health Checker section of the primary installer path has been moved to its own section and an installer 'phase' has been added to report on installer status. (BZ#1504593)

When updating the openshift-ansible package, all subpackages are now updated in order to keep them in sync. (BZ#1506971)

The NetworkManager dispatcher script responsible for configuring a host to use dnsmasq operated in a non-atomic manner, resulting in failed DNS queries during boot up. The script has been refactored to ensure that required services are verified before /etc/resolv.conf is reconfigured. (BZ#1410288)

Using the Ansible installer to install metrics with dynamic storage failed. Installation now fails if the parameter storage kind = 'dynamic' is set without enabling dynamic provisioning. (BZ#1415297)

An error occurred from the yum module during the upgrade process. Yum transactions are now retried. (BZ#1479533)

The 'registry-console' image stream did not have a source tag specified, causing it to be improperly imported.The source tag has been added to the image stream ensuring that it imports properly. (BZ#1480442)

When enabling API aggregation with the ovs-multitenant SDN driver, creating a global project failed due to a performance latency issue. While creating a global project, the netnamespace is now checked to ensure availability and the Ansible Playbook Bundle finishes the operation. (BZ#1487959)

The device mapper kernel modules may not have been loaded on a host if overlay2 storage was used, which prevented the gluster storage system from working properly. With this fix, the installer now ensures that when gluster is used the dm_thin_pool, dm_snapshot, and dm_mirror modules are loaded. (BZ#1490905)

Previously, if there was no DNS search path in /etc/resolv.conf, then the NetworkManager dispatcher would omit adding cluster.local to the search path. With this bug fix, the dispatcher script was updated to ensure that a search path is created if one did not already exist. (BZ#1496593)

The example inventories have been updated to clearly indicate that the NFS export directory must only consist of lowercase alphanumeric characters, hyphens or periods, and must start and end with an alphanumeric character. (BZ#1488366)

The quick installer tool, atomic-openshift-installer, was initially blocked for use with OpenShift Container Platform 3.7 due to a bug. This has now been fixed in the latest update. (BZ#1509112)

Messages were read into Fluentd’s memory buffer and were lost if the pod was restarted because Fluentd considered them read, but they were not pushed to storage. This caused the loss of any message not stored, but already read by Fluentd. This fix replaced the memory buffer with a file based buffer. As a result, the file buffered messages are pushed to storage once Fluentd restarts. (BZ#1460749)

Kibana visualizations and dashboard for monitoring container and pod logs allows administrator users, cluster-admin or cluster-reader, to view logs by deployment, namespace, pod, and container. The script es_load_kibana_ui_objects is used to load dashboards and other Kibana UI objects for the given user. To use, run oc exec $espod — es_load_kibana_ui_objects user-name. It exists inside the Elasticsearch and ES-OPS pod, and must be run inside those pods. Additionally, it requires some indices and other objects set up by the OpenShift Elasticsearch plug-in, so the user must login to Kibana or Elasticsearch before using this script. This will also add an index pattern for project.* and load the necessary index pattern file. Kibana visualizations and dashboard gives administrators an easier way to view Kubernetes/OpenShift related logs in the cluster, allowing admin users have graphs and a dashboard to use to view logs from OpenShift pods and containers. (BZ#1467963)

The execute bit in the downstream repo was previously not set for run.sh. (BZ#1474715)

The value of the buffer_chunk_limit is now configurable, and defaults to 1M. To configure the buffer_chunk_limit, set the value to the environment variable BUFFER_SIZE_LIMIT or openshift_logging_fluentd_buffer_size_limit in the Ansible inventory file. To cover various types of input, buffer_chunk_limit needs to be configurable. The “size of the emitted data exceeds buffer_chunk_limit" can be fixed by configuring buffer_chunk_limit. (BZ#1413147)

Role permissions were generated based upon the project, causing queries to be disallowed if they involved multiple indices. This fix generates role permissions based on the user and not the project, allowing users to query across multiple indices. (BZ#1445425)

The openshift-elasticsearch-plugin was creating ACL roles based on the provided name, which could include slashes and commas. This caused the dependent lib to not properly evaluate roles. This fix hashes the name when creating ACL roles so they no longer contain the invalid characters. Now, users can use kibana and logging. (BZ#1456584)

The ansible parameter name is confusing to use and does not properly reflect how it is consumed by Fluentd. This fix removed the parameter, allowing Fluentd to consistently collect logs based on the source it detects. (BZ#1466152)

Elasticsearch was logging to console logs, resulting Elasticsearch ending up in a feedback loop ingesting its own logs. This fix turned off console logs in favor of file logs. As a result, the feedback loop is broken but users will need to setup Elasticsearch log volume with file rotation to get ES logs. Additionally, oc logs against an Elasticsearch pod will no longer be sufficient to retrieve Elasticsearch pod logs. (BZ#1432607)

Elasticsearch default value for sharing storage between Elasticsearch instances was wrong. This caused the incorrect default value to be allowed an Elasticsearch pod starting up (when another Elasticsearch pod was shutting down) to create a new location on the PV for managing the storage volume, duplicating data, and in some instances, potentially causing data loss. With this fix, all Elasticsearch pods now run with node.max_local_storage_nodes set to 1. As a result, the Elasticsearch pods starting up and shutting down will no longer share the same storage and prevent the data duplication and data loss. (BZ#1460564)

Use underscores when providing memory switches to the Nodejs runtime instead of dashes. As a result, the Nodejs interpreter understands the request. (BZ#1464020)

The openshift_logging_purge_logging Ansible variable was introduced to purge logging persistent data. Because the openshift_logging_install_logging=false will keep persistent data, there was a need for a complete uninstall. As a result, there are no changes to openshift_logging_install_logging, with the additional variable openshift_logging_purge_logging for complete uninstall. (BZ#1467265)

In the configuration for the Fluentd systemd input plug-in, the read_from_head parameter was not set properly based on the environment variable JOURNAL_READ_FROM_HEAD or its corresponding Ansible parameter openshift_logging_fluentd_journal_read_from_head. Due to the problem, the full contents of pre-existing logs were indexed instead of the latest logs captured by “tail” when a pos_file does not exist, which happens when the logging system is initially deployed or a pos_file is deleted. With this bug fix, the parameter is correctly set. And based on the setting, if JOURNAL_READ_FROM_HEAD=true, all the logs are indexed; if JOURNAL_READ_FROM_HEAD=false, logs read from "tail" are indexed when a pos_file does not exist. (BZ#1488941)

When deploying logging-fluentd with secure-forward to send the collected logs to logging-mux, it requires openshift_logging_mux_client_mode=maximal with openshift_logging_use_mux=True in the ansible inventory if the Fluentd container and the mux container are on the same node. If openshift_logging_mux_client_mode=maximal is set without openshift_logging_use_mux=True, the mux secret directory /etc/fluent/muxkeys is mounted in the Fluentd container although the secret directory does not exist. It makes Fluentd hang when it tries to access the mux secrets at the startup time. This patch checks the value of openshift_logging_mux_client_mode and openshift_logging_use_mux in the Ansible playbook and if the former is true while the latter is false, then it does not mount the mux secret directory in the Fluentd container. Also, if the Fluentd start script finds the mux secret directory does not exist, it disables openshift_logging_mux_client_mode even if it is enabled. (BZ#1490647)

The json-file parser was assuming the "time" field was a Time object instead of a String object, which does not have a "utc" method, causing the logs to fill with the error. This fix checks the type of object for the "time" field, and convert the String to a Time object if necessary. As a result, json-file read time values are parsed correctly with no errors. (BZ#1491405)

The openshift-elasticsearch-plugin was creating ACL roles based on the provided name which could include slashes and commas. This caused the dependent lib to not properly evaluate roles. This fix hashes the name when creating ACL roles so they no longer contain the invalid characters. As a result, users can use Kibana and logging. (BZ#1494239)

Previously in the web console pod terminal, you could not enter third-level characters using the AltGr key such as ‘|’ (pipe) in some keyboard layouts. Now Alt+Gr-<character> combinations work properly in the web console pod terminal. (BZ#1292507)

In the web console, copying and pasting content from the terminal could result in extra spaces being added to the end of each line. Now when you copy content from the terminal, no extra spaces are added. (BZ#1395564)

The left navigation column did not support vertical scrolling. When the browser viewport was less than 440 pixels tall and wider than 768 pixels the bottom left navigation link was not accessible. The new left navigation column markup supports vertical scrolling. Now, all left navigation links are accessible at all browser viewport sizes and zoom levels. (BZ#1375134)

Previously, on iOS Safari, number inputs used the full keyboard rather than the number input. Now inputs that accept only numbers show the iOS number pad for easier entry. (BZ#1470976)

Previously, some requests for templates in the web console could timeout or take a long time to complete over high latency network connections. This could cause an error when loading the Add to Project page. The web console can now load templates using much less data, which fixes the problem. (BZ#1471033)

Clarifies help text on the Route creation and editing pages to make it clear that the CA certificates should be certificate chains. (BZ#1471155)

A known bug in Internet Explorer resulted in the layout of pod charts overflowing their containers on the overview page. As a result, the pod charts looked mis-aligned in the UI. The fix involved increasing the specificity on some CSS declarations so that they only apply when they are needed, which is during a deployment when the pod charts are being animated. As a result, the pod charts appear correctly aligned in Internet Explorer. (BZ#1473512)

A known bug in Internet Explorer resulted in the layout of catalog items taking up too much space. As a result, not all the catalog items were visible in Internet Explorer. The fix involved adding an additional CSS declaration as a workaround for IE. As a result, the catalog items now take up the correct space in IE. (BZ#1473615)

The code was using an empty envFrom entry when creating/editing the environment variable, causing a validation failure when adding or editing an environment variable using Deployment Configuration page of the web console. The user would receive an error that the deployment configuration is invalid. The envFrom entry is now properly submitted and the user can add or edit environment variables from the web console. (BZ#1502914)

Various errors were present in the source code that prevented Config maps were not available from the drop-down menu on the Edit Deployment Config page for pre and post-hooks when using Add Value from Config Map or Secret. These errors have been corrected. Config maps appear in the appropriate drop-downs. (BZ#1502914)

Previously, secrets with null values would display incorrectly when values were revealed on the secret details page. Now the web console will correctly display the secret key as having no value. (BZ#1510346)

Previously there was a quirk in the drag-and-drop behavior of the key value editor. While reordering an env var it might jump more than a single node at a time. This bug fix ensures that the drag-and-drop behavior will behave as expected. (BZ#1428991)

On the project overview, the Application drop-down menu was incorrectly set to overflow:hidden. As a result, when the application row is collapsed, the menu did not display fully. The overflow: hidden parameter has been removed and the menu is now fully visible. (BZ#1460153)

Previously, deleting a service account would ignore the SAs namespace. This means that the delete action from the web UI could delete multiple service account rolebindings under the service account tab if service accounts from different namespaces had the same name. The delete action on the SA tab will now respect the namespace and only delete the specified SA rolebinding from the correct namespace. (BZ#1507730)

The Configuration tab of the Deployment page in the web console was laid out in such a way that a large gap could appear when the right column contents were longer than the left column contents. The fix involved changing the layout markup so the gap does not appear. The result is there is no longer a gap between Volumes and Triggers when the right column content is longer than the left column content. (BZ#1505255)

Ansible installs with a caBundle on the service catalog API service resulting in a 500 Internal Server Error on the product overview page in the web console. The installer was changed to install with insecureSkipTLSVerify flag set to true. As a result, the product overview page works as expected. (BZ#1473523)

CronJobs are placed in batch/v2alpha1 group, whereas other batch resources are placed in batch/v1. Due to this fact, some API machinery does not handle multiversioning problems properly. The restmapper, which is responsible for matching resource with appropriate api group version to handle multi-versioned apis, was updated. Describing resources works as expected. (BZ#1480453)

The installer was configured to watch specific resources that do not support watching. As a result, the /var/log/messages file was reporting errors and warnings related to the issue. The installer has been corrected to not watch these resources and the errors/warnings are not generated. (BZ#1452206)

Creating project using project template does not use the substituted project name, but the namespace name. As a result, the user is not able to use parametrized name as a project name as the generated suffix or prefix might be dropped. The code was changed to allow the use of substituted project name when creating the namespace. (BZ#1454535)

Node status information was getting rate limited during heavy traffic causing some nodes to fall into not ready status. The code was changed to use a separate connection for node healthiness. As a result, node status is reported without any problems. (BZ#1464653)

Running multiple clusters in a single authorization zone in AWS requires resources be tagged. If the clusters are not tagged, the clusters will not work properly. The master controllers process will require a ClusterID on resources in order to run. Existing resources will need to be tagged manually. Multiple clusters in one az will work properly once tagged. (BZ#1468579)

An upstream patch caused an error with the oc apply command. The patch deleted an element from an array (eg. env) and then reordered or modified another array (eg. volumeMounts). The kubectl apply fails with the _unable to find api field in struct Container for the json field "$setElementOrder/env". The algorithm was updated so that it continues operation under described condition. The oc apply works without any problems. (BZ#1497325)

When either a certificate within the chain at serviceaccount/ca.crt or any of the certificates within the provided truststore file contain white space after the BEGIN CERTIFICATE declaration, the Java keytool rejects the certificate with an error, causing Origin Metrics to fail to start. As a workaround, Origin Metrics will now attempt to remove the spaces before feeding the certificate to the Keytool. Admins should ensure their certificates don’t contain such spaces. (BZ#1503450)

When deleting a large number of pods, the hawkular-metrics pod log reports Pool is busy errors. The condition was fixed upstream in Cassandra and clusters with a large number of pods should not report the Pool is busy error. (BZ#1451209)

When opening the metrics page in a disconnected environment, Hawkular attempted to connect to external web sites, such asfonts.googleapis.com. Because the cluster cannot connect to Internet, the metrics page loaded slowly. Changes were made upstream so that Hawkular does not attempt to connect to external web sites when there is no access to the Internet. As a result, in a disconnected environment, the metrics page loads properly. (BZ#1466403)

In Cassandra, it is possible that new generation objects (with the -Xmn flag) can exceed the maximum size of the Java memory heap (with the -Xmx flag). If that happens, the JVM will log a warning at start up, but Cassandra still starts. The code was changed to set the size of new generation objects at ¼ of the maximum heap size. (BZ#1471239)

Cassandra metrics would not start up if the commit log exceeded the limit applied to the log. An out-of-memory (OOM) condition would cause metrics to constantly start and stop. The commit log size is now based on total available memory. Also, log compression is no longer used, which will reduce the demand on resources. As a result, large logs should not affect metrics operation. (BZ#1473013)

When changes are made to software defined network (SDN) plugin, the master controller will fail to start when there are headless services in the cluster. As a result, when initializing OpenShift Container Platform, SDN fails to allow a nil service IP and OpenShift Container Platform was unable to start. The code was changed to allow nil as a valid value of srv.Spec.ClusterIP. OpenShift Container Platform SDN properly starts after changing network with headless service. (BZ#1451881)

The nodes local IP address is not part of the Open vSwitch (OVS) rules. If you deny 0.0.0.0/0 and allow a DNS name in the egress network policy, the node will not be able to reach that allowed address because DNS name resolution is blocked Adding the local node IP to the ovs allowed rule so that the name resolution will not be blocked. Also adding a note to the docs for the case when dns resolution does not happen on the node. OpenShift Container Platform can successfully block 0.0.0.0/0 as a cidrSelector and allow specific DNS names through. (BZ#1458849)

If the service network restart command is executed on a machine while the OpenShift Container Platform node process is running, a stop() function properly disables IP forwarding. However, the start() function was not re-enabling it. The code was changed to persist IP forwarding on nodes during network restarts. (BZ#1477716)

While upgrading nodes, if any invalid network CIDRs are detected, nodes might be unable to upgrade and will fail. The code was changed to not fail with invalid CIDRs. (BZ#1506017)

The Kubernetes CNI (Container Network Interface) plug-in generates errors if hostNetwork=true is configured for pods. This issue has been fixed. (BZ#1507257)

Because of upstream issues in Kubernetes, vSphere had networking problems when used with OpenShift Container Platform. The periodic resync of Kubernetes into OpenShift Container Platform included the required changes. vSphere now works correctly. (BZ#1433236)

Because of changes with upstream Kubernetes, the oc adm join-projects, oc adm isolate-projects and other commands that depend on the pod update operation will not work. The code was changed to fetch some required elements from the Container Runtime Interface (CRI) directly. As a result, the pod update operation works correctly and the commands work as expected. (BZ#1453190)

Because of default authorization, project administrators (standard user) were not able to manage network policies for their own projects. Changes to the code now allow project admins to create, delete, list the network polices in their own projects. (BZ#1461208)

An invalid HostSubnet could not be fixed. As a result, if a node with an invalid HostSubnet is restarted, the node assigned to the HostSubnet, would fail to start. The code has been changed to allow an invalid HostSubnet to be changed, using commands such as oc edit hostsubnet. (BZ#1466239)

Adding an IPv6 address to a host subnet as an egress resulted in a panic error. The code has been changed to better handle IPv6 addresses with a meaningful error message. (BZ#1500664)

Using ipfailover when a node fails ensures that a second node receives traffic. Previously, traffic went back to the first node once it is back up, potentially causing traffic imbalance. Now, using the --preemption-strategy="nopreempt" option, allows the administrator to control the default strategy, meaning that the strategy to switch to a higher priority node is suppressed. (BZ#1465987)

A log message similar to the following was repeatedly appearing:

LoadBalancerRR: Removing endpoints for ops-health-monitoring/pull-07062050z-ie:8080-tcp

Previously, the image for the default network diagnostics pod was mismatched, causing the diagnostics to fail. The image checking has been fixed, and the network diagnostics works without errors. (BZ#1481147)

Previously, conntrack entries for UDP traffic were not erased when an endpoint was added for a service that previously had no endpoints. This meant that the system could end up incorrectly caching a rule that would cause traffic to that service be dropped rather than being send to the new endpoint. The relevant conntrack entries have been changed to be deleted at the right time, meaning that the UDP services work correctly when endpoints are added and removed. (BZ#1487438)

Previously, network debug tests were showing errors regarding not being able to read stats from a changing pod. This was because, even though the container process had exited, but the cgroup wasn’t removed, leading to a Docker container with no tasks. The log spam has been reduced. (BZ#1328913)

Because of an outdated Go format, kubemarl-scale was consistently failing. The version of Golang was updated, stopping the failures. (BZ#1454239)

Previously, the HPA V1 was unable to get the metrics from the resource CPU. This was due to the custom setup of the HPA controller changing. The settings have been restored. (BZ#1458663)

Previously, multi-node environments produced “Failed to watch” errors. This was because the controller didn’t have permission to watch resources, which meant its behaviour was to retry every second by default. The controller has been given the permission to watch resources. (BZ#1465361)

Previously, the OpenShift master failed to start when using Openstack integration without Neutron LBaaS, which is not available in OpenShift. The issue now gives a warning instead of a failure, which mean the master will start successfully even if the LBaaS is not available. (BZ#1465722)

Previously, project volumes were not included in security context constraints, meaning that pods could not be used with projected volumes. The projected volumes have been added to the correct SCCs, and the projected volumes can be used as expected. (BZ#1448816)

Init containers with resource requests or limits were producing error messages. This was due to a mismatch in the sum of a pod’s container resources, resulting in the parent cgroup choosing the incorrect resource. The issue has been fixed upstream and the correct resources are being chosen. (BZ#1459826)

Previously, when a deployment configuration was created without any memory information when quota restrictions were in place, no error message would appear. The expected results were a “FailedCreate” event, much like with replication controllers. The “FailedCreate” event now appears when the pod immediately fails. (BZ#1465801)

A design limitation in previous versions does not account for memory-backed volumes against the pod’s cumulative memory limit. So, it is possible for a user to exhaust memory on the node by creating a large file in an memory-backed volume, regardless of the memory limit. Now, pod-level cgroups have been added to, among other things, enforce limits on memory-backed volumes, resulting in memory-backed volume sizes now being bound by cumulative pod memory limits. (BZ#1422049)

Previously, upgrading to 3.4 gave a “insufficient pods” error. This was due to a change in configuration from a max-pods variable to the smaller of 250 or 10 pods per core. The upgrade broke installations with fewer pods. The change has been made so that the max-pods variable has become the limiting variable. (BZ#1430484)

Previously, error messages in the status field of failed builds said “error” instead of an actual error message. This was because the status was showing the message from the Docker daemon returning the failed pod message. The message now returns a more helpful error message. (BZ#1449820)

Previously, registry pods were occasionally reporting liveness and readiness probe failures with the message http2: no cached connection was available. This was due to an upstream issue where the liveness and readiness probes get in the way of each other. The problem has been fixed upstream, and updated for OpenShift Container Platform version 3.7. (BZ#1454858)

Large clusters with a large amount of HPAs or unhealthy pods sent a large number of events if an object was unable to reach its desired state. This bug fix updates the event client to protect against spamming master components. As a result, this controls traffic to the masters and reduces writed to etcd. (BZ#1466933)

For all resources other than pod or PVCs, the quota controller would make a LIST call per namespace to determine current usage counts. This caused quota recalculation to take an extended period of time. This bug fix reduces LIST calls made by the resource quota controller by using shared informer caches. As a result, LIST operations made to the master were reduced and information was pulled from a shared cache in the controller. (BZ#1473370)

Previously, users were not able to to look up PVC information for the Drupal database without receiving scheduler log spam. This bug fix prevents unnecessary logging of a harmless error from a PVC-related scheduler predicate. (BZ#1475558)

Previously, messages originating from the AWS SDK were causing partial log entries due to new lines in the message itself. Error messages are now properly quoted so all messages are (BZ#1462445)

Previously, the help information included a redundant example. This bug fix removed the redundant example. As a result, the help information is now more concise. (BZ#1440620)

Previously, the code path automatically prepended the partition name to the vserver name. If the vserver was in a path of length more than 1, then the path was lost because only the partition name was prepended. This bug fix prepends the entire path of vserver instead of just concatenating the partition name and vserver name. (BZ#1465304)

Previously, if you had a router of a previous version of OpenShift Container Platform a 403 http status resulted when the router stats were accessed without credentials. This web browser did not prompt the user for a password so the stats were inaccessible. The code has been updated to return a 403 when no credentials are passed and the browser now prompts the user for a password, so the router stats are visible in a web browser. (BZ#1467257)

Previously, the IP failover keepalived image did not support IPV6 addresses or ranges, as well as IP address validation. Adding IPV6 addresses to the oc adm ipfailover command resulted in a new vrrp section pertaining to the wrong address. The code has been updated, and inputting invalid IPV4 and IPV6 addresses now return an error as expected. (BZ#1459960)

Previously, the x-forwarded header and its associated information, displayed the IPV6 form in IPV4 form. The ROUTER_IP_V4_V6_MODE environment variable has been created to control which form is displayed. (BZ#1471255)

Previously, the locking was overly broad, causing events to not be processed while an HAProxy reload was happening. This meant that route changes would take hours to process. The locking has been made more fine-grained, so that events can be processed in parallel. And changes are now processed within the time of two reloads of the router. (BZ#1471899)

An error in the router code caused by a missing locking around a router data structure was causing errors causing the router pod to occasionally crash and restart. The locking has been fixed, and the router now works as expected. (BZ#1473031)

When running the oc adm router --expose-metrics command, the router deployment failed because the generated deployment configuration object was not compatible. This was due to a background change upstream. A change has been made with the oc adm router command, and the command can now handle --expose-metrics. (BZ#1488954)

Previously, multiple service catalog objects named “default” were not a problem, but a change made them all top level. This bug fixes the object names to be unique. (BZ#1420543)

Previously, a fresh installation using the openshift-ansible method and with a service-catalog resulted in the service class being empty, resulting in the stage registry giving a bad response. The administrator would need to see the ASB logs and trigger a manual bootstrap. Now, if the bootstrap fails, the broker fails, and the kubelet retries the process until it works correctly. (BZ#1468173)

This bug fixes running the service-catalog binaries for the apiserver and controller manager when used with the --version option, which previously reported UNKNOWN, but now reports the correct value. (BZ#1476134, BZ#1475251)

Previously, when deleting a namespace, the Ansible Service Broker (ASB) attempted to execute deprovision playbook actions using a namespace in a "terminating" state. This led to the APB actions being rejected, because of the namespace terminating. As a result, deprovision fails, and both the APB deprovision sandbox and target namespace were not deleted. Now, instead of executing APB actions on namespace deletion, the records of the services to be deprovisioned are cleaned up, allowing kubernetes to delete the resources normally, meaning the target namespace is properly deleted by Kubernetes. (BZ#1476173)

The error message returned when a user does not have permission to modify a TemplateInstance is updated. (BZ#1460145)

Previously, only one annotation returned when both expose and base64-expose annotations were defined in template (per bind request). This issue is fixed in the latest release. (BZ#1463570)

Previously, Ansible Playbook Bundles (APB) that have been removed from their container catalog, appeared in Ansible Service Broker (ASB) as valid options even after bootstrap was performed. This issue is fixed now. (BZ#1463798)

Previously, there were inconsistency between the serviceclass and the server-broker. After creating a broker, the controller-manager only fetched the catalog once. This resulted in inability to updates the serviceclass unless the broker was recreated. This is fixed now. (BZ#1469448)

Previously, the Ansible service broker would fail on provisioning because of incorrect permissions. This is now fixed and Ansible service broker now has the required permissions for creating new namespaces and dynamic service account in these new namespace to run APBs. (BZ#1469485)

The oc version command did not get OpenShift version against the ansible deployed service catalog environment. The version information is added the command now reports correct information. (BZ#1471717)

Previously, when the Ansible Service Broker started it could not communicate to the configured registry, and therefore got no information about APBs. This was because of a missing setting in the ansible service broker configuration. The broker: bootstrap_on_startup: true setting is now added in the configuration which resolves this issue. (BZ#1471973)

Previously, the ansible service broker container would fail if the dockerhub credentials were not supplied because the encryption script required them. It is now reconfigured to use RHCC adapter and the dockerhub credentials are optional. (BZ#1464222)

Previously, bad data was being returned from the bootstrapped registry. This was because the broker failed to bootstrap and it used to error out due to a null pointer de-reference. The broker now has logic to avoid de-referencing null pointers if the data is corrupted. This issue is now resolved and the broker skips image with bad data and continues with next one. (BZ#1467905)

The Service Broker Installer was setting incorrect configuration values for launchapbonbind, this is fixed and configuration value is now set as launch_apb_on_bind. (BZ#1467948)

The role for Service Accounts used by the Ansible Service Broker is updated. The Broker runs under asb service account set to admin through a ClusterRoleBinding and APBs run under a temporary service account granted edit through a RoleBinding in the target namespace. (BZ#1470824)

Creating a new persistent volume claim (PVC) using OpenStack Cinder storageclass resulted in the PVC being stuck in Pending state. This bug fix re-configured the cloud provider openstack.conf to use OpenStack Keystone V3. As a result, dynamic provisioning of new Cinder volumes works as documented. (BZ#1491331)

Previously, the Gophercloud library used by OpenShift to communicate with the OpenStack API did not accept HTTP status 300 in pagination. It was not possible to dynamically provision OpenStack Cinder volumes. This bug fix upgrades the Gophercloud library in the OpenShift vendor directory. As a result, dynamic provisioning of new Cinder volumes works as documented. (BZ#1490768)

Previously, the default bootstrap policy allowed basic users to “get” storage classes, but not “list” storage classes. Basic users would receive an error message after issuing the oc get storagelcass storageclass_name command. This bug fix modified the bootstrap policy. As a result, basic users can now issue the oc get storagelcass storageclass_name command to receive specific storage classes. (BZ#1449608)

Previously, the lack of cloud provider configuration in the admission plug-in caused persistent volume (PV) creation to fail when attempting to create the PV in a zone other than master. This bug fix enables static PV provisioning in multizone environments. As a result, users can now statically provision PVs in zones other than master. (BZ#1454601)

Previously, when creating storage classes, users could not specify the fstype. This bug fix allows specifying the desired fstype when dynamically provisioning volumes with storage classes. As a result, storage classes now support file system configuration when creating dynamically provisioned volumes. (BZ#1469001)

Previously, it was not possible to dynamically provision ScaleIO volumes if the ScaleIO volume plug-in was not enabled. This bug fix enables the ScaleIO volume plug-in in OpenShift Container Platform 3.7. As a result, it is now possible to dynamically provision ScaleIO volumes. (BZ#1482274)

When trying to mount/unmount, the FlexVolume plug-in’s file system previously assumed that SELinux was supported. This assumption instructed docker to relabel the volume. If the FlexVolume plugin’s file system did not support file system relabeling, the container using the FlexVolume would fail to start. This bug fix added the selinuxRelabel capability, which allows FlexVolume plug-ins to report in their init call. As a result, FlexVolume plug-ins can now be configured to opt out of SELinux relabeling. (BZ#1484899)

Previously, the service catalog could not provide authentication when invoking the template service broker, which meant the template service broker API had to allow calls from unauthenticated clients. This bug fix allows the service catalog to use proper authentication to invoke the template service broker when issuing the oc cluster up command to run both. As a result, the template service broker APIs will now be secured, and will only be invokable by the service catalog (or another client with appropriate credentials). (BZ#1470628)

Previously, the master node upgrade took more disk space than was initially estimated. This caused the etcd member to report a no space left on device error message. This bug fix increased the estimation of disk space needed before the master node upgrade can start. As a result, a master node is properly upgraded with enough disk space left after the upgrade finishes. (BZ#1489182)

Previously, the upgrade playbooks incorrectly overwrote nondefault admissionConfig parameters while setting specific values required of the upgrade process. This bug fix removed this task as it is no longer necessary after upgrading from OpenShift Container Platform 3.4 to OpenShift Container Platform 3.5. (BZ#1486054)

Previously, the etcd v3 data migrated prior to the first etcd v2 snapshot being written. Without a v2 snapshot, the v3 data was not propagated properly to the remaining etcd members, which resulted in a loss of some v3 data. This bug fix checks to see if there is at least one v2 snapshot before etcd data migration proceeds. As a result, etcd v3 data is now properly distributed among all etcd members. (BZ#1501752)

When trying to upgrade OpenShift Container Platform with dedicated etcd from v3.6 to v3.7, the upgrade failed at the [Stop atomic-openshift-master-controllers] task due to the wrong hosts group. This bug fix corrected the host group to specify the masters group for controller restart. As a result, the upgrade now succeeds. (BZ#1504515)

Previously, if Ansible tags were used to evaluate some of the tasks in a set of playbooks, the conditional for including a task file was not properly evaluated. This caused the upgrade to fail. This bug fix allows the conditional to evaluate properly and skip running the task. (BZ#1464025)

Ansible playbooks now exit immediately when health checks fail. Previously, in some instances, a host failure would not result in the playbook exiting during failed health checks. This bug fix sets the any_errors_fatal play option to true, ensuring that the playbook exits as expected. (BZ#1484324)

Upgrades that made use of system reboots to restart services may have failed if hosts took longer than 5 minutes to restart. This bug fix increases the timeout to 10 minutes. As a result, the shutdown process is now faster. (BZ#1455836)

Prometheus Cluster Monitoring

Local Storage Persistent Volumes

CRI-O

Tenant-driven Storage Snapshotting

CLI Plug-ins

Enabling Static IPs for External Project Traffic

Service Catalog

Template Service Broker

OpenShift Ansible Broker

Ansible Playbook Bundles

Network Policy

Initial Experience

Add from Catalog and Add to Project

Search Catalog

Automated installation of CloudForms Inside OpenShift

Cron Jobs (formerly called Scheduled Jobs)

Kubernetes Deployments Support

StatefulSets, formerly known as PetSets

Require Explicit Quota to Consume a Resource

Mount Options

Installation of etcd, Docker Daemon, and Ansible Installer as System Containers

Running OpenShift Installer as a System Container

Integrated Approach to Adding Hawkular OpenShift Agent

Bind in Context

mux

When users are redirected to an external domain after successfully logging in, users should only be redirected to specific URLs. This is a known issue and will be fixed in OpenShift Container Platform 3.9. (BZ1525538)

The installer can not deploy system container-based installations when the specified registry requires authentication credentials in order to pull the required system container images. The fix for this depends on an update to the atomic command, which will be updated after OpenShift Container Platform 3.7 GA. (BZ#1505744)

A OpenShift Container Platform 3.7 master will return an unstructured response instead of structured JSON when an action is forbidden. This is a known issue and will be fixed in OpenShift Container Platform 3.8.

The volume snapshot Technology Preview feature may not be available to non-administrator users by default due to API RBAC settings. When the volume snapshot controller and provisioner are installed and run, the cluster administrator needs to configure the API access to the VolumeSnapshot objects by creating roles and cluster roles, then assigning them to the desired users or user groups. (BZ#1502945)

OpenShift Container Platform is unable to list known health checks. (BZ#1509157)

The current format of audit logs is difficult to consume. Some keys are duplicates and some are misleading in that they match wrong keys in the linux-audit dictionary. (BZ#1496176)

Limit NFS provisioning of the Quick Installer to hosts that are part of the cluster. If a host is outside of the cluster, where Ansible will not run, use the Advanced Installation method.