Red Hat JBoss Data Grid is available as a containerized xPaaS image that is designed for use with OpenShift. This image provides an in-memory distributed database so that developers can quickly access large amounts of data in a hybrid environment.
|
There are significant differences in supported configurations and functionality in the JBoss Data Grid xPaaS image compared to the full, non-PaaS release of JBoss Data Grid. |
This topic details the differences between the JBoss Data Grid xPaaS image and the full, non-PaaS release of JBoss Data Grid, and provides instructions specific to running and configuring the JBoss Data Grid xPaaS image. Documentation for other JBoss Data Grid functionality not specific to the JBoss Data Grid xPaaS image can be found in the JBoss Data Grid documentation on the Red Hat Customer Portal.
|
OpenShift Container Platform release 3.1 supports JBoss Data Grid release 6.6 and earlier. If you are using OpenShift release 3.2 and JBoss Data Grid release 7.1 or later, refer to the documentation in the Red Hat Customer Portal. |
There are several major functionality differences in the OpenShift JBoss Data Grid xPaaS image:
The JBoss Data Grid Management Console is not available to manage OpenShift JBoss Data Grid xPaaS images.
The JBoss Data Grid Management CLI is only bound locally. This means that you can only access the Management CLI of a container from within the pod.
Library mode is not supported.
Only JDBC is supported for a backing cache-store. Support for remote cache stores are present only for data migration purposes.
Clustering is achieved through one of two discovery mechanisms: Kubernetes or DNS. This is accomplished by configuring the JGroups protocol stack in clustered-openshift.xml with either the <openshift.KUBE_PING/> or <openshift.DNS_PING/> elements. By default KUBE_PING is the pre-configured and supported protocol.
For KUBE_PING to work the following steps must be taken:
The OPENSHIFT_KUBE_PING_NAMESPACE environment variable must be set (as seen in the Configuration Environment Variables). If this variable is not set, then the server will act as if it is a single-node cluster, or a cluster that consists of only one node.
The OPENSHIFT_KUBE_PING_LABELS environment variable must be set (as seen in the Configuration Environment Variables). If this variable is not set, then pods outside the application (but in the same namespace) will attempt to join.
Authorization must be granted to the service account the pod is running under to be allowed to Kubernetes' REST api. This is done on the command line:
Using the default service account in the myproject namespace:
oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default -n $(oc project -q)
Using the eap-service-account in the myproject namespace:
oc policy add-role-to-user view system:serviceaccount:$(oc project -q):eap-service-account -n $(oc project -q)
Once the above is configured images will automatically join the cluster as they are deployed; however, removing images from an active cluster, and therefore shrinking the cluster, is not supported.
Clients can access JBoss Data Grid via REST, HotRod, and memcached endpoints defined as usual in the cache’s configuration.
If a client attempts to access a cache via HotRod and is in the same project it will be able to receive the full cluster view and make use of consistent hashing; however, if it is in another project then the client will unable to receive the cluster view. Additionally, if the client is located outside of the project that contains the HotRod cache there will be additional latency due to extra network hops being required to access the cache.
| Only caches with an exposed REST endpoint will be accessible outside of OpenShift. |
A list of caches may be defined by the CACHE_NAMES environment variable. By default the following caches are created:
default
memcached
Each cache’s behavior may be controlled through the use of cache-specific environment variables, with each environment variable expecting the cache’s name as the prefix. For instance, consider the default cache, any configuration applied to this cache must begin with the DEFAULT_ prefix. To define the number of cache entry owners for each entry in this cache the DEFAULT_CACHE_OWNERS environment variable would be used.
A full list of these is found at Cache Environment Variables.
Datasources are automatically created based on the value of some environment variables.
The most important variable is the DB_SERVICE_PREFIX_MAPPING which defines JNDI mappings for datasources. It must be set to a comma-separated list of <name><database_type>=<PREFIX> triplet, where *name is used as the pool-name in the datasource, database_type determines which database driver to use, and PREFIX is the prefix used in the names of environment variables, which are used to configure the datasource.
For each <name>-database_type>=PREFIX triplet in the DB_SERVICE_PREFIX_MAPPING environment variable, a separate datasource will be created by the launch script, which is executed when running the image.
The <database_type> will determine the driver for the datasource. Currently, only postgresql and mysql are supported.
The <name> parameter can be chosen on your own. Do not use any special characters.
| The first part (before the equal sign) of the DB_SERVICE_PREFIX_MAPPING should be lowercase. |
The JBoss Data Grid xPaaS image contains Java drivers for MySQL, PostgreSQL, and MongoDB databases deployed. Datasources are generated only for MySQL and PostGreSQL databases.
| For MongoDB databases there are no JNDI mappings created because this is not a SQL database. |
The following examples demonstrate how datasources may be defined using the DB_SERVICE_PREFIX_MAPPING environment variable.
Consider the value test-postgresql=TEST.
This will create a datasource named java:jboss/datasources/test_postgresql. Additionally, all of the required settings, such as username and password, will be expected to be provided as environment variables with the TEST_ prefix, such as TEST_USERNAME and TEST_PASSWORD.
Multiple database mappings may also be specified; for instance, considering the following value for the DB_SERVICE_PREFIX_MAPPING environment variable: cloud-postgresql=CLOUD,test-mysql=TEST_MYSQL.
| Multiple datasource mappings should be separated with commas, as seen in the above example. |
This will create two datasources:
java:jboss/datasources/test_mysql
java:jboss/datasources/cloud_postgresql
MySQL datasource configuration, such as the username and password, will be expected with the TEST_MYSQL prefix, for example TEST_MYSQL_USERNAME. Similarly the PostgreSQL datasource will expect to have environment variables defined with the CLOUD_ prefix, such as CLOUD_USERNAME.
A full list of datasource environment variables may be found at Datasource Environment Variables.
To configure a new Security Domain the SECDOMAIN_NAME environment variable must be defined, which will result in the creation of a security domain named after the passed in value. This domain may be configured through the use of the Security Environment Variables.
A major difference in managing an OpenShift JBoss Data Grid xPaaS image is that there is no Management Console exposed for the JBoss Data Grid installation inside the image. Because images are intended to be immutable, with modifications being written to a non-persistent file system, the Management Console is not exposed.
However, the JBoss Data Grid Management CLI (JDG_HOME/bin/jboss-cli.sh) is still accessible from within the container for troubleshooting purposes.
First open a remote shell session to the running pod:
$ oc rsh <pod_name>
Then run the following from the remote shell session to launch the JBoss Data Grid Management CLI:
$ /opt/datagrid/bin/jboss-cli.sh
| Any configuration changes made using the JBoss Data Grid Management CLI on a running container will be lost when the container restarts. |
Making configuration changes to the JBoss Data Grid instance inside the JBoss Data Grid xPaaS image is different from the process you may be used to for a regular release of JBoss Data Grid.
The Red Hat xPaaS middleware images were automatically created during the installation of OpenShift along with the other default image streams and templates.
You can make changes to the JBoss Data Grid configuration in the xPaaS image using either the S2I templates, or by using a modified JBoss Data Grid xPaaS image.
The recommended method to run and configure the OpenShift JBoss Data Grid xPaaS image is to use the OpenShift S2I process together with the application template parameters and environment variables.
The S2I process for the JBoss Data Grid xPaaS image works as follows:
If there is a pom.xml file in the source repository, a Maven build is triggered with the contents of $MAVEN_ARGS environment variable.
By default the package goal is used with the openshift profile, including the system properties for skipping tests (-DskipTests) and enabling the Red Hat GA repository (-Dcom.redhat.xpaas.repo.redhatga).
The results of a successful Maven build are copied to JDG_HOME/standalone/deployments. This includes all JAR, WAR, and EAR files from the directory within the source repository specified by $ARTIFACT_DIR environment variable. The default value of $ARTIFACT_DIR is the target directory.
Any JAR, WAR, and EAR in the deployments source repository directory are copied to the JDG_HOME/standalone/deployments directory.
All files in the configuration source repository directory are copied to JDG_HOME/standalone/configuration.
| If you want to use a custom JBoss Data Grid configuration file, it should be named clustered-openshift.xml. |
All files in the modules source repository directory are copied to JDG_HOME/modules.
The JBoss Data Grid xPaaS image may come with multiple versions of OpenJDK installed, but only one is the default. For example, the JBoss Data Grid 6.5 xPaaS image comes with OpenJDK 1.7 and 1.8 installed, but OpenJDK 1.8 is the default.
If you want the JBoss Data Grid xPaaS image to use a different JDK version than the default, you must:
Ensure that your pom.xml specifies to build your code using the intended JDK version.
In the S2I application template, configure the image’s JAVA_HOME environment variable to point to the intended JDK version. For example:
name: "JAVA_HOME"
value: "/usr/lib/jvm/java-1.7.0"An alternative method is to make changes to the image, and then use that modified image in OpenShift.
The JBoss Data Grid configuration file that OpenShift uses inside the JBoss Data Grid xPaaS image is JDG_HOME/standalone/configuration/clustered-openshift.xml, and the JBoss Data Grid startup script is JDG_HOME/bin/openshift-launch.sh.
You can run the JBoss Data Grid xPaaS image in Docker, make the required configuration changes using the JBoss Data Grid Management CLI (JDG_HOME/bin/jboss-cli.sh), and then commit the changed container as a new image. You can then use that modified image in OpenShift.
| It is recommended that you do not replace the OpenShift placeholders in the JBoss Data Grid xPaaS configuration file, as they are used to automatically configure services (such as messaging, datastores, HTTPS) during a container’s deployment. These configuration values are intended to be set using environment variables. |
| Ensure that you follow the guidelines for creating images. |
The following information environment variables are designed to convey information about the image and should not be modified by the user:
| Variable Name | Description | Value |
|---|---|---|
JBOSS_DATAGRID_VERSION |
The full, non-PaaS release that the xPaaS image is based from. |
6.5.1.GA |
JBOSS_HOME |
The directory where the JBoss distribution is located. |
/opt/datagrid |
JBOSS_IMAGE_NAME |
Image name, same as Name label |
jboss-datagrid-6/datagrid65-openshift |
JBOSS_IMAGE_RELEASE |
Image release, same as Release label |
Example: dev |
JBOSS_IMAGE_VERSION |
Image version, same as Version label |
Example: 1.2 |
JBOSS_MODULES_SYSTEM_PKGS |
org.jboss.logmanager |
|
JBOSS_PRODUCT |
datagrid |
|
LAUNCH_JBOSS_IN_BACKGROUND |
Allows the data grid server to be gracefully shutdown even when there is no terminal attached. |
true |
Configuration environment variables are designed to conveniently adjust the image without requiring a rebuild, and should be set by the user as desired.
| Variable Name | Description | Value |
|---|---|---|
CACHE_CONTAINER_START |
Should this cache container be started on server startup, or lazily when requested by a service or deployment. Defaults to LAZY |
Example: EAGER |
CACHE_CONTAINER_STATISTICS |
Determines if the cache container collects statistics. Disable for optimal performance. Defaults to true. |
Example: false |
CACHE_NAMES |
List of caches to configure. Defaults to default,memcached, and each defined cache will be configured as a distributed-cache with a mode of SYNC. |
Example: addressbook,addressbook_indexed |
CONTAINER_SECURITY_CUSTOM_ROLE_MAPPER_CLASS |
Class of the custom principal to role mapper. |
Example: com.acme.CustomRoleMapper |
CONTAINER_SECURITY_IDENTITY_ROLE_MAPPER |
Set a role mapper for this cache container. Valid values are: identity-role-mapper,common-name-role-mapper,cluster-role-mapper,custom-role-mapper. |
Example: identity-role-mapper |
CONTAINER_SECURITY_ROLES |
Define role names and assign permissions to them. |
Example: admin=ALL,reader=READ,writer=WRITE |
DB_SERVICE_PREFIX_MAPPING |
Define a comma-separated list of datasources to configure. |
Example: test-mysql=TEST_MYSQL |
DEFAULT_CACHE |
Indicates the default cache for this cache container. |
Example: addressbook |
ENCRYPTION_REQUIRE_SSL_CLIENT_AUTH |
Whether to require client certificate authentication. Defaults to false. |
Example: true |
HOTROD_AUTHENTICATION |
If defined the hotrod-connectors will be configured with authentication in the ApplicationRealm. |
Example: true |
HOTROD_ENCRYPTION |
If defined the hotrod-connectors will be configured with encryption in the ApplicationRealm. |
Example: true |
HOTROD_SERVICE_NAME |
Name of the OpenShift service used to expose HotRod externally. |
Example: DATAGRID_APP_HOTROD |
INFINISPAN_CONNECTORS |
Comma separated list of connectors to configure. Defaults to hotrod,memcached,rest. Note that if authorization or authentication is enabled on the cache then memcached should be removed as this protocol is inherently insecure. |
Example: hotrod |
JAVA_OPTS_APPEND |
The contents of JAVA_OPTS_APPEND is appended to JAVA_OPTS on startup. |
Example: -Dfoo=bar |
JGROUPS_CLUSTER_PASSWORD |
A password to control access to JGroups. Needs to be set consistently cluster-wide. The image default is to use the OPENSHIFT_KUBE_PING_LABELS variable value; however, the JBoss application templates generate and supply a random value. |
Example: miR0JaDR |
MEMCACHED_CACHE |
The name of the cache to use for the Memcached connector. |
Example: memcached |
OPENSHIFT_KUBE_PING_LABELS |
Clustering labels selector. |
Example: application=eap-app |
OPENSHIFT_KUBE_PING_NAMESPACE |
Clustering project namespace. |
Example: myproject |
PASSWORD |
Password for the JDG user. |
Example: p@ssw0rd |
REST_SECURITY_DOMAIN |
The security domain to use for authentication and authorization purposes. Defaults to none (no authentication). |
Example: other |
TRANSPORT_LOCK_TIMEOUT |
Infinispan uses a distributed lock to maintain a coherent transaction log during state transfer or rehashing, which means that only one cache can be doing state transfer or rehashing at the same time. This constraint is in place because more than one cache could be involved in a transaction. This timeout controls the time to wait to acquire a distributed lock. Defaults to 240000. |
Example: 120000 |
USERNAME |
Username for the JDG user. |
Example: openshift |
The following environment variables all control behavior of individual caches; when defining these values for a particular cache substitute the cache’s name for CACHE_NAME.
| Variable Name | Description | Example Value |
|---|---|---|
<CACHE_NAME>_CACHE_TYPE |
Determines whether this cache should be distributed or replicated. Defaults to distributed. |
replicated |
<CACHE_NAME>_CACHE_START |
Determines if this cache should be started on server startup, or lazily when requested by a service or deployment. Defaults to LAZY. |
EAGER |
<CACHE_NAME>_CACHE_BATCHING |
Enables invocation batching for this cache. Defaults to false. |
true |
<CACHE_NAME>_CACHE_STATISTICS |
Determines whether or not the cache collects statistics. Disable for optimal performance. Defaults to true. |
false |
<CACHE_NAME>_CACHE_MODE |
Sets the clustered cache mode, ASYNC for asynchronous operations, or SYNC for synchronous operations. |
ASYNC |
<CACHE_NAME>_CACHE_QUEUE_SIZE |
In ASYNC mode this attribute can be used to trigger flushing of the queue when it reaches a specific threshold. Defaults to 0, which disables flushing. |
100 |
<CACHE_NAME>_CACHE_QUEUE_FLUSH_INTERVAL |
In ASYNC mode this attribute controls how often the asynchronous thread runs to flush the replication queue. This should be a positive integer that represents thread wakeup time in milliseconds. Defaults to 10. |
20 |
<CACHE_NAME>_CACHE_REMOTE_TIMEOUT |
In SYNC mode the timeout, in milliseconds, used to wait for an acknowledgement when making a remote call, after which the call is aborted and an exception is thrown. Defaults to 17500. |
25000 |
<CACHE_NAME>_CACHE_OWNERS |
Number of cluster-wide replicas for each cache entry. Defaults to 2. |
5 |
<CACHE_NAME>_CACHE_SEGMENTS |
Number of hash space segments per cluster. The recommended value is 10 * cluster size. Defaults to 80. |
30 |
<CACHE_NAME>_CACHE_L1_LIFESPAN |
Maximum lifespan, in milliseconds, of an entry placed in the L1 cache. Defaults to 0, indicating that L1 is disabled. |
100. |
<CACHE_NAME>_CACHE_EVICTION_STRATEGY |
Sets the cache eviction strategy. Available options are UNORDERED, FIFO, LRU, LIRS, and NONE (to disable eviction). Defaults to NONE. |
FIFO |
<CACHE_NAME>_CACHE_EVICTION_MAX_ENTRIES |
Maximum number of entries in a cache instance. If selected value is not a power of two the actual value will default to the least power of two larger than the selected value. A value of -1 indicates no limit. Defaults to 10000. |
-1 |
<CACHE_NAME>_CACHE_EXPIRATION_LIFESPAN |
Maximum lifespan, in milliseconds, of a cache entry, after which the entry is expired cluster-wide. Defaults to -1, indicating that the entries never expire. |
10000 |
<CACHE_NAME>_CACHE_EXPIRATION_MAX_IDLE |
Maximum idle time, in milliseconds, a cache entry will be maintained in the cache. If the idle time is exceeded, then the entry will be expired cluster-wide. Defaults to -1, indicating that the entries never expire. |
10000 |
<CACHE_NAME>_CACHE_EXPIRATION_INTERVAL |
Interval, in milliseconds, between subsequent runs to purge expired entries from memory and any cache stores. If you wish to disable the periodic eviction process altogether, then set the interval to -1. Defaults to 5000. |
-1 |
<CACHE_NAME>_CACHE_COMPATIBILITY_ENABLED |
Enables compatibility mode for this cache. Disabled by default. |
true |
<CACHE_NAME>_CACHE_COMPATIBILITY_MARSHALLER |
A marshaller to use for compatibility conversions. |
com.acme.CustomMarshaller |
<CACHE_NAME>_JDBC_STORE_TYPE |
Type of JDBC store to configure. This value may either be string or binary. |
string |
<CACHE_NAME>_JDBC_STORE_DATASOURCE |
Defines the jndiname of the datasource. |
java:jboss/datasources/ExampleDS |
<CACHE_NAME>_KEYED_TABLE_PREFIX |
Defines the prefix prepended to the cache name used when composing the name of the cache entry table. Defaults to ispn_entry. |
JDG |
<CACHE_NAME>_CACHE_INDEX |
The indexing mode of the cache. Valid values are NONE, LOCAL, and ALL. Defaults to NONE. |
ALL |
<CACHE_NAME>_CACHE_INDEXING_PROPERTIES |
Comma separated list of properties to pass on to the indexing system. |
default.directory_provider=ram |
<CACHE_NAME>_CACHE_SECURITY_AUTHORIZATION_ENABLED |
Enables authorization checks for this cache. Defaults to false. |
true |
<CACHE_NAME>_CACHE_SECURITY_AUTHORIZATION_ROLES |
Sets the valid roles required to access this cache. |
admin,reader,writer |
<CACHE_NAME>_CACHE_PARTITION_HANDLING_ENABLED |
If enabled, then the cache will enter degraded mode when it loses too many nodes. Defaults to true. |
false |
Datasource properties may be configured with the following environment variables:
| Variable Name | Description | Example Value |
|---|---|---|
<NAME><DATABASE_TYPE>_SERVICE_HOST_ |
Defines the database server’s hostname or IP to be used in the datasource’s connection_url property. |
192.168.1.3 |
<NAME>_DATABASE_TYPE>_SERVICE_PORT |
Defines the database server’s port for the datasource. |
5432 |
<PREFIX>_JNDI |
Defines the JNDI name for the datasource. Defaults to java:jboss/datasources/<name><database_type>_, where name and database_type are taken from the triplet definition. This setting is useful if you want to override the default generated JNDI name. |
java:jboss/datasources/test-postgresql |
<PREFIX>_USERNAME |
Defines the username for the datasource. |
admin |
<PREFIX>_PASSWORD |
Defines the password for the datasource. |
password |
<PREFIX>_DATABASE |
Defines the database name for the datasource. |
myDatabase |
<PREFIX>_TX_ISOLATION |
Defines the java.sql.Connection transaction isolation level for the database. |
TRANSACTION_READ_UNCOMMITTED |
<PREFIX>_TX_MIN_POOL_SIZE |
Defines the minimum pool size option for the datasource. |
1 |
<PREFIX>_TX_MAX_POOL_SIZE |
Defines the maximum pool size option for the datasource. |
20 |
The following environment variables may be defined to customize the environment’s security domain:
| Variable Name | Description | Example Value |
|---|---|---|
SECDOMAIN_NAME |
Define in order to enable the definition of an additional security domain. |
myDomain |
SECDOMAIN_PASSWORD_STACKING |
If defined, the password-stacking module option is enabled and set to the value useFirstPass. |
true |
SECDOMAIN_LOGIN_MODULE |
The login module to be used. Defaults to UsersRoles. |
UsersRoles |
SECDOMAIN_USERS_PROPERTIES |
The name of the properties file containing user definitions. Defaults to users.properties. |
users.properties |
SECDOMAIN_ROLES_PROPERTIES |
The name of the properties file containing role definitions. Defaults to roles.properties. |
roles.properties |
The following ports are exposed by default in the JBoss Data Grid xPaaS Image:
| Value | Description |
|---|---|
8443 |
Secure Web |
8778 |
- |
11211 |
memcached |
11222 |
internal hotrod |
11333 |
external hotrod |
| The external hotrod connector is only available if the HOTROD_SERVICE_NAME environment variables has been defined. |
In addition to viewing the OpenShift logs, you can troubleshoot a running JBoss Data Grid xPaaS Image container by viewing its logs. These are outputted to the container’s standard out, and are accessible with the following command:
$ oc logs -f <pod_name> <container_name>
| By default, the OpenShift JBoss Data Grid xPaaS Image does not have a file log handler configured. Logs are only sent to the container’s standard out. |