Chapter 11. Debezium Server (Technology Preview)
Debezium Server is a ready-to-use application that streams change events from a source database directly to messaging and storage systems without requiring Apache Kafka. You configure a source connector and a sink to define where events come from and where they are delivered.
Debezium Server is a Technology Preview feature. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
11.1. Debezium Server installation
Download and unpack the Debezium Server distribution to install the server on your system.
Procedure
- Open the Download red_hat_build_of_debezium page on the Red Hat Customer Portal.
- From the Releases tab, locate Debezium 3.6.3 Debezium Server and click Download Now to download the ZIP file.
- Extract the contents of the archive to the directory in which you want to install Debezium.
A directory named debezium-server is created with the following contents:
debezium-server/
|-- config
|-- debezium-server-dist-3.6.3.Final-redhat-00008-runner.jar
|-- jmx
|-- lib
|-- lib_metrics
|-- lib_opt
|-- run.bat
|-- run.sh
The lib directory stores dependencies, and the config directory contains configuration files. The run.bat or run.sh file starts the server.
Before you start the server, specify the source for the change events that you want to stream and the sink destination for those events. For details about how to configure Debezium Server, see the Configuration section.
11.1.1. Start the server
Run the Debezium Server startup script to begin streaming change events from your configured source connector to the configured sink.
Prerequisites
- Configure Debezium Server as described in Debezium Server configuration.
-
To use the Oracle connector, copy the Oracle JDBC driver to the
libdirectory before you start the server. -
To use Oracle XStream, copy both the Oracle JDBC driver and the required XStream API files to the
libdirectory. For more information, see Obtaining the Oracle JDBC driver and XStream API files.
Procedure
- Open a command prompt or terminal window.
-
Change to the
<ServerInstallDirectory>/debezium-serverdirectory. Start Debezium Server by entering one of the following commands:
On Linux, run the
run.shscript:./run.shOn Windows, run the
run.batscript:run.bat
11.2. Debezium Server configuration
Debezium Server uses MicroProfile Configuration for configuration. MicroProfile configuration is a standard for supplying configuration values to Java applications from multiple sources, including configuration files, environment variables, and system properties.
Configuration properties that contain the $ character are normally interpreted by Quarkus for property expansion. If a property value includes a $ character as a prefix, such as ${my.property}, the Microprofile configuration parser determines that the value is a property expression and attempts to expand it.
To prevent the parser from attempting to expand values that use this syntax, escape the $ prefix with a second $ character; that is, provide the value as $${my.property}. The value of the property is then passed to Debezium as ${my.property}.
The main configuration file is config/application.properties. There are multiple sections configured:
-
debezium.sourceis for source connector configuration; each instance of Debezium Server runs exactly one connector -
debezium.sinkis for the sink system configuration -
debezium.formatis for the output serialization format configuration -
debezium.transformsis for the configuration of message transformations -
debezium.predicatesis for the configuration of message transformation predicates
The following example shows an excerpt from a configuration file:
debezium.sink.type=http
debezium.sink.http.url=https://example.com/events
debezium.source.connector.class=io.debezium.connector.postgresql.PostgresConnector
debezium.source.offset.storage.file.filename=data/offsets.dat
debezium.source.offset.flush.interval.ms=0
debezium.source.database.hostname=localhost
debezium.source.database.port=5432
debezium.source.database.user=postgres
debezium.source.database.password=postgres
debezium.source.database.dbname=postgres
debezium.source.topic.prefix=tutorial
debezium.source.schema.include.list=inventoryThe configuration file in the preceding example specifies the following settings:
-
The source connector captures events from a schema named
inventory. If you want to capture all changes in the database, remove this line. Otherwise, update this line to correspond to your preferred schema or tables. -
The source offset is stored in a file named
offsets.datin thedatadirectory. To prevent startup errors, the directory that you specify must exists before you start the server.
After the server starts, it generates a sequence of log messages similar to the following example:
__ ____ __ _____ ___ __ ____ ______
--/ __ \/ / / / _ | / _ \/ //_/ / / / __/
-/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2020-05-15 11:33:12,628 INFO [io.deb.ser.DebeziumServer] (main) Consumer 'io.debezium.server.http.HttpChangeConsumer' instantiated
2020-05-15 11:33:12,754 INFO [org.apa.kaf.con.jso.JsonConverterConfig] (main) JsonConverterConfig values:
converter.type = key
decimal.format = BASE64
schemas.cache.size = 1000
schemas.enable = true
2020-05-15 11:33:12,757 INFO [org.apa.kaf.con.jso.JsonConverterConfig] (main) JsonConverterConfig values:
converter.type = value
decimal.format = BASE64
schemas.cache.size = 1000
schemas.enable = false
2020-05-15 11:33:12,763 INFO [io.deb.emb.EmbeddedEngine$EmbeddedConfig] (main) EmbeddedConfig values:
access.control.allow.methods =
access.control.allow.origin =
admin.listeners = null
bootstrap.servers = [localhost:9092]
client.dns.lookup = default
config.providers = []
connector.client.config.override.policy = None
header.converter = class org.apache.kafka.connect.storage.SimpleHeaderConverter
internal.key.converter = class org.apache.kafka.connect.json.JsonConverter
internal.value.converter = class org.apache.kafka.connect.json.JsonConverter
key.converter = class org.apache.kafka.connect.json.JsonConverter
listeners = null
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
offset.flush.interval.ms = 0
offset.flush.timeout.ms = 5000
offset.storage.file.filename = data/offsets.dat
offset.storage.partitions = null
offset.storage.replication.factor = null
offset.storage.topic =
plugin.path = null
rest.advertised.host.name = null
rest.advertised.listener = null
rest.advertised.port = null
rest.extension.classes = []
rest.host.name = null
rest.port = 8083
ssl.client.auth = none
task.shutdown.graceful.timeout.ms = 5000
topic.tracking.allow.reset = true
topic.tracking.enable = true
value.converter = class org.apache.kafka.connect.json.JsonConverter
2020-05-15 11:33:12,763 INFO [org.apa.kaf.con.run.WorkerConfig] (main) Worker configuration property 'internal.key.converter' is deprecated and may be removed in an upcoming release. The specified value 'org.apache.kafka.connect.json.JsonConverter' matches the default, so this property can be safely removed from the worker configuration.
2020-05-15 11:33:12,763 INFO [org.apa.kaf.con.run.WorkerConfig] (main) Worker configuration property 'internal.value.converter' is deprecated and may be removed in an upcoming release. The specified value 'org.apache.kafka.connect.json.JsonConverter' matches the default, so this property can be safely removed from the worker configuration.
2020-05-15 11:33:12,765 INFO [org.apa.kaf.con.jso.JsonConverterConfig] (main) JsonConverterConfig values:
converter.type = key
decimal.format = BASE64
schemas.cache.size = 1000
schemas.enable = true
2020-05-15 11:33:12,765 INFO [org.apa.kaf.con.jso.JsonConverterConfig] (main) JsonConverterConfig values:
converter.type = value
decimal.format = BASE64
schemas.cache.size = 1000
schemas.enable = true
2020-05-15 11:33:12,767 INFO [io.deb.ser.DebeziumServer] (main) Engine executor started
2020-05-15 11:33:12,773 INFO [org.apa.kaf.con.sto.FileOffsetBackingStore] (pool-3-thread-1) Starting FileOffsetBackingStore with file data/offsets.dat
2020-05-15 11:33:12,835 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) Starting PostgresConnectorTask with configuration:
2020-05-15 11:33:12,837 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) connector.class = io.debezium.connector.postgresql.PostgresConnector
2020-05-15 11:33:12,837 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) offset.flush.interval.ms = 0
2020-05-15 11:33:12,838 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) database.user = postgres
2020-05-15 11:33:12,838 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) database.dbname = postgres
2020-05-15 11:33:12,838 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) offset.storage.file.filename = data/offsets.dat
2020-05-15 11:33:12,838 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) database.hostname = localhost
2020-05-15 11:33:12,838 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) database.password = ********
2020-05-15 11:33:12,839 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) name = http
2020-05-15 11:33:12,839 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) topic.prefix = tutorial
2020-05-15 11:33:12,839 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) database.port = 5432
2020-05-15 11:33:12,839 INFO [io.deb.con.com.BaseSourceTask] (pool-3-thread-1) schema.include.list = inventory
2020-05-15 11:33:12,908 INFO [io.quarkus] (main) debezium-server 1.2.0-SNAPSHOT (powered by Quarkus 1.4.1.Final) started in 1.198s. Listening on: http://0.0.0.0:8080
2020-05-15 11:33:12,911 INFO [io.quarkus] (main) Profile prod activated.
2020-05-15 11:33:12,911 INFO [io.quarkus] (main) Installed features: [cdi, smallrye-health]11.2.1. Debezium Server source configuration properties
Source configuration properties specify which connector Debezium Server runs and configure offset and schema history storage outside of Kafka Connect.
Most source configuration properties correspond to properties that are documented for individual connectors. To use these properties in Debezium Server, prepend the debezium.source. prefix to the connector property name. Debezium Server also provides additional source configuration properties for settings that Kafka Connect normally handles automatically. For example, debezium.source.offset.storage and debezium.source.offset.storage.file.filename to configure file-based offset storage.
| Property | Default | Description |
|---|---|---|
| The name of the Java class implementing the source connector. | ||
|
| Class to use for storing and retrieving offsets for non-Kafka deployments. Available options
| |
| If using a file offset store (default), the file in which connector offsets are stored for non-Kafka deployments. | ||
| Defines how frequently the offsets are flushed into the file. | ||
|
| Some of the connectors (e.g MySQL, SQL Server, Db2, Oracle) track the database schema evolution over time and stores this data in a database schema history. This is by default based on Kafka. There are also other options available
| |
|
The name and location of the file to which |
11.2.2. Debezium Server format configuration options
Configure how Debezium Server serializes message keys and values. You can configure key and value formats independently by using either the default JSON format or a custom Kafka Connect Converter implementation.
| Property | Default | Description |
|---|---|---|
|
|
The name of the output format for key, one of | |
| Configuration properties passed to the key converter. | ||
|
|
The name of the output format for value, one of | |
| Configuration properties passed to the value converter. | ||
|
|
The name of the output format for value, one of | |
| Configuration properties passed to the header converter. |
11.2.3. Debezium Server transformation configuration options
Configure Kafka Connect Single Message Transformations (SMTs) that Debezium Server applies to messages before it delivers them to the sink. Specify the transformations to apply and the configuration options for each transformation.
The server supports single message transformations defined by Kafka Connect.
| Property | Default | Description [id="debezium-transforms"] |
|---|---|---|
|
| The comma separated list of symbolic names of transformations. | |
|
|
The name of Java class implementing the transformation with name | |
|
|
Configuration properties passed to the transformation with name | |
|
|
The name of the predicate to be applied to the transformation with name | |
|
| false |
Determines if the result of the predicate to the transformation with name |
11.2.4. Debezium Server predicates configuration options
Configure predicates that evaluate messages and determine when Debezium Server applies message transformations. Debezium Server supports the Kafka Connect predicate framework for conditional Single Message Transformations (SMTs). Predicate properties specify the predicates to use, their implementation classes, and their configuration options.
| Property | Default | Description [id="debezium-predicates"] |
|---|---|---|
|
| The comma separated list of symbolic names of predicates. | |
|
|
The name of Java class implementing the predicate with name | |
|
|
Configuration properties passed to the predicate with name |
11.2.5. Enabling message filtering in Debezium Server
You can enable the filter SMT in Debezium Server to selectively pass or discard change events based on expression-based conditions that you define.
Because filter expressions are evaluated by a scripting engine running inside the Debezium Server process, this feature is only safe to enable in environments where access to the server configuration is restricted to authorized operators. Scripting support is therefore disabled by default and must be explicitly enabled.
To enable message filtering, set the environment variable ENABLE_DEBEZIUM_SCRIPTING to true before starting Debezium Server. When yhou enable this setting, the startup script adds the debezium-scripting JAR and the JSR 223 script engine implementations (Groovy and GraalVM JavaScript) from the opt_lib directory of the Debezium Server distribution to the server classpath. These libraries are kept in opt_lib rather than on the default classpath precisely so that scripting is never available unless an operator makes a conscious decision to include them.
Additional resources
11.2.6. Debezium Server asynchronous engine properties
By default, Debezium Server uses the asynchronous embedded engine (AsyncEmbeddedEngine) to process change events. Use the following properties to configure record-processing threads, task lifecycle management, and engine shutdown behavior.
| Property | Default | Description |
| Threads allocated on demand, based on the workload and the number of available CPU cores. |
The number of threads that are available to process change event records. If no value is specified (the default), the engine uses the Java ThreadPoolExecutor to dynamically adjust the number of threads, based on the current workload. Maximum number of threads is number of CPU cores on given machine. If a value is specified, the engine uses the Java fixed thread pool method to create a thread pool with the specified number of threads. To use all available cores on given machine, set the placeholder value, | |
| 1000 | Maximum time, in milliseconds, that the engine allows for processing pending records after a task shutdown is called. | |
| 180,000 (3 min) | Time, in milliseconds, that the engine waits for a task’s lifecycle management operations (starting and stopping) to complete. |
11.2.7. Debezium Server Quarkus runtime configuration properties
Debezium Server runs on the Quarkus framework. All configuration properties that Quarkus exposes are available in Debezium Server. The following table lists commonly used Quarkus properties for configuring HTTP endpoint access and logging behavior.
| Property | Default | Description [id="debezium-quarkus-http-port"] |
|---|---|---|
|
| 8080 |
The port on which Debezium exposes Microprofile Health endpoint and other exposed status information. Health can be accessed on |
|
| INFO | The default log level for every log category. |
|
| true | Determine whether to enable the JSON console formatting extension, which disables "normal" console formatting. |
You can disable JSON logging by setting quarkus.log.console.json=false in the config/application.properties file. For details, see the example in the file.
11.2.8. Debezium Server sink configuration
Debezium Server supports implementations for delivering change events to different sink destinations. Configure the destination to which Debezium Server delivers change events. Set the debezium.sink.type property to specify the sink implementation, and then configure the properties that apply to the selected sink.
11.2.8.1. HTTP Client sink configuration
Configure the HTTP Client sink to deliver change events from Debezium Server to an HTTP endpoint for downstream processing. The HTTP Client sink supports optional JWT and OAuth 2.0 client credentials authentication, and provides options for configuring batched event delivery.
You can use the HTTP Client sink to integrate Debezium Server with Knative and enable it to act as a Knative Event Source.
| Property | Default | Description |
|---|---|---|
|
Must be set to | ||
|
The HTTP Server URL to stream events to. This can also be set by defining the | ||
| 60000 | The number of seconds to wait for a response from the server before timing out. (default of 60s) | |
| 5 | The number retries before exception is thrown (default 5 times). | |
| 1000 | The number of milliseconds to wait before another attempt to send record is made after failure (default of 1s). | |
| X-DEBEZIUM- | Headers will be prefixed with this value (defaults to X-DEBEZIUM-). | |
| true | Header values will be base64 encoded (defaults to true). | |
| false |
When set to | |
| 200 | The maximum number of events per HTTP request when batch mode is enabled. If the engine delivers more events than this limit, they are chunked into multiple requests. | |
| Specifies the type of authentication the HTTP client sink uses when connecting to an HTTP server. Supports one of the following options:
If you omit this property, the HTTP client sink does not use authentication headers for the connection | ||
| Specifies the username for JWT authentication. | ||
| Specifies the password for JWT authentication. | ||
|
Specifies the base URL (e.g., | ||
| Requested duration (in minutes) before the authentication token expires. | ||
|
| Requested duration (in minutes) before the refresh token expires. | |
|
The webhooks signing secret that Debezium uses to generate HMAC-SHA256 signatures for the webhook request. The secret must be Base64-encoded, with a size from 24 bytes to 64 bytes (192–512 bits). Optionally, you can add the prefix | ||
| Specifies the OAuth2 client ID for the client credentials grant. | ||
| Specifies the OAuth2 client secret for the client credentials grant. | ||
|
The URL of the OAuth2 token endpoint (e.g., | ||
|
Optional space-separated list of OAuth2 scopes to request (e.g., | ||
| client_secret_basic | Specifies how client credentials are sent to the token endpoint. Supports one of the following options:
| |
|
| POST |
The HTTP method to use for token requests. The OAuth2 specification requires |
|
Optional additional parameters to include in the token request. For example, some providers require an |
11.2.8.2. Apache Kafka sink configuration
Configure the Apache Kafka sink to publish change events from Debezium Server to Kafka topics. Use the sink properties to specify broker connection details and other Kafka-specific settings.
| Property | Default | Description |
|---|---|---|
|
Must be set to | ||
|
The Kafka sink adapter supports pass-through configuration. This means that all Kafka producer configuration properties are passed to the producer with the prefix removed. At least | ||
|
|
The maximum time, in milliseconds, that the server waits for a request to complete and return metadata for a record. The specified timeout also governs the interval that the server waits for Kafka to respond to a request. Set the value to |
11.2.8.2.1. Apache Kafka sink injection points
Customize Kafka sink behavior by providing implementations of supported extension points. For example, you can implement a custom StreamNameMapper to control how Debezium Server maps source streams to Kafka topics. If you do not provide a custom implementation, Debezium Server uses the default implementation.
| Interface | CDI classifier | Description |
|---|---|---|
| Custom implementation maps the original destination (topic) name into another Kafka topic. By default, the same name is used. |
Revised on 2026-10-14 18:57:30 UTC
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}