Package com.hazelcast.jet.cdc.postgres

Class PostgresCdcSources

java.lang.Object

com.hazelcast.jet.cdc.postgres.PostgresCdcSources

@EvolvingApi
public final class PostgresCdcSources
extends Object

Contains factory methods for creating change data capture sources based on PostgreSQL databases.

Since:

4.2

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	PostgresCdcSources.Builder	Builder for configuring a CDC source that streams change data from a PostgreSQL database to Hazelcast Jet.

Method Summary

Modifier and Type	Method	Description
static PostgresCdcSources.Builder	postgres(String name)	Creates a CDC source that streams change data from a PostgreSQL database to Hazelcast Jet.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

postgres

@Nonnull public static PostgresCdcSources.Builder postgres(@Nonnull String name)

Creates a CDC source that streams change data from a PostgreSQL database to Hazelcast Jet.

KNOWN ISSUE 1: If Jet can't reach the database when it attempts to start the source or if it looses the connection to the database from an already running source, it throws an exception and terminate the execution of the job. This behaviour is not ideal, would be much better to try to reconnect, at least for a certain amount of time. Future versions will address the problem.

KNOWN ISSUE 2: This source is based on the Debezium PostgreSQL Connector, which has some data loss issues when synchronizing snapshots with the stream of events coming from the Postgres WAL (Write-Ahead Log). For details see DBZ-2288. Currently the first streaming event that should come in right after the snapshot ends is lost. Sometimes even more events can be lost. The issue is actively being looked into and we will release a fix immediately as it's available.

Parameters:

name - name of this source, needs to be unique, will be passed to the underlying Kafka Connect source

Returns:

builder that can be used to set source properties and also to construct the source once configuration is done