Package com.hazelcast.jet.cdc.postgres

Class PostgresCdcSources

java.lang.Object
com.hazelcast.jet.cdc.postgres.PostgresCdcSources
@EvolvingApi
public final class PostgresCdcSources
extends Object
Contains factory methods for creating change data capture sources based on PostgreSQL databases.
Since:
4.2

  • Method Details

    • postgres

      @Nonnull public static PostgresCdcSources.Builder postgres​(@Nonnull String name)
      Creates a CDC source that streams change data from a PostgreSQL database to Hazelcast Jet.

      You can configure how the source will behave if the database connection breaks, by passing one of the retry strategies to setReconnectBehavior().

      The default reconnect behavior is never, which treats any connection failure as an unrecoverable problem and triggers the failure of the source and the entire job.

      Other behavior options, which specify that retry attempts should be made, will result in the source initiating reconnects to the database.

      There is a further setting influencing reconnect behavior, specified via setShouldStateBeResetOnReconnect(). The boolean flag passed in specifies what should happen to the connector's state on reconnect, whether it should be kept or reset. If the state is kept, then database snapshotting should not be repeated and streaming the WAL should resume at the position where it left off. If the state is reset, then the source will behave as on its initial start, so will do a database snapshot and will start tailing the WAL where it syncs with the database snapshot's end.

      You can also configure how often the source will send feedback about processed change record offsets to the backing database via setCommitPeriod(). The replication slots of the database will clean up their internal data structures based on this feedback. A commit period of 0 means that the source will commit offsets after every batch of change records. Also, important to note that periodic commits happen only in the case of jobs without processing guarantees. For jobs offering processing guarantees, the source will ignore this setting and commit offsets as part of the state snapshotting process. So the setting governing them will be JobConfig.setSnapshotIntervalMillis.

      Parameters:
      name - name of this source, needs to be unique, will be passed to the underlying Kafka Connect source
      Returns:
      builder that can be used to set source properties and also to construct the source once configuration is done