Class JdbcSinkBuilder<T>

Type Parameters:
T - type of the items the sink accepts

public class JdbcSinkBuilder<T>
extends Object
  • Method Details

    • updateQuery

      @Nonnull public JdbcSinkBuilder<T> updateQuery​(@Nonnull String updateQuery)
      The query to execute for each item. It should contain a parametrized query to which the bind function will bind values.

      Which type of statement to use?

      In exactly-once mode we recommend using an INSERT statement. Each parallel processor uses a separate transaction: if two processors try to update the same record, the later one will be deadlocked: it will be blocked waiting for a record lock but will never get it because the snapshot will not be able to complete and the lock owner will never commit.

      On the other hand, in at-least-once mode we recommend using a MERGE statement. If a unique key is derived from the items, this can give you exactly-once behavior through idempotence without using XA transactions: the MERGE statement will insert a record initially and overwrite the same record, if the job restarted and the same item is written again. If you don't have a unique key in the item, use INSERT with auto-generated key.

      updateQuery - the SQL statement to execute for each item
      this instance for fluent API
    • bindFn

      Set the function to bind values to a PreparedStatement created with the query set with updateQuery(String). The function should not execute the query, nor call commit() or any other method.
      bindFn - the bind function
      this instance for fluent API
    • jdbcUrl

      @Nonnull public JdbcSinkBuilder<T> jdbcUrl​(String connectionUrl)
      Sets the connection URL for the target database.

      If your job runs in exactly-once mode, don't use this method, but provide an XADataSource using dataSourceSupplier(SupplierEx) method, otherwise the job will fail. If your driver doesn't have an XADataSource implementation, also call exactlyOnce(false).

      See also dataSourceSupplier(SupplierEx).

      connectionUrl - the connection URL
      this instance for fluent API
    • dataSourceSupplier

      @Nonnull public JdbcSinkBuilder<T> dataSourceSupplier​(SupplierEx<? extends CommonDataSource> dataSourceSupplier)
      Sets the supplier of DataSource or XADataSource. One dataSource instance will be created on each member. For exactly-once guarantee an XADataSource must be used or the job will fail. If your driver doesn't have an XADataSource implementation, also call exactlyOnce(false).

      There's no need to use ConnectionPoolDataSource. One connection is given to each processor and that connection is held during the entire job execution.

      dataSourceSupplier - the supplier of data source
      this instance for fluent API
    • exactlyOnce

      @Nonnull public JdbcSinkBuilder<T> exactlyOnce​(boolean enabled)
      Sets whether the exactly-once mode is enabled for the sink. If exactly-once is enabled, the job must also be in exactly-once mode for that mode to be used, otherwise the sink will use the job's guarantee.

      Set exactly-once to false if you want your job to run in exactly-once, but want to reduce the guarantee just for the sink.

      enabled - whether exactly-once is allowed for the sink
      this instance for fluent API
    • build

      @Nonnull public Sink<T> build()
      Creates and returns the JDBC Sink with the supplied components.