public final class KafkaSinks extends Object
Modifier and Type | Method and Description |
---|---|
static <E,K,V> Sink<E> |
kafka(Properties properties,
FunctionEx<? super E,org.apache.kafka.clients.producer.ProducerRecord<K,V>> toRecordFn)
Returns a source that publishes messages to an Apache Kafka topic.
|
static <K,V> Sink<Map.Entry<K,V>> |
kafka(Properties properties,
String topic)
Convenience for
kafka(Properties, String, FunctionEx, FunctionEx)
which expects Map.Entry<K, V> as input and extracts its key and value
parts to be published to Kafka. |
static <E,K,V> Sink<E> |
kafka(Properties properties,
String topic,
FunctionEx<? super E,K> extractKeyFn,
FunctionEx<? super E,V> extractValueFn)
Convenience for
kafka(Properties, FunctionEx) which creates
a ProducerRecord using the given topic and the given key and value
mapping functions |
@Nonnull public static <E,K,V> Sink<E> kafka(@Nonnull Properties properties, @Nonnull FunctionEx<? super E,org.apache.kafka.clients.producer.ProducerRecord<K,V>> toRecordFn)
ProducerRecord
using the
supplied mapping function.
The source creates a single KafkaProducer
per cluster
member using the supplied properties
.
Behavior on job restart: the processor is stateless. On snapshot we only make sure that all async operations are done. If the job is restarted, duplicate events can occur. If you need exactly-once behavior, you must ensure idempotence on the application level.
IO failures are generally handled by Kafka producer and do not cause the processor to fail. Refer to Kafka documentation for details.
Default local parallelism for this processor is 2 (or less if less CPUs are available).
E
- type of stream itemK
- type of the key published to KafkaV
- type of the value published to Kafkaproperties
- producer properties which should contain broker
address and key/value serializerstoRecordFn
- function that extracts the key from the stream item@Nonnull public static <E,K,V> Sink<E> kafka(@Nonnull Properties properties, @Nonnull String topic, @Nonnull FunctionEx<? super E,K> extractKeyFn, @Nonnull FunctionEx<? super E,V> extractValueFn)
kafka(Properties, FunctionEx)
which creates
a ProducerRecord
using the given topic and the given key and value
mapping functionsE
- type of stream itemK
- type of the key published to KafkaV
- type of the value published to Kafkaproperties
- producer properties which should contain broker
address and key/value serializerstopic
- name of the Kafka topic to publish toextractKeyFn
- function that extracts the key from the stream itemextractValueFn
- function that extracts the value from the stream item@Nonnull public static <K,V> Sink<Map.Entry<K,V>> kafka(@Nonnull Properties properties, @Nonnull String topic)
kafka(Properties, String, FunctionEx, FunctionEx)
which expects Map.Entry<K, V>
as input and extracts its key and value
parts to be published to Kafka.K
- type of the key published to KafkaV
- type of the value published to Kafkaproperties
- producer properties which should contain broker
address and key/value serializerstopic
- Kafka topic name to publish toCopyright © 2019 Hazelcast, Inc.. All rights reserved.