Package | Description |
---|---|
com.hazelcast.jet.avro |
Apache Avro file read/write support for Hazelcast Jet.
|
com.hazelcast.jet.hadoop |
Apache Hadoop HDFS read/write support for Hazelcast Jet.
|
com.hazelcast.jet.kafka |
Apache Kafka reader/writer support for Hazelcast Jet.
|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
Modifier and Type | Method and Description |
---|---|
static Sink<org.apache.avro.generic.IndexedRecord> |
AvroSinks.files(String directoryName,
SupplierEx<org.apache.avro.Schema> schemaSupplier)
Convenience for
AvroSinks.files(String, SupplierEx,
SupplierEx) which uses GenericDatumWriter . |
static <R> Sink<R> |
AvroSinks.files(String directoryName,
SupplierEx<org.apache.avro.Schema> schemaSupplier,
Class<R> recordClass)
Convenience for
AvroSinks.files(String, SupplierEx,
SupplierEx) which uses either SpecificDatumWriter or
ReflectDatumWriter depending on the supplied recordClass . |
static <R> Sink<R> |
AvroSinks.files(String directoryName,
SupplierEx<org.apache.avro.Schema> schemaSupplier,
SupplierEx<org.apache.avro.io.DatumWriter<R>> datumWriterSupplier)
Returns a sink that that writes the items it receives to Apache Avro
files.
|
Modifier and Type | Method and Description |
---|---|
static <K,V> Sink<Map.Entry<K,V>> |
HdfsSinks.hdfs(org.apache.hadoop.mapred.JobConf jobConf)
Convenience for
HdfsSinks.hdfs(JobConf, FunctionEx,
FunctionEx) which expects Map.Entry<K, V> as
input and extracts its key and value parts to be written to HDFS. |
static <E,K,V> Sink<E> |
HdfsSinks.hdfs(org.apache.hadoop.mapred.JobConf jobConf,
FunctionEx<? super E,K> extractKeyF,
FunctionEx<? super E,V> extractValueF)
Returns a sink that writes to Apache Hadoop HDFS.
|
Modifier and Type | Method and Description |
---|---|
static <E,K,V> Sink<E> |
KafkaSinks.kafka(Properties properties,
FunctionEx<? super E,org.apache.kafka.clients.producer.ProducerRecord<K,V>> toRecordFn)
Returns a source that publishes messages to an Apache Kafka topic.
|
static <K,V> Sink<Map.Entry<K,V>> |
KafkaSinks.kafka(Properties properties,
String topic)
Convenience for
KafkaSinks.kafka(Properties, String, FunctionEx, FunctionEx)
which expects Map.Entry<K, V> as input and extracts its key and value
parts to be published to Kafka. |
static <E,K,V> Sink<E> |
KafkaSinks.kafka(Properties properties,
String topic,
FunctionEx<? super E,K> extractKeyFn,
FunctionEx<? super E,V> extractValueFn)
Convenience for
KafkaSinks.kafka(Properties, FunctionEx) which creates
a ProducerRecord using the given topic and the given key and value
mapping functions |
Modifier and Type | Method and Description |
---|---|
Sink<T> |
SinkBuilder.build()
Creates and returns the
Sink with the components you supplied to
this builder. |
Sink<T> |
JmsSinkBuilder.build()
Creates and returns the JMS
Sink with the supplied components. |
Sink<T> |
FileSinkBuilder.build()
Creates and returns the file
Sink with the supplied components. |
static <T extends Map.Entry> |
Sinks.cache(String cacheName)
Returns a sink that puts
Map.Entry s it receives into a Hazelcast
ICache with the specified name. |
static <T> Sink<T> |
Sinks.files(String directoryName)
Convenience for
Sinks.filesBuilder(java.lang.String) with the UTF-8 charset and with
overwriting of existing files. |
static <T> Sink<T> |
Sinks.fromProcessor(String sinkName,
ProcessorMetaSupplier metaSupplier)
Returns a sink constructed directly from the given Core API processor
meta-supplier.
|
static <T> Sink<T> |
Sinks.jdbc(String updateQuery,
String connectionUrl,
BiConsumerEx<PreparedStatement,T> bindFn)
Convenience for
Sinks.jdbc(String, SupplierEx,
BiConsumerEx) . |
static <T> Sink<T> |
Sinks.jdbc(String updateQuery,
SupplierEx<Connection> connectionSupplier,
BiConsumerEx<PreparedStatement,T> bindFn)
Returns a sink that connects to the specified database using the given
connectionSupplier , prepares a statement using the given updateQuery and inserts/updates the items. |
static <T> Sink<T> |
Sinks.jmsQueue(SupplierEx<javax.jms.ConnectionFactory> factorySupplier,
String name)
Convenience for
Sinks.jmsQueueBuilder(SupplierEx) . |
static <T> Sink<T> |
Sinks.jmsTopic(SupplierEx<javax.jms.ConnectionFactory> factorySupplier,
String name)
Convenience for
Sinks.jmsTopicBuilder(SupplierEx) . |
static <T> Sink<T> |
Sinks.list(IList<? super T> list)
Returns a sink that adds the items it receives to a Hazelcast
IList with the specified name. |
static <T> Sink<T> |
Sinks.list(String listName)
Returns a sink that adds the items it receives to a Hazelcast
IList with the specified name. |
static <T> Sink<T> |
Sinks.logger()
|
static <T> Sink<T> |
Sinks.logger(FunctionEx<? super T,String> toStringFn)
Returns a sink that logs all the data items it receives, at the INFO
level to the log category
WriteLoggerP . |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.map(IMap<? super K,? super V> map)
Returns a sink that puts
Map.Entry s it receives into the given
Hazelcast IMap . |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.map(String mapName)
Returns a sink that puts
Map.Entry s it receives into a Hazelcast
IMap with the specified name. |
static <T,K,V> Sink<T> |
Sinks.mapWithEntryProcessor(IMap<? super K,? super V> map,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends EntryProcessor<K,V>> toEntryProcessorFn)
Returns a sink that uses the items it receives to create
EntryProcessor s it submits to a Hazelcast IMap with the
specified name. |
static <E,K,V> Sink<E> |
Sinks.mapWithEntryProcessor(String mapName,
FunctionEx<? super E,? extends K> toKeyFn,
FunctionEx<? super E,? extends EntryProcessor<K,V>> toEntryProcessorFn)
Returns a sink that uses the items it receives to create
EntryProcessor s it submits to a Hazelcast IMap with the
specified name. |
static <T,K,V> Sink<T> |
Sinks.mapWithMerging(IMap<? super K,? super V> map,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn,
BinaryOperatorEx<V> mergeFn)
Returns a sink that uses the supplied functions to extract the key
and value with which to update a Hazelcast
IMap . |
static <K,V,V_IN extends V> |
Sinks.mapWithMerging(IMap<? super K,V> map,
BinaryOperatorEx<V> mergeFn)
Convenience for
Sinks.mapWithMerging(IMap, FunctionEx, FunctionEx,
BinaryOperatorEx) with Map.Entry as input item. |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.mapWithMerging(String mapName,
BinaryOperatorEx<? super V> mergeFn)
Convenience for
Sinks.mapWithMerging(String, FunctionEx, FunctionEx,
BinaryOperatorEx) with Map.Entry as input item. |
static <T,K,V> Sink<T> |
Sinks.mapWithMerging(String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn,
BinaryOperatorEx<V> mergeFn)
Returns a sink that uses the supplied functions to extract the key
and value with which to update a Hazelcast
IMap . |
static <K,V,E extends Map.Entry<K,V>> |
Sinks.mapWithUpdating(IMap<? super K,? super V> map,
BiFunctionEx<? super V,? super E,? extends V> updateFn)
Convenience for
Sinks.mapWithUpdating(IMap, FunctionEx,
BiFunctionEx) with Map.Entry as the input item. |
static <T,K,V> Sink<T> |
Sinks.mapWithUpdating(IMap<? super K,? super V> map,
FunctionEx<? super T,? extends K> toKeyFn,
BiFunctionEx<? super V,? super T,? extends V> updateFn)
Returns a sink that uses the supplied key-extracting and value-updating
functions to update a Hazelcast
IMap . |
static <K,V,E extends Map.Entry<K,V>> |
Sinks.mapWithUpdating(String mapName,
BiFunctionEx<? super V,? super E,? extends V> updateFn)
Convenience for
Sinks.mapWithUpdating(String, FunctionEx,
BiFunctionEx) with Map.Entry as the input item. |
static <T,K,V> Sink<T> |
Sinks.mapWithUpdating(String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
BiFunctionEx<? super V,? super T,? extends V> updateFn)
Returns a sink that uses the supplied key-extracting and value-updating
functions to update a Hazelcast
IMap . |
static <T> Sink<T> |
Sinks.noop()
Returns a sink which discards all received items.
|
static <T extends Map.Entry> |
Sinks.remoteCache(String cacheName,
ClientConfig clientConfig)
Returns a sink that puts
Map.Entry s it receives into a Hazelcast
ICache with the specified name in a remote cluster identified by
the supplied ClientConfig . |
static <T> Sink<T> |
Sinks.remoteList(String listName,
ClientConfig clientConfig)
Returns a sink that adds the items it receives to a Hazelcast
IList with the specified name in a remote cluster identified by the
supplied ClientConfig . |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.remoteMap(String mapName,
ClientConfig clientConfig)
Returns a sink that puts
Map.Entry s it receives into a Hazelcast
IMap with the specified name in a remote cluster identified by
the supplied ClientConfig . |
static <E,K,V> Sink<E> |
Sinks.remoteMapWithEntryProcessor(String mapName,
ClientConfig clientConfig,
FunctionEx<? super E,? extends K> toKeyFn,
FunctionEx<? super E,? extends EntryProcessor<K,V>> toEntryProcessorFn)
Returns a sink equivalent to
Sinks.mapWithEntryProcessor(java.lang.String, com.hazelcast.jet.function.FunctionEx<? super E, ? extends K>, com.hazelcast.jet.function.FunctionEx<? super E, ? extends com.hazelcast.map.EntryProcessor<K, V>>) , but for a map
in a remote Hazelcast cluster identified by the supplied ClientConfig . |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.remoteMapWithMerging(String mapName,
ClientConfig clientConfig,
BinaryOperatorEx<V> mergeFn)
Convenience for
Sinks.remoteMapWithMerging(java.lang.String, com.hazelcast.client.config.ClientConfig, com.hazelcast.jet.function.FunctionEx<? super T, ? extends K>, com.hazelcast.jet.function.FunctionEx<? super T, ? extends V>, com.hazelcast.jet.function.BinaryOperatorEx<V>) with Map.Entry as
input item. |
static <T,K,V> Sink<T> |
Sinks.remoteMapWithMerging(String mapName,
ClientConfig clientConfig,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn,
BinaryOperatorEx<V> mergeFn)
Returns a sink equivalent to
Sinks.mapWithMerging(String, BinaryOperatorEx) ,
but for a map in a remote Hazelcast cluster identified by the supplied
ClientConfig . |
static <K,V,E extends Map.Entry<K,V>> |
Sinks.remoteMapWithUpdating(String mapName,
ClientConfig clientConfig,
BiFunctionEx<? super V,? super E,? extends V> updateFn)
|
static <T,K,V> Sink<T> |
Sinks.remoteMapWithUpdating(String mapName,
ClientConfig clientConfig,
FunctionEx<? super T,? extends K> toKeyFn,
BiFunctionEx<? super V,? super T,? extends V> updateFn)
Returns a sink equivalent to
Sinks.mapWithUpdating(java.lang.String, com.hazelcast.jet.function.FunctionEx<? super T, ? extends K>, com.hazelcast.jet.function.BiFunctionEx<? super V, ? super T, ? extends V>) , but for a map
in a remote Hazelcast cluster identified by the supplied ClientConfig . |
static <T> Sink<T> |
Sinks.socket(String host,
int port)
Convenience for
Sinks.socket(String, int, FunctionEx,
Charset) with Object.toString as the conversion function and
UTF-8 as the charset. |
static <T> Sink<T> |
Sinks.socket(String host,
int port,
FunctionEx<? super T,? extends String> toStringFn)
Convenience for
Sinks.socket(String, int, FunctionEx,
Charset) with UTF-8 as the charset. |
static <T> Sink<T> |
Sinks.socket(String host,
int port,
FunctionEx<? super T,? extends String> toStringFn,
Charset charset)
Returns a sink that connects to the specified TCP socket and writes to
it a string representation of the items it receives.
|
Modifier and Type | Method and Description |
---|---|
SinkStage |
GeneralStage.drainTo(Sink<? super T> sink)
Attaches a sink stage, one that accepts data but doesn't emit any.
|
<T> SinkStage |
Pipeline.drainTo(Sink<? super T> sink,
GeneralStage<? extends T> stage0,
GeneralStage<? extends T> stage1,
GeneralStage<? extends T>... moreStages)
Attaches the supplied sink to two or more pipeline stages.
|
Copyright © 2019 Hazelcast, Inc.. All rights reserved.