Package | Description |
---|---|
com.hazelcast.jet.kafka |
Apache Kafka reader/writer support for Hazelcast Jet.
|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
com.hazelcast.jet.pipeline.test |
This package contains various mock sources to help with pipeline testing
and development.
|
Modifier and Type | Method and Description |
---|---|
static <K,V,T> StreamSource<T> |
KafkaSources.kafka(Properties properties,
FunctionEx<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,T> projectionFn,
String... topics)
Returns a source that consumes one or more Apache Kafka topics and emits
items from them as
Map.Entry instances. |
static <K,V> StreamSource<Map.Entry<K,V>> |
KafkaSources.kafka(Properties properties,
String... topics)
Convenience for
KafkaSources.kafka(Properties, FunctionEx, String...)
wrapping the output in Map.Entry . |
Modifier and Type | Method and Description |
---|---|
StreamSource<T> |
SourceBuilder.Stream.build()
Builds and returns the unbounded stream source.
|
StreamSource<T> |
SourceBuilder.TimestampedStream.build()
Builds and returns the timestamped stream source.
|
StreamSource<javax.jms.Message> |
JmsSourceBuilder.build()
Convenience for
JmsSourceBuilder.build(FunctionEx) . |
<T> StreamSource<T> |
JmsSourceBuilder.build(FunctionEx<? super javax.jms.Message,? extends T> projectionFn)
Creates and returns the JMS
StreamSource with the supplied
components and the projection function projectionFn . |
StreamSource<String> |
FileSourceBuilder.buildWatcher()
Convenience for
FileSourceBuilder.buildWatcher(BiFunctionEx) . |
<T> StreamSource<T> |
FileSourceBuilder.buildWatcher(BiFunctionEx<String,String,? extends T> mapOutputFn)
Builds a source that emits a stream of lines of text coming from files in
the watched directory (but not its subdirectories).
|
static <K,V> StreamSource<Map.Entry<K,V>> |
Sources.cacheJournal(String cacheName,
JournalInitialPosition initialPos)
Convenience for
Sources.cacheJournal(String, PredicateEx,
FunctionEx, JournalInitialPosition)
which will pass only CREATED and UPDATED
events and will project the event's key and new value into a Map.Entry . |
static <T,K,V> StreamSource<T> |
Sources.cacheJournal(String cacheName,
PredicateEx<? super EventJournalCacheEvent<K,V>> predicateFn,
FunctionEx<? super EventJournalCacheEvent<K,V>,? extends T> projectionFn,
JournalInitialPosition initialPos)
Returns a source that will stream the
EventJournalCacheEvent
events of a Hazelcast ICache with the specified name. |
static StreamSource<String> |
Sources.fileWatcher(String watchedDirectory)
A source to stream lines added to files in a directory.
|
static StreamSource<javax.jms.Message> |
Sources.jmsQueue(SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier,
String name)
Convenience for
Sources.jmsQueueBuilder(SupplierEx) . |
static StreamSource<javax.jms.Message> |
Sources.jmsTopic(SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier,
String name)
Convenience for
Sources.jmsTopicBuilder(SupplierEx) . |
static <K,V> StreamSource<Map.Entry<K,V>> |
Sources.mapJournal(IMap<? extends K,? extends V> map,
JournalInitialPosition initialPos)
Convenience for
Sources.mapJournal(IMap, PredicateEx,
FunctionEx, JournalInitialPosition)
which will pass only ADDED and UPDATED
events and will project the event's key and new value into a Map.Entry . |
static <T,K,V> StreamSource<T> |
Sources.mapJournal(IMap<? extends K,? extends V> map,
PredicateEx<? super EventJournalMapEvent<K,V>> predicateFn,
FunctionEx<? super EventJournalMapEvent<K,V>,? extends T> projectionFn,
JournalInitialPosition initialPos)
Returns a source that will stream
EventJournalMapEvent s of the
given Hazelcast IMap . |
static <K,V> StreamSource<Map.Entry<K,V>> |
Sources.mapJournal(String mapName,
JournalInitialPosition initialPos)
Convenience for
Sources.mapJournal(String, PredicateEx,
FunctionEx, JournalInitialPosition)
which will pass only ADDED and
UPDATED events and will project the
event's key and new value into a Map.Entry . |
static <T,K,V> StreamSource<T> |
Sources.mapJournal(String mapName,
PredicateEx<? super EventJournalMapEvent<K,V>> predicateFn,
FunctionEx<? super EventJournalMapEvent<K,V>,? extends T> projectionFn,
JournalInitialPosition initialPos)
Returns a source that will stream
EventJournalMapEvent s of the
Hazelcast IMap with the specified name. |
static <K,V> StreamSource<Map.Entry<K,V>> |
Sources.remoteCacheJournal(String cacheName,
ClientConfig clientConfig,
JournalInitialPosition initialPos)
Convenience for
Sources.remoteCacheJournal(String, ClientConfig,
PredicateEx, FunctionEx, JournalInitialPosition)
which will pass only
CREATED
and UPDATED
events and will project the event's key and new value
into a Map.Entry . |
static <T,K,V> StreamSource<T> |
Sources.remoteCacheJournal(String cacheName,
ClientConfig clientConfig,
PredicateEx<? super EventJournalCacheEvent<K,V>> predicateFn,
FunctionEx<? super EventJournalCacheEvent<K,V>,? extends T> projectionFn,
JournalInitialPosition initialPos)
Returns a source that will stream the
EventJournalCacheEvent
events of the Hazelcast ICache with the specified name from a
remote cluster. |
static <K,V> StreamSource<Map.Entry<K,V>> |
Sources.remoteMapJournal(String mapName,
ClientConfig clientConfig,
JournalInitialPosition initialPos)
Convenience for
Sources.remoteMapJournal(String, ClientConfig,
PredicateEx, FunctionEx, JournalInitialPosition)
which will pass only ADDED
and UPDATED events and will
project the event's key and new value into a Map.Entry . |
static <T,K,V> StreamSource<T> |
Sources.remoteMapJournal(String mapName,
ClientConfig clientConfig,
PredicateEx<? super EventJournalMapEvent<K,V>> predicateFn,
FunctionEx<? super EventJournalMapEvent<K,V>,? extends T> projectionFn,
JournalInitialPosition initialPos)
Returns a source that will stream the
EventJournalMapEvent
events of the Hazelcast IMap with the specified name from a
remote cluster. |
StreamSource<T> |
StreamSource.setPartitionIdleTimeout(long timeoutMillis)
Sets a timeout after which idle partitions will be excluded from
watermark coalescing.
|
static StreamSource<String> |
Sources.socket(String host,
int port)
Convenience for
socket(host, port, charset) with
UTF-8 as the charset. |
static StreamSource<String> |
Sources.socket(String host,
int port,
Charset charset)
Returns a source which connects to the specified socket and emits lines
of text received from it.
|
static <T> StreamSource<T> |
Sources.streamFromProcessor(String sourceName,
ProcessorMetaSupplier metaSupplier)
Returns an unbounded (event stream) source constructed directly from the given
Core API processor meta-supplier.
|
static <T> StreamSource<T> |
Sources.streamFromProcessorWithWatermarks(String sourceName,
Function<EventTimePolicy<? super T>,ProcessorMetaSupplier> metaSupplierFn,
boolean supportsNativeTimestamps)
Returns an unbounded (event stream) source that will use the supplied
function to create processor meta-suppliers as required by the Core API.
|
Modifier and Type | Method and Description |
---|---|
<T> StreamSourceStage<T> |
Pipeline.drawFrom(StreamSource<? extends T> source)
Returns a pipeline stage that represents an unbounded data source (i.e., an
event stream).
|
Modifier and Type | Method and Description |
---|---|
static StreamSource<SimpleEvent> |
TestSources.itemStream(int itemsPerSecond)
Returns a streaming source which generates events of type
SimpleEvent at
the specified rate infinitely. |
static <T> StreamSource<T> |
TestSources.itemStream(int itemsPerSecond,
GeneratorFunction<? extends T> generatorFn)
Returns a streaming source which generates events created by the
generatorFn at the specified rate infinitely. |
Copyright © 2019 Hazelcast, Inc.. All rights reserved.