Package | Description |
---|---|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
com.hazelcast.jet.pipeline.test |
This package contains various mock sources to help with pipeline testing
and development.
|
Modifier and Type | Method and Description |
---|---|
StreamStage<T> |
GeneralStage.addTimestamps(ToLongFunctionEx<? super T> timestampFn,
long allowedLag)
Adds a timestamp to each item in the stream using the supplied function
and specifies the allowed amount of disorder between them.
|
<R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items that belong to a given window.
|
<R> StreamStage<KeyedWindowResult<K,R>> |
StageWithKeyAndWindow.aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)
Attaches a stage that performs the given group-and-aggregate operation.
|
default <T1,R0,R1> StreamStage<WindowResult<Tuple2<R0,R1>>> |
StageWithWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that performs the given co-aggregate operations over
the items from this stage and
stage1 you supply. |
default <T1,R0,R1> StreamStage<KeyedWindowResult<K,Tuple2<R0,R1>>> |
StageWithKeyAndWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
<T1,R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate2(StreamStage<T1> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
<T1,R> StreamStage<KeyedWindowResult<K,R>> |
StageWithKeyAndWindow.aggregate2(StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
default <T1,T2,R0,R1,R2> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2> |
StageWithKeyAndWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStageWithKey<T2,? extends K> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
<T1,T2,R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
<T1,T2,R> StreamStage<KeyedWindowResult<K,R>> |
StageWithKeyAndWindow.aggregate3(StreamStageWithKey<T1,? extends K> stage1,
StreamStageWithKey<T2,? extends K> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from this stage as well as
stage1 and stage2 you supply. |
default <R> StreamStage<R> |
StreamStage.apply(FunctionEx<? super StreamStage<T>,? extends StreamStage<R>> transformFn)
Transforms
this stage using the provided transformFn and
returns the transformed stage. |
StreamStage<KeyedWindowResult<K,ItemsByTag>> |
WindowGroupAggregateBuilder.build()
Creates and returns a pipeline stage that performs a windowed
cogroup-and-aggregate operation on the stages registered with this
builder object.
|
StreamStage<WindowResult<ItemsByTag>> |
WindowAggregateBuilder.build()
Creates and returns a pipeline stage that performs a windowed
co-aggregation of the stages registered with this builder object.
|
<R> StreamStage<KeyedWindowResult<K,R>> |
WindowGroupAggregateBuilder1.build(AggregateOperation<?,? extends R> aggrOp)
Creates and returns a pipeline stage that performs a windowed
cogroup-and-aggregate of the pipeline stages registered with this builder object.
|
<A,R> StreamStage<WindowResult<R>> |
WindowAggregateBuilder1.build(AggregateOperation<A,R> aggrOp)
Creates and returns a pipeline stage that performs a windowed
co-aggregation of the pipeline stages registered with this builder
object.
|
<R> StreamStage<R> |
StreamHashJoinBuilder.build(BiFunctionEx<T0,ItemsByTag,R> mapToOutputFn)
Builds a new pipeline stage that performs the hash-join operation.
|
<R> StreamStage<R> |
StreamStageWithKey.customTransform(String stageName,
ProcessorMetaSupplier procSupplier) |
<R> StreamStage<R> |
StreamStage.customTransform(String stageName,
ProcessorMetaSupplier procSupplier) |
default <R> StreamStage<R> |
StreamStageWithKey.customTransform(String stageName,
ProcessorSupplier procSupplier) |
default <R> StreamStage<R> |
StreamStage.customTransform(String stageName,
ProcessorSupplier procSupplier) |
default <R> StreamStage<R> |
StreamStageWithKey.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
default <R> StreamStage<R> |
StreamStage.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
default StreamStage<WindowResult<T>> |
StageWithWindow.distinct()
Attaches a stage that passes through just the items that are distinct
within their window (no two items emitted for a window are equal).
|
default StreamStage<KeyedWindowResult<K,T>> |
StageWithKeyAndWindow.distinct()
Attaches a stage that passes through just the items that are distinct
within their window according to the grouping key (no two items emitted
for a window map to the same key).
|
StreamStage<T> |
StreamStage.filter(PredicateEx<T> filterFn) |
<S> StreamStage<T> |
StreamStageWithKey.filterStateful(long ttl,
SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn)
Attaches a stage that performs a stateful filtering operation.
|
default <S> StreamStage<T> |
StreamStageWithKey.filterStateful(SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn) |
<S> StreamStage<T> |
StreamStage.filterStateful(SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn) |
<C> StreamStage<T> |
StreamStage.filterUsingContext(ContextFactory<C> contextFactory,
BiPredicateEx<? super C,? super T> filterFn) |
<C> StreamStage<T> |
StreamStageWithKey.filterUsingContext(ContextFactory<C> contextFactory,
TriPredicate<? super C,? super K,? super T> filterFn) |
<C> StreamStage<T> |
StreamStage.filterUsingContextAsync(ContextFactory<C> contextFactory,
BiFunctionEx<? super C,? super T,? extends CompletableFuture<Boolean>> filterAsyncFn) |
<C> StreamStage<T> |
StreamStageWithKey.filterUsingContextAsync(ContextFactory<C> contextFactory,
TriFunction<? super C,? super K,? super T,CompletableFuture<Boolean>> filterAsyncFn) |
<R> StreamStage<R> |
StreamStage.flatMap(FunctionEx<? super T,? extends Traverser<? extends R>> flatMapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.flatMapStateful(long ttl,
SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> flatMapFn,
TriFunction<? super S,? super K,? super Long,? extends Traverser<R>> onEvictFn)
Attaches a stage that performs a stateful flat-mapping operation.
|
<S,R> StreamStage<R> |
StreamStage.flatMapStateful(SupplierEx<? extends S> createFn,
BiFunctionEx<? super S,? super T,? extends Traverser<R>> flatMapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.flatMapStateful(SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> flatMapFn) |
<C,R> StreamStage<R> |
StreamStage.flatMapUsingContext(ContextFactory<C> contextFactory,
BiFunctionEx<? super C,? super T,? extends Traverser<R>> flatMapFn) |
<C,R> StreamStage<R> |
StreamStageWithKey.flatMapUsingContext(ContextFactory<C> contextFactory,
TriFunction<? super C,? super K,? super T,? extends Traverser<? extends R>> flatMapFn) |
<C,R> StreamStage<R> |
StreamStage.flatMapUsingContextAsync(ContextFactory<C> contextFactory,
BiFunctionEx<? super C,? super T,? extends CompletableFuture<Traverser<R>>> flatMapAsyncFn) |
<C,R> StreamStage<R> |
StreamStageWithKey.flatMapUsingContextAsync(ContextFactory<C> contextFactory,
TriFunction<? super C,? super K,? super T,CompletableFuture<Traverser<R>>> flatMapAsyncFn) |
<K,T1_IN,T1,R> |
StreamStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
BiFunctionEx<T,T1,R> mapToOutputFn) |
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
StreamStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
TriFunction<T,T1,T2,R> mapToOutputFn) |
<R> StreamStage<R> |
StreamStage.map(FunctionEx<? super T,? extends R> mapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.mapStateful(long ttl,
SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends R> mapFn,
TriFunction<? super S,? super K,? super Long,? extends R> onEvictFn)
Attaches a stage that performs a stateful mapping operation.
|
<S,R> StreamStage<R> |
StreamStage.mapStateful(SupplierEx<? extends S> createFn,
BiFunctionEx<? super S,? super T,? extends R> mapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.mapStateful(SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends R> mapFn) |
<C,R> StreamStage<R> |
StreamStage.mapUsingContext(ContextFactory<C> contextFactory,
BiFunctionEx<? super C,? super T,? extends R> mapFn) |
<C,R> StreamStage<R> |
StreamStageWithKey.mapUsingContext(ContextFactory<C> contextFactory,
TriFunction<? super C,? super K,? super T,? extends R> mapFn) |
<C,R> StreamStage<R> |
StreamStage.mapUsingContextAsync(ContextFactory<C> contextFactory,
BiFunctionEx<? super C,? super T,? extends CompletableFuture<R>> mapAsyncFn) |
<C,R> StreamStage<R> |
StreamStageWithKey.mapUsingContextAsync(ContextFactory<C> contextFactory,
TriFunction<? super C,? super K,? super T,CompletableFuture<R>> mapAsyncFn) |
default <V,R> StreamStage<R> |
StreamStageWithKey.mapUsingIMap(IMap<K,V> iMap,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingIMap(IMap<K,V> iMap,
FunctionEx<? super T,? extends K> lookupKeyFn,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
default <V,R> StreamStage<R> |
StreamStageWithKey.mapUsingIMap(String mapName,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingIMap(String mapName,
FunctionEx<? super T,? extends K> lookupKeyFn,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingReplicatedMap(ReplicatedMap<K,V> replicatedMap,
FunctionEx<? super T,? extends K> lookupKeyFn,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingReplicatedMap(String mapName,
FunctionEx<? super T,? extends K> lookupKeyFn,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
StreamStage<T> |
StreamStage.merge(StreamStage<? extends T> other)
Attaches a stage that emits all the items from this stage as well as all
the items from the supplied stage.
|
default StreamStage<T> |
StreamStage.peek() |
default StreamStage<T> |
StreamStage.peek(FunctionEx<? super T,? extends CharSequence> toStringFn) |
StreamStage<T> |
StreamStage.peek(PredicateEx<? super T> shouldLogFn,
FunctionEx<? super T,? extends CharSequence> toStringFn) |
default <A,R> StreamStage<Map.Entry<K,R>> |
StreamStageWithKey.rollingAggregate(AggregateOperation1<? super T,A,? extends R> aggrOp) |
default <A,R> StreamStage<R> |
StreamStage.rollingAggregate(AggregateOperation1<? super T,A,? extends R> aggrOp) |
default <A,R> StreamStage<Map.Entry<K,R>> |
StreamStageWithKey.rollingAggregate(long ttl,
AggregateOperation1<? super T,A,? extends R> aggrOp)
Attaches a rolling aggregation stage.
|
StreamStage<T> |
StreamStage.setLocalParallelism(int localParallelism) |
StreamStage<T> |
StreamStage.setName(String name) |
StreamStage<T> |
StageWithWindow.streamStage()
Returns the pipeline stage associated with this object.
|
default StreamStage<T> |
StreamSourceStage.withIngestionTimestamps()
Declares that the source will assign the time of ingestion as the event
timestamp.
|
StreamStage<T> |
StreamSourceStage.withNativeTimestamps(long allowedLag)
Declares that the stream will use the source's native timestamps.
|
StreamStage<T> |
StreamSourceStage.withoutTimestamps()
Declares that the source will not assign any timestamp to the events it
emits.
|
StreamStage<T> |
StreamSourceStage.withTimestamps(ToLongFunctionEx<? super T> timestampFn,
long allowedLag)
Declares that the source will extract timestamps from the stream items.
|
Modifier and Type | Method and Description |
---|---|
<E> Tag<E> |
WindowAggregateBuilder1.add(StreamStage<E> stage)
Adds another stage that will contribute its data to the aggregate
operation to be performed.
|
<T,R> Tag<R> |
WindowAggregateBuilder.add(StreamStage<T> stage,
AggregateOperation1<? super T,?,? extends R> aggrOp)
Adds another stage that will contribute its data to the aggregate
operation to be performed.
|
default <T1,R0,R1> StreamStage<WindowResult<Tuple2<R0,R1>>> |
StageWithWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that performs the given co-aggregate operations over
the items from this stage and
stage1 you supply. |
<T1,R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate2(StreamStage<T1> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
<T1,T2,R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
<T1,T2,R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
StreamStage<T> |
StreamStage.merge(StreamStage<? extends T> other)
Attaches a stage that emits all the items from this stage as well as all
the items from the supplied stage.
|
Modifier and Type | Method and Description |
---|---|
default <R> StreamStage<R> |
StreamStage.apply(FunctionEx<? super StreamStage<T>,? extends StreamStage<R>> transformFn)
Transforms
this stage using the provided transformFn and
returns the transformed stage. |
default <R> StreamStage<R> |
StreamStage.apply(FunctionEx<? super StreamStage<T>,? extends StreamStage<R>> transformFn)
Transforms
this stage using the provided transformFn and
returns the transformed stage. |
Modifier and Type | Method and Description |
---|---|
static <T> FunctionEx<StreamStage<T>,StreamStage<T>> |
Assertions.assertCollectedEventually(int timeout,
ConsumerEx<? super List<T>> assertFn)
Collects all the received items into a list and runs the
assertFn
every time a new item is received. |
static <T> FunctionEx<StreamStage<T>,StreamStage<T>> |
Assertions.assertCollectedEventually(int timeout,
ConsumerEx<? super List<T>> assertFn)
Collects all the received items into a list and runs the
assertFn
every time a new item is received. |
Copyright © 2019 Hazelcast, Inc.. All rights reserved.