public interface Pipeline
The Pipeline
object is a container of all the stages defined on
a pipeline: the source stages obtained directly from it by calling drawFrom(BatchSource)
as well as all the stages attached (directly or
indirectly) to them.
Note that there is no simple one-to-one correspondence between pipeline stages and Core API's DAG vertices. Some stages map to several vertices (e.g., grouping and co-grouping are implemented as a cascade of two vertices) and some stages may be merged with others into a single vertex (e.g., a cascade of map/filter/flatMap stages can be fused into one vertex).
Modifier and Type | Method and Description |
---|---|
static Pipeline |
create()
Creates a new, empty pipeline.
|
<T> SinkStage |
drainTo(Sink<? super T> sink,
GeneralStage<? extends T> stage0,
GeneralStage<? extends T> stage1,
GeneralStage<? extends T>... moreStages)
Attaches the supplied sink to two or more pipeline stages.
|
<T> BatchStage<T> |
drawFrom(BatchSource<? extends T> source)
Returns a pipeline stage that represents a bounded (batch) data source.
|
<T> StreamSourceStage<T> |
drawFrom(StreamSource<? extends T> source)
Returns a pipeline stage that represents an unbounded data source (i.e., an
event stream).
|
DAG |
toDag()
Transforms the pipeline into a Jet DAG, which can be submitted for
execution to a Jet instance.
|
String |
toDotString()
Returns a DOT format (graphviz) representation of the Pipeline.
|
@Nonnull <T> BatchStage<T> drawFrom(@Nonnull BatchSource<? extends T> source)
T
- the type of source data itemssource
- the definition of the source from which the stage draws data@Nonnull <T> StreamSourceStage<T> drawFrom(@Nonnull StreamSource<? extends T> source)
T
- the type of source data itemssource
- the definition of the source from which the stage draws data@Nonnull <T> SinkStage drainTo(@Nonnull Sink<? super T> sink, @Nonnull GeneralStage<? extends T> stage0, @Nonnull GeneralStage<? extends T> stage1, @Nonnull GeneralStage<? extends T>... moreStages)
SinkStage
representing the sink. You need this method when you
want to drain more than one stage to the same sink. In the typical case
you'll use GeneralStage.drainTo(Sink)
instead.T
- the type of data being drained to the sink@Nonnull DAG toDag()
Copyright © 2019 Hazelcast, Inc.. All rights reserved.