Package com.hazelcast.jet.core

Class DAG

java.lang.Object

com.hazelcast.jet.core.DAG

All Implemented Interfaces:

DataSerializable, IdentifiedDataSerializable, Iterable<Vertex>

public class DAG
extends Object
implements IdentifiedDataSerializable, Iterable<Vertex>

Describes a computation to be performed by the Jet computation engine. A vertex represents a unit of data processing and an edge represents the path along which the data travels to the next vertex.

The work of a single vertex is parallelized and distributed, so that there are several instances of the Processor type on each member corresponding to it. Whenever possible, each instance should be tasked with only a slice of the total data and a partitioning strategy can be employed to ensure that the data sent to each vertex is collated by a partitioning key.

There are three basic kinds of vertices:

1. source with just outbound edges;
2. processor with both inbound and outbound edges;
3. sink with just inbound edges.

Data travels from sources to sinks and is transformed and reshaped as it passes through the processors.

Since:

3.0

Constructor Summary

Constructors

Constructor	Description
DAG()

Method Summary

Modifier and Type	Method	Description
DAG	edge(Edge edge)	Adds an edge to this DAG.
int	getClassId()	Returns type identifier for this class.
int	getFactoryId()	Returns DataSerializableFactory factory ID for this class.
List<Edge>	getInboundEdges(String vertexName)	Returns the inbound edges connected to the vertex with the given name.
List<Edge>	getOutboundEdges(String vertexName)	Returns the outbound edges connected to the vertex with the given name.
Vertex	getVertex(String vertexName)	Returns the vertex with the given name.
Iterator<Vertex>	iterator()	Returns an iterator over the DAG's vertices in topological order.
Vertex	newVertex(String name, SupplierEx<? extends Processor> simpleSupplier)	Creates a vertex from a Supplier<Processor> and adds it to this DAG.
Vertex	newVertex(String name, ProcessorMetaSupplier metaSupplier)	Creates a vertex from a ProcessorMetaSupplier and adds it to this DAG.
Vertex	newVertex(String name, ProcessorSupplier processorSupplier)	Creates a vertex from a ProcessorSupplier and adds it to this DAG.
void	readData(ObjectDataInput in)	Reads fields from the input stream
String	toDotString()	Returns a DOT format (graphviz) representation of the DAG.
String	toDotString(int defaultParallelism)	Returns a DOT format (graphviz) representation of the DAG and annotates the vertices using default parallelism with the supplied value.
com.hazelcast.internal.json.JsonObject	toJson(int defaultLocalParallelism)	Returns a JSON representation of the DAG.
String	toString()
String	toString(int defaultLocalParallelism)	Returns a string representation of the DAG.
DAG	vertex(Vertex vertex)	Adds a vertex to this DAG.
void	writeData(ObjectDataOutput out)	Writes object fields to output stream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Details

DAG

public DAG()

Method Details

newVertex

@Nonnull public Vertex newVertex(@Nonnull String name, @Nonnull SupplierEx<? extends Processor> simpleSupplier)

Creates a vertex from a Supplier<Processor> and adds it to this DAG.

Parameters:

name - the unique name of the vertex

simpleSupplier - the simple, parameterless supplier of Processor instances

See Also:

Vertex(String, SupplierEx)

newVertex

@Nonnull public Vertex newVertex(@Nonnull String name, @Nonnull ProcessorSupplier processorSupplier)

Creates a vertex from a ProcessorSupplier and adds it to this DAG.

Parameters:

name - the unique name of the vertex

processorSupplier - the supplier of Processor instances which will be used on all members

See Also:

Vertex(String, ProcessorSupplier)

newVertex

@Nonnull public Vertex newVertex(@Nonnull String name, @Nonnull ProcessorMetaSupplier metaSupplier)

Creates a vertex from a ProcessorMetaSupplier and adds it to this DAG.

Parameters:

name - the unique name of the vertex

metaSupplier - the meta-supplier of ProcessorSuppliers for each member

See Also:

Vertex(String, ProcessorMetaSupplier)

vertex

@Nonnull public DAG vertex(@Nonnull Vertex vertex)

Adds a vertex to this DAG. The vertex name must be unique.

edge

@Nonnull public DAG edge(@Nonnull Edge edge)

Adds an edge to this DAG. The vertices it connects must already be present in the DAG. It is an error to connect an edge to a vertex at the same ordinal as another existing edge. However, inbound and outbound ordinals are independent, so there can be two edges at the same ordinal, one inbound and one outbound.

Jet supports multigraphs, that is you can add two edges between the same tow vertices. However, they have to have different ordinals.

getInboundEdges

@Nonnull public List<Edge> getInboundEdges(@Nonnull String vertexName)

Returns the inbound edges connected to the vertex with the given name.

getOutboundEdges

@Nonnull public List<Edge> getOutboundEdges(@Nonnull String vertexName)

Returns the outbound edges connected to the vertex with the given name.

getVertex

@Nonnull public Vertex getVertex(@Nonnull String vertexName)

Returns the vertex with the given name.

iterator

@Nonnull public Iterator<Vertex> iterator()

Returns an iterator over the DAG's vertices in topological order.

Specified by:

iterator in interface Iterable<Vertex>

toString

@Nonnull public String toString()

Overrides:

toString in class Object

toString

@Nonnull public String toString(int defaultLocalParallelism)

Returns a string representation of the DAG.

Parameters:

defaultLocalParallelism - the local parallelism that will be shown if neither overridden on the vertex nor the preferred parallelism is defined by meta-supplier

toJson

@Nonnull public com.hazelcast.internal.json.JsonObject toJson(int defaultLocalParallelism)

Returns a JSON representation of the DAG.

Note: the exact structure of the JSON is unspecified.

Parameters:

defaultLocalParallelism - the local parallelism that will be shown if neither overridden on the vertex nor the preferred parallelism is defined by meta-supplier

toDotString

@Nonnull public String toDotString()

Returns a DOT format (graphviz) representation of the DAG.

toDotString

@Nonnull public String toDotString(int defaultParallelism)

Returns a DOT format (graphviz) representation of the DAG and annotates the vertices using default parallelism with the supplied value.

writeData

public void writeData(ObjectDataOutput out) throws IOException

Description copied from interface: DataSerializable

Writes object fields to output stream

Specified by:

writeData in interface DataSerializable

Parameters:

out - output

Throws:

IOException - if an I/O error occurs. In particular, an IOException may be thrown if the output stream has been closed.

readData

public void readData(ObjectDataInput in) throws IOException

Description copied from interface: DataSerializable

Reads fields from the input stream

Specified by:

readData in interface DataSerializable

Parameters:

in - input

Throws:

IOException - if an I/O error occurs. In particular, an IOException may be thrown if the input stream has been closed.

getFactoryId

public int getFactoryId()

Description copied from interface: IdentifiedDataSerializable

Returns DataSerializableFactory factory ID for this class.

Specified by:

getFactoryId in interface IdentifiedDataSerializable

Returns:

factory ID

getClassId

public int getClassId()

Description copied from interface: IdentifiedDataSerializable

Returns type identifier for this class. It should be unique per DataSerializableFactory.

Specified by:

getClassId in interface IdentifiedDataSerializable

Returns:

type ID