Interface ProcessorMetaSupplier
- All Superinterfaces:
Serializable
- Functional Interface:
- This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.
@FunctionalInterface public interface ProcessorMetaSupplier extends Serializable
ProcessorSupplier
instances. The starting point of
the chain leading to the eventual creation of Processor
instances
on each cluster member:
-
client creates
ProcessorMetaSupplier
as a part of the DAG; - serializes it and sends to a cluster member;
-
the member deserializes and uses it to create one
ProcessorSupplier
for each cluster member; -
serializes each
ProcessorSupplier
and sends it to its target member; -
the target member deserializes and uses it to instantiate as many instances
of
Processor
as requested by the parallelism property on the correspondingVertex
.
ProcessorSupplier
s this meta-supplier will
be given access to the Hazelcast instance and, in particular, its cluster topology
and partitioning services. It can use the information from these services to
precisely parameterize each Processor
instance that will be created on
each member.- Since:
- 3.0
-
Nested Class Summary
Nested Classes Modifier and Type Interface Description static interface
ProcessorMetaSupplier.Context
Context passed to the meta-supplier at init time on the member that received a job request from the client. -
Method Summary
Modifier and Type Method Description default void
close(Throwable error)
Called on coordinator member after execution has finished on all members, successfully or not.static ProcessorMetaSupplier
forceTotalParallelismOne(ProcessorSupplier supplier)
Variant offorceTotalParallelismOne(ProcessorSupplier, String)
where the node for the supplier will be chosen randomly.static ProcessorMetaSupplier
forceTotalParallelismOne(ProcessorSupplier supplier, String partitionKey)
Wraps the providedProcessorSupplier
into a meta-supplier that will only use the givenProcessorSupplier
on a single node.Function<? super Address,? extends ProcessorSupplier>
get(List<Address> addresses)
Called to create a mapping from memberAddress
to theProcessorSupplier
that will be sent to that member.default Map<String,String>
getTags()
Returns the metadata on this supplier, a string-to-string map.default void
init(ProcessorMetaSupplier.Context context)
Called on the cluster member that receives the client request, after deserializing the meta-supplier instance.static ProcessorMetaSupplier
of(int preferredLocalParallelism, FunctionEx<? super Address,? extends ProcessorSupplier> addressToSupplier)
Factory method that creates aProcessorMetaSupplier
from the supplied function that maps a cluster member address to aProcessorSupplier
.static ProcessorMetaSupplier
of(int preferredLocalParallelism, SupplierEx<? extends Processor> procSupplier)
Factory method that wraps the givenSupplier<Processor>
and uses it as the supplier of allProcessor
instances.static ProcessorMetaSupplier
of(int preferredLocalParallelism, ProcessorSupplier procSupplier)
Factory method that wraps the givenProcessorSupplier
and returns the same instance for each givenAddress
.static ProcessorMetaSupplier
of(FunctionEx<? super Address,? extends ProcessorSupplier> addressToSupplier)
Factory method that creates aProcessorMetaSupplier
from the supplied function that maps a cluster member address to aProcessorSupplier
.static ProcessorMetaSupplier
of(SupplierEx<? extends Processor> procSupplier)
Factory method that wraps the givenSupplier<Processor>
and uses it as the supplier of allProcessor
instances.static ProcessorMetaSupplier
of(ProcessorSupplier procSupplier)
Wraps the providedProcessorSupplier
into a meta-supplier that will always return it.static ProcessorMetaSupplier
preferLocalParallelismOne(SupplierEx<? extends Processor> procSupplier)
Variant ofpreferLocalParallelismOne(ProcessorSupplier)
where the suppliedSupplierEx<Processor>
will be wrapped into aProcessorSupplier
.static ProcessorMetaSupplier
preferLocalParallelismOne(ProcessorSupplier supplier)
Wraps the providedProcessorSupplier
into a meta-supplier that will always return it.default int
preferredLocalParallelism()
Returns the local parallelism the vertex should be configured with.
-
Method Details
-
getTags
Returns the metadata on this supplier, a string-to-string map. There is no predefined metadata; this facility exists to allow the DAG vertices to contribute some information to the execution planning phase.- Since:
- 4.0
-
preferredLocalParallelism
default int preferredLocalParallelism()Returns the local parallelism the vertex should be configured with. The default implementation returnsVertex.LOCAL_PARALLELISM_USE_DEFAULT
. -
init
Called on the cluster member that receives the client request, after deserializing the meta-supplier instance. Gives access to the Hazelcast instance's services and provides the parallelism parameters determined from the cluster size.- Throws:
Exception
-
get
@Nonnull Function<? super Address,? extends ProcessorSupplier> get(@Nonnull List<Address> addresses)Called to create a mapping from memberAddress
to theProcessorSupplier
that will be sent to that member. Jet calls this method with a list of all cluster members' addresses and the returned function must be a mapping that returns a non-null value for each given address.The method will be called once per job execution on the job's coordinator member.
init()
will have already been called. -
close
Called on coordinator member after execution has finished on all members, successfully or not. This method will be called afterProcessorSupplier.close(Throwable)
has been called on all available members. The job can be restarted later.If there is an exception during the creation of the execution plan, this method will be called regardless of whether the
init()
orget()
method have been called or not. If this method throws an exception, it will be logged and ignored; it won't be reported as a job failure.If you rely on the fact that this method is run once per cluster, it can happen that it is not called at all, if the coordinator member crashed.
- Parameters:
error
- the exception (if any) that caused the job to fail;null
in the case of successful job completion- Throws:
Exception
-
of
@Nonnull static ProcessorMetaSupplier of(int preferredLocalParallelism, @Nonnull ProcessorSupplier procSupplier)Factory method that wraps the givenProcessorSupplier
and returns the same instance for each givenAddress
.- Parameters:
preferredLocalParallelism
- the value to return frompreferredLocalParallelism()
procSupplier
- the processor supplier
-
of
Wraps the providedProcessorSupplier
into a meta-supplier that will always return it. ThepreferredLocalParallelism()
of the meta-supplier will beVertex.LOCAL_PARALLELISM_USE_DEFAULT
. -
of
@Nonnull static ProcessorMetaSupplier of(int preferredLocalParallelism, @Nonnull SupplierEx<? extends Processor> procSupplier)Factory method that wraps the givenSupplier<Processor>
and uses it as the supplier of allProcessor
instances. Specifically, returns a meta-supplier that will always return the result of callingProcessorSupplier.of(SupplierEx)
.- Parameters:
preferredLocalParallelism
- the value to return frompreferredLocalParallelism()
procSupplier
- the supplier of processors
-
of
Factory method that wraps the givenSupplier<Processor>
and uses it as the supplier of allProcessor
instances. Specifically, returns a meta-supplier that will always return the result of callingProcessorSupplier.of(SupplierEx)
. ThepreferredLocalParallelism()
of the meta-supplier will beVertex.LOCAL_PARALLELISM_USE_DEFAULT
. -
of
@Nonnull static ProcessorMetaSupplier of(int preferredLocalParallelism, @Nonnull FunctionEx<? super Address,? extends ProcessorSupplier> addressToSupplier)Factory method that creates aProcessorMetaSupplier
from the supplied function that maps a cluster member address to aProcessorSupplier
.- Parameters:
preferredLocalParallelism
- the value to return frompreferredLocalParallelism()
addressToSupplier
- the mapping from address to ProcessorSupplier
-
of
@Nonnull static ProcessorMetaSupplier of(@Nonnull FunctionEx<? super Address,? extends ProcessorSupplier> addressToSupplier)Factory method that creates aProcessorMetaSupplier
from the supplied function that maps a cluster member address to aProcessorSupplier
. ThepreferredLocalParallelism()
of the meta-supplier will beVertex.LOCAL_PARALLELISM_USE_DEFAULT
. -
preferLocalParallelismOne
@Nonnull static ProcessorMetaSupplier preferLocalParallelismOne(@Nonnull ProcessorSupplier supplier)Wraps the providedProcessorSupplier
into a meta-supplier that will always return it. ThepreferredLocalParallelism()
of the meta-supplier will be one, i.e., no local parallelization.The parallelism will be overridden if the
Vertex.localParallelism(int)
is set to a specific value. -
preferLocalParallelismOne
@Nonnull static ProcessorMetaSupplier preferLocalParallelismOne(@Nonnull SupplierEx<? extends Processor> procSupplier)Variant ofpreferLocalParallelismOne(ProcessorSupplier)
where the suppliedSupplierEx<Processor>
will be wrapped into aProcessorSupplier
. -
forceTotalParallelismOne
@Nonnull static ProcessorMetaSupplier forceTotalParallelismOne(@Nonnull ProcessorSupplier supplier)Variant offorceTotalParallelismOne(ProcessorSupplier, String)
where the node for the supplier will be chosen randomly. -
forceTotalParallelismOne
@Nonnull static ProcessorMetaSupplier forceTotalParallelismOne(@Nonnull ProcessorSupplier supplier, @Nonnull String partitionKey)Wraps the providedProcessorSupplier
into a meta-supplier that will only use the givenProcessorSupplier
on a single node. The node will be chosen according to thepartitionKey
supplied. This is mainly provided as a convenience for implementing non-distributed sources where data can't be read in parallel by multiple consumers. When used as a sink or intermediate vertex, the DAG should ensure that only the processor instance on the designated node receives any data, otherwise anIllegalStateException
will be thrown.The vertex containing the
ProcessorMetaSupplier
must have a local parallelism setting of 1, otherwise {code IllegalArgumentException} is thrown.- Parameters:
supplier
- the supplier that will be wrappedpartitionKey
- the supplier will only be created on the node that owns the supplied partition key- Returns:
- the wrapped
ProcessorMetaSupplier
- Throws:
IllegalArgumentException
- if vertex has local parallelism setting of greater than 1
-