Class EdgeConfig

All Implemented Interfaces:

public class EdgeConfig
extends Object
implements Serializable
A configuration object for a DAG Edge that holds fine-tuning parameters that influence its performance characteristics.
See Also:
Serialized Form
  • Field Details

  • Constructor Details

  • Method Details

    • setQueueSize

      public EdgeConfig setQueueSize​(int queueSize)
      Sets the capacity of processor-to-processor concurrent queues. The value is rounded upwards to the next power of 2.

      When data needs to travel between two processors on the same cluster member, Jet sends it over a concurrent single-producer, single-consumer (SPSC) queue of fixed capacity.

      Since there are several processors executing the logic of each vertex, and since the queues are SPSC, there will be senderParallelism * receiverParallelism queues representing the edge on each member. The edge capacity should strike a balance between performance and memory usage. The default of 1024 is a good size for simple DAGs and moderate parallelism.

      this instance for fluent API
    • getQueueSize

      public int getQueueSize()
      Returns the size of the SPSC queues used to implement this edge. The real queue size will be rounded upwards to the next power of 2.
    • setReceiveWindowMultiplier

      public EdgeConfig setReceiveWindowMultiplier​(int receiveWindowMultiplier)
      Sets the scaling factor used by the adaptive receive window sizing function.

      For each distributed edge the receiving member regularly sends flow-control ("ack") packets to its sender which prevent it from sending too much data and overflowing the buffers. The sender is allowed to send the data one receive window further than the last acknowledged byte and the receive window is sized in proportion to the rate of processing at the receiver.

      Ack packets are sent in regular intervals and the receive window multiplier sets the factor of the linear relationship between the amount of data processed within one such interval and the size of the receive window.

      To put it another way, let us define an ackworth as the amount of data processed between two consecutive ack packets. The receive window multiplier determines the number of ackworths the sender can be ahead of the last acked byte.

      The default value is 3. This setting has no effect on a non-distributed edge.

      this instance for fluent API
    • getReceiveWindowMultiplier

      public int getReceiveWindowMultiplier()
      the receive window multiplier
    • setPacketSizeLimit

      public EdgeConfig setPacketSizeLimit​(int packetSizeLimit)
      For a distributed edge, data is sent to a remote member via Hazelcast network packets. Each packet is dedicated to the data of a single edge, but may contain any number of data items. This setting limits the size of the packet in bytes. Packets should be large enough to drown out any fixed overheads, but small enough to allow good interleaving with other packets.

      Note that a single item cannot straddle packets, therefore the maximum packet size can exceed the value configured here by the size of a single data item.

      The default value is 16384. This setting has no effect on a non-distributed edge.

      this instance for fluent API
    • getPacketSizeLimit

      public int getPacketSizeLimit()
      Returns the limit on the network packet size, in bytes