ProducerConsumerElement overview

ProducerConsumerElement classes

The following class diagram shows the essential structure of the ProducerConsumerElement class:


This class diagram shows that a ProducerConsumerElement has:
  • A Tag object, which can be used to hold custom data associated with the element
  • A Slots store, which holds the link state for all links and overlays within the element
  • Methods corresponding to the messages sent/received between ProducerConsumers, which update the current link state within the Slots store
  • Methods for retrieving the opposite producer or consumer slot where there is a linked producer-consumer pair
  • Methods for retrieving the active producer or consumer slot for a particular link.

ProducerConsumerElement functionality

The ProducerConsumerElement is responsible for implementing the flow control logic for exchanging events between particular elements of connected producers and consumers. This makes it possible to precisely control the number of iterations that producers can run ahead of consumers throughout an application, allowing enough depth to give the required concurrency while ensuring consumers can always keep up.

Flow control

Basic event transmission cycle

The flow of data between ProducerConsumers is controlled by a four-way handshake consisting of:
  • Query - the consumer queries the producer for an event
  • Event - the producer supplies the event when it is able and the consumer processes the event
  • Release - the consumer sends a release back to the producer which frees up the associated resource for another consumer.
  • Released - the producer sends a released notification back to the consumer to inform it that the cycle is complete.

This query cycle is depicted in the following sequence diagram:


This diagram is illustrating the following points:
  • DerivedTaskA is sending an event to DerivedTaskB
  • The derived tasks each own their own ProducerConsumer task
  • The five small pink boxes represent scheduled work items
  • The messages between those scheduled work items are asynchronous
  • The transmission of one event involves sending four messages (Query, Event, Release, Released)

Event overlay

The query cycle described previously is the basic cycle that occurs for each event delivery. However, it is also permissible to overlay multiple such cycles so that multiple events can be delivered before any of the events are released.

The purpose of this event overlaying is to deliver resources as soon as possible (i.e. start processing the next iteration concurrently with the current iteration) to ensure maximum CPU utilization. The following rules apply to event overlay:
  • The next query must not be sent until the previous event is received
  • Events can only be overlaid up to the specified maximum overlay depth
  • Events can be released in any order

For a given element, each ProducerConsumerElement stores the state of each link for each active event query. The link state consists of:
  • Link Status - what stage of the cycle the link is currently in
  • Data - event data associated with the query
  • Producer->Consumer Linkage - link from an active producer event to an active consumer event

This information is stored in an optimized structure contained within the Slots class.

The following sequence diagram shows the simplest case of event overlaying, where two events (A and B) are active simultaneously:


There are several things to note from this sequence diagram:
  • The maximum overlay depth in this example is 2
  • Event B was not queried by the consumer until Event A was received
  • In this case, event B was released before event A even though the events were queried and delivered in the order A then B
  • The release for A and B could arrive at the same instant at the producer but they will be processed sequentially because of the element scope lock
  • Additional queries cannot be sent after the maximum overlay depth has been reached - until one or more events are released
  • The derived tasks implement specific logic for when to release events that depends on their function (in this example the derived task was waiting for two events to have arrived before releasing them)
  • This sequence shows event B arriving before event A was released but it could arrive after event A has been through the full query cycle.

As the maximum overlay depth is increased the number of possible interleavings of the handshaking messages increases rapidly and it is not practical (or particularly helpful) to try and represent them on a sequence diagram but the principle of operation is the same as with the dual-overlay described above.

Producer and consumer linkage

Producer and consumer slots can be linked together so that objects that produce an event in response to consuming one can track the relationship between the consumed and produced events. The slots store allows a consumer or producer slot to be retrieved from its related producer or consumer slot respectively. If there is no related slot then the returned slot ID would be set to NO_SLOT.

Consumer and producer slot IDs are simply indices into the slots store that enable the retrieval of the state data for a particular link. This state data includes the current link status and any custom data object (Tag) that may be associated with it.

Slot advance modes

Both producers and consumers can operate in one of two slot advance modes:
  • Advance Single - move to the next overlay when a single link has been populated
  • Advance All - move to the next overlay when all links have been populated
These two modes are used to implement the shared and exclusive branch and merge operations. For example aggregation of multiple input events into a single event uses the Advance All mode and keeps the aggregated events for each link in a single overlay. Alternatively when routing multiple input events, the inputs are independent of each other and are assigned a separate overlay and therefore this operation uses the Advance Single mode. Similar logic applies to the branching operations.

Last edited Oct 7, 2012 at 11:04 AM by jaorme, version 10


No comments yet.