Application Event Broker

The application event broker provides a bridge between the workflow engine and external asynchronous event sources. Asynchronous events may be generated by activity in external IT systems or they can be associated with calendar events or elapsed time intervals. The event broker enables a workflow to respond to such external events in one of two ways:
  1. by creating a new instance of a workflow process and associating it with the event
  2. by activating a transition in an existing workflow instance
In both cases the semantics of 'listening for an external event' are expressed using XPDL ExtendedAttributes and in both cases information can be extracted from the actual event into the listening process instance.

Application Event

The runtime entity which embodies the external event is known as an application event. Application events are categorized by their event type, which is generally descriptive of some recurrent business-relevant incident (e.g., OrderReceived, InvoicePaid, ShipmentDelivered, FaultLogged, NewCustomer, RequestForProposal, NewEmployee etc.). An application event carries with it application relevant event data (for example the actual purchase order, fault log details, employee details, etc.). A specific instance of a particular event type is uniquely identified by a set of event keys associated with the event data (such as PurchaseOrderNumber, InvoiceNumber, TicketNumber, EmployeeNumber, and so on).

Determining the Event Type

OBE-aware event sources can construct and publish OBE application events themselves. Conversely, non-OBE-aware event sources can simply publish the raw event data in whatever format suits them (Java event objects, XML documents, email messages, COBOL copybook records, text documents, etc.), in which case the OBE event broker assumes responsibility for pre-processing the raw event data into a usable application event object. Every event type to which the workflow engine is capable of responding is registered in the application event broker with event type meta-data. These meta-data not only describe what information a subscriber must supply in order to subscribe to a given event type, they can also describe to the event broker how to recognize an occurrence of the given event type by introspecting raw inbound event data, and how to compute the event keys from the raw event data. This ability to recognize application events reflectively is a powerful feature because it enables the workflow engine to integrate with target systems without imposing any OBE-specific requirements upon them. The event broker recognizes event types by the content type and schema of their event data.

MIME Content Type

The standard event broker implementations which ship with OBE implement their event classification scheme in terms of the standard MIME content types described in the Multi-purpose Internet Mail Extensions specification (MIME content types are also used by the Javabeans Activation Framework, which provides a framework for generic manipulation of arbitrary content). Some event sources (such as email or HTTP) already describe their payload in terms of its MIME content type. For others it is necessary for OBE to introspect the content to determine its type; it does this by examining the Java class of the raw event data.

Content Handlers

The Class2MimeType.properties configuration file maps Java class names to MIME content types (either as a content type or as a type/subtype combination). The MimeHandlers.properties configuration file registers a handler for a specific content type[/subtype] and also contains content type-specific hints that the handler can use to help it determine the MIME subtype. A content handler has another role: it transforms the raw event data into one or more application event objects. Note that through this transformation process a single inbound event message can produce multiple application events of potentially different types. This technique is used by default to 'burst apart' bundled objects such as multipart email messages. If you wish to do similar things to other content types you can achieve this by registering your own content handlers.

Schema

The concept of schema is a means of further refining event classification beyond the MIME content type/subtype. The event broker derives the content type and uses this to retrieve a suitable content handler, which then derives the schema name from the raw event data using hints encoded in the  MimeHandlers.properties file mentioned previously. The hints vary according to the content type - for example, for text/* content types the hints take the form of regular expressions which can be used to differentiate one text format from another according to the occurrence of certain keywords or patterns of tokens. Other content types use different schemes.

Event Persistence

It can happen that a workflow subscribes to a event after the event in question has already been received by the event broker. This can happen as a result of a race condition in a non-transactional request-response scenario, and typically results in the event being lost and the workflow getting 'stuck' waiting for an event that had already arrived and been discarded unprocessed because no subscription could be found. The OBE event brokers have a solution to this dilemma: it is possible to define a time to live for an event type, in which case instances of that event type will be stored for a specified period of time before being discarded. This allows them to be matched against subsequently received subscriptions.

Event Type Metadata

Event type metadata define an event type in terms of content type, schema, a qualifying condition, the scripting language used to evaluate expressions against event data, the name of a business calendar to use for temporal computations, and a set of event parameter definitions. The event parameters are an extension of the XPDL formal parameter concept and specify in addition a key expression that is evaluated to compute a key value for its respective parameter from an inbound event. These key values are used to match the event occurrence to corresponding subscriptions. By specifying OUT-type event parameters it is possible to enable subscribers to extract event key values into named process instance attributes.

The two application event broker implementations which come with OBE read and store these data in the configuration file BasicApplicationEventBroker.xml; entries have this format:
<event-type id="nmtoken" type="nmtoken"? threadsafe="true|false"?>
    <display-name .../>?
    <author .../>?
    <description .../>?
    <doc-url .../>?
    <content-type .../>?
    <schema .../>?
    <script-type .../>?
    <!-- This is a standard XPDL Condition element. -->
    <condition type="CONDITION|OTHERWISE|EXCEPTION|DEFAULTEXCEPTION" .../>?
    <!-- This is an extended XPDL FormalParameters element. -->
    <event-parameters>?
        <event-parameter id="nmtoken" mode="IN|INOUT|OUT"?>+
            <data-type>
                <!-- One of the following types. -->
                <basic-type type="STRING|FLOAT|INTEGER|REFERENCE|DATETIME|BOOLEAN|PERFORMER"/>?
                <declared-type type="refid"/>?
                <schema-type>xsd:any</schema-type>?
                <external-reference location="document-uri" xref="fragment" namespace="namespace-uri" />?
            </data-type>
            <description .../>?
            <key-expression .../>
        </event-parameter>
    </event-parameters>
    <time-to-live .../>?
    <calendar .../>?
</event-type>
content-type
Content type, must be a valid IANA-registered MIME media type, or a valid extension type.
schema
Arbitrary sub-classification.
script-type
The ID of the scripting language to use when evaluating key expressions.
condition
A qualifying condition, which must evaluate to true in order to produce a match on this event type.
event-parameter
An extension to the XPDL FormalParameter construct.
id
The parameter ID.
mode
The parameter mode: IN, INOUT or OUT. N.B. IN- and INOUT-type parameters must be listed before OUT-type parameters.
type
The data type of the parameter. One of: <basic-type>, <declared-type>, <schema-type>, <external-reference>.
key-expression
The expression which must be evaluated to compute the event key value for this parameter.
time-to-live
An XPDL Duration-style expression of form "<integer>[s|m|h|D|M|Y]".
calendar
The ID of the business calendar to use when evaluating time-to-live.

Publishing Event Data

Many IT systems can be configured or adapted to broadcast business event data to messaging middleware and at the lowest level this can be achieved by something as simple as a database trigger. All that is required is a means of injecting relevant business data into OBE's event broker. OBE provides both an API and a JMS subscriber in the form of a message driven EJB that listens to a predefined JMS queue. Other implementations are equally possible (polling a file system or POP3/IMAP mail store, or a Web Service interface, for example).

Event Subscription

The runtime entity which embodies the listening state is known as an event subscription. An event subscription associates an anticipated event occurrence with an event subscriber. The potential event occurrence is characterized as an event type plus its requisite set of event key values and/or a qualifying condition; the event subscriber is characterized as a set of correlation keys which uniquely identify the listening workflow process definition or transition (corresponding to cases 1 and 2 above, respectively). The application event broker recognizes two types of subscription: an application event subscription and a temporal event subscription.

Effective & Expiry Dates

A subscription can specify an effective date - the moment in time when it will become active and an expiry date - the moment in time when it ceases to be active.

Subscription Recurrence

An event subscription can be designated as recurrent, in which case it can potentially be triggered multiple times. A subscription can specify a finite or infinite number of recurrences. In either case a recurrent subscription expires when its expiry date, if any, is reached. Subscriptions from triggered workflows are typically recurrent whereas those for triggered transitions are not.

Application Event Subscription

This is a subscription to an asynchronous event representing activity in some external application. The subscription is specified by attaching an XPDL ExtendedAttribute named 'obe.Event' to a WorkflowProcess or Transition element.

Temporal Event Subscription

This is a subscription to a timed event that occurs after some specified interval of time as computed from the moment the subscription is created. The system uses it to implement activity deadlines and time-triggered workflows. A time-triggered workflow is implemented by attaching an XPDL ExtendedAttribute named 'obe.Timer' to a WorkflowProcess. A temporal subscription is implied by the use of the standard XPDL 'Deadline' element - these temporal event types are by definition recoverable. Note that temporal event subscriptions are not absolutely guaranteed to occur in strict chronological sequence, especially if the events are recoverable or closely spaced in time.

Recoverable Temporal Subscriptions

A temporal event subscription can specify that it is recoverable. Recoverable events are never lost, even if the system was down at the time they were scheduled to occur.  The system processes any missed recoverable events when it resumes normal operation.

Non-Recoverable Temporal Subscriptions

A non-recoverable subscription is not guaranteed to be processed if it was missed. The configuration property obe.events.max.tardiness determines the maximum age for a temporal subscription to be considered eligible for processing.

XPDL Extensions

An application event subscription is specified by an XPDL ExtendedAttribute with the name 'obe.Event', containing a nested <obe:Event> element. Application event subscriptions can be specified on a WorkflowProcess or a Transition. A temporal event subscription is specified by an XPDL ExtendedAttribute with the name 'obe.Timer', containing a nested <obe:Timer> element. Temporal event subscriptions can be specified only on a WorkflowProcess; time-triggered Transitions are associated with an <xpdl:Deadline> element at the Activity level, and the event subscription is created dynamically by the workflow engine.

The syntax is:
<xpdl:ExtendedAttribute Name="obe.Event">
    <obe:Event Id="nmtoken" Count="integer"?>
        <xpdl:ActualParameters>?
            <xpdl:ActualParameter ... />*
        </xpdl:ActualParameters>
    </obe:Event>
</xpdl:ExtendedAttribute>

<xpdl:ExtendedAttribute Name="obe.Timer">
    <obe:Timer Id="nmtoken" Count="integer"? Interval="string" Calendar="string" Recoverable="true|false"?>
        <xpdl:ActualParameters>?
            <xpdl:ActualParameter ... />*
        </xpdl:ActualParameters>
    </obe:Timer>
</xpdl:ExtendedAttribute>
Id
The event type ID, used as a lookup key by the factory.
Count
The maximum number of times the event subscription can be fulfilled before it expires. Default is -1 (infinite).
ActualParameters
Standard XPDL actual parameter list, to match the formal parameters declared the event type definition.
ActualParameter
Standard XPDL actual parameter semantics for IN-, INOUT- and OUT-type parameters respectively.
Interval
A standard XPDL Duration-style time interval. 
Calendar
The ID of the business calendar to use when applying the Interval.
Recoverable
Whether the temporal event is recoverable if missed during system down-time.
For example:

Let's say the following event types are declared in BasicApplicationEventBroker.xml:
<entry>
    <event-type id="OrderReceived" type="OBEEvent">
        <content-type>text/xml</content-type>
        <schema>http://www.xyz.com/ebiz/2005/order</schema>
        <condition type="CONDITION">/event/data/status = 'RAISE'</condition>
        <script-type>text/x-xpath</script-type>
        <display-name>Sales Order Received</display-name>
        <description>
            An XML sales order is received, via an inbound
            Web Service call or email message, for example.
        </description>
        <event-parameters>
            <event-parameter id="customerId" mode="OUT">
                <data-type>
                    <basic-type type="STRING"/>
                </data-type>
                <key-expression>string(/event/data/customer-id)</key-expression>
            </event-parameter>
            <event-parameter id="orderNumber" mode="OUT">
                <data-type>
                    <basic-type type="STRING"/>
                </data-type>
                <key-expression>string(/event/data/order-number)</key-expression>
            </event-parameter>
            <event-parameter id="order" mode="OUT">
                <data-type>
                    <external-reference location="order.xsd"/>
                </data-type>
                <key-expression>/event/data</key-expression>
            </event-parameter>
        </event-parameters>
    </event-type>
</entry>
<entry>
    <event-type id="OrderCancelled" type="OBEEvent">
        <content-type>text/xml</content-type>
        <schema>http://www.xyz.com/ebiz/2005/order</schema>
        <condition type="CONDITION">/event/data/status = 'CANCEL'</condition>
        <script-type>text/x-xpath</script-type>
        <display-name>Sales Order Received</display-name>
        <description>
            An XML order cancellation is received, via an inbound
            Web Service call or email message, for example.
        </description>
        <event-parameters>
            <event-parameter id="customerId" mode="IN">
                <data-type>
                    <basic-type type="STRING"/>
                </data-type>
                <key-expression>string(/event/data/customer-id)</key-expression>
            </event-parameter>
            <event-parameter id="orderNumber" mode="IN">
                <data-type>
                    <basic-type type="STRING"/>
                </data-type>
                <key-expression>string(/event/data/order-number)</key-expression>
            </event-parameter>
        </event-parameters>
    </event-type>
</entry>
<entry>
    <event-type id="timeout" type="OBEEvent">
        <content-type>application/x-java-object</content-type>
        <schema>java.util.Date</schema>
        <display-name>Timer Event</display-name>
        <description>A Timed Event</description>
        <event-parameters>
            <event-parameter id="datetime" mode="OUT">
                <data-type>
                    <basic-type type="DATETIME"/>
                </data-type>
                <key-expression>/event/data</key-expression>
            </event-parameter>
        </event-parameters>
    </event-type>
</entry>
The referenced order schema order.xsd is registered in BasicResourceRepository.xml thus:
<entry>
    <resource id="order.xsd" type="xsd">
        <display-name>Order Schema</display-name>
        <description>Defines the structure of an order</description>
        <systemId>http://www.xyz.com/ebiz/2005/order</systemId>
    </resource>
    <instance xsi:type="java:java.lang.String">order.xsd</instance>
</entry>
Here are some examples of ways in which matching event subscriptions could be created:
<!-- This workflow process is triggered whenever a customer order is received. -->
<WorkflowProcess Id="sales-order" Name="Sales Order Processing">
    ...
    <DataFields>
        <DataField Id="customerId">
            <DataType>
                <BasicType Type="STRING" />
            </DataType>
        </DataField>
        <DataField Id="orderNumber">
            <DataType>
                <BasicType Type="STRING" />
            </DataType>
        </DataField>
        <DataField Id="orderDocument">
            <DataType>
                <ExternalReference location="order.xsd" />
            </DataType>
        </DataField>
    </DataFields>
    ...
    <ExtendedAttributes>
        <ExtendedAttribute Name="obe.Event">
            <obe:Event Id="OrderReceived" Count="-1">
                <xpdl:ActualParameters>
                    <xpdl:ActualParameter>customerId</xpdl:ActualParameter>
                    <xpdl:ActualParameter>orderNumber</xpdl:ActualParameter>
                    <xpdl:ActualParameter>orderDocument</xpdl:ActualParameter>
                </xpdl:ActualParameters>
            </obe:Event>
        </ExtendedAttribute>
    </ExtendedAttributes>
    ...
    <Transitions>
        <Transition Id="cancelled" From="fulfil" To="notify-cancelled">
            <Description>
                This transition is triggered by the asynchronous receipt of an
                order cancellation message from a customer. The subscription is
                automatically cancelled by the completion of the 'fulfill'
                activity.
            </Description>
            <ExtendedAttribute Name="obe.Event">
                <obe:Event Id="OrderCancelled" Count="1">
                    <xpdl:ActualParameters>
                        <xpdl:ActualParameter>$customerId</xpdl:ActualParameter>
                        <xpdl:ActualParameter>$orderNumber</xpdl:ActualParameter>
                    </xpdl:ActualParameters>
                </obe:Event>
            </ExtendedAttribute>
        </Transition>
    <Transitions>
    ...
</WorkflowProcess>

<!--
    This workflow process is triggered at the end of each month, occurring twelve times between
    Midnight 31st January, 2006 UTC and Midnight 3rd January, 2007 (3 spare days, to allow recovery).
-->
<WorkflowProcess Id="month-end" Name="2006 Month End Processing">
    <ProcessHeader>
        <ValidFrom>2006-01-31T00:00:00+0000<ValidFrom>
        <ValidTo>2007-01-03T00:00:00+0000<ValidTo>
    </ProcessHeader>
    ...
    <DataFields>
        <DataField Id="datetime">
            <DataType>
                <BasicType Type="DATETIME" />
            </DataType>
        </DataField>
    </DataFields>
    ...
    <ExtendedAttributes>
        <ExtendedAttribute Name="obe.Timer">
            <obe:Timer Id="timeout" Count="12" Interval="1M" Calendar="SALES" Recoverable="true">
                <xpdl:ActualParameters>
                    <xpdl:ActualParameter>datetime</xpdl:ActualParameter>
                </xpdl:ActualParameters>
            </obe:Timer>
        </ExtendedAttribute>
    </ExtendedAttributes>
    ...
</WorkflowProcess>