Complex Event Processing (CEP)

What it is

The CEP GE analyses event data in real-time, generates immediate insight and enables instant response to changing conditions. While standard reactive applications are based on reactions to single events, the CEP GE reacts to situations rather than to single events. A situation is a condition that is based on a series of events that have occurred within a dynamic time window called processing context. Situations include composite events (e.g., sequence), counting operators on events (e.g., aggregation) and absence operators. The Proactive Technology Online is an implementation of the FIWARE CEP (Complex Event Processing) GE. The Proactive Technology Online is a scalable integrated platform to support the development, deployment, and maintenance of event-driven applications. The Proactive Technology Online authoring tool allows the definition of CEP applications using a web user interface. The Proactive Technology Online engine is a runtime tool that receives information on the occurrence of events from event producers, detects situations, and reports the detected situations to external consumers.

Why get it

The technology and implementations of CEP provide means to expressively and flexibly define and maintain the event processing logic of the application, and in runtime it is designed to meet all the functional and nonfunctional requirements without taking a toll on the application performance, removing one issue from the application developer’s and system managers concerns.

Avaliable for:

Requirements:

Architecture

The CEP architecture is composed from 4 main components:

  1. Build-time web based user interface
  2. Input Adapters
  3. Output Adapters
  4. Runtime Engine

Illustration of those components and their interactions are given in the figure below.

The application definitions, including the EPN, can be written by the application developer using CEP build-time web based user interface, by filling definition forms. The CEP build-time user interface generates a definition file which is sent to the CEP run-time. Alternatively this definition file, in JSON format, can be generated programmatically by any other application. At runtime, CEP receives incoming events through the input adapters or directly through its REST service. CEP processes those incoming events according to the application definitions and sends derived events through the output adapters.

3e
CEP High Level Architecture


CEP semantic layer allows the user to define producers and consumers for event data (see the figure above). Producers produce event data, and consumers consume the event data. The definitions of producers and consumer, which is specified during the application build time are translated into input and output adapters at CEP execution time. The physical entities representing the logical entities of producers and consumers in CEP are adapter instances.

3e2
Adapters layer representation


As can be seen in the above figure, an input adapter is defined for each producer, which defines how to pull the data from the source, how to format the data into CEP's object format before delivering it to the engine. The adapter is environment-agnostic but uses the environment-specific connector object, injected into the adapter during its creation, to connect to CEP runtime.

The consumers and their respective output adapters are operated in push mode – each time an event is published by the runtime it is pushed through environment-specific server connectors to the appropriate consumers, represented by their output adapters, which publish the event in the appropriate format to the designated resource.

The server connectors are environment-specific. They hide the implementation of the connectivity layer from the adapters, allowing them to be environment-agnostic.

Target Use

Complex Event Processing (CEP) is the analysis of event patterns in real-time to generate immediate insight and enable instant response to changing conditions. When the need is to respond to a specific event, the Context Broker GE is sufficient. You should consider using the CEP GE when there is a need to detect pattern over the incoming events occurring within some processing context (see the pattern examples in the previous section). Some functional requirements this technology addresses include event-based routing, observation, monitoring and event correlation. The technology and implementations of CEP provide means to expressively and flexibly define and maintain the event processing logic of the application, and in runtime it is designed to meet all the functional and non-functional requirements without taking a toll on the application performance, removing one issue from the application developer’s and system managers concerns.

For the primary user of the real-time processing generic enabler, namely the consumer of the information generated, the Complex Event Processing GE (CEP GE) addresses the user’s concerns of receiving the relevant events at the relevant time with the relevant data in a consumable format (relevant meaning that reacting or making use of the event is meaningful for the consumer/subscriber). The figure below depicts this role through a pseudo API derivedEvent(type,payload) by which, at the very least, an event object is received with the name of the event, derived out of the processing of other events, and its payload.

The designer of the event processing logic is responsible for creating event specifications and definitions (including where to receive them) from the data gathered by the Massive Data Gathering Generic Enabler. The designer should also be able to discover and understand existing event definitions. Therefore FIWARE, in providing an implementation of a Real-time CEP GE, will also provide the tools for the designer. In addition, APIs will be provided to allow generation of event specification and definitions programmatically, by an application or tools. In the figure below these roles are described as Designer and Programs making use of the pseudo API deploy definitions/instructions.

Finally, the CEP GE supports the roles of event system manager and operator, which could be played either by real people or management components. Actors playing these roles are responsible for managing configurations, monitor processing performance, handling problems, and monitoring the system’s health. They make use of the pseudo API configuration/tuning/monitoring for this purpose.

3e3
Interactions with and APIs of the Real-time CEP Generic Enabler

FIWARE Webpage

Complex Event Processing (CEP) - Proactive Technology Online

Documentation

Complex Event Processing (CEP) - Proactive Technology Online Documentation

Download

Complex Event Processing (CEP) - Proactive Technology Online Download

Fiware Academy

Complex Event Processing (CEP)Courses

Screenshots

Click on the images to enlarge them.