Designing monitoring systems for complex event processing in Big Data contexts

Abstract

Nowadays, the amount of data that is constantly being generated presents new challenges for the technical and scientific community, such as the challenge of ensuring Complex Event Processing (CEP) in Big Data contexts, which arises to meet current advanced analytical needs. Therefore, some works are dedicated to the design and implementation of integrated CEP systems in the context of Big Data, as it is an example the Intelligent Event Broker (IEB) on which this work is based on. The IEB is a collection of several components that are integrated and validated to create a homogeneous system that will process events in real time in Big Data contexts, focusing on a rule-based approach. Considering the complexity of the IEB in constantly running contexts, it is important to have the ability of monitoring the evolution of the system, to avoid its uncontrolled growth. To accomplish that, we have previously proposed a component named “Mapping and Drill-down System” for the IEB, composed of a Web visualization Platform and a graph database. The main goal of the work presented in this paper is propose an architecture for the Mapping and Drill-down System component to monitor, in real time, the IEB’s execution data, by collecting, processing, and efficiently storing it in a graph database for later visualization through the Web Visualization Platform. The graph database and the Web Visualization platform are the key components of the Mapping and Drill-down System. With this work, it will be easier to understand the behavior of the IEB in constantly running contexts, ensuring its controlled growth and helping the community in the design and development of CEP systems for Big Data contexts, especially in the monitoring component of such complex systems.