Reactive stream processing for data-centric publish/subscribe

TitleReactive stream processing for data-centric publish/subscribe
Publication TypeConference Paper
Year of Publication2015
AuthorsKhare, S., K. An, A. Gokhale, S. Tambe, and A. Meena
Refereed DesignationDoes Not Apply
Conference Name9th ACM International Conference on Distributed Event-Based Systems
Date Published06/2015
Conference LocationOslo, Norway
ISBN Number978-1-4503-3286-6

The Internet of Things (IoT) paradigm has given rise to a new class of applications wherein complex data analytics must be performed in real-time on large volumes of fast-moving and heterogeneous sensor-generated data. Such data streams are often unbounded and must be processed in a distributed and parallel manner to ensure timely processing and delivery to interested subscribers. Dataflow architectures based on event-based design have served well in such applications because events support asynchrony, loose coupling, and helps build resilient, responsive and scalable applications. However, a unified programming model for event processing and distribution that can naturally compose the processing stages in a dataflow while exploiting the inherent parallelism available in the environment and computation is still lacking. To that end, we investigate the benefits of blending Reactive Programming with data distribution frameworks for building distributed, reactive, and high-performance stream-processing applications. Specifically, we present insights from our study integrating and evaluating Microsoft .NET Reactive Extensions (Rx) with OMG Data Distribution Service (DDS), which is a standards-based publish/subscribe middleware suitable for demanding industrial IoT applications. Several key insights from both qualitative and quantitative evaluation of our approach are presented.