Research Area:  Edge Computing

Abstract:

   In this thesis, our goal is to enable and achieve effective and efficient real-time stream processing in a Geo-distributed infrastructure, by combining the power of central data centers and micro data centers. Our research focus is to address the challenges of distributing the stream processing applications and placing them closer to data sources and sinks. We enable applications to run in a Geo-distributed setting and provide solutions for the network-aware placement of distributed stream processing applications across Geo-distributed infrastructures. First, we evaluate Apache Storm, a widely used open-source distributed stream processing system, in the community network Cloud, as an example of a Geo-distributed infrastructure. Our evaluation exposes new requirements for stream processing systems to function in a Geo-distributed infrastructure. Second, we propose a solution to facilitate the optimal placement of the stream processing components on Geo-distributed infrastructures.
    We present a novel method for partitioning a Geo-distributed infrastructure into a set of computing clusters, each called a micro data center. According to our results, we can increase the minimum available bandwidth in the network and likewise, reduce the average latency to less than 50%. Next, we propose a parallel and distributed graph partitioned, called HoVerCut, for fast partitioning of streaming graphs. Since a lot of data can be presented in the form of graph, graph partitioning can be used to assign the graph elements to different data centers to provide data locality for efficient processing. Last, we provide an approach, called Span Edge that enables stream processing systems to work on a Geo-distributed infrastructure. Spen Edge unifies stream processing over the central and near-the-edge data centers (micro data centers). As a proof of concept, we implement Span Edge by extending Apache Storm that enables it to run across multiple data centers.

Name of the Researcher:  Hooman Peiro Sajjao

Name of the Supervisor(s):  Vladimir Vlassov

Year of Completion:  2016

University:  KTH Royal Institute of Technology

Thesis Link:   Home Page Url