Distributed graph processing: An approach based on overlay composition

In this day and age, human being is generating an unprecedented amount of data coming from multiple and heterogeneous sources, ranging from scientific devices to business transactions. In many contexts such data is modelled as a graph, however, due to its size, it is often infeasible to process it on a single machine. As a consequence, the adoption of parallel and distributed computing frameworks is the natural approach to face this problem. This paper proposes an high-level approach for large graphs processing. Our approach takes inspiration from overlay networks, a widely exploited approach for information dissemination, aggregation and computing orchestration in highly distributed systems. We propose Telos, a programming framework supporting the definition of computations on graphs as a composition of overlays, each devoted to a specific aim. Telos is implemented on the top of Apache Spark. A set of experimental results are presented to give a preliminary evidence of the effectiveness of our approach.