| dc.description.abstract | Advances in Internet of Things (IoT) have enabled a variety of latency-sensitive applications that reside at the edge of the network, for example, smart-grids, intelligent transportation, etc. These applications often involve a large number of sensors that generate volumes of data, which must be processed continuously in near real-time to gain insights, which are subsequently distributed to interested end-points for actuation. Thus, an IoT application can be visualized as a distributed dataflow graph where both data distribution and continuous/streaming data processing are two fundamental requirements. Although the Publish/Subscribe (pub/sub) communication pattern effectively meets the scalable data distribution needs of IoT applications, it lacks an intuitive programming model to also support continuous data processing which preserves the end-to-end data-flow structure. Additionally, to meet the response time requirements of latency-sensitive IoT applications, both data distribution and processing must be performed in a latency-aware manner. Although edge computing supports low latency requirement by virtue of executing the computations near the source of data and thereby improving application response times, the computational overhead of running multiple processing tasks on a resource-constrained edge device can quickly eclipse the network latency benefits of using the edge. To accrue the benefits of edge computing, this trade-off between computation overhead and network cost must be accounted for. Developing an edge-based, latency-aware data distribution and processing solution for IoT applications, which expressly takes this trade-off into account is the objective of this dissertation research. To that end, this dissertation makes the following contributions. First, it presents a unified programming model for programming IoT applications which supports both pub/sub data distribution and composable data stream processing to preserve the end-to-end data-flow application semantics. Second, it presents a novel data load balancing algorithm for publish/process/subscribe systems to meet the low latency data distribution requirements and scale to larger data volumes. Third, it presents a novel placement solution for distributing stream processing workloads on edge devices to minimize the end-to-end latency of processing. | |
