Please use this identifier to cite or link to this item:
Title: Run-time adaptation of a functional stream processing system
Authors: Cattermole, Adam Douglas Derwent
Issue Date: 2022
Publisher: Newcastle University
Abstract: Extracting value from streams of events generated by sensors and software has become key to the success of many important classes of applications, whether this be sensors for smart cities/buildings, or wearable healthcare devices. However, writing streaming data applications is not easy – developers are confronted with major challenges, including processing events arriving at varying rates from thousands to millions of events per second, distributing processing over a set of heterogeneous platforms including edge devices and cloud servers, and meeting non-functional requirements such as energy, networking, security and performance. The data within these applications can be largely dynamic, and requires the streaming system to adapt to the ever-changing demands. This thesis focuses on one challenge in distributed stream processing: automatically adapting the partitioning of the processing between the edge and the cloud without a loss of service. An example is when the event arrival rate increases and the edge processor can no longer meet performance requirements. Re-partitioning without loss of service involves moving computations between the edge and the cloud while events are still being processed. In this thesis the StrIoT system is introduced – a stream processing system that supports automatic re-partitioning of a streaming application. It is based on a set of functional stream operators, and the thesis describes how the run-time system can automatically adapt applications that use them. Results are presented from the evaluation of StrIoT on a real-world dataset of taxi journey information, using both cloud servers and an edge device, showing that performance can be improved with only a low, temporary impact during adaptation.
Description: PhD Thesis
Appears in Collections:School of Computing

Files in This Item:
File Description SizeFormat 
Cattermole A D D 2022.pdf17.51 MBAdobe PDFView/Open
dspacelicence.pdf43.82 kBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.