Bayesian online state and parameter estimation for streaming data

Abstract

With the advent of Big Data and the Internet of Things, data streams are ubiquitous, increasing the demand for real-time inference on sequential data at low computational cost. Inference for streaming time-series is tightly coupled with the problem of Bayesian online state and parameter inference. In this thesis we will focus mainly on Dynamic Generalised Linear Models, the class of models often chosen to model continuous and discrete time-series data. We will look at methods which solve the problem of estimating jointly states and parameters, both in online and offline scenarios. For the online scenario, when the parameters are known, we will look at the Kalman Filter and Sequential Monte Carlo methods (SMC) which provide estimations for the hidden latent states. We will then consider SMC extensions allowing for online joint state and parameter estimation. Offline methods, by definition, do not allow real-time estimation but typically provide superior results at higher time and computational costs. In this thesis we propose and evaluate a fully online, approximated version of a sequential, but not-online method (SMC2 ). This method approximates the true posterior, performing estimation over a sliding window of the most recent observations and so bounding the computational cost and operating in an online fashion, providing an acceptable approximation and, by employing particle rejuvenation through an MCMC move, delaying particle impoverishment problems. This thesis analyses online methods when applied to different real world datasets showing that SMC sufficient statistics-based-methods delay known problems, such as particle impoverishment, especially when applied to long running time-series, while providing reasonable estimations when compared to exact methods, such as Particle Marginal MetropolisHastings. State and observation forecasts will also be analysed as a performance metric. By benchmarking against a “gold standard” (offline) method, we can better understand the performance of online methods in challenging real-world scenarios.