Machine Learning and Probabilistic Methods for Network Security Assessment

Abstract

Computer networks comprised of many hosts are vulnerable to cyber attacks. One attack can take the form of the exploitation of multiple vulnerabilities in the network along with lateral movement between hosts. In order to analyse the security of a network, it is common practice to run a vulnerability scan to report the presence of vulnerabilities in the network and prioritise them with an importance score. The scoring mechanism used primarily in the literature and in industry ignores how multiple vulnerabilities could be used in conjunction with one another to achieve a goal that previously was not possible. Attack graphs are a common solution to this problem, where a scan along with the topology of the network is turned into a graph that models how hosts and vulnerabilities can be connected. For a large network these attack graphs can be thousands of nodes in size, so in order to gain insight from them in an automated way, they can be turned into Bayesian attack graphs (BAGs) to model the security of the network probabilistically. The aim of this thesis is to work towards the automation of gathering insight from vulnerability scans of a network, primarily through the generation of BAGs. The main contributions of this thesis are as follows: 1. Creation of a unified formalism for the structure of BAGs and how other graphs can be translated into this formalism. 2. Classification of vulnerabilities using neural networks. 3. Design and evaluation of a novel technique for approximation in the computation of access probabilities in BAGs (referred to in the literature as the static analysis of BAGs) with no requirement for the base graph to be acyclic. 4. Implementation and comparison of three stochastic simulation techniques for inference on BAGs with evidence (referred to in the literature as the dynamic analysis of BAGs), enabling security measure evaluation and sensitivity analysis. 5. Demonstration of a sensitivity analysis for BAG priors and a novel method for quick computation of sensitivities that is more readily analysed than the traditional technique. 6. Development and demonstration of a fully containerised pipeline to automatically process vulnerability scans and generate the corresponding attack graph. With a single formalism for attack graphs, alongside an open-source attack graph generation pipeline, our work serves to enable future progress and collaboration in the field of processing vulnerability scans using attack graphs by simplifying the process of generating the graphs and having a mathematical basis for their evaluation. We design, implement, and evaluate various techniques for calculations on BAGs. For the process of computation of access probabilities we provide an algorithm that requires no processing or trimming of the initial graph, and for inference on BAGs we recommend likelihood weighting as the best performing sampling technique of the three we implement. We also show how inference techniques can be applied to sensitivity analysis on BAGs, and provide a new method that allows for more efficient and interpretable sensitivity analysis, enabling more productive research into the area in future. This research was originally undertaken in collaboration with XQ Cyber.