Abstract

With the increase in networks capacities and number of online users, threats of different cyber attacks on computer networks also increased significantly, causing the loss of a vast amount of money every year to various organizations. This requires the need to identify and group these threats according to different attack types. Many anomaly detection systems have been introduced over the years based on different machine learning algorithms. More precisely, unsupervised learning algorithms have proven to be very effective. In many research studies, to build an effective ADS system, finite mixture models have been widely accepted as an essential clustering method.
In this thesis, we deploy different non-Gaussian mixture models that have been proven to model well bounded and semi-bounded data. These models are based on the Dirichlet family of distributions. The deployed models are tested with Geometric Area Analysis Technique (GAA) and with an adversarial learning framework.
Moreover, we build an effective hybrid anomaly detection system with finite and in-finite mixture models. In addition, we propose a feature selection approach based on the highest vote obtained. We evaluated the performance of mixture models with Geometric Area Analysis technique based on Trapezoidal Area Estimation (TAE) and the effect of adversarial learning on ADS performance via extensive experiments based on well-known data sets.