Abstract

This thesis studies the application and impact of deep learning methods in anomaly detection, a critical area within security applications. While deep learning's popularity is driven by its perceived ability to manage complex patterns in large datasets and perform feature engineering inherently, this thesis questions these assumptions. By revisiting feature selection and data augmentation techniques, this research evaluates their effectiveness in improving the performance of deep-learning-based anomaly detection methods. Furthermore, it examines the impact of other essential factors such as model choice (both traditional machine learning and deep learning), data balancing, and hyperparameter tuning on anomaly detection performance.

From these investigations, the thesis reports that the common beliefs surrounding deep learning are not universally valid, highlighting the need for a framework to evaluate the usefulness of features and data for specific cases. To address this gap, a new framework is proposed, guiding data users and anomaly detection tools toward optimal configurations, including feature selection, model selection, hyperparameters, and data augmentation techniques. The effectiveness of this framework is demonstrated using two major IoT datasets, offering insights into improving anomaly detection systems through strategic and evidence-based approaches.