Abstract

Nowadays, we observe a rapid growth of complex data in all formats due to the technological development. Thanks to the field of machine learning, we can automatically analyze and infer useful information from these data. In particular, data clustering is regarded as one of the most famous data analysis tools aiming at grouping data with similar patterns into the same cluster. Among existing clustering techniques, finite mixture models have shown great flexibility in data modeling. Mixture models are a common unsupervised learning technique that have been widely used to statistically approximate and analyse heterogenous data. The goal of using mixture models is to fit the data into an appropriate distribution. A crucial point is to estimate the prefect parameters of the distribution and the suitable number of clusters in the data. To do so, an entropy-based variational learning algorithm is proposed for the model selection (i.e. determination of the optimal number of components). We investigate if a given component is genuinely distributed according to a mixture model to select the optimal number of components that better suits our data.
In our work, we have used the variational inference framework that overcomes the over-fitting problem of maximum likelihood approaches and at the same time convergence is guaranteed. In addition, it decreases the computational complexity of purely Bayesian approaches. In recent researches the main concern when deploying mixture models has been the choice of distributions. The effectiveness of Dirichlet family of distributions has been proved in recent studies especially for non-Gaussian data.
In this thesis, an effective mixture model-based approach for clustering and modeling purposes has been proposed. Our contribution is the application of an entropy-based variational inference algorithm to learn the mixture models, namely, generalized inverted Dirichlet and inverted Beta-Liouville mixture models. The performance of the proposed model is evaluated on multiple real-world applications such as human activity recognition, images, texture and breast cancer datasets, where in each case we compare our results with popular and similar models.