Login | Register

A Study on Entropy-Based Variational Learning for Mixture Models


A Study on Entropy-Based Variational Learning for Mixture Models

Ahmadzadeh, Mohammad Sadegh (2020) A Study on Entropy-Based Variational Learning for Mixture Models. Masters thesis, Concordia University.

[thumbnail of Ahmadzadeh_MASc_S2021.pdf]
Text (application/pdf)
Ahmadzadeh_MASc_S2021.pdf - Accepted Version
Available under License Spectrum Terms of Access.


Nowadays, we observe a rapid growth of complex data in all formats due to the technological development. Thanks to the field of machine learning, we can automatically analyze and infer useful information from these data. In particular, data clustering is regarded as one of the most famous data analysis tools aiming at grouping data with similar patterns into the same cluster. Among existing clustering techniques, finite mixture models have shown great flexibility in data modeling. Mixture models are a common unsupervised learning technique that have been widely used to statistically approximate and analyse heterogenous data. The goal of using mixture models is to fit the data into an appropriate distribution. A crucial point is to estimate the prefect parameters of the distribution and the suitable number of clusters in the data. To do so, an entropy-based variational learning algorithm is proposed for the model selection (i.e. determination of the optimal number of components). We investigate if a given component is genuinely distributed according to a mixture model to select the optimal number of components that better suits our data.
In our work, we have used the variational inference framework that overcomes the over-fitting problem of maximum likelihood approaches and at the same time convergence is guaranteed. In addition, it decreases the computational complexity of purely Bayesian approaches. In recent researches the main concern when deploying mixture models has been the choice of distributions. The effectiveness of Dirichlet family of distributions has been proved in recent studies especially for non-Gaussian data.
In this thesis, an effective mixture model-based approach for clustering and modeling purposes has been proposed. Our contribution is the application of an entropy-based variational inference algorithm to learn the mixture models, namely, generalized inverted Dirichlet and inverted Beta-Liouville mixture models. The performance of the proposed model is evaluated on multiple real-world applications such as human activity recognition, images, texture and breast cancer datasets, where in each case we compare our results with popular and similar models.

Divisions:Concordia University > Gina Cody School of Engineering and Computer Science > Electrical and Computer Engineering
Item Type:Thesis (Masters)
Authors:Ahmadzadeh, Mohammad Sadegh
Institution:Concordia University
Degree Name:M. Sc.
Program:Electrical and Computer Engineering
Date:17 December 2020
Thesis Supervisor(s):Bouguila, Nizar
Keywords:machine learning, pattern recognition, mixture models
ID Code:987757
Deposited By: Mohammad Sadegh Ahmadzadeh
Deposited On:23 Jun 2021 16:37
Last Modified:23 Jun 2021 16:37
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.

Repository Staff Only: item control page

Downloads per month over past year

Research related to the current document (at the CORE website)
- Research related to the current document (at the CORE website)
Back to top Back to top