A unique and efficient Bayesian learning framework is proposed for the learning of asymmetric generalized Gaussian mixtures and hidden Markov models. This framework is based on Markov chain Monte Carlo (MCMC) sampling with hybrid Metropolis-Hastings within Gibbs sampling as the fundamental learning algorithm. The algorithm is integrated with the reversible jump MCMC (RJMCMC) technique to achieve a fully Bayesian learning framework for proposed models. A fully Bayesian learning framework allows self-adaptive learning where the two major challenges of mixture modelling, parameter estimation and model selection, are done automatically thereby making the learning process autonomous. Furthermore, feature selection is explored and incorporated in the learning process to enhance the capability of the models to weight and pick relevant features in multi-dimensional data. The proposed framework is tested through a wide range of applications: activity recognition, speaker recognition etc., and its performance is evaluated with multiple performance metrics to display the robustness of the approach.