This thesis targets the problem of poor performance of HMM-based classifiers. First, we study the effect of the structure on the performance of HMMs and see how the number of states and the topology can contribute to the classification performance. As a result, our investigation showed the topology has a stronger contribution to the classification performance than the number of states. Second, we propose a general two-stage framework that combines generative and discriminative models to reach a high performance in the classification of time-series data. In the first stage, HMMs are used to model the time-series data, then a fixed size score vector is extracted from this stage and used as the input to the discriminative model in the second stage. The framework showed a potential for combining generative and discriminative models for in time-series data classification and was able to achieve a recognition rate of 98.02%, with an increase of 3.83% over traditional HMM-based classifiers. (Abstract shortened by UMI.)