Loss of brain tissues and cognitive abilities are natural processes of aging, and they are related to each other. These changes in cognition and brain structure are different among the cognitively normal elderly and those with Alzheimer's disease (AD). Despite the great development in the longitudinal study of decline in brain volume and cognitive abilities, previous studies are limited by their small number of data collection waves and inadequate adjustments for important factors (such as a genetic factor). These limitations diminish the power to detect changes in brain tissues and cognitive abilities over a longer period of time. In this study, firstly, we aimed to explore the longitudinal association between cognitive abilities and global and regional structural brain variables among individuals with normal cognitive status, mild cognitive impairment (MCI), and AD using mixed effects models. Secondly, we investigated the effect of education on the relationship between cognition and brain structure. Lastly, we utilized latent class growth analysis in order to study the change in cognition between different MCI sub-classes based on their functional abilities. The data in this study were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI)which contained 6 time points over three years (n=686). The results showed that cognitive abilities decreased over time across different groups, and the rate of decline in cognition depended on the whole brain volume. Importantly, the effect of brain volume on the rate of decline in cognitive abilities was greater among MCI subjects who progressed to AD (pMCI) and participants with AD. Ventricle enlargement in the pMCI group also showed a significant influence on the rate of cognitive decline .Lastly, based on an assessment of functional abilities at baseline, this study demonstrated an efficient methodology to identify MCI subjects who are most at-risk for cognitive impairment progression.