This case-control study is aimed to investigate the correlation of altered functional connectivity (FC) in cerebellum with cognitive impairment in amnestic mild cognitive impairment (aMCI) and Alzheimer's disease (AD). The morphometric and resting-state FC MRI analysis including 46 participants with AD, 32 with aMCI and 42 age-matched normal controls (NCs) were conducted. We compared the cerebellar gray matter volume and cerebellar FC with cerebral cortical regions among three groups. To investigate the relationship of cerebellar FC with cognition, we measure the correlation of significant altered FC and individual cognitive domain. No significant morphometric differences of cerebellum was observed across three groups. The patients with AD had weaker cerebral cortical FCs in bilateral Crus I and left VIIb compared to NCs, and in bilateral Crus I compared to patients with aMCI. For patients with aMCI, the weaker FC were found between right Crus I, left VIIb and cerebral cortical regions compared to NCs. The strength of left cerebellar FC positively correlated with specific cognitive subdomains, including memory, executive function, visuospatial function, and global cognition in AD and aMCI. These findings demonstrated the alteration of cerebellar FC with cerebral cortical regions, and the correlation of cerebellar FC and cognitive impairment in AD and aMCI.