This study aimed to quantify the association between exposure to pandemic outbreaks and psychological health via a comprehensive meta-analysis. Literature retrieval, study selection, and data extraction were completed independently and in duplicate. Effect-size estimates were expressed as odds ratio (OR) with 95% confidence interval (CI). Data from 22 articles, involving 40,900 persons, were meta-analyzed. Overall analyses revealed a significant association of exposing to SARS-CoV-related pandemics with human mental health (OR, 1.32; 95% CI, 1.24-1.40; p < 0.001). Subgroup analyses showed that anxiety (OR, 1.37; 95% CI, 1.19-1.58; p < 0.001), depression (OR, 1.28; 95% CI, 1.15-1.42; p < 0.001), posttraumatic stress (OR, 1.36; 95% CI, 1.17-1.58; p < 0.001), and psychological distress (OR, 1.25; 95% CI, 1.11-1.40; p < 0.001) were all obviously related to pandemic diseases. In the context of infectious disease outbreaks, the mental health of general populations is clearly vulnerable. Therefore, all of us, especially health care workers, need special attention and psychological counseling to overcome pandemic together.