Background Age is closely related to the efficacy of treatment for non-small cell lung cancer (NSCLC) patients. Latest clinical trials have proved the better overall survival (OS) for the use of immune checkpoint inhibitors verse chemotherapy in NSCLC patients. However, we had no clear idea of the efficacy of them in elderly patients. So we conducted a meta-analysis to compare the efficacy of immune checkpoint inhibitors for NSCLC patients of different age groups and summarized overall treatment-related adverse events. Materials and methods PubMed, EMBASE, Web of Science and the Cochrane Library were searched for all clinical trials in NSCLC until 30th of April 2019. Eligible studies included randomized controlled trials (RCTs) comparing immune checkpoint inhibitors with chemotherapy in NSCLC patients. The hazard ratio (HRs) and 95% confidence intervals (CIs) of OS, progression-free survival or adverse events (AEs) were used. Results A total of 4994 patients from 8 RCTs were included. Immune checkpoint inhibitors significantly prolonged the OS (HR, 0.73; 95% CI, 0.61-0.89) versus chemotherapy in NSCLC patients who were less than 65 years old. Also, they prolonged the OS (HR, 0.74; 95% CI, 0.59-0.93) in NSCLC patients who were more than 65 years old. However, there was no statistical significance of OS (HR, 0.87; 95% CI, 0.57-1.30) among NSCLC patients who were more than 75 years old. It also showed that the single use of immune checkpoint inhibitors had fewer all-grade AEs. Conclusion Regardless of the NSCLC patients who were less or more than 65 years, immune checkpoint inhibitors could achieve better OS than chemotherapy. But there was no significant difference when NSCLC patients who were more than 75 years old. Older patient should be offered immune therapies if it is possible and the mechanism in old age treatment should be further studied.