Immune checkpoint inhibitors (ICIs), represented by anti-CTLA-4 or anti-PD-1/anti-PD-L1 pathway antibodies, have led to a revolution in cancer treatment modalities. ICIs have unique clinical benefits, such as effectiveness against a broad range of tumor types, strong overall impact on survival, and persistent responses after the cessation of therapy. However, only a subset of patients responds to these therapies, and a small proportion of patients even experience rapid progression or an increased risk of death. Therefore, it is imperative to optimize patient selection for treatment. This review focuses on the mechanisms of tumor escape from immune surveillance, the composition and activity of a preexisting immune infiltrate, the degree of tumor foreignness (as reflected by the mutational burden, expression of viral genes, and driver gene mutations), and host factors (including peripheral blood biomarkers, genetic polymorphisms, and gut microbiome) to summarize current evidence on the biomarkers of responses to ICIs and explore the future prospects in this field.