This study aimed to investigate determinants other than drug treatment for poor glycemic control in a large cross-sectional survey of Chinese men with type 2 diabetes (T2D). From March 2011 to June 2011, the Chinese Diabetes Association conducted a survey of patients with T2D who used oral antidiabetes drugs (OADs) in 77 hospitals located in four well-developed cities in China, as part of China's nationwide surveillance of the rate of poor glycemic control. A total of 29,502 patients were consecutively recruited from the participating tertiary hospitals, and 15,427 male subjects were used in the analysis. Poor glycemic control was defined by glycated hemoglobin < 7 %. Multivariate conditional logistic regression was used to identify the determinants of poor glycemic control. Up to 68.5 % (n = 10,600) of the surveyed men with T2D had poor glycemic control. In multivariate analysis, age below 40 years (OR 1.36, 95 % CI 1.19 to 1.65) and overweight/obesity were associated with poor glycemic control. The duration of diabetes was associated with poor glycemic control in a J-shaped manner, whereby being < 1 year as well as a parts per thousand yen6 years were associated with increased risk of poor glycemic control. On the other hand, self-monitoring of blood glucose was associated with lower risk of poor glycemic control (OR 0.84, 95 % CI 0.77 to 0.92). Young men aged below 40 years, overweight/obese, newly diagnosed T2D, and 6 years duration of diabetes were associated with increased risk of poor glycemic control.