Objective We aimed to investigate the effect of the triglyceride glucose (TyG) index on the association between diabetes and cardiovascular disease (CVD). Methods Data from 6,114 individuals were extracted and analyzed from the China Health and Retirement Longitudinal Survey (CHARLS) from 2011 to 2018. Logistic regression analyses were conducted to assess the relationship between diabetes and CVD across the various TyG index groups. The statistical method of subgroup analysis was used to determine the correlation between diabetes and CVD for each TyG index group by sex, history of hypertension and dyslipidemia, smoking, and drinking. Results Diabetes was positively associated with CVD risk after adjustment in 2011(odds ratio (OR) 1.475, 95% CI 1.243-1.752, P < 0.001). There was a gradient increase in the OR for new-onset CVD in 2018 due to diabetes at baseline across the value of the TyG index based on a fully adjusted model (P for trend < 0.05). The ORs of diabetes at baseline for CVD in 2018 were 1.657 (95% CI 0.928-2.983, P = 0.098), 1.834(95% CI 1.064-3.188, P = 0.037) and 2.234(95% CI 1.349-3.673, P = 0.006) for T1, T2 and T3 of the TyG index respectively. The gradient of increasing risk of CVD still existed among those with hypertension and nondrinkers in the subgroup analysis. Conclusion Elevated TyG index strengthens the correlation between diabetes mellitus and CVD in middle-aged and elderly Chinese adults.