Background Rheumatoid arthritis (RA) and osteoarthritis (OA) were two major types of joint diseases. This study aimed to explore the mechanism underlying OA and RA and analyze their difference by integrated analysis of multiple gene expression data sets. Methods Gene expression data sets of RA and OA were downloaded from The Gene Expression Omnibus. Shared and specific differentially expressed genes (DEGs) in RA and OA were identified by integrated analysis of multiple gene expression data sets. Functional annotation and protein-protein interaction (PPI) network construction of OA- and RA-specific DEGs were performed to further explore the molecular mechanisms underlying RA and OA and analyze the mechanism differences between them. Results Compared with normal controls, 3757 and 2598 DEGs were identified in RA and OA, respectively. Among them, 2176 DEGs were RA-specific DEGs and 1017 DEGs were OA-specific DEGs. Moreover, the expression of 17 DEGs played opposite pattern in RA and OA compared with normal controls. Chemokine signaling pathway and oxidative phosphorylation were significantly enriched pathways for RA- and OA-specific DEGs, respectively. BIRC2 and CSNK1E were respective hub genes of RA- and OA-specific PPI network. Conclusion Our findings provided clues for the specific mechanism and developing specific biomarkers for RA and OA.