Pulmonary cancer is one of the most dangerous cancers with a high incidence and mortality. An early accurate diagnosis and treatment of pulmonary cancer can observably increase the survival rates, where computer-aided diagnosis systems can largely improve the efficiency of radiologists. In this article, we propose a deep automated lung nodule diagnosis system based on three-dimensional convolutional neural network (3D-CNN) and support vector machine (SVM) with multiple kernel learning (MKL) algorithms. The system not only explores the computed tomography (CT) scans, but also the clinical information of patients like age, smoking history and cancer history. To extract deeper image features, a 34-layers 3D Residual Network (3D-ResNet) is employed. Heterogeneous features including the extracted image features and the clinical data are learned with MKL. The experimental results prove the effectiveness of the proposed image feature extractor and the combination of heterogeneous features in the task of lung nodule diagnosis.