BACKGROUND: Breast cancer has the highest cancer prevalence rate among the women worldwide. Early detection of breast cancer is crucial for successful treatment and reducing cancer mortality rate. However, tumor detection of breast ultrasound (US) image is still a challenging work in computer-aided diagnosis (CAD). OBJECTIVE: This study aims to develop a novel automated algorithm for breast tumor detection based on deep learning. METHODS: We proposed a new deep learning network named One-step model which have one input and two outputs, the first one was the segmentation result and the other one was used for false-positive reduction. The proposed One-step model includes three key components: Base-net, Seg-net, and Cls-net based on Anchor Box. The model chose DenseNet to construct Base-net, the decoder part of RefineNet as Seg-net, and connected several middle layers of Base-net and Seg-net to Cls-net. From the first output acquired by Base-net and Seg-net, the model detected a series of suspicious lesion regions. Then the second output from the Cls-net was used to recognize and reduce the false-positive regions. RESULTS: Experimental results showed that the new model achieved competitive detection result with 90.78% F1 score, which was 8.55% higher than Single Shot MultiBox Detector (SSD) method. In addition, running new model is also computational efficient and has comparative cost effect as SSD. CONCLUSIONS: We established a novel One-step model which improves location accuracy by generating more precise bounding box via Seg-net and removing false targets by another object detection network (Cls-net). On the other hand, a real-time detection of tumor is achieved by sharing the common Base-net. The experimental results showed that the new model performed well on various irregular and blurred ultrasound images. As a result, this study demonstrated feasibility of applying deep learning scheme to detect breast lesions depicting on US image.