Objective: The very first step for an autonomous lower gastrointestinal (GI) tract robot to carry out a diagnostic or a therapeutic task is to enter the lower GI. The natural entry point of the lower GI tract is the anus. Thus, to find the anus center is the very first step before entering the lower GI tract. However, to the authors' knowledge, there doesn't exist any results of detection of anus center.Methods: An image processing pipeline is proposed by combing several classical image methods including Otsu's method, the multiscale method and the threshold method and two deep neural networks, including Mask R-CNN and Inception-V3. Also, as a complementary result, the classification of healthy and diseased anus is determined by another Inception-V3, for healthy anus. Results: The positional error of the center detection by the proposed workflow is 2.15% averagely compared to the diagonal length, which is at the same level to that ten experienced proctologists. The proposed anal center detection method is applicable for both hairy anus and hairless anus. Also, the approach is valid for the anus in several common perianal diseases including perianal eczema, the mixed hemorrhoids, the anal fistula, the thrombotic external hemorrhoids, the internal hemorrhoids, and the external hemorrhoid.Conclusion: Anus center is detected by the proposed method with a similar accuracy to human doctors. Significance: This study provides the first solution for the anus center detection, enabling autonomous lower gastrointestinal tracts robot to enter anus without human guidance.