Mitochondrial homeostasis is closely associated with liver cancer progression via multiple mechanisms and is also a potential tumour-suppressive target in clinical practice. However, the role of mitochondrial fission in liver cancer cell viability has not been adequately investigated. Matrine, a type of alkaloid isolated from Sophoraflavescens, has been widely used to treat various types of cancer. However, the molecular effect of matrine on mitochondrial homeostasis is unclear. Therefore, the aim of the current study was to determine the role of mitochondrial fission in cell apoptosis, viability, migration and proliferation of HepG2 cells in vitro. The effect of matrine on mitochondrial fission and its mechanism were also explored. The results of our study showed that HepG2 cells treated with matrine had reduced viability, an increased apoptotic rate, a blunted migratory response, and impaired proliferation capacity. At the molecular level, matrine treatment activated mitochondrial fission, which promoted mitochondrial dysfunction, caused cellular oxidative stress, disrupted cellular energy metabolism and initiated cell apoptotic pathways. However, blockade of mitochondrial fission abolished the deleterious effects of matrine on HepG2 cells. Further, we demonstrated that the Mst1-JNK signalling axis was required for matrine-modulated mitochondrial fission. Matrine-mediated mitochondrial dysfunction was reversed by inhibiting Mst1-JNK pathways. Together, our results demonstrated that mitochondrial fission could be a potential upstream tumour-suppressive signal for liver cancer by modifying mitochondrial function and cell death. By contrast, matrine exerted an anticancer function in liver cancer by activating mitochondrial fission mediated by Mst1-JNK pathways.