To effectively treat bone diseases using bone regenerative medicine, there is an urgent need to develop safe and cheap drugs that can potently induce bone formation. Here, we demonstrate the osteogenic effects of icariin, the main active compound of Epimedium pubescens. Icariin induced osteogenic differentiation of preosteoblastic cells. The combination of icariin and a helioxanthin-derived small compound synergistically induced osteogenic differentiation of MC3T3-E1 cells to a similar extent to bone morphogenetic protein-2. Icariin enhanced the osteogenic induction activity of bone morphogenetic protein-2 in a fibroblastic cell line. Mineralization was enhanced by treatment with a combination of icariin and calcium-enriched medium. The in vivo anabolic effect of icariin was confirmed in a mouse calvarial defect model. Eight-week-old male C57BL/6N mice were transplanted with icariin-calcium phosphate cement (CPC) tablets or CPC tablets only (n = 5 for each), and bone regeneration was evaluated after 4 and 6 weeks. Significant new bone formation was observed in the icariin-CPC group at 4 weeks, and the new bone thickness had increased by 6 weeks. Obvious blood vessel formation was observed in the icariin-induced new bone. Treatment of senescence-accelerated mouse prone 1 and senescence-accelerated mouse prone 6 models further demonstrated that icariin was able to enhance bone formation in vivo. Therefore, icariin is a strong candidate for an osteogenic compound for use in bone tissue engineering.