A search for a heavy, CP-odd Higgs boson, A, decaying into a Z boson and a 125 GeV Higgs boson, h, with the ATLAS detector at the LHC is presented. The search uses proton–proton collision data at a centre-of-mass energy of 8 TeV corresponding to an integrated luminosity of 20.3 fb−1. Decays of CP-even h bosons to ττ or bb pairs with the Z boson decaying to electron or muon pairs are considered, as well as h → bb decays with the Z boson decaying to neutrinos. No evidence for the production of an A boson in these channels is found and the 95% confidence level upper limits derived for σ(gg → A) × BR(A → Zh) × BR(h → f ¯ f ) are 0.098–0.013 pb for f = τ and 0.57–0.014 pb for f = b in a range of mA = 220–1000 GeV. The results are combined and interpreted in the context of two-Higgs-doublet models.