A search for Higgs boson pair production in the bb¯ WW * decay mode is performed in the bb¯ lνqq final state using 36.1 fb −1 of proton-proton collision data at a centre-of-mass energy of 13 TeV recorded with the ATLAS detector at the Large Hadron Collider. No evidence of events beyond the background expectation is found. Upper limits on the non-resonant pp → HH production cross section of 10 pb and on the resonant production cross section as a function of the HH invariant mass are obtained. Resonant production limits are set for scalar and spin-2 graviton hypotheses in the mass range 500 to 3000 GeV.