In this paper a framework is presented to automatically extract a sequence of images of the background of a scene from a shaky film. That is, the input video sequence may have local and global motion but the output video must contain exclusively the static background scene. Applying robust procedures to this end is one of the main goals of this work, since the aim is to get a procedure not only resistant to low scale noise but to occasional high scale noise. The median is used as an estimate of the background, the median absolute deviation (MAD) is used to establish a threshold to locate foreground and M-estimation for regression is used to stabilize the video sequence.