In this paper we present a method for semi-automatic object tracking in video sequences using multiple features and a method for probabilistic relaxation to improve the tracking results producing smooth and accurate tracked borders. Starting from a given initial position of the object in the first frame the proposed method automatically tracks the object in the sequence modeling the a posteriori probabilities of a set of features such as: color, position and motion, depth, etc