In this paper we present a new method for joint denoising of depth and luminance images produced by time-of-flight camera. Here we assume that the sequence does not contain outlier points which can be present in the depth images. Our method first performs estimation of noise and signal covariance matrices and then performs vector denoising. Luminance image is segmented into similar contexts usina k-means algorithm, which are used for calculation of covariance matrices. Denoising results are compared with the ground truth images obtained by averaging of the multiple frames of the still scene.