Sparse subspace clustering (SSC), as an effective subspace clustering technique, has been widely applied in the remote sensing community, demonstrating a superior performance over the traditional methods such as k-means. In this paper, we propose a unified framework for hyperspectral image (HSI) clustering, which incorporates spatial information and label information in an SSC model, aiming at generating a more precise similarity matrix. The spatial information is included through a joint sparsity constraint on the coefficient matrix of each local region. Pixels within a local region are encouraged to select a common set of samples in the subspace-sparse representation, which greatly promotes the connectivity of the similarity matrix. We incorporate the available label information effectively within the same framework, by zeroing the entries of the sparse coefficient matrix, which correspond to the data points from different classes. An optimization algorithm is derived based on the alternating direction method of multipliers for the resulting model. Experimental results on real HSIs demonstrate a superior performance over the related state-of-the-art methods.