Convolutional neural network (CNN) is a powerful visual model which has a significant performance in various visual recognition issues, and attracted considerable attention in recent years. Due to the strong correlation between bands and small sample set (3S) problem, hyperspectral image classification issue remains a challenging problem. In this paper, we construct a novel multi-task framework with embedded convolutional neural network to obtain stronger discriminative capability for hyperspectral image classification. A variance of CNN, which is named Network in Network (NIN), is used to construct three sub-model. Here 1 × 1 convolutional filter is adopted and average pooling layer is employed to replace the full connect layer. The input of the first sub-model is spatial-spectral schroedinger Eigenmapes (SSSE) which can provide fused information of spatial and spectral information. Meanwhile, due to the significant capability of extracting spatial texture of the uniform local binary (ULBP), the histogram feature extracted from ULBP is used as the second sub-model input. Raw hyperspectral data is input to the last sub-model for supplying external information that SSSE and ULBP have lost. Experimental results demonstrate the effectiveness of the proposed model.