To improve the generalization of deep convolutional neural networks (CNN)

we proposed a discriminatively orthogonal feature generation method. By regularizing nonnegative outputs

orthogonal degree and correlation degree were optimized simultaneously

which helps to generate discriminatively orthogonal and sparse features. To adjust sparse degree for controlling network capacity

the technique of auto-adjusting regularization coefficient was proposed. To improve computational efficiency

a stochastically 2-class discriminatively orthogonal feature generation rule was further designed. Subsequently

a comparative experiment was conducted on handwritten digit set MNIST. In this experiment

the sparsity adjustment property of our method was verified. By means of deconvolution technique for visualization

it was further found that our method has a good property of focusing on local discriminant areas. Finally

our method was applied to Alzheimer's Disease MRI image analysis. The experimental results showed that our method outperforms some other representative methods and locates the importantly discriminant brain regions successfully.