A Survey of Regularization Methods for Deep Neural Network?

Mimicking the human psyche has been a core challenge in machine learning research. Deep Neural network inspired from the human Visual cortex system are powerful computational model represents the large features in a hierarchical way. Overfitting is a major problem in deep learning due to the presence of a large number of features. Dropout is a proficient and simple method to prevent co-adaptation of features and thus stymie to over fit. It simply drops hidden units with probability 0.5.Maxout a new activation unit built on dropout has improved accuracy on datasets. Other recent companions are DropConect, DropAll and stochastic pooling. Dropout achieved state-of-art results on many labelled benchmark datasets: MNIST, CIFAR-10, CIFAR-100 and SVHN. This paper reviews different techniques to reduce Overfitting in Neural Network.