Enhanced Handwritten Digit Recognition with a Hybrid Optimization Framework for Deep Learning Techniques

Abstract

One of the interesting applications for machine deep learning in the arena of information extraction is the recognition of handwritten numerals. Even though this is an established field, still it becomes difficult to recognize numbers with an efficient optimization. Due to higher levels of complexity and storage, training of such a model using large amount of data often fails. A convolutional neural network (CNN) with amalgamation of mini-batch and stochastic Hessian-free optimization (HybOpt) technique is used in this paper to get predictions that are accurate and converge more quickly. When solving quadratic equations, which intensely depend upon computation and storage, a second-order approximation is applied to escalate speed. The proposed technique is also using a repetitive minimization method for quicker convergence with an arbitrary initialization. The effectiveness of the proposed method is examined by doing study on standard MNIST dataset. The CNN is trained up to 15 layers using HybOpt and results are compared with existing optimization techniques such as MBSGD, SGD, SHF, HFO and NCG. The dataset was divided into 85% training and 15% testing images. Parameters for example accuracy, f1-score, precision and recall are found. Additionally, running time against epochs are also observed. In all the parameters, it was found that the proposed method produces the best results.