In deep learning models, neural networks significantly grow the speech recognition task. Speech Recognition works on human inputs that enable machines to react on inserted text, voice, or any other inputs. Automatic speech recognition (ASR) is based on two probability functions: an acoustic model that computes the probability of correspondence of an utterance to the input word sequence and a language model that calculates the prior probability of the meaning of the input. A deep neural network as an acoustic model has improved the performance of the ASR system. Various deep neural network algorithms are applied for automatic speech recognition, such as convolutional neural networks, deep belief networks, and recurrent neural networks are most commonly used. The application areas of speech recognition are Home automation, Interactive voice response, Mobile telephony, including smartphones, email, In-car systems, Health care, MilitaryTelephony, IoT, and other domains. Recently, transformers for speech recognition that achieve better performance. Future advancements of speech recognition using deep neural networks are Deep RNN - long short term memory for speech recognition, subword-based HMM-DNN speech recognition, Advancing RNN Transducer Technology for Speech Recognition, and many more.