Existing and upcoming instrumentation is collecting large amounts of astrophysical data, which require efficient and fast analysis techniques. We present a deep neural network architecture to analyze high-resolution stellar spectra and predict stellar parameters such as effective temperature, surface gravity, metallicity, and rotational velocity. With this study, we firstly demonstrate the capability of deep neural networks to precisely recover stellar parameters from a synthetic training set. Secondly, we analyze the application of this method to observed spectra and the impact of the synthetic gap (i.e., the difference between observed and synthetic spectra) on the estimation of stellar parameters, their errors, and their precision. Our convolutional network is trained on synthetic PHOENIX-ACES spectra in different optical and near-infrared wavelength regions. For each of the four stellar parameters, Teff, log g, [M/H], and v sin i, we constructed a neural network model to estimate each parameter independently. We then applied this method to 50 M dwarfs with high-resolution spectra taken with CARMENES (Calar Alto high-Resolution search for M dwarfs with Exo-earths with Near-infrared and optical Échelle Spectrographs), which operates in the visible (520-960 nm) and near-infrared wavelength range (960-1710 nm) simultaneously. Our results are compared with literature values for these stars. They show mostly good agreement within the errors, but also exhibit large deviations in some cases, especially for [M/H], pointing out the importance of a better understanding of the synthetic gap.