Context. The X-shooter Spectral Library (XSL) is an empirical stellar library at medium spectral resolution covering the wavelength range from 3000 Å to 24 800 Å. This library aims to provide a benchmark for stellar population studies. Aims: In this work, we present a uniform set of stellar atmospheric parameters, effective temperatures, surface gravities, and iron abundances for 754 spectra of 616 XSL stars. Methods: We used the full-spectrum fitting package ULySS with the empirical MILES library as reference to fit the ultraviolet-blue (UVB) and visible (VIS) spectra. We tested the internal consistency and we compared our results with compilations from the literature. Results: The stars cover a range of effective temperature 2900 < Teff < 38 000 K, surface gravity 0 < logg < 5.7, and iron abundance -2.5 < [Fe/H] < +1.0, with a couple of stars extending down to [Fe/H] = -3.9. The precisions of the measurements for the G- and K-type stars are 0.9%, 0.14, and 0.06 in Teff, logg, and [Fe/H], respectively. For the cool giants with logg < 1, the precisions are 2.1%, 0.21, and 0.22, and for the other cool stars these values are 1%, 0.14, and 0.10. For the hotter stars (Teff > 6500 K), these values are 2.6%, 0.20, and 0.10 for the three parameters. Full Tables A.1-A.3 are only available at the CDS via anonymous ftp to http://cdsarc.u-strasbg.fr (ftp://130.79.128.5) or via http://cdsarc.u-strasbg.fr/viz-bin/qcat?J/A+A/627/A138Based on ESO observations from run IDs 084.B-0869(A/B), 085.B-0751(A/B) and 189.B-0925(A/B/C/D).