Context. High-precision frequencies of acoustic modes in red giant stars are now available thanks to the long observing length and high quality of the light curves provided by the NASA Kepler mission, thus allowing the interior of evolved cool low-mass stars to be probed with an unprecedented level of detail. Aims: We characterize the acoustic signature of the helium second ionization zone in a sample of 18 low-mass low-luminosity red giants by exploiting new mode-frequency measurements derived from more than four years of Kepler observations. Methods: We analyzed the second frequency differences of radial acoustic modes in all the stars of the sample by using the Bayesian code Diamonds. Results: We find clear acoustic glitches due to the signature of helium second ionization in all the stars of the sample. We could measure the acoustic depth and the characteristic width of the acoustic glitches with a precision level on average around ~2% and ~8%, respectively. We find good agreement with theoretical predictions and existing measurements from the literature. Finally, we derive the amplitude of the glitch signal at νmax for the second differences and for the frequencies with an average precision of ~6%, obtaining values in the range 0.14-0.24 μHz and 0.08-0.33 μHz, respectively, which can be used to investigate the helium abundance in the stars. Appendix A is available in electronic form at http://www.aanda.org