Aims:We introduce analytical response functions and their main properties as an important diagnostic tool that help understand Stokes profile formation physics and the meaning of well-known behaviors of standard inversion codes of the radiative transfer equation often used to measure solar magnetic fields. Methods: A Milne-Eddington model atmosphere is used as an example where response functions are analytical. A sample spectral line has been chosen to show the main qualitative properties. Results: We show that analytic response functions readily provide explanations for various well-known behaviors of spectral lines, such as the sensitivity of visible lines to weak magnetic fields or the trade-offs often detected in inversion codes between the Milne-Eddington thermodynamic parameters. We also show that response functions are helpful in selecting sample wavelengths optimized for specific parameter diagnostics. Appendix A is only available in electronic form at http://www.aanda.org