Following our study on the incidence, morphology and kinematics of the ionized gas in early-type galaxies, we now address the question of what is powering the observed nebular emission. To constrain the likely sources of gas excitation, we resort to a variety of ancillary data we draw from complementary information on the gas kinematics, stellar populations and galactic potential from the SAURON data, and use the SAURON-specific diagnostic diagram juxtaposing the [OIII] λ5007/Hβ and [NI] λλ5197, 5200/Hβ line ratios. We find a tight correlation between the stellar surface brightness and the flux of the Hβ recombination line across our sample, which points to a diffuse and old stellar source as the main contributor of ionizing photons in early-type galaxies, with post-asymptotic giant branch (pAGB) stars being still the best candidate based on ionizing balance arguments. The role of AGN photoionization is confined to the central 2-3arcsec of an handful of objects with radio or X-ray cores. OB-stars are the dominant source of photoionization in 10 per cent of the SAURON sample, whereas for another 10 per cent the intense and highly ionized emission is powered by the pAGB population associated to a recently formed stellar subcomponent. Fast shocks are not an important source of ionization for the diffuse nebular emission of early-type galaxies since the required shock velocities can hardly be attained in the potential of our sample galaxies. Finally, in the most massive and slowly or non-rotating galaxies in our sample, which can retain a massive X-ray halo, the finding of a spatial correlation between the hot and warm phases of the interstellar medium (ISM) suggests that the interaction with the hot ISM provides an additional source of ionization besides old ultraviolet-bright stars. This is also supported by a distinct pattern towards lower values of the [OIII]/Hβ ratio. These results lead us to investigate the relative role of stellar and AGN photoionization in explaining the ionized gas emission observed in early-type galaxies by the Sloan Digital Sky Survey (SDSS). By simulating how our sample galaxies would appear if placed at further distance and targeted by the SDSS, we conclude that only in very few, if any, of the SDSS galaxies which display modest values for the equivalent width of the [OIII] line (less than ~2.4 Å) and low-ionization nuclear emission-line region like [OIII]/Hβ values the nebular emission is truly powered by an AGN.