We make use of a semi-analytical model of galaxy formation to investigate the origin of the observed correlation between [α/Fe] abundance ratios and stellar mass in elliptical galaxies. We implement a new galaxy-wide stellar initial mass function (IMF; top-heavy integrated galaxy initial mass function, TH-IGIMF) in the semi-analytic model SAG (acronym for semi-analytic galaxies) and evaluate its impact on the chemical evolution of galaxies. The star formation rate (SFR) dependence of the slope of the TH-IGIMF is found to be key to reproducing the correct [α/Fe]-stellar mass relation. Massive galaxies reach higher [α/Fe] abundance ratios because they are characterized by more top-heavy IMFs as a result of their higher SFR. As a consequence of our analysis, the value of the minimum embedded star cluster mass and of the slope of the embedded cluster mass function, which are free parameters involved in the TH-IGIMF theory, are found to be as low as 5 and 2 M⊙, respectively. A mild downsizing trend is present for galaxies generated assuming either a universal IMF or a variable TH-IGIMF. We find that, regardless of galaxy mass, older galaxies (with formation redshifts ≳2) are formed in shorter time-scales (≲2 Gyr), thus achieving larger [α/Fe] values. Hence, the time-scale of galaxy formation alone cannot explain the slope of the [α/Fe]-galaxy mass relation, but is responsible for the big dispersion of [α/Fe] abundance ratios at fixed stellar mass. We further test the hypothesis of a TH-IGIMF in elliptical galaxies by looking into mass-to-light ratios, and luminosity functions. Models with a TH-IGIMF are also favoured by these constraints. In particular, mass-to-light ratios agree with observed values for massive galaxies while being overpredicted for less massive ones; this overprediction is present regardless of the IMF considered.