Spectral energy distribution (SED) fitting is one of the most commonly used techniques to study the dust properties in active galactic nuclei (AGN). Works implementing this technique commonly use radiative transfer models that assume a variety of dust properties. Despite the key role of this aspect, limited effort has been put forward to explore the chemical composition, the role of different optical properties, and the grain size distribution of dust, all of which can have a substantial impact on the theoretical radiative transfer calculations. In this work, we explore the role of the dust chemical composition in the AGN dusty torus through SED fitting to Spitzer/IRS spectra of a sample of 49 nearby AGN with silicate features in emission. We implement a mineralogy model including the popular astronomical silicates and a set of oxides and amorphous silicates with different grain sizes. Wefind that best fits use principally porous alumina, periclase, and olivine. In terms of mass fractions, $\sim 99~{{\ \rm per\ cent}}$ of the dust is composed of dust grains of size $\rm {0.1 \, \mu m}$, with a $\lt 1~{{\ \rm per\ cent}}$ contribution from $\rm {3 \, \mu m}$ grains. Moreover, the astronomical silicates have a very low occurrence in the best fits, suggesting that they are not the most suited dust species to reproduce the silicate features in our sample.