The composition of the outer solar system is of particular interest because it holds the key to understanding the chemical evolution of the Solar System. Observations at the edge of the Solar System are difficult because of distance and size limitations. The Spitzer Space Telescope has provided a wealth of data for Kuiper Belt Objects (KBOs), the small inhabitants of this remote part of the Solar System past the orbit of Neptune, as well as for Centaurs, similar objects to the KBOs but with orbits that come closer to the Sun. Are these observations sufficient to tell us what the composition of these objects is? We briefly introduce spectral modeling, its strengths and limitations. Making use of synthetic surface reflectance spectra we assess the feasibility of determining the composition of Kuiper Belt Objects and Centaurs making use of Spitzer-IRAC data alone.