We examine the distribution of the [O/Fe] abundance ratio in stars across the Galactic disk using H-band spectra from the Apache Point Galactic Evolution Experiment (APOGEE). We minimize systematic errors by considering groups of stars with similar atmospheric parameters. The APOGEE measurements in the Sloan Digital Sky Survey data release 12 reveal that the square root of the star-to-star cosmic variance in the oxygen-to-iron ratio at a given metallicity is about 0.03-0.04 dex in both the thin and thick disk. This is about twice as high as the spread found for solar twins in the immediate solar neighborhood and the difference is probably associated to the wider range of galactocentric distances spanned by APOGEE stars. We quantify the uncertainties by examining the spread among stars with the same parameters in clusters; these errors are a function of effective temperature and metallicity, ranging between 0.005 dex at 4000 K and solar metallicity, to about 0.03 dex at 4500 K and [Fe/H] ≃ -0.6. We argue that measuring the spread in [O/Fe] and other abundance ratios provides strong constraints for models of Galactic chemical evolution.