The linearly polarized solar limb spectrum caused by the absorption and scattering of anisotropic radiation has a very rich diagnostic potential, given its sensitivity to the thermal, dynamic, and magnetic structure of the solar atmosphere. A crucial first step toward its scientific exploitation is understanding the physical origin of the observed spectral line polarization and its magnetic sensitivity via the Hanle and Zeeman effects. Here, we study the linear polarization signals observed in the IR triplet of O i at 777 nm, describing in detail the multilevel radiative transfer calculations that allowed us to decipher their physical origin. We investigate the sensitivity of the calculated scattering polarization signals to various modeling parameters, finding that the observed fractional linear polarization pattern originates mainly in the solar chromosphere, although the intensity profiles of the O i IR triplet come mainly from the lower photosphere. We find that the three lines are sensitive, via the Hanle effect, to magnetic fields with strengths between 0.01 and 30 G, in a extended region of the solar atmosphere. We show this through calculations of the response function to magnetic field perturbations in a semi-empirical model of the quiet Sun atmosphere. The dominant response of the linear polarization signals occurs at heights ∼ 1000 km above the visible model’s surface, which demonstrates that the scattering linear polarization signals of the oxygen IR triplet encode information on the magnetism of the solar chromosphere.