The weak-field approximation is one of the simplest models that allows us to relate the observed polarization induced by the Zeeman effect with the magnetic field vector present on the plasma of interest. It is usually applied for diagnosing magnetic fields in the solar and stellar atmospheres. A fully Bayesian approach to the inference of magnetic properties in unresolved structures is presented. The analytical expression for the marginal posterior distribution is obtained, from which we can obtain statistically relevant information about the model parameters. The role of a priori information is discussed and a hierarchical procedure is presented that gives robust results that are almost insensitive to the precise election of the prior. The strength of the formalism is demonstrated through an application to IMaX data. Bayesian methods can optimally exploit data from filter polarimeters given the scarcity of spectral information as compared with spectro-polarimeters. The effect of noise and how it degrades our ability to extract information from the Stokes profiles is analyzed in detail.