This review summarizes the observational constraints on the mass spectrum of compact objects in X-ray binaries. We currently have 20 X-ray binaries with confirmed black holes, based on dynamical information (i.e. mass in excess of 3 Msun). In two cases, V404 Cyg and GRS 1915+105, the black hole mass exceeds the maximum predicted by current Type Ib supernovae models and challenges black hole formation scenarios. The great majority of black hole binaries are members of the class of X-ray Transients, where long periods of quiescence enable spectroscopic studies of the faint donor stars. On the other hand, neutron star binaries are mostly found in persistent binaries, where reprocessed light from the accretion disc overwhelms the companion star and precludes mass estimates. New results, based on the detection of optical fluorescent lines from the donor star and X-ray burst oscillations, provide the best prospects for mass constraints of neutron stars in persistent X-ray binaries.