Models predict that planets with water-rich compositions may be common around low-mass stars, but definitive evidence for the existence of water worlds has remained elusive. Precise mass and radius measurements for small planets allow us to identify candidate water worlds that have bulk densities too low to be consistent with a rocky composition, and masses too low to retain any H-rich envelope against atmospheric escape. Dynamical interactions between adjacent planets in near-resonant multi-planet systems produce transit timing variations (TTVs) that can be observed with high-precision photometry, and measurements of these TTVs can be inverted to produce constraints on planetary densities. Over the past two years, we have collected dozens of high SNR transit observations using ground based telescopes, along with supplemental radial velocity and stellar XUV data for three nearby multi-planet M dwarf systems containing small, Earth-sized exoplanets. Using these data, we have identified several candidate water worlds that are some of the most favorable targets for atmospheric characterization with JWST, including some of the coldest known water world candidates.