Recent Atacama Large Millimeter/submillimeter Array (ALMA) observations of protoplanetary disks in the millimeter continuum have shown a variety of radial gaps, cavities, and spiral features. These substructures may be signposts for ongoing planet formation, and therefore these systems are promising targets for direct imaging planet searches in the near-infrared. To this end, we present results from a deep imaging survey in the band (3.8 μm) with the Keck/NIRC2 vortex coronagraph to search for young planets in 43 disks with resolved features in the millimeter continuum or evidence for gaps/central cavities from their spectral energy distributions. Although we do not detect any new point sources, using the vortex coronagraph allows for high sensitivity to faint sources at small angular separations (down to ∼0.″1), allowing us to place strong upper limits on the masses of potential gas giant planets. We compare our mass sensitivities to the masses of planets derived using ALMA observations, and while we are sensitive to ∼1 M Jup planets in the gaps in some of our systems, we are generally not sensitive to planets of the masses expected from the ALMA observations. In addition to placing upper limits on the masses of gas giant planets that could be interacting with the dust in the disks to form the observed millimeter substructures, we are also able to map the micron-sized dust as seen in scattered light for 8 of these systems. Our large sample of systems also allows us to investigate limits on planetary accretion rates and disk viscosities.