We explore whether stellar tidal streams can provide information on the secular, cosmological evolution of the Milky Way's gravitational potential and on the presence of subhalos. We carry out long-term (Δt~tHubble) N-body simulations of disrupting satellite galaxies in a semianalytic Galaxy potential in which the dark matter halo and the subhalos evolve according to a ΛCDM cosmogony. All simulations are constrained to end up with the same position and velocity at present. Our simulations account for (1) the secular evolution of the host halo's mass, size, and shape, (2) the presence of subhalos, and (3) dynamical friction. We find that tidal stream particles respond adiabatically to the Galaxy growth, so that, at present, the energy and angular momentum distribution is exclusively determined by the present Galaxy potential. In other words, all present-day observables can only constrain the present mass distribution of the Galaxy independent of its past evolution. We also show that if the full phase-space distribution of a tidal stream is available, we can accurately determine (1) the present Galaxy's shape and (2) the amount of mass loss from the stream's progenitor, even if this evolution spanned a cosmologically significant epoch.