We present the first detailed kinematic analysis of the proper motions (PMs) of stars in the Magellanic Bridge, from both the Gaia Data Release 2 catalog and from Hubble Space Telescope (HST) Advanced Camera for Surveys data. For the Gaia data, we identify and select two populations of stars in the Bridge region, young main-sequence (MS) and red giant stars. The spatial locations of the stars are compared against the known H I gas structure, finding a correlation between the MS stars and the H I gas. In the HST fields our signal comes mainly from an older MS and turnoff population, and the PM baselines range between ∼4 and 13 yr. The PMs of these different populations are found to be consistent with each other, as well as across the two telescopes. When the absolute motion of the Small Magellanic Cloud is subtracted out, the residual Bridge motions display a general pattern of pointing away from the Small Magellanic Cloud toward the Large Magellanic Cloud. We compare in detail the kinematics of the stellar samples against numerical simulations of the interactions between the Small and Large Magellanic Clouds, and find general agreement between the kinematics of the observed populations and a simulation in which the Clouds have undergone a recent direct collision.