Context. It has recently been discovered that the Galactic bulge is X-shaped, with the two southern arms of the X both crossing the lines of sight at l = 0 and | b| > 4, hence producing a double red clump in the bulge color magnitude diagram. Dynamical models predict the formation of X-shaped bulges as extreme cases of boxy-peanut bulges. However, since X-shaped bulges were known to be present only in external galaxies, models have never been compared to 3D kinematical data for individual stars. Aims: We study the orbital motion of Galactic bulge stars in the two arms (overdensities) of the X in the southern hemisphere. The goal is to provide observational constraints to bulge formation models that predict the formation of X-shapes through bar dynamical instabilities. Methods: Radial velocities have been obtained for a sample of 454 bulge giants, roughly equally distributed between the bright and the faint red clump, in a field at (l,b) = (0, -6). Proper motions were derived for all red clump stars in the same field by combining images from two epochs, which were obtained 11 years apart, with WFI at the 2.2 m at La Silla. The observed field contains the globular cluster NGC 6558, whose member stars were used to assess the accuracy of the proper motion measurement. At the same time, as a by-product, we provide the first proper motion measurement of NGC 6558. The proper motions for the spectroscopic subsample are analyzed for a subsample of 352 stars, taking into account the radial velocities and metallicities measured from near-infrared calcium triplet lines. Results: The radial velocity distribution of stars in the bright red clump, which traces the closer overdensity of bulge stars, shows an excess of stars moving towards the Sun. Similarly, an excess of stars receding from the Sun is seen in the far overdensity, which is traced by faint red clump stars. This is explained by the presence of stars on elongated orbits, which are most likely streaming along the arms of the X-shaped bulge. Proper motions for these stars are consistent with qualitative predictions of dynamical models of peanut-shaped bulges. Surprisingly, stars on elongated orbits have preferentially metal-poor (subsolar) metallicities, while the metal rich ones, in both overdensities, are preferentially found in more axisymmetric orbits. The observed proper motion of NGC 6558 has been measured as (μlcos (b),μb) = (0.30 ± 0.14, -0.43 ± 0.13), with a velocity dispersion of (σlcos(b),σb) = (1.8,1.7) mas/yr. This is the first proper motion measurement for this cluster. Based on observations taken with ESO telescopes at the La Silla Paranal Observatory under program IDs 163.O-0741(A), 085.D-0143(A) and 385.B-0735(B) and on observations taken with the Magellan telescope at the Las Campanas Observatory.Kinematics for IMACS and FLAMES spectra are only available at the CDS via anonymous ftp to http://cdsarc.u-strasbg.fr (ftp://130.79.128.5) or via http://cdsarc.u-strasbg.fr/viz-bin/qcat?J/A+A/555/A91