The 3D geometries of high-redshift galaxies remain poorly understood. We build a differentiable Bayesian model and use Hamiltonian Monte Carlo to efficiently and robustly infer the 3D shapes of star-forming galaxies in James Webb Space Telescope Cosmic Evolution Early Release Science observations with $\mathrm{log}{M}_{* }/{M}_{\odot }=9.0\mbox{--}10.5$ at z = 0.5–8.0. We reproduce previous results from the Hubble Space Telescope Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey in a fraction of the computing time and constrain the mean ellipticity, triaxiality, size, and covariances with samples as small as ∼50 galaxies. We find high 3D ellipticities for all mass–redshift bins, suggesting oblate (disky) or prolate (elongated) geometries. We break that degeneracy by constraining the mean triaxiality to be ∼1 for $\mathrm{log}{M}_{* }/{M}_{\odot }=9.0\mbox{--}9.5$ dwarfs at z > 1 (favoring the prolate scenario), with significantly lower triaxialities for higher masses and lower redshifts indicating the emergence of disks. The prolate population traces out a "banana" in the projected $b/a\mbox{--}\mathrm{log}a$ diagram with an excess of low-b/a, large- $\mathrm{log}a$ galaxies. The dwarf prolate fraction rises from ∼25% at z = 0.5–1.0 to ∼50%–80% at z = 3–8. Our results imply a second kind of disk settling from oval (triaxial) to more circular (axisymmetric) shapes with time. We simultaneously constrain the 3D size–mass relation and its dependence on 3D geometry. High-probability prolate and oblate candidates show remarkably similar Sérsic indices (n ∼ 1), nonparametric morphological properties, and specific star formation rates. Both tend to be visually classified as disks or irregular, but edge-on oblate candidates show more dust attenuation. We discuss selection effects, follow-up prospects, and theoretical implications.