We identify six ultra-diffuse galaxies (UDGs) outside clusters in three nearby isolated groups (0.014 < z < 0.026) using very deep imaging in three different Sloan Digital Sky Survey filters (g, r and I bands) from the IAC Stripe 82 Legacy Project. By comparing with the abundance of UDGs in rich galaxy clusters, we find that the density of UDGs (i.e. the number per unit mass of the host structure where they are located) decreases towards the most massive systems. This is compatible with a scenario where UDGs are formed preferentially outside clusters. In the periphery (D > 250 kpc) of our three groups, we identify a population of potential UDG progenitors (two of them confirmed spectroscopically). These progenitors have similar masses, shapes and sizes but are bluer, g - I ∼ 0.45 [and for this reason brighter, μg(0) < 24 mag arcsec-2] than traditional UDGs (g - I ∼ 0.76). Passive evolution of these progenitors will transform them into regular [i.e. μg(0) > 24 mag arcsec-2] UDGs after ∼6 Gyr. If confirmed, our observations support a scenario where UDGs are old, extended, low surface brightness dwarf galaxies (M⋆ ∼ 108 M⊙) born in the field, are later processed in groups and, ultimately, infall into galaxy clusters by group accretion.