High-resolution observations provide evidence of the existence of small-amplitude transverse oscillations in solar filament fine structures. These oscillations are believed to represent fast magnetohydrodynamic (MHD) waves, and the disturbances are seen to be damped on short timescales of the order of 1-4 periods. In this Letter, we propose that, due to the highly inhomogeneous nature of the filament plasma at the fine-structure spatial scale, the phenomenon of resonant absorption is likely to operate in the temporal attenuation of fast MHD oscillations. By considering transverse inhomogeneity in a straight flux tube model, we find that, for density inhomogeneities typical of filament threads, the decay times are of a few oscillatory periods only.