In a statistical analysis of Debrecen Photoheliographic Results sunspot area data we find that the logarithmic deviation (logD)' of the area decay rate D from the parabolic mean decay law (derived in the first paper in this series) follows a Gaussian probability distribution. As a consequence, the actual decay rate D and the time-averaged decay rate overline D are also characterized by approximately lognormal distributions, as found in an earlier work. The correlation time of (logD)' is about 3 days. We find a significant physical anticorrelation between (logD)' and the amount of plage magnetic flux of the same polarity in an annulus around the spot on Kitt Peak magnetograms. The anticorrelation is interpreted in terms of a generalization of the turbulent erosion model of sunspot decay to the case when the flux tube is embedded in a preexisting homogeneous `plage' field. The decay rate is found to depend inversely on the value of this plage field, the relation being very close to logarithmic, i.e., the plage field acts as multiplicative noise in the decay process. A Gaussian probability distribution of the field strength in the surrounding plage will then naturally lead to a lognormal distribution of the decay rates, as observed. It is thus suggested that, beside other multiplicative noise sources, the environmental effect of surrounding plage fields is a major factor in the origin of lognormally distributed large random deviations from the mean law in the sunspot decay rates.