Label smoothing

[knazeri]

I believe the label smoothing is not performed for fake samples as mentioned under Use Soft and Noisy Labels section in this repo. In the referenced paper by Salimans et. al. 2016 they mention that they smooth only the positive labels, leaving negative labels set to 0.

It was later explained by Goodfellow at NIPS 2016 Tutorial why label smoothing is done only for real samples:

It is important to not smooth the labels for the fake samples. Suppose we use a target of 1 − α for the real data and a target of 0 + β for the fake samples. When β is zero, then smoothing by α does nothing but scale down the optimal value of the discriminator. When β is nonzero, the shape of the optimal discriminator function changes. The discriminator will thus reinforce incorrect behavior in the generator; the generator will be trained either to produce samples that resemble the data or to produce samples that resemble the samples it already makes.