Unexplainable GAN losses, need help

[lennert130994]

So i have a simple GAN with dense layers and leakyReLU activation, nothing special.
When the GAN is training correctly and the generated samples look good, the training loss looks like this:

With blue being D and orange G.

A lot of times the GAN training goes wrong and the generated sample look like, well, shit. The training loss in this case looks like this:

Any help to what might cause this would be greatly appreciated!

[shaurov2253]

Hi, have you figured this out? I am trying to train a GAN and my losses look somewhat similar to your second figure.

[lennert130994]

<!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; font-size:11.0pt; font-family:"Calibri",sans-serif;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} .MsoChpDefault {mso-style-type:export-only;} @page WordSection1 {size:612.0pt 792.0pt; margin:70.85pt 70.85pt 70.85pt 70.85pt;} div.WordSection1 {page:WordSection1;} -->Hi, In my case it was caused by the GAN being to complex for the input data I fed it.So by doing some testing with smaller and less layers I eventually solved it.Also you need to play around with the complexity of the discriminator and generator so that they are balanced/competing. Let me know if you need more help. Regards,Lennert Sietsma  Van: shaurov2253Verzonden: donderdag 8 oktober 2020 02:17Aan: soumith/ganhacksCC: Lennert Sietsma; AuthorOnderwerp: Re: [soumith/ganhacks] Unexplainable GAN losses, need help (#57) Hi, have you figured this out? I am trying to train a GAN and my losses look somewhat similar to your second figure.—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe. 

[shaurov2253]

I am using a u-net for generator and 8-layer CNN for discriminator. I was also thinking that it may be happening because discriminator has too many layers. My understanding is the generator has much complex task than the discriminator (generating samples vs simple binary classification), which is why discriminator needs to be simpler to strike the balance. Thank you for your reply. If you got something to add, or think I'm thinking in the wrong direction. please let me know. Thanks again!