DOC: Adds Inigo's week 5 short blogpost #45
Conversation
|
🪓 PR closed, deleted preview at https://github.com/dipy/preview-html/tree/main/dipy.org/pull/45/ |
itellaetxe
requested review from
jhlegarreta,
skoudoro,
pjsjongsung,
WassCodeur,
robinroy03 and
deka27
There was a problem hiding this comment.
Hi @itellaetxe,
From my side, it looks good, just waiting that you address @jhlegarreta comments.
| Hi everyone! This week I have been on vacation, so I have not been able to work on the project as much as the previous weeks. However, I have been thinking about the next steps to take and I have decided to start with the conditional AutoEncoder. I have been reading some papers and I have found some interesting ideas that would be nice to implement. | ||
|
|
||
| While stuck in the Munich airport, I started to write some code for this (the weather was not very good and my flight was delayed, so I lost my connecting flight and I had to sleep in the airport). I found an implementation of a regression variational AutoEncoder `in this paper <https://doi.org/10.1007/978-3-030-32245-8_91>`_, where the authors implement a way to manipulate the latent space so that the samples (streamlines) that get projected into it are organized along a desired scalar parameter. | ||
| I thought this could be a good starting point for my conditional AutoEncoder because it basically provides a way to sample from the latent space in a controlled manner, where you can select the age of the streamlines you want to generate. Also, having the variational part makes the latent space more regularized, so our model is more resilient against overfitting, which is the main flaw of ordinary AutoEncoders. |
There was a problem hiding this comment.
Also, I am all that comfortable saying that overfitting is an inherent property to AEs. Do you have a scientific reference that supports this?
There was a problem hiding this comment.
Thnks for pointing this out. To be precise, I should rephrase this to "...overfitting, what could happen when non-regularized (vanilla) AutoEncoders learn the identity function, which leads to lack of generalizability outside the test set."
This is explained in this NeurIPS 2020 paper (PDF)
In conclusion, yes, saying that "AEs main flaw is (natural) overfitting" is too much. However, saying that they might overfit towards the identity function, which is the effect I was aware of at the time I wrote the blog, is more rigurous.
What do you think @jhlegarreta ?
There was a problem hiding this comment.
Thanks for the reference.
"which leads to lack of generalizability outside the test set." outside the training set ?
There was a problem hiding this comment.
True, sorry. Yes, I meant "outside the training set"
* DOC: Adds Inigo's week 5 short blogpost * FIX: Grammar issues and paragraph spacing * FIX: Corrects overfitting comment 65c3b20
Adds what I did on week 5, mainly vacation time.