DOC: Adds Iñigo's week 3 and 4 blogpost #42

itellaetxe · 2024-06-14T12:43:10Z

Adds @itellaetxe blogpost for Week 3.

itellaetxe · 2024-06-14T12:46:14Z

@skoudoro I still cannot request review from anyone in this repo. In the DIPY repo, I can, but not here.

robinroy03

LGTM. Good work 👍. Please see the minor typo below but that's everything.

robinroy03 · 2024-06-16T11:54:13Z

posts/2024/2024_06_14_Inigo_week_3.rst

+What is coming up next week
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+My mentors and I agreed on trying to transfer the weights of the pre-trained PyTorch model to my Keras implementation, because it may take less time than actually training the model. Thus, the strategy we devised for this to work is the following:
+1. Implement dataset loading using HDF5 files, as the original model uses them, and the TractoInferno dataset is contained in such files (it is approximately 75 GB)/


/ looks like a typo for the punctuation mark . (line 25 end)

True, TY for your review :)!

deka27 · 2024-06-19T04:23:34Z

LGTM. Cant review tho.

itellaetxe · 2024-06-21T09:15:21Z

This PR had not been merged and I did not want to create another branch in my fork, so I just pushed the new blog here (week 4). I also updated the PR title.

@WassCodeur, @deka27, I think you cannot review this for whatever reason, but could you comment something on the PR? (I already saw your comment on week 3 @deka27, ty)

@skoudoro, whenever you can, could you check the review request problem?

thanks!

WassCodeur · 2024-06-21T14:29:59Z

posts/2024/2024_06_21_Inigo_week_4.rst

@@ -0,0 +1,48 @@
+Week 4 into GSoC 2024: Weight transfer experiments, hardships, and results!


Hi @itellaetxe !

Great work,

I've added a few comments below

TY for the review Wachiou!

posts/2024/2024_06_21_Inigo_week_4.rst

WassCodeur · 2024-06-21T14:39:56Z

posts/2024/2024_06_21_Inigo_week_4.rst

+On the other hand I started implementing the dataset loading using HDF5 files, but I set that aside because it is not priority.
+Finally, my mentor `Jon Haitz <https://github.com/jhlegarreta>`_ kindly provided me with the weights of the PyTorch AE he trained on the FiberCup dataset, and he suggested an experiment consisting of encoding the FiberCup tractogram with my Keras model, and Decoding it with the PyTorch model to see if the Encoder works properly.
+This was indeed the case, as the PyTorch model effectively reconstructed the tractogram, but unfortunately the Keras encoder was not capable of giving the same result. Naturally, this suggests that the Keras Decoder implementation is still not similar enough to the PyTorch one, so there is still room
+for improvement. Despite not being successful, this experiment was very enlightening, and it gave me a lot of insight into the differences between the two implementations.


Cool, it's clear even not being too familiar I get it. Thank you for sharing

WassCodeur · 2024-06-21T14:46:06Z

posts/2024/2024_06_14_Inigo_week_3.rst

+
+
+What I did this week
+~~~~~~~~~~~~~~~~~~~~


LGTM, apart from that if you could leave a blank line after the titles, like here, it would be better for reading.

jhlegarreta

Thanks for this work.

Contents are well-explained. Thanks.

Two comments about the form:

Not sure how this looks like when it is deployed since I am unable to find the corresponding URL, but the text looks busy: I would group related sentences in real paragraphs and add white lines between paragraphs, otherwise it is hard to follow.
Although it is not that important here, remember that adding a commit message body is helpful. Let me know if you are unsure how to do this.

jhlegarreta · 2024-06-22T14:59:03Z

posts/2024/2024_06_21_Inigo_week_4.rst

+This was indeed the case, as the PyTorch model effectively reconstructed the tractogram, but unfortunately the Keras encoder was not capable of giving the same result. Naturally, this suggests that the Keras Decoder implementation is still not similar enough to the PyTorch one, so there is still room
+for improvement. Despite not being successful, this experiment was very enlightening, and it gave me a lot of insight into the differences between the two implementations.
+In a last effort to get to replicate the PyTorch model results, I went on to train the my Keras architecture on the FiberCup dataset with the same parameters as my mentor used in his `GESTA <https://doi.org/10.1016/j.media.2023.102761>`_ paper to see if the results I get are similar to the ones he got.
+Well, this resulted in amazing results, as you can check visually in the figure below. Note that none of the models were able to capture the depth dimension of the streamlines, but this is not concerning. It can be solved reducing the latent dimension size to 16 (it is 32 now).


Reducing ? Are we sure? I thought this was achieved when we increased it to 64 ? Can you please check this?

I remember I made an experiment with the latent dimension size set to 16, and the depth was captured. While I agree that it is indeed counterintuitive, I got that result. However, since I did not prove thoroughly that the problem is solved like this, I will just stick to saying, "it can be solved by changing the latent dimension size" to keep it more general.

jhlegarreta · 2024-06-22T15:01:45Z

posts/2024/2024_06_21_Inigo_week_4.rst

+As the Keras 1D convolutional output dimensions do not follow the same ordering as in PyTorch, (*[n, m, channels]* vs *[n, channels, m]*), the flattening behavior of the models was different, and thus, the fully connected layer of the Encoder (named ``fc1``) was receiving different inputs.
+To solve this, I first reshaped the output of the Keras 1D convolutional layer to match the PyTorch *channels first* convention, and then applied the flattening.
+This effectively resulted in a within-reasonable-error (MAE = 1e-6) output of the Encoder block. Problem solved!
+The Decoder block was a bit more challenging, because the PyTorch implementation was using linear interpolation in its ``torch.nn.Upsample`` layers. For this, I had to implement a custom layer in Keras that would perform the same operation,


Unnecessary line break.

jhlegarreta · 2024-06-22T15:02:05Z

posts/2024/2024_06_21_Inigo_week_4.rst

+The errors in the Decoder block are higher than in the Encoder but we assumed that a MAE of around 1e-3 is acceptable.
+On the other hand I started implementing the dataset loading using HDF5 files, but I set that aside because it is not priority.
+Finally, my mentor `Jon Haitz <https://github.com/jhlegarreta>`_ kindly provided me with the weights of the PyTorch AE he trained on the FiberCup dataset, and he suggested an experiment consisting of encoding the FiberCup tractogram with my Keras model, and Decoding it with the PyTorch model to see if the Encoder works properly.
+This was indeed the case, as the PyTorch model effectively reconstructed the tractogram, but unfortunately the Keras encoder was not capable of giving the same result. Naturally, this suggests that the Keras Decoder implementation is still not similar enough to the PyTorch one, so there is still room


Unnecessary line break.

jhlegarreta · 2024-06-22T15:08:36Z

posts/2024/2024_06_21_Inigo_week_4.rst

+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Next week we will start working on a conditional version of the AutoEncoder, which should give us the ability to generate tractograms conditioned on a specific scalar input. This will be a very interesting feature to have because we can get tractograms with properties of interest. Well, this is the main goal of this project.
+The sampling strategy for the tracts does not concern me for now because the code is already available in the `tractolearn <https://github.com/scil-vital/tractolearn>`_ repository, so we can postpone it for now.


"The sampling strategy for the tracts does not concern me for now because the code is" : "It is decided to focus on developping a conditional version of the autoencoder over adding the latent space sampling because the code for the latter is"

Sounds way better, thank you

jhlegarreta · 2024-06-22T15:15:15Z

posts/2024/2024_06_21_Inigo_week_4.rst

+Thus, matching the behavior of the PyTorch model with the Keras implementation became my objective. To achieve so, I run a common input through all the layers of both models sequentially, and systematically compared the outputs of each layer.
+In the Encoder block, I found all the outputs to be within a reasonable range of each other (MAE = 1e-6), except for the last two operations, which flatten the output of the 1D convolutional layers and then feed it to a fully connected layer.
+This was partially good news, because most of the Encoder was behaving as desired, but, the most challenging part was adapting the flattening and reshaping operations happening in the encoder and the decoder, respectively.
+As the Keras 1D convolutional output dimensions do not follow the same ordering as in PyTorch, (*[n, m, channels]* vs *[n, channels, m]*), the flattening behavior of the models was different, and thus, the fully connected layer of the Encoder (named ``fc1``) was receiving different inputs.


the flattening behavior of the models was different (the elements followed a different sorting when being concatenated into a 1D array)

If the above is accurate as to what was happening.

Yes, correct. Added the comment in parenthesis, it clarifies it further. TY

skoudoro · 2024-06-23T06:33:36Z

@skoudoro, whenever you can, could you check the review request problem?

it should be ok now, Can you confirm ?

Not sure how this looks like when it is deployed since I am unable to find the corresponding URL, but the text looks busy

I agree with @jhlegarreta. Also, one CI's seems to stop to work, that's why we do not have the URL. I need to check what's going on.

Also, this PR is not compiling. It would be great if you could try to compile locally @itellaetxe before requesting review. You will get all the errors early on and the process will be faster.

robinroy03

Hi, please see the comments below.

robinroy03 · 2024-06-23T19:36:50Z

posts/2024/2024_06_21_Inigo_week_4.rst

@@ -0,0 +1,53 @@
+Week 4 into GSoC 2024: Weight transfer experiments, hardships, and results!


We'll need a paragraph break here. (Image from blog.html)

Thanks for the comment @robinroy03, but I am not sure how to add it. Could you give me a hint, please?

@itellaetxe your new commit fixes it. The first paragraph will be highlighted on the blog page.

itellaetxe · 2024-06-24T08:25:13Z

@skoudoro, I can now request reviewers, I added you, @pjsjongsung, and @deka27. Thank you.
Also, I tried to build locally with make -C . clean && make -C . html. The build fails after autodoc fails to import the module numpydoc_test_module. This is the last part of the build log:

Naturally, I don't get any pages under the _build/html directory. Any ideas on this? I found this answer in stackoverflow that might come in handy.

@jhlegarreta, you are right and I remember you had already told me about this, but I did not look into it, sorry. I will correct the blog and add the commit body message. (For reference about the commit body message, I looked at [this answer in stackoverflow].(https://stackoverflow.com/a/36427485)

@robinroy03 thanks for the feedback, correcting it now.

Removes unnecessary line breaks. Groups similar sentences in paragraphs. Adds empty lines for readability.

github-actions · 2024-06-24T09:16:38Z

🪓 PR closed, deleted preview at https://github.com/dipy/preview-html/tree/main/dipy.org/pull/42/

robinroy03

The new commit fixes the issue I had raised. The blog is readable and the links work. Images are rendered properly.

skoudoro

Thanks all for all the review.

week 4 is still a bit dense and needs more empty lines.

However, I will go ahead and merge this PR.

Note: the URL was not appearing because the blog compilation was failing. At the end, everything works almost as expected but when it fails, the CI's should fails. I will try to look in to it this week

thanks @itellaetxe

* DOC: Adds Iñigo's week 3 blogpost * FIX: Corrects punctuation mark * DOC: Adds Iñigo's week 4 blogpost * FIX: Adds whitespaces after titles for readability * FIX: Improves readability of blog Removes unnecessary line breaks. Groups similar sentences in paragraphs. Adds empty lines for readability. 837daed

itellaetxe force-pushed the gsoc2024_inigo branch from 4d6d166 to fa3265f Compare June 14, 2024 12:44

robinroy03 approved these changes Jun 16, 2024

View reviewed changes

jhlegarreta approved these changes Jun 17, 2024

View reviewed changes

itellaetxe added 2 commits June 20, 2024 13:42

DOC: Adds Iñigo's week 3 blogpost

7af7d49

FIX: Corrects punctuation mark

7585dff

itellaetxe force-pushed the gsoc2024_inigo branch from ccfe4b9 to 7585dff Compare June 20, 2024 11:43

DOC: Adds Iñigo's week 4 blogpost

5b4879f

itellaetxe changed the title ~~DOC: Adds Iñigo's week 3 blogpost~~ DOC: Adds Iñigo's week 3 and 4 blogpost Jun 21, 2024

WassCodeur reviewed Jun 21, 2024

View reviewed changes

posts/2024/2024_06_21_Inigo_week_4.rst Show resolved Hide resolved

WassCodeur reviewed Jun 21, 2024

View reviewed changes

WassCodeur approved these changes Jun 21, 2024

View reviewed changes

FIX: Adds whitespaces after titles for readability

c24971c

WassCodeur approved these changes Jun 21, 2024

View reviewed changes

jhlegarreta reviewed Jun 22, 2024

View reviewed changes

robinroy03 suggested changes Jun 23, 2024

View reviewed changes

itellaetxe requested review from skoudoro, pjsjongsung and deka27 June 24, 2024 08:13

FIX: Improves readability of blog

79d32a5

Removes unnecessary line breaks. Groups similar sentences in paragraphs. Adds empty lines for readability.

robinroy03 approved these changes Jun 24, 2024

View reviewed changes

skoudoro approved these changes Jun 24, 2024

View reviewed changes

skoudoro merged commit 837daed into dipy:master Jun 24, 2024
3 checks passed

jhlegarreta mentioned this pull request Jul 1, 2024

DOC: Adds Inigo's week 5 short blogpost #45

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOC: Adds Iñigo's week 3 and 4 blogpost #42

DOC: Adds Iñigo's week 3 and 4 blogpost #42

itellaetxe commented Jun 14, 2024

itellaetxe commented Jun 14, 2024 •

edited

Loading

robinroy03 left a comment

robinroy03 Jun 16, 2024 •

edited

Loading

itellaetxe Jun 17, 2024

deka27 commented Jun 19, 2024

itellaetxe commented Jun 21, 2024

WassCodeur Jun 21, 2024

WassCodeur Jun 21, 2024

itellaetxe Jun 21, 2024

WassCodeur Jun 21, 2024

WassCodeur Jun 21, 2024

jhlegarreta left a comment

jhlegarreta Jun 22, 2024

itellaetxe Jun 24, 2024

jhlegarreta Jun 22, 2024

itellaetxe Jun 24, 2024

jhlegarreta Jun 22, 2024

itellaetxe Jun 24, 2024

jhlegarreta Jun 22, 2024

itellaetxe Jun 24, 2024

jhlegarreta Jun 22, 2024

itellaetxe Jun 24, 2024

skoudoro commented Jun 23, 2024 •

edited

Loading

robinroy03 left a comment

robinroy03 Jun 23, 2024

itellaetxe Jun 24, 2024

robinroy03 Jun 24, 2024

itellaetxe commented Jun 24, 2024 •

edited

Loading

github-actions bot commented Jun 24, 2024 •

edited

Loading

robinroy03 left a comment

skoudoro left a comment

		@@ -0,0 +1,48 @@
		Week 4 into GSoC 2024: Weight transfer experiments, hardships, and results!

		@@ -0,0 +1,53 @@
		Week 4 into GSoC 2024: Weight transfer experiments, hardships, and results!

DOC: Adds Iñigo's week 3 and 4 blogpost #42

DOC: Adds Iñigo's week 3 and 4 blogpost #42

Conversation

itellaetxe commented Jun 14, 2024

itellaetxe commented Jun 14, 2024 • edited Loading

robinroy03 left a comment

Choose a reason for hiding this comment

robinroy03 Jun 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deka27 commented Jun 19, 2024

itellaetxe commented Jun 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jhlegarreta left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

skoudoro commented Jun 23, 2024 • edited Loading

robinroy03 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itellaetxe commented Jun 24, 2024 • edited Loading

github-actions bot commented Jun 24, 2024 • edited Loading

robinroy03 left a comment

Choose a reason for hiding this comment

skoudoro left a comment

Choose a reason for hiding this comment

itellaetxe commented Jun 14, 2024 •

edited

Loading

robinroy03 Jun 16, 2024 •

edited

Loading

skoudoro commented Jun 23, 2024 •

edited

Loading

itellaetxe commented Jun 24, 2024 •

edited

Loading

github-actions bot commented Jun 24, 2024 •

edited

Loading