ENH: Improves blog appearance with custom figure margins and figures …

…for images with caption

posts/2024/2024_08_22_inigo_final_report.rst

@@ -44,6 +44,15 @@
 
    <span class="gsoc-title">Timeline</span>
 
+.. raw:: html  
+
+   <style>  
+   .custom-gsoc-margin {  
+       margin-top: 20px;  
+       margin-bottom: 20px;  
+   }  
+   </style>  
+
 .. image:: /_static/images/logos/gsoc-logo.png
    :height: 40
    :target: https://summerofcode.withgoogle.com/programs/2024/projects/dHajBmW3
@@ -72,8 +81,8 @@
    :category: gsoc
 
 -  **Name:** `Iñigo Tellaetxe Elorriaga <https://github.com/itellaetxe>`_
--  **Organization:** Python Software Foundation
--  **Sub-Organization:** DIPY
+-  **Organization:** `Python Software Foundation <https://www.python.org/psf-landing/>`_
+-  **Sub-Organization:** `DIPY <https://dipy.org>`_
 -  **Project:** `AI/ML in diffusion MRI processing <https://github.com/dipy/dipy/wiki/Google-Summer-of-Code-2024#project-5-project-ideas-using-aiml-in-diffusion-mri-processing>`_
 -  **Repository:** `TractoEncoder GSoC <https://github.com/itellaetxe/tractoencoder_gsoc>`_
 
@@ -113,18 +122,20 @@ The objective of the project is to generate synthetic human tractograms with tun
   Validated the results using the FiberCup dataset. The model is found in the ``ae_model.py`` module of the `TractoEncoder GSoC <https://github.com/itellaetxe/tractoencoder_gsoc>`_ repository. The architecture can be summarized as in the following image:
 
   .. image:: /_static/images/gsoc/2024/inigo/inigo_vanilla_autoencoder.png
+    :class: custom-gsoc-margin
     :alt: AE architecture diagram.
     :align: center
-    :width: 800
+    :width: 600
 
   * Weight and bias initializers are different in PyTorch compared to TensorFlow2, so the PyTorch behavior was replicated using custom initializers. Different weight & bias initialization strategies can lead to drastically different training results, so this step took extra care.
   * The upsampling layers of the Decoder block use linear interpolation in PyTorch by default, whereas in TensorFlow2, there is no native implementation for this, and nearest neighbor (NN) interpolation is used instead. This is a significant difference in implementations, and to replicate the PyTorch behavior a custom linear interpolating upsampling layer was implemented in TF2. However, after training with both NN and linear interpolation, the results were very similar, so the custom layer was not used in the final implementation. This work was developed in a `separate branch <https://github.com/itellaetxe/tractoencoder_gsoc/tree/feature_linear_upsampling>`_.
   * Training was run for 120 epochs, using a data set containing plausible and implausible streamlines. All the training experiments in my GSoC work were done with this data set. The figure below shows a summary of the replication results, which consist of running a set of unseen plausible streamlines through the model (encoder and decoder):
 
-    .. image:: /_static/images/gsoc/2024/inigo/fibercup_replicated.png
+    .. figure:: /_static/images/gsoc/2024/inigo/fibercup_replicated.png
+      :class: custom-gsoc-margin
       :alt: Replication of the FINTA architecture results on the FiberCup dataset.
       :align: center
-      :width: 800
+      :width: 600
 
     We see that the Keras result is very similar to the PyTorch result, though the fiber crossings are not as well resolved in the Keras result. This is not a thing to worry since this is not systematic, as seen in other sets of unseen data.
 
@@ -136,15 +147,17 @@ The objective of the project is to generate synthetic human tractograms with tun
   * The model is found in the ``vae_model.py`` module of the `TractoEncoder GSoC <https://github.com/itellaetxe/tractoencoder_gsoc>`_ repository. The architecture can be summarized in the figure below:
 
   .. image:: /_static/images/gsoc/2024/inigo/inigo_variational_autoencoder.png
+    :class: custom-gsoc-margin
     :alt: VAE architecture diagram.
     :align: center
-    :width: 800
+    :width: 600
 
   * It was necessary to implement Batch Normalization after the Encoder convolutional layers and exponential operator clipping, to prevent gradient explosion while training the model. The figure above displays that the encoder outputs the variance of the latent space vector (:math:`\sigma^2`), though it is a common practice to output the log-variance instead. This leads to using :math:`\sigma = e^{\frac{1}{2}\text{log}{\sigma^2}}`, which can get numerically quite unstable as it can shoot up very quickly when a sufficiently big log-variance is input. This modification allowed stable training and it was a major contribution to the robustness of the architecture. Shout out to my lab colleague `Jorge <https://github.com/jgarciacondado>`_ for the ideas and discussions on the topic.
   * Weighing the Kullback-Leibler loss component was also implemented, based on the `Beta-VAE <https://openreview.net/forum?id=Sy2fzU9gl>`_ work, aiming for a stronger disentanglement of the latent space. However, this parameter was never explored (it was always set to 1.0) due to its trade-off with the reconstruction accuracy, but it is a potential improvement for future work in the context of hyperparameter optimization.
   * The figure below shows a summary of the VAE results, also using the FiberCup dataset:
 
-    .. image:: /_static/images/gsoc/2024/inigo/vanilla_vae_120_epoch_results.png
+    .. figure:: /_static/images/gsoc/2024/inigo/vanilla_vae_120_epoch_results.png
+      :class: custom-gsoc-margin
       :alt: VAE architecture results on the FiberCup dataset.
       :align: center
       :width: 600
@@ -157,16 +170,18 @@ The objective of the project is to generate synthetic human tractograms with tun
   * The model is found in the ``cond_vae_model.py`` module of the `TractoEncoder GSoC <https://github.com/itellaetxe/tractoencoder_gsoc>`_ repository. The model was trained on the FiberCup dataset, and the conditioning variable in this case was chosen to be the length of the streamlines, hypothesizing that this is a relatively simple feature to capture by the model based on their geometry. The majority of the architecture is based on the VAE from the previous point (also based on the `FINTA <https://doi.org/10.1016/j.media.2021.102126>`_ architecture), to which I added two dense layers to output the :math:`\sigma_r` and :math:`\text{log}\sigma^2_r` of the regressed attribute, as well as the *generator* block. A diagram of the architecture can be seen below:
 
     .. image:: /_static/images/gsoc/2024/inigo/conditional_vae_architecture_diagram.png
+      :class: custom-gsoc-margin
       :alt: condVAE architecture diagram.
       :align: center
-      :width: 800
+      :width: 600
 
 
 * **Implemented validation strategies of the condVAE model** to check that the model can capture the variability of the conditioning variable.
 
   * By exploring the latent space of the VAE and condVAE models, we can compare the organization of the samples in the latent space, and see whether there is a difference aligned with the conditioning variable. After training for 64 epochs just to check how the model was progressing, I projected the 32-dimensional latent space using the t-SNE algorithm, to visualize it easily. This particular algorithm was chosen due to its popularity, speed, and availability in widespread libraries like `scikit-learn`. The projections only show the plausible fibers The results are shown in the figures below:
 
-    .. image:: /_static/images/gsoc/2024/inigo/latent_space_comparison_VAE_cVAE_colored_by_streamline_length.png
+    .. figure:: /_static/images/gsoc/2024/inigo/latent_space_comparison_VAE_cVAE_colored_by_streamline_length.png
+      :class: custom-gsoc-margin
       :alt: t-SNE latent space comparison between condVAE and VAE models.
       :align: center
       :width: 600
@@ -175,18 +190,20 @@ The objective of the project is to generate synthetic human tractograms with tun
 
   * Another aspect to validate was the capability of the model to correctly capture the conditioning variable in the training data. To do so, we retrained the model until we got a close-to-zero *label loss* (in charge of capturing this variability), and computed the :math:`MSE` and the :math:`R^2` metrics between the predicted and the true conditioning variable. In addition, we plotted the latent space 2D projection again. The results are shown in the figure below:
 
-    .. image:: /_static/images/gsoc/2024/inigo/vae_conditioning_validation.png
+    .. figure:: /_static/images/gsoc/2024/inigo/vae_conditioning_validation.png
+      :class: custom-gsoc-margin
       :alt: condVAE conditioning validation results.
       :align: center
       :width: 600
 
     We got an :math:`\text{MSE}=50.371` and an :math:`R^2=0.941`, and the latent space was organized in 7 clusters (the middle line is actually 2 lines, there is a small separation in the middle), which correspond to the 7 bundles of the FiberCup dataset, thus we saw that the model was indeed capable of predicting the conditioning variable fairly well. From left to right, the streamlines get shorter (the color gets lighter) with a gradient in the latent space that is aligned with the conditioning variable. The morphology of the leftmost clusters would indicate that they are the most variable in length, contrary to the middle clusters (the straighter lines), which are almost in the same horizontal coordinate. Having the t-SNE projection aligned with the conditioning variable is a good sign, that some dimension in the latent space was correctly tied to the conditioning variable.
 
-  * As the last experiment to validate the capabilities of this model and architecture, I proceeded to generate synthetic streamlines conditioned on their length. Theoretically, the model should be able to generate samples of specific lengths, and each length should be related to a specific bundle, as shorter fibers look different than the longer ones. Nonetheless, there was a major flaw in the generative process that my mentor `Jong Sung Park <https://github.com/pjsjongsung>`_ had detected just before going into this step. When looking at the architecture of the model, the generative process would start from the :math:`r` variable, setting it to a desired quantity, and then running the input through ``D5`` and ``D6``, then adding noise like :math:`z=\mu_{p_z}+\sigma_{p_z}\odot\epsilon`, and decoding this output. 
+  * As the last experiment to validate the capabilities of this model and architecture, I proceeded to generate synthetic streamlines conditioned on their length. Theoretically, the model should be able to generate samples of specific lengths, and each length should be related to a specific bundle, as shorter fibers look different than the longer ones. Nonetheless, there was a major flaw in the generative process that my mentor `Jong Sung <https://github.com/pjsjongsung>`_ had detected just before going into this step. When looking at the architecture of the model, the generative process would start from the :math:`r` variable, setting it to a desired quantity, and then running the input through ``D5`` and ``D6``, then adding noise like :math:`z=\mu_{p_z}+\sigma_{p_z}\odot\epsilon`, and decoding this output. 
 
   * Since the generator block is trying to predict the whole latent vector :math:`z` from a single number (:math:`r`), the model was going to probably have trouble getting the necessary geometrical variability, and overall problems for generating different things from the same number. The model was not designed for generating samples and rather to regress their associated conditioning variable, and this was a major issue that had not been foreseen. This was a good lesson to learn, and it was a good opportunity to think about the importance of the generative process in the model design, and how it should be aligned with the model's objectives. The figure below shows a set of generated samples of lengths 30 and 300 (left and right), seeing that the model was generating a very constant shape, only scaled in length:
 
     .. image:: /_static/images/gsoc/2024/inigo/streamlines_short_long.png
+      :class: custom-gsoc-margin
       :alt: condVAE conditioning validation results.
       :align: center
       :width: 600
@@ -204,6 +221,7 @@ The objective of the project is to generate synthetic human tractograms with tun
     * The model is found in the ``adv_ae_model.py`` module of the `TractoEncoder GSoC <https://github.com/itellaetxe/tractoencoder_gsoc>`_ repository. The proposed architecture can be summarized in the figure below:`
 
       .. image:: /_static/images/gsoc/2024/inigo/adversarial_ae_with_abr.png
+        :class: custom-gsoc-margin
         :alt: condVAE conditioning validation results.
         :align: center
         :width: 600
@@ -233,7 +251,7 @@ Apart from my project, I got to contribute to the DIPY project in other ways too
 
 * I opened a PR that got merged into the main DIPY repository, which was a `nomenclature issue with spherical harmonics <https://github.com/dipy/dipy/issues/2970>`_. It took some time to agree on how to solve it, but it was very nice to see that the community was open to discussing the issue and to find a solution that was good for everyone. This was the contribution that gave me access to GSoC with DIPY, and it was a very nice start to the journey. Link to the PR: https://github.com/dipy/dipy/pull/3086
 * I reviewed the code of my fellow GSoC students `Kaustav <https://github.com/deka27>`_, `Wachiou <https://github.com/WassCodeur>`_ and `Robin <https://github.com/robinroy03>`_. It felt very good to understand their projects and to be able to help them with their work, which is completely different from my project. I also reviewed their blogs and participated in the reviews I got from them. It was very pleasant to see how engaging the community is, and how everyone is willing to help each other.
-* Lastly, I opened an `issue in the dipy.org repository <https://github.com/dipy/dipy.org/issues/40>`_ that got solved thanks to a contribution from my mentor `Serge Koudoro <https://github.com/skoudoro>`_.
+* Lastly, I opened an `issue in the dipy.org repository <https://github.com/dipy/dipy.org/issues/40>`_ that got solved thanks to a contribution from my mentor `Serge <https://github.com/skoudoro>`_.
 
 |future-work-title|
 -------------------
@@ -256,7 +274,7 @@ Because the work is still in progress, no integration of the final version was d
 
 I am super grateful for the opportunity to participate in the Google Summer of Code program with DIPY. This journey has been amazing as learning and research experience. I am proud of the progress I have made and what I have achieved. Nevertheless, this is just the beginning of my journey with this project.
 
-I would like to thank again my mentors, Jon Haitz, Jong Sung and Serge, their time, patience, attention, support, and expertise throughout the program. They have been a key part of my success. On the other hand, I would like to thank my fellow GSoC students, Kaustav, Wachiou, and Robin, for their support and collaboration. It has been a pleasure to work with them and to learn from them.
+I would like to thank again my mentors, `Jon Haitz <https://github.com/jhlegarreta>`_, `Jong Sung <https://github.com/pjsjongsung>`_ and `Serge <https://github.com/skoudoro>`_, their time, patience, attention, support, and expertise throughout the program. They have been a key part of my success. On the other hand, I would like to thank my fellow GSoC students, `Kaustav <https://github.com/deka27>`_, `Wachiou <https://github.com/WassCodeur>`_, and `Robin <https://github.com/robinroy03>`_, for their support and collaboration. It has been a pleasure to work with them and to learn from them.
 
 
 |timeline-title|