Merge pull request #10 from t4d-gmbh/5-git-lfs

5 git lfs

source/content/ci_cd_for_repro/slide1.md

@@ -0,0 +1 @@
+## title

source/content/exercise/slide1.md

@@ -0,0 +1 @@
+## title

source/content/intro/slide1.md

@@ -0,0 +1 @@
+## title

source/content/lfs/comparison-git-gitlfs.md

@@ -0,0 +1,36 @@
+## <i class="fab fa-git"></i> vs. <i class="fab fa-git"></i> LFS: A Comparison
+
+{% if build == "pages" %}
+
+### <i class="fab fa-git"></i> 
+
+When you commit changes with <i class="fab fa-git"></i>, it creates objects to represent the state of your files at that point in time. 
+These objects are stored in the `.git/objects` directory.
+Each object is a snapshot of the file contents, and <i class="fab fa-git"></i>  uses pointers (hashes) to reference these objects.
+
+<i class="fab fa-git"></i>  is optimized for text files.
+Text files often have small, incremental changes, that <i class="fab fa-git"></i> can handle well by storing only the differences (deltas) between versions.
+
+Changes in binary files (e.g., images, videos, datasets) are different from text files because changes are not easily represented as deltas.
+When a binary file changes, <i class="fab fa-git"></i>  often stores the entire file again, leading to repository bloat and slow performance.
+
+### <i class="fab fa-git"></i> LFS
+
+<i class="fab fa-git"></i> LFS replaces large files with small pointer files. 
+These pointer files reference the actual content stored outside the main repository.
+
+The actual large files are stored in a seperate location (e.g., a remote server) which keeps the main repository lightweight and efficient.
+When you clone a repository with <i class="fab fa-git"></i> LFS, you only download the pointer files, not the actual large files.
+When you check out a file, <i class="fab fa-git"></i> LFS automatically downloads the large file content.
+Similarly, when you commit a large file, <i class="fab fa-git"></i> uploads the large file to the external storage and replaces it with a pointer file in the repository.
+
+{% else %}
+
+| <i class="fab fa-git"></i>  | <i class="fab fa-git"></i> LFS |
+|:---:|:-------:|
+| Optimized for text files | Optimized for large files |
+| Stores entire file contents | Stores large files externally |
+| Slows down with large files | Maintains repository performance |
+| Bloated repositories | Lightweight repositories |
+
+{% endif %}

source/content/lfs/gitLFS_UZH.md

@@ -0,0 +1,30 @@
+## <i class="fab fa-git"></i> LFS Availability at UZH
+
+{% if build == "pages" %}
+Popular <i class="fab fa-git"></i> hosting services like GitHub, GitLab, Bitbucket, and Azure DevOps have built-in support for <i class="fab fa-git"></i> LFS.
+
+In self-hosted <i class="fab fa-git"></i> server, it must be ensured that they have <i class="fab fa-git"></i> LFS support enabled which might come with installing and configuring additional software.
+
+This is because <i class="fab fa-git"></i> LFS stores large files in a separate storage location, which requires additional server-side support to manage the large files (e.g., storage, authentication, bandwidth, etc.).
+
+At the IMATH, <i class="fab fa-git"></i> LFS is NOT supported on the [IMATH GitLab](https://gitlab.imath.uzh.ch) instance.
+
+At the University of Zurich (UZH), <i class="fab fa-git"></i> LFS is supported on the [UZH GitLab](https://gitlab.uzh.ch) instance.
+There is a **size limit of 15 GB per project** (this includes all parts of a project, i.e. <i class="fab fa-git"></i> repository, LFS, etc.). 
+The data is stored in the [Switch Cloud](https://www.switch.ch/en/competencies/cloud), which means outside the UZH but within Switzerland (generally the UZH data protection regulations apply).
+
+Please note that GitLab is generally a place for collaborative software development rather than simply data storage. 
+For example, UZH has limited disk space and deletes all data after 12 months of user inactivity. 
+For long-term data storage, it is recommended to use [OneDrive](https://uzh-my.sharepoint.com/my) or [SwitchDrive](https://drive.switch.ch/).
+
+{% else %}
+
+- <i class="fab fa-git"></i> hosting services like GitHub, GitLab, and Bitbucket support <i class="fab fa-git"></i> LFS.
+- Self-hosted <i class="fab fa-git"></i> servers need additional configuration for <i class="fab fa-git"></i> LFS.
+- <i class="fab fa-git"></i> LFS is _not_ supported on the [IMATH GitLab instance](https://git.math.uzh.ch/).
+- <i class="fab fa-git"></i> LFS is supported on the [UZH GitLab instance](https://gitlab.uzh.ch/).
+  - **Size Limit**: 15 GB per project.
+  - **Storage**: [Switch Cloud](https://www.switch.ch/en/competencies/cloud) (Switzerland).
+  - **Data Retention**: Data is deleted after 12 months of inactivity.
+
+{% endif %}

source/content/lfs/how-to-use-git-lfs.md

@@ -0,0 +1,65 @@
+## How to use <i class="fab fa-git"></i> LFS
+
+{% if build == "pages" %}
+
+To use <i class="fab fa-git"></i> LFS, you need to install the <i class="fab fa-git"></i> LFS client on your local machine.
+
+### 1. Install Git LFS:
+   - Download and install from [git-lfs.github.com](https://git-lfs.github.com).
+   - Initialize in your repository by running the following command in your repository directory:
+     ```bash
+     git lfs install
+     ```
+
+### 2. Track Large Files:
+   - Specify the file types to be tracked by <i class="fab fa-git"></i> LFS. For example, to track all `.pdf` files, run:
+     ```bash
+     git lfs track "*.pdf"
+     ```
+
+### 3. Commit and Push:
+
+- Add the `.gitattributes` file:
+
+  The `.gitattributes` file is a configuration file used by <i class="fab fa-git"></i> to manage attributes for paths in your repository.
+  It allows you to define how certain files should be handled by <i class="fab fa-git"></i>, including aspects like line endings, mark files as binary to prevent text-based operations on them, and more. 
+  This file is particularly useful when working with <i class="fab fa-git"></i> LFS to specify which files should be managed by LFS.
+
+  ```bash
+  git add .gitattributes
+  ```
+
+  Example: 
+  If you want to use Git LFS for large media files like images and videos, your .gitattributes file might look like this:
+
+  ```bash
+  # Use LFS for image files
+  *.jpg filter=lfs diff=lfs merge=lfs -text
+  *.png filter=lfs diff=lfs merge=lfs -text
+
+  # Use LFS for video files
+  *.mp4 filter=lfs diff=lfs merge=lfs -text
+  ```
+
+- Add, commit, and push large files:
+  ```bash
+  git add <large-file>
+  git commit -m "Add large file"
+  git push
+  ```
+
+{% else %}
+
+- Install Git LFS from [git-lfs.github.com](https://git-lfs.github.com) and initialize it with `git lfs install`.
+- Track large files by specifying file types, e.g., `git lfs track "*.pdf"`.
+- Add the `.gitattributes` file to manage file attributes and specify LFS management for certain files.
+    Example `.gitattributes` entries for images and videos:
+    ```bash
+    *.jpg filter=lfs diff=lfs merge=lfs -text
+    *.png filter=lfs diff=lfs merge=lfs -text
+    *.mp4 filter=lfs diff=lfs merge=lfs -text
+    ```
+- Add, commit, and push large files using Git LFS.
+
+{% endif %}
+

source/content/lfs/index.md

@@ -1,15 +1,24 @@
-# <i class="fab fa-git"></i> LFS
+# <i class="fab fa-git"></i> Large File System (LFS) 
 {% if slide %}
 <!-- BUILDING THE SLIDES -->
 ```{toctree}
 :maxdepth: 2
 
-./slide1
-
+./why-use-git-lfs.md
+./comparison-git-gitlfs.md
+./how-to-use-git-lfs.md
+./gitLFS_UZH.md
 ```
+
 {% else %}
 <!-- BUILDING THE PAGES -->
 <!-- build the page content here -->
-```{include} ./slide1.md
+```{include} ./why-use-git-lfs.md
+```
+```{include} ./comparison-git-gitlfs.md
+```
+```{include} ./how-to-use-git-lfs.md
+```
+```{include} ./gitLFS_UZH.md
 ```
 {% endif %}

source/content/lfs/why-use-git-lfs.md

@@ -0,0 +1,20 @@
+## Why Using <i class="fab fa-git"></i> LFS?
+
+{% if build == "pages" %}
+Data management is a critical aspect of scientific research and is often not versioned or tracked leading to data loss, duplication, or errors.
+
+Tracking large files with <i class="fab fa-git"></i> can be challenging. 
+As the size of your repository grows, it can slows down the <i class="fab fa-git"></i> operations, such as cloning, fetching, and pushing because <i class="fab fa-git"></i> stores the entire history of the repository locally.
+To address this issue, <i class="fab fa-git"></i> Large File Storage (LFS) provides a solution for managing large files in your <i class="fab fa-git"></i> repositories.
+
+Git Large File Storage (LFS) is a <i class="fab fa-git"></i> extension that replaces large files in your repository with text pointers while storing the file contents on a remote server. 
+This approach allows you to work with large files in your repository without slowing down the <i class="fab fa-git"></i> operations.
+
+{% else %}
+
+- **Data Management**: Proper data management is crucial in scientific research to prevent data loss, duplication, and errors.
+- **Efficiency**: Traditional <i class="fab fa-git"></i>  struggles with large files, leading to slow performance and bloated repositories.
+- **Collaboration**: Ensures team members can work with large files without conflicts or performance issues.
+- **History Tracking**: Maintains a history of changes, making it easier to revert to previous versions if needed.
+
+{% endif %}

source/content/project_mgmt_tools/slide1.md

@@ -0,0 +1 @@
+## title

source/content/submodules/slide1.md

@@ -0,0 +1 @@
+## title

source/content/versioning_vs_reproducibility/slide1.md

@@ -0,0 +1 @@
+## title