Restructuring: new folder structure which is more accessible (Simulation-Software-Engineering#67)

* New folder structure

* Add draft for new README in 00_organization

* Tabular format for chapter README

* Restructuring building_and_packaging chapter

* Correct README in building_and_packaging

* Rectifying table in building_and_packaging

* overview --> README for version control chapter

* Renaming vm containers exercise file

* Remvoing demos from README and making separate files

* Updating PDF creation workflow depending on new file names

* Moving git basics in a files _demo.md

* Updating markdown creation list

* Updating main README

* Further corrections to files

* Removing packaging keyword from all file names

* Separating git trick demo

* Splitting git and containers demo into smaller parts

* Modifying links in the timetable

* Add GitHub action to check links

* Remove cron job schedule fron check-links action

* Removing build/ directory from CMake example

* Removing debian from file name

* Consistency in intro file names

* Remove code added unintentionally

* Remove CMakeLists.txt from exercise

* Adding documentation chapter to new structure

* Changing to linkchecker GitHub action from SSE/homepage repo

* Modifying linkchecker action so that it finds all markdown files in the repo

* Adding more options to awesome_bot command in linkchecker Action

Author: IshaanDesai

.github/workflows/create-pdfs-from-markdown.yml

@@ -26,14 +26,6 @@ jobs:
           name: slides
           path: slides
           retention-days: 7
-      - name: Compile PDFs (texts)
-        run: |
-          ./scripts/create-pdf-from-markdown.sh text
-      - uses: actions/upload-artifact@master
-        with:
-          name: texts
-          path: texts
-          retention-days: 7
       - name: Compile PDFs (quizzes)
         run: |
           ./scripts/create-pdf-from-markdown.sh quiz
@@ -50,9 +42,9 @@ jobs:
           name: texts
           path: texts
           retention-days: 7
-      - name: Compile PDFs (notes)
+      - name: Compile PDFs (demos)
         run: |
-          ./scripts/create-pdf-from-markdown.sh notes
+          ./scripts/create-pdf-from-markdown.sh demo
       - uses: actions/upload-artifact@master
         with:
           name: texts

.github/workflows/linkchecker.yml

@@ -0,0 +1,24 @@
+name: Link checker
+
+on:
+  push:
+    branches: [ '*' ]
+  pull_request:
+    branches: [ '*' ]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Ruby 2.6
+      uses: actions/setup-ruby@v1
+      with:
+        ruby-version: 2.6.x
+    - name: Check links
+      #FIXME Need to allow duplicates (same links on the same page)
+      run: |
+        gem install awesome_bot
+        awesome_bot --white-list https://gitlab-sim.informatik.uni-stuttgart.de/  --allow-ssl --allow-redirect --allow-dupe $(find . -iname "*.md" -type f)

00_organization/README.md

@@ -0,0 +1,15 @@
+# Organization and Introduction to Research Software Engineering
+
+Learning goals:
+
+- Lecture, lab organization, and the challenge
+- What is research software engineering and what is special about it?
+- Why do we need research software engineering?
+
+| Duration | Content |
+| --- | --- |
+| 20 minutes | [`challenge_intro_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/00_organization/challenge_intro_slides.md) |
+| 40 minutes | [`course_intro_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/00_organization/course_intro_slides.md) |
+| 10 minutes | [`organizational_remarks_week3_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/00_organization/organizational_remarks_week3_slides.md) |
+| 5 minutes | [`organizational_remarks_week7_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/00_organization/organizational_remarks_week7_slides.md) |
+| 15 minutes | [`rse_basics.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/organization/material/rse_basics_slides.md) |

organization/material/challenge_intro_slides.md 00_organization/challenge_intro_slides.md

@@ -0,0 +1,19 @@
+# Git -- the standard version control system
+
+Learning goals:
+
+- Refresh and organize students' existing knowledge on git (learn how to learn more).
+- Students can explain difference between merge and rebase and when to use what.
+- How to use git workflows to organize research software development in a team.
+- Get to know a few useful GitHub/GitLab standards and a few helpful tools.
+
+| Duration | Content |
+| --- | --- |
+| 75 minutes | [`cheat_sheet_exercise.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/01_version_control/cheat_sheet_exercise.md) |
+| 30 minutes | [`git_quiz.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/01_version_control/git_quiz.md) |
+| 40 minutes | [`git_basics_demo.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/01_version_control/git_basics_demo.md)
+| 30-45 minutes| [`my_favorite_neat_little_git_trick_demo.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/01_version_control/my_favorite_neat_little_git_trick_demo.md) |
+| 15 minutes | [`intro_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/01_version_control/intro_slides.md) |
+| 25 minutes | [`merge_rebase_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/01_version_control/merge_rebase_slides.md) |
+| 10 minutes | [`standards_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/01_version_control/standards_slides.md) |
+| 20 minutes | [`workflow_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/01_version_control/workflow_slides.md) |

organization/material/intro_course_slides.md 00_organization/course_intro_slides.md

@@ -0,0 +1,102 @@
+# Git Demo
+
+## Recap of Git basics
+
+- Expert level poll on git: ask students to estimate their level.
+    - Beginner: I have hardly ever used Git
+    - User: pull, commit, push, status, diff
+    - Developer: fork, branch, merge, checkout
+    - Maintainer: rebase, squash, cherry-pick, bisect
+    - Owner: submodules
+
+![git overview picture from py-rse](https://merely-useful.tech/py-rse/figures/git-cmdline/git-remote.png)
+
+- `git --help`, `git commit --help`
+- incomplete statement `git comm`
+
+- There is a difference between Git and hosting services ([*forges*](https://en.wikipedia.org/wiki/Forge_(software)))
+    - [GitHub](https://github.com/)
+    - [GitLab](https://about.gitlab.com/), open-source, hosted e.g. at [IPVS](https://gitlab-sim.informatik.uni-stuttgart.de)
+    - [Bitbucket](https://bitbucket.org/product/)
+    - [SourceForge](https://sourceforge.net/)
+    - many more
+    - often, more than just hosting, also DevOps
+
+- Give outlook on remainder of Git chapter: *How I work with Git*, quiz, advanced topics (workflows, rebase, standards), *my neat little Git trick*
+
+## How I work with Git
+
+Starting remarks:
+
+- There is not *the one solution* how to do things with Git. I simply show what I typically use.
+- Don't use a client if you don't understand the command line `git`
+
+- (1) Look at GitHub
+    - [preCICE repository](https://github.com/precice/precice)
+    - default branch `develop`
+    - fork -> my fork
+
+- (2) Working directory:
+    - ZSH shell shows git branches
+    - `git remote -v` (I have upstream, myfork, ...)
+    - mention difference between ssh and https (also see GitHub)
+    - get newest changes `git pull upstream develop`
+    - `git log` -> I use special format, see `~/.gitconfig`,
+    - check log on GitHub; explain short hash
+    - `git branch`
+    - `git branch add-demo-feature`
+    - `git checkout add-demo-feature`
+
+- (3) First commit
+    - `git status` -> always tells you what you can do
+    - `vi src/action/Action.hpp` -> add `#include "MagicHeader.hpp"`
+    - `git diff`, `git diff src/com/Action.hpp`, `git diff --color-words`
+    - `git status`, `git add`, `git status`
+    - `git commit` -> "Include MagicHeader in Action.hpp"
+    - `git status`, `git log`, `git log -p`, `git show`
+
+- (4) Change or revert things
+    - I forgot to add sth: `git reset --soft HEAD~1`, `git status`
+    - `git diff`, `git diff HEAD` because already staged
+    - `git log`
+    - `git commit`
+    - actually all that is nonsense: `git reset --hard HEAD~1`
+    - modify again, all nonsense before committing: `git checkout src/action/Action.hpp`
+
+- (5) Stash
+    - while working on unfinished feature, I need to change / test this other thing quickly, too lazy for commits / branches
+    - `git stash`
+    - `git stash pop`
+
+- (6) Create PR
+    - create commit again
+    - preview what will be in PR: `git diff develop..add-demo-feature`
+    - `git push -u myfork add-demo-feature` -> copy link
+    - explain PR template
+    - explain target branch
+    - explain "Allow edits by maintainers"
+    - cancel
+    - my fork -> branches -> delete
+
+- (7) Check out someone else's work
+    - have a look at an existing PR, look at all tabs, show suggestion feature
+    - but sometimes we want to really build and try sth out ...
+    - `git remote -v`
+    - `git remote add alex [email protected]:ajaust/precice.git` if I don't have remote already (or somebody else)
+    - `git fetch alex`
+    - `git checkout -t alex/[branch-name]`
+    - I could now also push to `ajaust`'s remote
+
+## Further reading
+
+### Quick things
+
+- [Video: Git in 15 minutes: basics, branching, no remote](https://www.youtube.com/watch?v=USjZcfj8yxE)
+- [The GitHub Blog: Commits are snapshots, not diffs](https://github.blog/2020-12-17-commits-are-snapshots-not-diffs/)
+- Chapters [6](https://merely-useful.tech/py-rse/git-cmdline.html) and [7](https://merely-useful.tech/py-rse/git-advanced.html) of Research Software Engineering with Python
+- [Podcast All Things Git: History of VC](https://www.allthingsgit.com/episodes/the_history_of_vc_with_eric_sink.html)
+- [git purr](https://girliemac.com/blog/2017/12/26/git-purr/)
+
+### References
+
+- [Official documentation](http://git-scm.com/doc)

organization/material/organizational_remarks_week3_slides.md 00_organization/organizational_remarks_week3_slides.md

@@ -0,0 +1,22 @@
+# My favorite neat little Git trick
+
+- Students present in 3-5 mins their favorite *neat little Git trick* (a tool, a command, a configuration, a GitHub thing, ...).
+- Preparation should not take longer than 15 mins, should be a demo.
+- Lecturers also prepare some.
+
+Collected examples:
+
+- Click on line number on GitHub and then `blame` to study history of file
+- `git reflog` to resurrect accidentally deleted branches
+- GitHub uses [gitignore templates](https://github.com/github/gitignore) if you create a `.gitignore` file from GitHub
+- [GitExplorer](https://gitexplorer.com/): a nice way to generate important git command
+- Shortcut `t` to search project files on GitHub
+- Shortcut `ctrl` + `k` to never use mouse on GitHub again
+- `git effort` (from `git-extras`repository): Show effort statistics on file(s) in repository
+- `git commit -m "Title of commit" -d "Description of commit` to specify a commit message and description without going to vim
+- Use [aheadfork](https://github.com/mbomb007/aheadfork) to find GitHub forks that are ahead
+- Use [permanent links](https://docs.github.com/en/github/writing-on-github/working-with-advanced-formatting/creating-a-permanent-link-to-a-code-snippet#linking-to-code) on GitHub to refer to and display code snippets in issues
+- Coding in the cloud with GitHub [Codespaces](https://github.com/features/codespaces) and VS Code: type `.` in a repository to jump into the editor
+- One can create empty commits, i.e. a commit without any changes, using the `--allow-empty` options. This can be useful for debugging, for example. The full command could look like this: `git commit --allow-empty -m 'This commit is empty'`
+- You can generate a list of contributors sorted by the number of contributions using `git shortlog --summary --numbered --email.
+- If you have some contributors with several different email addresse and/or configured names, you can merge them via aliases in a file called [`.mailmap`](https://git-scm.com/docs/gitmailmap) that should reside in the root of the repository.

organization/material/organizational_remarks_week7_slides.md 00_organization/organizational_remarks_week7_slides.md

@@ -0,0 +1,22 @@
+# Virtualization and Containers
+
+Learning goals:
+
+- What is the difference between virtualization and containers?
+- When to use virtual machines and when containers.
+- How to work with virtual machines (VirtualBox) and how to manage these with Vagrant.
+- Building containers with Docker and Singularity.
+- Understand pros and cons of different container technologies.
+- Student can set up their own containers tailored to their requirements.
+
+| Duration | Content |
+| --- | --- |
+| 10 minutes | [`containers_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/container_slides.md), [`containers_demo.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/container_demo.md) |
+| 10 minutes | [`containers_homework_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/containers_homework_slides.md) |
+| 65 minutes | [`docker_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/docker_slides.md), [`docker_demo.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/docker_demo.md) |
+| 5 minutes | [`intro_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/intro_slides.md) |
+| 20 minutes | [`singularity_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/singularity_slides.md), [`singularity_demo.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/singularity_demo.md) |
+| 25 minutes | [`vagrant_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/vagrant_slides.md) |
+| 40 minutes | [`virtualbox_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/virtualbox_slides.md), [`virtualbox_demo.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/virtualbox_demo.md) |
+| 10 minutes | [`virtualmachines_slides.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/virtualmachines_slides.md), [`virtualmachines_demo.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/virtualmachines_demo.md) |
+| 90 minutes | [`virtualmachines_containers_exercise.md`](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/02_virtualization_and_containers/virtualmachines_containers_exercise.md) |

organization/material/rse_basics_slides.md 00_organization/rse_basics_slides.md

@@ -0,0 +1,60 @@
+# Containers Demo
+
+## Introduction to Containers
+
+- What are the differences between VMs and containers?
+    - Containers
+        - Low(er) overhead than virtual machines.
+        - Container operates in "fenced off" part of the operating system (`cgroups`).
+        - Container runs kernel of the host OS -> Does **not** run its own OS in.
+        - Operating system (OS) needs to be compatible with underlying OS. Cannot run different OS than host. (TODO: Verify this)
+            - **Note**: Windows 10 can run Linux containers! (Due to Windows Subsystem for Linux?!)
+        - A process for which (and its childs) special rules apply.
+- Shortly recap what we have learned about containers.
+    - Fenced-off, relies on capabilities of OS etc.
+- LXD/LXC and its container registry [Linux containers](https://linuxcontainers.org/)
+- Short and incomplete overview over container technologies: Docker, Singularity, lxc/lxd, podman...
+
+## Concluding Remarks and Discussion about Learning Goals
+
+- Virtual machines are a good abstraction layer that are very flexible. They can run different OS, can be moved etc. VirtualBox is a popular VM solution (hypervisor.)
+- There are tools like Vagrant to manage them more conveniently than with the GUI or CLI.
+- Containers are a light weight alternative to VMs. They are basically "special" processes that run more or less isolated. Therefore, they do not need to bring everything (kernel, libraries...) themselves, but rely on the functionalities of the Host. Example: Containers "run" on the same kernel as the Host. Cannot mix OSes (easily).
+- Docker is popular for container framework encapsulating environments and applications. We will use it also more for testing (CI/CD, DevOps). This will come in future lectures.
+- Singularity is a bit more "niche", but important in the computing/simulation business as it focuses on self-contained applications that can easily run on different platforms instead of isolation compared to Docker.
+- For the exercise you should ideally have Docker, VirtualBox, Vagrant (maybe Singularity installed).
+- Show slides from the intro with learning goals again.
+    - Can you answer the questions now?
+
+## Further Reading
+
+### References
+
+- [Docker documentation](https://docs.docker.com/)
+- [Singularity documentation](https://sylabs.io/docs/)
+
+### Virtualization tools
+
+- [VirtualBox](https://www.virtualbox.org/)
+- [VirtualBox Manual](https://www.virtualbox.org/manual/UserManual.html)
+- [Overview of different disk formats](https://www.parallels.com/blogs/ras/vdi-vs-vhd-vs-vmdk/)
+- [Ubuntu 18.04 virtual machine setup](https://codebots.com/docs/ubuntu-18-04-virtual-machine-setup)
+- [Vagrant](https://www.vagrantup.com/)
+
+### Containers
+
+- [Docker](https://www.docker.com/)
+- [Docker Hub](https://hub.docker.com/)
+- [Singularity](https://sylabs.io/)
+- [Sarus](https://user.cscs.ch/tools/containers/sarus/)
+- [lxc/lxd](https://linuxcontainers.org/)
+- [podman](https://podman.io/)
+- [Linux containers](https://linuxcontainers.org/)
+- Singularity paper: [Singularity: Scientific containers for mobility of compute](https://doi.org/10.1371/journal.pone.0177459)
+
+### Other
+
+- ["Should I use Vagrant or Docker for creating an isolated environment?"](https://stackoverflow.com/questions/16647069/should-i-use-vagrant-or-docker-for-creating-an-isolated-environment)
+- [Malicious Docker Hub Container Images Used for Cryptocurrency Mining](https://www.trendmicro.com/vinfo/fr/security/news/virtualization-and-cloud/malicious-docker-hub-container-images-cryptocurrency-mining)
+- ["How To Make Package Managers Cry"](https://archive.fosdem.org/2018/schedule/event/how_to_make_package_managers_cry/)
+- [Open Container Initiative (OCI)](https://opencontainers.org/)