Cloning sources

Contents

1. About
2. Prepare source code directory
3. Create Python virtual environment
4. Install reana-dev helper script
5. Fork all repositories
6. Clone all repositories
7. Next steps

About

REANA uses github.com/reanahub organisation. The source code is organised in many different repositories. In order to do full-scale REANA development, we should fork all the repositories and clone them locally. This page explains how to facilitate this work.

Prepare source code directory

We start by preparing a new directory that will hold the sources. We'll assume ~/project/reana/src:

$ mkdir -p ~/project/reana/src
$ cd ~/project/reana/src

Create Python virtual environment

REANA comes with a reana-dev helper package for developers that automatise some of the processes across many different repositories. We can use it to facilitate forking sources as well.

We shall therefore start by preparing a Python virtual environment where we shall be working. We'll call it reana.

It is important to select an appropriate Python version to use. When developing REANA and using live code reloading and debugging features, it comes handy to use the same Python version locally as the one used in the REANA cluster components inside the containers. At the time of this writing, it is Python version 3.12. (Note that you can verify reana-server's Dockerfile to see whether this is still the current version.) Using the same version will help to ensure that any *.pyc or .eggs packages will be the same when mounting local file system into the containers, or that we compile proper container package dependencies when using pip-compile outside of the container. We therefore specify Python 3.12 as the Python version to use for the reana virtual environment.

To create a virtual environment named reana with Python 3.12, follow these steps:

Step 1: Install virtualenvwrapper

$ pip install virtualenvwrapper

Step 2: Configure virtualenvwrapper

Find the location of virtualenvwrapper.sh and add it to your shell configuration file (~/.bashrc or ~/.zshrc). You can use the which command to locate it:

$ which virtualenvwrapper.sh

Once you have the path to virtualenvwrapper.sh, add the following line to your shell configuration file and make sure to run it or open a new terminal to apply the changes:

$ source /path/to/virtualenvwrapper.sh

Replace /path/to/virtualenvwrapper.sh with the actual path you obtained from the which command.

Step 3: Create the reana virtual environment with Python 3.12

Now, you can create the reana virtual environment with Python 3.12 using mkvirtualenv:

$ mkvirtualenv reana -p python3.12
(reana) $ python -V

You should see output similar to this, indicating that Python 3.12 is in use:

Python 3.12.4

Now you have a Python virtual environment named "reana" with Python 3.12 set up and ready to use.

Install reana-dev helper script

Let's install the reana-dev helper script into the newly created environment:

(reana) $ pip install git+https://github.com/reanahub/reana.git#egg=reana

In order to simplify the authentication with GitHub, we set up SSH keys for password-less access. We can run ssh-agent manually:

(reana) $ eval "$(ssh-agent -s)"
(reana) $ ssh-add ~/.ssh/id_rsa

Fork all repositories

We now use reana-dev helper script to assist in forking all repositories. Assuming our GitHub user name is johndoe, the REANA convention uses upstream for github.com/reanahub/some-package and origin for its github.com/johndoe/some-package fork.

For instance, you can run the following one-liner to open browser tabs in Firefox for the manual forking. (You can pass your desired browser via -b argument, or simply use open for macOS.)

(reana) $ eval "$(reana-dev git-fork -c ALL -b firefox)"

Another option, if you have installed the gh command-line tool for GitHub, is to fork repositories fully automatically without any manual intervention:

(reana) $ reana-dev git-fork -c ALL --automatic

Clone all repositories

Now that you have forked all REANA repositories into your personal GitHub space, we can clone them locally using another reana-dev helper script command:

(reana) $ reana-dev git-clone -c ALL -u johndoe

This command clones the repositories from your personal fork and sets up upstream and origin branches accordingly.

If you later want to sync your forks with the origin, you can run:

reana-dev git-upgrade -c ALL

This will fetch the upstream, merge, and push to your fork.

Next steps

We now have ~/project/reana/src directory populated with all the cloned source code repositories. We can proceed to building and deploying REANA cluster following Using production-like development mode.