Migrate Processes across Different Architectures

This page shows how to migrate a running process from x86-64 to aarch64 once the entire setup is in place. The steps demonstrated here are the same for both emulated as well as bare-metal environments.

Directory Structure

To migrate a process, we need to make sure that the ELF images are present on both x86-64 and aarch64 machines in the exact same folder structure. For example if we want to spawn and migrate the bt benchmark from SNU_NPB the folder structure should look like this in both x86-64 and aarch64 side:

.
├── bin
│   ├── bt_aarch64
│   └── bt_x86-64
└── bt

The folder containing the binary bt should contain another directory called bin. This directory should contain both bt_x86-64 and bt_aarch64. The file bt in the outermost directory should be a copy of the x86-64 binary on x86-64 machine and aarch64 binary on aarch64 machine.

SNU-NPB benchmarks

In order to create this directory structure:

1. Navigate to Transproc directory with the command

cd /path/to/TransProc/test/SNU_NPB_SER_C

2. On the x86-64 host, run the script setup_x86.sh but simply running

./setup_x86.sh

3. On the aarch64 host, run the script setup_arm.sh but simply running

./setup_arm.sh

This will create the aforementioned directory structure in your home folder.

There are 4 main steps binary migration:

• Spawn : Spawn a process and suspend it at a migration point
• Dump : Criu dump the process
• Recode : Transform the process images to be restored on destination
• Copy : Copy the criu artifacts to destination
• Restore : Criu restore the process on destination

Once the folder structure is in place, we are now ready to spawn an application and do live heterogeneous migration. We support two types of benchmarks, SNU-NPB benchmarks do not run within a loop and hence need to be manually held at a migration point. Redis runs within a loop and can be scheduled to be held at migration point on demand.

Spawn

This example uses bt as the benchmark that gets live migrated, bt can be replaced with any SNU-NPB benchmarks. On the x86-64 server, first navigate to the bt directory in your home directory.

cd ~/bt/

This directory should contain a Makefile with recipes that will help in live migrating the process. To launch bt with the TransProc monitor run:

make spawn

Dump

Once the process is spawned under the TransProc monitor, it will be suspended at a migration point. To dump it we have another Makefile target it place. You will need another terminal to run the following command from the process' directory context:

make dump

This command will dump the process.

Transform

Now the process is ready to be transformed such that it can be restored on the aarch64 server. Run the following command to transform the images:

make transform

Scp

The transformed files are now ready to be copied to the aarch64 server. To do this run:

make scp

Restore

In a terminal on the aarch64 server, navigate to the bt directory inside your home directory with the command:

cd ~/bt/

Now run the following command to restore the process:

make restore

Redis

For redis, first create the redis directory structure by navigating to /path/to/TransProc/test/redis and run the script: ./setup-x86.sh and x86-64 the server and ./setup-arm.sh on the arm server.

Now navigate to the redis-server directory in your home directory with

cd ~/redis-server

Next, run redis on x86-64:

./redis-server

Trap

To trap the process stay in the directory context of redis-server from another terminal and run:

make attach

Dump

Once the process is suspended at a migration point dump it:

make dump

Transform

This is the step that transforms the criu binaries such that they can be restored on aarch64. Run:

make transform

Copy

This step involves copying over the transformed images to destination. Copy the images to the same directory in the destination. Run:

make scp

Restore

The files get copied to the exact same directory structure on the aarch64 server. On the aarch64 server, navigate to redis-server directory with:

cd ~/redis-server

Now run:

make restore

This will spawn the migrated process.

NOTE:

Please note that the Makefiles contain two variables a)TRANSPROC and b)DEST which need to be changed. $TRANSPROC needs to be changed to the path to TransProc directory on the servers where heterogeneous migration is to be reproduced and $DEST needs to be changed to the IP address of the aarch64 server.