Table of Contents

• DREAMPlaceFPGA
  • FPGA Placement: Overview
  • Global Placer
  • Packer-Legalizer
  • Performance
  • Target Architecture
• Developers
• Publications
• External Dependencies
• Cloning the Repository
• Build Instructions
  • To install Python dependency
  • To install with Docker
  • To Build
  • Cmake Options
• Sample Benchmarks
• Running DREAMPlaceFPGA
• JSON Configurations
• Bug Report
• Copyright

DREAMPlaceFPGA

DREAMPlaceFPGA is an Open-Source GPU-Accelerated Placer for Large Scale Heterogeneous FPGAs using a Deep Learning Toolkit.

FPGA Placement: An Overview

Placement is a crucial and computationally intensive step in the FPGA design flow that determines the physical locations of various heterogeneous instances in the design. In general, placement consists of three stages - global placement (GP), packing/clustering and legalization (LG), and detailed placement (DP).

• Stages in an FPGA Placement

With a synthesized logic-mapped design netlist and the FPGA architecture description as inputs, the global placer obtains roughly legal locations for all the design instances. Based on the global placement (GP) solution, the packer clusters the FFs and LUTs to be placed on the corresponding sites. Then the legalizer assigns all the instances to their corresponding physical sites on the FPGA to obtain a legal placement. A placement is legal when an instance occupies a site of the same type, and all instances can be routed with the limited routed resources available on an FPGA. The disruptive movement of instances to the fixed physical site locations during legalization results in quality degradation, and thus detailed placement (DP) further refines the legalized placement.

Among the various placement stages, the global placement and pack-legalize stages are accelerated in DREAMPlaceFPGA.

Global Placer

By leveraging the open-source ASIC placement framework, DREAMPlace, we build an open-source placement framework for FPGAs that is based on the elfPlace algorithm. The global placement flow in DREAMPlaceFPGA:

The operators for wirelength, density, and instance area update are accelerated on a GPU.

Packer-Legalizer

Starting with a flat global placement solution, the direct legalization (DL) algorithm allows for clustering (or) packing of LUTs and FFs, followed by legalization to their respective sites. The packing-legalization flow in DREAMPlaceFPGA:

DREAMPlaceFPGA enhances the DL algorithm and accelerates it on a GPU.

Performance

DREAMPlaceFPGA outperforms elfPlace (GPU) by 19% for global placementruntime. On the ISPD'2016 benchmark suite, DREAMPlaceFPGA is 5.3× faster for global placement, 2.2× faster for packing-legalization and 2.4× faster for overall placement than 16-thread elfPlace (CPU), with a slight increase in (+0.6%) placement HPWL and (+0.9%) routed wirelength. For more details, please refer to the 'publications'.

The runtime results vary based on the hardware used. The above mentioned results are based on a Linux machine with an Intel i9-7900 CPU (running at 3.30 GHz) and an NVIDIA Titan Xp (Pascal) GPU.

Target Architecture

Support for the following architectures are available

Simplified Xilinx Ultrascale Architecture

• The ISPD'2016 benchmarks, which employs a simplified Xilinx Ultrascale architecture, can be run on DREAMPlaceFPGA.
• Interchange Format (IF) support is not available for this architecture.

Ultrascale+ Architecture

• Limited support is available for Ultrascale+ architecture GNL designs through the Interchange Format (IF). Please refer to IF_README for more details.

The elfPlace (CPU) binary is available to run the legalization and detailed placement stages, when DREAMPlaceFPGA is used to only run the global placement stage. DREAMPlaceFPGA runs on both CPU and GPU. If installed on a machine without GPU, multi-threaded CPU support is available.

Developers

• Rachel Selina Rajarathnam, UTDA, ECE Department, The University of Texas at Austin
• Zixuan Jiang, UTDA, ECE Department, The University of Texas at Austin
• Zhili Xiong, UTDA, ECE Department, The University of Texas at Austin

Publications

• Rachel Selina Rajarathnam, Mohamed Baker Alawieh, Zixuan Jiang, Mahesh A. Iyer, and David Z. Pan, "DREAMPlaceFPGA: An Open-Source Analytical Placer for Large Scale Heterogeneous FPGAs using Deep-Learning Toolkit", 27th IEEE/ACM Asian and South Pacific Design Automation Conference (ASP-DAC), 2022.
• Rachel Selina Rajarathnam, Zixuan Jiang, Mahesh A. Iyer, and David Z. Pan, "DREAMPlaceFPGA-PL: An Open-Source GPU-Accelerated Packer-Legalizer for Heterogeneous FPGAs", ACM International Symposium on Physical Design (ISPD), 2023.

External Dependencies

• Python 2.7 or Python 3.5/3.6/3.7

• CMake version 3.8.2 or later

• Pytorch 1.6/1.7/1.8

  • Other 1.0 versions may also work, but not tested.
  • Version 2.0 or higher is not compatible!

• GCC

  • Recommend GCC 5.1 or later.
  • Other compilers may also work, but not tested.

• cmdline

  • a command line parser for C++

• Flex

  • lexical analyzer employed in the bookshelf parser

• Bison

  • parser generator employed in the bookshelf parser

• Boost

  • Need to install and visible for linking

• Limbo

  • Integrated as a submodule: the bookshelf parser is modified for FPGAs.

• Flute

  • Integrated as a submodule

• CUB

  • Integrated as a git submodule

• munkres-cpp

  • Integrated as a git submodule

• CUDA 9.1 or later (Optional)

  • If installed and found, GPU acceleration will be enabled.
  • Otherwise, only CPU implementation is enabled.

• GPU architecture compatibility 6.0 or later (Optional)

  • Code has been tested on GPUs with compute compatibility 6.0, 7.0, and 7.5.
  • Please check the compatibility of the GPU devices.
  • The default compilation target is compatibility 6.0. This is the minimum requirement and lower compatibility is not supported for the GPU feature.

• Cairo (Optional)

  • If installed and found, the plotting functions will be faster by using C/C++ implementation.
  • Otherwise, python implementation is used.

• pycapnp

  • Required Python bindings for Interchange Format (IF) support.

Cloning the Repository

To pull git submodules in the root directory

git submodule init
git submodule update

Or alternatively, pull all the submodules when cloning the repository.

git clone --recursive https://github.com/rachelselinar/DREAMPlaceFPGA.git

Build Instructions

To install Python dependency

There is an aternative way to install Dreamplace using docker, if you want to use the docker, skip this step and goes to the next step, "To install with Docker"

At the root directory:

pip install -r requirements.txt 

For example, if the repository was cloned in directory ~/Downloads, then the root directory is ~/Downloads/DREAMPlaceFPGA

You can also use a python virtual environment to install all the required packages to run DREAMPlaceFPGA

To install with Docker

You can use the Docker container to avoid building all the dependencies yourself.

1. Install Docker on Linux.(Win and Mac are not tested)

2. To enable the GPU features, install NVIDIA-docker; otherwise, skip this step.

3. Get the docker image using one of the options Build the image locally

   docker build . --file Dockerfile --tag <username>/dreamplace_fpga:1.0
   

   replace <username> with a username, for instance 'utda_macro_placer'.

4. Enter bash environment of the container. Mount the repo and all the Designs into the Docker, which allows the Docker container to directly access and modify these files

   To run on a Linux machine without GPU:

   docker run -it -v $(pwd):/DREAMPlaceFPGA <username>/dreamplace_fpga:1.0 bash
   

   To run on a Linux machine with GPU: (Docker verified on NVIDIA GPUs with compute capability 6.1, 7.5, and 8.0)

   docker run --gpus 1 -it -v $(pwd):/DREAMPlaceFPGA <username>/dreamplace_fpga:1.0 bash
   

   For example to run on a Linux machine without GPU:

   docker run -it -v $(pwd):/DREAMPlaceFPGA utda_macro_placer/dreamplace_fpga:1.0 bash
   

5. Go to the DREAMPlaceFPGA directory in the Docker, which is the root directory of the project

   cd /DREAMPlaceFPGA_mlcad
   

To Build

At the root directory,

mkdir build 
cd build 
cmake .. -DCMAKE_INSTALL_PREFIX=path_to_root_dir
make
make install

if you are using the docker, using the following at the root directory,

rm -rf build
mkdir build 
cd build 
cmake .. -DCMAKE_INSTALL_PREFIX=/DREAMPlaceFPGA_mlcad -DPYTHON_EXECUTABLE=$(which python)
make
make install

Third party submodules are automatically built except for Boost.

For example,

~/Downloads/DREAMPlaceFPGA: mkdir build; cd build

~/Downloads/DREAMPlaceFPGA/build: cmake . . -DCMAKE_INSTALL_PREFIX=~/Downloads/DREAMPlaceFPGA

~/Downloads/DREAMPlaceFPGA/build: make; make install

The directory ~/Downloads/DREAMPlaceFPGA/build is the install dir

When there are changes to packages or parser code, it is necessary to delete contents of build directory for a clean build and proper operation.

rm -r build

For example,

~/Downloads/DREAMPlaceFPGA: rm -r build

Cmake Options

Here are the available options for CMake.

• CMAKE_INSTALL_PREFIX: installation or root directory
  • Example cmake -DCMAKE_INSTALL_PREFIX=path/to/root/directory
• CMAKE_CUDA_FLAGS: custom string for NVCC (default -gencode=arch=compute_60,code=sm_60)
  • Example cmake -DCMAKE_CUDA_FLAGS=-gencode=arch=compute_60,code=sm_60
• CMAKE_CXX_ABI: 0|1 for the value of _GLIBCXX_USE_CXX11_ABI for C++ compiler, default is 0.
  • Example cmake -DCMAKE_CXX_ABI=0
  • It must be consistent with the _GLIBCXX_USE_CXX11_ABI for compling all the C++ dependencies, such as Boost and PyTorch.
  • PyTorch in default is compiled with _GLIBCXX_USE_CXX11_ABI=0, but in a customized PyTorch environment, it might be compiled with _GLIBCXX_USE_CXX11_ABI=1.

Sample Benchmarks

DREAMPlaceFPGA requires IO instances to be fixed and only accepts inputs in Bookshelf format.

• 4 sample benchmarks for Xilinx Ultrascale Architecture in bookshelf format with fixed IOs are provided. Refer to ISPD'2016 contest for more information.
• 10 GNL designs based on Xilinx Ultrascale+ Architecture are provided. These benchmarks need to be converted from Interchange Format (IF) to bookshelf format before running DREAMPlaceFPGA. Refer to IF_README for more details.

The sample benchmarks and designs can be found in the benchmarks directory.

Running DREAMPlaceFPGA

Before running, ensure that all python dependent packages have been installed. Go to the root directory and run with JSON configuration file.

python dreamplacefpga/Placer.py <benchmark>.json

Run from ~/Downloads/DREAMPlaceFPGA directory

For example:

python dreamplacefpga/Placer.py test/FPGA-example1.json

~/Downloads/DREAMPlaceFPGA: python dreamplacefpga/Placer.py test/FPGA-example1.json

If you are not using the GPU, change the gpu flag in the *.json file to 0.

Unit tests for some of the pytorch operators are provided. For instance, to run unit test for hpwl, use the below command:

python unitest/ops/hpwl_unitest.py

Note: If your machine does not have an NVIDIA GPU, set the 'gpu' flag in JSON configuration file to '0' to run on CPU.

JSON Configurations

The most frequently used options in the JSON file are listed below. For the complete list of available options, please refer to paramsFPGA.json.

JSON Parameter	Default	Description
aux_input	required for bookshelf	input .aux file
gpu	1	enable GPU acceleration or run on CPU
num_threads	8	number of CPU threads
num_bins_x	512	number of bins in horizontal direction
num_bins_y	512	number of bins in vertical direction
global_place_stages	required	global placement configuration of each stage, a dictionary of {"num_bins_x", "num_bins_y", "iteration", "learning_rate"}, learning_rate is relative to bin size. Note: The maximum number of iterations can be set using "iteration" and default value is 2000.
routability_opt_flag	0	Set to enable routability optimization. Note: If this flag is not set for large designs, it makes legalization very difficult.
gamma	5.0	initial coefficient for log-sum-exp and weighted-average wirelength
random_seed	1000	random seed
scale_factor	0.0	scale factor to avoid numerical overflow; 0.0 means not set
result_dir	results	result directory for output
global_place_flag	1	whether to run global placement
legalize_flag	1	whether to run legalization on DREAMPlaceFPGA else legalization and detailed placement are run using elfPlace
dtype	float32	data type, float32 (or) float64
plot_flag	0	whether to plot solution or not (Increases runtime)
deterministic_flag	1	Ensures reproducible run-to-run results (Slightly increases runtime)

Bug Report

Please report bugs to rachelselina dot r at utexas dot edu.

Copyright

This software is released under BSD 3-Clause "New" or "Revised" License. Please refer to LICENSE for details.