Skip to content

Commit

Permalink
First commit
Browse files Browse the repository at this point in the history
  • Loading branch information
dantegd committed Apr 22, 2020
0 parents commit 5bd002a
Show file tree
Hide file tree
Showing 46 changed files with 7,224 additions and 0 deletions.
27 changes: 27 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#cpp code owners
cpp/ @rapidsai/cuml-cpp-codeowners
cpp/ @rapidsai/cugraph-cpp-codeowners

#python code owners
python/ @rapidsai/cuml-python-codeowners
python/ @rapidsai/cugraph-python-codeowners

#cmake code owners
**/CMakeLists.txt @rapidsai/cuml-cmake-codeowners
**/CMakeLists.txt @rapidsai/cugraph-cmake-codeowners
**/cmake/ @rapidsai/cuml-cmake-codeowners
**/cmake/ @rapidsai/cugraph-cmake-codeowners
python/setup.py @rapidsai/cuml-cmake-codeowners
python/setup.py @rapidsai/cugraph-cmake-codeowners
build.sh @rapidsai/cuml-cmake-codeowners
build.sh @rapidsai/cugraph-cmake-codeowners
**/build.sh @rapidsai/cuml-cmake-codeowners
**/build.sh @rapidsai/cugraph-cmake-codeowners

#build/ops code owners
.github/ @rapidsai/ops-codeowners
ci/ @rapidsai/ops-codeowners
conda/ @rapidsai/ops-codeowners
**/Dockerfile @rapidsai/ops-codeowners
**/.dockerignore @rapidsai/ops-codeowners
docker/ @rapidsai/ops-codeowners
26 changes: 26 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
name: Bug report
about: Create a bug report to help us improve RAFT
title: "[BUG]"
labels: "? - Needs Triage, bug"
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.

**Steps/Code to reproduce bug**
Follow this guide http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports to craft a minimal bug report. This helps us reproduce the issue you're having and resolve the issue more quickly.

**Expected behavior**
A clear and concise description of what you expected to happen.

**Environment details (please complete the following information):**
- Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)]
- Method of RAFT install: [conda, Docker, or from source]
- If method of install is [Docker], provide `docker pull` & `docker run` commands used


**Additional context**
Add any other context about the problem here.
35 changes: 35 additions & 0 deletions .github/ISSUE_TEMPLATE/documentation-request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
name: Documentation request
about: Report incorrect or needed documentation
title: "[DOC]"
labels: "? - Needs Triage, doc"
assignees: ''

---

## Report incorrect documentation

**Location of incorrect documentation**
Provide links and line numbers if applicable.

**Describe the problems or issues found in the documentation**
A clear and concise description of what you found to be incorrect.

**Steps taken to verify documentation is incorrect**
List any steps you have taken:

**Suggested fix for documentation**
Detail proposed changes to fix the documentation if you have any.

---

## Report needed documentation

**Report needed documentation**
A clear and concise description of what documentation you believe it is needed and why.

**Describe the documentation you'd like**
A clear and concise description of what you want to happen.

**Steps taken to search for needed documentation**
List any steps you have taken:
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for RAFT
title: "[FEA]"
labels: "? - Needs Triage, feature request"
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I wish I could use RAFT to do [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context, code examples, or references to existing implementations about the feature request here.
10 changes: 10 additions & 0 deletions .github/ISSUE_TEMPLATE/submit-question.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
name: Submit question
about: Ask a general question about RAFT
title: "[QST]"
labels: "? - Needs Triage, question"
assignees: ''

---

**What is your question?**
44 changes: 44 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
<!--
Thank you for contributing to RAFT :)
Here are some guidelines to help the review process go smoothly.
1. Please write a description in this text box of the changes that are being
made.
2. Please ensure that you have written units tests for the changes made/features
added.
3. If you are closing an issue please use one of the automatic closing words as
noted here: https://help.github.com/articles/closing-issues-using-keywords/
4. If your pull request is not ready for review but you want to make use of the
continuous integration testing facilities please label it with `[WIP]`.
5. If your pull request is ready to be reviewed without requiring additional
work on top of it, then remove the `[WIP]` label (if present) and replace
it with `[REVIEW]`. If assistance is required to complete the functionality,
for example when the C/C++ code of a feature is complete but Python bindings
are still required, then add the label `[HELP-REQ]` so that others can triage
and assist. The additional changes then can be implemented on top of the
same PR. If the assistance is done by members of the rapidsAI team, then no
additional actions are required by the creator of the original PR for this,
otherwise the original author of the PR needs to give permission to the
person(s) assisting to commit to their personal fork of the project. If that
doesn't happen then a new PR based on the code of the original PR can be
opened by the person assisting, which then will be the PR that will be
merged.
6. Once all work has been done and review has taken place please do not add
features or make changes out of the scope of those requested by the reviewer
(doing this just add delays as already reviewed code ends up having to be
re-reviewed/it is hard to tell what is new etc!). Further, please do not
rebase your branch on master/force push/rewrite history, doing any of these
causes the context of any comments made by reviewers to be lost. If
conflicts occur against master they should be resolved by merging master
into the branch used for making the pull request.
Many thanks in advance for your cooperation!
-->
46 changes: 46 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
## common
__pycache__
*.pyc
*~
\#*
.#*
*.o
*.so
*.dylib
.cache
.coverage
.vscode
*.swp
*.pytest_cache
htmlcov
build/
build_prims/
dist/
python/**/**/*.cpp
python/external_repositories
python/record.txt
log
.ipynb_checkpoints
.DS_Store
dask-worker-space/
## eclipse
.project
.cproject
.settings
.ptp-sync-folder

## Pycharm
.idea

## ccls
.ccls-cache
.ccls

## profiling
*.qdrep
*.qdrep.cache
*.qdstrm
*.nvprof

## doxygen build check inside ci/checks/style.sh
doxygen_check/
152 changes: 152 additions & 0 deletions BUILD.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
# RAFT Build and Development Guide

- [Usage of RAFT by downstream projects](usage-of-raft-by-downstream-projects)
- [C++ Integration](c-integration)
- [Python/Cython Integration](pythoncython-integration)
- [Building and running tests](building-and-running-tests)
- [CI Process](ci-process)
- [Development Guide](developer-guide)
- [Local Development](local-development)
- [Submitting PRs](submitting-prs)



## Usage of RAFT by downstream projects

### C++ Integration

The C++ component of RAFT is header only, so it can be easily configured using CMake by consuming libraries. Since this repo is intended to be included by downstream repos, the recommended way of accomplishing that is using CMake's git cloning functionality:


```cmake
if(DEFINED ENV{RAFT_PATH})
message(STATUS "RAFT_PATH environment variable detected.")
message(STATUS "RAFT_DIR set to $ENV{RAFT_PATH}")
set(RAFT_DIR ENV{RAFT_PATH})
else(DEFINED ENV{RAFT_PATH})
message(STATUS "RAFT_PATH environment variable NOT detected, cloning RAFT")
set(RAFT_GIT_DIR ${CMAKE_CURRENT_BINARY_DIR}/raft CACHE STRING "Path to RAFT repo")
ExternalProject_Add(raft
GIT_REPOSITORY [email protected]:dantegd/barge.git
GIT_TAG pinned_commit/git_tag/branch
PREFIX ${RAFT_GIT_DIR}
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND "")
set(RAFT_INCLUDE_DIR ${RAFT_DIR}/src/raft/cpp/include CACHE STRING "RAFT include variable")
endif(DEFINED ENV{RAFT_PATH})
```

This create the variable `$RAFT_INCLUDE_DIR` variable that can be used in `include_directories`, and then the related header files can be included when needed.

### Python/Cython Integration

The Python and Cython code have been designed to be included in projects that use RAFT, as opposed to a distributable by itself. To use:

- The file `setuputils.py` is included in RAFT's `python` folder. Copy the file to your repo, in a location where it can be imported by `setup.py`
- In your setup.py, use the function `use_raft_package`, for example for cuML:


```python
# Optional location of C++ build folder that can be configured by the user
libcuml_path = get_environment_option('CUML_BUILD_PATH')
# Optional location of RAFT that can be confugred by the user
raft_path = get_environment_option('RAFT_PATH')

use_raft_package(raft_path, libcuml_path)
```

The usage of RAFT by the consuming repo's python code follows the rules:
1. If the environment variable `RAFT_PATH` points to the RAFT repo, then that will be used.
2. If there is a C++ build folder that has cloned RAFT already, setup.py will use that RAFT.
3. If none of the above happened, then setup.py will clone RAFT and use it directly.

- After `setup.py` calls the `use_raft_package` function, the RAFT python code will be included (via a symlink) in the consuming repo package, under a raft subfolder. So for example, `cuml` python package includes RAFT in `cuml.raft`.


## Building and running tests

Since RAFT is not meant to create any artifact on itself, but be included in other projects, the build infrastructure is focused only on testing.

The base folder in the repository contains a `build.sh` script that builds both the C++ and Python code, which is the recommended way of building the tests.

To run C++ tests:

```bash
./test_raft
```

To run Python tests:

```bash
cd python
python -m pytest
```

To build manually, you can also use `CMake` and setup.py directly. For C++:

```bash
cd cpp
mkdir build
cd build
cmake ..
```

There is no `install` target currently.

For python:

```bash
cd python
python setup.py build_ext --inplace
```


## CI Process

PRs submitted to RAFT will always run the RAFT tests (once GPUCI is enabled). Additionally, RAFT has convenience functionality to run tests of the following projects that use RAFT: cuML and cuGraph.

To run these other tests, turn `ON` the variables in `ci/prtest.config` in your PR:

```bash
RUN_CUGRAPH_LIBCUGRAPH_TESTS=OFF
RUN_CUGRAPH_PYTHON_TESTS=OFF

RUN_CUML_LIBCUML_TESTS=OFF
RUN_CUML_PRIMS_TESTS=OFF
RUN_CUML_PYTHON_TESTS=OFF
```

This will make it so that CI in the PR will clone and build the respective repository, but the repository **will be built using the fork/branch of RAFT in the PR**. This allows to test changes in RAFT without the need of opening PRs in the other repositories.

Before merging the PR, those variables need to be returned to `OFF`.


## Developer Guide

### Local Development

To help working with RAFT and consuming projects as seamless as possible, this section describes how a typical workflow looks like and gives some guidelines for developers working in projects that affect code in both RAFT and at least one downstream repository.

Using as an example developer working on cuML and RAFT, we recommend the following:

- Create two working folders: one containing the cloned cuML repository and the other the cloned RAFT one.
- Create environment variable `RAFT_PATH` pointing to the location of the RAFT path.
- Work on same named branches in both repos/folders.

This will facilitate development, and the `RAFT_PATH` variable will make it so that the downstream repository, in this case cuML, builds using the locally cloned RAFT (as descrbed in the first step).

### Submitting PRs

If you are submitting changes to RAFT itself, without changing downstream repos, you can use the config file located in `ci/prtest.config` to trigger RAFT's CI to run tests of downstream repositories.

If you have changes to both RAFT and at least one downstream repo, then:

- It is recommended to open a PR to both repositories (for visibility and CI tests).
- Change the pinned branch/commit in the downstream repo PR to point to the fork and branch used for the RAFT PR to make CI run tests
- If your changes might affect usage of RAFT by other downnstream repos, alert reviewers and open a github issue or PR in that downstream repo as approproate.
- The PR to RAFT will be merged first, so that the downstream repo PR pinned branch/commit can be returned to the main RAFT branch and run CI with it.
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# RAFT 0.14.0 (Date TBD)


## New Features
- Initial RAFT version

## Improvements


## Bug Fixes
Loading

0 comments on commit 5bd002a

Please sign in to comment.