-
Notifications
You must be signed in to change notification settings - Fork 203
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit 5bd002a
Showing
46 changed files
with
7,224 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
#cpp code owners | ||
cpp/ @rapidsai/cuml-cpp-codeowners | ||
cpp/ @rapidsai/cugraph-cpp-codeowners | ||
|
||
#python code owners | ||
python/ @rapidsai/cuml-python-codeowners | ||
python/ @rapidsai/cugraph-python-codeowners | ||
|
||
#cmake code owners | ||
**/CMakeLists.txt @rapidsai/cuml-cmake-codeowners | ||
**/CMakeLists.txt @rapidsai/cugraph-cmake-codeowners | ||
**/cmake/ @rapidsai/cuml-cmake-codeowners | ||
**/cmake/ @rapidsai/cugraph-cmake-codeowners | ||
python/setup.py @rapidsai/cuml-cmake-codeowners | ||
python/setup.py @rapidsai/cugraph-cmake-codeowners | ||
build.sh @rapidsai/cuml-cmake-codeowners | ||
build.sh @rapidsai/cugraph-cmake-codeowners | ||
**/build.sh @rapidsai/cuml-cmake-codeowners | ||
**/build.sh @rapidsai/cugraph-cmake-codeowners | ||
|
||
#build/ops code owners | ||
.github/ @rapidsai/ops-codeowners | ||
ci/ @rapidsai/ops-codeowners | ||
conda/ @rapidsai/ops-codeowners | ||
**/Dockerfile @rapidsai/ops-codeowners | ||
**/.dockerignore @rapidsai/ops-codeowners | ||
docker/ @rapidsai/ops-codeowners |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
--- | ||
name: Bug report | ||
about: Create a bug report to help us improve RAFT | ||
title: "[BUG]" | ||
labels: "? - Needs Triage, bug" | ||
assignees: '' | ||
|
||
--- | ||
|
||
**Describe the bug** | ||
A clear and concise description of what the bug is. | ||
|
||
**Steps/Code to reproduce bug** | ||
Follow this guide http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports to craft a minimal bug report. This helps us reproduce the issue you're having and resolve the issue more quickly. | ||
|
||
**Expected behavior** | ||
A clear and concise description of what you expected to happen. | ||
|
||
**Environment details (please complete the following information):** | ||
- Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)] | ||
- Method of RAFT install: [conda, Docker, or from source] | ||
- If method of install is [Docker], provide `docker pull` & `docker run` commands used | ||
|
||
|
||
**Additional context** | ||
Add any other context about the problem here. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
--- | ||
name: Documentation request | ||
about: Report incorrect or needed documentation | ||
title: "[DOC]" | ||
labels: "? - Needs Triage, doc" | ||
assignees: '' | ||
|
||
--- | ||
|
||
## Report incorrect documentation | ||
|
||
**Location of incorrect documentation** | ||
Provide links and line numbers if applicable. | ||
|
||
**Describe the problems or issues found in the documentation** | ||
A clear and concise description of what you found to be incorrect. | ||
|
||
**Steps taken to verify documentation is incorrect** | ||
List any steps you have taken: | ||
|
||
**Suggested fix for documentation** | ||
Detail proposed changes to fix the documentation if you have any. | ||
|
||
--- | ||
|
||
## Report needed documentation | ||
|
||
**Report needed documentation** | ||
A clear and concise description of what documentation you believe it is needed and why. | ||
|
||
**Describe the documentation you'd like** | ||
A clear and concise description of what you want to happen. | ||
|
||
**Steps taken to search for needed documentation** | ||
List any steps you have taken: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
--- | ||
name: Feature request | ||
about: Suggest an idea for RAFT | ||
title: "[FEA]" | ||
labels: "? - Needs Triage, feature request" | ||
assignees: '' | ||
|
||
--- | ||
|
||
**Is your feature request related to a problem? Please describe.** | ||
A clear and concise description of what the problem is. Ex. I wish I could use RAFT to do [...] | ||
|
||
**Describe the solution you'd like** | ||
A clear and concise description of what you want to happen. | ||
|
||
**Describe alternatives you've considered** | ||
A clear and concise description of any alternative solutions or features you've considered. | ||
|
||
**Additional context** | ||
Add any other context, code examples, or references to existing implementations about the feature request here. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
--- | ||
name: Submit question | ||
about: Ask a general question about RAFT | ||
title: "[QST]" | ||
labels: "? - Needs Triage, question" | ||
assignees: '' | ||
|
||
--- | ||
|
||
**What is your question?** |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
<!-- | ||
Thank you for contributing to RAFT :) | ||
Here are some guidelines to help the review process go smoothly. | ||
1. Please write a description in this text box of the changes that are being | ||
made. | ||
2. Please ensure that you have written units tests for the changes made/features | ||
added. | ||
3. If you are closing an issue please use one of the automatic closing words as | ||
noted here: https://help.github.com/articles/closing-issues-using-keywords/ | ||
4. If your pull request is not ready for review but you want to make use of the | ||
continuous integration testing facilities please label it with `[WIP]`. | ||
5. If your pull request is ready to be reviewed without requiring additional | ||
work on top of it, then remove the `[WIP]` label (if present) and replace | ||
it with `[REVIEW]`. If assistance is required to complete the functionality, | ||
for example when the C/C++ code of a feature is complete but Python bindings | ||
are still required, then add the label `[HELP-REQ]` so that others can triage | ||
and assist. The additional changes then can be implemented on top of the | ||
same PR. If the assistance is done by members of the rapidsAI team, then no | ||
additional actions are required by the creator of the original PR for this, | ||
otherwise the original author of the PR needs to give permission to the | ||
person(s) assisting to commit to their personal fork of the project. If that | ||
doesn't happen then a new PR based on the code of the original PR can be | ||
opened by the person assisting, which then will be the PR that will be | ||
merged. | ||
6. Once all work has been done and review has taken place please do not add | ||
features or make changes out of the scope of those requested by the reviewer | ||
(doing this just add delays as already reviewed code ends up having to be | ||
re-reviewed/it is hard to tell what is new etc!). Further, please do not | ||
rebase your branch on master/force push/rewrite history, doing any of these | ||
causes the context of any comments made by reviewers to be lost. If | ||
conflicts occur against master they should be resolved by merging master | ||
into the branch used for making the pull request. | ||
Many thanks in advance for your cooperation! | ||
--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
## common | ||
__pycache__ | ||
*.pyc | ||
*~ | ||
\#* | ||
.#* | ||
*.o | ||
*.so | ||
*.dylib | ||
.cache | ||
.coverage | ||
.vscode | ||
*.swp | ||
*.pytest_cache | ||
htmlcov | ||
build/ | ||
build_prims/ | ||
dist/ | ||
python/**/**/*.cpp | ||
python/external_repositories | ||
python/record.txt | ||
log | ||
.ipynb_checkpoints | ||
.DS_Store | ||
dask-worker-space/ | ||
## eclipse | ||
.project | ||
.cproject | ||
.settings | ||
.ptp-sync-folder | ||
|
||
## Pycharm | ||
.idea | ||
|
||
## ccls | ||
.ccls-cache | ||
.ccls | ||
|
||
## profiling | ||
*.qdrep | ||
*.qdrep.cache | ||
*.qdstrm | ||
*.nvprof | ||
|
||
## doxygen build check inside ci/checks/style.sh | ||
doxygen_check/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,152 @@ | ||
# RAFT Build and Development Guide | ||
|
||
- [Usage of RAFT by downstream projects](usage-of-raft-by-downstream-projects) | ||
- [C++ Integration](c-integration) | ||
- [Python/Cython Integration](pythoncython-integration) | ||
- [Building and running tests](building-and-running-tests) | ||
- [CI Process](ci-process) | ||
- [Development Guide](developer-guide) | ||
- [Local Development](local-development) | ||
- [Submitting PRs](submitting-prs) | ||
|
||
|
||
|
||
## Usage of RAFT by downstream projects | ||
|
||
### C++ Integration | ||
|
||
The C++ component of RAFT is header only, so it can be easily configured using CMake by consuming libraries. Since this repo is intended to be included by downstream repos, the recommended way of accomplishing that is using CMake's git cloning functionality: | ||
|
||
|
||
```cmake | ||
if(DEFINED ENV{RAFT_PATH}) | ||
message(STATUS "RAFT_PATH environment variable detected.") | ||
message(STATUS "RAFT_DIR set to $ENV{RAFT_PATH}") | ||
set(RAFT_DIR ENV{RAFT_PATH}) | ||
else(DEFINED ENV{RAFT_PATH}) | ||
message(STATUS "RAFT_PATH environment variable NOT detected, cloning RAFT") | ||
set(RAFT_GIT_DIR ${CMAKE_CURRENT_BINARY_DIR}/raft CACHE STRING "Path to RAFT repo") | ||
ExternalProject_Add(raft | ||
GIT_REPOSITORY [email protected]:dantegd/barge.git | ||
GIT_TAG pinned_commit/git_tag/branch | ||
PREFIX ${RAFT_GIT_DIR} | ||
CONFIGURE_COMMAND "" | ||
BUILD_COMMAND "" | ||
INSTALL_COMMAND "") | ||
set(RAFT_INCLUDE_DIR ${RAFT_DIR}/src/raft/cpp/include CACHE STRING "RAFT include variable") | ||
endif(DEFINED ENV{RAFT_PATH}) | ||
``` | ||
|
||
This create the variable `$RAFT_INCLUDE_DIR` variable that can be used in `include_directories`, and then the related header files can be included when needed. | ||
|
||
### Python/Cython Integration | ||
|
||
The Python and Cython code have been designed to be included in projects that use RAFT, as opposed to a distributable by itself. To use: | ||
|
||
- The file `setuputils.py` is included in RAFT's `python` folder. Copy the file to your repo, in a location where it can be imported by `setup.py` | ||
- In your setup.py, use the function `use_raft_package`, for example for cuML: | ||
|
||
|
||
```python | ||
# Optional location of C++ build folder that can be configured by the user | ||
libcuml_path = get_environment_option('CUML_BUILD_PATH') | ||
# Optional location of RAFT that can be confugred by the user | ||
raft_path = get_environment_option('RAFT_PATH') | ||
|
||
use_raft_package(raft_path, libcuml_path) | ||
``` | ||
|
||
The usage of RAFT by the consuming repo's python code follows the rules: | ||
1. If the environment variable `RAFT_PATH` points to the RAFT repo, then that will be used. | ||
2. If there is a C++ build folder that has cloned RAFT already, setup.py will use that RAFT. | ||
3. If none of the above happened, then setup.py will clone RAFT and use it directly. | ||
|
||
- After `setup.py` calls the `use_raft_package` function, the RAFT python code will be included (via a symlink) in the consuming repo package, under a raft subfolder. So for example, `cuml` python package includes RAFT in `cuml.raft`. | ||
|
||
|
||
## Building and running tests | ||
|
||
Since RAFT is not meant to create any artifact on itself, but be included in other projects, the build infrastructure is focused only on testing. | ||
|
||
The base folder in the repository contains a `build.sh` script that builds both the C++ and Python code, which is the recommended way of building the tests. | ||
|
||
To run C++ tests: | ||
|
||
```bash | ||
./test_raft | ||
``` | ||
|
||
To run Python tests: | ||
|
||
```bash | ||
cd python | ||
python -m pytest | ||
``` | ||
|
||
To build manually, you can also use `CMake` and setup.py directly. For C++: | ||
|
||
```bash | ||
cd cpp | ||
mkdir build | ||
cd build | ||
cmake .. | ||
``` | ||
|
||
There is no `install` target currently. | ||
|
||
For python: | ||
|
||
```bash | ||
cd python | ||
python setup.py build_ext --inplace | ||
``` | ||
|
||
|
||
## CI Process | ||
|
||
PRs submitted to RAFT will always run the RAFT tests (once GPUCI is enabled). Additionally, RAFT has convenience functionality to run tests of the following projects that use RAFT: cuML and cuGraph. | ||
|
||
To run these other tests, turn `ON` the variables in `ci/prtest.config` in your PR: | ||
|
||
```bash | ||
RUN_CUGRAPH_LIBCUGRAPH_TESTS=OFF | ||
RUN_CUGRAPH_PYTHON_TESTS=OFF | ||
|
||
RUN_CUML_LIBCUML_TESTS=OFF | ||
RUN_CUML_PRIMS_TESTS=OFF | ||
RUN_CUML_PYTHON_TESTS=OFF | ||
``` | ||
|
||
This will make it so that CI in the PR will clone and build the respective repository, but the repository **will be built using the fork/branch of RAFT in the PR**. This allows to test changes in RAFT without the need of opening PRs in the other repositories. | ||
|
||
Before merging the PR, those variables need to be returned to `OFF`. | ||
|
||
|
||
## Developer Guide | ||
|
||
### Local Development | ||
|
||
To help working with RAFT and consuming projects as seamless as possible, this section describes how a typical workflow looks like and gives some guidelines for developers working in projects that affect code in both RAFT and at least one downstream repository. | ||
|
||
Using as an example developer working on cuML and RAFT, we recommend the following: | ||
|
||
- Create two working folders: one containing the cloned cuML repository and the other the cloned RAFT one. | ||
- Create environment variable `RAFT_PATH` pointing to the location of the RAFT path. | ||
- Work on same named branches in both repos/folders. | ||
|
||
This will facilitate development, and the `RAFT_PATH` variable will make it so that the downstream repository, in this case cuML, builds using the locally cloned RAFT (as descrbed in the first step). | ||
|
||
### Submitting PRs | ||
|
||
If you are submitting changes to RAFT itself, without changing downstream repos, you can use the config file located in `ci/prtest.config` to trigger RAFT's CI to run tests of downstream repositories. | ||
|
||
If you have changes to both RAFT and at least one downstream repo, then: | ||
|
||
- It is recommended to open a PR to both repositories (for visibility and CI tests). | ||
- Change the pinned branch/commit in the downstream repo PR to point to the fork and branch used for the RAFT PR to make CI run tests | ||
- If your changes might affect usage of RAFT by other downnstream repos, alert reviewers and open a github issue or PR in that downstream repo as approproate. | ||
- The PR to RAFT will be merged first, so that the downstream repo PR pinned branch/commit can be returned to the main RAFT branch and run CI with it. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
# RAFT 0.14.0 (Date TBD) | ||
|
||
|
||
## New Features | ||
- Initial RAFT version | ||
|
||
## Improvements | ||
|
||
|
||
## Bug Fixes |
Oops, something went wrong.