Skip to content

Commit

Permalink
Merge pull request #634 from rapidsai/branch-22.08
Browse files Browse the repository at this point in the history
[RELEASE] cuspatial v22.08
  • Loading branch information
raydouglass authored Aug 17, 2022
2 parents 96613ba + 66d4739 commit 5ea1978
Show file tree
Hide file tree
Showing 135 changed files with 4,885 additions and 2,908 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -146,4 +146,4 @@ ENV/
notebooks/taxi2016.csv
notebooks/taxi2017.csv
notebooks/tzones_lonlat.json
notebooks/cu_taxi_zones.*
notebooks/cu_taxi.zones.*
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/pycqa/isort
rev: 5.6.4
rev: 5.10.1
hooks:
- id: isort
args: ["--settings-path=python/cuspatial/setup.cfg"]
Expand Down
43 changes: 43 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,46 @@
# cuSpatial 22.08.00 (17 Aug 2022)

## 🚨 Breaking Changes

- Pairwise Point to Point Distance; Rename Folder `distances` to `distance` ([#558](https://github.com/rapidsai/cuspatial/pull/558)) [@isVoid](https://github.com/isVoid)

## 🐛 Bug Fixes

- Fix Broken Link in NYC Taxi Notebook ([#616](https://github.com/rapidsai/cuspatial/pull/616)) [@isVoid](https://github.com/isVoid)
- Add missing rmm includes ([#590](https://github.com/rapidsai/cuspatial/pull/590)) [@harrism](https://github.com/harrism)
- Fix failed automerge 22.06 into 22.08 ([#562](https://github.com/rapidsai/cuspatial/pull/562)) [@harrism](https://github.com/harrism)
- Bump cuspatial python version in scikit-build ([#550](https://github.com/rapidsai/cuspatial/pull/550)) [@isVoid](https://github.com/isVoid)

## 📖 Documentation

- Defer loading of `custom.js` ([#631](https://github.com/rapidsai/cuspatial/pull/631)) [@galipremsagar](https://github.com/galipremsagar)
- Use documented header template for `doxygen` ([#619](https://github.com/rapidsai/cuspatial/pull/619)) [@galipremsagar](https://github.com/galipremsagar)
- Fix issues with day & night modes in python docs ([#613](https://github.com/rapidsai/cuspatial/pull/613)) [@isVoid](https://github.com/isVoid)

## 🚀 New Features

- Add NVBench and linestring distance benchmark ([#577](https://github.com/rapidsai/cuspatial/pull/577)) [@isVoid](https://github.com/isVoid)
- Pairwise Point to Point Distance; Rename Folder `distances` to `distance` ([#558](https://github.com/rapidsai/cuspatial/pull/558)) [@isVoid](https://github.com/isVoid)

## 🛠️ Improvements

- Bump `gdal` version ([#623](https://github.com/rapidsai/cuspatial/pull/623)) [@ajschmidt8](https://github.com/ajschmidt8)
- Build benchmarks in CI ([#597](https://github.com/rapidsai/cuspatial/pull/597)) [@vyasr](https://github.com/vyasr)
- Add benchmark for points_in_spatial_window ([#595](https://github.com/rapidsai/cuspatial/pull/595)) [@harrism](https://github.com/harrism)
- Update isort to version 5.10.1 ([#589](https://github.com/rapidsai/cuspatial/pull/589)) [@vyasr](https://github.com/vyasr)
- Header-only Refactor of `point_in_polygon` ([#587](https://github.com/rapidsai/cuspatial/pull/587)) [@isVoid](https://github.com/isVoid)
- Remove GeoArrow glue code replacing gpu storage with cudf.Series and host storage with pyarrow ([#585](https://github.com/rapidsai/cuspatial/pull/585)) [@thomcom](https://github.com/thomcom)
- Create `pygeoarrow` and use it for cuSpatial feature storage and i/o ([#583](https://github.com/rapidsai/cuspatial/pull/583)) [@thomcom](https://github.com/thomcom)
- Header-only refactoring of `points_in_spatial_window` ([#579](https://github.com/rapidsai/cuspatial/pull/579)) [@harrism](https://github.com/harrism)
- Update Python version support. ([#574](https://github.com/rapidsai/cuspatial/pull/574)) [@bdice](https://github.com/bdice)
- Combine `device_atomics` ([#561](https://github.com/rapidsai/cuspatial/pull/561)) [@isVoid](https://github.com/isVoid)
- Relocate Utility Files ([#560](https://github.com/rapidsai/cuspatial/pull/560)) [@isVoid](https://github.com/isVoid)
- Fuse `transform` and `copy_if` operations in `quadtree_point_in_polygon` ([#559](https://github.com/rapidsai/cuspatial/pull/559)) [@trxcllnt](https://github.com/trxcllnt)
- Remove `itstimestamp` and `types.hpp` ([#554](https://github.com/rapidsai/cuspatial/pull/554)) [@isVoid](https://github.com/isVoid)
- Change build.sh to find C++ library by default and avoid shadowing CMAKE_ARGS ([#543](https://github.com/rapidsai/cuspatial/pull/543)) [@vyasr](https://github.com/vyasr)
- Add missing Thrust includes ([#539](https://github.com/rapidsai/cuspatial/pull/539)) [@bdice](https://github.com/bdice)
- Refactor Hausdorff distance to header-only API ([#538](https://github.com/rapidsai/cuspatial/pull/538)) [@harrism](https://github.com/harrism)

# cuSpatial 22.06.00 (7 Jun 2022)

## 🐛 Bug Fixes
Expand Down
30 changes: 21 additions & 9 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,13 @@ ARGS=$*
# script, and that this script resides in the repo dir!
REPODIR=$(cd $(dirname $0); pwd)

VALIDARGS="clean libcuspatial cuspatial tests -v -g -n -h --allgpuarch --show_depr_warn"
VALIDARGS="clean libcuspatial cuspatial tests benchmarks -v -g -n -h --allgpuarch --show_depr_warn"
HELP="$0 [clean] [libcuspatial] [cuspatial] [tests] [-v] [-g] [-n] [-h] [-l] [--show_depr_warn] [--cmake-args=\"<args>\"]
clean - remove all existing build artifacts and configuration (start over)
libcuspatial - build the libcuspatial C++ code only
cuspatial - build the cuspatial Python package
tests - build tests
benchmarks - build benchmarks
-v - verbose build mode
-g - build for debug
-n - no install step
Expand All @@ -41,6 +42,7 @@ BUILD_DIRS="${LIBCUSPATIAL_BUILD_DIR} ${CUSPATIAL_BUILD_DIR}"
# Set defaults for vars modified by flags to this script
VERBOSE_FLAG=""
BUILD_TESTS=OFF
BUILD_BENCHMARKS=OFF
BUILD_TYPE=Release
BUILD_ALL_GPU_ARCH=0
INSTALL_TARGET=install
Expand Down Expand Up @@ -68,12 +70,12 @@ function cmakeArgs {
# There are possible weird edge cases that may cause this regex filter to output nothing and fail silently
# the true pipe will catch any weird edge cases that may happen and will cause the program to fall back
# on the invalid option error
CMAKE_ARGS=$(echo $ARGS | { grep -Eo "\-\-cmake\-args=\".+\"" || true; })
if [[ -n ${CMAKE_ARGS} ]]; then
# Remove the full CMAKE_ARGS argument from list of args so that it passes validArgs function
ARGS=${ARGS//$CMAKE_ARGS/}
EXTRA_CMAKE_ARGS=$(echo $ARGS | { grep -Eo "\-\-cmake\-args=\".+\"" || true; })
if [[ -n ${EXTRA_CMAKE_ARGS} ]]; then
# Remove the full EXTRA_CMAKE_ARGS argument from list of args so that it passes validArgs function
ARGS=${ARGS//$EXTRA_CMAKE_ARGS/}
# Filter the full argument down to just the extra string that will be added to cmake call
CMAKE_ARGS=$(echo $CMAKE_ARGS | grep -Eo "\".+\"" | sed -e 's/^"//' -e 's/"$//')
EXTRA_CMAKE_ARGS=$(echo $EXTRA_CMAKE_ARGS | grep -Eo "\".+\"" | sed -e 's/^"//' -e 's/"$//')
fi
fi
}
Expand Down Expand Up @@ -115,6 +117,15 @@ if hasArg tests; then
BUILD_TESTS=ON
fi

if hasArg benchmarks; then
BUILD_BENCHMARKS=ON
fi

# Append `-DFIND_CUSPATIAL_CPP=ON` to EXTRA_CMAKE_ARGS unless a user specified the option.
if [[ "${EXTRA_CMAKE_ARGS}" != *"DFIND_CUSPATIAL_CPP"* ]]; then
EXTRA_CMAKE_ARGS="${EXTRA_CMAKE_ARGS} -DFIND_CUSPATIAL_CPP=ON"
fi

# If clean given, run it prior to any other steps
if hasArg clean; then
# If the dirs to clean are mounted dirs in a container, the
Expand Down Expand Up @@ -146,9 +157,10 @@ if (( ${NUMARGS} == 0 )) || hasArg libcuspatial; then
${CUSPATIAL_CMAKE_CUDA_ARCHITECTURES} \
-DCMAKE_CXX11_ABI=ON \
-DBUILD_TESTS=${BUILD_TESTS} \
-DBUILD_BENCHMARKS=${BUILD_BENCHMARKS} \
-DDISABLE_DEPRECATION_WARNING=${BUILD_DISABLE_DEPRECATION_WARNING} \
-DCMAKE_BUILD_TYPE=${BUILD_TYPE} \
${CMAKE_ARGS} \
${EXTRA_CMAKE_ARGS} \
..

cmake --build . -j ${PARALLEL_LEVEL} ${VERBOSE_FLAG}
Expand All @@ -162,8 +174,8 @@ fi
if (( ${NUMARGS} == 0 )) || hasArg cuspatial; then

cd ${REPODIR}/python/cuspatial
python setup.py build_ext -j${PARALLEL_LEVEL:-1} --inplace -- -DCMAKE_PREFIX_PATH=${INSTALL_PREFIX} -DCMAKE_LIBRARY_PATH=${LIBCUSPATIAL_BUILD_DIR} ${CMAKE_ARGS}
python setup.py build_ext -j${PARALLEL_LEVEL:-1} --inplace -- -DCMAKE_PREFIX_PATH=${INSTALL_PREFIX} -DCMAKE_LIBRARY_PATH=${LIBCUSPATIAL_BUILD_DIR} ${EXTRA_CMAKE_ARGS}
if [[ ${INSTALL_TARGET} != "" ]]; then
python setup.py install --single-version-externally-managed --record=record.txt -- ${CMAKE_ARGS}
python setup.py install --single-version-externally-managed --record=record.txt -- -DCMAKE_PREFIX_PATH=${INSTALL_PREFIX} -DCMAKE_LIBRARY_PATH=${LIBCUSPATIAL_BUILD_DIR} ${EXTRA_CMAKE_ARGS}
fi
fi
2 changes: 1 addition & 1 deletion ci/gpu/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ if [[ -z "$PROJECT_FLASH" || "$PROJECT_FLASH" == "0" ]]; then

gpuci_logger "Build cuSpatial"
cd "$WORKSPACE"
./build.sh clean libcuspatial cuspatial tests --cmake-args=\"-DFIND_CUSPATIAL_CPP=ON\"
./build.sh clean libcuspatial cuspatial tests

###############################################################################
# TEST - Run libcuspatial and cuSpatial Unit Tests
Expand Down
4 changes: 2 additions & 2 deletions conda/environments/cuspatial_dev_cuda11.0.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ channels:
dependencies:
- clang=11.1.0
- clang-tools=11.1.0
- cudf=22.06.*
- cudf=22.08.*
- cudatoolkit=11.0
- gdal>=3.0.2
- geopandas>=0.9.0,<0.10.0a0
- geopandas>=0.11.0
- cmake>=3.20.1,!=3.23.0
- cython>=0.29,<0.30
- scikit-build>=0.13.1
Expand Down
4 changes: 2 additions & 2 deletions conda/environments/cuspatial_dev_cuda11.1.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ channels:
dependencies:
- clang=11.1.0
- clang-tools=11.1.0
- cudf=22.06.*
- cudf=22.08.*
- cudatoolkit=11.1
- gdal>=3.0.2
- geopandas>=0.9.0,<0.10.0a0
- geopandas>=0.11.0
- cmake>=3.20.1,!=3.23.0
- cython>=0.29,<0.30
- scikit-build>=0.13.1
Expand Down
4 changes: 2 additions & 2 deletions conda/environments/cuspatial_dev_cuda11.2.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ channels:
dependencies:
- clang=11.1.0
- clang-tools=11.1.0
- cudf=22.06.*
- cudf=22.08.*
- cudatoolkit=11.2
- gdal>=3.0.2
- geopandas>=0.9.0,<0.10.0a0
- geopandas>=0.11.0
- cmake>=3.20.1,!=3.23.0
- cython>=0.29,<0.30
- scikit-build>=0.13.1
Expand Down
5 changes: 4 additions & 1 deletion conda/recipes/cuspatial/build.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@

# Copyright (c) 2018-2019, NVIDIA CORPORATION.

# Ignore conda-provided CMAKE_ARGS for the Python build.
unset CMAKE_ARGS

# This assumes the script is executed from the root of the repo directory
./build.sh cuspatial --cmake-args=\"-DFIND_CUSPATIAL_CPP=ON\"
./build.sh cuspatial
3 changes: 3 additions & 0 deletions conda/recipes/cuspatial/conda_build_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,6 @@ cuda_compiler:

sysroot_version:
- "2.17"

gdal_version:
- ">=3.4.3,<3.4.4a0"
4 changes: 2 additions & 2 deletions conda/recipes/cuspatial/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ requirements:
- python
- cudf {{ minor_version }}.*
- rmm {{ minor_version }}.*
- gdal >=3.3.0,<3.4.0a0
- geopandas >=0.9.0,<0.10.0a0
- gdal {{ gdal_version }}
- geopandas >=0.11.0

test: # [linux64]
imports: # [linux64]
Expand Down
2 changes: 1 addition & 1 deletion conda/recipes/libcuspatial/build.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2018-2022, NVIDIA CORPORATION.

# build cuspatial with verbose output
./build.sh -v libcuspatial tests --allgpuarch -n
./build.sh -v libcuspatial tests benchmarks --allgpuarch -n
2 changes: 1 addition & 1 deletion conda/recipes/libcuspatial/conda_build_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ cmake_version:
- ">=3.20.1"

gdal_version:
- ">=3.3.0,<3.4.0a0"
- ">=3.4.3,<3.4.4a0"

gtest_version:
- "1.10.0"
Expand Down
8 changes: 7 additions & 1 deletion cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ include(rapids-find)

rapids_cuda_init_architectures(CUSPATIAL)

project(CUSPATIAL VERSION 22.06.00 LANGUAGES C CXX CUDA)
project(CUSPATIAL VERSION 22.08.00 LANGUAGES C CXX CUDA)

# Needed because GoogleBenchmark changes the state of FindThreads.cmake,
# causing subsequent runs to have different values for the `Threads::Threads` target.
Expand Down Expand Up @@ -103,6 +103,7 @@ rapids_cpm_init()
# find or add cuDF
include(cmake/thirdparty/CUSPATIAL_GetCUDF.cmake)


###################################################################################################
# - library targets -------------------------------------------------------------------------------

Expand All @@ -121,6 +122,7 @@ add_library(cuspatial
src/spatial/haversine.cu
src/spatial/hausdorff.cu
src/spatial/linestring_distance.cu
src/spatial/point_distance.cu
src/spatial/lonlat_to_cartesian.cu
src/trajectory/derive_trajectories.cu
src/trajectory/trajectory_bounding_boxes.cu
Expand Down Expand Up @@ -207,6 +209,10 @@ if(CUSPATIAL_BUILD_BENCHMARKS)
GIT_SHALLOW TRUE
OPTIONS "BENCHMARK_ENABLE_TESTING OFF"
"BENCHMARK_ENABLE_INSTALL OFF")

# Find or install NVBench
include(${rapids-cmake-dir}/cpm/nvbench.cmake)
rapids_cpm_nvbench()
add_subdirectory(benchmarks)
endif()

Expand Down
35 changes: 32 additions & 3 deletions cpp/benchmarks/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -40,17 +40,37 @@ target_include_directories(cuspatial_benchmark_common
function(ConfigureBench CMAKE_BENCH_NAME)
add_executable(${CMAKE_BENCH_NAME} ${ARGN})
set_target_properties(${CMAKE_BENCH_NAME}
PROPERTIES RUNTIME_OUTPUT_DIRECTORY "$<BUILD_INTERFACE:${CUSPATIAL_BINARY_DIR}/gbenchmarks>"
PROPERTIES RUNTIME_OUTPUT_DIRECTORY "$<BUILD_INTERFACE:${CUSPATIAL_BINARY_DIR}/benchmarks>"
INSTALL_RPATH "\$ORIGIN/../../../lib"
)
target_link_libraries(${CMAKE_BENCH_NAME} PRIVATE benchmark::benchmark_main cuspatial_benchmark_common)
install(
TARGETS ${CMAKE_BENCH_NAME}
COMPONENT testing
COMPONENT benchmark
DESTINATION bin/benchmarks/libcuspatial
EXCLUDE_FROM_ALL
)
endfunction(ConfigureBench)
endfunction()

# This function takes in a benchmark name and benchmark source for nvbench benchmarks and handles
# setting all of the associated properties and linking to build the benchmark
function(ConfigureNVBench CMAKE_BENCH_NAME)
add_executable(${CMAKE_BENCH_NAME} ${ARGN})
set_target_properties(
${CMAKE_BENCH_NAME}
PROPERTIES RUNTIME_OUTPUT_DIRECTORY "$<BUILD_INTERFACE:${CUSPATIAL_BINARY_DIR}/benchmarks>"
INSTALL_RPATH "\$ORIGIN/../../../lib"
)
target_link_libraries(
${CMAKE_BENCH_NAME} PRIVATE cuspatial_benchmark_common nvbench::main
)
install(
TARGETS ${CMAKE_BENCH_NAME}
COMPONENT benchmark
DESTINATION bin/benchmarks/libcuspatial
EXCLUDE_FROM_ALL
)
endfunction()

###################################################################################################
### benchmark sources #############################################################################
Expand All @@ -61,3 +81,12 @@ endfunction(ConfigureBench)

ConfigureBench(HAUSDORFF_BENCH
hausdorff_benchmark.cpp)

ConfigureNVBench(DISTANCES_BENCH
pairwise_linestring_distance.cu)

ConfigureNVBench(POINTS_IN_RANGE_BENCH
points_in_range.cu)

ConfigureNVBench(POINT_IN_POLYGON_BENCH
point_in_polygon.cu)
71 changes: 71 additions & 0 deletions cpp/benchmarks/fixture/rmm_pool_raii.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
/*
* Copyright (c) 2022, NVIDIA CORPORATION.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#include <rmm/mr/device/cuda_memory_resource.hpp>
#include <rmm/mr/device/owning_wrapper.hpp>
#include <rmm/mr/device/per_device_resource.hpp>
#include <rmm/mr/device/pool_memory_resource.hpp>

namespace cuspatial {

/**
* @brief An RAII class setting up RMM memory pool for `nvbench` benchmarks
*
* This is a temporary solution before templated fixtures tests are supported
* in `nvbench`. Similarly to `cuspatial::benchmark`, creating this RAII object in
* each benchmark will ensure that the RAPIDS Memory Manager pool mode is used
* in benchmarks, which eliminates memory allocation / deallocation performance
* overhead from the benchmark.
*
* Example:
*
* void my_benchmark(nvbench::state& state) {
* cuspatial::rmm_pool_raii pool_raii;
* state.exec([](nvbench::launch& launch) {
* // benchmark stuff
* });
* }
*
* NVBENCH_BENCH(my_benchmark);
*/
class rmm_pool_raii {
private:
// memory resource factory helpers
inline auto make_cuda() { return std::make_shared<rmm::mr::cuda_memory_resource>(); }

inline auto make_pool()
{
return rmm::mr::make_owning_wrapper<rmm::mr::pool_memory_resource>(make_cuda());
}

public:
rmm_pool_raii()
{
mr = make_pool();
rmm::mr::set_current_device_resource(mr.get()); // set default resource to pool
}

~rmm_pool_raii()
{
rmm::mr::set_current_device_resource(nullptr);
mr.reset();
}

private:
std::shared_ptr<rmm::mr::device_memory_resource> mr;
};

} // namespace cuspatial
Loading

0 comments on commit 5ea1978

Please sign in to comment.