Header file organization

[harrism]

Now that the header-only API refactoring is complete, I'm working on code organization after merging the previously "experimental" headers and source into the main include and src directories.

Here is a view of the include/cuspatial directory, excluding include/cuspatial/detail

(rapids) coder ➜ ~/cuspatial/cpp/include/cuspatial $ tree -L 2 -I detail
.
├── bounding_box.cuh
├── column
│   └── geometry_column_view.hpp
├── constants.hpp
├── cuda_utils.hpp
├── derive_trajectories.cuh
├── distance
│   ├── hausdorff.hpp
│   ├── haversine.hpp
│   ├── linestring_distance.hpp
│   ├── point_distance.hpp
│   ├── point_linestring_distance.hpp
│   └── point_polygon_distance.hpp
├── error.hpp
├── geometry
│   ├── box.hpp
│   ├── linestring_ref.cuh
│   ├── polygon_ref.cuh
│   ├── segment.cuh
│   └── vec_2d.hpp
├── geometry_collection
│   ├── multilinestring_ref.cuh
│   ├── multipoint_ref.cuh
│   └── multipolygon_ref.cuh
├── hausdorff.cuh
├── haversine.cuh
├── iterator_factory.cuh
├── linestring_bounding_boxes.cuh
├── linestring_bounding_box.hpp
├── linestring_distance.cuh
├── linestring_intersection.cuh
├── linestring_intersection.hpp
├── linestring_polygon_distance.cuh
├── pairwise_point_in_polygon.cuh
├── pairwise_point_in_polygon.hpp
├── point_distance.cuh
├── point_in_polygon.cuh
├── point_in_polygon.hpp
├── point_linestring_distance.cuh
├── point_linestring_nearest_points.cuh
├── point_linestring_nearest_points.hpp
├── point_polygon_distance.cuh
├── point_quadtree.cuh
├── point_quadtree.hpp
├── points_in_range.cuh
├── polygon_bounding_boxes.cuh
├── polygon_bounding_box.hpp
├── projection.hpp
├── range
│   ├── multilinestring_range.cuh
│   ├── multipoint_range.cuh
│   ├── multipolygon_range.cuh
│   └── range.cuh
├── sinusoidal_projection.cuh
├── spatial_join.cuh
├── spatial_join.hpp
├── spatial_window.hpp
├── traits.hpp
├── trajectory_distances_and_speeds.cuh
├── trajectory.hpp
└── types.hpp

5 directories, 56 files

I have an in-progress PR #1084 to consolidate and improve this. So far, that PR

• Consolidates bounding_boxes APIs / headers and name files consistently -- combined header bounding_boxes.cuh/.hpp for point_bounding_boxes, linestring_bounding_boxes, and polygon_bounding_boxes. (Note, I noticed we don't have a column-API function for point_bounding_boxes -- I will file an issue for this)
• Consolidate trajectory APIs under a single header.
• Consolidate distance headers into the distance directory, as started by @isVoid.
• Consolidate pairwise and all-pairs point-in-polygon APIs under a single header and DRY up implementation (turns out they are very similar and can share 90% of the dispatch code and some of the kernel code).

Here is the current state of the tree in that PR:

(rapids) coder ➜ ~/cuspatial/cpp/include/cuspatial $ tree -L 2 -I detail
.
├── bounding_boxes.cuh
├── bounding_boxes.hpp
├── column
│   └── geometry_column_view.hpp
├── constants.hpp
├── cuda_utils.hpp
├── distance
│   ├── hausdorff.cuh
│   ├── hausdorff.hpp
│   ├── haversine.cuh
│   ├── haversine.hpp
│   ├── linestring_distance.cuh
│   ├── linestring_distance.hpp
│   ├── linestring_polygon_distance.cuh
│   ├── point_distance.cuh
│   ├── point_distance.hpp
│   ├── point_linestring_distance.cuh
│   ├── point_linestring_distance.hpp
│   ├── point_polygon_distance.cuh
│   └── point_polygon_distance.hpp
├── error.hpp
├── geometry
│   ├── box.hpp
│   ├── linestring_ref.cuh
│   ├── polygon_ref.cuh
│   ├── segment.cuh
│   └── vec_2d.hpp
├── geometry_collection
│   ├── multilinestring_ref.cuh
│   ├── multipoint_ref.cuh
│   └── multipolygon_ref.cuh
├── iterator_factory.cuh
├── linestring_intersection.cuh
├── linestring_intersection.hpp
├── point_in_polygon.cuh
├── point_in_polygon.hpp
├── point_linestring_nearest_points.cuh
├── point_linestring_nearest_points.hpp
├── point_quadtree.cuh
├── point_quadtree.hpp
├── points_in_range.cuh
├── projection.hpp
├── range
│   ├── multilinestring_range.cuh
│   ├── multipoint_range.cuh
│   ├── multipolygon_range.cuh
│   └── range.cuh
├── sinusoidal_projection.cuh
├── spatial_join.cuh
├── spatial_join.hpp
├── spatial_window.hpp
├── traits.hpp
├── trajectory.cuh
├── trajectory.hpp
└── types.hpp

5 directories, 50 files

I want to discuss options for further organization. @isVoid began moving all the disparate distance headers under a distance directory. That is one approach. A second approach would be to consolidate headers, but leave the hierarchy of algorithm headers fairly flat.

We already have directories for data structure headers, e.g. column, geometry, geometry_collection, and range. I think that it makes sense to keep these grouped in directories. Thrust takes a similar approach with its various iterator headers.

So let me explicitly lay out two options.

Option 1: Flat hierarchy with related functions grouped in common headers:

(rapids) coder ➜ ~/cuspatial/cpp/include/cuspatial $ tree -L 2 -I detail
.
├── bounding_boxes.cuh
├── bounding_boxes.hpp
├── column
│   └── geometry_column_view.hpp
├── constants.hpp
├── cuda_utils.hpp
├── distance.cuh
├── distance.hpp
├── error.hpp
├── geometry
│   ├── box.hpp
│   ├── linestring_ref.cuh
│   ├── polygon_ref.cuh
│   ├── segment.cuh
│   └── vec_2d.hpp
├── geometry_collection
│   ├── multilinestring_ref.cuh
│   ├── multipoint_ref.cuh
│   └── multipolygon_ref.cuh
├── iterator_factory.cuh
├── intersection.cuh
├── intersection.hpp
├── point_in_polygon.cuh
├── point_in_polygon.hpp
├── nearest_points.cuh
├── nearest_points.hpp
├── point_quadtree.cuh
├── point_quadtree.hpp
├── points_in_range.cuh
├── projection.hpp
├── range
│   ├── multilinestring_range.cuh
│   ├── multipoint_range.cuh
│   ├── multipolygon_range.cuh
│   └── range.cuh
├── projection.cuh
├── spatial_join.cuh
├── spatial_join.hpp
├── spatial_window.hpp
├── traits.hpp
├── trajectory.cuh
├── trajectory.hpp
└── types.hpp

4 directories, 39 files

Advantages:

• shallow hierarchy potentially makes headers easier to find and include.
• cuDF takes a similar approach.
• Similar to Thrust, STL.
• Easier to decide where to put header files.
• Consolidation reduces the number of headers.

Disadvantages:

• Users may experience higher compilation time when including headers if they only need one function from a header with many functions, compared to keeping them in separate headers.

Option 2: related functions grouped in directories

(rapids) coder ➜ ~/cuspatial/cpp/include/cuspatial $ tree -L 2 -I detail
.
├── bounds
│   ├── bounding_boxes.cuh
│   └── bounding_boxes.hpp
├── clip
│   ├──points_in_range.cuh
│   └──spatial_window.hpp
├── column
│   └── geometry_column_view.hpp
├── constants.hpp
├── cuda_utils.hpp
├── distance
│   ├── hausdorff.cuh
│   ├── hausdorff.hpp
│   ├── haversine.cuh
│   ├── haversine.hpp
│   ├── linestring_distance.cuh
│   ├── linestring_distance.hpp
│   ├── linestring_polygon_distance.cuh
│   ├── point_distance.cuh
│   ├── point_distance.hpp
│   ├── point_linestring_distance.cuh
│   ├── point_linestring_distance.hpp
│   ├── point_polygon_distance.cuh
│   └── point_polygon_distance.hpp
├── error.hpp
├── geometry
│   ├── box.hpp
│   ├── linestring_ref.cuh
│   ├── polygon_ref.cuh
│   ├── segment.cuh
│   └── vec_2d.hpp
├── geometry_collection
│   ├── multilinestring_ref.cuh
│   ├── multipoint_ref.cuh
│   └── multipolygon_ref.cuh
├── iterator_factory.cuh
├── index
│   ├── point_quadtree.cuh
│   └── point_quadtree.hpp
├── intersection
│   ├── linestring_intersection.cuh
│   └── linestring_intersection.hpp
├── join
│   ├──spatial_join.cuh
│   └──spatial_join.hpp
├── nearest_points
│   ├── point_linestring_nearest_points.cuh
│   └── point_linestring_nearest_points.hpp
├── point_in_polygon
│   ├── point_in_polygon.cuh
│   └── point_in_polygon.hpp
├── projection
│   ├──sinusoidal_projection.cuh
│   └──sinusoidal_projection.hpp
├── range
│   ├── multilinestring_range.cuh
│   ├── multipoint_range.cuh
│   ├── multipolygon_range.cuh
│   └── range.cuh
├── traits.hpp
├── trajectory
│   ├──trajectory.cuh
│   └──trajectory.hpp
└── types.hpp

17 directories, 50 files

Advantages:

• A logical directory hierarchy can help organization
• Allows separating functions into their own files without clutter, with potential benefits to compile time.

Disadvantages

• Can be harder to find the header you need. How deep do we allow hierarchy to go?
• Can be difficult to decide how to group headers. Some could go in multiple groups.
• Group naming leads to bike shedding. Then again everything leads to bike shedding.

[jarmak-nv]

Gut feeling, I prefer option 2. Grouping together feels more intuitive to me if I come at it from an outsider's perspective.

However, I also lean towards option 1 being a better option for dev usage, it's straightforward, clear and just works, it just might get big and a little unwieldy as we continue to grow but that's not a problem for people who are in the thick of it frequently.

IMO this impacts dev more than users, so we should use option 1.

[isVoid]

I'm also in favor of option 1. Fewer headers means lower maintenance cost. Frankly speaking I'm not sure if I like how distance is organized now, it seems too fragmented. In terms of longer complication time, if this is what cuDF team has received complaints about, then we should also look into solutions, otherwise there's a long way until we hit that performance drop point. For those who are using the header only API, the template API are the main portion of the compilation time anyway.

[harrism]

Thanks for all the feedback @jarmak-nv and @isVoid . PR ready for review: #1097