Skip to content

Commit

Permalink
Add some (but not all) docs for the Rust crate (#696)
Browse files Browse the repository at this point in the history
Includes:

- A breaking cahnge to `Table::from_arrow_and_geometry` to make the
argument order the same as `Table::try_new`
- Removal of a leftover `dbg!` statement in
[src/array/coord/separated/array.rs](0ab9dc0#diff-05c34ee401b527590650de1809d1edcc54957ce8a5c47128371bc83729ba2392)
- Moved two single-file `mod-name/mod.rs` modules up a level (`table.rs`
and `chunked_array.rs`)
- Some README tweaks, including adding badges
- `#![deny(missing_docs)]` to `lib.rs`, and then
`#![allow(missing_docs)]` on the "deep" modules (e.g. `algorithm`,
`array`)
- Docs to the rest of the modules, with examples almost always

NB I've updated #689 with
a checklist to track progress on the "deep" modules

---------

Co-authored-by: Kyle Barron <[email protected]>
  • Loading branch information
gadomski and kylebarron authored Aug 13, 2024
1 parent 77381c3 commit 9eaf839
Show file tree
Hide file tree
Showing 36 changed files with 1,506 additions and 149 deletions.
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ bench = false # TODO fix this benchmark
required-features = ["parquet_compression"]

[package.metadata.docs.rs]
rustdoc-args = ["--cfg", "docsrs"]
features = [
"csv",
"flatgeobuf",
Expand Down
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# `geoarrow-rs`
# geoarrow-rs

[![GitHub Workflow Status (CI)](https://img.shields.io/github/actions/workflow/status/geoarrow/geoarrow-rs/ci.yml?branch=main)](https://github.com/geoarrow/geoarrow-rs/actions/workflows/ci.yml)
[![docs.rs](https://img.shields.io/docsrs/geoarrow?label=docs.rs)](https://docs.rs/geoarrow/latest/geoarrow/)
[![Crates.io](https://img.shields.io/crates/v/geoarrow)](https://crates.io/crates/geoarrow)
![Crates.io](https://img.shields.io/crates/l/geoarrow)

A Rust implementation of the [GeoArrow](https://github.com/geoarrow/geoarrow) specification and bindings to [GeoRust algorithms](https://github.com/georust/geo) for efficient spatial operations on GeoArrow memory.

Expand All @@ -20,7 +25,7 @@ This repository also includes [Python bindings](https://github.com/geoarrow/geoa
Add this to your `Cargo.toml`:

```toml
geoarrow = "0.1"
geoarrow = "0.2"
```

## References
Expand Down
7 changes: 3 additions & 4 deletions js/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# `geoarrow-wasm`
# geoarrow-wasm

[![GitHub Workflow Status (WASM)](https://img.shields.io/github/actions/workflow/status/geoarrow/geoarrow-rs/wasm.yml?label=WASM&branch=main)](https://github.com/geoarrow/geoarrow-rs/actions/workflows/wasm.yml)

Efficient, vectorized geospatial operations in WebAssembly.

Expand All @@ -21,7 +23,6 @@ I wrote a [blog post](https://kylebarron.dev/blog/geos-wasm) about this that goe

Most users will use this by installing the prebuilt JavaScript package. This is published to NPM as [`geoarrow-wasm`](https://npmjs.com/package/geoarrow-wasm).


### From Rust

Advanced users can also depend on these Rust-Wasm bindings directly, enabling you to add custom operations on top of these bindings and generating your own WebAssembly bundles. This means you can reuse all the binding between JavaScript and WebAssembly and focus on implementing your algorithms. This package is published to crates.io as [`geoarrow-wasm`](https://crates.io/crates/geoarrow-wasm).
Expand All @@ -31,5 +32,3 @@ Advanced users can also depend on these Rust-Wasm bindings directly, enabling yo
- [Prototyping GeoRust + GeoArrow in WebAssembly](https://observablehq.com/@kylebarron/prototyping-georust-geoarrow-in-webassembly)

## How it Works


2 changes: 1 addition & 1 deletion js/src/io/flatgeobuf.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,6 @@ pub fn read_flatgeobuf(file: &[u8], batch_size: Option<usize>) -> WasmResult<Tab
..Default::default()
};
let geo_table = _read_flatgeobuf(&mut cursor, options)?;
let (schema, batches) = geo_table.into_inner();
let (batches, schema) = geo_table.into_inner();
Ok(Table::new(schema, batches))
}
4 changes: 2 additions & 2 deletions js/src/io/geojson.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ pub fn read_geojson(file: &[u8], batch_size: Option<usize>) -> WasmResult<Table>
// assert_parquet_file_not_empty(parquet_file)?;
let mut cursor = Cursor::new(file);
let geo_table = _read_geojson(&mut cursor, batch_size)?;
let (schema, batches) = geo_table.into_inner();
let (batches, schema) = geo_table.into_inner();
Ok(Table::new(schema, batches))
}

Expand All @@ -39,7 +39,7 @@ pub fn read_geojson(file: &[u8], batch_size: Option<usize>) -> WasmResult<Table>
#[wasm_bindgen(js_name = writeGeoJSON)]
pub fn write_geojson(table: Table) -> WasmResult<Vec<u8>> {
let (schema, batches) = table.into_inner();
let rust_table = geoarrow::table::Table::try_new(schema, batches)?;
let rust_table = geoarrow::table::Table::try_new(batches, schema)?;
let mut output_file: Vec<u8> = vec![];
_write_geojson(rust_table, &mut output_file)?;
Ok(output_file)
Expand Down
8 changes: 4 additions & 4 deletions js/src/io/parquet/async.rs
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ impl ParquetFile {
)
.build()?;
let table = stream.read_table().await?;
let (schema, batches) = table.into_inner();
let (batches, schema) = table.into_inner();
Ok(Table::new(schema, batches))
}
#[wasm_bindgen]
Expand Down Expand Up @@ -267,11 +267,11 @@ impl ParquetDataset {

let mut all_batches = vec![];
tables.into_iter().for_each(|table| {
let (_schema, table_batches) = table.into_inner();
let (table_batches, _schema) = table.into_inner();
all_batches.extend(table_batches);
});
let table = geoarrow::table::Table::try_new(output_schema, all_batches)?;
let (schema, batches) = table.into_inner();
let table = geoarrow::table::Table::try_new(all_batches, output_schema)?;
let (batches, schema) = table.into_inner();
Ok(Table::new(schema, batches))
}

Expand Down
4 changes: 2 additions & 2 deletions js/src/io/parquet/sync.rs
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ pub fn read_geoparquet(file: Vec<u8>) -> WasmResult<Table> {
)?
.build()?;
let geo_table = reader.read_table()?;
let (schema, batches) = geo_table.into_inner();
let (batches, schema) = geo_table.into_inner();
Ok(Table::new(schema, batches))
}

Expand All @@ -47,7 +47,7 @@ pub fn read_geoparquet(file: Vec<u8>) -> WasmResult<Table> {
#[wasm_bindgen(js_name = writeGeoParquet)]
pub fn write_geoparquet(table: Table) -> WasmResult<Vec<u8>> {
let (schema, batches) = table.into_inner();
let mut rust_table = geoarrow::table::Table::try_new(schema, batches)?;
let mut rust_table = geoarrow::table::Table::try_new(batches, schema)?;
let mut output_file: Vec<u8> = vec![];
_write_geoparquet(&mut rust_table, &mut output_file, &Default::default())?;
Ok(output_file)
Expand Down
4 changes: 3 additions & 1 deletion python/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# `geoarrow.rust`: Python bindings to `geoarrow-rs`
# geoarrow.rust

[![GitHub Workflow Status (Python)](https://img.shields.io/github/actions/workflow/status/geoarrow/geoarrow-rs/python.yml?branch=main)](https://github.com/geoarrow/geoarrow-rs/actions/workflows/python.yml)

This folder contains Python bindings to the [GeoArrow Rust implementation](https://github.com/geoarrow/geoarrow-rs).

Expand Down
5 changes: 0 additions & 5 deletions python/core/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion python/core/src/ffi/from_python/table.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ impl<'a> FromPyObject<'a> for GeoTable {
batches.push(batch);
}

let table = geoarrow::table::Table::try_new(schema, batches)
let table = geoarrow::table::Table::try_new(batches, schema)
.map_err(|e| PyValueError::new_err(e.to_string()))?;
let table = table
.downcast(true)
Expand Down
4 changes: 2 additions & 2 deletions python/core/src/interop/util.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,11 @@ pub(crate) fn import_pyogrio(py: Python) -> PyGeoArrowResult<Bound<PyModule>> {
}

pub(crate) fn table_to_pytable(table: geoarrow::table::Table) -> PyTable {
let (schema, batches) = table.into_inner();
let (batches, schema) = table.into_inner();
PyTable::new(batches, schema)
}

pub(crate) fn pytable_to_table(table: PyTable) -> Result<geoarrow::table::Table, GeoArrowError> {
let (batches, schema) = table.into_inner();
geoarrow::table::Table::try_new(schema, batches)
geoarrow::table::Table::try_new(batches, schema)
}
8 changes: 4 additions & 4 deletions python/core/src/io/parquet/reader.rs
Original file line number Diff line number Diff line change
Expand Up @@ -574,10 +574,10 @@ impl ParquetDataset {

let mut all_batches = vec![];
tables.into_iter().for_each(|table| {
let (_schema, table_batches) = table.into_inner();
let (table_batches, _schema) = table.into_inner();
all_batches.extend(table_batches);
});
let table = Table::try_new(output_schema, all_batches)
let table = Table::try_new(all_batches, output_schema)
.map_err(PyGeoArrowError::GeoArrowError)?;
Ok(table_to_pytable(table))
})?;
Expand Down Expand Up @@ -609,10 +609,10 @@ impl ParquetDataset {

let mut all_batches = vec![];
tables.into_iter().for_each(|table| {
let (_schema, table_batches) = table.into_inner();
let (table_batches, _schema) = table.into_inner();
all_batches.extend(table_batches);
});
let table = Table::try_new(output_schema, all_batches)
let table = Table::try_new(all_batches, output_schema)
.map_err(PyGeoArrowError::GeoArrowError)?;
Ok(table_to_pytable(table).to_arro3(py)?)
})
Expand Down
2 changes: 2 additions & 0 deletions src/algorithm/mod.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
//! Vectorized algorithms implemented on and returning GeoArrow arrays.
#![allow(missing_docs)] // FIXME

pub mod broadcasting;
pub mod geo;
pub mod geo_index;
Expand Down
2 changes: 1 addition & 1 deletion src/algorithm/native/explode.rs
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ impl ExplodeTable for Table {
schema_builder.push(field.clone());
let schema = schema_builder.finish();

Table::try_new(schema.into(), new_batches)
Table::try_new(new_batches, schema.into())
} else {
// No take is necessary; nothing happens
Ok(self.clone())
Expand Down
3 changes: 3 additions & 0 deletions src/array/coord/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@ pub use separated::{SeparatedCoordBuffer, SeparatedCoordBufferBuilder};
/// buffers as XXXX and YYYY.
#[derive(Default, Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum CoordType {
/// Interleaved coordinates.
#[default]
Interleaved,

/// Separated coordinates.
Separated,
}
1 change: 0 additions & 1 deletion src/array/coord/separated/array.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ pub struct SeparatedCoordBuffer<const D: usize> {
}

fn check<const D: usize>(buffers: &[ScalarBuffer<f64>; D]) -> Result<()> {
dbg!(buffers);
if !buffers.windows(2).all(|w| w[0].len() == w[1].len()) {
return Err(GeoArrowError::General(
"all buffers must have the same length".to_string(),
Expand Down
5 changes: 5 additions & 0 deletions src/array/metadata.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ use crate::error::GeoArrowError;
/// this value is omitted, edges will be interpreted as planar.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub enum Edges {
/// Follow a spherical path rather than a planar.
///
/// See [the geoarrow
/// specification](https://github.com/geoarrow/geoarrow/blob/main/extension-types.md#extension-metadata)
/// for more information aobut how `edges` should be used.
#[serde(rename = "spherical")]
Spherical,
}
Expand Down
2 changes: 2 additions & 0 deletions src/array/mod.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
//! Implementations of immutable GeoArrow arrays plus builders to more easily create arrays.
#![allow(missing_docs)] // FIXME

pub use binary::{WKBArray, WKBBuilder, WKBCapacity};
pub use cast::{AsChunkedGeometryArray, AsGeometryArray};
pub use coord::{
Expand Down
Loading

0 comments on commit 9eaf839

Please sign in to comment.