|
21 | 21 |
|
22 | 22 | Welcome to the [Rust][rust] implementation of [Apache Arrow], the popular in-memory columnar format.
|
23 | 23 |
|
24 |
| -This repo contains the following main components: |
| 24 | +This repository contains the following crates: |
25 | 25 |
|
26 | 26 | | Crate | Description | Latest API Docs | README |
|
27 | 27 | | ------------------ | ---------------------------------------------------------------------------- | ------------------------------------------------ | --------------------------------- |
|
28 | 28 | | [`arrow`] | Core functionality (memory layout, arrays, low level computations) | [docs.rs](https://docs.rs/arrow/latest) | [(README)][arrow-readme] |
|
29 | 29 | | [`arrow-flight`] | Support for Arrow-Flight IPC protocol | [docs.rs](https://docs.rs/arrow-flight/latest) | [(README)][flight-readme] |
|
30 |
| -| [`object-store`] | Support for object store interactions (aws, azure, gcp, local, in-memory) | [docs.rs](https://docs.rs/object_store/latest) | [(README)][objectstore-readme] | |
31 | 30 | | [`parquet`] | Support for Parquet columnar file format | [docs.rs](https://docs.rs/parquet/latest) | [(README)][parquet-readme] |
|
32 | 31 | | [`parquet_derive`] | A crate for deriving RecordWriter/RecordReader for arbitrary, simple structs | [docs.rs](https://docs.rs/parquet-derive/latest) | [(README)][parquet-derive-readme] |
|
33 | 32 |
|
34 | 33 | The current development version the API documentation in this repo can be found [here](https://arrow.apache.org/rust).
|
35 | 34 |
|
| 35 | +Note: previously the [`object_store`] crate was also part of this repository, |
| 36 | +but it has been moved to the [arrow-rs-object-store repository] |
| 37 | + |
36 | 38 | [apache arrow]: https://arrow.apache.org/
|
37 | 39 | [`arrow`]: https://crates.io/crates/arrow
|
38 | 40 | [`parquet`]: https://crates.io/crates/parquet
|
39 | 41 | [`parquet_derive`]: https://crates.io/crates/parquet-derive
|
40 | 42 | [`arrow-flight`]: https://crates.io/crates/arrow-flight
|
41 |
| -[`object-store`]: https://crates.io/crates/object-store |
| 43 | +[arrow-rs-object-store repository]: https://github.com/apache/arrow-rs-object-store |
42 | 44 |
|
43 | 45 | ## Release Versioning and Schedule
|
44 | 46 |
|
45 |
| -### `arrow` and `parquet` crates |
46 |
| - |
47 | 47 | The Arrow Rust project releases approximately monthly and follows [Semantic
|
48 | 48 | Versioning].
|
49 | 49 |
|
@@ -72,21 +72,6 @@ Planned Release Schedule
|
72 | 72 | [ticket #5368]: https://github.com/apache/arrow-rs/issues/5368
|
73 | 73 | [semantic versioning]: https://semver.org/
|
74 | 74 |
|
75 |
| -### `object_store` crate |
76 |
| - |
77 |
| -The [`object_store`] crate is released independently of the `arrow` and |
78 |
| -`parquet` crates and follows [Semantic Versioning]. We aim to release new |
79 |
| -versions approximately every 2 months. |
80 |
| - |
81 |
| -[`object_store`]: https://crates.io/crates/object_store |
82 |
| - |
83 |
| -Planned Release Schedule |
84 |
| - |
85 |
| -| Approximate Date | Version | Notes | |
86 |
| -| ---------------- | -------- | --------------------------------------- | |
87 |
| -| Feb 2025 | `0.12.0` | Major, potentially breaking API changes | |
88 |
| -| Apr 2025 | `0.12.1` | Minor, NO breaking API changes | |
89 |
| - |
90 | 75 | ### Guidelines for `panic` vs `Result`
|
91 | 76 |
|
92 | 77 | In general, use panics for bad states that are unreachable, unrecoverable or harmful.
|
@@ -128,20 +113,18 @@ maintainers.
|
128 | 113 |
|
129 | 114 | There are several related crates in different repositories
|
130 | 115 |
|
131 |
| -| Crate | Description | Documentation | |
132 |
| -| ------------------------ | ------------------------------------------- | --------------------------------------- | |
133 |
| -| [`datafusion`] | In-memory query engine with SQL support | [(README)][datafusion-readme] | |
134 |
| -| [`ballista`] | Distributed query execution | [(README)][ballista-readme] | |
135 |
| -| [`object_store_opendal`] | Use [`opendal`] as [`object_store`] backend | [(README)][object_store_opendal-readme] | |
136 |
| -| [`parquet_opendal`] | Use [`opendal`] for [`parquet`] Arrow IO | [(README)][parquet_opendal-readme] | |
| 116 | +| Crate | Description | Documentation | |
| 117 | +| ------------------- | ------------------------------------------------------------ | ---------------------------------- | |
| 118 | +| [`object_store`] | Object Storage (aws, azure, gcp, local, in-memory) interface | [(README)](object_store-readme) | |
| 119 | +| [`datafusion`] | In-memory query engine with SQL support | [(README)][datafusion-readme] | |
| 120 | +| [`ballista`] | Distributed query execution | [(README)][ballista-readme] | |
| 121 | +| [`parquet_opendal`] | Use [`opendal`] for [`parquet`] Arrow IO | [(README)][parquet_opendal-readme] | |
137 | 122 |
|
138 | 123 | [`datafusion`]: https://crates.io/crates/datafusion
|
139 | 124 | [`ballista`]: https://crates.io/crates/ballista
|
140 |
| -[`object_store_opendal`]: https://crates.io/crates/object_store_opendal |
141 |
| -[`opendal`]: https://crates.io/crates/opendal |
142 |
| -[object_store_opendal-readme]: https://github.com/apache/opendal/blob/main/integrations/object_store/README.md |
143 | 125 | [`parquet_opendal`]: https://crates.io/crates/parquet_opendal
|
144 | 126 | [parquet_opendal-readme]: https://github.com/apache/opendal/blob/main/integrations/parquet/README.md
|
| 127 | +[object_store-readme]: https://github.com/apache/arrow-rs-object-store/blob/main/README.md |
145 | 128 |
|
146 | 129 | Collectively, these crates support a wider array of functionality for analytic computations in Rust.
|
147 | 130 |
|
@@ -170,18 +153,18 @@ a great place to meet other contributors and get guidance on where to contribute
|
170 | 153 | The Rust implementation uses [GitHub issues][issues] as the system of record for new features and bug fixes and
|
171 | 154 | this plays a critical role in the release process.
|
172 | 155 |
|
173 |
| -For design discussions we generally collaborate on Google documents and file a GitHub issue linking to the document. |
| 156 | +For design discussions we generally use GitHub issues. |
174 | 157 |
|
175 | 158 | There is more information in the [contributing] guide.
|
176 | 159 |
|
177 | 160 | [rust]: https://www.rust-lang.org/
|
| 161 | +[`object_store`]: https://crates.io/crates/object-store |
178 | 162 | [arrow-readme]: arrow/README.md
|
179 | 163 | [contributing]: CONTRIBUTING.md
|
180 | 164 | [parquet-readme]: parquet/README.md
|
181 | 165 | [flight-readme]: arrow-flight/README.md
|
182 | 166 | [datafusion-readme]: https://github.com/apache/datafusion/blob/main/README.md
|
183 | 167 | [ballista-readme]: https://github.com/apache/datafusion-ballista/blob/main/README.md
|
184 |
| -[objectstore-readme]: object_store/README.md |
185 | 168 | [parquet-derive-readme]: parquet_derive/README.md
|
186 | 169 | [issues]: https://github.com/apache/arrow-rs/issues
|
187 | 170 | [discussions]: https://github.com/apache/arrow-rs/discussions
|
0 commit comments