Skip to content

Commit d93cf79

Browse files
Add roadmap to readme (#1616)
* Add roadmap to readme * Link to datafusion-contrib * Update multi column comparisons
1 parent a96bb5e commit d93cf79

File tree

1 file changed

+54
-0
lines changed

1 file changed

+54
-0
lines changed

README.md

+54
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,60 @@ datafusion = "6.0.0"
141141

142142
DataFusion also includes a simple command-line interactive SQL utility. See the [CLI reference](https://arrow.apache.org/datafusion/cli/index.html) for more information.
143143

144+
# Roadmap
145+
146+
A quarterly roadmap will be published to give the DataFusion community visibility into the priorities of the projects contributors. This roadmap is not binding.
147+
148+
## 2022 Q1
149+
150+
### DataFusion Core
151+
152+
- Publish official Arrow2 branch
153+
- Implementation of memory manager (i.e. to enable spilling to disk as needed)
154+
155+
### Benchmarking
156+
157+
- Inclusion in Db-Benchmark with all quries covered
158+
- All TPCH queries covered
159+
160+
### Performance Improvements
161+
162+
- Predicate evaluation
163+
- Improve multi-column comparisons (that can't be vectorized at the moment)
164+
- Null constant support
165+
166+
### New Features
167+
168+
- Read JSON as table
169+
- Simplify DDL with Datafusion-Cli
170+
- Add Decimal128 data type and the attendant features such as Arrow Kernel and UDF support
171+
- Add new experimental e-graph based optimizer
172+
173+
### Ballista
174+
175+
- Begin work on design documents and plan / priorities for development
176+
177+
### Extensions ([datafusion-contrib](https://github.com/datafusion-contrib]))
178+
179+
- Stable S3 support
180+
- Begin design discussions and prototyping of a stream provider
181+
182+
## Beyond 2022 Q1
183+
184+
There is no clear timeline for the below, but community members have expressed interest in working on these topics.
185+
186+
### DataFusion Core
187+
188+
- Custom SQL support
189+
- Split DataFusion into multiple crates
190+
- Push based query execution and code generation
191+
192+
### Ballista
193+
194+
- Evolve architecture so that it can be deployed in a multi-tenant cloud native environment
195+
- Ensure Ballista is scalable, elastic, and stable for production usage
196+
- Develop distributed ML capabilities
197+
144198
# Status
145199

146200
## General

0 commit comments

Comments
 (0)