Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark Doris Connector Release Note 1.3.0 #159

Open
JNSimba opened this issue Nov 9, 2023 · 3 comments
Open

Spark Doris Connector Release Note 1.3.0 #159

JNSimba opened this issue Nov 9, 2023 · 3 comments

Comments

@JNSimba
Copy link
Member

JNSimba commented Nov 9, 2023

Feature & Improvement

  1. Support Spark3.3, 3.4
  2. DorisWriter write memory optimization [optimize][refractor] Optimizing memory usage for writing data #140
  3. Supports Map, Struct reading and writing, and Array type writing
  4. Support adding hidden delimiters when writing in CSV format
  5. Write supports two-phase commit [feature] support two phase commit #122
  6. Add auto-redirect configuration to write data through direct connection fe
  7. Writing supports OverWrite mode
  8. Structured Streaming supports Row format DataFrame writing doris
  9. Optimize some log output

Bug

  1. Fixed the issue where doris.filter.query pushdown does not take effect in some scenarios
  2. Fix the problem of writing null pointer to String type
  3. Fix Structured Streaming writing exception problem

Thanks

@CodeCooker17
@daikon12
@gnehil
@huanccwang
@JNSimba
@shoothzj
@wolfboys

@JNSimba JNSimba changed the title [Enhancement] Release Note 1.3.0 Nov 9, 2023
@JNSimba JNSimba changed the title Release Note 1.3.0 Spark Doris Connector Release Note 1.3.0 Nov 9, 2023
@JNSimba JNSimba pinned this issue Nov 9, 2023
@cjj2010
Copy link

cjj2010 commented Nov 29, 2023

Feature & Improvement

  1. Support Spark3.3, 3.4
  2. DorisWriter write memory optimization [optimize][refractor] Optimizing memory usage for writing data #140
  3. Supports Map, Struct reading and writing, and Array type writing
  4. Support adding hidden delimiters when writing in CSV format
  5. Write supports two-phase commit [feature] support two phase commit #122
  6. Add auto-redirect configuration to write data through direct connection fe
  7. Writing supports OverWrite mode
  8. Structured Streaming supports Row format DataFrame writing doris
  9. Optimize some log output

Bug

  1. Fixed the issue where doris.filter.query pushdown does not take effect in some scenarios
  2. Fix the problem of writing null pointer to String type
  3. Fix Structured Streaming writing exception problem

Thanks

@CodeCooker17 @daikon12 @gnehil @huanccwang @JNSimba @shoothzj @wolfboys

I see that the latest version on sonatype is 1.2.0. When will 1.3.0 be officially released
image

@milton4code
Copy link

Feature & Improvement

  1. Support Spark3.3, 3.4
  2. DorisWriter write memory optimization [optimize][refractor] Optimizing memory usage for writing data #140
  3. Supports Map, Struct reading and writing, and Array type writing
  4. Support adding hidden delimiters when writing in CSV format
  5. Write supports two-phase commit [feature] support two phase commit #122
  6. Add auto-redirect configuration to write data through direct connection fe
  7. Writing supports OverWrite mode
  8. Structured Streaming supports Row format DataFrame writing doris
  9. Optimize some log output

Bug

  1. Fixed the issue where doris.filter.query pushdown does not take effect in some scenarios
  2. Fix the problem of writing null pointer to String type
  3. Fix Structured Streaming writing exception problem

Thanks

@CodeCooker17 @daikon12 @gnehil @huanccwang @JNSimba @shoothzj @wolfboys

I see that the latest version on sonatype is 1.2.0. When will 1.3.0 be officially released image

I am the same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants