Releases · StarRocks/starrocks-connector-for-apache-spark

What's Changed

This release mainly includes some features and improvements.

Features

Support Apache Spark 3.5 #89

Support "starrocks.filter.query" for Spark SQL read #92

Support to read json column #100

Improvements

Refine the error log if starrocks.columns contains a non-existed column #97

RestService logs response status and entity for debug #98

add output col names test #99

What's Changed

This release mainly includes some features and improvements for loading data to StarRocks.

NOTICE

Take note of the some changes when you upgrade the Spark connector to this version. For details, see Upgrade Spark connector.

Features

Improvements

Remove useless dependency, and make the Spark connector JAR file lightweight. #55 #57
Replace fastjson with jackson. #58
Add the missing Apache license header. #60
Do not package the MySQL JDBC driver in the Spark connector JAR file. #63
Support to configure timezone parameter and become compatible with Spark Java8 API datetime. #64
Optimize row-string converter to reduce CPU costs. #68
The starrocks.fe.http.url parameter supports to add a http scheme. #71
The interface BatchWrite#useCommitCoordinator is implemented to run on DataBricks 13.1 #79
Add the hint of checking the privileges and parameters in the error log. #81

Bug fixes

Parse escape characters in the CSV related parameters column_seperator and row_delimiter. #85

Doc

Refactor the docs. #66
Add examples of load data to BITMAP and HLL columns. #70
Add examples of Spark applications written in Python. #72
Add examples of loading ARRAY-type data. #75
Add examples for performing partial updates and conditional updates on Primary Key tables. #80

scutzou, hellolilyliuyi, banmoy