BACKPORT-TO-STABLE: Backported fixes for Drill 1.21.2 #2860

jnturton · 2023-12-31T06:14:06Z

[BACKPORT-TO-STABLE]: fixes for Drill 1.21.2

Description

Fixes and minor enhancements since Drill 1.21.1 backported from the master branch.

cgivre · 2024-01-01T14:30:28Z

@jnturton
Did we add #2795 to this?

jnturton · 2024-01-02T05:27:43Z

@jnturton Did we add #2795 to this?

@cgivre yes, it's here.

luocooong

LGTM +1

…e#2796)

… the ElasticSearch plugin (apache#2795)

* chore(deps): Update Yauaa to 7.19.2 * Additional fix, please squash on merge

Bumps [h2](https://github.com/h2database/h2database) from 2.1.210 to 2.2.220. - [Release notes](https://github.com/h2database/h2database/releases) - [Commits](h2database/h2database@version-2.1.210...version-2.2.220) --- updated-dependencies: - dependency-name: com.h2database:h2 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…e#2818)

…pache#2820)

…apache#2822) * Shut down MiniDFS cluster in TestInboundImpersonation.selectChainedView. * Ensure that MiniDFS is using simple auth in BaseTestImpersonation.

…/rdbms-metastore (apache#2812)

…pache#2839) * Add a 2GB swap file when setting up the Actions Runner. * Remove /proc/sys/vm/drop_caches snake oil. * Tune the heap and direct memory limits applied to the test suite.

…ding Parquet v2 data page (apache#2838)

…apache#2861) The EasyOutOfMemory test category is a clever workaround to run Drill unit tests in the memory constrained CI environment provided by Github Actions that has served us well but that is no longer needed now that we set up swap space in the CI workflow. We can retire it for simpler and faster CI testing.

This increase is not applicable to master where the removal of Drill's shaded Guava has shrunk the JDBC driver JAR size.

* Update Netty to 4.1.101

…s_override_file action (apache#2859)

* Upgrade Janino 3.1.8 → 3.1.11. * Update classb/ and 3rdparty/ lib separation for jakarta packages.

apache#2865) * Upgrade ZooKeeper 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0

cgivre · 2024-01-03T18:52:59Z

@jnturton
I'd like to include DRILL-8470, and possibly DRILL-8415 and DRILL-8465 in this. What do you think? I'm on the fence about DRILL-8465 as that does seem like more of a new feature.

* Upgrade MongoDB driver 4.3.3 → 4.11.1 * Upgrade DeltaLake 2.1.0 → 3.0.0 * Upgrade Metadata-Extractor 2.18.0 → 2.19.0

Allows the readers using the EVF layer to write to UNION and LIST vectors whether or not those vectors are projected by the query. EVF creates "dummy" vectors and writers for non-projected columns. This feature previously worked for other column types: this PR adds support for UNION and LIST.

* Upgrade Jackson 2.14.3 → 2.16.1. * Do some list sorting in java-exec's drill-module.conf. * Bump JDBC jar size to 54.5Mb.

jnturton · 2024-01-10T06:39:10Z

@jnturton I'd like to include DRILL-8470, and possibly DRILL-8415 and DRILL-8465 in this. What do you think? I'm on the fence about DRILL-8465 as that does seem like more of a new feature.

8415 and 8470 are here. 8465 is still in analysis and design. We still need the new 8475 (NOTICE file incomplete) that I marked as a blocker and which is being worked on by @CalvinKirs and me.

cgivre · 2024-01-10T14:33:34Z

@jnturton I'd like to include DRILL-8470, and possibly DRILL-8415 and DRILL-8465 in this. What do you think? I'm on the fence about DRILL-8465 as that does seem like more of a new feature.

8415 and 8470 are here. 8465 is still in analysis and design. We still need the new 8475 (NOTICE file incomplete) that I marked as a blocker and which is being worked on by @CalvinKirs and me.

I think we should drop 8465 from 1.21.2. It sounds like once the notices are done we should be ready to cut a RC!

…countered (apache#2874) (apache#2875)

* Update Javadoc instructions in Release.md. * Disable forced pushes to the master branch.

… on affinity node and some on non-affinity node (apache#2885)

…n read data from InputStream (apache#2891)

…n read data from Stream with container (apache#2889)

…exception during the process of building a hash table (apache#2887) (apache#2888)

…parquet reading (apache#2898)

jnturton requested review from luocooong and cgivre December 31, 2023 06:14

jnturton force-pushed the 1.21 branch 4 times, most recently from 3b4694b to 958a2c3 Compare December 31, 2023 14:11

luocooong approved these changes Jan 2, 2024

View reviewed changes

cgivre and others added 21 commits January 3, 2024 10:40

[MINOR UPDATE] Fix Error In HTTP Plugin Docs

1c3a44c

DRILL-8426: Fix endless retrying zk set data for a large query (apach…

cd0db53

…e#2796)

DRILL-8385: Add support for disabling SSL certificate verification in…

10d8002

… the ElasticSearch plugin (apache#2795)

DRILL-8428: ElasticSearch Config Missing Getters (apache#2797)

33577e5

[DRILL-8429]: Bump Jackson to 2.14.3 (apache#2798)

0b328b3

DRILL-8430: Create JacksonUtils (apache#2800)

b9152eb

DRILL-8158: Remove non-reproducible entry (apache#2805)

351ed56

DRILL-8438: Bump YAUAA to 7.19.2 (apache#2808)

20df284

* chore(deps): Update Yauaa to 7.19.2 * Additional fix, please squash on merge

Update bug_report.md

e05b2c2

DRILL-8442: Fix DeltaRowGroupScan deserialization

655eea4

DRILL-8449: fix typo in width property in FreeMarker templates (apach…

257856e

…e#2818)

DRILL-8451: options and profile pages have bad order symbols style (a…

b19a253

…pache#2820)

[MINOR UPDATE]: Fix Hadoop 3 impersonation test errors during CI runs (…

7b5d640

…apache#2822) * Shut down MiniDFS cluster in TestInboundImpersonation.selectChainedView. * Ensure that MiniDFS is using simple auth in BaseTestImpersonation.

DRILL-8452: Library upgrades (apache#2823)

822801b

[MINOR UPDATE] Bump sqlite-jdbc from 3.30.1 to 3.41.2.2 in /metastore…

bfa10a5

…/rdbms-metastore (apache#2812)

DRILL-8456: Upgrade POI and Commons-Compress (apache#2833)

f6d1644

[MINOR UPDATE]: Improve the Github CI memory constraint workarounds (a…

ddb3c85

…pache#2839) * Add a 2GB swap file when setting up the Actions Runner. * Remove /proc/sys/vm/drop_caches snake oil. * Tune the heap and direct memory limits applied to the test suite.

DRILL-8459 Bump Avro to 1.11.3 (apache#2841)

ac71ec0

DRILL-8458: Use correct size of definition level bytes slice when rea…

3f0d530

…ding Parquet v2 data page (apache#2838)

DRILL-8461: Prevent XXE Attacks in XML Format Plugin (apache#2845)

dbb7ffb

pjfanning and others added 5 commits January 3, 2024 10:40

DRILL-8466. upgrade to logback 1.3.14 (apache#2854)

40af40c

Increase jdbc-all-jar-maxsize to 57000000 for the hadoop-2 profile.

8f3a298

This increase is not applicable to master where the removal of Drill's shaded Guava has shrunk the JDBC driver JAR size.

DRILL-8467: Update Netty to 4.1.101 (apache#2857)

19afef6

* Update Netty to 4.1.101

DRILL-8468: Drill doesn't perform drill.exec.storage.action_on_plugin…

78ace46

…s_override_file action (apache#2859)

jnturton force-pushed the 1.21 branch from 9f6080c to 78ace46 Compare January 3, 2024 08:42

jnturton added 2 commits January 3, 2024 16:19

DRILL-8445: Upgrade Janino 3.1.8 → 3.1.11 (apache#2864)

6c83fc6

* Upgrade Janino 3.1.8 → 3.1.11. * Update classb/ and 3rdparty/ lib separation for jakarta packages.

DRILL-8460: Upgrade ZooKeeper 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 (

649c32c

apache#2865) * Upgrade ZooKeeper 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0

cgivre assigned jnturton Jan 3, 2024

cgivre and others added 4 commits January 5, 2024 08:13

DRILL-8470: Bump MongoDB Driver to Latest Version (apache#2862)

659fe97

* Upgrade MongoDB driver 4.3.3 → 4.11.1 * Upgrade DeltaLake 2.1.0 → 3.0.0 * Upgrade Metadata-Extractor 2.18.0 → 2.19.0

DRILL-8415: Upgrade Jackson 2.14.3 → 2.16.1 (apache#2866)

f091162

* Upgrade Jackson 2.14.3 → 2.16.1. * Do some list sorting in java-exec's drill-module.conf. * Bump JDBC jar size to 54.5Mb.

[MINOR UPDATE]: Update the copyright year in NOTICE (apache#2868)

77f24b8

DRILL-8478. HashPartition memory leak when OutOfMemoryException is en…

5e44632

…countered (apache#2874) (apache#2875)

jnturton force-pushed the 1.21 branch from 77aa125 to 5e44632 Compare January 23, 2024 15:11

jnturton and others added 11 commits April 12, 2024 16:54

[MINOR UPDATE]: Enable master branch protection (apache#2872)

611fb43

* Update Javadoc instructions in Release.md. * Disable forced pushes to the master branch.

DRILL-8475: Update the binary distributions LICENSE (apache#2879)

e392327

Fix typos in LicenseHeadersAndNotices.md (apache#2883)

e653567

DRILL-8482:Assign region throw exception when some region is deployed…

9eddd1a

… on affinity node and some on non-affinity node (apache#2885)

DRILL-8141: Ability to query XML root attributes (apache#2884)

7db6140

DRILL-8485: HashJoinPOP memory leak is caused by an oom exception whe…

5a377d3

…n read data from InputStream (apache#2891)

[MINOR UPDATE] Upgrade Actions (apache#2894)

36f0ccc

[MINOR UPDATE] Update Commons-Compress due to CVEs (apache#2892)

298f599

DRILL-8484: HashJoinPOP Memory Leak is Caused by an OOM Exception whe…

f0c07cf

…n read data from Stream with container (apache#2889)

DRILL-8483: SpilledRecordBatch memory leak when the program threw an …

264537f

…exception during the process of building a hash table (apache#2887) (apache#2888)

DRILL-8486: fix handling of long variable length entries during bulk …

0e6b21d

…parquet reading (apache#2898)

jnturton merged commit 0e6b21d into apache:1.21 Apr 12, 2024
0 of 5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BACKPORT-TO-STABLE: Backported fixes for Drill 1.21.2 #2860

BACKPORT-TO-STABLE: Backported fixes for Drill 1.21.2 #2860

jnturton commented Dec 31, 2023 •

edited

Loading

cgivre commented Jan 1, 2024

jnturton commented Jan 2, 2024

luocooong left a comment

cgivre commented Jan 3, 2024

jnturton commented Jan 10, 2024

cgivre commented Jan 10, 2024

BACKPORT-TO-STABLE: Backported fixes for Drill 1.21.2 #2860

BACKPORT-TO-STABLE: Backported fixes for Drill 1.21.2 #2860

Conversation

jnturton commented Dec 31, 2023 • edited Loading

[BACKPORT-TO-STABLE]: fixes for Drill 1.21.2

Description

cgivre commented Jan 1, 2024

jnturton commented Jan 2, 2024

luocooong left a comment

Choose a reason for hiding this comment

cgivre commented Jan 3, 2024

jnturton commented Jan 10, 2024

cgivre commented Jan 10, 2024

jnturton commented Dec 31, 2023 •

edited

Loading