v1.2.659-nightly

What's Changed

Exciting New Features ✨

• feat(query): Support rename dictionary ddl. by @Winnie-Hong0927 in #16754
• feat(query): add last_day, previous_day, next_day function by @Freejww in #16740
• feat: implement StringColumn using StringViewArray by @andylokandy in #16610
• feat: Implement WAL-based RaftLog storage by @drmingdrmer in #16776
• feat(query): expand trim/ltrim/rtrim by @TCeason in #16802
• feat: support pushdown predicate into iceberg engine by @sundy-li in #16650
• feat(query): TopN window operator by @forsaken628 in #16726
• feat(query): support SHOW DROP DATABASES by @TCeason in #16811
• feat: uuid function from v4 to v7 by @BohuTANG in #16827
• feat: refine the rule PushDownFilterWindowTopN in case top n is equal to 0. by @forsaken628 in #16830

Thoughtful Bug Fix 🔧

• fix: token and cookie session id miss match. by @youngsofun in #16786
• fix: clustering_information panic if string domain is none by @zhyass in #16792
• fix(query): fix broken file log layout by @zhang2014 in #16803
• fix: re-ensure dir before upgrading by @drmingdrmer in #16805
• fix: purge may not work on tables after a flash back operation by @SkyFan2002 in #16812
• fix(storage): recluster endless loop by @zhyass in #16831
• fix: incorrect table data disk cache key by @dantengsky in #16837
• fix(query): fix group by with alias column can't bind the column by @b41sh in #16804
• fix(cluster): fix the possibility of connection leak when the cluster state is broken. by @zhang2014 in #16842
• fix(query): fix aggregator_groups_builder to work with string view by @sundy-li in #16843
• fix(query): check storage request method with arg list-type by @everpcpc in #16849
• fix: replace risky dma_buffer_as_vec implementations by @forsaken628 in #16829
• fix(query): fix and check total_buffer_len and total_bytes_len by @sundy-li in #16854

Code Refactor 🎉

• refactor(base): add stacktrace to replace backtrace by @zhang2014 in #16643
• refactor: when upgrading, do not import already purged garbage log entries by @drmingdrmer in #16809
• refactor: align metrics field names with current behavior by @drmingdrmer in #16826
• refactor(base): support stacktrace in cluster mode by @zhang2014 in #16821
• refactor: refactor cte binder and fix materialized cte used in subquery by @xudong963 in #16785

Build/Testing/CI Infra Changes 🔌

• ci: Enable incremental back to see how it works by @Xuanwo in #16817

Documentation 📔

• docs: add stream to readme by @BohuTANG in #16810

Others 📒

• chore: remove obsolete version by @everpcpc in #16789
• chore: add snapshot location cache for attached table by @dantengsky in #16795
• chore: update meta-service version info and compatibility doc by @drmingdrmer in #16800
• chore(query): embedded public key of license by @zhang2014 in #16813
• chore(query): refactor common/arrow removes arrow/io crates by @sundy-li in #16808
• chore: purge dropped view metadata during vacuum by @dantengsky in #16819
• chore(query): keep queries detail and profile log format by @zhang2014 in #16822
• chore(ci): test with different license by @everpcpc in #16815
• chore(base): rollback to kernel signal handler after dump backtrace by @zhang2014 in #16828
• chore: remove unused codes by @zhyass in #16834
• chore: fix typo in ColumnCacheKeyBuilder by @dantengsky in #16838
• chore(cluster): disable parallel commit of cluster tasks by @zhang2014 in #16851
• chore(query): remove SliceExt by @sundy-li in #16853
• chore(ci): separate build profile by @everpcpc in #16855

Full Changelog: v1.2.654-nightly...v1.2.659-nightly

v1.2.654-nightly

What's Changed

Exciting New Features ✨

• feat(query): function about convert_timezone #16177 by @florann in #16181
• feat(functions): add new function: map_pick by @hanxuanliang in #15573
• feat(metrics): new storage_http_requests_count by @everpcpc in #16769
• feat(functions): add map_insert function by @hanxuanliang in #15567
• feat: filter null value before join by @xudong963 in #16722

Thoughtful Bug Fix 🔧

• fix: Files::remove_file_in_batch should not swallow errors by @dantengsky in #16761
• fix(query): add tls config for udf client by @everpcpc in #16782

Code Refactor 🎉

• refactor: change blocking pruning process to non-blocking channel by @dqhl76 in #16718
• refactor: ensure last-purged log-id cursor in exported from databend-meta service by @drmingdrmer in #16759
• refactor: Consolidate MetaStorageError variants by @drmingdrmer in #16760
• refactor(storage): refactor read virtual columns by @b41sh in #16758
• refactor: remove location_prefix, which is no longer supported by @dantengsky in #16757

Others 📒

• chore: make VACUUM TEMPORARY FILES killable by @dantengsky in #16751
• chore: tweak batch deletion concurrency by @dantengsky in #16770
• chore: refine raft-types import path by @drmingdrmer in #16764

New Contributors

• @florann made their first contribution in #16181

Full Changelog: v1.2.653-nightly...v1.2.654-nightly

v1.2.653-nightly

What's Changed

Exciting New Features ✨

• feat: HTTP handler use cookie to pass session id to and from clients. by @youngsofun in #16735
• feat: impl opendal::HttpFetch for StorageHttpClient by @everpcpc in #16738

Thoughtful Bug Fix 🔧

• fix: date_add/sub need respect timezone by @TCeason in #16721
• fix(query): remove single quotes for system.columns comment by @TCeason in #16732
• fix: HTTP handle lost rows when result set is wide and large. by @youngsofun in #16748
• fix: keep column statistics of all NULL column by @dantengsky in #16753

Code Refactor 🎉

• refactor: Enforce all deps must be declared at root by @Xuanwo in #16741
• refactor: Remove not used async-std by @Xuanwo in #16745

Others 📒

• chore: tweak cargo feature "jemalloc" by @dantengsky in #16693
• chore: remove flaky test by @TCeason in #16739
• chore: refine vacuum by @dantengsky in #16743
• chore: Remove not used dep config by @Xuanwo in #16746
• chore(query): Introduce VecExt and SliceExt to check unsafe bounds in debug mode by @sundy-li in #16747

Full Changelog: v1.2.652-nightly...v1.2.653-nightly

v1.2.652-nightly

What's Changed

Exciting New Features ✨

• feat: support map lambda functions by @Dragonliu2018 in #16683
• feat(query): add api /v1/verify by @everpcpc in #16665
• feat: support calculating table storage size(including time travel) by @SkyFan2002 in #16698
• feat(query): add table function udf_echo && expose more metrics of udf client by @sundy-li in #16696
• feat(query): Settings clause by @TCeason in #16669
• feat: implement is_not_null selectivity based on null count in stats by @xudong963 in #16730

Thoughtful Bug Fix 🔧

• fix: JEAllocator should properly handle nullptr returned by @dantengsky in #16692
• fix(metrics): use f64::MAX as histogram bound to display +Inf properly by @flaneur2020 in #16701
• fix(query): use custom connector for udf client by @everpcpc in #16697
• fix: MetaGrpcClient deadlock when drop by @drmingdrmer in #16727

Code Refactor 🎉

• refactor: streamline system.temp_files table scan by @dantengsky in #16659
• refactor: Align versions for arrow, opendal, hyper, tonic, prost by @Xuanwo in #16704
• refactor: refine column statistic map serialization by @dantengsky in #16728
• refactor: MetaGrpcClient needs to specify timeout to auto-reconnect to restarted server by @drmingdrmer in #16733
• refactor: replace direct GitHub dependencies with published crates.io versions by @drmingdrmer in #16700

Others 📒

• chore: display port number on error from bind to port failure by @lyang24 in #16695
• chore: remove SortColumnDescription.is_nullable by @forsaken628 in #16699
• chore: remove transaction keyword by @SkyFan2002 in #16707
• chore: add timeout for detect_region operation to prevent hanging request by @dantengsky in #16709
• chore(query): add current_query_id and database id for admin api by @zhang2014 in #16703
• chore(query): index scalar support variant by @sundy-li in #16719
• chore: display log.file.limit and prefix_filter by @TCeason in #16713
• chore: avoid returning error during computing binary comparison selectivity by @xudong963 in #16725
• chore: Simplifying BlockMetaInfo Definitions by @forsaken628 in #16716

New Contributors

• @lyang24 made their first contribution in #16695

Full Changelog: v1.2.651-nightly...v1.2.652-nightly

v1.2.651-nightly

What's Changed

Exciting New Features ✨

• feat: Bump OpenDAL to 0.50.1 by @Xuanwo in #16661

Thoughtful Bug Fix 🔧

• fix(query): sort spilling may hang by @forsaken628 in #16672

Code Refactor 🎉

• refactor: refine cast variant to map by @andylokandy in #16691

Full Changelog: v1.2.650-nightly...v1.2.651-nightly

v1.2.650-nightly

What's Changed

Exciting New Features ✨

• feat: support explain decorrelated plan by @xudong963 in #16681
• feat: add ARRAY_SIZE alias for ARRAY_LENGTH by @ding-young in #16685
• feat(query): support json_object_delete and json_object_pick function by @b41sh in #16682

Thoughtful Bug Fix 🔧

• fix(ci): flaky test by @zhyass in #16664
• fix: insert long sql again by @sundy-li in #16684

Others 📒

• chore: upon startup, meta-service output human-readable error if trying to open future version on-disk data by @drmingdrmer in #16679
• chore(query): modify rbac test result display by @TCeason in #16686
• chore: adjust default value of setting max_storage_io_requests by @SkyFan2002 in #16680
• chore(query): returns nan for aggregate_kurtosis and aggregate_skewness by @sundy-li in #16688

New Contributors

• @ding-young made their first contribution in #16685

Full Changelog: v1.2.649-nightly...v1.2.650-nightly

v1.2.649-nightly

What's Changed

Exciting New Features ✨

• feat: add deterministic block level sampling for small datasets by @xudong963 in #16670

Thoughtful Bug Fix 🔧

• fix(query): create table need fail if storage format is invalid by @TCeason in #16663
• fix(query): forbid explain explain statement by @TCeason in #16654
• fix(storage): fix refresh virtual column using async api by @b41sh in #16656
• fix(query): insert with long sql parse error by @sundy-li in #16678

Code Refactor 🎉

• refactor(query): use arrow-flight to exchange datas by @sundy-li in #16657

Others 📒

• chore(query): remove hash join build detail status by @zhang2014 in #16674

Full Changelog: v1.2.648-nightly...v1.2.649-nightly

v1.2.648-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix(query): check_timestamp/date -> clamp_timestamp/date by @TCeason in #16634
• fix: deadlock in table lock by @zhyass in #16632
• fix(query): sort spilling use arrow file format painc by @forsaken628 in #16658

Code Refactor 🎉

• refactor: delete user_procedure and refactor procedure interpreter by @TCeason in #16532

Others 📒

• chore(query): add warn log if vacuum temporary file is error by @zhang2014 in #16646
• chore: resolve todo in dynamic sample by @xudong963 in #16629
• chore(query): enable loser tree merge sort default by @forsaken628 in #16649
• chore(query): replace arrow2 with arrow-rs in serialize_column and deserialize_column by @forsaken628 in #16655

Full Changelog: v1.2.647-nightly...v1.2.648-nightly

v1.2.647-nightly

What's Changed

Exciting New Features ✨

• feat: support sampling table with block and row level simultaneously by @xudong963 in #16613
• feat: add unload options overwrite, include_query_id and use_raw_path. by @youngsofun in #16614
• feat(function): Add mode Aggregate function. by @Freejww in #16627
• feat(query): Support use parquet format when spilling by @forsaken628 in #16612
• feat(query): support json_object_insert function by @b41sh in #16636
• feat: http handler support request forwarding. by @youngsofun in #16637
• feat: Implemented a new sql explain analyze graphical by @Maricaya in #16543

Thoughtful Bug Fix 🔧

• fix: lock mgr may failed to release lock in time by @dantengsky in #16611
• fix(query): support subquery in pivot by @Dragonliu2018 in #16631

Code Refactor 🎉

• refactor: Unify the set and unset structures by @TCeason in #16622
• refactor(query): make delta table support partition prunning by @sundy-li in #16621
• refactor(storage): inverted index support calculate the score by @b41sh in #16609
• refactor: some http APIs do not need sticky. by @youngsofun in #16644

Others 📒

• chore: reset recluster block size by @zhyass in #16623
• chore(query): add http address info to cluster node info by @zhang2014 in #16620
• chore: drop guard when acquire lock failed by @zhyass in #16616
• chore: revert “lock mgr may failed to release lock in time (#16611)" by @dantengsky in #16630
• chore(query): reduce unsafe codes in kernels by @sundy-li in #16633

New Contributors

• @Dragonliu2018 made their first contribution in #16631
• @Maricaya made their first contribution in #16543

Full Changelog: v1.2.646-nightly...v1.2.647-nightly

v1.2.646-nightly

What's Changed

Exciting New Features ✨

• feat(query): Reduce serialisation memory usage when spilling to local disk by @forsaken628 in #16580
• feat(query): Support show dictionaries DQL. by @Winnie-Hong0927 in #16602

Thoughtful Bug Fix 🔧

• fix: subquery returns wrong results under cluster by @xudong963 in #16606
• fix(query): ntile frame should be unbounded by @TCeason in #16604

Code Refactor 🎉

• refactor: isolate session_id and session states of diff users. by @youngsofun in #16592

Others 📒

• chore: refine table sample tests by @xudong963 in #16600
• chore(query): add more logs on aggregation by @sundy-li in #16552

Full Changelog: v1.2.645-nightly...v1.2.646-nightly