v0.3.0

What's Changed

Features

• feat(datanode): iImplement the heartbeat response handler by @WenyXu in #1547
• feat(mito): Combine the original and procedure's implementation by @evenyag in #1468
• feat(servers): Add metrics based on axum's example by @evenyag in #1638
• feat(servers): collect samples by metric by @evenyag in #1706
• feat(tql): add initial implementation for explain & analyze by @etolbakov in #1427
• feat: Add FlushPicker to flush regions periodically by @evenyag in #1559
• feat: Add a global TTL option for all tables by @evenyag in #1679
• feat: Add app version metric by @evenyag in #1685
• feat: Add column supports at first or after the existing columns by @lizhemingi in #1621
• feat: Add metrics for cache hit/miss for object store cache by @nearsyh in #1405
• feat: Add request type and result code to grpc metrics by @evenyag in #1664
• feat: Compress manifest and checkpoint by @Taylor-lagrange in #1497
• feat: Drop table by procedure by @evenyag in #1401
• feat: Log error in GreptimeRequestHandler by @evenyag in #1507
• feat: Record process metrics by @evenyag in #1646
• feat: Remove store from procedure config by @evenyag in #1489
• feat: Switch to the procedure framework by @evenyag in #1448
• feat: Trigger flush based on global write buffer size by @evenyag in #1585
• feat: Use LocalScheduler framework to implement FlushScheduler by @evenyag in #1531
• feat: acquire table_id from region_id by @WenyXu in #1656
• feat: adapt window reader to order rules by @v0y4g3r in #1671
• feat: add WindowedReader by @v0y4g3r in #1532
• feat: add cache for catalog kv backend by @Fengys123 in #1592
• feat: add catalog/schema/table count as catalog metrics by @sunng87 in #1499
• feat: add delete WAL in drop_region by @DevilExileSu in #1577
• feat: add information_schema as exception of cross schema check by @sunng87 in #1551
• feat: add invalidate cache step by @WenyXu in #1658
• feat: add invalidate table cache handler by @WenyXu in #1633
• feat: add metrics for ingested row count by @sunng87 in #1645
• feat: add metrics for protocol interfaces by @sunng87 in #1495
• feat: add semantic_type to information_schema.columns by @sunng87 in #1530
• feat: add support for information_schema.columns by @sunng87 in #1500
• feat: add table id and engine to information_schema.TABLES by @haohuaijin in #1407
• feat: adopt REPLACE interceptor and quit all processes on exit by @waynexia in #1478
• feat: broadcast with mailbox by @fengjiachun in #1661
• feat: bump opendal to 0.33 by @killme2008 in #1479
• feat: change default selector in meta from "LeaseBased" to "LoadBased" by @Fengys123 in #1598
• feat: clean procedure's state after it is done by @evenyag in #1543
• feat: convert values to vector directly by @waynexia in #1704
• feat: distributed planner basic by @waynexia in #1599
• feat: enable tokio console in cluster mode by @MichaelScofield in #1512
• feat: exists API for KVStore by @fengjiachun in #1695
• feat: expose output_ordering on scan plan by @waynexia in #1425
• feat: ignore mailbox message into stat by @fengjiachun in #1560
• feat: impl instant query and add tests by @waynexia in #1452
• feat: impl instant query interface by @waynexia in #1410
• feat: impl literal only PromQL query by @waynexia in #1641
• feat: implement /api/v1/labels for prometheus by @haohuaijin in #1580
• feat: implement CloseRegionHandler by @WenyXu in #1569
• feat: implement JsonOpener and CsvOpener by @WenyXu in #1367
• feat: implement OpenTableHandler by @WenyXu in #1567
• feat: implement ParquetFileReaderFactory by @WenyXu in #1423
• feat: implement Prometheus-compatible API in gRPC by @waynexia in #1449
• feat: implement COPY for cluster by @MichaelScofield in #1388
• feat: implement parsing format from hashmap by @WenyXu in #1420
• feat: improve and distinguish different errors for IllegalInsertData by @NiwakaDev in #1503
• feat: improve metrics and log level by @sunng87 in #1470
• feat: insert with stream by @fengjiachun in #1703
• feat: make RepeatedTask invoke remove_outdated_meta method by @NiwakaDev in #1578
• feat: make grpc insert requests in a batch by @MichaelScofield in #1687
• feat: make log level and destination configurable from config files by @DevilExileSu in #1444
• feat: manage multiple engine procedure in the engine manager by @evenyag in #1434
• feat: memtable stats by @v0y4g3r in #1591
• feat: merge scan for distributed execution by @waynexia in #1660
• feat: meta metrics by @fengjiachun in #1538
• feat: metasrv mailbox by @fengjiachun in #1481
• feat: meter write request by @shuiyisong in #1447
• feat: metrics for storage engine by @sunng87 in #1574
• feat: migrate substrait to datafusion official implementation by @waynexia in #1238
• feat: more configurable logging levels by @sunng87 in #1630
• feat: open catalogs and schemas in parallel by @gitccl in #1527
• feat: optimizer rule to pass expected output ordering hint by @waynexia in #1675
• feat: prevent sensitive information (key, password, secrets etc.) from being printed in plain by @DevilExileSu in #1501
• feat: procedures for file table engine by @evenyag in #1417
• feat: region failover procedure by @MichaelScofield in #1558
• feat: serialize/deserialize support for PromQL plans by @waynexia in #1684
• feat: show create table by @killme2008 in #1336
• feat: sql dialect for different protocols by @killme2008 in #1631
• feat: support "delete" in distributed mode by @MichaelScofield in #1441
• feat: support /api/v1/label/<label_name>/values from Prometheus by @gitccl in #1604
• feat: support azblob storage. by @QuenKar in #1659
• feat: support creating the physical plan for JSON and CSV files by @WenyXu in #1424
• feat: support frontend-meta heartbeat by @Fengys123 in #1555
• feat: support to copy table from/to CSV and JSON format file by @WenyXu in #1475
• feat: support to create external table by @WenyXu in #1372
• feat: support to create parquet format external table by @WenyXu in #1463
• feat: support to load missing region by @WenyXu in #1651
• feat: support user config manifest compression by @Taylor-lagrange in #1579
• feat: supports CTE query by @killme2008 in #1674
• feat: table metrics by @discord9 in #1469
• feat: time_zone variable for mysql connections by @sunng87 in #1607
• feat: update dashboard to v0.2.2 by @ZonaHex in #1426
• feat: update dashboard to v0.2.3 by @ZonaHex in #1430
• feat: update dashboard to v0.2.4 by @ZonaHex in #1553
• feat: update dashboard to v0.2.5 by @ZonaHex in #1563
• feat: update dashboard to v0.2.6 by @ZonaHex in #1700
• feat: update pgwire to 0.14 by @sunng87 in #1504
• feat: update table metadata in lock by @MichaelScofield in #1634
• feat: use server time if it's not specified by @waynexia in #1480
• feat: window inferer by @v0y4g3r in #1648

Breaking Changes

• feat!: reorganize the storage layout by @killme2008 in #1609

Fixes

• fix(table-procedure): Open table in RegisterCatalog state by @evenyag in #1617
• fix: unwrap() None in NULL value exist multi-field table during prometheus query_range by @haohuaijin in #1571
• fix: Copy from must follow the order of table fields issue by @WenyXu in #1521
• fix: Print source error in subprocedure failure message by @evenyag in #1683
• fix: add log on leader stepdown by @fengjiachun in #1450
• fix: add s3 region info by @WenyXu in #1486
• fix: add s3 region info by @WenyXu in #1492
• fix: allow cross-schema query on information_schema by @sunng87 in #1568
• fix: avoid writing manifest and wal if no files are actually flushed by @v0y4g3r in #1698
• fix: checkpoint GC task also deletes the file with the last version by @evenyag in #1491
• fix: clippy warning by @waynexia in #1669
• fix: correct schema/table count in catalog metrics by @sunng87 in #1565
• fix: disable dashboard by @v0y4g3r in #1494
• fix: do not alias relation before join by @waynexia in #1693
• fix: do not change timestamp index column while planning aggr by @waynexia in #1688
• fix: enable region failover test by @WenyXu in #1699
• fix: execute sql query in another catalog by @Fengys123 in #1457
• fix: failed to insert data with u8 by @Fengys123 in #1701
• fix: fix datanode cannot start while failing to open tables by @WenyXu in #1601
• fix: fix doesn't release closed regions issue by @WenyXu in #1596
• fix: fix type casting issue by @WenyXu in #1652
• fix: fix typo by @WenyXu in #1649
• fix: frontend opt should respect http addr in config file when no com… by @v0y4g3r in #1456
• fix: grpc client keepalive by @v0y4g3r in #1461
• fix: incorrect show create table output by @NiwakaDev in #1514
• fix: insert distributed table if partition column has default value by @NiwakaDev in #1498
• fix: install python3-pip in Dockerfile by @yaoyinnan in #1644
• fix: invalidate route cache on renaming table by @Fengys123 in #1691
• fix: invalidate table route cache by @WenyXu in #1663
• fix: meta admin API get catalog table name error by @paomian in #1603
• fix: no active datanode when frontend start by @Fengys123 in #1533
• fix: object store cache bug by @killme2008 in #1482
• fix: object store caching bug, #1466 by @killme2008 in #1467
• fix: prepare statement doesn't support insert clause by @killme2008 in #1680
• fix: recompile&register scripts as UDF on reboot by @discord9 in #1421
• fix: refreshing Dashboard returns 404 by @MichaelScofield in #1562
• fix: remove region number validation by @WenyXu in #1593
• fix: respect MySQL timestamp format by @v0y4g3r in #1510
• fix: sequence out of range by @fengjiachun in #1597
• fix: sqlness failed due to region failover by @MichaelScofield in #1690
• fix: support restart sqlness in distributed mode by @waynexia in #1443
• fix: table scan without projection by @waynexia in #1404
• fix: the latest number of regions by @fengjiachun in #1546
• fix: uses nextest in the Release CI by @waynexia in #1582
• fix: vector and matrix in Prometheus use different field by @waynexia in #1520
• fix: wal replay ignore manifest entries by @v0y4g3r in #1612
• fix: wrong max_table_id log in remote catalog manager by @haohuaijin in #1516
• fix: wrong next column in manifest by @lizhemingi in #1440

Others

• build: Download assets to cargo output dir by @evenyag in #1476
• build: change release CI to only run test on linux by @waynexia in #1548
• chore(datasource): make CompressionType follow the style of the guide by @WenyXu in #1522
• chore(deps): bump arrow/parquet to 40.0, datafuson to the latest HEAD by @waynexia in #1677
• chore(deps): update substrait dep in client by @waynexia in #1453
• chore(toolchain): update rust-toolchain to 2023-05-03 by @waynexia in #1524
• chore: Bump OpenDAL to v0.36 by @Xuanwo in #1678
• chore: add len() to Bytes and StringBytes by @shuiyisong in #1455
• chore: add a uniform prefix to the metrics using the official recommendation of by @paomian in #1590
• chore: add boxed error for custom error map by @shuiyisong in #1635
• chore: add configurator to http server by @paomian in #1488
• chore: add fmt for statement query by @paomian in #1588
• chore: add grpc query interceptor by @shuiyisong in #1626
• chore: add metrics for table route getting by @Fengys123 in #1636
• chore: add some metrics for grpc client by @paomian in #1398
• chore: don't render reproduce as shell in issue template by @waynexia in #1397
• chore: fix code styling by @WenyXu in #1623
• chore: fix some typo and add deriv to plan in promql by @haohuaijin in #1438
• chore: ignore symbol link target file by @waynexia in #1696
• chore: reduce the number of requests for meta by @Fengys123 in #1647
• chore: reject table creation when partitions exceeds peer number by @Fengys123 in #1654
• chore: release 0.2.0 by @v0y4g3r in #1413
• chore: remove Release prefix from release name by @v0y4g3r in #1409
• chore: remove info log by @WenyXu in #1483
• chore: remove redundant code. by @DevilExileSu in #1502
• chore: remove useless Option type in plugins by @paomian in #1544
• chore: truncate route-table by @fengjiachun in #1619
• chore: unify code styling by @WenyXu in #1523
• chore: update RustPython depend by @discord9 in #1406
• chore: upgrade promql-parser version by @yuanbohan in #1484
• chore: use alicloud imagehub by @daviderli614 in #1418
• ci: always upload sqlness log by @waynexia in #1692
• ci: enable test on release by @waynexia in #1428
• ci: set whether it is the latest release by using 'ncipollo/release-action and update install.sh by @zyy17 in #1400
• ci: upgrade nightly release tag from v0.2.0 to v0.3.0 by @zyy17 in #1446
• docs: Add docs about schema structs by @evenyag in #1373
• docs: add play section by @sunng87 in #1528
• docs: fix timestamp rendering in readme by @v0y4g3r in #1399
• docs: rfc of distributed planner by @waynexia in #1554
• docs: update readme by @killme2008 in #1549
• refactor(frontend): adjust code structure by @waynexia in #1615
• refactor: Holds histogram in the timer to avoid clone labels if possible by @evenyag in #1653
• refactor: add "table engine" to datanode heartbeat. by @QuenKar in #1616
• refactor: add scan_to_stream() to Table trait to postpone the stream generation by @waynexia in #1639
• refactor: catalog by @v0y4g3r in #1454
• refactor: change mailbox_messages to mailbox_message by @WenyXu in #1557
• refactor: default catalog and schema are created at Metasrv by @Fengys123 in #1391
• refactor: move code out of loop by @WenyXu in #1657
• refactor: move rpc's commons to common-meta by @fengjiachun in #1625
• refactor: move the common part of the heartbeat response handler to common by @WenyXu in #1627
• refactor: parallelize open table by @shuiyisong in #1392
• refactor: refactor BufferedWriter by @WenyXu in #1439
• refactor: refactor stmt_options_to_table_options by @WenyXu in #1403
• refactor: remove redundant plugins argument by @sunng87 in #1436
• refactor: remove unnecessary async from RepeatedTask::start by @evenyag in #1545
• refactor: remove useless error by @v0y4g3r in #1624
• refactor: remove version column by @v0y4g3r in #1576
• refactor: replace Copy Format with datasource Format by @WenyXu in #1435
• refactor: set the filters for testing logs by @MichaelScofield in #1637
• refactor: support layered configuration by @zyy17 in #1535
• refactor: support parsing env list by @zyy17 in #1595
• refactor: update proto and rename incorrect region_id fields by @evenyag in #1670
• test: Fix s3 region in test by @evenyag in #1493
• test: add SELECT ... LIMIT ... test cases for distributed mode by @MichaelScofield in #1419
• test: add basic cases for distributed TQL by @waynexia in #1437
• test: add integration case to check dashboard path by @waynexia in #1422
• test: add tests for external table by @WenyXu in #1460
• test: add write test for region failover by @WenyXu in #1673
• test: fix sqlx compatibility and adds integration test for sqlx by @sunng87 in #1686
• test: move instances tests to "tests-integration" by @MichaelScofield in #1573

New Contributors

• @nearsyh made their first contribution in #1405
• @daviderli614 made their first contribution in #1418
• @DevilExileSu made their first contribution in #1444
• @NiwakaDev made their first contribution in #1498
• @gitccl made their first contribution in #1527
• @Taylor-lagrange made their first contribution in #1497
• @QuenKar made their first contribution in #1616
• @yaoyinnan made their first contribution in #1644

Full Changelog: v0.2.0...v0.3.0