From dc229f25d6e13ea0f7a0603a15be077ded9b67b9 Mon Sep 17 00:00:00 2001 From: aishwarya24 Date: Thu, 31 Oct 2024 12:40:36 -0400 Subject: [PATCH 1/2] 2.23.1.0 rn --- .../preview/releases/yba-releases/v2.23.md | 263 ++++++++++++++++++ .../preview/releases/ybdb-releases/v2.23.md | 229 +++++++++++++++ docs/data/currentVersions.json | 6 +- 3 files changed, 495 insertions(+), 3 deletions(-) diff --git a/docs/content/preview/releases/yba-releases/v2.23.md b/docs/content/preview/releases/yba-releases/v2.23.md index 0612c3187143..e793adfb2770 100644 --- a/docs/content/preview/releases/yba-releases/v2.23.md +++ b/docs/content/preview/releases/yba-releases/v2.23.md @@ -15,6 +15,269 @@ What follows are the release notes for all releases in the YugabyteDB Anywhere ( For an RSS feed of all release series, point your feed reader to the [RSS feed for releases](../index.xml). +## v2.23.1.0 - September 13, 2024 {#v2.23.1.0} + +**Build:** `2.23.1.0-b214` + +### Download + + + +### Change log + +
+ View the detailed changelog + +### Improvements + +* Allows adding an extra imagePullSecret to the StatefulSet in Kubernetes-based YBA 2.20.x installations. PLAT-13465 +* Enables the addition of an extra imagePullSecret to the StatefulSet in Kubernetes based installations. PLAT-13465 +* Allows efficient data management by cleaning only known directories in replicated storage and blocking migration if installRoot is a subdirectory. PLAT-14912 +* Ensures safe replicated migration by checking if the install root is a subdirectory of the replicated storage path and only cleaning up known directories, preventing accidental data loss. PLAT-14912 +* Provides a new subtask for improved validation of the customCA config for on-premises universes on multiple actions, including create universe, edit universe, and rotate certificates, with relevant configurations being `runtimeConfig` and `skip_cert_validation`. PLAT-4899 +* Allows 10 retries for IAM credentials fetch during backup to recover from transient network issues, enhanced with improved logging for failure scenarios. PLAT-13910 +* Ensures successful creation of air-gapped universes in recent YBA versions by addressing installation issues related to Chrony, locales, and en_US.UTF-8. PLAT-15052,PLAT-8144 +* Eliminates needless certificate rotation during gflag upgrades, optimizing YBA performance. PLAT-14008 +* Stores and displays edit universe task parameters in the audit log post Universe Edit, Gflags update, or Software upgrade. PLAT-14418 +* Allows setting volume and network interface tags in AWS during tag modifications in edit universe. PLAT-14603 +* Allows validation check for illegal characters in xCluster replication configuration name, enhancing error messaging. PLAT-14863 +* Allows time-filtering of core files collected in the support bundle. PLAT-14552 +* Removes the setting of optimized memory and tablet split GFlags for YBM to avoid potential conflicts. PLAT-15209 +* Shifts Task Details UI to use the Audit Log API instead of the separate task details API. PLAT-14938 +* Displays Point-In-Time Recovery (PITR) column and restore window details on backup list page. PLAT-15287 +* Enhances user experience by displaying a tooltip for ReadOnly users explaining why actions on the release page are disabled. PLAT-15189 +* Allows locally tested distributed replication failover to validate the safetime restoration, ensuring the data consistency after failover. PLAT-15212 +* Simplifies the frozen universe error message in YBA to help users understand the cause of a failed task. PLAT-15545 +* Adjusts default scrape timeout setting for Prometheus Jobs to avoid metrics calculation problems. PLAT-15625 + +### Bug fixes + +* Introduces health checks for connection pooling in universe configurations to monitor tservers and proxies. PLAT-12226 +* Introduces command to specify Linux version and trigger Linux OD upgrades in Universe configuration. PLAT-12297,PLAT-11553 +* Adds a precheck for libselinux bindings in system python3 during provisioning workflow. PLAT-12435 +* Enables option to use UTC for cron expression calculations when scheduling backups. PLAT-12510 +* Allows scripts like zip_purge_logs or clean_cores to work in ubuntu cronjobs by adding default of whoami output. PLAT-12610 +* Redacts `ysql_hba_conf_csv` value from logs to mask LDAP bind password. PLAT-13707 +* Updates user login flow and RBAC logic to include group memberships for OIDC and LDAP groups. PLAT-14227,PLAT-14226 +* Allows creating storage configurations with Minio using a predefined server region, and improves handling of S3 endpoints and global bucket access to prevent API failure. PLAT-14308 +* Enables connection pooling adjustments and port overriding in the Create Universe flow with a user interface. PLAT-14337,PLAT-12223,PLAT-12225,PLAT-14338 +* Enables continuous backups feature with extended configurations and cloud support, ensuring safer and robust data backups. PLAT-14450 +* Enables HTTPS support for Prometheus in `yba-ctl createBackup`, enhancement in the backup-restore process to include support for NFS, GCS, Azure storage, and retaining the last three backups for each major version. PLAT-14450 +* Enables point-in-time restoration, by saving and using specific metadata in the YB-Controller's success marker file, effectively returning comprehensive backup metadata and validating the restoration process. PLAT-14601 +* Enables a point-in-time restore feature and implements preflight changes for backup and restore functions. PLAT-14601 +* Preflight checks now simultaneously validate all supported NTP services including chrony, ntp(d), and timesyncd. PLAT-14695 +* Restores swamper targets in YBA even if exclude_prometheus is in backup for consistent universe metrics. PLAT-14743 +* Allows node agent installation to use bind IP differently from node IP for DNS name based registration. PLAT-14786 +* Ensures SAS token is masked in the backup configuration response and yb_backup logs for enhanced security. PLAT-14788 +* Reduces false DB node restart alerts by ignoring minor time updates in NTP syncs (up to 10 seconds). PLAT-14867 +* Integrates Query View for easier identification and management of Query Based Anomalies in YBA. PLAT-14870 +* Enables update of latest master addresses in under-replicated masters on the newly added node. PLAT-14891 +* Ensures optional resource group fields for AZU providers are not treated as mandatory in the Edit Region modal. PLAT-14900 +* Solves unexpected errors when editing/creating a provider with an incorrect resource group at the region level. PLAT-14901 +* Enhances clock drift health check by adding support for NTP, enabling time drift metrics collection even when NTP is used instead of Chrony. PLAT-14955 +* Allows manual provisioning of on-prem DB nodes by making SSH fields optional and graying out the `connect` option when SSH key is not provided. PLAT-14958,PLAT-14959 +* Allows manual provisioning of on-prem nodes without need for an SSH key. PLAT-14960 +* Precheck for node agent installation ensures correct permissions for execution in the installer directory. PLAT-14973 +* Clears `pg_data_11` directory on on-prem node release for seamless pg15 migration. PLAT-14993 +* Stores the YNP version on the disk and compares it during preflight, ensuring proper node_exporter installation. PLAT-14995,PLAT-14974,PLAT-14802 +* Enhances the UI by adding a helper message for pgaudit log level, removing the Logs Tab when YSQL is disabled, deleting non-functional Learn More link from Edit PG Compatibility Dialog, and hiding the Download metrics button. PLAT-14998,PLAT-14986,PLAT-14869,PLAT-15003 +* Allows installing NodeAgent as a user-level unit in Systemd mode. PLAT-15015 +* Enhances the create universe API with connection pooling support which introduces two new parameters `enableConnectionPooling` and `internalYsqlServerRpcPort`. PLAT-15036,PLAT-12222,PLAT-14333 +* Now includes Prometheus user in Yugabyte group and modifies yb_home_dir permissions to heighten security. PLAT-14974,PLAT-15045 +* Enables users to export logs without using `sudo` by creating a log directory with correct permissions. PLAT-15046 +* Allows successful connection pooling when ysqlAuth is enabled by using the correct postgres port. PLAT-15087 +* Enables user to configure logging level for YNP and fixes minor bugs causing YNP installation failure. PLAT-15153,PLAT-15132 +* Allows users to skip the cluster consistency check by adjusting runtime configurations in rare cases. PLAT-15175 +* Introduces a new endpoint to show the state of YugabyteDB master load balancing in Platform UI, assisting users in identifying the operational status and performance impacts. PLAT-5874 +* Refines the backup script to support relative paths and trailing slash at the end of the output path. PLAT-6779 +* Automatically sets the `yba-ctl` backup scripts to support the use of Prometheus HTTPS option, addressing backup interruptions for on-premise installations. PLAT-14522 +* Introduces a reboot-required module for YNP. PLAT-14564 +* Restores the ability for node agent install to handle bind IP differently than node IP for DNS name-based registration. PLAT-14786 +* Now deletes unattached OS disks on Azure after an OS upgrade to reduce clutter and waste. PLAT-14883 +* Enables the `connect to pod` action for Kubernetes universes even without access keys, but grays out the `connect` dialog when no SSH key is provided for on-premises providers with manual provisioning. PLAT-14959 +* Adds a precheck to confirm node addition to the provider. PLAT-15044 +* Enables database version checking for connection pooling. PLAT-15088 +* Allows dropping of `write_read_test` and `consistency_check` tables when disabling YSQL and recreation when re-enabling YSQL. PLAT-15097 +* Captures debug logs from bash scripts when log level is set to DEBUG. PLAT-15132 +* Enables new restore modal with Point in Time Recovery (PITR) support, enhancing backup restoration capabilities. PLAT-15180 +* Ensures the toggle for unsupported database versions is disabled in the User Interface. PLAT-15216 +* Introduces advanced dateTime option for collecting Prometheus metrics and restricts Connection Pooling for Kubernetes. PLAT-14052,PLAT-15237 +* Enables listing of backup schedules in a scrollable manner with editing, deleting, interval inspection, and backup table visibility functions. PLAT-15247 +* Allows usage of pagination when querying LDAP server and adds an API parameter to sync specific groups only. PLAT-15295 +* Updates the YBA node agent TLS certificates to use stronger and more secure ciphers. PLAT-15306 +* Allows passing the region_name during YNP installation, fixing installation failures. PLAT-15325 +* Revises disk mount preflight checks and enhances check output formatting. PLAT-15133,PLAT-15332 +* Sets 755 permissions on node-agent service file to ensure it restarts after node reboot in YNP deployment. PLAT-15345 +* Corrects the Node Addition Precheck logic for handling missing provider_ids. PLAT-15355 +* Enables retrieving userName from attribute lists if not present in the distinguished name (DN) on Ldap_db_sync API. PLAT-14882 +* Upgrades the Python requests library and urllib3 library to the latest versions eliminating high and medium vulnerabilities. PLAT-11243 +* Upgrades ion java library to enhance security and address vulnerability {{}}. PLAT-13833 +* Upgrades Grpc library to handle known vulnerabilities and ensures compatibility with latest Python and Java versions. PLAT-13936 +* Clarifies the usage of node agent's silent parameters to enhance user-friendliness. PLAT-14976 +* Introduces a soft threshold for auto master failover, allowing for early schedule enablement and user cancellation via the UI. PLAT-15000 +* Allows unlimited retries to start yb-node service. PLAT-15041 +* Ensures Disaster Recovery (DR) config page loads successfully even when numerous database scoped DR configurations exist on the same YBA. PLAT-15168 +* Reverts a commit to address Cloud NodeOps case failure when re-adding a previously removed node. PLAT-13800 +* Includes the `pg_max_mem_mb` flag in the configuration phase to fix cgroup modification failures during instance type changes. PLAT-15156 +* Upgrades the YBC client and server version to 2.2.0.0-b5 increasing backup efficiency by applying deadlines on a per-RPC basis. PLAT-15169 +* Introduces a password policy to disallow back-tick in database passwords, preventing potential code injection. PLAT-10119 +* Allows the collection of audit logs, HA config, and xCluster data in the support bundle. PLAT-10264 +* Allows multiple DR configurations or transaction replications for each database, preventing conflicts and errors. PLAT-10586 +* Adds a consistency check task at the end of universe creation and alerts for out-of-sync universes. PLAT-11389 +* Adds new gauge metrics for `last backup time` and `last backup size` on the HA page for active instances. PLAT-12905 +* Enables rolling N nodes at a time during upgrades for multi-AZ/region k8s universes, configurable as a percentage of universe size. PLAT-12933 +* Enables rolling restart of N nodes simultaneously during upgrades for multi-AZ(region) universes once `batch_roll_enabled` is activated. PLAT-12934 +* Adds functionality to collect the open file count per process for master/tserver and other agents. PLAT-13095 +* Allows cloud provider edits to be retried. PLAT-13285 +* Includes a link to the Prometheus federate job on YBA standby instances for users to validate federation configurations and discover current lag. PLAT-13384 +* Upgrades Python setuptools to enhance security and compatibility by addressing listed vulnerabilities. PLAT-13836 +* Upgrades AWS SDK to version 1.55.5 addressing vulnerabilities and enhancing security. PLAT-13845 +* Upgrades jsonpath to 2.9.0, handling exceptions if the set path does not already exist. PLAT-13848 +* Allows shrink volume size adjustments on both primary and asynchronous clusters in K8 universe UI, keeping more action items enabled after a failed task. PLAT-13921 +* Adds `XCLUSTER` action and updates UI RBAC wrapper for improved permissions management in xCluster replication. PLAT-13957 +* Refactors backup commands in YBA CLI to guarantee compatibility with various inputs and allow JSON and pretty outputs. PLAT-14039 +* Enables viewing tables and their metrics for DB scoped replication via get API, along with YBA metadata. PLAT-14077 +* Adds a retry policy for GcpProjectApiClient to handle transient server errors. PLAT-14144 +* Adds a validate endpoint for all upgrade types and introduces `runOnlyPrechecks` and `skipNodeChecks` parameters for faster and customizable upgrade task execution. PLAT-14220 +* Allows rolling up to N nodes simultaneously in multi-AZ universes, determines maximum roll size by smallest AZ, and automatically falls back to batch size 1 if pre-check fails. PLAT-14299 +* Expands Prometheus scraping to collect additional container and kubelet metrics for pods matching the `yb` regex. PLAT-14351 +* Keeps the master in the quorum during instance resizes to prevent unnecessary removal. PLAT-14483 +* Allows communication ports to be altered during edit operations in a universe. PLAT-14595 +* Modifies JSON output format in CLI to display elements as a list of JSON objects, enhancing readability. PLAT-14620 +* Prevents reducing replication factor (RF) in edit universe, allowing only changes to RF placement. PLAT-14678 +* Displays a warning/banner to indicate an ongoing master failover process in the user interface, enhancing visibility and control. PLAT-14742 +* Switches to the new XCluster sync API in the UI for an updated user interaction. PLAT-14760 +* Now allows better handling of service restarts by removing bad start limit values from yb_bind and otel-collector systemd units. PLAT-14779 +* Enables retention of chosen storage type when changing instance type in edit mode, removing default reset. PLAT-14784 +* Enables the monitoring of internally scheduled jobs through the addition of REST APIs. PLAT-14785 +* Introduces a basic API that displays the status and estimated idle time of the master tablet load balancer. PLAT-14787 +* Adds `Region name` option and ensures regions are searched within the provider, fixing the issue of missing region metadata when adding provisioned nodes via Node Agent. PLAT-14790 +* Enables adding Environment Access Restrictions (EAR) configurations for AWS, Azure, GCP, and HCV. PLAT-14805 +* Allows enabling specific DB flags for memory optimization and enforcement of tablet creation and splitting limits in new universes. PLAT-14819 +* Eliminates duplication in table configs and adds missing YCQL index tables to replication during GET calls and metrics. PLAT-14835 +* Makes API authentication faster by identifying users using new `userUUID$apiToken`, reducing call time. PLAT-14850 +* Introduces new per-process metrics for better monitoring of CPU usage, memory usage, and IO operations. PLAT-14875 +* Unveils new per-process metrics in non-k8 environments, governed by a runtime configuration. PLAT-14876 +* Resets the failed counter for auto-master failover during a new submission, allowing restart of the failover process. PLAT-14884 +* Updates minor text changes for Point In Time Recovery (PITR) configuration step for concise error messages. PLAT-14892 +* Marks eligible universes for node agent migration, beginning the migration process. PLAT-14915 +* Keeps DB audit logging runtime configuration enabled on a universe even when turned off. PLAT-14925 +* Now introduces capability to override Master, TServer, and Node Exporter ports while editing Universe in Non-k8 environment. PLAT-14928 +* Allows correct update of status for downloaded releases in K8s Operator, fixing Release Reconciler regression. PLAT-14948 +* Adds validation to prevent decimal input for PITR retention period values. PLAT-14951 +* Adds guardrails and improves non-root install settings with new prechecks for the yba installer. PLAT-14956 +* Allows superadmin users to click on `use same/different replica` during DR repair without encountering permission errors. PLAT-14963 +* Boosts the default value of yb.query_stats.slow_queries.query_length to 16KB. PLAT-14981 +* Allows enabling or disabling DDL atomicity check using a feature flag, set to on by default, for improved error inspection during health checks. PLAT-15011 +* Ensure xCluster metrics only fetch from running universes to avoid failure and allow addition/removal of the exact table in a single API request, even when bootstrap is needed. PLAT-14945,PLAT-15017,PLAT-15006 +* Replaces the `$` sign in the API token with a safer character to prevent bash command disruptions. PLAT-15027 +* Adds a bootstrap summary page to xCluster modals demonstrating an overview of selected tables for bootstrapping and enabling skipping of bootstrapping. PLAT-13757,PLAT-15005,PLAT-15032 +* Enables regular sync of gflag template file to actual gflag file used by master/tserver service in k8s_parent.py. PLAT-15035 +* Allows creating bi-directional xCluster configurations without triggering bootstrap in the reverse direction. PLAT-15039 +* Sets the default Point-in-Time Recovery (PITR) retention period from runtime configs when configuring Disaster Recovery (DR). PLAT-15042 +* Triggers an immediate sync after completing a promotion to mitigate potential disruptions in instance state caused by long replication frequencies. PLAT-15089 +* Restricts visibility of the newer Scheduled Backup Policies tab to only when the "enableBackupPITR" feature is enabled. PLAT-15100 +* Restricts DB scoped replication creation until universes are upgraded to the minimum supported version. PLAT-15113 +* Allows retrieval of simple DB objects without DB Sync using a query parameter in GET xcluster/DR API. PLAT-15139 +* Allows setting XCluster table status to `DroppedFromTarget` if the replicated table gets dropped from the target. PLAT-15148 +* Upgrades YBC client and server version to 2.2.0.0-b5, adjusting client deadline to each RPC call, not per client. PLAT-15152 +* Integrates collection of Prometheus metrics directly into the support bundle, allowing customers to set a specific time-range for metric collection, filter by export or job, and select individual nodes, enhancing offline analysis capabilities. PLAT-3593 +* Allows creation of a single Kubernetes Load Balancer service for the entire universe instead of one per Availability Zone (AZ), offering a more streamlined way to manage services when a universe is entirely contained within a single namespace. PLAT-10592 +* Enables support for installing and managing global services in Kubernetes provided it's not an MCS-based universe, utilizes new helm naming, and applies new labels on the Universe. PLAT-10592 +* Allows retrying switchover and failover tasks upon failure, supports switchover aborting, boosts pitr restore task performance, and enhances failover/switchover task execution time. PLAT-10706 +* Enables batch collection of customer task info, preventing memory overload, for YBA metadata support bundle. PLAT-12326 +* Increases default data points in metric graphs to 250 for better resolution and makes it runtime configurable. PLAT-13984 +* Enables support for multiple transactional xCluster configurations per universe in YBA user interface. PLAT-14158 +* Corrects node status change issue that kept decommissioned on-prem nodes from being reused. PLAT-14590 +* Enables editing of EAR configs using YBA CLI for AWS, Azure, GCP, and Hashicorp. PLAT-14808 +* Enables Master Key Rotation (MKR) in the universe security operations for enhanced data security. PLAT-14809 +* Allows editing of Point in Time Recovery (PITR) parameters and stores these during Disaster Recovery (DR) creation. PLAT-14888 +* Allows the use of default values or user-specified parameters for pitr creation in database addition. PLAT-14888 +* Allows installing node agents on desired universes for public cloud providers, offers renaming options for clarity, handles node scaling, and tracks status transitions when node-agent client is disabled. PLAT-14897,PLAT-14916 +* Allows selection of the latest gflag groups by version, with adjusted group values for `ENHANCED_PG_COMPATIBILITY` in 2024.1.3. PLAT-15016 +* Enables support for creating DB scoped DR configurations in the UI and includes updates to display schema change mode information, enhancing user experience and understanding. PLAT-15071,PLAT-15062 +* For tables dropped on target, the YBA UI now considers them as unconfigured and preselected, enhancing the need for a bootstrap check and facilitating a detailed bootstrap response. PLAT-15079 +* Enhances high availability (HA) logging and backup metrics for precise monitoring, and corrects the `HA Last Backup Size` graph display. PLAT-14092,PLAT-15091 +* Enables toggling on/off roll N nodes in k8s through a new runtime configuration. PLAT-15101 +* Determines the client type for node-agent installation using the universe marker field, allowing specific client usage based on the migration status. PLAT-15145 +* Offers more accurate auto flag checks for xCluster/DR configurations by aligning source and target universe flags. PLAT-15151 +* Restores visibility of Persistent Volume Stats metrics data on the K8 universe. PLAT-15181 +* Ensures `describe` and `create` CLI commands return JSON objects not lists, for a more appropriate output. PLAT-15194 +* Allows addition and updating of `image-bundle` field with custom AMI images for CSPs and displays a message requesting image bundles usage when deprecated SSH fields are used. PLAT-13998,PLAT-15215 +* Modifies package installation command to use `legacy-peer-deps`, allowing continued use of react version 17 despite dependency issues with react-bootstrap-table. PLAT-15228 +* Enhances task details drawer with `Start time`, correct headers, `Collapse all` button, and subtaskInfo and fixes incorrect Universe Diff comparison and missing Query Regex on logs page. PLAT-15231 +* Upgrades YBC to version 2.2.0.0-b6, enables verbose logging by default, and adds retries for DEADLINE_EXCEEDED errors in YBC Java client. PLAT-15232,PLAT-15214 +* Redesigned backup schedules to correctly tag backups under their schedule names, not the user login. PLAT-15261 +* Now disallows non-namespaced universes to pass Namespaced services in serviceEndpoints overrides. PLAT-15262 +* Updates pitr params for DR config and enforces not-null constraints. PLAT-15274 +* Allows use of the `set_dbs` endpoint for editing table selections in DB scoped Disaster Recovery configurations, enhancing consistency and error handling while adding new databases. PLAT-15288 +* Ensures `GET /configs` does not get blocked by managing DR config and XCluster config deletion within a single transaction. PLAT-15297 +* Allows task completion notifications only when tasks are genuinely finished, preventing premature notifications. PLAT-15300 +* Removes parallel query from the enhanced PG parity GFlag group in all deployments. PLAT-15302 +* Fixes error with K8s Universes throwing 'interface {} is string, not float64' and assures correct float fields conversion for K8s. PLAT-15320 +* Ensure up-to-date build files for improved Play Framework assets caching, preventing YBA UI breakdown after upgrades. PLAT-15322 +* Corrects error handling in DDL atomicity checks, ensuring accurate results even when YSQL is down on one node. PLAT-15326 +* Enables resolution of POD_IP for MCS cases with newly introduced gflags. PLAT-15344 +* Enables new group to role mappings for OIDC/LDAP with `enableNewAuthAndMappings` for revamped UI. PLAT-13377 +* Showcases catalog cache misses metric in the YSQL section of the YBA dashboard, offering both regular and outlier table views. PLAT-14744 +* Revised the `since yba version` and corrected RBAC values for the new release API, also made the upload API external. PLAT-14756 +* Ensures "Assign Public IP" field in Azure remains enabled but unchecked by default. PLAT-14871 +* Enables creation of Scheduled Backup Policy modal under the feature flags "enableBackupPITR". PLAT-14965 +* Introduces a new v2 API for listing all YBC Gflags metadata, emulating the existing YBA list_gflags API. PLAT-14801 +* Reduces bundle size by removing unused dependencies from UI packages. PLAT-14949 +* Corrects UI validation for the hba conf flag during OIDC setup, preventing errors during GFlag edits. PLAT-15167 +* Fixes an issue in Kubernetes universes where altering replication factor and node count, then resetting, won't reset replication factor. PLAT-15182 +* Corrects terminology inconsistency in the Kubernetes universe by displaying `Pods` instead of `Nodes` in the final configuration summary. PLAT-15187 +* Allows updating of scheduling frequency and cron expression when editing the scheduled backup. PLAT-15244 +* Enhances YB CLI to list, describe, and delete Encryption At Rest configurations across various commands. PLAT-14807,PLAT-14806 +* Enables listing, describing, and deleting Encryption in Transit (EIT) configurations. PLAT-14812,PLAT-14813 +* Upgrades YBC client and server versions to 2.2.0.0-b4 to eliminate a deadlock occurring amid task operations. PLAT-14909 +* Corrects UI display error where selected tables appeared unselected during restart replication. PLAT-14982 +* Mitigates create universe failures for k8s due to race conditions by using thread-safe collections. PLAT-14987 +* Allows easier selection of index tables when users choose the select unreplicated tables option, enhancing index table management. PLAT-14988 +* Streamlines the Postgres setup process with improved initdb operation, eliminating permission issues. PLAT-15028 +* Splits the installation process into two steps: software layout and data layout for more flexibility. PLAT-15029 +* Allows installation without setting up the data directory, enabling disk remounting to another dataless install. PLAT-15030 +* Renames `Provider Configurations` page title to `Integrations` for consistency with new tab page. PLAT-15061 +* Allows adding and editing EIT configurations using the YBA CLI command format. PLAT-14810,PLAT-14811 +* Adds two guardrails to the installer for safer without-data installation and appropriate data disk size checks. PLAT-15031 +* Limits character count for table names to avoid overlapping in the schema tabs. PLAT-15068 +* Resolves issues with missing graphs and inconsistent RCA display in TP Secondary Dashboard within YBA. PLAT-15105 +* Disables excessive logs related to `Explicitly set HTTP header 'Transfer-Encoding:chunked` for a quieter logging experience. PLAT-11189 +* Allows Prometheus log file to write logs, enhancing clarity and avoiding confusion in the status output. PLAT-11305 +* Removes exception stack trace logs during backup for non-operator controlled universes. PLAT-12232 +* Collects universe logs in support bundle considering the `log_dir` GFlag for both master and tserver. PLAT-12433 +* Enables hostname in UBI-8 images and improves support bundle collection on Kubernetes. PLAT-14045 +* Corrects argument parsing failure in disk_io_failure_detection_py3.py script when values contain `=`. PLAT-14435 +* Fixes Hashicorp vault's KMS configuration to retain custom key names on edits and displays the key name under KMS config's Show Details area. PLAT-14968,PLAT-14966 +* Fixes error in `Average YSQL operations latency` alert which was incorrectly measuring in microseconds. PLAT-15404,PLAT-15406 +* Ensures proper escaping of strings in JSON templates and adds logging, fixing issues in webhook template placeholders. PLAT-15607 +* Now enables IMDSv2 by default on UI and backend to enhance EC2 instance security. PLAT-14030 +* Changes YBA callhome URL to new diagnostics endpoint and assures only non-sensitive, unique diagnostics data is sent. PLAT-15205 +* Stops downloading sha1 during release creation, as only sha256 values are expected. PLAT-15791 +* Allows setting `master_join_existing_cluster` GFlag during Helm install and universe configuration in K8s. PLAT-15034 +* Eliminates the necessity for clock sync check in node_health.py for Kubernetes universes. PLAT-15196 +* Enables successful installation/upgrade of YBA on K8s with TLS through Jenkins. PLAT-15219 +* Prevents sensitive information leaks in YBA logs by modifying gflag values in Java and Python layers. PLAT-15307 +* Ensures the `update_lb_config` task correctly performs its work rather than clearing out prematurely. PLAT-15349 +* Upgrades the Postgres version in YBA helm charts to the latest, enhancing security. PLAT-15352 +* Allows using UTC for cron expressions in backup schedules to correct prior faulty commits. PLAT-15379 +* Offers more precise error codes during YSQL dump and snapshot failures in YBC instead of the generic `COMMAND_FAILED` status. PLAT-15557 +* Adds a retry system for dump-entities check before node destruction, reducing potential inaccuracies. PLAT-15608 +* Enhances per-process IO write stats calculation by considering `write_bytes` and `cancelled_write_bytes`. PLAT-15440,PLAT-15672 +* Allows root install directory change on rehydration to avoid "No directory found" error messages by setting YBA to re-trigger the path fixing code. PLAT-15695 +* Prevents null pointer exception (NPE) when updating the status of backup/restore tasks in k8s operator. PLAT-15282 +* Allows retrieval of shasum file for URL downloads and adds SHA256 to downloaded releases, aiding in artifact validation. PLAT-15580,PLAT-15581 + +
+ ## v2.23.0.0 - September 13, 2024 {#v2.23.0.0} **Build:** `2.23.0.0-b710` diff --git a/docs/content/preview/releases/ybdb-releases/v2.23.md b/docs/content/preview/releases/ybdb-releases/v2.23.md index 1e1391a9973f..a868921856e7 100644 --- a/docs/content/preview/releases/ybdb-releases/v2.23.md +++ b/docs/content/preview/releases/ybdb-releases/v2.23.md @@ -15,6 +15,235 @@ What follows are the release notes for the YugabyteDB v2.23 release series. Cont For an RSS feed of all release series, point your feed reader to the [RSS feed for releases](../index.xml). +## v2.23.1.0 - November 1, 2024 {#v2.23.1.0} + +**Build:** `2.23.1.0-b214` + +### Downloads + + + +**Docker:** + +```sh +docker pull yugabytedb/yugabyte:2.23.1.0-b214 +``` + +### Change log + +
+ View the detailed changelog + +### Improvements + +#### YSQL + +* Changes the CatalogCacheMisses metric name to showcase separate cache misses for each catalog cache, assisting users in easily preloading necessary catalog tables. {{}} +* Enables in-place updates for non-key columns in indexes, reducing number of flushes. {{}} +* Introduces the `yb_make_next_ddl_statement_nonincrementing` configuration parameter to reduce catalog version increments during many consecutive DDLs, lowering memory and latency spikes. {{}} +* Updates advisory lock error message with a workaround hint instead of `not yet implemented.` {{}} +* Allows certain ALTER VIEW commands in the parser without requiring additional changes. {{}} +* Includes a modification to ysql_dump and Postgres that ensures all pg_class and pg_type OID preservation across a backup/restore when passing the include-yb-metadata flag. {{}} +* Streamlines sys table prefetching by automatically resetting catalog read time. {{}} +* Simplifies several ybc_pggate functions for better value return and removes unused functions. {{}} +* Revamps the restoration of old backups by only applying the check for `pg_class OID` if the new YSQL configuration parameter, `yb_ignore_heap_pg_class_oids`, is enabled. {{}} +* Removes unused `connected_database_` field from PgSession class for code cleanliness. {{}} +* Unmodified columns on main table will continue to be written out when update optimizations are enabled, removing dependency on the feature controlled by `ysql_skip_row_lock_for_update`. {{}} +* Adds table-level Prometheus metrics for catcache misses at node-level, viewable at `:13000/prometheus-metrics` endpoint. {{}} +* Auto analyze service now tracks mutation counts from DDL changes and eliminates redundant analysis of indexes. {{}},{{}} +* Adds serialization/deserialization and equality checking mechanism for update optimization metadata. {{}} +* Allows adding a primary key using a unique index by performing a table rewrite. {{}} +* Enables the batch execution of explicit row lock requests in Pg, improving the handling and performance of such requests. {{}} +* Allows validation of foreign-key and check constraints with `ALTER TABLE ...VALIDATE CONSTRAINT`. {{}} +* Added yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables for yb_db_admin role. {{}} +* Allows preloading of foreign key lists in relcache, avoiding on-demand master fetches, controlled by `yb_enable_fkey_catcache`. {{}} +* Refines the PatchStatus function by storing relation id directly in the relations vector, avoiding the need to call the PgSession::LoadTable method. {{}} + +#### YCQL + +* Updates YCQL tables `system_schema.functions` and `system_schema.aggregates` to align with Cassandra, aiding Cortex migration. {{}} + +#### DocDB + +* Reduces risk of WAL over-garbage collection in xCluster and avoids unnecessary flushes by modifying the WAL retention policy. {{}} +* Enables reduction of duplicate code and custom flag filtering through `GetFlagInfos` relocation. {{}} +* Refactors in-memory TSDescriptors to use SysServerEntryPB for further application towards persistent TSDescriptors in the system catalog. {{}} +* Accelerates xCluster setup for tables by reducing setup time, storing all intermediate state in-memory, and limiting writes to `sys_catalog`. {{}} +* Allows adding new CatalogEntries without modifying the sys-catalog-tool through using helper functions. {{}} +* Refactors method on main table scan for abstract sourcing of point-lookup keys, enabling keys from secondary vector index. {{}} +* Enables reading stored vector index data from DocDB. {{}} +* Removes unnecessary commented code for a cleaner and more efficient codebase. {{}} +* Enables scanning of the `sequences_data` table for xCluster replication, aiding in the bootstrapping process by integrating current sequence information into the WAL from the source universe. {{}} +* Introduces `DEFINE_NEW_INSTALL_VALUE` macro to enable controlled rollout of new features in fresh clusters. {{}} +* Enables taking into account of changed data during a vector index data update. {{}} +* Adds utilities for HNSW vector index implementation, benchmarking, and file reading. {{}} +* Removes the `CallVisitor` template function to simplify the sys-catalog-tool code. {{}} +* Allows each sequence to have an update in the WAL for seamless xCluster replication. {{}} +* Introduces a command-line tool for building, validating, and benchmarking HNSW indexes with Usearch implementation. {{}} +* Offers support for different vector coordinate types including vector-related templates. {{}} +* Imports hnswlib header-only library for enhancements. {{}} +* Introduces a new gflag `max_disk_throughput_mbps` for automated control of write rejections when disk is full, replacing `reject_writes_min_disk_space_aggressive_check_mb`. {{}} +* Allows easier debugging with additional logging and updated log prefixes. {{}} +* Ensures that ddl_queue and replicated_ddls do not colocate on the same tablet in colocated databases. {{}} +* Integrates hnswlib into the vector indexing framework, adding wrapper and allowing different distance calculations. {{}} +* Eliminates the capability to rename xCluster replication groups, reducing potential confusion. {{}} +* Enables sequences replication feature in source universe by creating `sequences_data` table and incorporating an outgoing stream. {{}} +* Enables shared memory by default for executing read and write pg client queries to enhance performance. {{}} +* Streamlines automatic syncing of inline third-party dependencies and breaks down `thirdparty_tool` into separate modules. {{}} +* Enables automatic application of DDL changes in xCluster using `automatic_ddl_mode` proto field. {{}} +* Enables display of cluster_uuid in tserver /varz endpoint for better test and call_home tracking. {{}} +* Unutilized non-transactional snapshot code gets removed for a cleaner transactional snapshot flow. {{}} +* Resets the `cdc_wal_retention_time_secs` value on table removal from xCluster replication and applies updates dynamically, eliminating need to drop and recreate replication. {{}} +* Ensures correct functioning of `search_path` by skipping its value's enquote during transaction setup. {{}} +* Enhances sequence generation consistency by making backend process stick to a logical connection when initializing a sequence variable with `nextval` function. {{}} +* Ensures DDL operations success with a sleep interval after database catalog version updates when Connection Manager is active. {{}} +* Updates the callhome URL to use HTTPS for enhanced security. {{}} +* Includes PID in yb_active_session_history for easy correlation with actual processes in YSQL, YCQL, and background activities. {{}} +* Allows reduction of thread stack size to avoid Linux hugepage backing. {{}} +* Allows explicit addition of gFlags to `gflag_allowlist.txt` for secure callhome data collection, plus `version_info` in tserver data. {{}} +* Adds a metric to alert users when the master follower heartbeat delay is too high. {{}} +* Removes the need to set `set_cdc_min_replicated_index` during xCluster bootstrap for improved log retention. {{}} +* Ensures the user interface accurately displays disk size, even when multiple data directories are used. {{}} + +#### CDC + +* Allows dynamic table addition alongside table removal from CDC streams without needing to disable or restart. {{}} +* Adds a yb-admin option to disable dynamic table addition when creating new streams. {{}} +* Enables default non-eligible table cleanup in CDC streams, replaces the flag `enable_cleanup_of_non_eligible_tables_from_cdcsdk_stream` with `cdcsdk_enable_dynamic_table_addition_with_table_cleanup`, and removes flag protection for table identification. {{}} +* Syncs table removal from a CDC stream to a more performant async process, allowing both manual and automatic removal via the `RemoveUserTableFromCDCSDKStream` and `RemoveTablesFromCDCSDKStream` RPCs respectively. {{}} +* Disables `wal_level` checks for logical replication in YSQL as PG WAL is not used. {{}} +* Changes the default `wal_level` to `logical` to ensure compatibility with logical replication clients. {{}} +* Enables asynchronous removal of user tables from CDC streams by adjusting how background threads process and persist stream metadata. {{}} +* Enables tablet splitting on tables under CDCSDK stream by default using the GFLAG `enable_tablet_split_of_cdcsdk_streamed_tables`. {{}} +* Adds a tag for the slot name attribute in the CDC metrics. {{}} + +#### yugabyted + +* Handles OBJECT_NOT_FOUND errors during SELECT/UPDATE/DELETE after TRUNCATE when Connection Manager is enabled. {{}} +* Enables bitmap scans and removes size-based fetching in the pg parity feature by updating GFlags. {{}} +* Updates to the pg parity testcase now remove size-based fetching and include enabling bitmap scans. {{}} +* Changes the flag name from `enable_pg_parity_tech_preview` to `enable_pg_parity_early_access` in all branches. {{}} +* Stops sending gflags details in the callhome diagnostics to eliminate redundant data. {{}} + +### Bug fixes + +#### YSQL + +* Removes unnecessary DCHECK for smoother rolling upgrades from release 2.20, enhancing `ysql_enable_db_catalog_version_mode` gflag functionality. {{}} +* Fixes an issue related to regular expression pushdown causing inconsistency and crashes by introducing thread local cache. {{}} +* Resets catalog read time after table prefetching to prevent outdated readings for on-demand loading from a master. {{}} +* Corrects IndexScan's indextlist to improve accuracy of variable references in subquery scenarios. {{}} +* Prevents individually moving a colocated table to a different tablespace, forcing users to move all tables collectively. {{}} +* Minimizes read restart errors during ANALYZE operation by using a fresh read time for each table, finding an optimal balance between read oldness and transactionality. {{}} +* Reduces the memory consumption of the `fk_reference_cache_` in large transactions, thereby ensuring minimal use of RAM. {{}} +* Prevents retry of schema mismatch errors in batched execution mode to maintain transaction atomicity. {{}} +* Revamped `Storage Read Requests` metric now better reflects true RPC count in `PrecastRequestSender` usage. {{}},{{}} +* Resolves `INSERT ON CONFLICT` bugs in temporary tables to ensure consistent outputs. {{}} +* Fixes a TupleDesc reference leak warning in nested INSERT ON CONFLICT. {{}} +* Restores unintentional grammar changes to YSQL ALTER SCHEMA command for bug fix. {{}} +* Fixes repeated calls in `DeleteTableInternal`, preventing `YbAdminSnapshotScheduleTest.SysCatalogRetention` failure on asan builds. {{}} +* Resolves potential memory corruption by mutating pushdown expressions outside the per tuple context. {{}} +* Eliminates the publishing of empty metrics from the `statementType` enum on the YSQL Metrics endpoint. {{}} +* Simplifies the process of identifying identical data by using storage equality, not semantic equality, during UPDATE queries, handling problems that arise when data types lack equality operators or when user-defined data types exhibit unusual semantic equality. {{}} +* Addresses a bug in YSQL that caused crashes while costing an index scan on a hash index with included columns using CBO. {{}} +* Adds more logs to aid in debugging a transaction status mismatch issue. {{}} +* Fixes the hanging issue when creating a colocated table by correcting the unconditional reading of the `relkind` column. {{}} +* Adds `log_dist` option to auto_explain docs for YSQL. {{}} +* Addresses the issue of invalid Attnum with Bitmap Scan +subplans +nested loops previously encountered in Index Scans. {{}} +* Adds table name to table-level metrics on `/metrics` endpoint for better identification. {{}} +* Fixes a bug that caused incorrect deducing of commit/abort status for transactions which do not change the table's schema. {{}} +* Adds HELP and TYPE metadata to YSQL Prometheus metrics endpoint for better metrics accessibility. {{}} +* Corrects an issue causing heap-buffer-overflow errors during ysqlconnmgr ASAN tests. {{}} +* Fixes a crash caused by using the `pg_hint_plan` with the `hint_table` enabled. {{}} +* Updates the copyright string from `YugaByteDB` to the correct version `YugabyteDB`, ensuring error-free linter runs. {{}} +* Allows for a modified calculation of net connections in YSQL connection manager with the warmup random mode. {{}} +* Prevents background parallel workers from executing check/assign hooks of YSQL configuration parameters, eliminating potential crashes. {{}} +* Modifies backup/restore process to skip column name checks for indexes, allowing for successful restoration even with renamed columns. {{}} +* Fixes crashes caused by using a RowComparisonExpression on a reordered primary key index. {{}} +* Replaces deprecated function `shared_ptr::unique` to resolve compilation errors on macOS. {{}} +* Improves cache re-invalidation for `ALTER TABLE` commands to avoid schema version mismatch errors within the same session. {{}} +* Streamlines ysql_dump process by eliminating faulty statements for colocated table unique constraint, enhancing database reliability. {{}} +* Introduces batching for the `INSERT ON CONFLICT` command to enhance performance by minimizing alternating read and write operations. {{}} +* Fixes the flaw in the current DDL atomicity workflow where only the first table's schema was compared, even if it doesn't change after a schema version increment. {{}} +* Alters encoding setup order to ensure `UTF8` use instead of `SQL_ASCII` and adds `pg_collation` to preloaded tables to prevent specific errors. {{}} +* Corrects a typo to prevent incorrect loading of pg_collation catalog caches, eliminating associated performance issues. {{}} + +#### YCQL + +* Eliminates unexpected error during shell type definition update across multiple sessions. {{}} + +#### DocDB + +* Prevents tablet bootstrap from getting stuck when replaying a truncate operation. {{}} +* Limits the response size of ListSnapshotSchedules RPC by excluding unnecessary per-object details to prevent overflow issues. {{}} +* Reduces the risk of deadlocks when making active read/write requests to a table right after a tablet-split. {{}} +* Simplifies handling of large RPC responses by catching errors earlier and appropriately using `narrow_cast`. {{}} +* Fixes errors from executing rpc callback twice due to lack of return after detecting disabled wait-queue or deadlock features. {{}} +* Switches write query duration from `CoarseMonoClock` to `MonoClock` for precise nanosecond granularity. {{}} +* Sets the `close_timestamp_micros` field for partially copied log segments to prevent time-based policy violation. {{}} +* Eliminates persistent logging issue after dropping a colocated table by removing unnecessary logs. {{}} +* Corrects leader balancing for geopartitioned tables to prevent early exits and ensure accurate global leader sorting. {{}} +* Adds `database_oid` to the `TSLocalLockManager` locking contract to distinguish objects from different databases. {{}} +* Adds debugging information for retryable requests rejected by followers despite acceptance by leaders. {{}} +* Corrects typo and includes minimum allowed value in the validation message for `xcluster_checkpoint_max_staleness_secs` flag. {{}} +* Prevents tablet partition collision during `ImportSnapshot` by not cloning `DELETED` or `REPLACED` tablets. {{}} +* Allows pg_cron to correctly handle SIGTERM by using `quickdie` instead of `die` or `pg_cron_sigterm`. {{}} +* Allows the retry of checkpoint if `leader not ready` or `service unavailable` errors occur, and prevents client retries on a failed checkpoint by storing an `InternalError` status. {{}} +* Restores `transaction_aware` field in `master_backup.proto` for compatibility with older DB versions. {{}} +* Enhances tablet limit flags by adding more details to the main flag and adjusting the help strings. {{}} +* Adds filtering for bootstrap intent iterators based on `min_replay_txn_start_ht` to prevent unnecessary SST files buildup during bootstrap. {{}} +* Fixes a compile error caused by a mismatch in the declaration of `protobuf_message_total_bytes_limit`. {{}} +* Reduces the potential for core dump errors during Point In Time Recovery (PITR) by utilizing shared_ptr in callbacks. {{}} +* Renames the `yql_endpoint_tserver_uuid` field to `top_level_node_id` in `/rpcz` to maintain consistency. {{}} +* Ensures correct and consistent updating of `grptablespace` column in `pg_yb_tablegroup` after `ALTER TABLE` command. {{}} +* Ensures the `CREATE TABLE ...TABLESPACE pg_default` command in colocated databases aligns with PostgreSQL semantics. {{}} +* Now treats YSQL configuration parameters in a case-insensitive manner, avoiding variable duplication and potential failures, aligning closer with Postgres behavior. {{}} +* Ensures node restarts successfully with `secure` enabled by introducing a timeout framework. {{}} +* Restores loading of old dumps that lack index PG class OIDs by enhancing the flag `yb_ignore_pg_class_oids`. {{}} +* Promotes cluster start-up when master_webserver_port is customized by considering user input during the `validate_and_set_configs` step. {{}} +* Allows the packed row size to exceed its limit during repacking, preventing tserver crash after `alter table add column` commands with a default value. {{}} +* The update stabilizes WriteBuffer, preventing potential crashes when constructed over non-zeroed memory. {{}} +* Adds a null check to prevent a system crash when cleaning up a recently applied transaction state after a flush operation. {{}} +* Resolves the shutdown issue with the shared exchange causing TServer to hang by properly managing concurrent threads and session destruction, and sets `pg_client_use_shared_memory` to false by default on Mac to enhance performance. {{}} +* Ensures "create table" requests for colocated tables don't fail due to mistaken tablet limit checks. {{}} + +#### CDC + +* Refactors CDCSDK to prevent data loss during workload with single shard transactions by ensuring WAL messages are committed before being replicated. {{}} +* Reduces disk IO overhead by optimizing the tablet-meta update process in CDCService. {{}} +* Introduces three new yb-admin commands for managing user tables in a CDC stream and addresses a race condition issue by removing an optimization in the `UpdateCheckpointForTabletEntriesInCDCState` method. {{}},{{}},{{}} +* Obsoletes expired or unpolled tables from CDC stream and state table, reducing unnecessary resource retention and potential blocking of split tablet deletion. {{}} +* Filters out records related to index tables in colocated databases to prevent GetChanges call failures. {{}} +* Ensures old-tuples for updates are only sent when replica identity is FULL, matching PostgreSQL behavior. {{}} +* Enhances table removal from a CDC stream by fetching tablets from the local master copy instead of the entire cdc_state table, enhancing performance. {{}} +* Prevents crashes that occur due to concurrent GetChanges calls on the same producer tablet. {{}} + +#### yugabyted + +* Offers corrected DB OID in the function `calculate_table_size`, now matching `YBCGetDatabaseOid(rel)` for improved reliability. {{}} +* Enables a node to successfully join using the `join` flag by implementing a retry mechanism. {{}} + +
+ ## v2.23.0.0 - September 13, 2024 {#v2.23.0.0} **Build:** `2.23.0.0-b710` diff --git a/docs/data/currentVersions.json b/docs/data/currentVersions.json index aa77a62e4851..35323405c7ec 100644 --- a/docs/data/currentVersions.json +++ b/docs/data/currentVersions.json @@ -5,9 +5,9 @@ "series": "v2.23", "alias": "preview", "display": "v2.23 (Preview)", - "version": "2.23.0.0", - "versionShort": "2.23.0", - "appVersion": "2.23.0.0-b710", + "version": "2.23.1.0", + "versionShort": "2.23.1", + "appVersion": "2.23.1.0-b216", "isStable": false, "initialRelease": "2024-09-13" }, From 4759eef3aaeecddcbb0b328f3b68668bd3b5d573 Mon Sep 17 00:00:00 2001 From: aishwarya24 Date: Thu, 31 Oct 2024 12:44:19 -0400 Subject: [PATCH 2/2] build number update --- docs/content/preview/releases/yba-releases/v2.23.md | 4 ++-- docs/content/preview/releases/ybdb-releases/v2.23.md | 10 +++++----- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/content/preview/releases/yba-releases/v2.23.md b/docs/content/preview/releases/yba-releases/v2.23.md index e793adfb2770..fd6371fb747a 100644 --- a/docs/content/preview/releases/yba-releases/v2.23.md +++ b/docs/content/preview/releases/yba-releases/v2.23.md @@ -17,13 +17,13 @@ For an RSS feed of all release series, point your feed reader to the [RSS feed f ## v2.23.1.0 - September 13, 2024 {#v2.23.1.0} -**Build:** `2.23.1.0-b214` +**Build:** `2.23.1.0-b216` ### Download