polardbx-sql-5.4.19-20241103_17309482

RELEASE NOTE

Feature enhancement

Added support for automatic identification of partitioned tables without GSI during scaling, and partition pruning for queries issued by UPSERT statements based on primary tables. For more information, see Global Secondary Index (GSI).
Added support for automatic cleanup of columnar Plan Cache when shutting down columnar nodes.
Added support for TRUNCATE TABLE operations on tables with columnar indexes. For details, see Columnar Index DDL Statement Restrictions.
Optimized GSI construction performance (concurrent backfilling based on globally ordered primary key ranges). For more information, see Global Secondary Index (GSI).
Optimized SPM (SQL Plan Management) bound columnar plan selection to no longer be affected by global columnar routing switch.
Optimized Metric View to display system metric information.
Optimized resource limits and scheduling efficiency for logical and physical backfill operations.
Optimized HLL (HyperLogLog) collection logic to avoid scanning unique keys.
Optimized performance overhead of MDL lock wait detection. For more information, see MDL Optimization (Metadata Lock).
Optimized columnar index selection logic (SQL with Apply operators no longer routes to columnar nodes). For more information, see Row-Column Routing Mechanism.
Optimized workload type determination logic, configuring columnar query workload type as AP by default. For more information, see Hybrid Load HTAP.
Optimized SHOW DDL statement performance for viewing complex DDL operations.
Optimized High Availability (HA) system's health check intervals and timeout parameters to reduce HA system recovery time.
Optimized scaling DDL Task granularity settings to improve scaling scheduling efficiency.
Added support for executing columnar index-related DDL statements on primary instances (configurable switch, enabled by default). For more information, see Configure Columnar Index Switch.
Added support for binding DN HINT in SPM (SQL Plan Management).
Added support for partition splitting in columnar indexes.
Added support for execution plan reuse regardless of semicolon presence at SQL statement end.
Added support for automatic routing of single-table queries with columnar indexes to read-only instances.
Added support for disabling built-in database names like MySQL when creating databases.
Added support for subqueries in JOIN ON conditions during single-table pushdown.
Added support for DDL operations on primary tables with columnar indexes by default.
Added default support for using columnar indexes in ANALYZE TABLE.
Added support for creating function indexes on JSON type fields when engine version is MySQL 8.0.
Optimized scaling process and source table existence verification during rollback.
Added SPM (SQL Plan Management) support for columnar read-only instances.
Added support for automatic quote addition or removal for type-mismatched parameter values.
Added support for string type SortKey in index pruning functionality for columnar read-only instances.
Added support for automatically scheduling columnar queries from primary instances to columnar read-only instances.
Optimized aggregate (AGG) function execution logic: default pushdown aggregation in row storage scenarios; default two-phase aggregation in columnar storage scenarios.
Optimized hot value index selection logic.
Optimized NOT IN expression performance in columnar read-only instance queries.
Added Plan Cache support for columnar read-only instances.
Added support for scheduling columnar queries to columnar read-only instances.
Optimized performance of SQL statements containing JOIN keywords.
Optimized performance of UPDATE statements with ORDER BY LIMIT clauses on partitioned tables.
Optimized display strategy for transaction-related variables.
Optimized distributed deadlock detection mechanism.
Added support for execution plan reuse regardless of semicolon presence at SQL statement end.
Added support for subqueries in JOIN ON conditions during single-table pushdown.
Optimized cardinality estimation in extreme data skew situations.
Optimized scaling process and source table existence verification during rollback.
Added automatic force index functionality in certain scenarios.
Added support for precision specification in float and double data types.
Added throttling support for columnar read-only and read-only instances.
Optimized Information_schema.tables data display.
Optimized DDL concurrency limits for ALTER TABLE related physical executions.
Increased DN node wait and concurrent throttling thresholds.
Added support for reading snapshot data using AS OF TSO syntax.
Optimized Distinct Agg query plans.
Added support for INSERT INTO TABLENAME() VALUES() syntax.
Added support for table-less UNION syntax in UPDATE SET clauses.
Optimized logical REPLACE pushdown judgment, supporting REPLACE statement pushdown in certain scenarios under RC isolation level.
Disabled schema-unspecified DDL execution in INFORMATION_SCHEMA database.
Added encrypted table support for physical replication.
Added support for scaling maintenance windows.
Added support for specifying table compression-related parameters like KEY_BLOCK_SIZE when creating tables.
Optimized execution speed of GSI creation and table partition key changes.

Bugfix

Fixed an issue where executing TRUNCATE operations multiple times on the same table with the table recycle bin feature enabled in DRDS mode databases could cause metadata errors. For more information, see AUTO Mode Database and DRDS Mode Database.
Fixed an issue where executing SET GLOBAL statements with regular privilege accounts on new instances would directly report errors instead of returning warnings.
Fixed an issue where single value frequencies in statistics were estimated as 0 in histograms.
Fixed an issue where TTL definitions would be lost when executing CREATE TABLE WITH TTL statements after CDC transmission to downstream Replicas. For more information, see TTL.
Fixed an issue where Sort operation results in columnar optimizer were lost, leading to unexpected results.
Fixed an issue where hidden primary keys became unavailable when executing TRUNCATE in RECYCLEBIN mode.
Fixed an issue where SAMPLE HINT was ineffective on storage nodes when SAMPLE RATE was expressed in scientific notation.
Fixed an issue where the system incorrectly matched execution plans for multiple IN parameters during preprocessing when SQL statements contained IN clauses with only one parameter.
Fixed issues with SQL parser failing to parse ALTER statements containing implicit table groups and incorrect handling of marking statements. For more information, see Table Group Types.
Fixed an issue where FORCE INDEX CCI was ineffective for single tables (non-partitioned tables). For more information, see FORCE INDEX.
Fixed an issue where SHOW DDL STATUS displayed incorrect rates during concurrent backfilling.
Fixed duplicate Feedback information in statistics.
Fixed an issue where single value frequencies were estimated as 0 in histogram statistics.
Fixed an issue where executing SET GLOBAL statements with newly created regular privilege accounts on new instances would directly report errors instead of returning warnings.
Fixed validation errors when changing column types for lock-free columns.
Fixed an issue where incorrect PARTIAL_SELECTION processing caused result errors when NULL values were present in the same Chunk during Selection generation.
Fixed an issue where FORCE INDEX CCI was ineffective for single tables.
Fixed compatibility issues with certain SHOW statements (SHOW INDEX FROM, SHOW TABLE STATUS, DESC TABLE/SHOW COLUMNS FROM, SHOW COLLATION, SHOW CHARACTER SET, SHOW PRIVILEGES) in columnar read-only instances.
Fixed an issue where incorrect filter condition transmission to OUTER JOIN led to incorrect result output in certain scenarios.
Fixed potential null pointer exceptions when executing INSERT IGNORE during sub-partition changes.
Fixed an issue where NDV (Number of Distinct Values) collection would incorrectly delete column statistics during table topology changes.
Fixed an issue where threads building snapshots would block when CDC was constructing full Schema snapshots.
Fixed an issue where automatic FORCE INDEX failed due to missing backticks (`) for indexes containing reserved keywords.
Fixed incorrect results caused by column disorder after partition pruning when using Lookup Join (BKAJoin) optimization for statements containing Null-Safe equals operator (<=>). For more information, see Lookup Join (BKAJoin).
Fixed residual DML pushdown flags when changing lock-free column types.
Fixed an issue where ALTER TABLE column rename operations would hang indefinitely when new column names had different case than original names.
Fixed an issue where INFORMATION_SCHEMA.TABLE data wasn't updating in real-time when DN node engine version was MySQL 8.0.
Fixed errors when querying INFORMATION_SCHEMA.STATISTICS table in PREPARE mode.
Fixed columnar index table creation failures caused by default table group reassignment triggered by partition changes.
Fixed potential unexpected results when filtering aggregation results without grouping.
Fixed columnar archive table creation failures when time column values in TTL tables were zero or empty.
Fixed SQL parser issues with ALTER statements containing implicit table groups and incorrect handling of marking statements.
Fixed CDC downstream synchronization errors caused by random suffixes in generated CDC SQL during dynamic CCI partition addition for TTL tables.
Fixed unexpected DirectPlan pushdown behavior in cross-database single table INSERT SELECT operations.
Fixed issues with creating and querying database users through Navicat.
Fixed continuous Warning prompts when connecting to databases through DataGrip.
Fixed inability to query database views through DBeaver.
Fixed residual DML pushdown flags when changing lock-free column types.
Fixed identical SQL template IDs between queries with NODE HINT and other queries.
Fixed potential validation failures when adding NOT NULL constraints during lock-free column type changes.
Fixed potential unexpected results when filtering aggregation results without grouping.
Fixed histogram serialization failures caused by special characters in statistics.
Fixed errors when querying INFORMATION_SCHEMA.STATISTICS table in PREPARE mode.
Fixed DDL operations hanging when ALTER TABLE column declaration changes involved case-sensitive differences.
Fixed unexpected results with STR_TO_DATE('2020-08-01 00:00:00', '%Y-%m-%d %T').
Fixed limitation where WHERE clause left values only supported integers in range [-263,263-1].
Fixed lack of support for union statements in Baseline.
Fixed unexpected concurrency calculations for SPM (SQL Plan Management) execution plans in MPP (Massively Parallel Processing) mode.
Fixed security issues with AES_DECRYPT function in high-concurrency environments.
Fixed histogram serialization failures caused by special characters in statistics.
Fixed loss of columnar attribute distribution during execution plan deserialization.
Fixed failures when modifying sequences with names longer than 64 characters.
Fixed template ID inconsistencies caused by case differences in aliases when using aggregate (AGG) functions.
Fixed ability to create mysql and performance_schema type databases using CREATE DATABASE statements.
Fixed potential unexpected results when executing SQL with LIMIT in columnar read-only instances.
Fixed column mismatch issues when updating child tables with foreign key constraints.
Fixed query errors when executing SQL containing aggregate functions with union subqueries.
Fixed potential validation failures when changing column types and adding NOT NULL constraints in lock-free scenarios.
Fixed Join type inference failures when pushing predicate conditions into subqueries.
Fixed column matching failures in INSERT SELECT statements containing aggregate (AGG) functions.
Changed default value of switch controlling DDL operations on primary tables with columnar indexes to enabled.
Fixed memory overflow issues caused by excessive cache pool maximum memory configuration.
Fixed errors when executing IN(NULL) queries on indexes containing string type columns.
Fixed incorrect encoding of TO_BASE64() function results on BLOB fields.
Fixed incorrect results after JoinAggToJoinAggSemiJoinRule optimization.
Fixed ineffective SAMPLE HINT when using scientific notation for SAMPLE RATE.
Fixed routing and value change issues when inserting data using TIMESTAMP() function.
Fixed limitation where WHERE clause left values only supported integers in range [-263,263-1].
Fixed unexpected DDL rollbacks.
Fixed long processing times for IN pruning when pushed-down table count exceeds 9.
Fixed potential failures when converting string types to non-string types using lock-free column type changes.
Fixed partial effectiveness of baseline delete commands across nodes.
Fixed insert errors after adding auto_increment to fields in tables without sequences using lock-free column type changes.
Fixed isolation level string decoding errors due to case sensitivity and underscore issues.
Fixed inaccurate execution plan cost calculations for baseline add/fix commands with IN parameters.
Fixed errors caused by missing CDC marking actions when recreating databases after DROP DATABASE failures.
Fixed ineffective single table sample hints in statistics.
Fixed UGSI existing data routing errors potentially caused by OMC 2.0.
Fixed foreign key constraint addition failures when processing self-referencing table indexes and character set detection.
Fixed parsing errors for large values in LIMIT x, y.
Fixed UGSI migration errors after enabling physical replication.
Fixed failures when performing lock-free column changes on single tables and broadcast tables.

发布说明

特性更新

扩缩容期间新增支持自动识别未包含GSI的分区表，并对UPSERT语句下发的查询按照主表进行分区裁剪。更多信息，请参见全局二级索引（GSI）。
关闭列存节点时新增支持自动清理列存Plan Cache。
含有列存索引的表新增支持TRUNCATE TABLE操作。具体信息，请参见列存索引DDL语句限制。
优化构建GSI的性能（构建时按照全局有序的主键范围并发回填）。更多信息，请参见全局二级索引（GSI）。
优化SPM（SQL Plan Management， SQL执行计划管理）绑定的列存计划选择不再受全局列存路由开关影响。
优化Metric View，以显示系统的Metric的信息。
优化对逻辑Backfill和物理Backfill的资源限制和调度效率。
优化HLL（HyperLogLog，一种概率算法的名称）的采集逻辑，以避免扫描唯一键。
优化MDL锁等待检测的性能开销。更多信息，请参见MDL优化（元数据锁）。
优化列存索引的选择逻辑（包含Apply算子的SQL不再路由至列存节点）。更多信息，请参见行列路由机制。
优化负载类型判定逻辑，将列存查询负载类型默认配置为AP类型。更多信息，请参见混合负载HTAP。
优化SHOW DDL语句对查看复杂DDL的性能。
优化高可用性（High Availability, HA）系统的探活间隔及其超时参数，以减少HA系统的恢复时间。
优化扩缩容DDL Task的粒度设置，以提升扩缩容的调度效率。
主实例新增支持执行列存索引相关DDL语句（该功能可配置开关，默认开启）。更多信息，请参见配置列存索引开关。
SPM（SQL Plan Management， SQL执行计划管理）新增支持绑定DN HINT。
列存索引新增支持分区分裂。
新增SQL结尾有无分号都支持复用执行计划。
新增含有列存索引的单表查询语句支持自动路由到只读实例的功能。
新增支持创建数据库时禁用MySQL等内置数据库名。
新增单表下推时JOIN ON条件中支持子查询的功能。
新增支持默认可以对含有列存索引的主表进行DDL操作的功能。
新增ANALYZE TABLE默认支持使用列存索引。
引擎版本为MySQL 8.0时，新增支持对JSON类型的字段创建函数索引。
优化扩缩容过程并在回滚的时候检验源表是否存在。
新增列存只读实例支持SPM（SQL Plan Management， SQL执行计划管理）。
新增支持自动给类型不匹配的参数值添加或移除引号。
列存只读实例新增支持索引裁剪功能中的SortKey为字符类型。
新增支持将主实例上的列存查询自动调度到列存只读实例上执行。
优化聚合（AGG）类函数的执行逻辑：在行存场景下，默认采用下推聚合；在列存场景下，默认采用两阶段聚合。
优化热点值的索引选择逻辑。
优化列存只读实例查询中NOT IN表达式的性能。
列存只读实例新增支持Plan Cache（执行计划缓存）。
新增支持列存查询可以调度到列存只读实例上执行。
优化包含JOIN关键字SQL的性能。
优化包含ORDER BY LIMIT子句的UPDATE语句在分区表上执行的性能。
优化事务相关变量的展示策略。
优化分布式死锁检测机制。
新增SQL语句末尾带分号或不带分号都可以复用执行计划的功能。
新增单表下推时JOIN ON条件中支持子查询的功能。
优化数据极端倾斜情况下的基数估计。
优化扩缩容过程并在回滚的时候检验源表是否存在。
新增部分场景下自动force index的功能。
新增支持float和double数据类型的精度指定。
列存只读和只读实例新增支持限流操作。
优化Information_schema.tables的数据显示。
优化ALTER TABLE相关物理执行的DDL并发限制。
增大DN节点wait和concurrent的限流阈值。
新增支持使用AS OF TSO语法读取快照数据。
优化Distinct Agg的查询计划。
新增支持INSERT INTO TABLENAME() VALUES() 语法。
新增支持在UPDATE SET子句中使用无表UNION的语法。
优化逻辑REPLACE下推判断，支持RC隔离级别下的部分场景下推REPLACE语句。
禁止在INFORMATION_SCHEMA库中执行不指定SCHEMA的 DDL。
新增物理复制支持加密表。
新增支持扩缩容运维窗。
创建表时新增支持指定KEY_BLOCK_SIZE等与表压缩功能相关的参数配置。
优化创建GSI以及表分区键变更的执行速度。

问题修复

修复DRDS模式数据库，开启表回收站功能后，对同一个表多次执行TRUNCATE操作，可能导致元数据错误的问题。更多信息，请参见AUTO模式数据库与DRDS模式数据库。
修复新建实例的普通权限账号执行SET GLOBAL语句，会直接报错而不是返回Warning的问题。
修复统计信息中单值频率在直方图中估算为0的问题。
修复执行CREATE TABLE WITH TTL语句时，通过CDC透传到下游Replias后，TTL定义会丢失的问题。更多信息，请参见TTL。
修复列存优化器Sort操作的结果丢失，导致结果不符合预期的问题。
修复RECYCLEBIN模式下执行TRUNCATE，导致隐藏主键不可用问题。
修复SAMPLE RATE用科学计数法表达，导致存储节点SAMPLE HINT不生效的问题。
修复SQL语句含有IN子句且其中只有一个参数时，系统在预处理阶段错误匹配到多个IN参数对应的执行计划的问题。
修复SQL解析器对含有隐式表组的ALTER语句解析失败，以及打标语句不能被正确处理的问题。更多信息，请参见表组类型。
修复对单表（没有做拆分的表）执行FORCE INDEX CCI不生效的问题。更多信息，请参见FORCE INDEX。
修复SHOW DDL STATUS在并发回填时，速率显示错误的问题。
修复统计信息中Feedback信息重复的问题。
修复统计信息中单值频率在直方图中估算为0的问题。
修复使用新建实例的新建普通权限账号执行SET GLOBAL语句会直接报错而不是返回Warning的问题。
修复对列类型为无锁时，变更校验报错的问题。
修复在生成Selection时，同一个Chunk内含有NULL值，导致PARTIAL_SELECTION处理不正确继而引起结果错误的问题。
修复对单表执行FORCE INDEX CCI不生效的问题。
修复列存只读实例部分SHOW语句（SHOW INDEX FROM、SHOW TABLE STATUS、DESC TABLE/SHOW COLUMNS FROM、SHOW COLLATION、SHOW CHARACTER SET、SHOW PRIVILEGES）的兼容性问题 。
修复在某些场景下因过滤条件错误透传至外连接（OUTER JOIN）而导致结果输出不正确的问题。
修复子分区变更过程中，执行INSERT IGNORE会有概率出现空指针报错的问题。
修复表拓扑变更时，NDV（唯一值个数）采集会误删列统计信息的问题。
修复CDC在构建全量Schema快照时，构建快照的线程发生Block的问题。
修复对于含有保留关键字的索引，执行自动FORCE INDEX未加反引号（`）导致SQL报错的问题。
修复对包含Null-Safe等于操作符（<=>）的语句，使用Lookup Join (BKAJoin)优化技术时，分区键裁剪后查询列错乱从而导致结果不正确的问题。更多信息，请参见Lookup Join (BKAJoin)。
修复无锁变更列的类型时，系统会禁止DML下推标记残留的问题。
修复在执行ALTER TABLE修改列名时，如果新的列名大小写与原有列名不一致，导致DDL一直处于执行状态无法完成的问题。
修复DN节点的引擎版本为MySQL 8.0时，INFORMATION_SCHEMA.TABLE中的数据不能实时更新的问题。
修复在PREPARE模式下查询INFORMATION_SCHEMA.STATISTICS表报错的问题。
修复创建列存索引表时因为分区变更触发默认表组被重新分配，导致列存索引表创建失败的问题。
修复对无分组的聚合结果进行过滤时，结果可能不符合预期的问题。
修复当TTL表中时间类型的列值为零或空，导致列存归档表创建失败的问题。
修复SQL解析器对含有隐式表组的ALTER语句解析失败，以及打标语句不能被正确处理的问题。
修复TTL表在动态添加CCI分区时生成的CDC SQL含有随机后缀，导致CDC下游同步异常的问题。
修复跨库单表INSERT SELECT时，DirectPlan下推不符合预期的问题。
修复无法通过Navicat创建和查询数据库用户的问题。
修复通过DataGrip连接数据库时持续不断提示Warning信息的问题。
修复无法通过DBeaver查询数据库视图的问题。
修复无锁变更列的类型时，系统会禁止DML下推标记残留的问题。
修复带有NODE HINT的SQL模板ID与其他SQL模板ID相同的问题。
修复在无锁变更列的类型添加NOT NULL约束时，可能会导致校验不通过的问题。
修复对无分组的聚合结果进行过滤时，结果可能不符合预期的问题。
修复特殊字符会导致统计信息直方图序列化失败的问题。
修复在PREPARE模式下查询INFORMATION_SCHEMA.STATISTICS表报错的问题。
修复在执行ALTER TABLE修改列声明时，如果新的列名大小写与原有列名不一致，会导致DDL一直处于执行状态无法完成的问题。
修复STR_TO_DATE('2020-08-01 00:00:00', '%Y-%m-%d %T')结果不符合预期的问题.
修复SQL语句中WHERE子句的左值仅支持范围在[-263,263-1]内的整数的问题。
修复Baseline（基线）不支持union语句的问题。
修复SPM（SQL Plan Management，SQL执行计划管理）执行计划在MPP（Massively Parallel Processing，大规模并行处理）模式下并发度计算不符合预期的问题。
修复AES_DECRYPT函数在高并发环境下存在的安全问题。
修复特殊字符导致统计信息直方图序列化失败的问题。
修复执行计划在反序列化过程中丢失列存属性distribution的问题。
修复对名称长度超过64个字符的sequence进行变更操作会失败的问题。
修复使用聚合（AGG）类函数时，别名大小写不同，导致模板ID不一致的问题。
修复CREATE DATABASE语句可以创建mysql、performance_schema类型数据库的问题。
修复在列存只读实例执行含有LIMIT的SQL时，结果有概率不符合预期的问题。
修复对有外键约束的子表进行更新操作时，外键的列不匹配的问题。
修复执行含有聚合函数联合子查询的SQL时，查询报错的问题。
修复在无锁情况下变更列的类型并添加NOT NULL约束时，可能导致校验不通过的问题。
修复谓词条件下推到子查询内部时，Join类型推导失败的问题。
修复包含聚合（AGG）函数的INSERT SELECT语句，列匹配失败的问题。
修改控制主表包含列存索引能否被进行DDL操作开关的默认值为开启。
修复因缓存池最大内存配置过大而导致内存溢出的问题。
修复对包含字符串类型列的索引执行IN(NULL)查询时报错的问题。
修复对BLOB字段执行TO_BASE64()函数后结果编码不正确的问题。
修复SQL经过JoinAggToJoinAggSemiJoinRule优化后导致结果不正确的问题。
修复SAMPLE RATE用科学计数法的方式使用时导致SAMPLE HINT不生效的问题。
修复使用TIMESTAMP()函数插入数据时路由和数值变化的问题。
修复SQL语句中WHERE子句的左值仅支持范围在[-263,263-1]内的整数的问题。
修复DDL非预期回滚的问题。
修复下推表数量超过9个时，IN裁剪耗时长的问题。
修复使用无锁列类型变更将字符串类型转为非字符串类型时，有概率失败的问题。
修复baseline delete指令在部分节点生效，部分节点没有生效的问题。
修复使用无锁列类型变更给没有sequence的表的字段添加auto_incement后，导致该表插入数据报错的问题。
修复隔离级别字符串因为大小写和下划线问题，设置后可能解码出错的问题。
修复baseline add/fix指令对于携带IN参数的执行计划代价计算不准确的问题。
修复DROP DATABASE失败后，重新创建数据库时遗漏掉CDC打标动作导致报错的问题。
修复统计信息单表sample hint不生效的问题。
修复OMC 2.0可能会导致UGSI存量数据路由错误的问题。
修复在处理自引用表索引及字符集检测时，导致外键约束添加失败的问题。
修复LIMIT x, y当数值过大时解析报错的问题。
修复开启物理复制后迁移ugsi报错的问题。
修复对单表和广播表进行无锁列变更时会执行失败的问题。