Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[#26066] YSQL: pg_upgrade --check improvements
Summary: When `pg_upgrade --check` fails, it writes out the list of incompatible objects in files in the data directory. This is not ideal for users, since they will need to retrieve and analyze those files. It also makes the use of the check via `yugabyted`, `YBA` and `YBM` less user friendly. This change send logs those objects to the console output, in addition to the files. Additionally, instead of exiting the check on the first error, the tool now lists all the errors. In the actual upgrade, it continues to exit on the first error like before. - Cleaned up some print formatting errors with yb specific checks. - Disabling the call to `check_new_cluster`. Since all the related checks (check_new_cluster_is_empty, check_databases_are_compatible, check_loadable_libraries, installed user, check_for_prepared_transactions and check_for_new_tablespace_dir) are not applicable for yb. - Disabled `check_for_tables_with_oids` since it is not relevant in yb. - Migrated all shell upgrade tests to c++. - Added `TEST_SETUP_SUPER` macro that will call `SetUp` on the super class, and exit if the test has been marked as skipped. - Moved `CheckUpgradeCompatibilityGuc` and `UsersAndRoles` to the new `ysql_major_upgrade_check-test.cc` file Fixes #26066 Jira: DB-15392 Test Plan: Pg15UpgradeTest.BasicTablespace Pg15UpgradeTest.DroppedColumnTest YsqlMajorUpgradeCheckTest **Failed check:** ``` yugabyte=# ALTER DATABASE postgres WITH allow_connections FALSE; ALTER DATABASE yugabyte=# CREATE TABLE system_composite_test (id int primary key, authid pg_authid); CREATE TABLE yugabyte=# CREATE TABLE reg_check (a int, b regproc); CREATE TABLE yugabyte=# CREATE FUNCTION ident(integer) yugabyte-# RETURNS integer yugabyte-# AS $$ yugabyte$# BEGIN yugabyte$# RETURN $1; yugabyte$# END; yugabyte$# $$ LANGUAGE plpgsql; CREATE FUNCTION yugabyte=# CREATE OPERATOR !!! (LEFTARG = integer, PROCEDURE = ident); CREATE OPERATOR yugabyte=# CREATE AGGREGATE array_accum (ANYELEMENT)(sfunc = array_append, stype = yugabyte(# ANYARRAY, initcond = '{}'); CREATE AGGREGATE yugabyte=# CREATE TABLE sql_identifier_test (id int primary key, d information_schema.sql_identifier); CREATE TABLE ``` ``` $ ./build/latest/postgres/bin/pg_upgrade --check -U yugabyte --old-socketdir /tmp/.yb.127.0.0.1:5433 --old-port 5433 --old-datadir ~/yugabyte-data/node-1/disk-1/pg_data_11 Performing Consistency Checks on Old Live Server ------------------------------------------------ Checking cluster versions ok Checking attributes of the 'yugabyte' user ok Checking for all 3 system databases ok Checking database connection settings fatal postgres All non-template0 databases must allow connections, i.e. their pg_database.datallowconn must be true. Your installation contains non-template0 databases with their pg_database.datallowconn set to false. Consider allowing connection for all non-template0 databases or drop the databases which do not allow connections. A list of databases with the problem is printed above and in the file: /Users/hsunder/yugabyte-data/node-1/disk-1/pg_data_11/pg_upgrade_output.d/20250218T134505.203/databases_with_datallowconn_false.txt Checking for system-defined composite types in user tables fatal In database: yugabyte public.system_composite_test.authid Your installation contains system-defined composite type(s) in user tables. These type OIDs are not stable across PostgreSQL versions, so this cluster cannot currently be upgraded. You can drop the problem columns and restart the upgrade. A list of the problem columns is printed above and in the file: /Users/hsunder/yugabyte-data/node-1/disk-1/pg_data_11/pg_upgrade_output.d/20250218T134505.203/tables_using_composite.txt Checking for reg* data types in user tables fatal In database: yugabyte public.reg_check.b Your installation contains one of the reg* data types in user tables. These data types reference system OIDs that are not preserved by pg_upgrade, so this cluster cannot currently be upgraded. You can drop the problem columns and restart the upgrade. A list of the problem columns is printed above and in the file: /Users/hsunder/yugabyte-data/node-1/disk-1/pg_data_11/pg_upgrade_output.d/20250218T134505.203/tables_using_reg.txt Checking for user-defined postfix operators fatal In database: yugabyte (oid=16393) public.!!! (pg_catalog.int4, NONE) Your installation contains user-defined postfix operators, which are not supported anymore. Consider dropping the postfix operators and replacing them with prefix operators or function calls. A list of user-defined postfix operators is printed above and in the file: /Users/hsunder/yugabyte-data/node-1/disk-1/pg_data_11/pg_upgrade_output.d/20250218T134505.203/postfix_ops.txt Checking for incompatible polymorphic functions fatal In database: yugabyte aggregate: public.array_accum(anyelement) Your installation contains user-defined objects that refer to internal polymorphic functions with arguments of type "anyarray" or "anyelement". These user-defined objects must be dropped before upgrading and restored afterwards, changing them to refer to the new corresponding functions with arguments of type "anycompatiblearray" and "anycompatible". A list of the problematic objects is printed above and in the file: /Users/hsunder/yugabyte-data/node-1/disk-1/pg_data_11/pg_upgrade_output.d/20250218T134505.203/incompatible_polymorphics.txt Checking for invalid "sql_identifier" user columns fatal In database: yugabyte public.sql_identifier_test.d Your installation contains the "sql_identifier" data type in user tables. The on-disk format for this data type has changed, so this cluster cannot currently be upgraded. You can drop the problem columns and restart the upgrade. A list of the problem columns is printed above and in the file: /Users/hsunder/yugabyte-data/node-1/disk-1/pg_data_11/pg_upgrade_output.d/20250218T134505.203/tables_using_sql_identifier.txt Failure, exiting ``` **Successful check:** ``` yugabyte=# DROP TABLE tbl2; DROP TABLE yugabyte=# DROP TABLE system_composite_test; DROP TABLE yugabyte=# ALTER DATABASE postgres WITH allow_connections TRUE; ALTER DATABASE ``` ``` $ ./build/latest/postgres/bin/pg_upgrade --check -U yugabyte --old-socketdir /tmp/.yb.127.0.0.1:5433 --old-port 5433 --old-datadir ~/yugabyte-data/node-1/disk-1/pg_data_11 Performing Consistency Checks on Old Live Server ------------------------------------------------ Checking cluster versions ok Checking attributes of the 'yugabyte' user ok Checking for all 3 system databases ok Checking database connection settings ok Checking for system-defined composite types in user tables ok Checking for reg* data types in user tables ok Checking for user-defined postfix operators ok Checking for incompatible polymorphic functions ok Checking for invalid "sql_identifier" user columns ok *Clusters are compatible* ``` Reviewers: telgersma, smishra Reviewed By: telgersma Subscribers: ybase, yql Differential Revision: https://phorge.dev.yugabyte.com/D41932
- Loading branch information