Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix an issue in parsing default column charset #582

Merged

Conversation

YAtOff
Copy link
Contributor

@YAtOff YAtOff commented Nov 29, 2023

Description

When parsing the default charset optional metadata, when handling the column index/collation number pairs, the column index is in the range of the number of columns with charset (not the number of all the columns).
Like how it's done here: https://github.com/ClickHouse/ClickHouse/blob/001b67863a57c4d734d94e375e6fdeb4a9407951/src/Core/MySQL/MySQLReplication.cpp#L353

Type of Change

  • Bug fix

Checklist

  • I have read and understood the CONTRIBUTING.md document.
  • I have checked there aren't any other open Pull Requests for the desired changes.
  • This PR resolves an issue #334.

Tests

  • All tests have passed.
  • New tests have been added to cover the changes. (describe below if applicable).

When parsing the default charset optional metadata, when handling the
column index/collation number pairs, the column index is in the range
of the number of columns with charset (not the number of all the
columns).
@sean-k1
Copy link
Collaborator

sean-k1 commented Nov 29, 2023

@YAtOff
Really Cool Pr!
Thanks a lot for spotting that issue and fixing it!
It seems like I didn't realize that the keys of 'column_charset_collation' were the order of non-numeric columns while handling that logic.

Can you describe your mysql variable? for history

  • binlog_format:
  • binlog_row_image:
  • binlog_row_metadata:
  • binlog_row_value_options:

and Can you fix lint too?

 black pymysqlreplication

@sean-k1 sean-k1 self-requested a review November 29, 2023 21:13
@YAtOff
Copy link
Contributor Author

YAtOff commented Nov 30, 2023

MySQL variables

Name Value
binlog_format ROW
binlog_row_image FULL
binlog_row_metadata MINIMAL
binlog_row_value_options -

@@ -1005,14 +1005,17 @@ def _parsed_column_charset_by_default_charset(
column_type_detect_function,
):
column_charset = []
position = 0
Copy link
Collaborator

@sean-k1 sean-k1 Nov 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@YAtOff
If binlog_row_metadata is FULL, you're probably right.

but your binlog_row_metadata = "MINIMAL" you can not enter this method.
because your self.__optional_meta_data value is False

 def _sync_column_info(self):
        if not self.__optional_meta_data: 
            # If optional_meta_data is False Do not sync Event Time Column Schemas
            return
        if len(self.optional_metadata.column_name_list) == 0:
            # May Be Now BINLOG_ROW_METADATA = FULL But Before Action BINLOG_ROW_METADATA Mode = MINIMAL
            return

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, my mistake. It's full.
But if it's minimal the tests pass without asserts.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@YAtOff Cool.

@sean-k1 sean-k1 merged commit bedcad7 into julien-duponchelle:main Dec 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants