Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CALCITE-6565] Invalid unparse for CHAR without precision in MssqlSqlDialect #3944

Merged
merged 1 commit into from
Sep 6, 2024

Conversation

ChengJie1053
Copy link
Member

@ChengJie1053 ChengJie1053 commented Sep 2, 2024

reference:#3804

image

image

new RelDataTypeSystemImpl() {
@Override public int getDefaultPrecision(SqlTypeName typeName) {
if (typeName == SqlTypeName.CHAR) {
return RelDataType.PRECISION_NOT_SPECIFIED;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indentation is off

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, thank you for helping me review the code, I will modify it

@NobiGo NobiGo added the LGTM-will-merge-soon Overall PR looks OK. Only minor things left. label Sep 3, 2024
Copy link

sonarqubecloud bot commented Sep 3, 2024

@Akanksha-kedia
Copy link

@ChengJie1053 @mihaibudiu @NobiGo @dynamix @mikehinchey I tried reaching the community but could not so really sorry for commenting in pr.

Description:

When translating Hive view definitions to a format that Trino can interpret and process, the Coral library is converting DECIMAL data types to DECIMAL(12,0) in its convertQuery() method. This is causing a type mismatch when the actual data in the view is of type DECIMAL(19,0), leading to a VIEW_IS_STALE exception in Trino.

Scenario:

A Hive view is created with a column of type DECIMAL(19,0) that holds values such as 12345623423423.
This view is read by Trino, which uses the Coral library for translation.
Coral incorrectly converts the DECIMAL(19,0) type to DECIMAL(12,0).
When the view is queried, Trino checks if the actual data type (DECIMAL(19,0)) can be coerced to the defined data type (DECIMAL(12,0)).
This check fails, leading to a VIEW_IS_STALE exception.

Suspected Fix:

The suspected fix involves updating the convertQuery() method in Coral to correctly translate DECIMAL data types without reducing precision. The method should be able to handle and preserve larger DECIMAL types such as DECIMAL(19,0). This change will ensure that the translated data type in Trino matches the actual data type in Hive, avoiding the VIEW_IS_STALE exception.

Proposed Solution:

Update the convertQuery() method in the Coral library to correctly handle and preserve the original precision of DECIMAL data types during translation. This would involve investigating how the current conversion to DECIMAL(12,0) is happening and updating the logic to maintain the original precision of the data type. We also need to ensure that this change does not adversely affect other parts of the Coral library or its functionality.

@mihaibudiu
Copy link
Contributor

This should be an issue in JIRA. Do you have a JIRA account? You can ask for one, it's free.
https://issues.apache.org

I was thinking to file your text as an issue, but I couldn't find any Coral library in the Calcite source tree.
Is this an issue with Calcite?

@caicancai
Copy link
Member

caicancai commented Sep 4, 2024

This should be an issue in JIRA. Do you have a JIRA account? You can ask for one, it's free. https://issues.apache.org

I was thinking to file your text as an issue, but I couldn't find any Coral library in the Calcite source tree. Is this an issue with Calcite?

@Akanksha-kedia @mihaibudiu Is this the Coral library you are referring to https://github.com/linkedin/coral ?

@mihaibudiu
Copy link
Contributor

I don't know what the Coral library is.

@NobiGo NobiGo merged commit 5b298c6 into apache:main Sep 6, 2024
22 of 35 checks passed
@NobiGo NobiGo changed the title [CALCITE-6423] Invalid unparse for CHAR without precision in MssqlSqlDialect [CALCITE-6565] Invalid unparse for CHAR without precision in MssqlSqlDialect Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
LGTM-will-merge-soon Overall PR looks OK. Only minor things left.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants