-
Notifications
You must be signed in to change notification settings - Fork 424
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[VL] Follow-ups for #7304 #7340
Conversation
Run Gluten Clickhouse CI |
@@ -97,7 +98,7 @@ object SparkArrowUtil { | |||
.map( | |||
field => | |||
toArrowField( | |||
ConverterUtils.normalizeStructFieldName(field.name), | |||
normalizeColName(field.name), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zhztheplayer, looks normalizeColName
is not equivalent to normalizeStructFieldName
which makes the conversion disabled for CH backend.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems gluten-arrow only used for velox backend.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. Let's keep gluten-arrow
an individual module so future refactors can be made easier. Minor code duplications like this one will be acceptable.
Follow-ups for #7304
To keep the new module dependency graph defined in #7264.
cc @zml1206 @PHILO-HE