Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance data schema generation for empty response #14918

Merged
merged 1 commit into from
Jan 28, 2025

Conversation

Jackie-Jiang
Copy link
Contributor

This is a followup on #14836 with the following enhancements:

  • Handle MV column type properly
  • Handle the case when different number of columns are returned in single-stage vs multi-stage engine
  • More exception handling to prevent it from throwing exception

@codecov-commenter
Copy link

codecov-commenter commented Jan 25, 2025

Codecov Report

Attention: Patch coverage is 0% with 39 lines in your changes missing coverage. Please review.

Project coverage is 63.73%. Comparing base (59551e4) to head (5c7ac6c).
Report is 1632 commits behind head on master.

Files with missing lines Patch % Lines
...g/apache/pinot/query/parser/utils/ParserUtils.java 0.00% 37 Missing ⚠️
...sthandler/BaseSingleStageBrokerRequestHandler.java 0.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #14918      +/-   ##
============================================
+ Coverage     61.75%   63.73%   +1.98%     
- Complexity      207     1471    +1264     
============================================
  Files          2436     2708     +272     
  Lines        133233   151539   +18306     
  Branches      20636    23398    +2762     
============================================
+ Hits          82274    96580   +14306     
- Misses        44911    47704    +2793     
- Partials       6048     7255    +1207     
Flag Coverage Δ
custom-integration1 100.00% <ø> (+99.99%) ⬆️
integration 100.00% <ø> (+99.99%) ⬆️
integration1 100.00% <ø> (+99.99%) ⬆️
integration2 0.00% <ø> (ø)
java-11 63.71% <0.00%> (+2.00%) ⬆️
java-21 63.62% <0.00%> (+2.00%) ⬆️
skip-bytebuffers-false 63.72% <0.00%> (+1.98%) ⬆️
skip-bytebuffers-true 63.58% <0.00%> (+35.86%) ⬆️
temurin 63.73% <0.00%> (+1.98%) ⬆️
unittests 63.72% <0.00%> (+1.98%) ⬆️
unittests1 56.31% <0.00%> (+9.42%) ⬆️
unittests2 34.01% <0.00%> (+6.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@albertobastos albertobastos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the enhancements Jackie, looks good to me.

Comment on lines +59 to +62
* Response data schema can be inaccurate or incomplete in several forms:
* 1. No result table at all (when all segments have been pruned on broker).
* 2. Data schema has all columns set to default type (STRING) (when all segments pruned on server).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have contradictory feelings about explaining why some issues can happen in the generic method that solves them. On the one hand, it is useful to have the reasons centralized. On the other hand, it is a clear candidate for documentation that could be desynchronized with the code (ie we change one of these cases and forget to change the javadoc).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need a dev focus documentation to document the internal behaviors. It is always a hard problem to keep the doc in sync with code though. I'd prefer having it in javadoc for now for future developers to know the history

@Jackie-Jiang Jackie-Jiang force-pushed the enhance_empty_response_schema branch from 2c3dea9 to 5c7ac6c Compare January 27, 2025 19:46
@gortiz gortiz merged commit 3c5bc43 into apache:master Jan 28, 2025
21 checks passed
@Jackie-Jiang Jackie-Jiang deleted the enhance_empty_response_schema branch January 28, 2025 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants