-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add the Faker connector #23691
base: master
Are you sure you want to change the base?
Add the Faker connector #23691
Conversation
plugin/trino-faker/src/main/java/io/trino/plugin/faker/ColumnInfo.java
Outdated
Show resolved
Hide resolved
plugin/trino-faker/src/main/java/io/trino/plugin/faker/ColumnInfo.java
Outdated
Show resolved
Hide resolved
plugin/trino-faker/src/main/java/io/trino/plugin/faker/ColumnInfo.java
Outdated
Show resolved
Hide resolved
plugin/trino-faker/src/main/java/io/trino/plugin/faker/FakerColumnHandle.java
Outdated
Show resolved
Hide resolved
plugin/trino-faker/src/main/java/io/trino/plugin/faker/FakerColumnHandle.java
Outdated
Show resolved
Hide resolved
...rc/main/resources/docker/trino-product-tests/conf/environment/multinode-all/faker.properties
Show resolved
Hide resolved
plugin/trino-faker/src/main/java/io/trino/plugin/faker/FakerPageSourceProvider.java
Outdated
Show resolved
Hide resolved
for (int i = 0; i < positions; i++) { | ||
generators.get(column).accept(output, completedRows + i); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should loop over the columns and append N positions at a time to each column, that is more efficient than looping over columns for each position
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit confused, isn't this already the case, since this is an inner loop? I just extracted generators.get(column)
outside this loop.
plugin/trino-faker/src/main/java/io/trino/plugin/faker/FakerPageSourceProvider.java
Outdated
Show resolved
Hide resolved
@raunaqmorarka thanks a lot for the review, everything looks much nicer now! |
Description
The purpose of this connector would be similar to tpch/tpcds - generate random data to be used for all kinds of testing. The main difference is that this works with all Trino data types and any kind of schema. It's also using the Datafaker library, which allows generating more sophisticated random data in multiple languages.
For more details, including usage examples, see https://github.com/nineinchnick/trino-faker/
I'll add documentation if the general idea of adding this connector will be approved.
Additional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
(x) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: