[GLUTEN-8851][VL] feat: Support cudf #9229
Conversation
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
3 similar comments
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
| This docker image contains Spark at env $SPARK_HOME, Gluten at /opt/gluten, take a try if you are interested on it. | ||
| The Gluten has been built with Spark3.4. | ||
| ``` | ||
| docker pull apache/gluten:centos-9-jdk8-cudf |
There was a problem hiding this comment.
it looks like this docker image is not ready, I assume this is the image in this docker file: https://github.com/apache/incubator-gluten/pull/9229/files#diff-395a263b6f69bd7ef4991face3666e100acdd60e47f4b28e0737adbbf5fb945a
shall we make a pull request with this docker image first, after merge it will be pushed to docker hub
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Can you help review again? Thanks!@zhouyuan |
| name: digests-centos-9-jdk8-cudf-${{ matrix.os }} | ||
| path: ${{ runner.temp }}/digests/* | ||
| if-no-files-found: error | ||
| retention-days: 1 |
There was a problem hiding this comment.
we may also need to add this new image in the merge section
https://github.com/apache/incubator-gluten/pull/9229/files#diff-e0ed06ed167d42ded61bb8913c0f02d8a691900a916546329d70b7ae1a14e994R318
|
Run Gluten Clickhouse CI on x86 |
| .internal() | ||
| .doc("Enable or disable cudf support.") | ||
| .booleanConf | ||
| .createWithDefault(true) |
There was a problem hiding this comment.
Several questions:
- Could we add an "experimental" mark in the option
doc()? - Should we turn it off by default?
- What happens from user side if CPP code is not built with
--enable_gpubut this option is turned on?
There was a problem hiding this comment.
Cudf will not make effect without built with --enable_gpu even if the config set to true.
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
Enable this feature by compiling with
--enable_gpu.Now only support OrderBy operator.
Use this branch https://github.com/apache/incubator-gluten/tree/cudf