Skip to content

Commit

Permalink
[minor][doc] Fix Iceberg config in README
Browse files Browse the repository at this point in the history
  • Loading branch information
aajisaka committed Nov 10, 2023
1 parent bf23d5e commit 203892f
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -546,7 +546,8 @@ group by 1
- the latest connector for iceberg in AWS marketplace uses Ver 0.14.0 for Glue 3.0, and Ver 1.2.1 for Glue 4.0 where Kryo serialization fails when writing iceberg, use "org.apache.spark.serializer.JavaSerializer" for spark.serializer instead, more info [here](https://github.com/apache/iceberg/pull/546)
- For Athena version 2: The adapter is compatible with the Iceberg Connector from AWS Marketplace with Glue 3.0 as Fulfillment option and 0.12.0-2 (Feb 14, 2022) as Software version)
- For Glue 4.0, to add the following configurations in dbt-profile:
```--conf spark.sql.catalog.glue_catalog=org.apache.iceberg.spark.SparkCatalog
```
--conf spark.sql.catalog.glue_catalog=org.apache.iceberg.spark.SparkCatalog
--conf spark.sql.catalog.glue_catalog.warehouse=s3://<PATH_TO_YOUR_WAREHOUSE>
--conf spark.sql.catalog.glue_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog
--conf spark.sql.catalog.glue_catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO
Expand Down

0 comments on commit 203892f

Please sign in to comment.