Connect iceberg-rest catalog to third-party s3 storage and hive metastore.

[thierrycz]

hello folks,

i'm trying to use the iceberg rest catalog whith a hive metastore and an on prem s3 service wich doesn't surpport virtual host.

the hive metastore conection is working fine, but i can't figure out how to passe the s3.path-style-access=true parameter to the res-catalog.

i get an unknow host error on my-bucket.my_endpoint

i tried

gravitino.iceberg-rest.s3-path-style-access = true
gravitino.iceberg-rest.s3.path-style-access = true
gravitino.iceberg-rest.bypass.s3.path-style-access = true
gravitino.bypass.s3.path-style-access = true
gravitino.bypass.fs.s3a.path.style.access = true

with no luck till now

the others s3 values are like

gravitino.iceberg-rest.s3-endpoint = http://my_endpoint:9020

gravitino.iceberg-rest.s3-access-key-id = my-id
gravitino.iceberg-rest.s3-secret-access-key = my-secret
gravitino.iceberg-rest.s3-region = us-east-1
gravitino.iceberg-rest.io-impl = org.apache.iceberg.aws.s3.S3FileIO
gravitino.iceberg-rest.warehouse = s3://my-bucket/catalog/gravitino

any tips would be greatly appreciated.

[FANNG1]

Could you do some debug to check whether the properties are setting successfully?

1. enable debug before start Gravitino server.

export GRAVITINO_DEBUG_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9999"

2. set break point in S3FileIOProperties
3. start server and do some test to check whether it works.

I could set it successfully by gravitino.iceberg-rest.s3.path-style-access = true

[thierrycz]

it's working fine, thank you