You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Cannot load seed files when using AWS Athena.
Describe the solution you'd like
Update athena__load_seed in load_seed.sql to write seed files from Tuva's s3 bucket to a temp table. Final table (written to client's s3 bucket) is created from temp table.
Update seed column type definitions to use real or int for approximate and exact numeric data types (respectively) when target.type == 'athena'
Describe alternatives you've considered
I've explored using dbt-athena-external-tables to stage each seed file via run-operation, but ultimately it was easier to modify load_seed.sql as dbt-athena-external-tables is still experimental and not yet merged into the dbt-athena adapter. That said, dbt-external-tables looks promising for other data warehouses and could be worth looking into.
Additional context
I have this working in my environment (fork here). Going to do a little more testing/code clean-up before submitting a PR.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Cannot load seed files when using AWS Athena.
Describe the solution you'd like
athena__load_seed
inload_seed.sql
to write seed files from Tuva's s3 bucket to a temp table. Final table (written to client's s3 bucket) is created from temp table.real
orint
for approximate and exactnumeric
data types (respectively) whentarget.type == 'athena'
Describe alternatives you've considered
I've explored using dbt-athena-external-tables to stage each seed file via
run-operation
, but ultimately it was easier to modifyload_seed.sql
asdbt-athena-external-tables
is still experimental and not yet merged into thedbt-athena
adapter. That said,dbt-external-tables
looks promising for other data warehouses and could be worth looking into.Additional context
I have this working in my environment (fork here). Going to do a little more testing/code clean-up before submitting a PR.
The text was updated successfully, but these errors were encountered: