Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Athena Compatibility for seeds #711

Open
sah129 opened this issue Feb 12, 2025 · 1 comment
Open

AWS Athena Compatibility for seeds #711

sah129 opened this issue Feb 12, 2025 · 1 comment

Comments

@sah129
Copy link

sah129 commented Feb 12, 2025

Is your feature request related to a problem? Please describe.
Cannot load seed files when using AWS Athena.

Describe the solution you'd like

  • Update athena__load_seed in load_seed.sql to write seed files from Tuva's s3 bucket to a temp table. Final table (written to client's s3 bucket) is created from temp table.
  • Update seed column type definitions to use real or int for approximate and exact numeric data types (respectively) when target.type == 'athena'

Describe alternatives you've considered
I've explored using dbt-athena-external-tables to stage each seed file via run-operation, but ultimately it was easier to modify load_seed.sql as dbt-athena-external-tables is still experimental and not yet merged into the dbt-athena adapter. That said, dbt-external-tables looks promising for other data warehouses and could be worth looking into.

Additional context
I have this working in my environment (fork here). Going to do a little more testing/code clean-up before submitting a PR.

@aneiderhiser
Copy link
Contributor

Sounds great @sah129 -- thanks for working on this!

@aneiderhiser aneiderhiser added the bug Something isn't working label Feb 19, 2025
@aneiderhiser aneiderhiser removed the bug Something isn't working label Feb 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

2 participants