Adbc ingestion

[dejandbt]

Context

Changes proposed in this pull request

• [List the main changes you've made]

Guidance to review

[Help reviewers know where to focus their efforts, or flag controversial decisions]

Relevant links

• [List any useful links]

Checklist:

• My code follows the style guidelines of this project
• New and existing unit tests pass locally with my changes
• I've changed or updated relevant documentation

[wpfl-dbt]

Couple of comments and questions.

• I'd prefer production SQL be implemented in SQLAlchemy unless there's no other way
• The Engine object has all the information you need in it to build the adbc_driver_postgresql.dbapi object
• Why haven't you just replaced batch_ingest()? It feels like having a single function that can be used on each table would be more extensible in the future, especially as we're considering new tables
• In batch_ingest() you'll see an isolate_table() function. It's interesting for a couple of reasons:
  • It shows some of the ways you can deal with indices, constraints, and table creation in pure SQLAlchemy
  • It's necessary because DeclarativeMeta objects (like Clusters, Contains etc) actually contain the entire Metadata of the ORM. This insight is hugely important because it means from a single DeclarativeMeta object you can access not only the constraints and indices of the table you're working with, but other table's foreign keys into that table. I believe that all the information you need to do the swap should be in the DeclarativeMeta object

[wpfl-dbt]

We test using pytest, though we do use unittest for mocks.